We call svFindFirstNumberedKey() to check whether we have any NETMASK
set. Since commit 9085c5c3a9 ('ifcfg-rh: rename
svFindFirstKeyWithPrefix() to svFindFirstNumberedKey() for finding
NETMASK') that function would no longer find the "NETMASK" without
number.
Fix that, by letting nms_ifcfg_rh_utils_is_numbered_tag() return TRUE
for the tag itself. This also makes more sense, because it matches our
common understanding what numbered tags are.
Adjust the other callers that don't want this behavior to explicitly
check.
Fixes: 9085c5c3a9 ('ifcfg-rh: rename svFindFirstKeyWithPrefix() to svFindFirstNumberedKey() for finding NETMASK')
Previously, setting or getting a variable required to scan all lines.
Note that frequently we would look up variables that didn't actually
exist, which we could only determine after searching the entire list.
Also, since we needed to handle having the same variable specified
multiple times (where the last occurrence wins), we always had to search
all keys and couldn't stop when finding the first key. Well, technically
we could have searched in reverse order for the getter, but that wasn't
done. For the setter we wanted to delete all but the last occurrences,
so to find them, we really had to search them all.
We want to support profiles with hundreds or thousands of addresses and routes.
This does not scale well.
Add an hash table to find the variables in constant time.
Test this commit and the parent commit:
$ git clean -fdx &&
CFLAGS=-O2 ./autogen.sh --with-more-asserts=0 &&
./tools/run-nm-test.sh -m src/settings/plugins/ifcfg-rh/tests/test-ifcfg-rh &&
perf stat -r 50 -B src/settings/plugins/ifcfg-rh/tests/test-ifcfg-rh 1>/dev/null
Before:
Performance counter stats for 'src/settings/plugins/ifcfg-rh/tests/test-ifcfg-rh' (50 runs):
330.94 msec task-clock:u # 0.961 CPUs utilized ( +- 0.33% )
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
1,081 page-faults:u # 0.003 M/sec ( +- 0.07% )
1,035,923,116 cycles:u # 3.130 GHz ( +- 0.29% )
1,800,084,022 instructions:u # 1.74 insn per cycle ( +- 0.01% )
362,313,301 branches:u # 1094.784 M/sec ( +- 0.02% )
6,259,421 branch-misses:u # 1.73% of all branches ( +- 0.13% )
0.34454 +- 0.00116 seconds time elapsed ( +- 0.34% )
Now:
Performance counter stats for 'src/settings/plugins/ifcfg-rh/tests/test-ifcfg-rh' (50 runs):
329.78 msec task-clock:u # 0.962 CPUs utilized ( +- 0.39% )
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
1,084 page-faults:u # 0.003 M/sec ( +- 0.05% )
1,036,130,698 cycles:u # 3.142 GHz ( +- 0.13% )
1,799,851,979 instructions:u # 1.74 insn per cycle ( +- 0.01% )
360,374,338 branches:u # 1092.756 M/sec ( +- 0.01% )
6,160,796 branch-misses:u # 1.71% of all branches ( +- 0.08% )
0.34287 +- 0.00133 seconds time elapsed ( +- 0.39% )
So, not much difference. But this is not surprising, because test-ifcfg-rh loads and
writes predominantly ifcfg files with few variables. The difference should be visible
when having large files.
svFindFirstKeyWithPrefix() only had one caller: to find whether there are
any NETMASK variables set. NETMASK is a numbered variable, so we should only
find variables that indeed follow the pattern. Since there was only
one caller, rename and repurpose the function.
When we write a connection profile to ifcfg-rh file, we first load the
possibly existing file and modify it. The purpose is to preserve
variables that we don't know about, keep comments and preserve the order
of the variables.
Note that the writer sets a bunch of variables according to the
profile's setting. At various places the writer would explicitly
clear variables with svUnsetValue(). However, that was problematic:
- we would not unset all variables that we care about. We really should
not leave previous variables if they make no sense anymore for the
profile. The only thing we want to preserve are entirely unknown keys
and comments. Note that when the writer omits to clear an unset variable,
it usually does so assuming that the reader would anyway ignore the
key, become some other key renders it irrelevant. Given the complexity
of the reader and writer, that is often not the case and hard to ensure.
We might have simply forgotten a svUnsetValue(), which was an easy
to make mistake and hard to find (because you'd have to test with
a pre-existing profile that happens to contain that key, which leaves
countless combinations for testing.
That means, a profile written by the writter might be interpreted
differently by the reader depending on which pre-existing keys were set.
- it was cumbersome to explicitly call svUnsetValue().
Note that for numbered tags in particular we would iterate the keys
trying to unset them. For example for addresses (like "IPADDR5") we
would iterate over the first 256 IPADDR keys, trying to unset them.
That is horrible. For one, it doesn't cover the case where there might
be more than 256 addresses. Also, it adds a significant overhead every
time.
While writing a ifcfg file currently is O(n^2) because setting one key
is O(l), with l being the number of keys/lines. So, if you set n keys
in a file with l lines, you get O(n*l). Which is basically O(n^2),
because the number of lines and the number of keys to set usually
corresponds.
So when setting 256 times IPADDR, the overall complexity was still
O(n^2 + 256 * n) and didn't change. However, the 256 factor here can
be very significant.
We should not explicitly unset variables, we should always unset all
known variables that we don't explicitly set.
The svUnsetValue() calls are still there. They will be dropped next.
Helper function to remove all variables that are still dirty (not
visited) and well-known.
Also add svWriteFileWithoutDirtyWellknown() to clear the lines
before persisting to disk.
This adds a lot of meta-data about how we handle ifcfg-rh.
We will use this to prune/delete all variables that are not explicitly
set (dirty) but also well-known.
We could now easily emit a warning when an ifcfg-rh file contains
unused key.
We also could add more meta-data for each key. For example, we write
different files (ifcfg- and keys- files). We could add flags to indicate
that variables are valid in certain files. Currently that's not done.
Also, for simple properties we could associate the key with the
NMSetting property, and treat does generically, like keyfile does.
Anyway, there are potentials. For now, we will use this to clear dirty
variables.
Previously, IS_NUMBERED_TAG() could only be called with a C literal.
Add is_numbered_tag() which can be called with any C string.
Also, IS_NUMBERED_TAG_PARSE() and IS_NUMBERED_TAG() didn't do exactly
the same. I think they should. The only difference was if the number
was larger than 2^63-1. Now IS_NUMBERED_TAG() starts ignoring such
keys, which is fine.
By default, all lines are now marked as dirty. Whenever we modify/set
a line, it becomes non-dirty. That will be used later to prune lines
that are dirty, that is, not yet visited.
I hesitated to add this to libnm, because it's hardly used.
However, we already fetch the property during GetManagedObjects(),
we we should make it accessible, instead of requiring the user to
make another D-Bus call.
Otherwise, this function cannot really be used via generated bindings.
Also, it's the only way to actually retrieve the set vlan-ids, without
it, you wouldn't know which ones are set.
Fixes: a9b4532fa7 ('libnm-core: add SR-IOV setting')
After a user entered a secret it would get stored in the readline
history data structure (in plain text) and eventually get leaked.
This commit instructs readline to not store any secret in its
history and fixes a non-related memory leak.
Found by covscan:
NetworkManager-1.22.0/src/dhcp/nm-dhcp-nettools.c:945: check_return:
Calling "g_file_set_contents" without checking return value (as is
done elsewhere 16 out of 20 times).
Fixes: 9f89516928 ('dhcp: nettools: read/write lease files')
Found by covscan:
NetworkManager-1.22.0/shared/nm-glib-aux/nm-dbus-aux.c:361:
missing_va_end: va_end was not called for "ap".
Fixes: ce36494c0a ('shared: add nm_dbus_error_is() helper')
After 1.22.0 is released, merge it back into master so that
1.22.0 is part of the history of master. That means,
$ git log --first-parent master
will also traverse 1.22.0 and 1.22-rc*.
Also bump the micro version to 1.23.1-dev to indicate that this is
after 1.22.0 is out.
When NetworkManager starts, NMSecretAgentOld gets a name-owner changed
signal and registers right away.
Especially since commit ce0e898fb4 ('libnm: refactor caching of D-Bus
objects in NMClient') this hits a race where NetworkManager does not yet
export the org.freedesktop.NetworkManager.AgentManager interface and
the registration fails:
GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.NetworkManager.AgentManager” on object at path /org/freedesktop/NetworkManager/AgentManager
Previously, when NMClient recevied a name-owner changed, that would
block the main loop long enough to avoid the race. Note that NMClient
has nothing to do with NMSecretAgentOld, however in practice all
applications that use NMSecretAgentOld also use NMClient.
While we should fix the race server-side, we also need to work around it
in the client. Retry.
Also, make the async request actually cancellable and actually honor the passed
GCancellable.
Check output:
$ LIBNM_CLIENT_DEBUG=trace ./clients/cli/nmcli agent secret |& grep secret-agent
libnm-dbus: <trace> [21399.04862] secret-agent[2f2af4ee102d7570]: create new instance
libnm-dbus: <trace> [21399.04863] secret-agent[2f2af4ee102d7570]: init-sync
libnm-dbus: <trace> [21404.08147] secret-agent[2f2af4ee102d7570]: name owner changed: (null)
libnm-dbus: <trace> [21404.09085] secret-agent[2f2af4ee102d7570]: name owner changed: ":1.2504"
libnm-dbus: <trace> [21404.09085] secret-agent[2f2af4ee102d7570]: register: starting asynchronous registration...
libnm-dbus: <trace> [21404.09178] secret-agent[2f2af4ee102d7570]: register: registration failed with error "GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.NetworkManager.AgentManager” on object at path /org/freedesktop/NetworkManager/AgentManager". Retry in 0 msec...
libnm-dbus: <trace> [21404.09178] secret-agent[2f2af4ee102d7570]: register: retry registration...
libnm-dbus: <trace> [21404.09195] secret-agent[2f2af4ee102d7570]: register: registration failed with error "GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.NetworkManager.AgentManager” on object at path /org/freedesktop/NetworkManager/AgentManager". Retry in 4 msec...
libnm-dbus: <trace> [21404.09236] secret-agent[2f2af4ee102d7570]: register: retry registration...
[...]
libnm-dbus: <trace> [21405.01782] secret-agent[2f2af4ee102d7570]: register: registration failed with error "GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.NetworkManager.AgentManager” on object at path /org/freedesktop/NetworkManager/AgentManager". Retry in 128 msec...
libnm-dbus: <trace> [21405.03063] secret-agent[2f2af4ee102d7570]: register: retry registration...
libnm-dbus: <trace> [21405.03068] secret-agent[2f2af4ee102d7570]: register: registration failed with error "GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.NetworkManager.AgentManager” on object at path /org/freedesktop/NetworkManager/AgentManager". Retry in 128 msec...
libnm-dbus: <trace> [21405.04354] secret-agent[2f2af4ee102d7570]: register: retry registration...
libnm-dbus: <trace> [21406.01097] secret-agent[2f2af4ee102d7570]: register: registration succeeded
(cherry picked from commit f0d3243f2b)
This change is of course right and read nicer. Also, the GTask captures the
current g_main_context_get_thread_default(). We will need that next.
(cherry picked from commit cff4e937ac)
RegisterWithCapabilities() is supported since NetworkManager 0.9.9.1. Of course,
we don't support such old server anymore (also, because we require the standard
D-Bus interfaces like ObjectManager).
(cherry picked from commit 263aa63caa)
"nm-glib-aux/nm-logging-fwd.h" provides macros like _LOGD() to be reused
by various parts which implement logging (by defining _NMLOG() accordingly).
libnm also has logging, however it uses different logging levels
aside LOGD_DEBUG.
Instead, implement _LOGD() using a define _LOGL_DEBUG, so that libnm can
redefine thos _LOGL_DEBUG defines and use the _LOGD() macro.
(cherry picked from commit 1b00fd2fd2)
When NetworkManager starts, NMSecretAgentOld gets a name-owner changed
signal and registers right away.
Especially since commit ce0e898fb4 ('libnm: refactor caching of D-Bus
objects in NMClient') this hits a race where NetworkManager does not yet
export the org.freedesktop.NetworkManager.AgentManager interface and
the registration fails:
GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.NetworkManager.AgentManager” on object at path /org/freedesktop/NetworkManager/AgentManager
Previously, when NMClient recevied a name-owner changed, that would
block the main loop long enough to avoid the race. Note that NMClient
has nothing to do with NMSecretAgentOld, however in practice all
applications that use NMSecretAgentOld also use NMClient.
While we should fix the race server-side, we also need to work around it
in the client. Retry.
Also, make the async request actually cancellable and actually honor the passed
GCancellable.
Check output:
$ LIBNM_CLIENT_DEBUG=trace ./clients/cli/nmcli agent secret |& grep secret-agent
libnm-dbus: <trace> [21399.04862] secret-agent[2f2af4ee102d7570]: create new instance
libnm-dbus: <trace> [21399.04863] secret-agent[2f2af4ee102d7570]: init-sync
libnm-dbus: <trace> [21404.08147] secret-agent[2f2af4ee102d7570]: name owner changed: (null)
libnm-dbus: <trace> [21404.09085] secret-agent[2f2af4ee102d7570]: name owner changed: ":1.2504"
libnm-dbus: <trace> [21404.09085] secret-agent[2f2af4ee102d7570]: register: starting asynchronous registration...
libnm-dbus: <trace> [21404.09178] secret-agent[2f2af4ee102d7570]: register: registration failed with error "GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.NetworkManager.AgentManager” on object at path /org/freedesktop/NetworkManager/AgentManager". Retry in 0 msec...
libnm-dbus: <trace> [21404.09178] secret-agent[2f2af4ee102d7570]: register: retry registration...
libnm-dbus: <trace> [21404.09195] secret-agent[2f2af4ee102d7570]: register: registration failed with error "GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.NetworkManager.AgentManager” on object at path /org/freedesktop/NetworkManager/AgentManager". Retry in 4 msec...
libnm-dbus: <trace> [21404.09236] secret-agent[2f2af4ee102d7570]: register: retry registration...
[...]
libnm-dbus: <trace> [21405.01782] secret-agent[2f2af4ee102d7570]: register: registration failed with error "GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.NetworkManager.AgentManager” on object at path /org/freedesktop/NetworkManager/AgentManager". Retry in 128 msec...
libnm-dbus: <trace> [21405.03063] secret-agent[2f2af4ee102d7570]: register: retry registration...
libnm-dbus: <trace> [21405.03068] secret-agent[2f2af4ee102d7570]: register: registration failed with error "GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.NetworkManager.AgentManager” on object at path /org/freedesktop/NetworkManager/AgentManager". Retry in 128 msec...
libnm-dbus: <trace> [21405.04354] secret-agent[2f2af4ee102d7570]: register: retry registration...
libnm-dbus: <trace> [21406.01097] secret-agent[2f2af4ee102d7570]: register: registration succeeded
RegisterWithCapabilities() is supported since NetworkManager 0.9.9.1. Of course,
we don't support such old server anymore (also, because we require the standard
D-Bus interfaces like ObjectManager).
"nm-glib-aux/nm-logging-fwd.h" provides macros like _LOGD() to be reused
by various parts which implement logging (by defining _NMLOG() accordingly).
libnm also has logging, however it uses different logging levels
aside LOGD_DEBUG.
Instead, implement _LOGD() using a define _LOGL_DEBUG, so that libnm can
redefine thos _LOGL_DEBUG defines and use the _LOGD() macro.
