Eventually we should do a coordinated shutdown when NetworkManager exits.
That is a large work, ensuring that all asynchronous actions are cancellable
(in time), and that we wait for all pending operations to complete.
Add a function nm_shutdown_register_watchdog(), so that objects can register
themselves, to block shutdown until they are destroyed. It's not yet used,
because actually iterating the mainloop during shutdown can only be done,
once the code is prepared to be ready for that. But we already need the
function, so that we can refactor individual parts of the code, in preparation
of using it in the future.
Use the same form everywhere: "TRANSLATORS" instead of "Translators".
The manual also seems to prefer the upper-case form [1].
$ sed 's/\<Translators\>: /TRANSLATORS: /g' $(git grep -l Translators) -i
[1] https://www.gnu.org/software/gettext/manual/gettext.html
NMSettings exposes a cached list of all connection. We don't need
to clone it. Note that this is not save against concurrent modification,
meaning, add/remove of connections in NMSettings will invalidate the
list.
However, it wasn't save against that previously either, because
altough we cloned the container (GSList), we didn't take an additional
reference to the elements.
This is purely a performance optimization, we don't need to clone the
list. Also, since the original list is of type "NMConnection *const*",
use that type insistently, instead of dependent API requiring GSList.
IMO, GSList is anyway not a very nice API for many use cases because
it requires an additional slice allocation for each element. It's
slower, and often less convenient to use.
Previously, we used the generated GDBusInterfaceSkeleton types and glued
them via the NMExportedObject base class to our NM types. We also used
GDBusObjectManagerServer.
Don't do that anymore. The resulting code was more complicated despite (or
because?) using generated classes. It was hard to understand, complex, had
ordering-issues, and had a runtime and memory overhead.
This patch refactors this entirely and uses the lower layer API GDBusConnection
directly. It replaces the generated code, GDBusInterfaceSkeleton, and
GDBusObjectManagerServer. All this is now done by NMDbusObject and NMDBusManager
and static descriptor instances of type GDBusInterfaceInfo.
This adds a net plus of more then 1300 lines of hand written code. I claim
that this implementation is easier to understand. Note that previously we
also required extensive and complex glue code to bind our objects to the
generated skeleton objects. Instead, now glue our objects directly to
GDBusConnection. The result is more immediate and gets rid of layers of
code in between.
Now that the D-Bus glue us more under our control, we can address issus and
bottlenecks better, instead of adding code to bend the generated skeletons
to our needs.
Note that the current implementation now only supports one D-Bus connection.
That was effectively the case already, although there were places (and still are)
where the code pretends it could also support connections from a private socket.
We dropped private socket support mainly because it was unused, untested and
buggy, but also because GDBusObjectManagerServer could not export the same
objects on multiple connections. Now, it would be rather straight forward to
fix that and re-introduce ObjectManager on each private connection. But this
commit doesn't do that yet, and the new code intentionally supports only one
D-Bus connection.
Also, the D-Bus startup was simplified. There is no retry, either nm_dbus_manager_start()
succeeds, or it detects the initrd case. In the initrd case, bus manager never tries to
connect to D-Bus. Since the initrd scenario is not yet used/tested, this is good enough
for the moment. It could be easily extended later, for example with polling whether the
system bus appears (like was done previously). Also, restart of D-Bus daemon isn't
supported either -- just like before.
Note how NMDBusManager now implements the ObjectManager D-Bus interface
directly.
Also, this fixes race issues in the server, by no longer delaying
PropertiesChanged signals. NMExportedObject would collect changed
properties and send the signal out in idle_emit_properties_changed()
on idle. This messes up the ordering of change events w.r.t. other
signals and events on the bus. Note that not only NMExportedObject
messed up the ordering. Also the generated code would hook into
notify() and process change events in and idle handle, exhibiting the
same ordering issue too.
No longer do that. PropertiesChanged signals will be sent right away
by hooking into dispatch_properties_changed(). This means, changing
a property in quick succession will no longer be combined and is
guaranteed to emit signals for each individual state. Quite possibly
we emit now more PropertiesChanged signals then before.
However, we are now able to group a set of changes by using standard
g_object_freeze_notify()/g_object_thaw_notify(). We probably should
make more use of that.
Also, now that our signals are all handled in the right order, we
might find places where we still emit them in the wrong order. But that
is then due to the order in which our GObjects emit signals, not due
to an ill behavior of the D-Bus glue. Possibly we need to identify
such ordering issues and fix them.
Numbers (for contrib/rpm --without debug on x86_64):
- the patch changes the code size of NetworkManager by
- 2809360 bytes
+ 2537528 bytes (-9.7%)
- Runtime measurements are harder because there is a large variance
during testing. In other words, the numbers are not reproducible.
Currently, the implementation performs no caching of GVariants at all,
but it would be rather simple to add it, if that turns out to be
useful.
Anyway, without strong claim, it seems that the new form tends to
perform slightly better. That would be no surprise.
$ time (for i in {1..1000}; do nmcli >/dev/null || break; echo -n .; done)
- real 1m39.355s
+ real 1m37.432s
$ time (for i in {1..2000}; do busctl call org.freedesktop.NetworkManager /org/freedesktop org.freedesktop.DBus.ObjectManager GetManagedObjects > /dev/null || break; echo -n .; done)
- real 0m26.843s
+ real 0m25.281s
- Regarding RSS size, just looking at the processes in similar
conditions, doesn't give a large difference. On my system they
consume about 19MB RSS. It seems that the new version has a
slightly smaller RSS size.
- 19356 RSS
+ 18660 RSS
nm_match_spec_device_by_pllink() does not support matching on all parameters,
unlike nm_match_spec_device(). The reason is that certain parameters are
only available when having a NMDevice instance.
Add an argument "match_device_type", so that the caller can inject the
device type to be used. Note that for NMDevice, the device-type is
nm_device_get_type_description(), which usually depends on the device
class only. The only caller of nm_match_spec_device_by_pllink() is the
wifi factory, and it already knows that it wants to create a device of
type NMDeviceWifi. Hence, it knows and can specify "wifi" as
match_device_type.
Add a variant of nm_device_spec_match_list() that looks up the match
paramters from a platform link instance.
Usually, we have a NMDevice instance that we use for matching.
However, at some places (like inside the device factory's
create_device() method), we might not have a NMDevice instance
to get the match paramters.
Add an alternative form, that gets the match paramters from a platform
link instance.
The code is placed inside src/NetworkManagerUtils.c, because
src/nm-core-utils.c is supposed to be independent of platform.
The matching works fuzzy and is not reliable. That is why we store
which connection should be assumed after restart in the state file
of NetworkManager.
In that case, we don't need to do a full check (with the possibility
of a false-reject). Just check for the minimum required properties:
the type and slave-type.
Yes, if the user modifies the connection while restarting NM, then
we might wrongly assume a connection that no longer would match.
But NM should not read minds, it should do as indicated.
Replace the usage of g_str_hash() with our own nm_str_hash().
GLib's g_str_hash() uses djb2 hashing function, just like we
do at the moment. The only difference is, that we use a diffrent
seed value.
Note, that we initialize the hash seed with random data (by calling
getrandom() or reading /dev/urandom). That is a change compared to
before.
This change of the hashing function and accessing the random pool
might be undesired for libnm/libnm-core. Hence, the change is not
done there as it possibly changes behavior for public API. Maybe
we should do that later though.
At this point, there isn't much of a change. This patch becomes
interesting, if we decide to use a different hashing algorithm.
When we generate the connection in nm_device_generate_connection(), we
add all routes that have rt_source (roundtrip(NM_IP_CONFIG_SOURCE_USER)).
Especially since commit e470e13922, this
includes automatically added host-routes to the gateway, added by
ip4_config_merge_and_apply().
Later, during nm_utils_match_connection() this route most not prevent
matching. Either nm_device_generate_connection() should not add it, or
nm_utils_match_connection() should ignore it.
I think adjusting the matching is better, because ip-configs are used
for several things, including exposing routes on D-Bus. We don't want
to hide this route on D-Bus.
Fixes: e470e13922https://bugzilla.redhat.com/show_bug.cgi?id=1487384
Kernel requires that routes have a host part of zero. For NetworkManager
configuration we allow non-zero host parts (but ignore them). Fix
route_compare() to ignore the host part.
This has only effect during assuming connections. That means, on
restart NM would fail to match a connection with static routes
if it has a non-zero host part. So, the impact is rather small.
(cherry picked from commit 034b7fb51c)
Now we only search for a candiate with matching UUID. No need to
first lookup all activatable connections, just find the candidate
by UUID and see if it is activatable.
If the connection candidate contains a static route
without route-metric, but overrides the default
route-metric, matching would have failed:
ipv4.route-metric: 200
ipv4.routes: { ip = 192.168.5.0/32 }
Then, we must not compare existing routes using the device's
metric, but must use the overwrite from the connection.
Keep the include paths clean and separate. We use directories to group source
files together. That makes sense (I guess), but then we should use this
grouping also when including files. Thus require to #include files with their
path relative to "src/".
Also, we build various artifacts from the "src/" tree. Instead of having
individual CFLAGS for each artifact in Makefile.am, the CFLAGS should be
unified. Previously, the CFLAGS for each artifact differ and are inconsistent
in which paths they add to the search path. Fix the inconsistency by just
don't add the paths at all.
Extend the "ethernet.cloned-mac-address" and "wifi.cloned-mac-address"
settings. Instead of specifying an explicit MAC address, the additional
special values "permanent", "preserve", "random", "random-bia", "stable" and
"stable-bia" are supported.
"permanent" means to use the permanent hardware address. Previously that
was the default if no explict cloned-mac-address was set. The default is
thus still "permanent", but it can be overwritten by global
configuration.
"preserve" means not to configure the MAC address when activating the
device. That was actually the default behavior before introducing MAC
address handling with commit 1b49f941a6.
"random" and "random-bia" use a randomized MAC address for each
connection. "stable" and "stable-bia" use a generated, stable
address based on some token. The "bia" suffix says to generate a
burned-in address. The stable method by default uses as token the
connection UUID, but the token can be explicitly choosen via
"stable:<TOKEN>" and "stable-bia:<TOKEN>".
On a D-Bus level, the "cloned-mac-address" is a bytestring and thus
cannot express the new forms. It is replaced by the new
"assigned-mac-address" field. For the GObject property, libnm's API,
nmcli, keyfile, etc. the old name "cloned-mac-address" is still used.
Deprecating the old field seems more complicated then just extending
the use of the existing "cloned-mac-address" field, although the name
doesn't match well with the extended meaning.
There is some overlap with the "wifi.mac-address-randomization" setting.
https://bugzilla.gnome.org/show_bug.cgi?id=705545https://bugzilla.gnome.org/show_bug.cgi?id=708820https://bugzilla.gnome.org/show_bug.cgi?id=758301
"NetworkManagerUtils.h" contains a bunch of helper tools for core
daemon ("src/").
Unfortunately, it has dependencies to other parts of core,
such as "nm-device.h" and "nm-platform.h". Split out a part
of tools that are independent so that they can be used without
dragging in other dependencies.
"nm-core-utils.h" should only use libnm-core, "nm-logging.h"
and shared.
"NetworkManagerUtils.h" should provide all "nm-core-utils.h" and
possibly other utilities that have larger dependencies.
- All internal source files (except "examples", which are not internal)
should include "config.h" first. As also all internal source
files should include "nm-default.h", let "config.h" be included
by "nm-default.h" and include "nm-default.h" as first in every
source file.
We already wanted to include "nm-default.h" before other headers
because it might contains some fixes (like "nm-glib.h" compatibility)
that is required first.
- After including "nm-default.h", we optinally allow for including the
corresponding header file for the source file at hand. The idea
is to ensure that each header file is self contained.
- Don't include "config.h" or "nm-default.h" in any header file
(except "nm-sd-adapt.h"). Public headers anyway must not include
these headers, and internal headers are never included after
"nm-default.h", as of the first previous point.
- Include all internal headers with quotes instead of angle brackets.
In practice it doesn't matter, because in our public headers we must
include other headers with angle brackets. As we use our public
headers also to compile our interal source files, effectively the
result must be the same. Still do it for consistency.
- Except for <config.h> itself. Include it with angle brackets as suggested by
https://www.gnu.org/software/autoconf/manual/autoconf.html#Configuration-Headers
When NM tries to match a generated connection to a persistent one, it
considers also the metric of static routes. However, if the field is
set to -1 (use default value for the device) on the persistent
connection, the comparison will always fail because the generated
connection contains the actual value read from kernel.
To fix the issue, modify check_possible_match() to deal correctly with
-1 and translate it to the expected value for the current device when
performing the comparison.
This allows connections with static routes and default metric to
properly be re-assumed when NM is restarted.
https://bugzilla.redhat.com/show_bug.cgi?id=1302532
The compiler might be able to optimize the switch better.
But more importantly, it has the type information of the enum
and can give warnings about unmentioned enum values.
Point in case:
# ip link add link dummy0 name dummy0.vlan type vlan id 4095
RTNETLINK answers: Numerical result out of range
This potentially causes existing (invalid) connections to disappear
as they now fail verification.
Instead of adjusting the range of the GObject property
NM_SETTING_VLAN_ID, reject it during vlan. This is a bit more
forgiving to an older client that isn't aware of this new restriction,
so he can first set the value without raising a critical warning.
When a connection should be assumed and the generated connection did not
contain a wired setting, the connection did not match due to S390 properties.
Such a connection should be allowed to match to a connection with a wired
setting with default (empty) S390 properties.
This can happen when there is a VLAN profile configured that contains a wired
setting in it and NetworkManager is (re)started.
Example/reproducer:
$ nmcli con add type vlan con-name vlan-test autoconnect no dev em1 id 44
$ nmcli con mod vlan-test eth.mtu 1450 (modify the connection, so that it has a wired setting)
$ nmcli con up vlan-test (activate the connection)
$ sudo systemctl restart NetworkManager
$ nmcli device
check that 'vlan-test' connection is active on em1.44 device
(and not the auto-generated em1.44)
https://bugzilla.redhat.com/show_bug.cgi?id=1276343
RFC7217 introduces an alternative mechanism for creating addresses during
stateless IPv6 address configuration. It's supposed to create addresses whose
host part stays stable in a particular network but changes when the hosts
enters another network to mitigate possibility of tracking the host movement.
It can be used alongside RFC 4941 privacy extensions (temporary addresses)
and replaces the use of RFC 4862 interface identifiers.
The address creation mode is controlld by ip6.addr_gen_mode property
(ADDR_GEN_MODE in ifcfg-rh), with values of "stable-privacy" and "eui-64",
defaulting to "eui-64" if unspecified.
The host part of an address is computed by hashing a system-specific secret
salted with various stable values that identify the connection with a secure
hash algorithm:
RID = F(Prefix, Net_Iface, Network_ID, DAD_Counter, secret_key)
For NetworkManager we use these parameters:
* F()
SHA256 hash function.
* Prefix
This is a network part of the /64 address
* Net_Iface
We use the interface name (e.g. "eth0"). This ensures the address won't
change with the change of interface hardware.
* Network_ID
We use the connection UUID here. This ensures the salt is different for
wireless networks with a different SSID as suggested by RFC7217.
* DAD_Counter
A per-address counter that increases with each DAD failure.
* secret_key
We store the secret key in /var/lib/NetworkManager/secret_key. If it's
shorter than 128 bits then it's rejected. If the file is not present we
initialize it by fetching 256 pseudo-random bits from /dev/urandom on
first use.
Duplicate address detection uses IDGEN_RETRIES = 3 and does not utilize the
IDGEN_DELAY delay (despite it SHOULD). This is for ease of implementation
and may change in future. Neither parameter is currently configurable.
Having a static string buffer for convenience is useful not only
for platform. Define the string buffer in NetworkManagerUtils.h,
so that all to-string functions can reuse *one* buffer.
Of course, this has the potential danger, that different
to-string method might reuse the same buffer. Hence, low-level
library functions are adviced to use their own buffer, because
an upper level might already use the global buffer for another
string.
Support a new configuration option
[.config]
enable=<ENABLED>
for configuration snippets.
This new [.config] section is only relevant within the snippet itself
and it is not merged into the combined configuration.
Currently only the "enable" key is supported. If the "enable" key is
missing, it obviously defaults to being enabled. It allows snippets
to be skipped from loading. The main configuration "NetworkManager.conf"
cannot be skipped.
<ENABLED> can be a boolean value (false), to skip a configuration
snippet from loading.
It can also be a string to match against the NetworkManager version,
like "enable=nm-version-min:1.1,nm-version-min:1.0.6"
There are several motivations for this:
- the user can disable an entire configuration snippet by toggeling
one entry.
This generalizes the functionality of the global-dns.enable
setting, but in a way that applies to configuration on a per-file
basis.
- for developing, we often switch between different versions of
NetworkManager. Thus, we might want to use different configuration.
E.g. before global-dns options, I want to use "dns=none" and manage
resolv.conf myself. Now, I can use global-dns setting to do that.
That can be achieved with something like the following (not exactly,
it's an example only):
[.config]
enable=nm-version-min:1.1
[main]
dns=default
[global-dns-domain-*]
nameserver=127.0.0.1
Arguably, this would be more awesome, if we would bump our micro devel
version (1.1.0) more often while developing 1.2.0 (*hint*).
- in principle, packages could drop configuration snippets and enable
them based on the NetworkManager version.
- with the "env:" spec, you can enable/disable snippets by configuring
an environment variable. Again, useful for testing and developing.
This is effectively the same as nm_config_parse_boolean(). The difference is,
that "nm-config.c" is not available to the interface-helper, thus any
code used by interface-helper (like "NetworkManager.c") cannot use this
function.
Still don't drop nm_config_parse_boolean() entirely, because it's better
to have the explicit notion of parsing a string in the config-context.
I ended up not using the function. But I'd still keep this patch.
