During srv_shutdown() we do
p.stdin.close()
p.kill()
Usually, the kill wins and the service just drops off the bus:
libnm-dbus[3201919]: <debug> [438617.45324] nmclient[40f7938626f3f5f0]: name owner changed: ":1.1" -> (null)
libnm-dbus[3201919]: <debug> [438617.45332] nmclient[40f7938626f3f5f0]: release all
at which point all objects in NMClient get destroyed and the signals get
emitted in the order:
libnm-dbus[3201919]: <trace> [438617.45574] nmclient[40f7938626f3f5f0]: [nmclient] emit "device-removed" signal for /org/freedesktop/NetworkManager/Devices/1
nmcli[out]: eth0: device removed
libnm-dbus[3201919]: <trace> [438617.45590] nmclient[40f7938626f3f5f0]: [nmclient] emit "any-device-removed" signal for /org/freedesktop/NetworkManager/Devices/1
libnm-dbus[3201919]: <trace> [438617.45593] nmclient[40f7938626f3f5f0]: [nmclient] emit "connection-removed" signal for /org/freedesktop/NetworkManager/Settings/Connectio>
nmcli[out]: con-1: connection profile removed
However, sometimes the stub service notices that stdin was closed and it
sends signals about shutting down:
libnm-dbus[3201061]: <trace> [438226.44965] nmclient[401639659459c316]: interfaces-removed: [/org/freedesktop/NetworkManager/Settings] receive interface remove event for >
libnm-dbus[3201061]: <trace> [438226.44966] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager/Settings]: changed-type 0x01 linked
libnm-dbus[3201061]: <trace> [438226.44967] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager/Settings]: changed-type 0x01 consumed
libnm-dbus[3201061]: <trace> [438226.44968] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager/Settings]: changed-type 0x02 linked
libnm-dbus[3201061]: <trace> [438226.44969] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager/Settings]: unregister NMClient from D-Bus object
libnm-dbus[3201061]: <trace> [438226.44971] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager/Settings]: drop D-Bus instance
libnm-dbus[3201061]: <trace> [438226.44971] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager/Settings]: set D-Bus object state unlinked
libnm-dbus[3201061]: <trace> [438226.44972] nmclient[401639659459c316]: [nmclient] emit "connection-removed" signal for /org/freedesktop/NetworkManager/Settings/Connectio>
nmcli[out]: con-1: connection profile removed
libnm-dbus[3201061]: <trace> [438226.44992] nmclient[401639659459c316]: interfaces-removed: [/org/freedesktop/NetworkManager] receive interface remove event for interface>
libnm-dbus[3201061]: <trace> [438226.44994] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager]: changed-type 0x01 linked
libnm-dbus[3201061]: <trace> [438226.44995] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager]: changed-type 0x01 consumed
libnm-dbus[3201061]: <trace> [438226.44996] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager]: changed-type 0x02 linked
libnm-dbus[3201061]: <trace> [438226.44998] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager]: unregister NMClient from D-Bus object
libnm-dbus[3201061]: <trace> [438226.44999] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager]: drop D-Bus instance
libnm-dbus[3201061]: <trace> [438226.45000] nmclient[401639659459c316]: [/org/freedesktop/NetworkManager]: set D-Bus object state unlinked
libnm-dbus[3201061]: <trace> [438226.45001] nmclient[401639659459c316]: [nmclient] emit "device-removed" signal for /org/freedesktop/NetworkManager/Devices/1
nmcli[out]: eth0: device removed
libnm-dbus[3201061]: <trace> [438226.45005] nmclient[401639659459c316]: [nmclient] emit "any-device-removed" signal for /org/freedesktop/NetworkManager/Devices/1
nmcli[out]: NetworkManager is stopped
libnm-dbus[3201061]: <debug> [438226.45545] nmclient[401639659459c316]: name owner changed: ":1.1" -> (null)
libnm-dbus[3201061]: <debug> [438226.45550] nmclient[401639659459c316]: release all
The fix is to accept the events in any order.
The test stub service watches stdin, and if it gets closed the service
will shut down. Note that the service does not catch any signals, so
sending a signal will kill the service right away.
The previous code first closed stdin, and then killed the process.
That can result in different outcomes on D-Bus. Usually the signal
gets received first, and the test service just drops off the bus.
Sometimes it notices the closing of stdin and shuts actively down.
That can make a difference, especially for the test_monitor() test which
runs the monitor while stopping the service.
We could just always kill the stub service to get consistent behavior.
However, that doesn't seem very useful. Instead, randomize the behavior
to easier see how the behavior differs.
For debugging libnm, LIBNM_CLIENT_DEBUG can be very useful.
As the tests compare stdout/stderr from nmcli with expected output, just
enabling it will break the tests. However, in combination with
LIBNM_CLIENT_DEBUG_FILE this can be very useful.
Preserve and pass on the environment variables, if set.
Sort imports by name. Also avoid "from signal import SIGINT". I find
it ugly to import names in the current namespace. "SIGINT" should be
refered to by its full name, including the package/namespace.
This will allow to find some memory leaks and memory corruptions.
The bulk of the nmcli calls are still not hooked up with valgrind.
Since we call nmcli a thousand time, we could not just run valgrind with
all of them. We would have instead to enable it randomly. This is
more work.
The base class is not used, and it's not clear that it would be useful.
Sure, we could extend "test-client.py" will various non-nmcli tests. That
might make sense. And then it might make sense to have more unit test classes.
So far, we don't need that. Drop the unused base class NmTestBase.
Under normal circumstances, the timeout is not supposed to be hit.
I see it hit on gitlab-ci. Was that because the machine was very
busy? It's hard to say whether there was a legitimate problem here,
and more importantly, what that problem was.
Try to increase the timeout. If there is a real problem, we probably
will still hit the timeout.
In case the D-Bus interfaces start dropping off (typically all off them go
one by one when the object is being deleted), don't reset all the properties.
In particular, keep most properties around, only tear down "o" and "ao",
so that the object dependencies get torn down, but we still get enough
properties around to identify what the dead object was its heyday.
One example of where this is not good is when the device-removed signal
is emmitted, the device no longer has the ifname:
$ nmcli monitor
<quit NetworkManager>
(null): device removed
(null): device removed
...
ModuleNotFoundError was only introduced in later python 3 versions.
Use just "ImportError", which is the parent class anyway.
Fixes: f7e484c8ed ('tests: fix "test-client.py" ignoring missing "NM" module')
(cherry picked from commit 9902373c6d)
It can be useful to choose a different "ipv6.addr-gen-mode". And it can be
useful to override the default for a set of profiles.
For example, in cloud or in a data center, stable-privacy might not be
the best choice. Add a mechanism to override the default via global defaults
in NetworkManager.conf:
# /etc/NetworkManager/conf.d/90-ipv6-addr-gen-mode-override.conf
[connection-90-ipv6-addr-gen-mode-override]
match-device=type:ethernet
ipv6.addr-gen-mode=0
"ipv6.addr-gen-mode" is a special property, because its default depends on
the component that configures the profile.
- when read from disk (keyfile and ifcfg-rh), a missing addr-gen-mode
key means to default to "eui64".
- when configured via D-Bus, a missing addr-gen-mode property means to
default to "stable-privacy".
- libnm's ip6-config::addr-gen-mode property defaults to
"stable-privacy".
- when some tool creates a profile, they either can explicitly
set the mode, or they get the default of the underlying mechanisms
above.
- nm-initrd-generator explicitly sets "eui64" for profiles it creates.
- nmcli doesn' explicitly set it, but inherits the default form
libnm's ip6-config::addr-gen-mode.
- when NM creates a auto-default-connection for ethernet ("Wired connection 1"),
it inherits the default from libnm's ip6-config::addr-gen-mode.
Global connection defaults only take effect when the per-profile
value is set to a special default/unset value. To account for the
different cases above, we add two such special values: "default" and
"default-or-eui64". That's something we didn't do before, but it seams
useful and easy to understand.
Also, this neatly expresses the current behaviors we already have. E.g.
if you don't specify the "addr-gen-mode" in a keyfile, "default-or-eui64"
is a pretty clear thing.
Note that usually we cannot change default values, in particular not for
libnm's properties. That is because we don't serialize the default
values to D-Bus/keyfile, so if we change the default, we change
behavior. Here we change from "stable-privacy" to "default" and
from "eui64" to "default-or-eui64". That means, the user only experiences
a change in behavior, if they have a ".conf" file that overrides the default.
https://bugzilla.redhat.com/show_bug.cgi?id=1743161https://bugzilla.redhat.com/show_bug.cgi?id=2082682
See-also: https://github.com/coreos/fedora-coreos-tracker/issues/907https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1213
The property wait-activation-delay will delay the activation of an
interface the specified amount of milliseconds. Please notice that it
could be delayed some milliseconds more due to other events in
NetworkManager.
This could be used in multiple scenarios where the user needs to define
an arbitrary delay e.g LACP bond configure where the LACP negotiation
takes a few seconds and traffic is not allowed, so they would like to
use nm-online and a setting configured with this new property to wait
some seconds. Therefore, when nm-online is finished, LACP bond should be
ready to receive traffic.
The delay will happen right before the device is ready to be activated.
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1248https://bugzilla.redhat.com/show_bug.cgi?id=2008337
Introduction of a new setting ipv4.link-local, which enables
link-local IP addresses concurrently with other IP address assignment
implementations such as dhcp or manually.
No way is implemented to obtain a link-local address as a fallback when
dhcp does not respond (as dhcpd does, for example). This could be be
added later.
To maintain backward compatibility with ipv4.method ipv4.link-local has
lower priority than ipv4.method. This results in:
* method=link-local overrules link-local=disabled
* method=disabled overrules link-local=enabled
Furthermore, link-local=auto means that method defines whether
link-local is enabled or disabled:
* method=link-local --> link-local=enabled
* else --> link-local=disabled
The upside is, that this implementation requires no normalization.
Normalization is confusing to implement, because to get it really
right, we probably should support normalizing link-local based on
method, but also vice versa. And since the method affects how other
properties validate/normalize, it's hard to normalize that one, so that
the result makes sense. Normalization is also often not great to the
user, because it basically means to modify the profile based on other
settings.
The downside is that the auto flag becomes API and exists because
we need backward compatibility with ipv4.method.
We would never add this flag, if we would redesign "ipv4.method"
(by replacing by per-method-specific settings).
Defining a default setting for ipv4.link-local in the global
configuration is also supported.
The default setting for the new property can be "default", since old
users upgrading to a new version that supports ipv4.link-local will not
have configured the global default in NetworkManager.conf. Therefore,
they will always use the expected "auto" default unless they change
their configuration.
Co-Authored-By: Thomas Haller <thaller@redhat.com>
The source file now contains UTF-8 (non ASCII) characters. Python2
doesn't like that:
"./src/tests/client/test-client.sh" "." "." "/usr/bin/python"
File "/builddir/build/BUILD/NetworkManager-1.39.2/src/tests/client/test-client.py", line 1730
SyntaxError: Non-ASCII character '\xe2' in file /builddir/build/BUILD/NetworkManager-1.39.2/src/tests/client/test-client.py on line 1730, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details
This test calls "nmcli g" with a bogus D-Bus bus address. We expect
a failure on stderr.
On alpine:latest, the error however looks slightly different:
b'size: 258\nlocation: src/tests/client/test-client.py:test_offline()/1\ncmd: $NMCLI g\nlang: C\nreturncode: 1\nstderr: 136 bytes\n>>>\nError: Could not create NMClient object: Key/Value pair 0, *invalid*, in address element *very:invalid* does not contain an equal sign.\n\n<<<\n'
On ubuntu:16.04 and debian:9 we got:
b"size: 258\nlocation: src/tests/client/test-client.py:test_offline()/1\ncmd: $NMCLI g\nlang: C\nreturncode: 1\nstderr: 136 bytes\n>>>\nError: Could not create NMClient object: Key/Value pair 0, 'invalid', in address element 'very:invalid' does not contain an equal sign.\n\n<<<\n"
On fedora and most recent systemd we got:
b'size: 258\nlocation: src/tests/client/test-client.py:test_offline()/1\ncmd: $NMCLI g\nlang: C\nreturncode: 1\nstderr: 136 bytes\n>>>\nError: Could not create NMClient object: Key/Value pair 0, ?invalid?, in address element ?very:invalid? does not contain an equal sign.\n\n<<<\n'
This depends on the glib version (whether to print `%s', '%s', or “%s”).
Also, as we run the application with lang=C, so that libc (I think)
replaces Unicode with an ASCII character. Here musl and glibc behave
differently.
Workaround by replace the unexpected text.
Let the replace_arr parameter of Util.replace_text() only contain
callables, and don't special case the argument. Previously, we
either expected a regex or a 2-tuple, and the code would check
each explicitly.
Aside that the tuples are hard to follow, it also makes
Util.replace_text() strongly tied to those types. By instead accepting
and arbitrary function, each element can implement its own replacement.
Also, make text that was replaced by regex atomic. Meaning, if we
first match (and replace) a certain part of the text with the regex,
then the replacement cannot be split again. This is done by returning
a 1-tuple from the replace function.
Seems the previous code did not work properly:
With python36-3.6.8-38.module+el8.5.0+12207+5c5719bc.x86_6 on rhel-8.6:
Traceback (most recent call last):
File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 1157, in f
func(self)
File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 1724, in test_offline
replace_stdout=replace_uuids,
File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 797, in call_nmcli
frame,
File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 997, in _call_nmcli
self.async_start(wait_all=sync_barrier)
File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 1032, in async_start
async_job.wait_and_complete()
File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 670, in wait_and_complete
self._complete_cb(self, return_code, stdout, stderr)
File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 919, in complete_cb
stdout = Util.replace_text(stdout, replace_stdout)
File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 362, in replace_text
if Util.is_regex_pattern(v_search):
File "/root/nm-build/NetworkManager/src/tests/client/test-client.py", line 208, in is_regex_pattern
t = re.Pattern
AttributeError: module 're' has no attribute 'Pattern'
On this python version, re.compile() give an object of type
_sre.SRE_Pattern.
# python -c 'import re; print(type(re.compile("a")))'
<class '_sre.SRE_Pattern'>
Fixes: beebde9e56 ('client/test: allow matching and replacing regex-es in nmcli output')
Set extra variables after the pre-defined ones have been set. This
allows overriding then.
In particular, this allows overriding DBUS_SESSION_BUS_ADDRESS so that
the test can check the behavior of the client is correct when it's set
to some garbage.
Add a new property to specify the minimum time interval in
milliseconds for which dynamic IP configuration should be tried before
the connection succeeds.
This property is useful for example if both IPv4 and IPv6 are enabled
and are allowed to fail. Normally the connection succeeds as soon as
one of the two address families completes; by setting a required
timeout for e.g. IPv4, one can ensure that even if IP6 succeeds
earlier than IPv4, NetworkManager waits some time for IPv4 before the
connection becomes active.
Most string properties can be either %NULL (unset) or a non-empty
string.
For a few properties, like "gsm.apn", also the empty word is a valid
value. That makes it problematic to use from nmcli, because
nmcli connection modify "$PROFILE" gsm.apn ""
means to reset the default (NULL). How to configure the empty word?
For the APN, "" has a specific meaning, distinct from NULL, so we
need to be able to represent that.
The other problem with nmcli is that
nmcli -g gsm.apn connection show "$PROFILE"
is supposed to give you a value that you an set again, like
X="$(nmcli -g gsm.apn connection show "$PROFILE"; echo x)"
nmcli connection modify "$PROFILE2" gsm.apn "${X%$'\n'x}"
but for %NULL and "" the output would be the same.
The "solution" to that is interpreting "" as NULL (like we always did)
and a non-empty string that contains all whitespace, like a string with
one whitespace less. This way, all values can be expressed.
Note that in case of "gsm.apn", the string is anyway internally
normalized with g_strstrip(), so a string with all whitespace was
not expressable.
This patch is introducing the wired setting accept-all-mac-addresses
property. The value corresponds to the kernel flag IFF_PROMISC.
When accept-all-mac-address is enabled, the interface will accept all
the packets without checking the destination mac address.
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
