Report an error if the data type of the address-labels received via DBus
is not the expected.
Also, fix the assignment of the labels to their corresponding addresses.
As they are matched by array position, if any invalid address was
received, the array of addresses that we generate is shorter than the
array of address-labels. We were not considering this so we were
assigning the address-labels to incorrect addresses. Fix it by moving the
assignment of the labels to _nm_utils_ip4_addresses_from_variant, where
we still have the information of what the original position in the array
the address had.
Invalid addresses received via DBUS were just ignored and filtered out,
only emitting a warning to the logs. If there were still some valid
addresses, those were configured and the client was unaware of the
errors. Only if there was not any valid address at all and method was
manual, an error was returned from `verify`, but not reflecting the
real cause:
ipv4.addresses: this property cannot be empty for 'method=manual'
Check for invalid addresses errors in the `_from_dbus` functions. With
NM_SETTING_PARSE_FLAG_STRICT, parsing is aborted on first error and
error is returned. With NM_SETTING_PARSE_BEST_EFFORT, we keep parsing
and set only the valid values.
Actually, the invalid addresses were dropped in a helper function that
converts from GVariant to NMIPAddress. As it is part of the public API,
we can't change now its signature to add the GError argument. Instead,
create a new internal function and call it from the public one. The
public function will ignore the error, as it was doing previously, but
it won't emit any warning to avoid spamming the logs (we don't even
know if ignoring the invalid values was intentional when calling the
function). The new internal function might be made public in
the future, deprecating the other, but probably it is not necessary
because clients are never going to receive invalid addresses from the
daemon.
Do the same as explained above for DNS entries and routes.
Also, fix the documentation of nm_utils_ip_routes_to/from_dbus, which
said that it accepts new style routes but described the old style ones.
With enabled assertions via LIBNM_CLIENT_DEBUG=WARN or
LIBNM_CLIENT_DEBUG=ERROR, still print the warning/error message to the
default destination, along the trace/debug messages.
For example, when you set LIBNM_CLIENT_DEBUG_FILE, then we want that
those messages end up in the file too, not only in g_log() output.
Also, g_warning() prints to stderr. If you set
LIBNM_CLIENT_DEBUG="WARN,trace,stdout", then we printed the warning to
stderr and the trace messages to stdout.
All debug messages should and up at the same place, and the g_warning()
and g_critical() messages are additional.
Also because glib's g_log() supports its own redirection and suppression
mechanism.
Previously, it was odd. The enum values like NML_DBUS_LOG_LEVEL_DEBUG were
actually the bit mask of all the levels "debug", "warn" and "error".
On the other hand, when parsing _nml_dbus_log_level, that variable only contained
the flags that were exactly requested. E.g. when setting LIBNM_CLIENT_DEBUG=trace,
then _nml_dbus_log_level only contained the trace flag 0x02. That was useful,
because with "LIBNM_CLIENT_DEBUG=warn,trace" the "warn" flag was not redundant,
it was used to enable printing via g_warning(). That was confusing.
Now, "LIBNM_CLIENT_DEBUG=warn,trace" is the same as "LIBNM_CLIENT_DEBUG=trace".
To enable printing via g_warning(), use "LIBNM_CLIENT_DEBUG=WARN,trace".
With this, we don't need this backward representation of the flags. Invert
it. The level enums are now just single bits.
Document LIBNM_CLIENT_DEBUG under nm_utils_print().
Also, add an alias "warn" for "warning" flag.
Also, no longer special treat "error" and "warning" flags to indicate
printing via g_criticial()/g_warning(). Previously, you could get
assertions via
$ G_DEBUG=fatal-warnings LIBNM_CLIENT_DEBUG=error,warning,trace nmcli
or you could enable all messages (including <error>/<warn> level)
without assertions via
$ G_DEBUG=fatal-warnings LIBNM_CLIENT_DEBUG=trace nmcli
However, it was not possible to enable only <error>/<warn> levels
without those assertions.
Now, "error"/"warn"/"warning" behave just like "debug"/"trace" to enable
message up to the specified level. It only implies printing to stderr
(or stdout or file, depending on "stdout" flag and
LIBNM_CLIENT_DEBUG_FILE).
Now, to enable redirect to g_warning()/g_error() use the new keywords
"ERROR"/"WARN"/"WARNING".
For testing, we probably want to enable such assertions. So to be
mostly backward compatible, we can run with
$ G_DEBUG=fatal-warnings LIBNM_CLIENT_DEBUG=error,warning,WARN nmcli
with that, the "error","warning" flags are redundant on newer libnm and
the WARN is ignored on older libnm.
This option was only introduced only to allow keeping the old behavior
in RHEL7, while the default order was changed from 'ifindex' to 'name'
in RHEL8. The usefulness of this option is questionable, as 'name'
together with predictable interface names should give predictable order.
When not using predictable interface names, the name is unpredictable
but so is the ifindex.
https://issues.redhat.com/browse/NMT-926https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1814
Now that we require glib 2.42, we can use G_PARAM_EXPLICIT_NOTIFY flag.
The benefit is that this flag saves a notification, when the property
value does not change.
The downside is, that implementations of set_property() must remember to
emit _notify() when required. This is somewhat alleviated by using
_nm_setting_property_set_property_direct(), which does this
automatically.
Se the flag for G_PARAM_EXPLICIT_NOTIFY for direct boolean properties.
For now, only do it for boolean properties, because of the danger of
getting this wrong. We must review all callers to make sure that they
don't implement set_properties() and don't forget to notify.
We will deprecate "connection.master" for "connection.controller". The
old property will become an alias for the new one. That means, when
setting the property we must emit notifications for both the old and the
new property.
Add "NMSettInfoProperty.direct_also_notify" for that. This is not fully
flexible, as it only works for direct properties (duh) and only allows
to specify one addtitional GParamSpec (of the same NMSetting). It is
however sufficient for our use.
We have nm_gobject_notify_together(), which accepts a list of
_PropertyEnums arguments. Add nm_gobject_notify_together_by_pspec(),
which can use a param spec.
direct_hook is a union, which currently only has one member (the set function
for a direct string). Extending this might make sense, for other set functions
(e.g. overwriting setting a strv array).
However, then the name was bad. The union is already for the set-function of
direct properties, it's not a place for various kinds of hooks (as it is
a union).
Rename.
"nm-glib" h contains compat wrappers for older glib versions. This file
used to be copied over to VPN plugins, to use the same compat code. It
was thus interesting to also have compat code for glib versions, that
were no longer supported by NetworkManager itself.
This was fine. But glib 2.42 is more than 8 years old. At this point,
there really is no need to support that, even if you copy the file out
of NetworkManager source tree.
Drop those compat wrappers.
The base image for the "check-tree" test got bumped to Fedora 39. This
brings a new python-black version (23.7.0 vs. 22.8.0) and requires
reformatting.
Maybe we should stick to 22.8.0, via `pip install`. But it seems better
to just follow the latest black version (the one from current Fedora).
So do the reformatting instead.
https://black.readthedocs.io/en/stable/change_log.html#id38
Let's not have unexpected, non-thread-safe functions somewhere deep down.
NML3ConfigData -- as a data structure -- is not thread-safe, nor aims it to
be. However, our code(!) should be thread-safe. That means, it should be
possible to call our code on separate data from multiple threads.
Violating that is a code smell and a foot gun.
This basically means that code should not access global data (unless
thread-local) or that the access to global-data needs to be
synchronized.
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1806
This patch add support to HSR/PRP interface. Please notice that PRP
driver is represented as HSR too. They are different drivers but on
kernel they are integrated together.
HSR/PRP is a network protocol standard for Ethernet that provides
seamless failover against failure of any network component. It intends
to be transparent to the application. These protocols are useful for
applications that request high availability and short switchover time
e.g electrical substation or high power inverters.
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1791
Dynamic added routes i.e ECMP single-hop routes, are not managed by l3cd
as the other ones. Therefore, they need to be tracked properly and
marked as dirty when they are.
Without this, the state of those ECMP single hop routes is not properly
tracked, and they are for example not removed by NML3Cfg when they
should.
Usually, routes to be configured originate from the combined
NML3ConfigData and are resolved early during a commit. For example,
_obj_states_update_all() creates for each such route an obj_state_hash
entry. Let's call those static, or "non-dynamic".
Later, we can receive additional routes. We get them back from NMNetns
during nm_netns_ip_route_ecmp_commit() (_commit_collect_routes()).
Let's call them "dynamic".
For those routes, we also must track an obj-state.
Now we have two reasons why an obj-state is tracked. The "non-dynamic"
and the "dynamic" one. Add two flags "os_dynamic" and "os_non_dynamic"
to the ObjStateData and consider the flags at the necessary places.
Co-authored-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
When a single-hop route is merged with another single-hop route from a
different interface creating a new ECMP route, NetworkManager needs to
schedule a commit on the l3cfg that is managing the first single-hop
route. Otherwise, the single-hop will continue to be there until a new
commit is scheduled.
Fixes: d9d33e2acc ('netns: fix configuring onlink routes for ECMP routes')
nm_netns_ip_route_ecmp_commit() returns the ECMP routes that didn't find
a merge-partner. It expects NML3Cfg to configure such routes in
platform.
Note that these routes have a positive "weight", which was used for
tracking the ECMP information in NML3Cfg and NMNetns.
But in kernel, single-hop routes don't have a weight. All routes in
NMPlatform will have a zero weight. While it somewhat works to call
nm_platform_ip_route_add() with a single-hop route of non-zero weight,
the result will be a different(!) route. This means for example that
nm_platform_ip_route_sync() will search the NMPlatform cache for whether
the routes exist, but in the cache single-hop routes with positive weight
never exist. Previously this happened and nm_platform_ip_route_sync()
would not delete those routes when it should.
It's also important because NML3Cfg tracks the object states of routes
that it wants to configure, vs routes in kernel (NMPlatform cache). The
route in kernel has a weight of zero. The route we want to configure
must also have a weight of zero.
The solution is that nm_netns_ip_route_ecmp_commit() does not tell
NML3Cfg to add a route, that cannot be added. Instead, it must first
normalize the weight to zero, to have a "regular" single-hop route.
Fixes: 5b5ce42682 ('nm-netns: track ECMP routes')
It wouldn't work otherwise. The object state is used to track routes
and compare them to what we find in platform.
A "metric_any" is useful at higher layers, to track a route where the
metric is decided by somebody else. But at the point when we add such an
object to the object-state, a fixed metric must be chosen.
Assert for that.
The hash/cmp functions were wrong with respect to IPv4 routes. Fix that.
- the weight was cast to a guint8, which is too small to hold all
values.
- NM_PLATFORM_IP_ROUTE_CMP_TYPE_ID comparison would normalize a zero
weight to one
NM_CMP_DIRECT(NM_MAX(a->weight, 1u), NM_MAX(b->weight, 1u));
That was very wrong.
- the handling of the weight depends on the n_nexthops parameter.
See _ip4_route_weight_normalize().
The remarkable thing is that upper layers find it useful to track IPv4
single-hop routes with a non-zero weight. Consequently, this is honored
by NM_PLATFORM_IP_ROUTE_CMP_TYPE_ID to treat single-hop routes
different, when their weight differs. However, adding such a route in
kernel will not work. To kernel, single-hop routes have no weight. While
the route exists as distinct in our hash tables, according to the
implemented identity, it never exists in kernel (or NMPlatform cache).
Well, you can call nm_platform_ip_route_add() with such a route, but the
result will have a weight of zero (making it a different route). See also
nm_platform_ip_route_normalize().
This works all mostly fine. The only thing to take care is that you
cannot look into the NMPlatform cache and ever find a IPv4 single-hop
route with a non-zero weight. If you preform such a lookup, realize that
such routes don't exist in platform. You can however normalize the
weight to zero first (nm_platform_ip_route_normalize()) and look for a
similar route with a zero weight.
Fixes: 1bbdecf5e1 ('platform: manage ECMP routes')
Users should not be allowed to start or stop a wifi-p2p scan unless
they have some kind of permission. Since we already have the
"org.freedesktop.NetworkManager.wifi.scan" permission for wifi scans,
check that.
Fixes: dd0c59c468 ('core/devices: Add DBus methods to start/stop a P2P find')
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1795
It also doesn't make much sense to have this. We may use a
GPtrArray to construct and keep track of a (dynamic) strv list.
Then we add the strings to the GPtrArray one by one.
We almost never will want to create a GPtrArray based on a strv array.
It doesn't scale to export all addresses/routes on D-Bus as properties.
In particular not combined with PropertiesChanged signal. On a busy
system, this causes severe performance issues. It also doesn't seem very
useful. Routes and addresses are complex things (e.g. policy routing).
If you want to do anything serious, you must check netlink (or find
another way to get the information).
Note that NMPlatform already ignores routes of certain protocols
(ip_route_is_alive()). It also does not expose most route attributes,
making the output only useful for very limited cases (e.g. displaying to
the user for information).
Limit the number of exported entries to 100.
Try adding 100K routes one-by-one. Run a `nmcli monitor` instance.
Re-nice the nmcli process and/or keep the CPUs busy. Then start a script
that adds 100k routes. Observe. Glib's D-Bus worker thread receives the
messages and queues them for the main thread. The main thread is too
slow to process them, the memory consumption grows very quickly in Giga
bytes. Afterwards, the memory also is not returned to the operation
system, either because of fragmentation or because the libc allocator
does anyway not return heap memory.
It doesn't work to expose an unlimited number of objects on D-Bus. At
least not with an API, that sends the full list of all routes, whenever
a route changes. Nobody can use that feature either, because the only
use is a quick overview in `nmcli` output or a GUI. If you see 100+
routes there, that becomes unmanageable anyway. Instead use netlink if
you want to handle the full list of addresses/routes (or some other
API).
It doesn't scale. If you add 100k routes one-by-one, then upon each
change from platform, we will send the growing list of the routes on
D-Bus.
That is too expensive. Especially, if you imagine that the receiving end
is a NMClient instance. There is a D-Bus worker thread that queues the
received GVariant messages, while the main thread may not be able to
process them fast enough. In that case, the memory keeps growing very
fast and due to fragmentation it is not freed.
Instead, rate limit updates to 3 per second.
Note that the receive buffer of the netlink socket can fill up and we
loose messages. Therefore, already on the lowest level, we may miss
addresses/routes. Next, on top of NMPlatform, NMIPConfig listens to
NM_L3_CONFIG_NOTIFY_TYPE_PLATFORM_CHANGE_ON_IDLE events. Thereby it
further will miss intermediate state (e.g. a route that exists only for
a short moment).
Now adding another delay and rate limiting on top of that, does not make
that fundamentally different, we anyway didn't get all intermediate states
from netlink. We may miss addresses/routes that only exist for a short
amount of time. This makes "the problem" worse, but not fundamentally
new.
We can only get a (correct) settled state, after all events are
processed. And we never know, whether there isn't the next event
just waiting to be received.
Rate limiting is important to not overwhelm D-Bus clients. In reality,
none of the users really need this information, because it's also
incomplete. Users who really need to know addresses/routes should use
netlink or find another way (a way that scales and where they explicitly
request this information).
If you add a large number of addresses/routes, then the output of
`nmcli` is unusable. It also doesn't seem too useful.
Limit the number to show up to 10 addresses and 10 routes.
If there are more than 10 addresses, then print an 11th line with
inet4 ... N more
Actually, if there are exactly 11 addresses, then don't waste an extra
line to print "1 more". Instead, still print the 11th address. Same for
routes.
This was forgotten to implement. But we cannot just forget about it.
Libnm emits a warning about unknown properties, exactly to catch such
bugs. Properties that are not implemented, must be marked to be ignored.
Next, support for this property will be added. But that introduces new
API, which cannot be backported. Hence, first fix the problem by marking
the property as ignored. This is a backportable change.
$ LIBNM_CLIENT_DEBUG="warning" G_DEBUG=fatal-warnings nmcli
(process:270215): nm-WARNING **: 15:22:56.125: libnm-dbus: <warn > nmclient[8094a8c217aae461]: get-managed-objects: [/org/freedesktop/NetworkManager/Devices/5]: ignore unknown property org.freedesktop.NetworkManager.Device.IPTunnel.FwMark
Trace/breakpoint trap (core dumped)
Fixes: 351c562491 ('devices: support VTI tunnels')
This commit fixes the build process for the documentation that was previously
unable to build separately via meson due to a dependency issue.
Previously, trying to build the API documentation via `ninja NetworkManager-doc`
failed due to missing dependencies (for example, `nm-dbus-types.xml` was not built).
I believe this happens due to some different handling of static paths vs. custom_target
by meson in this case.
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1801
Fixes: 03637ad8b5 ('build: add initial support for meson build system')
For most strv or string properties, we cannot distinguish between
NULL/unset/default and empty.
It would be difficult to enter in nmcli or grasp how it differs. There
are probably many bugs, where we accept empty strings, and fail to
handle them correctly.
Anyway. For most strv arrays, and empty array and NULL/unset/default are
treated the same. That means, g_object_get() tends to always return NULL
(never an empty strv array) and g_object_set() of an empty strv array
will internally leave the GArray at NULL.
For a few properties, there is a difference. See "ipv[46].dns-options".
See also "clear_emptyunset_fcn" hook in libnm-setting.
Add a way to handle such strv properties with the "direct" mechanism.
