It's not the task of the ifcfg reader to pre-normalize profiles
to truncate the DNS server list. It's only nm_connection_verify()'s
task to indicate what is valid and what not.
Increase the number to something excessive. Note that the parsing
scales with O(n^2). So don't have it totally unbounded and have an
overall limit (of 10000 entries).
Now that NetworkManager on Fedora 33 and RHEL 9 no longer writes
ifcfg-rh files by default ([1]), ifup/ifdown became less useful.
Possibly users shouldn't use it and it would be fine that new-style profiles
(keyfile) no longer work with these commands. But this is deemed as too
disruptive for users.
Note that our previous ifup/ifdown compat scripts only honored the argument
to be part of the ifcfg filename. That was not what initscripts were doing,
which called `need_config()` function that searched also the contents of
the files. With this extension, ifup/ifdown gets smarter too, to better
guess what the user might have wanted.
Extend the script by making it smarter, and to work with connection profile
names.
With this extension we further solidify ifup/ifdown as part of NetworkManager
command line API. That is problematic, because these tools pollute the
$PATH, by not having a clear NM-specific name. Also, these scripts
should only exist on Fedora/RHEL, which makes their usage non-portable
to other distros.
Also, other distros already ship different tools with name ifup/ifdown.
Extending the use of these scripts is thus undesirable, as it furthers
distro-specific commands.
Still, these arguments seem to not hold and users need to be "helped".
As Fedora users cannot be expected to unlearn "ifup" today, there is no
reason to assume they could in a few years. This likely means we will
never get rid of these scripts.
Also, if we truly would make ifup/ifdown part of NetworkManager, then a better
implementation would be that nmcli honors being called with these names.
That is not done, because nmcli's implementation currently is not as
nice to make that extension trivial (as it should be). It also would
mean to embrace ifup/ifdown officially. A shell script works well enough
as a hack.
[1] https://fedoraproject.org/wiki/Changes/NetworkManager_keyfile_instead_of_ifcfg_rhhttps://bugzilla.redhat.com/show_bug.cgi?id=1954607https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/936
The "utils" part does not seem useful in the name.
Note that we also have NMStrBuf, which is named nm_str_buf_*().
There is an unfortunate similarity between the two, but it's still
distinct enough (in particular, because one takes an NMStrBuf and
the other not).
Naming is important, because the name of a thing should give you a good
idea what it does. Also, to find a thing, it needs a good name in the
first place. But naming is also hard.
Historically, some strv helper API was named as nm_utils_strv_*(),
and some API had a leading underscore (as it is internal API).
This was all inconsistent. Do some renaming and try to unify things.
We get rid of the leading underscore if this is just a regular
(internal) helper. But not for example from _nm_strv_find_first(),
because that is the implementation of nm_strv_find_first().
- _nm_utils_strv_cleanup() -> nm_strv_cleanup()
- _nm_utils_strv_cleanup_const() -> nm_strv_cleanup_const()
- _nm_utils_strv_cmp_n() -> _nm_strv_cmp_n()
- _nm_utils_strv_dup() -> _nm_strv_dup()
- _nm_utils_strv_dup_packed() -> _nm_strv_dup_packed()
- _nm_utils_strv_find_first() -> _nm_strv_find_first()
- _nm_utils_strv_sort() -> _nm_strv_sort()
- _nm_utils_strv_to_ptrarray() -> nm_strv_to_ptrarray()
- _nm_utils_strv_to_slist() -> nm_strv_to_gslist()
- nm_utils_strv_cmp_n() -> nm_strv_cmp_n()
- nm_utils_strv_dup() -> nm_strv_dup()
- nm_utils_strv_dup_packed() -> nm_strv_dup_packed()
- nm_utils_strv_dup_shallow_maybe_a() -> nm_strv_dup_shallow_maybe_a()
- nm_utils_strv_equal() -> nm_strv_equal()
- nm_utils_strv_find_binary_search() -> nm_strv_find_binary_search()
- nm_utils_strv_find_first() -> nm_strv_find_first()
- nm_utils_strv_make_deep_copied() -> nm_strv_make_deep_copied()
- nm_utils_strv_make_deep_copied_n() -> nm_strv_make_deep_copied_n()
- nm_utils_strv_make_deep_copied_nonnull() -> nm_strv_make_deep_copied_nonnull()
- nm_utils_strv_sort() -> nm_strv_sort()
Note that no names are swapped and none of the new names existed
previously. That means, all the new names are really new, which
simplifies to find errors due to this larger refactoring. E.g. if
you backport a patch from after this change to an old branch, you'll
get a compiler error and notice that something is missing.
Configuration can have [device*] and [connection*] settings and both
can include a 'match-device=' key, which is a list of device-specs.
Introduce a new 'allowed-connections' key for [device*] sections,
which specifies a list of connection-specs to indicate which
connections can be activated on the device.
With this, it becomes possible to have a device configuration like:
[device-enp1s0]
match-device=interface-name:enp1s0
allowed-connections=except:origin:nm-initrd-generator
so that NM in the real root ignores connections created by the
nm-initrd-generator, and starts activating a persistent
connection. This requires also setting 'keep-configuration=no' to not
generate an assumed connection.
This is a normalization employed by NMSettingIPConfig.gateway.
Also rework NMSettingIPConfig.set_property() to no longer assert against
valid input. We want to pass there untrusted strings from D-Bus,
asserting is a horrible idea. Instead, either normalize the string or
keep the invalid text that will be rejected by verify().
This is like using nm_ascii_is_ctrl_or_del() instead of
nm_ascii_is_ctrl() in the previous version of the patch.
We thus now always will switch to ANSIC escaping if we see
a ASCII DEL character. That is probable desirable, but either
way should not make a big difference (because we can parse
the DEL character both in regular quotation and in ANSIC quotation).
The patch is however larger, to also take the opportunity to only check
for nm_ascii_is_regular() in the "fast path". The behavior is the same
as changing nm_ascii_is_ctrl() to nm_ascii_is_ctrl_or_del().
Problems of this patch:
- the code does not differentiate between an ifcfg file and an alias
file. Different shell variables are honored however depending on the
context and the warning should reflect that.
- there are no warnings about /etc/sysconfig/network. The main problem
is that we read this file for every ifcfg file we parse, and we would
need to ratelimit the number of warnings. Another problem is that
the file likely contains keys that we intentionally don't support.
We would need a new way to omit warnings about those lines.
Example:
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=dhcp
DEFROUTE=yes
STABLE_ID=$'xxx\xF4yy'
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
XX=foo
XX1=foo'
'
IPV6_AUTOCONF=yes xxxx
IPV6_DEFROUTE=yesx
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=xxx
UUID=9d8ed7ff-3cdd-4336-9e26-3e978dc87102
ONBOOT=no
<warn> [...] ifcfg-rh: ifcfg,/etc/sysconfig/network-scripts/ifcfg-xxx:6: key STABLE_ID does not contain valid UTF-8 and is treated as ""
<debug> [...] ifcfg-rh: ifcfg,/etc/sysconfig/network-scripts/ifcfg-xxx:9: key XX is unknown and ignored
<warn> [...] ifcfg-rh: ifcfg,/etc/sysconfig/network-scripts/ifcfg-xxx:10: key XX1 is badly quoted and is treated as ""
<warn> [...] ifcfg-rh: ifcfg,/etc/sysconfig/network-scripts/ifcfg-xxx:11: invalid line ignored
<warn> [...] ifcfg-rh: ifcfg,/etc/sysconfig/network-scripts/ifcfg-xxx:12: key IPV6_AUTOCONF is badly quoted and is treated as ""
<warn> [...] ifcfg-rh: ifcfg,/etc/sysconfig/network-scripts/ifcfg-xxx:13: key IPV6_DEFROUTE is duplicated and the early occurrence ignored
https://bugzilla.redhat.com/show_bug.cgi?id=1959656
ifcfg files are a text format. It makes no sense to ever accept
non-UTF-8 blobs. If binary data is to be encoded in a ifcfg file, then
the upper layers must escape/encode it in valid UTF-8.
Let svUnescape() silently reject any binary "text". This will lead to treat such
strings as empty strings "". This is no different than some invalid
quoting: the string is not parsable as (UTF-8) text and will be treated
as such.
This is potentially a breaking change. But the benefit is that all the
upper layers can rely on only getting valid UTF-8 strings. For example,
a non-UTF-8 string cannot be converted to a "s" GVariant (of course not,
it's not a string). But our nm_connection_verify() commonly does not
check that all strings are in fact valid UTF-8. So a user who edits
an ifcfg file could inject non-valid strings, and cause assertion
failures later on.
It's actually easy to provoke a crash (or at least an assertion failure)
by writing an ifcfg file with certain keys as binary.
Note that you can either reproduce the binary files by writing non-UTF-8
"strings" dirctly, or by using \x, \u, or \U escape sequences.
Note that also '\0' gets rejected and renders the string as invalid
(i.e. as empty). Before the returned string would have been simply
truncated and the rest ignored. Such NUL bytes can only be produced
using the escape sequences, because the ifcfg reader already (silently)
truncates the file on the first binary NUL.
Note that previously the check
if (s[slen] < ' ') {
...
return (*to_free = _escape_ansic(s));
}
would be TRUE for all UTF-8 characters if `char` is signed. That means,
depending on the compiler, we would always ANSI escape all UTF-8
characters. With this patch, we no longer do that!
Instead, valid unicode gets now preserved (albeit quoted).
On the other hand, always ANSIC escape invalid UTF-8 (regardless of the
compiler). ifcfg-rh is really a text based format. If a caller wants to store
binary data, they need to escape it first, for example with some own escaping
scheme, base64 or bin2hexstr.
A caller passing a non-text to svEscape() is likely a bug already and
they should have not done that.
Still, let svEscape() handle that by using ANSIC escaping. That works
as far as escaping is concerned, but likely later will be a problem
during unescaping, when the reader expects a valid UTF-8 string.
svEscape() is in no place to signal a sensible error, so proceed the
best it can, by escaping.
Add a new property to specify the minimum time interval in
milliseconds for which dynamic IP configuration should be tried before
the connection succeeds.
This property is useful for example if both IPv4 and IPv6 are enabled
and are allowed to fail. Normally the connection succeeds as soon as
one of the two address families completes; by setting a required
timeout for e.g. IPv4, one can ensure that even if IP6 succeeds
earlier than IPv4, NetworkManager waits some time for IPv4 before the
connection becomes active.
We have two GKeyfile files (timestamps and seen-bssids).
When a profile was deleted while NetworkManager was running, then
entries were removed from these keyfiles. But if a profile disappeared
while NetworkManger was stopped, then those UUIDs piled up.
This also happens if you have temporary connections in /run and reboot.
We need a way to garbage collect entries that are no longer relevant.
As the keyfile databases only get loaded once from disk, we will prune
all UUIDs for which we have no more connection loaded, on the first time
we write out the files again.
Note what this means: if you "temporarily" remove a connection profile
(without NetworkManager noticing) and restore it later, then the additional
information might have been pruned. There is no way how NetworkManager
could know that this UUID is coming back. The alternative is what we did
before: pile them up indefinitely. That seems more problematic.
Previously, there was no limit how many seen-bssids are tracked.
That seems problematic, also because there is no API how to get
rid of an excessive list of entries.
We should limit the number of entries. Add an (arbitrary) limit
of 30.
But this means that we drop the surplus of entries, and for that it
seems important to keep the newest, most recently seen entries.
Previously, entries were merely sorted ASCIIbetically. Now, honor
their order (with most recently seen first).
Also, normalize the BSSIDs. From internal code, we should only get
normalize strings, but when we load them from disk, they might be bogus.
As we might cut of the list, we don't want that invalid entries
cut of valid ones. And of course, invalid entries make no sense at
all.
ifcfg-rh plugin never stored the seen bssid list to file, and
keyfile no longer does, and it's no longer parsed from GVariant.
So there is actually no way how anything could be set here.
The seen-bssids should only be populate from
"/var/lib/NetworkManager/seen-bssids". Nowhere else.
This function is badly named, because it has no NMHostnameManager self
argument. It's just a simple function that entirely operates on a string
argument.
Move it away from "nm-hostname-manager.h" to "libnm-glib-aux/nm-shared-utils.h".
Hostname handling is complicated enough. Simple string validation
functions should not obscure the view on the complicated parts.
We somehow need to encode an NMSettingEthtool instance that has all
options unset. Previously, that would result in no "$ETHTOOL_OPTS"
variable and thus the reader would loose a previously existing setting.
Hack it by writing a bogus
ETHTOOL_OPTS="-A $IFACE"
line.
It just seems ugly to call g_getenv() repeatedly. Environment variables
must not change (in a multi-threaded program after other threads start),
so determine the mode once and cache it.
For the umpteenth time: it is not ifcfg-rh writers decision to decide
what are valid configurations and only persist settings based on
some other settings.
If s390-options would only be allowed together with subchannels, then
this is alone nm_connection_verify()'s task to ensure.
Reproduce with
$ nmcli connection add type ethernet autoconnect no con-name zz ethernet.s390-options bridge_role=primary
Related: https://bugzilla.redhat.com/show_bug.cgi?id=1935842
Fixes: 16bccfd672 ('core: handle s390 options more cleanly')
We now have a cached list of NMSettingsConnection instances,
sorted by their autoconnect priority.
However, the sort order nm_settings_connection_cmp_autoconnect_priority()
depends on various properties of the connection:
- "connection.autoconnect" and "connection.autoconnect-priority"
- the timestamp
- "connection.uuid"
These properties almost never change, so it's a waste that every call
to nm_settings_get_connections_sorted_by_autoconnect_priority() needs
to check whether the sort order is still satisfied.
We can do better by tracking when the sort order might have been
destroyed and only check in those (much fewer) cases.
Note that we end up calling nm_settings_get_connections_sorted_by_autoconnect_priority()
a lot, so this makes a difference.
The GetSettings() call is not the only place where we convert a
NMConnection to D-Bus. However it is one of the most prominent ones
with a measurable performance overhead.
The connection seldom changes, so it makes sense to cache it.
Note that GetSettings() is the only caller that specifies an option,
thus it's the only caller that converts a NMConnection to variant
in this particular way. That means, other callers don't benefit from
this caching and we could not cache the variant in the NMConnection
instance itself, because those callers use different parameters.
Turns out, we call nm_settings_get_connection_clone() *a lot* with sort order
nm_settings_connection_cmp_autoconnect_priority_p_with_data().
As we cache the (differently sorted) list of connections, also cache
the presorted list. The only complication is that every time we still
need to check whether the list is still sorted, because it would be
more complicated to invalidate the list when an entry changes which
affects the sort order. Still, such a check is usually successful
and requires "only" N-1 comparisons.
Since commit 207cf3d5d4 ('libnm: normalize "connection.uuid"') the
"connection.uuid" is normalized to be a valid UUID and all lower case.
That means, if we have .nmmeta files on disk with a previously valid,
but now invalid UUID, the meta file is no longer going to match.
Reject such file outright as invalid. If we really wanted to preserve
backward compatibility, then we would have to also normalize the
filename when we read it. However, that means, that suddenly we might
have any number of compatible .nmmeta files that normalize to the same
UUID, like the files
71088c75dec54119ab41be71bc10e736aaaabbbb.nmmeta
F95D40B4-578A-5E68-8597-39392249442B.nmmeta
f95d40b4-578a-5e68-8597-39392249442b.nmmeta
Having multiple places for the nmmeta file is complicated to handle.
Also, we often have the connection profile (and the normalized UUID)
first, and then check whether it has a .nmmeta file. If we would support
those unnormalized file names, we would have to visit all file names and
try to normalize it, to find those with a matching UUID.
Non-normalized UUIDs really should not be used and they already are not
working anymore for the .nmmeta file. This commit only outright rejects
them. This is a change in behavior, but the behavior change happened
earlier when we started normalizing "connection.uuid".
"uuid" is returned from nms_keyfile_nmmeta_check_filename(),
and contains "$UUID.nmmeta". We must compare only the first
"uuid_len" bytes.
Fixes: 064544cc07 ('settings: support storing "shadowed-storage" to .nmmeta files')
Most of the time, we care about whether we have a normalized UUID.
nm_uuid_is_valid_full() only exists for a particular case where we want
to use the function in a header, without including "nm-uuid.h". In that
case, we actually also care about normalized UUIDs.
If the TC setting contains no qdiscs and filters, it is lost after a
write-read cycle. Fix this by adding a new property to indicate the
presence of the (empty) setting.
In the past, the UUID was only loosely validate and would accept
forms that are not valid. This was fixed by commit 207cf3d5d4 ('libnm:
normalize "connection.uuid"'). Now the UUID is always strictly valid
and lower case.
Thus, don't use the fuzzy nm_utils_is_uuid() from libnm but the exact
check nm_uuid_is_valid_full().
Note that this is only used for assertions in the header file. We thus
don't want to drag in "libnm-glib-aux/nm-uuid.h". Instead, we forward
declare the function.
lgtm.com warns about declarations are block scope, so fix that too by
moving the declaration at file scope.
lgtm.com warns about function declarations inside blocks.
*sigh*. I think it's well understood what this code means, and it is not
done by accident. Still, let's make the tool happy in this case.
Previously, we would allocate a buffer of the worst case, that is,
4 times the number of bytes, in case all of them require octal escaping.
Coverity doesn't like _escape_ansic() for another reason:
Error: NULL_RETURNS (CWE-476): [#def298]
NetworkManager-1.31.5/src/core/settings/plugins/ifcfg-rh/shvar.c:161: returned_null: "g_malloc" returns "NULL".
NetworkManager-1.31.5/src/core/settings/plugins/ifcfg-rh/shvar.c:161: alias: Assigning: "q" = "dest = g_malloc(strlen(source) * 4UL + 1UL + 3UL)". Both pointers are now "NULL".
NetworkManager-1.31.5/src/core/settings/plugins/ifcfg-rh/shvar.c:163: dereference: Incrementing a pointer which might be null: "q".
# 161| q = dest = g_malloc(strlen(source) * 4 + 1 + 3);
# 162|
# 163|-> *q++ = '$';
# 164| *q++ = '\'';
# 165|
It doesn't recognize that g_malloc() shouldn't return NULL (because
we never request zero bytes).
I am not sure how to avoid that, but let's rework the code to first count
how many characters we exactly need. It think that should also help with
the coverity warning.
Doing exact allocation requires first to count the number of required
bytes. It still might be worth it, because we might keep the allocated
strings a bit longer around.
When the ACCEPT_ALL_MAC_ADDRESSES key is found by the wired reader, the
wired setting was not being created.
Fixes: d946aa0c50 ('wired-setting: add support to accept-all-mac-addresses')
Signed-off-by: Fernando Fernandez Mancera <ffmancera@riseup.net>
D-Bus 1.3.1 (2010) introduced the standard "PropertiesChanged" signal
on "org.freedesktop.DBus.Properties". NetworkManager is old, and predates
this API. From that time, it still had it's own PropertiesChanged signal
that are emitted together with the standard ones. NetworkManager
supports the standard PropertiesChanged signal since it switched to
gdbus library in version 1.2.0 (2016).
These own signals are deprecated for a long time already ([1], 2016), and
are hopefully not used by anybody anymore. libnm-glib was using them and
relied on them, but that library is gone. libnm does not use them and neither
does plasma-nm.
Hopefully no users are left that are affected by this API break.
[1] 6fb917178a
Introducing ethtool PAUSE support with:
* ethtool.pause-autoneg on/off
* ethtool.pause-rx on/off
* ethtool.pause-tx on/off
Limitations:
* When `ethtool.pause-autoneg` is set to true, the `ethtool.pause-rx`
and `ethtool.pause-tx` will be ignored. We don't have warning for
this yet.
Unit test case included.
Signed-off-by: Gris Ge <fge@redhat.com>
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/829
It's not the task of the writer to mangle/normalize profiles. If a profile
for a virtual device can have an [ethernet] setting, then unsuitable values
like s390 options must be either rejected by nm_connection_verify() or normalized
by nm_connection_normalize(). In no way it's right that the writer simple
pretends they are not set.
