Via Update2() D-Bus API there are three ways how a profile can be stored
(or migrated) to in-memory:
- NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY
- NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED
- NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_ONLY
With the recent rework of settings I dropped NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY
and it had the same meaning as NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED.
However, the way NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED was implemented is
problematic. The problem is that it leaves the profile on disk but creates an
in-memory representation which shadows the persistent storage. Later,
when storing the profile to disk again, a new filename is chosen.
This allows via D-Bus API to toggle between NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED
and NM_SETTINGS_UPDATE2_FLAG_TO_DISK, and thereby pilling up profiles on disk.
Also, there is no D-Bus API to do anything sensible with these leaked, shadowed
profiles on disk.
Note that if we have a read-only profile in /usr/lib or in ifupdown
plugin, then the problem is not made any worse. That is, because via D-Bus
API such profiles can be made in-memory, and afterwards stored to /etc.
Thereby too the profile gets duplicate on disk, but this game only
works once. Afterwards, you cannot repeat it to create additional
profiles on disk. It means, you can only leak profiles once, and only
if they already exist in read-only storage to begin with.
This problem with NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED already existed
before the settings-delegate-storage rework, and is unrelated to whether in-memory
profiles now happen to be persisted to /run.
Note that NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_ONLY is simple and does not suffer
from this problem. When you move a profile to in-memory-only, it gets deleted from
persistent storage and no duplication happens.
The problem is that NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED used to
forget about the profile that it shadows, and that is wrong.
So, first re-add proper support for NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY. This
works by remembering the "shadowed-storage" path for in-memory profiles.
When later saving such a profile to disk again, the shadowed-storage
will be re-used. Likewise, when deleting such a profile, the shadowed
storage will be deleted.
Note that we keep NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED and it
also remembers the shadowed storage (but without "owning" it). That means,
when such a profile gets saved to disk again, the orginal storage is
reused too. As such, during future updates it behaves just like
NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY. The difference is when deleting
such a profile. In this case, the profile is left on storage and a
tombstone gets written. So, how is this better than before and why even
keep this complicated flag?
First, we keep this flag because we really want the ansible role to be
able to do in-memory changes only. That implies being able to delete a
profile from NetworkManager's view, but not from persistent storage. Without
this flag there is no way to do that. You can only modify an on-disk profile
by shadowing it, but you could not delete it form NetworkManager's view
while keeping it on disk.
The new form of NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED is safe and avoids
the duplication problem because also for tombstones it remembers the original
"shadowed-storage". That is, when the profile gets recreated later via
D-Bus API AddConnection, then the re-created profile will still reference
and reuse the shadowed storage that it had before deletion.
Say, you configure "plugins=ifcfg-rh,keyfile" and have an ifcfg file for
a certain profile. Then you make it IN-MEMORY-DETACHED and delete it.
The result is that the ifcfg file still exists, but there is a tombstone
nmmeta file that shadows the profile.
Now, if you want to re-add the same profile (same UUID) and store it to
persistent storage, then first it will try to persist the profile via
the ifcfg-rh plugin. That may not be possible, for example because the
connection type is not supported by the plugin.
Now, you could write it to /etc as keyfile, but then there would still
be a higher priority profile. Note that after calling
_add_connection_to_first_plugin() we issue
_connection_changed_track (self, new_storage, new_connection, TRUE);
(note the prioritize=TRUE parameter). So, in the first moment, all looks
good. However it is not because upon first reload the change gets
reverted and the other profile resurfaces.
The problem are that all settings plugins (except keyfile) may be
completely unable to persist a profile. The real and only solution is
not to use any settings plugins except keyfile.
In a previous version (that never was merged), the "loaded-path" of nmmeta
files was used to prioritize profiles. Since that was not done, we
should not add profiles that we know have lower priority than existing
profiles.
Before, the .nmmeta file could only contain one piece of information:
the loaded-path. This was persisted to disk by writing a "$UUID.nmmeta"
symlink that links to the loaded-path. Also, in practice this is used
for tombstones, so the only valid loaded-path is "/dev/null" (all other
paths are ignored).
Extend the .nmmeta file format to also be able to store additional data: the
shadowed-storage path. We will need that later but the idea is that if
we have a tombstone on disk, then this tombstone might explicitly shadow
another file. The use is when re-adding a profile with the same UUID, then
the existing storage is used (instead of creating a new file). This will
be necessary with Update2(NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED)
flag. This flag first allows to clone a profile from persistent storage
to a profile in /run. Later, when this profile gets deleted, the
original profile will be left on disk. If the same profile then gets
re-created with AddConnection(), then the original filename must be
taken over again. This is to avoid duplication of profiles on disk.
Note that this piece of information is relevent per-UUID, and as such
it's correct to store it in the .nmmeta file. That is related to the
"shadowed-storage" information that we store in the [.nmmeta] section
of keyfiles.
When we make runtime only changes, we may leave the profile in persistent
storage and store a overlay profile in /run. However, in various cases we need
to remember the original path.
Add code to store and retrieve that path from the keyfile.
Note that this data is written inside the keyfile in /run. That is because
this piece of information really depends on that particular keyfile, and not
on the profile/UUID. That is why we write it to the [.nmmeta] section of the
keyfile and not to the .nmmeta file (which is per-UUID).
This patch only adds the backend to write and load the setting from
disk. It's not yet used.
Currently, meta-data has a very narrow use: as tombstones.
Later, we will need to store additional per UUID meta-data. For example,
when a profile becomes unsaved, we may need to remember the original
filename.
Refactor the code for that. This is for the most part just renaming
and slightly different handling of the fields.
Make use of the new API. Note that AddConnection2() covers all
functionality of AddConnection() and AddConnectionUnsaved(). Let's
only use one API for all.
There is a minor downside to this patch: now nmcli requires
libnm 1.20 API. Note that libnm's nm_client_add_connection2()
makes an effort to avoid AddConnection2() under the hood to
still work against older server versions. So, you can use nmcli
with libnm 1.20 to talk to older versions of NetworkManager.
But with this change nmcli strictly requires libnm 1.20. I think that is
sensible because commonly nmcli requires a libnm version that is as new
as itself.
Also, the value of allowing nmcli to talk to older NetworkManager
versions is during package upgrade (where the daemon might not be
restarted). This is much less concern w.r.t. to updating the nmcli/libnm
combo, which is commonly packaged together.
It should be possible to add a profile with autoconnect blocked form the
start. Update2() has a %NM_SETTINGS_UPDATE2_FLAG_BLOCK_AUTOCONNECT flag to
block autoconnect, and so we need something similar when adding a connection.
As the existing AddConnection() and AddConnectionUnsaved() API is not
extensible, add AddConnection2() that has flags and room for additional
arguments.
Then add and implement the new flag %NM_SETTINGS_ADD_CONNECTION2_FLAG_BLOCK_AUTOCONNECT
for AddConnection2().
Note that libnm's nm_client_add_connection2() API can completely replace
the existing nm_client_add_connection_async() call. In particular, it
will automatically prefer to call the D-Bus methods AddConnection() and
AddConnectionUnsaved(), in order to work with server versions older than
1.20. The purpose of this is that when upgrading the package, the
running NetworkManager might still be older than the installed libnm.
Anyway, so since nm_client_add_connection2_finish() also has a result
output, the caller needs to decide whether he cares about that result.
Hence it has an argument ignore_out_result, which allows to fallback to
the old API. One might argue that a caller who doesn't care about the
output results while still wanting to be backward compatible, should
itself choose to call nm_client_add_connection_async() or
nm_client_add_connection2(). But instead, it's more convenient if the
new function can fully replace the old one, so that the caller does not
need to switch which start/finish method to call.
https://bugzilla.redhat.com/show_bug.cgi?id=1677068
When modifying a connection profile that happens to be active on a
device, then most of the changes don't take effect immediately.
Only after a full re-activation or reapply (nmcli device reapply)
does the configuration of the active device change (the
"applied-connection").
With two execptions: "connection.zone" and "connection.metered" take
effect immediately. I think this is historic, but also to facilitate
firewall-cmd to modify a profile and change the zone right away.
Anyway, I think it should be possible to modify a profile without
changes to the runtime. Add a flag to prevent reapplying these
properties right away.
https://bugzilla.redhat.com/show_bug.cgi?id=1677070
- don't let no_auto_default_from_file() do any preprocessing of
the lines that it reads. It merely splits the lines at '\n'
and utf8safe-unescapes them.
This was previously duplicated also by NMConfigData's property
setter. We don't need to do it twice.
- sort the lines. This makes the entire handling O(n*ln(n)) instead
of O(n^2). Also, sorting effectively normalizes the content, and
it's desirable to have one true representation of what we write.
For devices that have no real MAC address (virtual devices) it makes no
sense to store the MAC address to "no-auto-default.state" file. Also,
because we later would not match the MAC address during
nm_match_spec_device().
Instead, extend the format and add a "interface-name:=$IFACE" match-spec.
Maybe we generally should prefer the interface-name over the MAC
address. Anyway, for now, just extend the previously non-working case.
Currently "no-auto-default.state" contains only MAC addresses in ASCII
representation.
Next, we will also want to write there interface names. Interface names
in Linux don't enforce any encoding, so they can contain almost all
characters (except NUL and '/'). In particular '\n', which
we use as line separator.
If we want to store there interface names, we need to properly escape
and unescape them. Use our nm_utils_str_utf8safe_*() API for that.
For one, nm_config_get_no_auto_default_for_device() uses
nm_device_spec_match_list(). This ignores fake MAC addresses.
Maybe it should not do that, but it's also not clear what it
would mean for the function to consider them.
As such, it makes not sense trying to persist such MAC addresses
to "/var/lib/NetworkManager/no-auto-default.state". For the moment,
just do nothing.
This still leaves the problem how we prevent the device from generating
a auto-default connection. But this patch is no change in behavior, because
it didn't work anyway.
nm_config_set_no_auto_default_for_device() is called by NMSettings,
so it makes sense that also NMSettings checks whether the device is
blocked.
Of course, there is little difference in practice.
The only downside is that most device types don't implement
new_default_connection(). So the previous form performed the
cheaper check first. On the other hand, we do expect to have
profiles for the devices anyway.
With plain "interface-name:$IFNAME" globbing is enabled. So this behaves
wrong if there are special characters like '*' or '?'.
Also, it behaves wrong if the first character of the interface name happens
to be '='.
Make an explicit match.
The define is better, because then we can grep for all the occurances
where they are used. The plain text like "mac:" is not at all unique in
our source-tree.
Rejecting %NULL for a "is-a" function can be annoying. Of course, %NULL is
not a valid name. But it's sufficient that the function just returns
%FALSE in that case, and not assert against the input not being %NULL.
Asserting might be useful to catch bugs, but rejecting %NULL as input
is more cumbersome to the caller than helping with catching bugs.
Something similar was also recently done for nm_utils_is_uuid().
In particular when calling nm_utils_strv_sort() with a positive length
argument, then this is not a %NULL terminated strv arrary. That may mean
that it makes sense for the input array to contain %NULL strings.
Use a strcmp() function that accepts %NULL too.
While this is not used at the moment, I think nm_utils_strv_sort()
should accept %NULL strings beause otherwise it's a possibly unexpected
restriction of its API. The function should handle sensible input gracefully.
It is like strcmp(), but has a signature suitable for GCompareDataFunc.
This is necessary with nm_utils_ptrarray_find_binary_search()
to search a sorted strv array.
The fault is here really C, which doesn't allow inline static functions.
So, you need all kinds of slightly different flavors for the same
callbacks (with or without user-data).
Note that glib2 internally just casts strcmp() to GCompareDataFunc ([1]),
relying on the fact how arguments are passed to the function and
ignoring the additional user-data argument. But I find that really
ugly and probably not permissible in general C. Dunno whether POSIX
would guarantee for this to work. I'd rather not do such function
pointer casts.
[1] 0c0cf59858/glib/garray.c (L1792)
