This helper code is already used by several of our unit tests.
Compile it as a separate library.
Previously, the source code lingered unmotivated under "shared/",
which is confusing.
We want that there is a clear hierarchy and dependency between
modules/directories.
But tests sometimes break that.
For example, "shared/nm-glib-aux/tests" link against libnm-glib-aux
and libnm-log-null. But "shared/nm-log-null" depends on "shared/nm-glib-aux".
Hence, for the test there is a circular dependency.
Break that, by including all tests at the end.
The alternative would be ugly in different ways.
"libnm-core/" is rather complicated. It provides a static library that
is linked into libnm.so and NetworkManager. It also contains public
headers (like "nm-setting.h") which are part of public libnm API.
Then we have helper libraries ("libnm-core/nm-libnm-core-*/") which
only rely on public API of libnm-core, but are themself static
libraries that can be used by anybody who uses libnm-core. And
"libnm-core/nm-libnm-core-intern" is used by libnm-core itself.
Move "libnm-core/" to "src/". But also split it in different
directories so that they have a clearer purpose.
The goal is to have a flat directory hierarchy. The "src/libnm-core*/"
directories correspond to the different modules (static libraries and set
of headers that we have). We have different kinds of such modules because
of how we combine various code together. The directory layout now reflects
this.
Currently "src/" mostly contains the source code of the daemon.
I say mostly, because that is not true, there are also the device,
settings, wwan, ppp plugins, the initrd generator, the pppd and dhcp
helper, and probably more.
Also we have source code under libnm-core/, libnm/, clients/, and
shared/ directories. That is all confusing.
We should have one "src" directory, that contains subdirectories. Those
subdirectories should contain individual parts (libraries or
applications), that possibly have dependencies on other subdirectories.
There should be a flat hierarchy of directories under src/, which
contains individual modules.
As the name "src/" is already taken, that prevents any sensible
restructuring of the code.
As a first step, move "src/" to "src/core/". This gives space to
reorganize the code better by moving individual components into "src/".
For inspiration, look at systemd's "src/" directory.
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/743
We have many static helper libraries, like libnm-glib-aux or libnm-core.
These can be statically linked in any end-binary as internal API. However, they
must only be linked once.
Also, we have various plugins (device, settings, ppp, wwan) which are
dlopened by NetworkManager. They should use the symbols from
NetworkManager core. It is important that they do not link with the
static libraries already, because also NetworkManager core links with
it, so these symbols will be duplicate.
As the symbols are internal, you might think that it is not a real
problem to duplicate them. However, there are also global variables,
like the hash tables for NMRefStr or the seed for NMHash. These global
variables must be only be used once, and hence also these symbols must
no be duplicated.
Fix that by adding a new dependency that is for the core plugins. This
dependency only has "include_directories" but not "link_with".
Naming for us is hard, because everything is an "nm". However, let's
standardize on the term "core" for the daemon, and not "daemon".
Eventually I would like to move the daemon from "src/" to "src/core/",
rename the dep in anticipation of that.
We want to move platform code to "shared/nm-platform". However, platform
code uses the logging infrastructure from the daemon, there is thus
an odd circular dependency.
Solve that by moving the "src/nm-logging.[hc]" to a new helper library
in "shared/nm-log-core".
NetworkManager core is huge. We should try to split out
parts that are independent.
Platform code is already mostly independent. But due to having it
under "src/", there is no strict separation/layering which determines
the parts that can work independently. So, while the code is mostly
independent (in practice), that is not obvious from looking at the
source tree. It thus still contributes to cognitive load.
Add a shared library "shared/nm-platform", which should have no
dependencies on libnm-core or NetworkManager core.
In a first step, move the netlink code there. More should follow.
NML3Cfg already handles IPv4 ACD. IPv4LL is just a small additional
layer on top of that, so it makes sense that it also is handled by
NML3Cfg.
Also, the overall goal is that multiple NMDevice and NMVpnConnection
instances can cooperate independently. So if multiple "users" enable
IPv4LL on an interface, then we should only run it once. This is
achieved by NML3IPv4LL's API where users register what they want,
and NML3IPv4LL figures out what that means as a whole.
Also, we thus will no longer need to use sd_ipv4ll/n-ipv4ll, because
we implement it ourself.
Currently, NMDevice does ACD. It intercepts certain NMIP4Config
instances, and tries to perform ACD on the addresses. I think this
functionality should be handled by NML3Cfg instead.
For one, NML3Cfg sees all configurations, and can perform ACD for all
(relevant) addresses. Also, it moves logic away from NMDevice and makes
the functionality available without an NMDevice. As such, it also will
allow that independent "controllers" contribute NML3ConfigData instances
and ACD will performed for all of them (as requested).
This will be our implementation for IPv4 ACD (https://tools.ietf.org/html/rfc5227)
based on nettools' n-acd library.
The code is not actually tested yes, because NMDevice did not yet switch
over to use NML3Cfg. Once that happens, surely issues with this patch
will be found that will need fixing.
We use a linker version script "NetworkManager.ver", to hide
symbols from NetworkManager that are not used. That is important
due to our habit of using internal helper libraries that we link
statically everywhere, without handpicking the symbols we actually
need. We want the tooling to get rid of unnecessary symbols.
However, NetworkManager loads shared libraries for settings and device
plugins. These libraries require symbols from the NetworkManager binary,
but which one depends on build options. Hence, we also generate
"NetworkManager.ver" by the "tools/create-exports-NetworkManager.sh"
script.
For that the script uses "nm" to find symbols that are undefined in the
plugin libraries but defined in NetworkManager. With autotools the
script looked at "./src/.libs/libNetworkManager.a" to find the present
symbols. Note that for meson that already didn't work, and we build
instead an intermediate NetworkManager binary first (with all symbols
exposed). With LTO, "nm" doesn't find all symbols in
"./src/.libs/libNetworkManager.a", and consequently they are not
exported and dropped/hidden.
This also causes unit tests to fail with LTO, because our test script
"tools/check-exports.sh" catches such bugs.
Fix that by also with autotools generate a complete "NetworkManager-all-sym"
binary that is used to generate "NetworkManager.ver", before rebuilding
"NetworkManager" again.
NMIP[46]Config will become much simpler than it is today.
It's sole responsibility will be to expose current settings
on D-Bus, in it's function as a NMDBusObject subtype.
However, it still make sense to let them share a common base class.
Add it.
Currently NMIP4Config and NMIP6Config both track the data to be
configured, they expose properties on D-Bus, and they have logic for
capturing and applying settings to platform.
We will split that.
- NMIP4Config and NMIP6Config will expose data on D-Bus.
- NML3Cfg will have the logic for handling IP configuration.
- NML3ConfigData will track data to be configured.
NML3ConfigData mirrors NMIP4Config/NMIP6Config in many aspects. For now,
this duplicates a lot of code. More will be done later. Eventually,
NMIP4Config/NMIP6Config will drop the duplicated functionality.
The advantage is that the API is now the same for IPv4 and IPv6: it's
all nm_dhcp_config_*() and we can (easier) treat the address family
generically.
We still need two distinct GObject types, mainly because of the
glue code for exposing the object on D-Bus as NMDBusObject. Of course,
that could be solved differently, but as it is, it's quite nice.
Add VRF support to the daemon. When the device we are activating is a
VRF or a VRF's slave, put routes in the table specified by the VRF
connection.
Also, introduce a VRF device type in libnm.
Keyfile support was initially added under GPL-2.0+ license as part of
core. It was moved to "libnm-core" in commit 59eb5312a5 ('keyfile: merge
branch 'th/libnm-keyfile-bgo744699'').
"libnm-core" is statically linked with by core and "libnm". In
the former case under terms of GPL-2.0+ (good) and in the latter case
under terms of LGPL-2.1+ (bad).
In fact, to this day, "libnm" doesn't actually use the code. The linker
will probably remove all the GPL-2.0+ symbols when compiled with
gc-sections or LTO. Still, linking them together in the first place
makes "libnm" only available under GPL code (despite the code
not actually being used).
Instead, move the GPL code to a separate static library
"shared/nm-keyfile/libnm-keyfile.la" and only link it to the part
that actually uses the code (and which is GPL licensed too).
This fixes the license violation.
Eventually, it would be very useful to be able to expose keyfile
handling via "libnm". However that is not straight forward due to the
licensing conflict.
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/381
Move it to shared as it's useful for clients as well.
Move and rename nm_dbus_manager_new_auth_subject_from_context() and
nm_dbus_manager_new_auth_subject_from_message() in nm-dbus-manager.c
as they're needed there.
The targets that involve the use of the `NetworkManager` library,
built in the `src` build file have been improved by applying a set
of changes:
- Indentation has been fixed.
- Set of objects used in targets have been grouped together.
- Aritificial dependencies used to group dependencies and custom
compiler flags have been removed and their use replaced with
proper dependencies and compiler flags to avoid any confussion.
The `libnm-core` build file has been improved by applying a set of
changes:
- Indentation has been fixed to be consistent.
- Library variable names have been changed to `lib{name}` pattern
following their filename pattern.
- `shared` prefix has been removed from all variables using it.
- Dependencies have been reviewed to store the necessary data.
- The use of the libraries and dependencies created in this file
has been reviewed through the entire source code. This has
required the addition or the removal of different libraries and
dependencies in different targets.
- Some files used directly with the `files` function have been moved
to their nearest path build file because meson stores their full
path seamessly and they can be used anywhere later.
The `nm-default.h` header is used widely in the code by many
targets. This header includes different headers and needs different
libraries depending the compilation flags.
A new set of `*nm_default_dep` dependencies have been created to
ease the inclusion of different directorires and libraries.
This allows cleaner build files and avoiding linking unnecessary
libraries so this has been applied allowing the removal of some
dependencies involving the linking of unnecessary libraries.
The `shared` build file has been improved by applying a set of
changes:
- Indentation has been fixed to be consistent.
- Unused libraries and dependencies have been removed.
- Dependencies have been reviewed to store the necessary data.
- Set of objects used in targets have been grouped together.
- Header files have been removed from sources lists as it's
unnecessary.
- Library variable names have been changed to `lib{name}` pattern
following their filename pattern.
- `shared` prefix has been removed from all variables using it.
- `version_header` its related configuration `version_conf`
variables have been renamed to `nm_version_macro*` following
its input and final file names.
Completely rework how settings plugin handle connections and how
NMSettings tracks the list of connections.
Previously, settings plugins would return objects of (a subtype of) type
NMSettingsConnection. The NMSettingsConnection was tightly coupled with
the settings plugin. That has a lot of downsides.
Change that. When changing this basic relation how settings connections
are tracked, everything falls appart. That's why this is a huge change.
Also, since I have to largely rewrite the settings plugins, I also
added support for multiple keyfile directories, handle in-memory
connections only by keyfile plugin and (partly) use copy-on-write NMConnection
instances. I don't want to spend effort rewriting large parts while
preserving the old way, that anyway should change. E.g. while rewriting ifcfg-rh,
I don't want to let it handle in-memory connections because that's not right
long-term.
--
If the settings plugins themself create subtypes of NMSettingsConnection
instances, then a lot of knowledge about tracking connections moves
to the plugins.
Just try to follow the code what happend during nm_settings_add_connection().
Note how the logic is spread out:
- nm_settings_add_connection() calls plugin's add_connection()
- add_connection() creates a NMSettingsConnection subtype
- the plugin has to know that it's called during add-connection and
not emit NM_SETTINGS_PLUGIN_CONNECTION_ADDED signal
- NMSettings calls claim_connection() which hocks up the new
NMSettingsConnection instance and configures the instance
(like calling nm_settings_connection_added()).
This summary does not sound like a lot, but try to follow that code. The logic
is all over the place.
Instead, settings plugins should have a very simple API for adding, modifying,
deleting, loading and reloading connections. All the plugin does is to return a
NMSettingsStorage handle. The storage instance is a handle to identify a profile
in storage (e.g. a particular file). The settings plugin is free to subtype
NMSettingsStorage, but it's not necessary.
There are no more events raised, and the settings plugin implements the small
API in a straightforward manner.
NMSettings now drives all of this. Even NMSettingsConnection has now
very little concern about how it's tracked and delegates only to NMSettings.
This should make settings plugins simpler. Currently settings plugins
are so cumbersome to implement, that we avoid having them. It should not be
like that and it should be easy, beneficial and lightweight to create a new
settings plugin.
Note also how the settings plugins no longer care about duplicate UUIDs.
Duplicated UUIDs are a fact of life and NMSettings must handle them. No
need to overly concern settings plugins with that.
--
NMSettingsConnection is exposed directly on D-Bus (being a subtype of
NMDBusObject) but it was also a GObject type provided by the settings
plugin. Hence, it was not possible to migrate a profile from one plugin to
another.
However that would be useful when one profile does not support a
connection type (like ifcfg-rh not supporting VPN). Currently such
migration is not implemented except for migrating them to/from keyfile's
run directory. The problem is that migrating profiles in general is
complicated but in some cases it is important to do.
For example checkpoint rollback should recreate the profile in the right
settings plugin, not just add it to persistent storage. This is not yet
properly implemented.
--
Previously, both keyfile and ifcfg-rh plugin implemented in-memory (unsaved)
profiles, while ifupdown plugin cannot handle them. That meant duplication of code
and a ifupdown profile could not be modified or made unsaved.
This is now unified and only keyfile plugin handles in-memory profiles (bgo #744711).
Also, NMSettings is aware of such profiles and treats them specially.
In particular, NMSettings drives the migration between persistent and non-persistent
storage.
Note that a settings plugins may create truly generated, in-memory profiles.
The settings plugin is free to generate and persist the profiles in any way it
wishes. But the concept of "unsaved" profiles is now something explicitly handled
by keyfile plugin. Also, these "unsaved" keyfile profiles are persisted to file system
too, to the /run directory. This is great for two reasons: first of all, all
profiles from keyfile storage in fact have a backing file -- even the
unsaved ones. It also means you can create "unsaved" profiles in /run
and load them with `nmcli connection load`, meaning there is a file
based API for creating unsaved profiles.
The other advantage is that these profiles now survive restarting
NetworkManager. It's paramount that restarting the daemon is as
non-disruptive as possible. Persisting unsaved files to /run improves
here significantly.
--
In the past, NMSettingsConnection also implemented NMConnection interface.
That was already changed a while ago and instead users call now
nm_settings_connection_get_connection() to delegate to a
NMSimpleConnection. What however still happened was that the NMConnection
instance gets never swapped but instead the instance was modified with
nm_connection_replace_settings_from_connection(), clear-secrets, etc.
Change that and treat the NMConnection instance immutable. Instead of modifying
it, reference/clone a new instance. This changes that previously when somebody
wanted to keep a reference to an NMConnection, then the profile would be cloned.
Now, it is supposed to be safe to reference the instance directly and everybody
must ensure not to modify the instance. nmtst_connection_assert_unchanging()
should help with that.
The point is that the settings plugins may keep references to the
NMConnection instance, and so does the NMSettingsConnection. We want
to avoid cloning the instances as long as they are the same.
Likewise, the device's applied connection can now also be referenced
instead of cloning it. This is not yet done, and possibly there are
further improvements possible.
--
Also implement multiple keyfile directores /usr/lib, /etc, /run (rh #1674545,
bgo #772414).
It was always the case that multiple files could provide the same UUID
(both in case of keyfile and ifcfg-rh). For keyfile plugin, if a profile in
read-only storage in /usr/lib gets modified, then it gets actually stored in
/etc (or /run, if the profile is unsaved).
--
While at it, make /etc/network/interfaces profiles for ifupdown plugin reloadable.
--
https://bugzilla.gnome.org/show_bug.cgi?id=772414https://bugzilla.gnome.org/show_bug.cgi?id=744711https://bugzilla.redhat.com/show_bug.cgi?id=1674545
This is inspired by the existing systemd integration, with a few differences:
* This parses the WPAD option, which systemd requested, but did not use.
* We hook into the DAD handling, only making use of the configured address
once DAD has completed successfully, and declining the lease if it fails.
There are still many areas of possible improvement. In particular, we need
to ensure the parsing of all options are compliant, as n-dhcp4 treats all
options as opaque, unlike sd-dhcp4. We probably also need to look at how
to handle failures and retries (in particular if we decline a lease).
We need to query the current MTU at client startu, as well as the hardware
broadcast address. Both these are provided by the kernel over netlink, so
it should simply be a matter of hooking that up with NM's netlink layer.
Contribution under LGPL2.0+, in addition to stated licenses.
The library is called "libnm_core". So the dependency should be called
"libnm_core_dep", like in all other cases.
(cherry picked from commit c27ad37c27)
Routing rules are unlike addresses or routes not tied to an interface.
NetworkManager thinks in terms of connection profiles. That works well
for addresses and routes, as one profile configures addresses and routes
for one device. For example, when activating a profile on a device, the
configuration does not interfere with the addresses/routes of other
devices. That is not the case for routing rules, which are global, netns-wide
entities.
When one connection profile specifies rules, then this per-device configuration
must be merged with the global configuration. And when a device disconnects later,
the rules must be removed.
Add a new NMPRulesManager API to track/untrack routing rules. Devices can
register/add there the routing rules they require. And the sync method will
apply the configuration. This is be implemented on top of NMPlatform's
caching API.
Like also done for autotools, create and use intermediate libraries
from "shared/nm-utils/".
Also, replace "shared_dep" by "shared_nm_utils_base_dep". We don't
need super fine-grained selection of what we link. We can always
link in "shared/libnm-utils-base.a", and let the linker throw away
unsed parts.
For better or worse, we already pull in large parts of systemd sources.
I need a base64 decode implementation (because glib's g_base64_decode()
cannot reject invalid encodings). Instead of coming up with my own or
copy-paste if from somewhere, reuse systemd's unbase64mem().
But for that, make systemd's basic bits an independent static library
first because I will need it in libnm-core.
This doesn't really change anything except making "libnm-systemd-core.la"
an indpendent static library that could be used from "libnm-core". We
shall still be mindful about which internal code of systemd we use, and only
access functionality that is exposed via "systemd/nm-sd-utils-shared.h".
For P2P connections it makes sense to bind the connection to the status
of the operation that is being done. One example is that a wifi display
(miracast) P2P connection should be shut down when streaming fails for
some reason.
This new helper class allows binding a connection to the presence of a
DBus path meaning that it will be torn down if the process disappears.
This is independent functionality that only depends on linux API
and glib.
Note how "nm-logging" uses this for getting the timestamps. This
makes "nm-logging.c" itself dependen on "src/nm-core-utils.c",
for little reason.
