Find a file
Thomas Haller 9eddf9fb09 settings: track profiles on disk that are shadowed by in-memory connections
Via Update2() D-Bus API there are three ways how a profile can be stored
(or migrated) to in-memory:

  - NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY
  - NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED
  - NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_ONLY

With the recent rework of settings I dropped NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY
and it had the same meaning as NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED.

However, the way NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED was implemented is
problematic. The problem is that it leaves the profile on disk but creates an
in-memory representation which shadows the persistent storage. Later,
when storing the profile to disk again, a new filename is chosen.
This allows via D-Bus API to toggle between NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED
and NM_SETTINGS_UPDATE2_FLAG_TO_DISK, and thereby pilling up profiles on disk.
Also, there is no D-Bus API to do anything sensible with these leaked, shadowed
profiles on disk.

Note that if we have a read-only profile in /usr/lib or in ifupdown
plugin, then the problem is not made any worse. That is, because via D-Bus
API such profiles can be made in-memory, and afterwards stored to /etc.
Thereby too the profile gets duplicate on disk, but this game only
works once. Afterwards, you cannot repeat it to create additional
profiles on disk. It means, you can only leak profiles once, and only
if they already exist in read-only storage to begin with.

This problem with NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED already existed
before the settings-delegate-storage rework, and is unrelated to whether in-memory
profiles now happen to be persisted to /run.

Note that NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_ONLY is simple and does not suffer
from this problem. When you move a profile to in-memory-only, it gets deleted from
persistent storage and no duplication happens.

The problem is that NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED used to
forget about the profile that it shadows, and that is wrong.

So, first re-add proper support for NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY. This
works by remembering the "shadowed-storage" path for in-memory profiles.
When later saving such a profile to disk again, the shadowed-storage
will be re-used. Likewise, when deleting such a profile, the shadowed
storage will be deleted.

Note that we keep NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED and it
also remembers the shadowed storage (but without "owning" it). That means,
when such a profile gets saved to disk again, the orginal storage is
reused too. As such, during future updates it behaves just like
NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY. The difference is when deleting
such a profile. In this case, the profile is left on storage and a
tombstone gets written. So, how is this better than before and why even
keep this complicated flag?
First, we keep this flag because we really want the ansible role to be
able to do in-memory changes only. That implies being able to delete a
profile from NetworkManager's view, but not from persistent storage. Without
this flag there is no way to do that. You can only modify an on-disk profile
by shadowing it, but you could not delete it form NetworkManager's view
while keeping it on disk.

The new form of NM_SETTINGS_UPDATE2_FLAG_IN_MEMORY_DETACHED is safe and avoids
the duplication problem because also for tombstones it remembers the original
"shadowed-storage". That is, when the profile gets recreated later via
D-Bus API AddConnection, then the re-created profile will still reference
and reuse the shadowed storage that it had before deletion.
2019-07-25 23:27:49 +02:00
clients cli: use nm_client_add_connection2() API from nmcli 2019-07-25 15:26:49 +02:00
contrib all: codespell fixes 2019-07-24 11:30:19 +02:00
data settings: drop ibft settings plugin 2019-06-20 16:06:44 +02:00
dispatcher dispatcher: tolerate lack of connection path 2019-07-24 13:30:41 +02:00
docs libnm-core: add ovs-dpdk setting 2019-06-14 12:10:20 +02:00
examples examples: add examples/python/gi/nm-update2.py example script 2019-07-25 22:02:00 +02:00
introspection core,libnm: add AddConnection2() D-Bus API to block autoconnect from the start 2019-07-25 15:26:49 +02:00
libnm core,libnm: add AddConnection2() D-Bus API to block autoconnect from the start 2019-07-25 15:26:49 +02:00
libnm-core settings: track profiles on disk that are shadowed by in-memory connections 2019-07-25 23:27:49 +02:00
m4 all: drop emacs file variables from source files 2019-06-11 10:04:00 +02:00
man man: update nm-openswitch example 2019-07-09 12:05:32 +02:00
po po: update Brazilian Portuguese (pt_BR) translation 2019-07-15 09:55:09 +02:00
shared shared: add nm_g_slice_free() helper 2019-07-25 15:26:49 +02:00
src settings: track profiles on disk that are shadowed by in-memory connections 2019-07-25 23:27:49 +02:00
tools settings: rework tracking settings connections and settings plugins 2019-07-16 19:09:08 +02:00
vapi all: goodbye libnm-glib 2019-04-16 15:52:27 +02:00
.dir-locals.el misc: add toplevel .dir-locals file that tells Emacs to show trailing whitespace 2013-03-08 15:15:28 +01:00
.gitignore settings: drop ibft settings plugin 2019-06-20 16:06:44 +02:00
.gitlab-ci.yml gitlab-ci: workaround build failure in Debian:sid due to iproute2 issue 2019-07-12 10:45:36 +02:00
.mailmap mailmap: update user 2018-10-01 12:02:55 +02:00
.travis.yml all: goodbye libnm-glib 2019-04-16 15:52:27 +02:00
AUTHORS misc: update maintainers and authors 2016-04-21 13:39:03 -05:00
autogen.sh all: goodbye libnm-glib 2019-04-16 15:52:27 +02:00
ChangeLog all: point git references to the GitLab instance 2018-08-27 11:36:56 +02:00
config-extra.h.meson build: remove duplicate and unused RUNDIR define 2019-05-17 21:24:18 +02:00
config-extra.h.mk build/autotools: generate "config-extra.h" via makefile "config-extra.h.mk" 2019-06-17 17:42:09 +02:00
config.h.meson systemd: merge branch systemd into master 2019-07-05 09:13:53 +02:00
configure.ac systemd: merge branch systemd into master 2019-07-05 09:13:53 +02:00
CONTRIBUTING CONTRIBUTING: update section about assertions in NetworkManager 2019-06-26 09:53:54 +02:00
COPYING docs: create new master NM documentation module 2011-02-16 16:24:16 -06:00
linker-script-binary.ver iface-helper/build: add linker version script 2016-10-13 21:33:33 +02:00
linker-script-devices.ver devices/build: use one linker-script-devices.ver for all device plugins 2016-10-13 21:36:06 +02:00
linker-script-settings.ver settings/build: add linker version script for settings plugins 2016-10-13 21:33:33 +02:00
MAINTAINERS misc: update maintainers and authors 2016-04-21 13:39:03 -05:00
Makefile.am settings: rework tracking settings connections and settings plugins 2019-07-16 19:09:08 +02:00
Makefile.examples examples: add examples/python/gi/nm-update2.py example script 2019-07-25 22:02:00 +02:00
Makefile.glib all: drop emacs file variables from source files 2019-06-11 10:04:00 +02:00
Makefile.vapigen build: fix make always re-making vapigen target 2016-10-21 18:46:03 +02:00
meson.build systemd: merge branch systemd into master 2019-07-05 09:13:53 +02:00
meson_options.txt dhcp: add nettools dhcp4 client 2019-07-05 11:04:32 +02:00
NetworkManager.pc.in build: update NetworkManager.pc 2013-01-29 16:17:30 -05:00
NEWS settings: drop ibft settings plugin 2019-06-20 16:06:44 +02:00
README all: drop empty first line from sources 2019-06-11 10:15:06 +02:00
TODO all: say Wi-Fi instead of "wifi" or "WiFi" 2018-11-29 17:53:35 +01:00
valgrind.suppressions all: goodbye libnm-glib 2019-04-16 15:52:27 +02:00
zanata.xml po: add Zanata configuration 2016-04-05 14:35:53 +02:00

******************
NetworkManager core daemon has moved to gitlab.freedesktop.org!

git clone https://gitlab.freedesktop.org/NetworkManager/NetworkManager.git
******************


Networking that Just Works
--------------------------

NetworkManager attempts to keep an active network connection available at all
times.  The point of NetworkManager is to make networking configuration and
setup as painless and automatic as possible.  NetworkManager is intended to
replace default route, replace other routes, set IP addresses, and in general
configure networking as NM sees fit (with the possibility of manual override as
necessary).  In effect, the goal of NetworkManager is to make networking Just
Work with a minimum of user hassle, but still allow customization and a high
level of manual network control.  If you have special needs, we'd like to hear
about them, but understand that NetworkManager is not intended for every
use-case.

NetworkManager will attempt to keep every network device in the system up and
active, as long as the device is available for use (has a cable plugged in,
the killswitch isn't turned on, etc).  Network connections can be set to
'autoconnect', meaning that NetworkManager will make that connection active
whenever it and the hardware is available.

"Settings services" store lists of user- or administrator-defined "connections",
which contain all the settings and parameters required to connect to a specific
network.  NetworkManager will _never_ activate a connection that is not in this
list, or that the user has not directed NetworkManager to connect to.


How it works:

The NetworkManager daemon runs as a privileged service (since it must access
and control hardware), but provides a D-Bus interface on the system bus to
allow for fine-grained control of networking.  NetworkManager does not store
connections or settings, it is only the mechanism by which those connections
are selected and activated.

To store pre-defined network connections, two separate services, the "system
settings service" and the "user settings service" store connection information
and provide these to NetworkManager, also via D-Bus.  Each settings service
can determine how and where it persistently stores the connection information;
for example, the GNOME applet stores its configuration in GConf, and the system
settings service stores its config in distro-specific formats, or in a distro-
agnostic format, depending on user/administrator preference.

A variety of other system services are used by NetworkManager to provide
network functionality: wpa_supplicant for wireless connections and 802.1x
wired connections, pppd for PPP and mobile broadband connections, DHCP clients
for dynamic IP addressing, dnsmasq for proxy nameserver and DHCP server
functionality for internet connection sharing, and avahi-autoipd for IPv4
link-local addresses.  Most communication with these daemons occurs, again,
via D-Bus.


Why doesn't my network Just Work?

Driver problems are the #1 cause of why NetworkManager sometimes fails to
connect to wireless networks.  Often, the driver simply doesn't behave in a
consistent manner, or is just plain buggy.  NetworkManager supports _only_
those drivers that are shipped with the upstream Linux kernel, because only
those drivers can be easily fixed and debugged.  ndiswrapper, vendor binary
drivers, or other out-of-tree drivers may or may not work well with
NetworkManager, precisely because they have not been vetted and improved by the
open-source community, and because problems in these drivers usually cannot
be fixed.

Sometimes, command-line tools like 'iwconfig' will work, but NetworkManager will
fail.  This is again often due to buggy drivers, because these drivers simply
aren't expecting the dynamic requests that NetworkManager and wpa_supplicant
make.  Driver bugs should be filed in the bug tracker of the distribution being
run, since often distributions customize their kernel and drivers.

Sometimes, it really is NetworkManager's fault.  If you think that's
the case, please file a bug at:

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/issues

Attaching NetworkManager debug logs from the journal (or wherever your
distribution directs syslog's 'daemon' facility output, as
/var/log/messages or /var/log/daemon.log) is often very helpful, and
(if you can get) a working wpa_supplicant config file helps
enormously.  See the logging section of file
contrib/fedora/rpm/NetworkManager.conf for how to enable debug logging
in NetworkManager.