During shutdown, we will need to still iterate the main loop
to do a coordinated shutdown. Currently we do not, and we just
exit, leaving a lot of objects hanging.
If we are going to fix that, we need during shutdown tell
NMDBusManager to reject all future operations.
Note that property getters and "GetManagerObjects" call is not
blocked. It continues to work.
Certainly for some operations, we want to allow them to be called even
during shutdown. However, these have to opt-in.
This also fixes an uglyness, where nm_dbus_manager_start() would
get the set-property-handler and the @manager as user-data. However,
NMDBusManager will always outlife NMManager, hence, after NMManager
is destroyed, the user-data would be a dangling pointer. Currently
that is not an issue, because
- we always leak NMManager
- we don't run the mainloop during shutdown
We currently start the bus manager only after the creation of a
NMManager because the NMManager is needed to handle set-property bus
calls. However, objects created by NMManager
(e.g. NMDnsSystemdResolved) need a bus connection and so their
initialization currently fail.
To fix this, split nm_dbus_manager_start() in two parts: first only
create the connection and acquire the bus. After this step the
NMManager can be set up. In the second step, set NMManager as the
set-property handler and start exporting objects on the bus.
Fixes: 297d4985ab
GVariant is immutable and can nicely be shared and cached.
Cache the property variants. This makes getting the properties
faster, at the expense of using some extra memory.
Tested with https://tratt.net/laurie/src/multitime/
$ multitime -n 200 -s 0 bash -c 'echo -n .; exec busctl call org.freedesktop.NetworkManager /org/freedesktop org.freedesktop.DBus.ObjectManager GetManagedObjects &>/dev/null'
# Mean Std.Dev. Min Median Max
# real(before) 0.013+/-0.0000 0.001 0.012 0.013 0.019
# real(after) 0.013+/-0.0000 0.002 0.011 0.012 0.034
$ multitime -n 100 -s 0 bash -c 'for i in {1..5}; do busctl call org.freedesktop.NetworkManager /org/freedesktop org.freedesktop.DBus.ObjectManager GetManagedObjects &>/dev/null & done; wait; echo -n .'
# Mean Std.Dev. Min Median Max
# real(before) 0.040+/-0.0000 0.002 0.037 0.040 0.049
# real(after) 0.037+/-0.0000 0.002 0.034 0.036 0.045
$ multitime -n 30 -s 0 bash -c 'for i in {1..100}; do busctl call org.freedesktop.NetworkManager /org/freedesktop org.freedesktop.DBus.ObjectManager GetManagedObjects &>/dev/null & done; wait; echo -n .'
# Mean Std.Dev. Min Median Max
# real(before) 0.704+/-0.0000 0.016 0.687 0.701 0.766
# real(after) 0.639+/-0.0000 0.013 0.622 0.637 0.687
$ multitime -n 200 -s 0 bash -c 'echo -n .; exec nmcli &>/dev/null'
# Mean Std.Dev. Min Median Max
# real(before) 0.092+/-0.0000 0.005 0.081 0.092 0.119
# real(after) 0.092+/-0.0000 0.005 0.080 0.091 0.123
$ multitime -n 100 -s 0 bash -c 'for i in {1..5}; do nmcli &>/dev/null & done; wait; echo -n .'
# Mean Std.Dev. Min Median Max
# real(before) 0.436+/-0.0000 0.043 0.375 0.424 0.600
# real(after) 0.413+/-0.0000 0.022 0.380 0.410 0.558
$ multitime -n 20 -s 0 bash -c 'for i in {1..100}; do nmcli &>/dev/null & done; wait; echo -n .'
# Mean Std.Dev. Min Median Max
# real(before) 8.796+/-0.1070 0.291 8.073 8.818 9.247
# real(after) 8.736+/-0.0893 0.243 8.017 8.780 9.101
The time savings are small, but that is because caching mostly speeds up
the GetManagedObjects calls, and that is only a small part of the entire
nmcli call from client side.
We use a private D-Bus socket for example for DHCP clients to report back
at unix:path=/var/run/NetworkManager/private-dhcp.
By default, gdbus will enable the authentication mechanisms EXTERNAL
and DBUS_COOKIE_SHA1. However, DBUS_COOKIE_SHA1 requires a /root/.dbus-keyrings
directory, which is not available to NetworkManager as it is started with
ProtectHome=read-only. And writing to /root would be a bad idea anyway.
This leads to a warning
NetworkManager[10962]: Error adding entry to keyring: Error creating directory “/root/.dbus-keyrings”: Read-only file system
Disable all but the EXTERNAL mechanism.
See-also: https://dbus.freedesktop.org/doc/dbus-specification.html#auth-mechanismshttps://bugzilla.gnome.org/show_bug.cgi?id=793116https://github.com/NetworkManager/NetworkManager/pull/79
Previously, we used the generated GDBusInterfaceSkeleton types and glued
them via the NMExportedObject base class to our NM types. We also used
GDBusObjectManagerServer.
Don't do that anymore. The resulting code was more complicated despite (or
because?) using generated classes. It was hard to understand, complex, had
ordering-issues, and had a runtime and memory overhead.
This patch refactors this entirely and uses the lower layer API GDBusConnection
directly. It replaces the generated code, GDBusInterfaceSkeleton, and
GDBusObjectManagerServer. All this is now done by NMDbusObject and NMDBusManager
and static descriptor instances of type GDBusInterfaceInfo.
This adds a net plus of more then 1300 lines of hand written code. I claim
that this implementation is easier to understand. Note that previously we
also required extensive and complex glue code to bind our objects to the
generated skeleton objects. Instead, now glue our objects directly to
GDBusConnection. The result is more immediate and gets rid of layers of
code in between.
Now that the D-Bus glue us more under our control, we can address issus and
bottlenecks better, instead of adding code to bend the generated skeletons
to our needs.
Note that the current implementation now only supports one D-Bus connection.
That was effectively the case already, although there were places (and still are)
where the code pretends it could also support connections from a private socket.
We dropped private socket support mainly because it was unused, untested and
buggy, but also because GDBusObjectManagerServer could not export the same
objects on multiple connections. Now, it would be rather straight forward to
fix that and re-introduce ObjectManager on each private connection. But this
commit doesn't do that yet, and the new code intentionally supports only one
D-Bus connection.
Also, the D-Bus startup was simplified. There is no retry, either nm_dbus_manager_start()
succeeds, or it detects the initrd case. In the initrd case, bus manager never tries to
connect to D-Bus. Since the initrd scenario is not yet used/tested, this is good enough
for the moment. It could be easily extended later, for example with polling whether the
system bus appears (like was done previously). Also, restart of D-Bus daemon isn't
supported either -- just like before.
Note how NMDBusManager now implements the ObjectManager D-Bus interface
directly.
Also, this fixes race issues in the server, by no longer delaying
PropertiesChanged signals. NMExportedObject would collect changed
properties and send the signal out in idle_emit_properties_changed()
on idle. This messes up the ordering of change events w.r.t. other
signals and events on the bus. Note that not only NMExportedObject
messed up the ordering. Also the generated code would hook into
notify() and process change events in and idle handle, exhibiting the
same ordering issue too.
No longer do that. PropertiesChanged signals will be sent right away
by hooking into dispatch_properties_changed(). This means, changing
a property in quick succession will no longer be combined and is
guaranteed to emit signals for each individual state. Quite possibly
we emit now more PropertiesChanged signals then before.
However, we are now able to group a set of changes by using standard
g_object_freeze_notify()/g_object_thaw_notify(). We probably should
make more use of that.
Also, now that our signals are all handled in the right order, we
might find places where we still emit them in the wrong order. But that
is then due to the order in which our GObjects emit signals, not due
to an ill behavior of the D-Bus glue. Possibly we need to identify
such ordering issues and fix them.
Numbers (for contrib/rpm --without debug on x86_64):
- the patch changes the code size of NetworkManager by
- 2809360 bytes
+ 2537528 bytes (-9.7%)
- Runtime measurements are harder because there is a large variance
during testing. In other words, the numbers are not reproducible.
Currently, the implementation performs no caching of GVariants at all,
but it would be rather simple to add it, if that turns out to be
useful.
Anyway, without strong claim, it seems that the new form tends to
perform slightly better. That would be no surprise.
$ time (for i in {1..1000}; do nmcli >/dev/null || break; echo -n .; done)
- real 1m39.355s
+ real 1m37.432s
$ time (for i in {1..2000}; do busctl call org.freedesktop.NetworkManager /org/freedesktop org.freedesktop.DBus.ObjectManager GetManagedObjects > /dev/null || break; echo -n .; done)
- real 0m26.843s
+ real 0m25.281s
- Regarding RSS size, just looking at the processes in similar
conditions, doesn't give a large difference. On my system they
consume about 19MB RSS. It seems that the new version has a
slightly smaller RSS size.
- 19356 RSS
+ 18660 RSS
The next commit will completely rework NMBusManager and replace
NMExportedObject by a new type NMDBusObject.
Originally, NMDBusObject was added along NMExportedObject to ease
the rework and have compilable, intermediate stages of refactoring. Now,
I think the new name is better, because NMDBusObject is very strongly related
to the bus manager and the old name NMExportedObject didn't make that
clear.
I also slighly prefer the name NMDBusObject over NMBusObject, hence
for consistancy, also rename NMBusManager to NMDBusManager.
This commit only renames the file for a nicer diff in the next commit.
It does not actually update the type name in sources. That will be done
later.
Our gdbus generated types use the same names as their corresponding
"real" types, but with "NM" changed to "NMDBus".
Unfortunately, that means that introspection/nmdbus-manager.c (the
generated type for src/nm-manager.c) uses the same type name as the
entirely unrelated src/nm-dbus-manager.c.
Fix this by removing the "d" from src/nm-dbus-manager.c. (We could
rename the generated type instead, but then it becomes inconsistent
with all the other generated types, and we're already using it as
"NMDBusManager" in libnm/nm-manager.c.)
Add NMExportedObject, make it the base class of all D-Bus-exported
types, and move the nm-properties-changed-signal logic into it. (Also,
make NMSettings use the same properties-changed code as everything
else, which it was not previously doing, presumably for historical
reasons).
(This is mostly just shuffling code around at this point, but
NMExportedObject will be more important in the gdbus port, since
gdbus-codegen doesn't do a very good job of supporting objects that
export multiple interfaces [as each NMDevice subclass does, for
example], so we will need more glue/helper code in NMExportedObject
then.)
Rather than randomly including one or more of <glib.h>,
<glib-object.h>, and <gio/gio.h> everywhere (and forgetting to include
"nm-glib-compat.h" most of the time), rename nm-glib-compat.h to
nm-glib.h, include <gio/gio.h> from there, and then change all .c
files in NM to include "nm-glib.h" rather than including the glib
headers directly.
(Public headers files still have to include the real glib headers,
since nm-glib.h isn't installed...)
Also, remove glib includes from header files that are already
including a base object header file (which must itself already include
the glib headers).
Add a file containing the defines like DBUS_INTERFACE_DBUS from
dbus-shared.h, and use it from the gdbus-using files.
Also, convert a bunch of other places that were previously hardcoding
the string values to use the defines instead, and fix the ifcfg-rh
plugin to properly namespace its own D-Bus-related defines.
Saw some g_warning() about g_object_weak_unref() trying to unref a non
registed reference. While this does not fix it, let's assert that
situation a step earlier to ease debugging.
Also, move g_object_weak_ref() closer to adding the object into the
@exported hash.
[thaller@redhat.com: rewrote commit message, change to register_object()]
The class itself is not thread-safe, so no need for guarding
the creation with g_once_init_*().
Also, assert against multiple creation and log a line when
creating the singleton. The getter is now more similar to what
is created by NM_DEFINE_SINGLETON_GETTER().
Don't have the singleton instance of NMDBusManager owned by
the main function. Instead use NM_DEFINE_SINGLETON_DESTRUCTOR()
which also logs what's happening.
This is really a no-op because the directory will be created by
nm_main_utils_check_pidfile() anyway, but make sure the permissions
are synchronized. Now that resolv.conf is symlink into NMRUNDIR,
if normal users don't have permissions to read NMRUNDIR then
DNS won't work.
config.h should be included from every .c file, and it should be
included before any other include. Fix that.
(As a side effect of how I did this, this also changes us to
consistently use "config.h" rather than <config.h>. To the extent that
it matters [which is not much], quotes are more correct anyway, since
we're talking about a file in our own build tree, not a system
include.)
The messages logged by nm-dbus-manager.c are not very useful, but amount to
a significant part of DEBUG logging. Log those messages with the lower TRACE priority.
Signed-off-by: Thomas Haller <thaller@redhat.com>
Since we already intenalize the @tag to a GQuark, just use
the constant string, instead of duplicating the string.
Signed-off-by: Thomas Haller <thaller@redhat.com>
"NetworkManager.h"'s name (and non-standard capitalization) suggest
that it's some sort of high-level super-important header, but it's
really just low-level D-Bus stuff. Rename it to "nm-dbus-interface.h"
and likewise "NetworkManagerVPN.h" to "nm-vpn-dbus-interface.h"
When running test programs, don't try to create a private bus, since
it will fail if the user isn't root or if NetworkManager is currently
running, and it isn't what we want anyway.
These are (most likely) only warnings and not severe bugs.
Some of these changes are mostly made to get a clean run of
Coverity without any warnings.
Error found by running Coverity scan
https://bugzilla.redhat.com/show_bug.cgi?id=1025894
Co-Authored-By: Jiří Klimeš <jklimes@redhat.com>
Signed-off-by: Thomas Haller <thaller@redhat.com>
When freeing one of the collections such as GArray, GPtrArray, GSList,
etc. it is common that the items inside the connections must be
freed/unrefed too.
The previous code often iterated over the collection first with
e.g. g_ptr_array_foreach and passing e.g. g_free as GFunc argument.
For one, this has the problem, that g_free has a different signature
GDestroyNotify then the expected GFunc. Moreover, this can be
simplified either by setting a clear function
(g_ptr_array_set_clear_func) or by passing the destroy function to the
free function (g_slist_free_full).
Signed-off-by: Thomas Haller <thaller@redhat.com>
Change the way that nm-properties-changed-signal works, and parse the
dbus-binding-tool-generated info to get the exact list of properties
that it's expected to export.
This makes NM_PROPERTY_PARAM_NO_EXPORT unnecessary, and also fixes the
problem of properties like NMDevice:hw-address being exported on
classes where it shouldn't be.
Since dbus senders are faked for private connections, we can't just
call dbus_bus_get_unix_user() on fake senders. They need to be
checked against the NMDBusManager's list of private connections
first.
For cases where NM may run without a bus daemon in root-only
environments, like an initramfs. For disconnection, since private
connection just get a disconnect message instead of NameOwnerChanged
signals broadcast by a bus daemon, just synthesize the NameOwnerChanged
signals using our fake owner name. It's just easier to do this rather
than modify any code that cares about disconnects.
Note that the new private socket is only enabled if built with
dbus-glib >= 0.100 as there are bugs in previous versions in the
implementation of dbus_g_proxy_new_for_peer() which clients must
use to talk to the private socket.
When providing a service on the bus daemon and a private connection,
we'll need to track objects so we can register them with the
private connection too. Thus all registration/unregistration
calls have to go through the NMDBusManager, not straight to
dbus-glib.
Remove the org.freedesktop.NetworkManagerSystemSettings bus name and
have everybody talk to org.freedesktop.NetworkManager. Now that we have
a single settings service that's embedded in the main daemon, we don't
need separate names anymore.
