fdo-mirrors/NetworkManager
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/NetworkManager/NetworkManager.git synced 2025-12-21 23:50:08 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
NetworkManager/libnm/nm-device.c

3007 lines
92 KiB
C
Raw Normal View History

all: SPDX header conversion $ find * -type f |xargs perl contrib/scripts/spdx.pl $ git rm contrib/scripts/spdx.pl
2019-09-10 11:19:01 +02:00
// SPDX-License-Identifier: LGPL-2.1+
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/*
all: unify format of our Copyright source code comments ```bash readarray -d '' FILES < <( git ls-files -z \ ':(exclude)po' \ ':(exclude)shared/c-rbtree' \ ':(exclude)shared/c-list' \ ':(exclude)shared/c-siphash' \ ':(exclude)shared/c-stdaux' \ ':(exclude)shared/n-acd' \ ':(exclude)shared/n-dhcp4' \ ':(exclude)src/systemd/src' \ ':(exclude)shared/systemd/src' \ ':(exclude)m4' \ ':(exclude)COPYING*' ) sed \ -e 's/^\(--\|#\| \*\) *\(([cC]) *\)\?Copyright \+\(\(([cC])\) \+\)\?\(\(20\|19\)[0-9][0-9]\) *[-–] *\(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/\1 C1pyright#\5 - \7#\9/' \ -e 's/^\(--\|#\| \*\) *\(([cC]) *\)\?Copyright \+\(\(([cC])\) \+\)\?\(\(20\|19\)[0-9][0-9]\) *[,] *\(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/\1 C2pyright#\5, \7#\9/' \ -e 's/^\(--\|#\| \*\) *\(([cC]) *\)\?Copyright \+\(\(([cC])\) \+\)\?\(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/\1 C3pyright#\5#\7/' \ -e 's/^Copyright \(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/C4pyright#\1#\3/' \ -i \ "${FILES[@]}" echo ">>> untouched Copyright lines" git grep Copyright "${FILES[@]}" echo ">>> Copyright lines with unusual extra" git grep '\<C[0-9]pyright#' "${FILES[@]}" | grep -i reserved sed \ -e 's/\<C[0-9]pyright#\([^#]*\)#\(.*\)$/Copyright (C) \1 \2/' \ -i \ "${FILES[@]}" ``` https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/298
2019-10-01 09:20:35 +02:00
* Copyright (C) 2007 - 2008 Novell, Inc.
* Copyright (C) 2007 - 2018 Red Hat, Inc.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*/
build: cleanup default includes - "gsystem-local-alloc.h" and <gio/gio.h> are already included via "nm-default.h". No need to include them separately. - include "nm-macros-internal.h" via "nm-default.h" and drop all explict includes. - in the modified files, ensure that we always include "config.h" and "nm-default.h" first. As second, include the header file for the current source file (if applicable). Then follow external includes and finally internal nm includes. - include nm headers inside source code files with quotes - internal header files don't need to include default headers. They can savely assume that "nm-default.h" is already included and with it glib, nm-glib.h, nm-macros-internal.h, etc.
2016-02-12 14:44:52 +01:00
#include "nm-default.h"
all: cleanup includes and let "nm-default.h" include "config.h" - All internal source files (except "examples", which are not internal) should include "config.h" first. As also all internal source files should include "nm-default.h", let "config.h" be included by "nm-default.h" and include "nm-default.h" as first in every source file. We already wanted to include "nm-default.h" before other headers because it might contains some fixes (like "nm-glib.h" compatibility) that is required first. - After including "nm-default.h", we optinally allow for including the corresponding header file for the source file at hand. The idea is to ensure that each header file is self contained. - Don't include "config.h" or "nm-default.h" in any header file (except "nm-sd-adapt.h"). Public headers anyway must not include these headers, and internal headers are never included after "nm-default.h", as of the first previous point. - Include all internal headers with quotes instead of angle brackets. In practice it doesn't matter, because in our public headers we must include other headers with angle brackets. As we use our public headers also to compile our interal source files, effectively the result must be the same. Still do it for consistency. - Except for <config.h> itself. Include it with angle brackets as suggested by https://www.gnu.org/software/autoconf/manual/autoconf.html#Configuration-Headers
2016-02-19 14:57:48 +01:00
build: cleanup default includes - "gsystem-local-alloc.h" and <gio/gio.h> are already included via "nm-default.h". No need to include them separately. - include "nm-macros-internal.h" via "nm-default.h" and drop all explict includes. - in the modified files, ensure that we always include "config.h" and "nm-default.h" first. As second, include the header file for the current source file (if applicable). Then follow external includes and finally internal nm includes. - include nm headers inside source code files with quotes - internal header files don't need to include default headers. They can savely assume that "nm-default.h" is already included and with it glib, nm-glib.h, nm-macros-internal.h, etc.
2016-02-12 14:44:52 +01:00
#include "nm-device.h"
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
udev: drop libgudev in favor of libudev libgudev is just a wrapper around libudev. We can use libudev directly and drop the dependency for libgudev.
2017-03-12 15:54:02 +01:00
#include <libudev.h>
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
#include "nm-glib-aux/nm-dbus-aux.h"
libnm: rename NetworkManager.h and NetworkManagerVPN.h "NetworkManager.h"'s name (and non-standard capitalization) suggest that it's some sort of high-level super-important header, but it's really just low-level D-Bus stuff. Rename it to "nm-dbus-interface.h" and likewise "NetworkManagerVPN.h" to "nm-vpn-dbus-interface.h"
2014-07-05 16:23:30 -04:00
#include "nm-dbus-interface.h"
libnm: add nm-types.h, to avoid include loops Add nm-types.h defining all the type structs, to avoid future include loops. Clean up the includes in all of the installed headers.
2014-10-09 09:12:42 -04:00
#include "nm-active-connection.h"
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
#include "nm-device-bt.h"
libnm: add nm-types.h, to avoid include loops Add nm-types.h defining all the type structs, to avoid future include loops. Clean up the includes in all of the installed headers.
2014-10-09 09:12:42 -04:00
#include "nm-dhcp4-config.h"
#include "nm-dhcp6-config.h"
#include "nm-ip4-config.h"
#include "nm-ip6-config.h"
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
#include "nm-object-private.h"
#include "nm-remote-connection.h"
libnm: drop NM_TYPE_OBJECT_ARRAY, use G_TYPE_PTR_ARRAY Use G_TYPE_PTR_ARRAY for GPtrArray-of-NMObject-valued properties, because it has better introspection/bindings support. As with the strdict change in libnm-core, we need to manually copy the array in get_property() implementations, to preserve the standard semantics that get_property() returns a copy, not the internal array. (This patch also changes those properties so that they are always non-NULL until dispose(); previously some of them could be either NULL or 0-length at different times.)
2014-08-26 08:31:04 -04:00
#include "nm-core-internal.h"
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
#include "nm-utils.h"
libnm: rename nm-dbus-helpers-private.h to nm-dbus-helpers.h The .h file should have the same name as the .c file.
2014-08-19 09:52:41 -04:00
#include "nm-dbus-helpers.h"
libnm: add NMDeviceTun
2015-09-04 18:11:15 +02:00
#include "nm-device-tun.h"
libnm-core: add nm-core-types.h, remove cross-includes Add nm-core-types.h, typedefing all of the GObject types in libnm-core; this is needed so that nm-setting.h can reference NMConnection in addition to nm-connection.h referencing NMSetting. Removing the cross-includes from the various headers causes lots of fallout elsewhere. (In particular, nm-utils.h used to include nm-connection.h, which included every setting header, so any file that included nm-utils.h automatically got most of the rest of libnm-core without needing to pay attention to specifics.) Fix this up by including nm-core-internal.h from those files that are now missing includes.
2014-10-21 22:09:52 -04:00
#include "nm-setting-connection.h"
shared: move udev helper to separate directory "shared/nm-udev-aux" We built (among others) two libraries from the sources in "shared/nm-utils": "libnm-utils-base.la" and "libnm-utils-udev.la". It's confusing. Instead use directories so there is a direct correspondence between these internal libraries and the source files. (cherry picked from commit 2973d682533a83c4d0738e2baef6991f8f0cba87)
2019-04-15 07:39:31 +02:00
#include "nm-udev-aux/nm-udev-utils.h"
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
/*****************************************************************************/
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
NM_GOBJECT_PROPERTIES_DEFINE (NMDevice,
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
PROP_INTERFACE,
PROP_UDI,
all: add device.path property Add a device property to expose its path as reported in the ID_PATH udev property.
2020-06-10 15:17:39 +02:00
PROP_PATH,
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
PROP_DRIVER,
PROP_DRIVER_VERSION,
PROP_FIRMWARE_VERSION,
PROP_CAPABILITIES,
PROP_REAL,
PROP_MANAGED,
PROP_AUTOCONNECT,
PROP_FIRMWARE_MISSING,
PROP_NM_PLUGIN_MISSING,
PROP_IP4_CONFIG,
PROP_DHCP4_CONFIG,
PROP_IP6_CONFIG,
PROP_STATE,
PROP_STATE_REASON,
PROP_PRODUCT,
PROP_VENDOR,
PROP_DHCP6_CONFIG,
PROP_IP_INTERFACE,
PROP_DEVICE_TYPE,
PROP_ACTIVE_CONNECTION,
PROP_AVAILABLE_CONNECTIONS,
PROP_PHYSICAL_PORT_ID,
PROP_MTU,
PROP_METERED,
PROP_LLDP_NEIGHBORS,
PROP_IP4_CONNECTIVITY,
PROP_IP6_CONNECTIVITY,
libnm: export interface flags Add libnm support for the new InterfaceFlags property of NMDevice.
2019-10-10 10:04:06 +02:00
PROP_INTERFACE_FLAGS,
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
PROP_HW_ADDRESS,
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
);
enum {
STATE_CHANGED,
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
LAST_SIGNAL
};
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
static guint signals[LAST_SIGNAL] = { 0 };
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
enum {
PROPERTY_O_IDX_ACTIVE_CONNECTION,
PROPERTY_O_IDX_IP4_CONFIG,
PROPERTY_O_IDX_IP6_CONFIG,
PROPERTY_O_IDX_DHCP4_CONFIG,
PROPERTY_O_IDX_DHCP6_CONFIG,
_PROPERTY_O_IDX_NUM,
};
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
typedef struct _NMDevicePrivate {
NMLDBusPropertyO property_o[_PROPERTY_O_IDX_NUM];
NMLDBusPropertyAO available_connections;
GPtrArray *lldp_neighbors;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
char *driver;
char *driver_version;
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
char *hw_address;
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
char *interface;
char *ip_interface;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
char *firmware_version;
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
char *physical_port_id;
char *udi;
all: add device.path property Add a device property to expose its path as reported in the ID_PATH udev property.
2020-06-10 15:17:39 +02:00
char *path;
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
guint32 capabilities;
guint32 device_type;
guint32 ip4_connectivity;
guint32 ip6_connectivity;
guint32 metered;
guint32 mtu;
guint32 state;
guint32 state_reason;
guint32 interface_flags;
bool firmware_missing;
bool nm_plugin_missing;
bool autoconnect;
bool managed;
bool real;
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
bool hw_address_is_new:1;
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
guint32 old_state;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm/udev: cache and reuse udev instance for NMDevice No need to create a separate NMUdevClient instance for all devices. Instead, have one "struct udev" instance in NMClient and pass it down during object construction.
2017-03-20 12:27:17 +01:00
struct udev *udev;
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
char *type_description;
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
char *product;
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
char *vendor;
char *short_vendor;
char *description;
char *bus_name;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
} NMDevicePrivate;
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
G_DEFINE_ABSTRACT_TYPE (NMDevice, nm_device, NM_TYPE_OBJECT);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
#define NM_DEVICE_GET_PRIVATE(self) _NM_GET_PRIVATE_PTR(self, NMDevice, NM_IS_DEVICE, NMObject)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
/*****************************************************************************/
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
static gboolean connection_compatible (NMDevice *device, NMConnection *connection, GError **error);
static NMLldpNeighbor *nm_lldp_neighbor_dup (NMLldpNeighbor *neighbor);
/*****************************************************************************/
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
struct _NMLldpNeighbor {
guint refcount;
GHashTable *attrs;
};
libnm: define NMLldpNeighbor as boxed type GLib introspection requires all types returned by public functions to be GObjects, basic types or boxed types in order to correctly manage resources. NMLldpNeighbor was a plain struct and GI complained with: Warning: NM: nm_lldp_neighbor_new: return value: Invalid non-constant return of bare structure or union; register as boxed type or (skip) To fix this define NMLldpNeighbor as a boxed type. Fixes: d3d2b494008322a4f1452cf873393ca85166022b
2015-10-16 16:11:42 +02:00
G_DEFINE_BOXED_TYPE (NMLldpNeighbor, nm_lldp_neighbor, nm_lldp_neighbor_dup, nm_lldp_neighbor_unref)
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
/*****************************************************************************/
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
static void
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
nm_device_init (NMDevice *self)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
{
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
NMDevicePrivate *priv;
priv = G_TYPE_INSTANCE_GET_PRIVATE (self, NM_TYPE_DEVICE, NMDevicePrivate);
self->_priv = priv;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
priv->old_state = NM_DEVICE_STATE_UNKNOWN;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
/*****************************************************************************/
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
static void
_notify_event_state_changed (NMClient *client,
NMClientNotifyEventWithPtr *notify_event)
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
{
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
gs_unref_object NMDevice *self = notify_event->user_data;
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (self);
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
libnm: fix logging message about device's state change signal The device instance might already be removed from the cache. At that point, _nm_object_get_client(self) returns %NULL. Use the correct NMClient instance.
2020-01-03 13:49:31 +01:00
NML_NMCLIENT_LOG_T (client,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
"[%s] emit Device's StateChanged signal %u -> %u, reason: %u",
_nm_object_get_path (self),
(guint) priv->old_state,
(guint) priv->state,
(guint) priv->state_reason);
g_signal_emit (self,
signals[STATE_CHANGED],
0,
(guint) priv->state,
(guint) priv->old_state,
(guint) priv->state_reason);
}
static NMLDBusNotifyUpdatePropFlags
_notify_update_prop_state_reason (NMClient *client,
NMLDBusObject *dbobj,
const NMLDBusMetaIface *meta_iface,
guint dbus_property_idx,
GVariant *value)
{
NMDevice *self = NM_DEVICE (dbobj->nmobj);
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (self);
guint32 new_state = NM_DEVICE_STATE_UNKNOWN;
guint32 reason = NM_DEVICE_STATE_REASON_NONE;
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
/* We ignore the "State" property and the "StateChanged" signal of the device.
* This information is redundant to the "StateReason" property, and we rely
* on that one alone. In the best case, the information is identical. If it
* would not be, then we stick to the information from "StateReason" property. */
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
if (value)
g_variant_get (value, "(uu)", &new_state, &reason);
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
if ( priv->state == new_state
&& priv->state_reason == reason) {
/* no changes. */
return NML_DBUS_NOTIFY_UPDATE_PROP_FLAGS_NONE;
}
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
if (priv->state != new_state) {
priv->old_state = priv->state;
priv->state = new_state;
_nm_client_queue_notify_object (client,
self,
obj_properties[PROP_STATE]);
}
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
if (priv->state_reason != reason) {
priv->state_reason = reason;
_nm_client_queue_notify_object (client,
self,
obj_properties[PROP_STATE_REASON]);
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
}
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_client_notify_event_queue_with_ptr (client,
NM_CLIENT_NOTIFY_EVENT_PRIO_GPROP + 1,
_notify_event_state_changed,
g_object_ref (self));
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return NML_DBUS_NOTIFY_UPDATE_PROP_FLAGS_NONE;
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
}
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
static NMLDBusNotifyUpdatePropFlags
_notify_update_prop_lldp_neighbors (NMClient *client,
NMLDBusObject *dbobj,
const NMLDBusMetaIface *meta_iface,
guint dbus_property_idx,
GVariant *value)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
{
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
NMDevice *self = NM_DEVICE (dbobj->nmobj);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (self);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
gs_unref_ptrarray GPtrArray *old = NULL;
gs_unref_ptrarray GPtrArray *new = NULL;
GVariantIter *attrs_iter;
GVariantIter iter;
new = g_ptr_array_new_with_free_func ((GDestroyNotify) nm_lldp_neighbor_unref);
if (value) {
g_variant_iter_init (&iter, value);
while (g_variant_iter_next (&iter, "a{sv}", &attrs_iter)) {
GVariant *attr_variant;
const char *attr_name;
NMLldpNeighbor *neigh;
neigh = nm_lldp_neighbor_new ();
while (g_variant_iter_next (attrs_iter, "{&sv}", &attr_name, &attr_variant))
g_hash_table_insert (neigh->attrs, g_strdup (attr_name), attr_variant);
g_ptr_array_add (new, neigh);
g_variant_iter_free (attrs_iter);
}
}
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
old = g_steal_pointer (&priv->lldp_neighbors);
priv->lldp_neighbors = g_steal_pointer (&new);
return NML_DBUS_NOTIFY_UPDATE_PROP_FLAGS_NOTIFY;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
/*****************************************************************************/
libnm: initialize @device_type of device objects Before switching to the ObjectManager, the D-Bus property was used to decide the actual type of the device and the property set manually by each subclass in its _init() function. Now we determine objects type based on their D-Bus interface and therefore we can handle the property like all others, ensuring that we return a known value in get_property() to avoid warnings in GLib. This fixes the missing initialization of the property which causes regressions on clients as nm-applet. Fixes: 1f5b48a59eb46c40cb10bf4381b2b21a19a9f471
2016-11-12 14:09:08 +01:00
static NMDeviceType
coerce_type (NMDeviceType type)
{
switch (type) {
case NM_DEVICE_TYPE_ETHERNET:
case NM_DEVICE_TYPE_WIFI:
case NM_DEVICE_TYPE_BT:
case NM_DEVICE_TYPE_OLPC_MESH:
introspection: add o.fd.NM.Device.OvsInterface interface
2017-10-02 08:39:09 +02:00
case NM_DEVICE_TYPE_OVS_INTERFACE:
introspection: add o.fd.NM.Device.OvsPort interface
2017-10-02 08:39:09 +02:00
case NM_DEVICE_TYPE_OVS_PORT:
introspection: add o.fd.NM.Device.OvsBridge interface
2017-10-02 08:39:09 +02:00
case NM_DEVICE_TYPE_OVS_BRIDGE:
libnm: initialize @device_type of device objects Before switching to the ObjectManager, the D-Bus property was used to decide the actual type of the device and the property set manually by each subclass in its _init() function. Now we determine objects type based on their D-Bus interface and therefore we can handle the property like all others, ensuring that we return a known value in get_property() to avoid warnings in GLib. This fixes the missing initialization of the property which causes regressions on clients as nm-applet. Fixes: 1f5b48a59eb46c40cb10bf4381b2b21a19a9f471
2016-11-12 14:09:08 +01:00
case NM_DEVICE_TYPE_WIMAX:
case NM_DEVICE_TYPE_MODEM:
case NM_DEVICE_TYPE_INFINIBAND:
case NM_DEVICE_TYPE_BOND:
case NM_DEVICE_TYPE_TEAM:
case NM_DEVICE_TYPE_BRIDGE:
case NM_DEVICE_TYPE_VLAN:
case NM_DEVICE_TYPE_ADSL:
core,libnm: introduce NMDeviceMacsec At the moment the device only exposes the current link status, but cannot create new links.
2016-06-30 18:20:22 +02:00
case NM_DEVICE_TYPE_MACSEC:
libnm: initialize @device_type of device objects Before switching to the ObjectManager, the D-Bus property was used to decide the actual type of the device and the property set manually by each subclass in its _init() function. Now we determine objects type based on their D-Bus interface and therefore we can handle the property like all others, ensuring that we return a known value in get_property() to avoid warnings in GLib. This fixes the missing initialization of the property which causes regressions on clients as nm-applet. Fixes: 1f5b48a59eb46c40cb10bf4381b2b21a19a9f471
2016-11-12 14:09:08 +01:00
case NM_DEVICE_TYPE_MACVLAN:
case NM_DEVICE_TYPE_VXLAN:
case NM_DEVICE_TYPE_IP_TUNNEL:
case NM_DEVICE_TYPE_TUN:
case NM_DEVICE_TYPE_VETH:
case NM_DEVICE_TYPE_GENERIC:
case NM_DEVICE_TYPE_UNUSED1:
case NM_DEVICE_TYPE_UNUSED2:
case NM_DEVICE_TYPE_UNKNOWN:
core: add support for dummy devices Add support for creating dummy devices. This commit adds a D-Bus interface 'org.freedesktop.NetworkManager.Device.Dummy' which is used primarily for determining the device type but does not carry any properties.
2017-01-31 14:14:33 +01:00
case NM_DEVICE_TYPE_DUMMY:
device: add NMDevicePPP The new device type represents a PPP interface, and will implement the activation of new-style PPPoE connections, i.e. the ones that don't claim the parent device.
2017-06-06 15:55:08 +02:00
case NM_DEVICE_TYPE_PPP:
devices: add NMDeviceWpan
2018-03-09 16:26:25 +01:00
case NM_DEVICE_TYPE_WPAN:
devices: add NMDevice6Lowpan
2018-05-22 16:25:54 +02:00
case NM_DEVICE_TYPE_6LOWPAN:
core: introduce NMDeviceWireGuard For now, the device only exposes partial link status (not including peers). It cannot create new links.
2018-03-13 13:42:38 +00:00
case NM_DEVICE_TYPE_WIREGUARD:
core,wifi-p2p: Fix Wi-Fi P2P device type The device type was set to the GType rather than a new value in the NMDeviceType enum. Add the corresponding enum entry, fix the device type and set the routing priority to the same value as generic devices. (cherry picked from commit 8d9365a973025976a15e5e7adb26a6f791957b7c)
2019-03-07 21:20:28 +01:00
case NM_DEVICE_TYPE_WIFI_P2P:
core,libnm: add VRF support Add VRF support to the daemon. When the device we are activating is a VRF or a VRF's slave, put routes in the table specified by the VRF connection. Also, introduce a VRF device type in libnm.
2019-12-05 10:36:54 +01:00
case NM_DEVICE_TYPE_VRF:
libnm: initialize @device_type of device objects Before switching to the ObjectManager, the D-Bus property was used to decide the actual type of the device and the property set manually by each subclass in its _init() function. Now we determine objects type based on their D-Bus interface and therefore we can handle the property like all others, ensuring that we return a known value in get_property() to avoid warnings in GLib. This fixes the missing initialization of the property which causes regressions on clients as nm-applet. Fixes: 1f5b48a59eb46c40cb10bf4381b2b21a19a9f471
2016-11-12 14:09:08 +01:00
return type;
}
return NM_DEVICE_TYPE_UNKNOWN;
}
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
/*****************************************************************************/
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
static void
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
register_client (NMObject *nmobj,
NMClient *client,
NMLDBusObject *dbobj)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
{
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (nmobj);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
priv->udev = _nm_client_get_udev (client);
if (priv->udev)
udev_ref (priv->udev);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
NM_OBJECT_CLASS (nm_device_parent_class)->register_client (nmobj, client, dbobj);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
/*****************************************************************************/
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
static void
finalize (GObject *object)
{
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (object);
all: use nm_clear_pointer() instead of g_clear_pointer() g_clear_pointer() would always cast the destroy notify function pointer to GDestroyNotify. That means, it lost some type safety, like GPtrArray *ptr_arr = ... g_clear_pointer (&ptr_arr, g_array_unref); Since glib 2.58 ([1]), g_clear_pointer() is also more type safe. But this is not used by NetworkManager, because we don't set GLIB_VERSION_MIN_REQUIRED to 2.58. [1] https://gitlab.gnome.org/GNOME/glib/-/commit/f9a9902aac826ab4aecc25f6eb533a418a4fa559 We have nm_clear_pointer() to avoid this issue for a long time (pre 1.12.0). Possibly we should redefine in our source tree g_clear_pointer() as nm_clear_pointer(). However, I don't like to patch glib functions with our own variant. Arguably, we do patch g_clear_error() in such a manner. But there the point is to make the function inlinable. Also, nm_clear_pointer() returns a boolean that indicates whether anything was cleared. That is sometimes useful. I think we should just consistently use nm_clear_pointer() instead, which does always the preferable thing. Replace: sed 's/\<g_clear_pointer *(\([^;]*\), *\([a-z_A-Z0-9]\+\) *)/nm_clear_pointer (\1, \2)/g' $(git grep -l g_clear_pointer) -i
2020-03-23 11:09:24 +01:00
nm_clear_pointer (&priv->lldp_neighbors, g_ptr_array_unref);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
g_free (priv->interface);
g_free (priv->ip_interface);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
g_free (priv->udi);
all: add device.path property Add a device property to expose its path as reported in the ID_PATH udev property.
2020-06-10 15:17:39 +02:00
g_free (priv->path);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
g_free (priv->driver);
g_free (priv->driver_version);
g_free (priv->firmware_version);
g_free (priv->product);
g_free (priv->vendor);
g_free (priv->short_vendor);
g_free (priv->description);
g_free (priv->bus_name);
g_free (priv->type_description);
g_free (priv->physical_port_id);
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
g_free (priv->hw_address);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
nm_clear_pointer (&priv->udev, udev_unref);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
G_OBJECT_CLASS (nm_device_parent_class)->finalize (object);
}
static void
get_property (GObject *object,
guint prop_id,
GValue *value,
GParamSpec *pspec)
{
NMDevice *device = NM_DEVICE (object);
switch (prop_id) {
case PROP_DEVICE_TYPE:
libnm: Fix reporting of unknown device types nm_device_get_device_type would report the device type as it was send on DBus, while fetching the property would mean that only a known device types is reported. Make both results consistent by coercing in nm_device_get_device_type rather than when setting the property. (cherry picked from commit a6a185ba00c6218d9b1ace6e3bd6b57347198246)
2019-03-07 21:31:47 +01:00
g_value_set_enum (value, nm_device_get_device_type (device));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
break;
case PROP_UDI:
g_value_set_string (value, nm_device_get_udi (device));
break;
all: add device.path property Add a device property to expose its path as reported in the ID_PATH udev property.
2020-06-10 15:17:39 +02:00
case PROP_PATH:
g_value_set_string (value, nm_device_get_path (device));
break;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
case PROP_INTERFACE:
g_value_set_string (value, nm_device_get_iface (device));
break;
case PROP_IP_INTERFACE:
g_value_set_string (value, nm_device_get_ip_iface (device));
break;
case PROP_DRIVER:
g_value_set_string (value, nm_device_get_driver (device));
break;
case PROP_DRIVER_VERSION:
g_value_set_string (value, nm_device_get_driver_version (device));
break;
case PROP_FIRMWARE_VERSION:
g_value_set_string (value, nm_device_get_firmware_version (device));
break;
case PROP_CAPABILITIES:
libnm: make use of GParamSpecFlags and GParamSpecEnum Make enum- and flags-valued properties use GParamSpecEnum and GParamSpecFlags, for better introspectability/bindability. This requires no changes outside libnm-core/libnm since the expected data size is still the same with g_object_get()/g_object_set(), and GLib will internally convert between int/uint and enum/flags GValues when using g_object_get_property()/g_object_set_property().
2014-06-26 16:47:46 -04:00
g_value_set_flags (value, nm_device_get_capabilities (device));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
break;
libnm-glib/libnm: add support for "real" NMDevice property
2014-10-06 11:34:02 -05:00
case PROP_REAL:
g_value_set_boolean (value, nm_device_is_real (device));
break;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
case PROP_MANAGED:
g_value_set_boolean (value, nm_device_get_managed (device));
break;
case PROP_AUTOCONNECT:
g_value_set_boolean (value, nm_device_get_autoconnect (device));
break;
case PROP_FIRMWARE_MISSING:
g_value_set_boolean (value, nm_device_get_firmware_missing (device));
break;
libnm: add nm-plugin-missing property indicating NM device plugin not available
2015-04-14 14:31:42 +02:00
case PROP_NM_PLUGIN_MISSING:
g_value_set_boolean (value, nm_device_get_nm_plugin_missing (device));
break;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
case PROP_IP4_CONFIG:
g_value_set_object (value, nm_device_get_ip4_config (device));
break;
case PROP_DHCP4_CONFIG:
g_value_set_object (value, nm_device_get_dhcp4_config (device));
break;
case PROP_IP6_CONFIG:
g_value_set_object (value, nm_device_get_ip6_config (device));
break;
case PROP_DHCP6_CONFIG:
g_value_set_object (value, nm_device_get_dhcp6_config (device));
break;
case PROP_STATE:
libnm: make use of GParamSpecFlags and GParamSpecEnum Make enum- and flags-valued properties use GParamSpecEnum and GParamSpecFlags, for better introspectability/bindability. This requires no changes outside libnm-core/libnm since the expected data size is still the same with g_object_get()/g_object_set(), and GLib will internally convert between int/uint and enum/flags GValues when using g_object_get_property()/g_object_set_property().
2014-06-26 16:47:46 -04:00
g_value_set_enum (value, nm_device_get_state (device));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
break;
case PROP_STATE_REASON:
libnm: simplify NMDevice:state-reason The NMDevice:state-reason property was awkward to work with since it was a tuple of two values (state and reason), and likewise, although we had nm_device_get_state() to return just the state, there was no corresponding function to get just the reason. Fix this; make the state-reason property contain just the NMDeviceStateReason, and make nm_device_get_state_reason() return just that.
2014-08-25 10:51:26 -04:00
g_value_set_uint (value, nm_device_get_state_reason (device));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
break;
case PROP_ACTIVE_CONNECTION:
g_value_set_object (value, nm_device_get_active_connection (device));
break;
case PROP_AVAILABLE_CONNECTIONS:
libnm: drop NM_TYPE_OBJECT_ARRAY, use G_TYPE_PTR_ARRAY Use G_TYPE_PTR_ARRAY for GPtrArray-of-NMObject-valued properties, because it has better introspection/bindings support. As with the strdict change in libnm-core, we need to manually copy the array in get_property() implementations, to preserve the standard semantics that get_property() returns a copy, not the internal array. (This patch also changes those properties so that they are always non-NULL until dispose(); previously some of them could be either NULL or 0-length at different times.)
2014-08-26 08:31:04 -04:00
g_value_take_boxed (value, _nm_utils_copy_object_array (nm_device_get_available_connections (device)));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
break;
case PROP_PRODUCT:
g_value_set_string (value, nm_device_get_product (device));
break;
case PROP_VENDOR:
g_value_set_string (value, nm_device_get_vendor (device));
break;
case PROP_PHYSICAL_PORT_ID:
g_value_set_string (value, nm_device_get_physical_port_id (device));
break;
case PROP_MTU:
g_value_set_uint (value, nm_device_get_mtu (device));
break;
core,libnm: add 'metered' property to NMDevice
2015-04-29 16:34:38 +02:00
case PROP_METERED:
g_value_set_uint (value, nm_device_get_metered (device));
break;
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
case PROP_LLDP_NEIGHBORS:
g_value_set_boxed (value, nm_device_get_lldp_neighbors (device));
break;
libnm: add support for per-device & per-AF connectivity status
2018-07-03 20:42:50 +02:00
case PROP_IP4_CONNECTIVITY:
g_value_set_enum (value, nm_device_get_connectivity (device, AF_INET));
break;
case PROP_IP6_CONNECTIVITY:
g_value_set_enum (value, nm_device_get_connectivity (device, AF_INET6));
break;
libnm: export interface flags Add libnm support for the new InterfaceFlags property of NMDevice.
2019-10-10 10:04:06 +02:00
case PROP_INTERFACE_FLAGS:
g_value_set_uint (value, nm_device_get_interface_flags (device));
break;
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
case PROP_HW_ADDRESS:
g_value_set_string (value, nm_device_get_hw_address (device));
break;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
static void
set_property (GObject *object,
guint prop_id,
const GValue *value,
GParamSpec *pspec)
{
NMDevice *self = NM_DEVICE (object);
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (self);
gboolean b;
switch (prop_id) {
case PROP_AUTOCONNECT:
b = g_value_get_boolean (value);
if (priv->autoconnect != b)
nm_device_set_autoconnect (NM_DEVICE (object), b);
break;
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
/* TODO: statistics interface not yet implemented. */
const NMLDBusMetaIface _nml_dbus_meta_iface_nm_device_statistics = NML_DBUS_META_IFACE_INIT (
NM_DBUS_INTERFACE_DEVICE_STATISTICS,
NULL,
NML_DBUS_META_INTERFACE_PRIO_NONE,
NML_DBUS_META_IFACE_DBUS_PROPERTIES (
NML_DBUS_META_PROPERTY_INIT_TODO ("RefreshRateMs", "u" ),
NML_DBUS_META_PROPERTY_INIT_TODO ("RxBytes", "t" ),
NML_DBUS_META_PROPERTY_INIT_TODO ("TxBytes", "t" ),
),
);
const NMLDBusMetaIface _nml_dbus_meta_iface_nm_device = NML_DBUS_META_IFACE_INIT_PROP (
NM_DBUS_INTERFACE_DEVICE,
nm_device_get_type,
NML_DBUS_META_INTERFACE_PRIO_PARENT_TYPE,
NML_DBUS_META_IFACE_DBUS_PROPERTIES (
NML_DBUS_META_PROPERTY_INIT_O_PROP ("ActiveConnection", PROP_ACTIVE_CONNECTION, NMDevicePrivate, property_o[PROPERTY_O_IDX_ACTIVE_CONNECTION], nm_active_connection_get_type, .is_always_ready = TRUE ),
NML_DBUS_META_PROPERTY_INIT_B ("Autoconnect", PROP_AUTOCONNECT, NMDevicePrivate, autoconnect ),
NML_DBUS_META_PROPERTY_INIT_AO_PROP ("AvailableConnections", PROP_AVAILABLE_CONNECTIONS, NMDevicePrivate, available_connections, nm_remote_connection_get_type, .is_always_ready = TRUE ),
NML_DBUS_META_PROPERTY_INIT_U ("Capabilities", PROP_CAPABILITIES, NMDevicePrivate, capabilities ),
NML_DBUS_META_PROPERTY_INIT_U ("DeviceType", PROP_DEVICE_TYPE, NMDevicePrivate, device_type ),
NML_DBUS_META_PROPERTY_INIT_O_PROP ("Dhcp4Config", PROP_DHCP4_CONFIG, NMDevicePrivate, property_o[PROPERTY_O_IDX_DHCP4_CONFIG], nm_dhcp4_config_get_type ),
NML_DBUS_META_PROPERTY_INIT_O_PROP ("Dhcp6Config", PROP_DHCP6_CONFIG, NMDevicePrivate, property_o[PROPERTY_O_IDX_DHCP6_CONFIG], nm_dhcp6_config_get_type ),
NML_DBUS_META_PROPERTY_INIT_S ("Driver", PROP_DRIVER, NMDevicePrivate, driver ),
NML_DBUS_META_PROPERTY_INIT_S ("DriverVersion", PROP_DRIVER_VERSION, NMDevicePrivate, driver_version ),
NML_DBUS_META_PROPERTY_INIT_B ("FirmwareMissing", PROP_FIRMWARE_MISSING, NMDevicePrivate, firmware_missing ),
NML_DBUS_META_PROPERTY_INIT_S ("FirmwareVersion", PROP_FIRMWARE_VERSION, NMDevicePrivate, firmware_version ),
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
NML_DBUS_META_PROPERTY_INIT_FCN ("HwAddress", 0, "s", _nm_device_notify_update_prop_hw_address ),
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
NML_DBUS_META_PROPERTY_INIT_S ("Interface", PROP_INTERFACE, NMDevicePrivate, interface ),
NML_DBUS_META_PROPERTY_INIT_U ("InterfaceFlags", PROP_INTERFACE_FLAGS, NMDevicePrivate, interface_flags ),
NML_DBUS_META_PROPERTY_INIT_IGNORE ("Ip4Address", "u" ),
NML_DBUS_META_PROPERTY_INIT_O_PROP ("Ip4Config", PROP_IP4_CONFIG, NMDevicePrivate, property_o[PROPERTY_O_IDX_IP4_CONFIG], nm_ip4_config_get_type ),
NML_DBUS_META_PROPERTY_INIT_U ("Ip4Connectivity", PROP_IP4_CONNECTIVITY, NMDevicePrivate, ip4_connectivity ),
NML_DBUS_META_PROPERTY_INIT_O_PROP ("Ip6Config", PROP_IP6_CONFIG, NMDevicePrivate, property_o[PROPERTY_O_IDX_IP6_CONFIG], nm_ip6_config_get_type ),
NML_DBUS_META_PROPERTY_INIT_U ("Ip6Connectivity", PROP_IP6_CONNECTIVITY, NMDevicePrivate, ip6_connectivity ),
NML_DBUS_META_PROPERTY_INIT_S ("IpInterface", PROP_IP_INTERFACE, NMDevicePrivate, ip_interface ),
NML_DBUS_META_PROPERTY_INIT_FCN ("LldpNeighbors", PROP_LLDP_NEIGHBORS, "aa{sv}", _notify_update_prop_lldp_neighbors ),
NML_DBUS_META_PROPERTY_INIT_B ("Managed", PROP_MANAGED, NMDevicePrivate, managed ),
NML_DBUS_META_PROPERTY_INIT_U ("Metered", PROP_METERED, NMDevicePrivate, metered ),
NML_DBUS_META_PROPERTY_INIT_U ("Mtu", PROP_MTU, NMDevicePrivate, mtu ),
NML_DBUS_META_PROPERTY_INIT_B ("NmPluginMissing", PROP_NM_PLUGIN_MISSING, NMDevicePrivate, nm_plugin_missing ),
all: add device.path property Add a device property to expose its path as reported in the ID_PATH udev property.
2020-06-10 15:17:39 +02:00
NML_DBUS_META_PROPERTY_INIT_S ("Path", PROP_PATH, NMDevicePrivate, path ),
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
NML_DBUS_META_PROPERTY_INIT_S ("PhysicalPortId", PROP_PHYSICAL_PORT_ID, NMDevicePrivate, physical_port_id ),
NML_DBUS_META_PROPERTY_INIT_B ("Real", PROP_REAL, NMDevicePrivate, real ),
NML_DBUS_META_PROPERTY_INIT_IGNORE ("State", "u" ),
NML_DBUS_META_PROPERTY_INIT_FCN ("StateReason", PROP_STATE_REASON, "(uu)", _notify_update_prop_state_reason ),
NML_DBUS_META_PROPERTY_INIT_S ("Udi", PROP_UDI, NMDevicePrivate, udi ),
),
.base_struct_offset = G_STRUCT_OFFSET (NMDevice, _priv),
);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
static void
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
nm_device_class_init (NMDeviceClass *klass)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
{
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
GObjectClass *object_class = G_OBJECT_CLASS (klass);
NMObjectClass *nm_object_class = NM_OBJECT_CLASS (klass);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
g_type_class_add_private (klass, sizeof (NMDevicePrivate));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
object_class->get_property = get_property;
object_class->set_property = set_property;
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
object_class->finalize = finalize;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
nm_object_class->register_client = register_client;
_NM_OBJECT_CLASS_INIT_PRIV_PTR_INDIRECT (nm_object_class, NMDevice);
libnm: add init_dbus() virtual method to NMObject Rather than having each object type override constructed() to call _nm_object_register_properties(), have NMObject call a virtual method on the subclass to ask it to register them. Move some code around in nm-client.c and nm-object.c so that all D-Bus-related initialization happens in init_dbus(), and non-D-Bus-related stuff stays in construct(). (This simplifies the next commit.)
2014-05-15 14:24:56 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_NM_OBJECT_CLASS_INIT_PROPERTY_O_FIELDS_N (nm_object_class, NMDevicePrivate, property_o);
_NM_OBJECT_CLASS_INIT_PROPERTY_AO_FIELDS_1 (nm_object_class, NMDevicePrivate, available_connections);
klass->connection_compatible = connection_compatible;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:interface:
*
* The interface of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_INTERFACE] =
g_param_spec_string (NM_DEVICE_INTERFACE, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:ip-interface:
*
* The IP interface of the device which should be used for all IP-related
* operations like addressing and routing.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_IP_INTERFACE] =
g_param_spec_string (NM_DEVICE_IP_INTERFACE, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:device-type:
*
* The numeric type of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_DEVICE_TYPE] =
g_param_spec_enum (NM_DEVICE_DEVICE_TYPE, "", "",
NM_TYPE_DEVICE_TYPE,
NM_DEVICE_TYPE_UNKNOWN,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:udi:
*
* An operating-system specific device hardware identifier; this is not
* unique to a specific hardware device across reboots or hotplugs. It
* is an opaque string which for some device types (Bluetooth, Modem)
* contains an identifier provided by the underlying hardware service daemon
* such as Bluez or ModemManager, and clients can use this property to
* request more information about the device from those services.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_UDI] =
g_param_spec_string (NM_DEVICE_UDI, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
all: add device.path property Add a device property to expose its path as reported in the ID_PATH udev property.
2020-06-10 15:17:39 +02:00
/**
* NMDevice:path:
*
* The device path as exposed by the udev property ID_PATH.
*
* The string is backslash escaped (C escaping) for invalid
* characters. The escaping can be reverted with g_strcompress(),
* however the result may not be valid UTF-8.
*
* Since: 1.26
**/
obj_properties[PROP_PATH] =
g_param_spec_string (NM_DEVICE_PATH, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:driver:
*
* The driver of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_DRIVER] =
g_param_spec_string (NM_DEVICE_DRIVER, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:driver-version:
*
* The version of the device driver.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_DRIVER_VERSION] =
g_param_spec_string (NM_DEVICE_DRIVER_VERSION, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:firmware-version:
*
* The firmware version of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_FIRMWARE_VERSION] =
g_param_spec_string (NM_DEVICE_FIRMWARE_VERSION, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:capabilities:
*
* The capabilities of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_CAPABILITIES] =
g_param_spec_flags (NM_DEVICE_CAPABILITIES, "", "",
NM_TYPE_DEVICE_CAPABILITIES,
NM_DEVICE_CAP_NONE,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm-glib/libnm: add support for "real" NMDevice property
2014-10-06 11:34:02 -05:00
/**
* NMDevice:real:
*
* Whether the device is real or is a placeholder device that could
* be created automatically by NetworkManager if one of its
* #NMDevice:available-connections was activated.
*
* Since: 1.2
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_REAL] =
g_param_spec_boolean (NM_DEVICE_REAL, "", "",
FALSE,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm-glib/libnm: add support for "real" NMDevice property
2014-10-06 11:34:02 -05:00
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:managed:
*
* Whether the device is managed by NetworkManager.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_MANAGED] =
g_param_spec_boolean (NM_DEVICE_MANAGED, "", "",
FALSE,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:autoconnect:
*
* Whether the device can auto-activate a connection.
libnm: deprecate synchronous/blocking API in libnm Note that D-Bus is fundamentally asynchronous. Doing blocking calls on top of D-Bus is odd, especially for libnm's NMClient. That is because NMClient essentially is a client-side cache of the objects from the D-Bus interface. This cache should be filled exclusively by (asynchronous) D-Bus events (PropertiesChanged). So, making a blocking D-Bus call means to wait for a response and return it, while queuing all messages that are received in the meantime. Basically there are three ways how a synchronous API on NMClient could behave: 1) the call just calls g_dbus_connection_call_sync(). This means that libnm sends a D-Bus request via GDBusConnection, and blockingly waits for the response. All D-Bus messages that get received in the meantime are queued in the GMainContext that belongs to NMClient. That means, none of these D-Bus events are processed until we iterate the GMainContext after the call returns. The effect is, that NMClient (and all cached objects in there) are unaffected by the D-Bus request. Most of the synchronous API calls in libnm are of this kind. The problem is that the strict ordering of D-Bus events gets violated. For some API this is not an immediate problem. Take for example nm_device_wifi_request_scan(). The call merely blockingly tells NetworkManager to start scanning, but since NetworkManager's D-Bus API does not directly expose any state that tells whether we are currently scanning, this out of order processing of the D-Bus request is a small issue. The problem is more obvious for nm_client_networking_set_enabled(). After calling it, NM_CLIENT_NETWORKING_ENABLED is still unaffected and unchanged, because the PropertiesChanged signal from D-Bus is not yet processed. This means, while you make such a blocking call, NMClient's state does not change. But usually you perform the synchronous call to change some state. In this form, the blocking call is not useful, because NMClient only changes the state after iterating the GMainContext, and not after the blocking call returns. 2) like 1), but after making the blocking g_dbus_connection_call_sync(), update the NMClient cache artificially. This is what nm_manager_check_connectivity() does, to "fix" bgo#784629. This also has the problem of out-of-order events, but it kinda solves the problem of not changing the state during the blocking call. But it does so by hacking the state of the cache. I think this is really wrong because the state should only be updated from the ordered stream of D-Bus messages (PropertiesChanged signal and similar). When libnm decides to modify the state, there may be already D-Bus messages queued that affect this very state. 3) instead of calling g_dbus_connection_call_sync(), use the asynchronous g_dbus_connection_call(). If we would use a sepaate GMainContext for all D-Bus related calls, we could ensure that while we block for the response, we iterate that internal main context. This might be nice, because all events are processed in order and after the blocking call returns, the NMClient state is up to date. The are problems however: current blocking API does not do this, so it's a significant change in behavior. Also, it might be unexpected to the user that during the blocking call the entire content of NMClient's cache might change and all pointers to the cache might be invalidated. Also, of course NMClient would invoke signals for all the changes that happen. Another problem is that this would be more effort to implement and it involves a small performance overhead for all D-Bus related calls (because we have to serialize all events in an internal GMainContext first and then invoke them on the caller's context). Also, if the users wants this behavior, they could implement it themself by running libnm in their own GMainContext. Note that libnm might have bugs to make that really working, but that should be fixed instead of adding such synchrnous API behavior. Read also [1], for why blocking calls are wrong. [1] https://smcv.pseudorandom.co.uk/2008/11/nonblocking/ So, all possible behaviors for synchronous API have severe behavioural issues. Mark all this API as deprecated. Also, this serves the purpose of identifying blocking D-Bus calls in libnm. Note that "deprecated" here does not really mean that the API is going to be removed. We don't break API. The user may: - continue to use this API. It's deprecated, awkward and discouraged, but if it works, by all means use it. - use asynchronous API. That's the only sensible way to use D-Bus. If libnm lacks a certain asynchronous counterpart, it should be added. - use GDBusConnection directly. There really isn't anything wrong with D-Bus or GDBusConnection. This deprecated API is just a wrapper around g_dbus_connection_call_sync(). You may call it directly without feeling dirty. --- The only other remainging API is the synchronous GInitable call for NMClient. That is an entirely separate beast and not particularly wrong (from an API point of view). Note that synchronous API in NMSecretAgentOld, NMVpnPluginOld and NMVpnServicePlugin as not deprecated here. These types are not part of the D-Bus cache and while they have similar issues, it's less severe because they have less state.
2019-09-04 13:58:43 +02:00
*
* The property setter is a synchronous D-Bus call. This is deprecated since 1.22.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_AUTOCONNECT] =
g_param_spec_boolean (NM_DEVICE_AUTOCONNECT, "", "",
libnm: change default value for NMDevice:autoconnect property Yes, by default (server side) devices do autoconnect. But that does not mean an NMObject, that has its GObject property not set via D-Bus shall default to TRUE. Default values preferably should be FALSE, because that is what we get by default (memset(0)).
2019-10-20 11:14:52 +02:00
FALSE,
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
G_PARAM_READWRITE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:firmware-missing:
*
* When %TRUE indicates the device is likely missing firmware required
* for its operation.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_FIRMWARE_MISSING] =
g_param_spec_boolean (NM_DEVICE_FIRMWARE_MISSING, "", "",
FALSE,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: add nm-plugin-missing property indicating NM device plugin not available
2015-04-14 14:31:42 +02:00
/**
* NMDevice:nm-plugin-missing:
*
* When %TRUE indicates that the NetworkManager plugin for the device
* is not installed.
*
* Since: 1.2
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_NM_PLUGIN_MISSING] =
g_param_spec_boolean (NM_DEVICE_NM_PLUGIN_MISSING, "", "",
FALSE,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add nm-plugin-missing property indicating NM device plugin not available
2015-04-14 14:31:42 +02:00
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:ip4-config:
*
* The #NMIP4Config of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_IP4_CONFIG] =
g_param_spec_object (NM_DEVICE_IP4_CONFIG, "", "",
NM_TYPE_IP_CONFIG,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:dhcp4-config:
*
libnm: create NMDhcpConfig as parent of NMDhcp4Config and NMDhcp6Config As with NMIP4Config and NMIP6Config, merge the two DHCP config classes into one in the public API.
2014-10-22 13:48:18 -04:00
* The IPv4 #NMDhcpConfig of the device.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_DHCP4_CONFIG] =
g_param_spec_object (NM_DEVICE_DHCP4_CONFIG, "", "",
NM_TYPE_DHCP_CONFIG,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:ip6-config:
*
libnm: create NMIPConfig as parent of NMIP4Config and NMIP6Config Create NMIPConfig as the parent of NMIP4Config and NMIP6Config, and remove the two subclasses from the public API; while it's convenient to still have both internally, they are now identical to the outside world.
2014-10-22 13:48:18 -04:00
* The IPv6 #NMIPConfig of the device.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_IP6_CONFIG] =
g_param_spec_object (NM_DEVICE_IP6_CONFIG, "", "",
NM_TYPE_IP_CONFIG,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:dhcp6-config:
*
libnm: create NMDhcpConfig as parent of NMDhcp4Config and NMDhcp6Config As with NMIP4Config and NMIP6Config, merge the two DHCP config classes into one in the public API.
2014-10-22 13:48:18 -04:00
* The IPv6 #NMDhcpConfig of the device.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_DHCP6_CONFIG] =
g_param_spec_object (NM_DEVICE_DHCP6_CONFIG, "", "",
NM_TYPE_DHCP_CONFIG,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: add support for per-device & per-AF connectivity status
2018-07-03 20:42:50 +02:00
/**
* NMDevice:ip4-connectivity:
*
* The IPv4 connectivity state of the device.
*
* Since: 1.16
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_IP4_CONNECTIVITY] =
g_param_spec_enum (NM_DEVICE_IP4_CONNECTIVITY, "", "",
NM_TYPE_CONNECTIVITY_STATE,
NM_CONNECTIVITY_UNKNOWN,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add support for per-device & per-AF connectivity status
2018-07-03 20:42:50 +02:00
/**
* NMDevice:ip6-connectivity:
*
* The IPv6 connectivity state of the device.
*
* Since: 1.16
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_IP6_CONNECTIVITY] =
g_param_spec_enum (NM_DEVICE_IP6_CONNECTIVITY, "", "",
NM_TYPE_CONNECTIVITY_STATE,
NM_CONNECTIVITY_UNKNOWN,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add support for per-device & per-AF connectivity status
2018-07-03 20:42:50 +02:00
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:state:
*
* The state of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_STATE] =
g_param_spec_enum (NM_DEVICE_STATE, "", "",
NM_TYPE_DEVICE_STATE,
NM_DEVICE_STATE_UNKNOWN,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:state-reason:
*
libnm: simplify NMDevice:state-reason The NMDevice:state-reason property was awkward to work with since it was a tuple of two values (state and reason), and likewise, although we had nm_device_get_state() to return just the state, there was no corresponding function to get just the reason. Fix this; make the state-reason property contain just the NMDeviceStateReason, and make nm_device_get_state_reason() return just that.
2014-08-25 10:51:26 -04:00
* The reason for the device state.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_STATE_REASON] =
g_param_spec_uint (NM_DEVICE_STATE_REASON, "", "",
0, G_MAXUINT32, 0,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:active-connection:
*
* The #NMActiveConnection object that "owns" this device during activation.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_ACTIVE_CONNECTION] =
g_param_spec_object (NM_DEVICE_ACTIVE_CONNECTION, "", "",
NM_TYPE_ACTIVE_CONNECTION,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
gobject-introspection: made several fixes to the annotations https://bugzilla.gnome.org/show_bug.cgi?id=794658
2018-03-24 15:18:21 +00:00
* NMDevice:available-connections: (type GPtrArray(NMRemoteConnection))
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: drop NM_TYPE_OBJECT_ARRAY, use G_TYPE_PTR_ARRAY Use G_TYPE_PTR_ARRAY for GPtrArray-of-NMObject-valued properties, because it has better introspection/bindings support. As with the strdict change in libnm-core, we need to manually copy the array in get_property() implementations, to preserve the standard semantics that get_property() returns a copy, not the internal array. (This patch also changes those properties so that they are always non-NULL until dispose(); previously some of them could be either NULL or 0-length at different times.)
2014-08-26 08:31:04 -04:00
* The available connections of the device
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_AVAILABLE_CONNECTIONS] =
g_param_spec_boxed (NM_DEVICE_AVAILABLE_CONNECTIONS, "", "",
G_TYPE_PTR_ARRAY,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:vendor:
*
* The vendor string of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_VENDOR] =
g_param_spec_string (NM_DEVICE_VENDOR, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:product:
*
* The product string of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_PRODUCT] =
g_param_spec_string (NM_DEVICE_PRODUCT, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:physical-port-id:
*
* The physical port ID of the device. (See
* nm_device_get_physical_port_id().)
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_PHYSICAL_PORT_ID] =
g_param_spec_string (NM_DEVICE_PHYSICAL_PORT_ID, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice:mtu:
*
* The MTU of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_MTU] =
g_param_spec_uint (NM_DEVICE_MTU, "", "",
libnm: change default value for NMDevice:mtu property Default values should preferably be zero and/or a value that indicates that the property is unknown/unset. In practice, this property is not unset because it's present on the D-Bus API.
2019-10-20 11:14:52 +02:00
0, G_MAXUINT32, 0,
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
core,libnm: add 'metered' property to NMDevice
2015-04-29 16:34:38 +02:00
/**
* NMDevice:metered:
*
* Whether the device is metered.
*
* Since: 1.2
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_METERED] =
g_param_spec_uint (NM_DEVICE_METERED, "", "",
0, G_MAXUINT32, NM_METERED_UNKNOWN,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
core,libnm: add 'metered' property to NMDevice
2015-04-29 16:34:38 +02:00
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
/**
* NMDevice:lldp-neighbors:
*
* The LLDP neighbors.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_LLDP_NEIGHBORS] =
g_param_spec_boxed (NM_DEVICE_LLDP_NEIGHBORS, "", "",
G_TYPE_PTR_ARRAY,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
libnm: export interface flags Add libnm support for the new InterfaceFlags property of NMDevice.
2019-10-10 10:04:06 +02:00
/**
* NMDevice:interface-flags:
*
* The interface flags.
*
* Since: 1.22
**/
obj_properties[PROP_INTERFACE_FLAGS] =
g_param_spec_uint (NM_DEVICE_INTERFACE_FLAGS, "", "",
0, G_MAXUINT32, 0,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
/**
* NMDevice:hw-address:
*
* The hardware address of the device.
*
* Since: 1.24
**/
obj_properties[PROP_HW_ADDRESS] =
g_param_spec_string (NM_DEVICE_HW_ADDRESS, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nml_dbus_meta_class_init_with_properties (object_class, &_nml_dbus_meta_iface_nm_device);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDevice::state-changed:
* @device: the device object that received the signal
* @new_state: the new state of the device
* @old_state: the previous state of the device
* @reason: the reason describing the state change
*
* Notifies the state change of a #NMDevice.
**/
signals[STATE_CHANGED] =
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
g_signal_new ("state-changed",
G_OBJECT_CLASS_TYPE (object_class),
G_SIGNAL_RUN_FIRST,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
0, NULL, NULL, NULL,
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
G_TYPE_NONE, 3,
G_TYPE_UINT, G_TYPE_UINT, G_TYPE_UINT);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_iface:
* @device: a #NMDevice
*
* Gets the interface name of the #NMDevice.
*
* Returns: the interface of the device. This is the internal string used by the
* device, and must not be modified.
**/
const char *
nm_device_get_iface (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return _nml_coerce_property_str_not_empty (NM_DEVICE_GET_PRIVATE (device)->interface);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_ip_iface:
* @device: a #NMDevice
*
* Gets the IP interface name of the #NMDevice over which IP traffic flows
* when the device is in the ACTIVATED state.
*
* Returns: the IP traffic interface of the device. This is the internal string
* used by the device, and must not be modified.
**/
const char *
nm_device_get_ip_iface (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return _nml_coerce_property_str_not_empty (NM_DEVICE_GET_PRIVATE (device)->ip_interface);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_device_type:
* @device: a #NMDevice
*
* Returns the numeric type of the #NMDevice, ie Ethernet, Wi-Fi, etc.
*
* Returns: the device type
**/
NMDeviceType
nm_device_get_device_type (NMDevice *self)
{
g_return_val_if_fail (NM_IS_DEVICE (self), NM_DEVICE_TYPE_UNKNOWN);
libnm: Fix reporting of unknown device types nm_device_get_device_type would report the device type as it was send on DBus, while fetching the property would mean that only a known device types is reported. Make both results consistent by coercing in nm_device_get_device_type rather than when setting the property. (cherry picked from commit a6a185ba00c6218d9b1ace6e3bd6b57347198246)
2019-03-07 21:31:47 +01:00
return coerce_type (NM_DEVICE_GET_PRIVATE (self)->device_type);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_udi:
* @device: a #NMDevice
*
* Gets the Unique Device Identifier of the #NMDevice.
*
* Returns: the Unique Device Identifier of the device. This identifier may be
* used to gather more information about the device from various operating
* system services like udev or sysfs.
**/
const char *
nm_device_get_udi (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: add and use _nml_coerce_property_*() Our NMObject implementations should behave in a similar manner. For example, string properties should be coerced the a consistent manner. Add functions _nml_coerce_property_*() for that. Of course, they are trivial. Their value is not that they encapsulate some complex implementation, but that they convey a general concept of how we want to handle certain properties in NMClient's object cache.
2019-10-22 16:51:09 +02:00
return _nml_coerce_property_str_not_empty (NM_DEVICE_GET_PRIVATE (device)->udi);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
all: add device.path property Add a device property to expose its path as reported in the ID_PATH udev property.
2020-06-10 15:17:39 +02:00
/**
* nm_device_get_path:
* @device: a #NMDevice
*
* Gets the path of the #NMDevice as exposed by the udev property ID_PATH.
*
* Returns: the path of the device.
*
* The string is backslash escaped (C escaping) for invalid characters. The escaping
* can be reverted with g_strcompress(), however the result may not be valid UTF-8.
*
* Since: 1.26
**/
const char *
nm_device_get_path (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
return _nml_coerce_property_str_not_empty (NM_DEVICE_GET_PRIVATE (device)->path);
}
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* nm_device_get_driver:
* @device: a #NMDevice
*
* Gets the driver of the #NMDevice.
*
* Returns: the driver of the device. This is the internal string used by the
* device, and must not be modified.
**/
const char *
nm_device_get_driver (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: add and use _nml_coerce_property_*() Our NMObject implementations should behave in a similar manner. For example, string properties should be coerced the a consistent manner. Add functions _nml_coerce_property_*() for that. Of course, they are trivial. Their value is not that they encapsulate some complex implementation, but that they convey a general concept of how we want to handle certain properties in NMClient's object cache.
2019-10-22 16:51:09 +02:00
return _nml_coerce_property_str_not_empty (NM_DEVICE_GET_PRIVATE (device)->driver);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_driver_version:
* @device: a #NMDevice
*
* Gets the driver version of the #NMDevice.
*
* Returns: the version of the device driver. This is the internal string used by the
* device, and must not be modified.
**/
const char *
nm_device_get_driver_version (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: add and use _nml_coerce_property_*() Our NMObject implementations should behave in a similar manner. For example, string properties should be coerced the a consistent manner. Add functions _nml_coerce_property_*() for that. Of course, they are trivial. Their value is not that they encapsulate some complex implementation, but that they convey a general concept of how we want to handle certain properties in NMClient's object cache.
2019-10-22 16:51:09 +02:00
return _nml_coerce_property_str_not_empty (NM_DEVICE_GET_PRIVATE (device)->driver_version);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_firmware_version:
* @device: a #NMDevice
*
* Gets the firmware version of the #NMDevice.
*
* Returns: the firmware version of the device. This is the internal string used by the
* device, and must not be modified.
**/
const char *
nm_device_get_firmware_version (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: add and use _nml_coerce_property_*() Our NMObject implementations should behave in a similar manner. For example, string properties should be coerced the a consistent manner. Add functions _nml_coerce_property_*() for that. Of course, they are trivial. Their value is not that they encapsulate some complex implementation, but that they convey a general concept of how we want to handle certain properties in NMClient's object cache.
2019-10-22 16:51:09 +02:00
return _nml_coerce_property_str_not_empty (NM_DEVICE_GET_PRIVATE (device)->firmware_version);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_type_description:
* @device: a #NMDevice
*
* Gets a (non-localized) description of the type of device that
* @device is.
*
* Returns: the type description of the device. This is the internal
* string used by the device, and must not be modified.
**/
const char *
nm_device_get_type_description (NMDevice *device)
{
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (device);
const char *desc, *typename;
device: add nm_device_get_type_description() function Add a function to get a concise representation of the device type. libnm already has nm_device_get_type_description() for that and it is shown by nmcli -f GENERAL.TYPE device show Reimplement that function for nm-core. Just take care that the two implementations don't diverge.
2015-05-15 15:00:07 +02:00
/* BEWARE: this function should return the same value
* as nm_device_get_type_description() in nm-core. */
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
if (priv->type_description)
libnm: add and use _nml_coerce_property_*() Our NMObject implementations should behave in a similar manner. For example, string properties should be coerced the a consistent manner. Add functions _nml_coerce_property_*() for that. Of course, they are trivial. Their value is not that they encapsulate some complex implementation, but that they convey a general concept of how we want to handle certain properties in NMClient's object cache.
2019-10-22 16:51:09 +02:00
return _nml_coerce_property_str_not_empty (priv->type_description);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
if (NM_DEVICE_GET_CLASS (device)->get_type_description) {
desc = NM_DEVICE_GET_CLASS (device)->get_type_description (device);
if (desc)
return desc;
}
typename = G_OBJECT_TYPE_NAME (device);
if (g_str_has_prefix (typename, "NMDevice"))
typename += 8;
priv->type_description = g_ascii_strdown (typename, -1);
libnm: add and use _nml_coerce_property_*() Our NMObject implementations should behave in a similar manner. For example, string properties should be coerced the a consistent manner. Add functions _nml_coerce_property_*() for that. Of course, they are trivial. Their value is not that they encapsulate some complex implementation, but that they convey a general concept of how we want to handle certain properties in NMClient's object cache.
2019-10-22 16:51:09 +02:00
return _nml_coerce_property_str_not_empty (priv->type_description);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
NMLDBusNotifyUpdatePropFlags
_nm_device_notify_update_prop_hw_address (NMClient *client,
NMLDBusObject *dbobj,
const NMLDBusMetaIface *meta_iface,
guint dbus_property_idx,
GVariant *value)
{
NMDevice *self = NM_DEVICE (dbobj->nmobj);
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (self);
gboolean is_new = (meta_iface == &_nml_dbus_meta_iface_nm_device);
gboolean changed = FALSE;
if ( !is_new
&& priv->hw_address_is_new) {
/* once the instance is marked to honor the new property, the
* changed signal for the old variant gets ignored. */
goto out;
}
if (!value) {
if (nm_clear_g_free (&priv->hw_address))
changed = TRUE;
goto out;
}
priv->hw_address_is_new = is_new;
nm_utils_strdup_reset (&priv->hw_address,
_nml_coerce_property_str_not_empty (g_variant_get_string (value, NULL)));
/* always emit a changed signal here, even if "priv->hw_address" might be unchanged.
* We want to emit the signal because we received a PropertiesChanged signal on D-Bus,
* even if nothing actually changed. */
changed = TRUE;
out:
if (changed) {
_nm_client_queue_notify_object (client,
self,
obj_properties[PROP_HW_ADDRESS]);
}
return NML_DBUS_NOTIFY_UPDATE_PROP_FLAGS_NONE;
}
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* nm_device_get_hw_address:
* @device: a #NMDevice
*
* Gets the current a hardware address (MAC) for the @device.
*
* Returns: the current MAC of the device, or %NULL.
* This is the internal string used by the device, and must not be modified.
**/
const char *
nm_device_get_hw_address (NMDevice *device)
{
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
NMDevicePrivate *priv;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
priv = NM_DEVICE_GET_PRIVATE (device);
nm_assert (!nm_streq0 (priv->hw_address, ""));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
return priv->hw_address;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_capabilities:
* @device: a #NMDevice
*
* Gets the device' capabilities.
*
* Returns: the capabilities
**/
NMDeviceCapabilities
nm_device_get_capabilities (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), 0);
return NM_DEVICE_GET_PRIVATE (device)->capabilities;
}
/**
* nm_device_get_managed:
* @device: a #NMDevice
*
* Whether the #NMDevice is managed by NetworkManager.
*
* Returns: %TRUE if the device is managed by NetworkManager
**/
gboolean
nm_device_get_managed (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), 0);
return NM_DEVICE_GET_PRIVATE (device)->managed;
}
libnm,libnm-glib: add Device.Managed setter
2015-03-23 13:31:10 +01:00
/**
* nm_device_set_managed:
* @device: a #NMDevice
* @managed: %TRUE to make the device managed by NetworkManager.
*
* Enables or disables management of #NMDevice by NetworkManager.
*
* Since: 1.2
libnm: deprecate synchronous/blocking API in libnm Note that D-Bus is fundamentally asynchronous. Doing blocking calls on top of D-Bus is odd, especially for libnm's NMClient. That is because NMClient essentially is a client-side cache of the objects from the D-Bus interface. This cache should be filled exclusively by (asynchronous) D-Bus events (PropertiesChanged). So, making a blocking D-Bus call means to wait for a response and return it, while queuing all messages that are received in the meantime. Basically there are three ways how a synchronous API on NMClient could behave: 1) the call just calls g_dbus_connection_call_sync(). This means that libnm sends a D-Bus request via GDBusConnection, and blockingly waits for the response. All D-Bus messages that get received in the meantime are queued in the GMainContext that belongs to NMClient. That means, none of these D-Bus events are processed until we iterate the GMainContext after the call returns. The effect is, that NMClient (and all cached objects in there) are unaffected by the D-Bus request. Most of the synchronous API calls in libnm are of this kind. The problem is that the strict ordering of D-Bus events gets violated. For some API this is not an immediate problem. Take for example nm_device_wifi_request_scan(). The call merely blockingly tells NetworkManager to start scanning, but since NetworkManager's D-Bus API does not directly expose any state that tells whether we are currently scanning, this out of order processing of the D-Bus request is a small issue. The problem is more obvious for nm_client_networking_set_enabled(). After calling it, NM_CLIENT_NETWORKING_ENABLED is still unaffected and unchanged, because the PropertiesChanged signal from D-Bus is not yet processed. This means, while you make such a blocking call, NMClient's state does not change. But usually you perform the synchronous call to change some state. In this form, the blocking call is not useful, because NMClient only changes the state after iterating the GMainContext, and not after the blocking call returns. 2) like 1), but after making the blocking g_dbus_connection_call_sync(), update the NMClient cache artificially. This is what nm_manager_check_connectivity() does, to "fix" bgo#784629. This also has the problem of out-of-order events, but it kinda solves the problem of not changing the state during the blocking call. But it does so by hacking the state of the cache. I think this is really wrong because the state should only be updated from the ordered stream of D-Bus messages (PropertiesChanged signal and similar). When libnm decides to modify the state, there may be already D-Bus messages queued that affect this very state. 3) instead of calling g_dbus_connection_call_sync(), use the asynchronous g_dbus_connection_call(). If we would use a sepaate GMainContext for all D-Bus related calls, we could ensure that while we block for the response, we iterate that internal main context. This might be nice, because all events are processed in order and after the blocking call returns, the NMClient state is up to date. The are problems however: current blocking API does not do this, so it's a significant change in behavior. Also, it might be unexpected to the user that during the blocking call the entire content of NMClient's cache might change and all pointers to the cache might be invalidated. Also, of course NMClient would invoke signals for all the changes that happen. Another problem is that this would be more effort to implement and it involves a small performance overhead for all D-Bus related calls (because we have to serialize all events in an internal GMainContext first and then invoke them on the caller's context). Also, if the users wants this behavior, they could implement it themself by running libnm in their own GMainContext. Note that libnm might have bugs to make that really working, but that should be fixed instead of adding such synchrnous API behavior. Read also [1], for why blocking calls are wrong. [1] https://smcv.pseudorandom.co.uk/2008/11/nonblocking/ So, all possible behaviors for synchronous API have severe behavioural issues. Mark all this API as deprecated. Also, this serves the purpose of identifying blocking D-Bus calls in libnm. Note that "deprecated" here does not really mean that the API is going to be removed. We don't break API. The user may: - continue to use this API. It's deprecated, awkward and discouraged, but if it works, by all means use it. - use asynchronous API. That's the only sensible way to use D-Bus. If libnm lacks a certain asynchronous counterpart, it should be added. - use GDBusConnection directly. There really isn't anything wrong with D-Bus or GDBusConnection. This deprecated API is just a wrapper around g_dbus_connection_call_sync(). You may call it directly without feeling dirty. --- The only other remainging API is the synchronous GInitable call for NMClient. That is an entirely separate beast and not particularly wrong (from an API point of view). Note that synchronous API in NMSecretAgentOld, NMVpnPluginOld and NMVpnServicePlugin as not deprecated here. These types are not part of the D-Bus cache and while they have similar issues, it's less severe because they have less state.
2019-09-04 13:58:43 +02:00
*
libnm: advise using D-Bus instead of deprecated synchronous methods With 1.22, various synchronous functions for invoking D-Bus methods were deprecated. The reason was that D-Bus is fundamentally asynchronous, and providing synchronous API in NMClient is inherently wrong. That is because NMClient essentially is a cache of the D-Bus API, and invoking g_dbus_connection_call_sync() messes up the order of events from D-Bus. In particular, when the synchronous function completes, the content of the cache does not yet reflect the change. Since they got deprecated, the question is with what to replace them. Instead of adding a (e.g.) nm_client_networking_set_enabled_async() for nm_client_networking_set_enabled(), just expect the user to call D-Bus directly. D-Bus itself defines a reasonable API, and with GDBusConnection it is fine (and convenient) to just call D-Bus operations directly. Often libraries try to abstract D-Bus by providing convenience wrappers around D-Bus API. I think that often is wrong and unnecessary. Note that libnm's NMClient does a lot more than just wrapping simple D-Bus calls. It provides a complete client-side cache of the D-Bus interface. As such, what libnm's NMClient does is more than simple wrappers around D-Bus. NMClient is a reasonable thing to do. However, it is unnecessary to add API like nm_client_networking_set_enabled_async() that only calls g_dbus_connection_call(). Don't pretend that we would need such trivial wrappers in libnm. Instead, recommend to use g_dbus_connection_call(). Or alternatively, the convenience wrappers nm_client_dbus_call() and nm_client_dbus_set_property().
2020-03-13 18:24:37 +01:00
* Deprecated: 1.22: Use the async command nm_client_dbus_set_property() on
* nm_object_get_path(), interface %NM_DBUS_INTERFACE_DEVICE to set the
* "Managed" property to a "(b)" boolean value.
libnm: deprecate nm_device_set_managed() and nm_device_set_autoconnect() API These setters not only invoke a synchronous D-Bus call (ignoring the return value). They also modify the content of the cache client-side, bypassing the information that we receive via notifications from the server. Also, they don't emit property changed signals, but in any case they are broken beyond repair. Fully mark them as deprecated. Note that they were already marked as _NM_DEPRECATED_SYNC_METHOD. However, that does not actually mark the API as deprecated, because fully deprecating all synchronous methods is premature at this point.
2019-10-28 17:18:39 +01:00
* This function is deprecated because it calls a synchronous D-Bus method
* and modifies the content of the NMClient cache client side. Also, it does
* not emit a property changed signal.
libnm,libnm-glib: add Device.Managed setter
2015-03-23 13:31:10 +01:00
**/
void
nm_device_set_managed (NMDevice *device, gboolean managed)
{
g_return_if_fail (NM_IS_DEVICE (device));
managed = !!managed;
NM_DEVICE_GET_PRIVATE (device)->managed = managed;
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_client_set_property_sync_legacy (_nm_object_get_client (device),
_nm_object_get_path (device),
NM_DBUS_INTERFACE_DEVICE,
"Managed",
"b",
managed);
libnm,libnm-glib: add Device.Managed setter
2015-03-23 13:31:10 +01:00
}
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* nm_device_get_autoconnect:
* @device: a #NMDevice
*
* Whether the #NMDevice can be autoconnected.
*
* Returns: %TRUE if the device is allowed to be autoconnected
**/
gboolean
nm_device_get_autoconnect (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), FALSE);
return NM_DEVICE_GET_PRIVATE (device)->autoconnect;
}
/**
* nm_device_set_autoconnect:
* @device: a #NMDevice
* @autoconnect: %TRUE to enable autoconnecting
*
* Enables or disables automatic activation of the #NMDevice.
libnm: deprecate synchronous/blocking API in libnm Note that D-Bus is fundamentally asynchronous. Doing blocking calls on top of D-Bus is odd, especially for libnm's NMClient. That is because NMClient essentially is a client-side cache of the objects from the D-Bus interface. This cache should be filled exclusively by (asynchronous) D-Bus events (PropertiesChanged). So, making a blocking D-Bus call means to wait for a response and return it, while queuing all messages that are received in the meantime. Basically there are three ways how a synchronous API on NMClient could behave: 1) the call just calls g_dbus_connection_call_sync(). This means that libnm sends a D-Bus request via GDBusConnection, and blockingly waits for the response. All D-Bus messages that get received in the meantime are queued in the GMainContext that belongs to NMClient. That means, none of these D-Bus events are processed until we iterate the GMainContext after the call returns. The effect is, that NMClient (and all cached objects in there) are unaffected by the D-Bus request. Most of the synchronous API calls in libnm are of this kind. The problem is that the strict ordering of D-Bus events gets violated. For some API this is not an immediate problem. Take for example nm_device_wifi_request_scan(). The call merely blockingly tells NetworkManager to start scanning, but since NetworkManager's D-Bus API does not directly expose any state that tells whether we are currently scanning, this out of order processing of the D-Bus request is a small issue. The problem is more obvious for nm_client_networking_set_enabled(). After calling it, NM_CLIENT_NETWORKING_ENABLED is still unaffected and unchanged, because the PropertiesChanged signal from D-Bus is not yet processed. This means, while you make such a blocking call, NMClient's state does not change. But usually you perform the synchronous call to change some state. In this form, the blocking call is not useful, because NMClient only changes the state after iterating the GMainContext, and not after the blocking call returns. 2) like 1), but after making the blocking g_dbus_connection_call_sync(), update the NMClient cache artificially. This is what nm_manager_check_connectivity() does, to "fix" bgo#784629. This also has the problem of out-of-order events, but it kinda solves the problem of not changing the state during the blocking call. But it does so by hacking the state of the cache. I think this is really wrong because the state should only be updated from the ordered stream of D-Bus messages (PropertiesChanged signal and similar). When libnm decides to modify the state, there may be already D-Bus messages queued that affect this very state. 3) instead of calling g_dbus_connection_call_sync(), use the asynchronous g_dbus_connection_call(). If we would use a sepaate GMainContext for all D-Bus related calls, we could ensure that while we block for the response, we iterate that internal main context. This might be nice, because all events are processed in order and after the blocking call returns, the NMClient state is up to date. The are problems however: current blocking API does not do this, so it's a significant change in behavior. Also, it might be unexpected to the user that during the blocking call the entire content of NMClient's cache might change and all pointers to the cache might be invalidated. Also, of course NMClient would invoke signals for all the changes that happen. Another problem is that this would be more effort to implement and it involves a small performance overhead for all D-Bus related calls (because we have to serialize all events in an internal GMainContext first and then invoke them on the caller's context). Also, if the users wants this behavior, they could implement it themself by running libnm in their own GMainContext. Note that libnm might have bugs to make that really working, but that should be fixed instead of adding such synchrnous API behavior. Read also [1], for why blocking calls are wrong. [1] https://smcv.pseudorandom.co.uk/2008/11/nonblocking/ So, all possible behaviors for synchronous API have severe behavioural issues. Mark all this API as deprecated. Also, this serves the purpose of identifying blocking D-Bus calls in libnm. Note that "deprecated" here does not really mean that the API is going to be removed. We don't break API. The user may: - continue to use this API. It's deprecated, awkward and discouraged, but if it works, by all means use it. - use asynchronous API. That's the only sensible way to use D-Bus. If libnm lacks a certain asynchronous counterpart, it should be added. - use GDBusConnection directly. There really isn't anything wrong with D-Bus or GDBusConnection. This deprecated API is just a wrapper around g_dbus_connection_call_sync(). You may call it directly without feeling dirty. --- The only other remainging API is the synchronous GInitable call for NMClient. That is an entirely separate beast and not particularly wrong (from an API point of view). Note that synchronous API in NMSecretAgentOld, NMVpnPluginOld and NMVpnServicePlugin as not deprecated here. These types are not part of the D-Bus cache and while they have similar issues, it's less severe because they have less state.
2019-09-04 13:58:43 +02:00
*
libnm: advise using D-Bus instead of deprecated synchronous methods With 1.22, various synchronous functions for invoking D-Bus methods were deprecated. The reason was that D-Bus is fundamentally asynchronous, and providing synchronous API in NMClient is inherently wrong. That is because NMClient essentially is a cache of the D-Bus API, and invoking g_dbus_connection_call_sync() messes up the order of events from D-Bus. In particular, when the synchronous function completes, the content of the cache does not yet reflect the change. Since they got deprecated, the question is with what to replace them. Instead of adding a (e.g.) nm_client_networking_set_enabled_async() for nm_client_networking_set_enabled(), just expect the user to call D-Bus directly. D-Bus itself defines a reasonable API, and with GDBusConnection it is fine (and convenient) to just call D-Bus operations directly. Often libraries try to abstract D-Bus by providing convenience wrappers around D-Bus API. I think that often is wrong and unnecessary. Note that libnm's NMClient does a lot more than just wrapping simple D-Bus calls. It provides a complete client-side cache of the D-Bus interface. As such, what libnm's NMClient does is more than simple wrappers around D-Bus. NMClient is a reasonable thing to do. However, it is unnecessary to add API like nm_client_networking_set_enabled_async() that only calls g_dbus_connection_call(). Don't pretend that we would need such trivial wrappers in libnm. Instead, recommend to use g_dbus_connection_call(). Or alternatively, the convenience wrappers nm_client_dbus_call() and nm_client_dbus_set_property().
2020-03-13 18:24:37 +01:00
* Deprecated: 1.22: Use the async command nm_client_dbus_set_property() on
* nm_object_get_path(), %NM_DBUS_INTERFACE_DEVICE to set "AutoConnect" property to a "(b)" value.
libnm: deprecate nm_device_set_managed() and nm_device_set_autoconnect() API These setters not only invoke a synchronous D-Bus call (ignoring the return value). They also modify the content of the cache client-side, bypassing the information that we receive via notifications from the server. Also, they don't emit property changed signals, but in any case they are broken beyond repair. Fully mark them as deprecated. Note that they were already marked as _NM_DEPRECATED_SYNC_METHOD. However, that does not actually mark the API as deprecated, because fully deprecating all synchronous methods is premature at this point.
2019-10-28 17:18:39 +01:00
* This function is deprecated because it calls a synchronous D-Bus method
* and modifies the content of the NMClient cache client side.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
void
nm_device_set_autoconnect (NMDevice *device, gboolean autoconnect)
{
g_return_if_fail (NM_IS_DEVICE (device));
NM_DEVICE_GET_PRIVATE (device)->autoconnect = autoconnect;
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_client_set_property_sync_legacy (_nm_object_get_client (device),
_nm_object_get_path (device),
NM_DBUS_INTERFACE_DEVICE,
"AutoConnect",
"b",
autoconnect);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_firmware_missing:
* @device: a #NMDevice
*
* Indicates that firmware required for the device's operation is likely
* to be missing.
*
* Returns: %TRUE if firmware required for the device's operation is likely
* to be missing.
**/
gboolean
nm_device_get_firmware_missing (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), 0);
return NM_DEVICE_GET_PRIVATE (device)->firmware_missing;
}
libnm: add nm-plugin-missing property indicating NM device plugin not available
2015-04-14 14:31:42 +02:00
/**
* nm_device_get_nm_plugin_missing:
* @device: a #NMDevice
*
* Indicates that the NetworkManager plugin for the device is not installed.
*
* Returns: %TRUE if the device plugin not installed.
*
* Since: 1.2
**/
gboolean
nm_device_get_nm_plugin_missing (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), FALSE);
return NM_DEVICE_GET_PRIVATE (device)->nm_plugin_missing;
}
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* nm_device_get_ip4_config:
* @device: a #NMDevice
*
libnm: create NMIPConfig as parent of NMIP4Config and NMIP6Config Create NMIPConfig as the parent of NMIP4Config and NMIP6Config, and remove the two subclasses from the public API; while it's convenient to still have both internally, they are now identical to the outside world.
2014-10-22 13:48:18 -04:00
* Gets the current IPv4 #NMIPConfig associated with the #NMDevice.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: create NMIPConfig as parent of NMIP4Config and NMIP6Config Create NMIPConfig as the parent of NMIP4Config and NMIP6Config, and remove the two subclasses from the public API; while it's convenient to still have both internally, they are now identical to the outside world.
2014-10-22 13:48:18 -04:00
* You can alternatively use nm_active_connection_get_ip4_config(), which also
* works with VPN connections.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: create NMIPConfig as parent of NMIP4Config and NMIP6Config Create NMIPConfig as the parent of NMIP4Config and NMIP6Config, and remove the two subclasses from the public API; while it's convenient to still have both internally, they are now identical to the outside world.
2014-10-22 13:48:18 -04:00
* Returns: (transfer none): the IPv4 #NMIPConfig, or %NULL if the device is not
* activated.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: create NMIPConfig as parent of NMIP4Config and NMIP6Config Create NMIPConfig as the parent of NMIP4Config and NMIP6Config, and remove the two subclasses from the public API; while it's convenient to still have both internally, they are now identical to the outside world.
2014-10-22 13:48:18 -04:00
NMIPConfig *
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
nm_device_get_ip4_config (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return nml_dbus_property_o_get_obj (&NM_DEVICE_GET_PRIVATE (device)->property_o[PROPERTY_O_IDX_IP4_CONFIG]);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_dhcp4_config:
* @device: a #NMDevice
*
libnm: create NMDhcpConfig as parent of NMDhcp4Config and NMDhcp6Config As with NMIP4Config and NMIP6Config, merge the two DHCP config classes into one in the public API.
2014-10-22 13:48:18 -04:00
* Gets the current IPv4 #NMDhcpConfig associated with the #NMDevice.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: create NMDhcpConfig as parent of NMDhcp4Config and NMDhcp6Config As with NMIP4Config and NMIP6Config, merge the two DHCP config classes into one in the public API.
2014-10-22 13:48:18 -04:00
* You can alternatively use nm_active_connection_get_dhcp4_config(), which also
* works with VPN connections.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: create NMDhcpConfig as parent of NMDhcp4Config and NMDhcp6Config As with NMIP4Config and NMIP6Config, merge the two DHCP config classes into one in the public API.
2014-10-22 13:48:18 -04:00
* Returns: (transfer none): the IPv4 #NMDhcpConfig, or %NULL if the device is
* not activated or not using DHCP.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: create NMDhcpConfig as parent of NMDhcp4Config and NMDhcp6Config As with NMIP4Config and NMIP6Config, merge the two DHCP config classes into one in the public API.
2014-10-22 13:48:18 -04:00
NMDhcpConfig *
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
nm_device_get_dhcp4_config (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return nml_dbus_property_o_get_obj (&NM_DEVICE_GET_PRIVATE (device)->property_o[PROPERTY_O_IDX_DHCP4_CONFIG]);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_ip6_config:
* @device: a #NMDevice
*
libnm: create NMIPConfig as parent of NMIP4Config and NMIP6Config Create NMIPConfig as the parent of NMIP4Config and NMIP6Config, and remove the two subclasses from the public API; while it's convenient to still have both internally, they are now identical to the outside world.
2014-10-22 13:48:18 -04:00
* Gets the current IPv6 #NMIPConfig associated with the #NMDevice.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: create NMIPConfig as parent of NMIP4Config and NMIP6Config Create NMIPConfig as the parent of NMIP4Config and NMIP6Config, and remove the two subclasses from the public API; while it's convenient to still have both internally, they are now identical to the outside world.
2014-10-22 13:48:18 -04:00
* You can alternatively use nm_active_connection_get_ip6_config(), which also
* works with VPN connections.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: create NMIPConfig as parent of NMIP4Config and NMIP6Config Create NMIPConfig as the parent of NMIP4Config and NMIP6Config, and remove the two subclasses from the public API; while it's convenient to still have both internally, they are now identical to the outside world.
2014-10-22 13:48:18 -04:00
* Returns: (transfer none): the IPv6 #NMIPConfig or %NULL if the device is not activated.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: create NMIPConfig as parent of NMIP4Config and NMIP6Config Create NMIPConfig as the parent of NMIP4Config and NMIP6Config, and remove the two subclasses from the public API; while it's convenient to still have both internally, they are now identical to the outside world.
2014-10-22 13:48:18 -04:00
NMIPConfig *
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
nm_device_get_ip6_config (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return nml_dbus_property_o_get_obj (&NM_DEVICE_GET_PRIVATE (device)->property_o[PROPERTY_O_IDX_IP6_CONFIG]);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_dhcp6_config:
* @device: a #NMDevice
*
libnm: create NMDhcpConfig as parent of NMDhcp4Config and NMDhcp6Config As with NMIP4Config and NMIP6Config, merge the two DHCP config classes into one in the public API.
2014-10-22 13:48:18 -04:00
* Gets the current IPv6 #NMDhcpConfig associated with the #NMDevice.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: create NMDhcpConfig as parent of NMDhcp4Config and NMDhcp6Config As with NMIP4Config and NMIP6Config, merge the two DHCP config classes into one in the public API.
2014-10-22 13:48:18 -04:00
* You can alternatively use nm_active_connection_get_dhcp6_config(), which also
* works with VPN connections.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: create NMDhcpConfig as parent of NMDhcp4Config and NMDhcp6Config As with NMIP4Config and NMIP6Config, merge the two DHCP config classes into one in the public API.
2014-10-22 13:48:18 -04:00
* Returns: (transfer none): the IPv6 #NMDhcpConfig, or %NULL if the device is
* not activated or not using DHCPv6.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: create NMDhcpConfig as parent of NMDhcp4Config and NMDhcp6Config As with NMIP4Config and NMIP6Config, merge the two DHCP config classes into one in the public API.
2014-10-22 13:48:18 -04:00
NMDhcpConfig *
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
nm_device_get_dhcp6_config (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return nml_dbus_property_o_get_obj (&NM_DEVICE_GET_PRIVATE (device)->property_o[PROPERTY_O_IDX_DHCP6_CONFIG]);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
libnm: add support for per-device & per-AF connectivity status
2018-07-03 20:42:50 +02:00
/**
* nm_device_get_connectivity:
* @device: a #NMDevice
* @addr_family: network address family
*
* The connectivity state of the device for given address family.
* Supported address families are %AF_INET for IPv4, %AF_INET6
* for IPv6 or %AF_UNSPEC for any.
*
* Returns: the current connectivity state
*
* Since: 1.16
**/
NMConnectivityState
nm_device_get_connectivity (NMDevice *device, int addr_family)
{
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (device);
switch (addr_family) {
case AF_INET:
return priv->ip4_connectivity;
case AF_INET6:
return priv->ip6_connectivity;
case AF_UNSPEC:
return NM_MAX (priv->ip4_connectivity, priv->ip6_connectivity);
default:
g_return_val_if_reached (NM_CONNECTIVITY_UNKNOWN);
}
}
libnm: export interface flags Add libnm support for the new InterfaceFlags property of NMDevice.
2019-10-10 10:04:06 +02:00
/**
* nm_device_get_interface_flags:
* @device: a #NMDevice
*
* Gets the interface flags of the device.
*
* Returns: the flags
*
* Since: 1.22
**/
NMDeviceInterfaceFlags
nm_device_get_interface_flags (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NM_DEVICE_INTERFACE_FLAG_NONE);
return NM_DEVICE_GET_PRIVATE (device)->interface_flags;
}
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* nm_device_get_state:
* @device: a #NMDevice
*
* Gets the current #NMDevice state.
*
* Returns: the current device state
**/
NMDeviceState
nm_device_get_state (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NM_DEVICE_STATE_UNKNOWN);
return NM_DEVICE_GET_PRIVATE (device)->state;
}
/**
* nm_device_get_state_reason:
* @device: a #NMDevice
*
libnm: simplify NMDevice:state-reason The NMDevice:state-reason property was awkward to work with since it was a tuple of two values (state and reason), and likewise, although we had nm_device_get_state() to return just the state, there was no corresponding function to get just the reason. Fix this; make the state-reason property contain just the NMDeviceStateReason, and make nm_device_get_state_reason() return just that.
2014-08-25 10:51:26 -04:00
* Gets the reason for entering the current #NMDevice state.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: simplify NMDevice:state-reason The NMDevice:state-reason property was awkward to work with since it was a tuple of two values (state and reason), and likewise, although we had nm_device_get_state() to return just the state, there was no corresponding function to get just the reason. Fix this; make the state-reason property contain just the NMDeviceStateReason, and make nm_device_get_state_reason() return just that.
2014-08-25 10:51:26 -04:00
* Returns: the reason for entering the current device state
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: simplify NMDevice:state-reason The NMDevice:state-reason property was awkward to work with since it was a tuple of two values (state and reason), and likewise, although we had nm_device_get_state() to return just the state, there was no corresponding function to get just the reason. Fix this; make the state-reason property contain just the NMDeviceStateReason, and make nm_device_get_state_reason() return just that.
2014-08-25 10:51:26 -04:00
NMDeviceStateReason
nm_device_get_state_reason (NMDevice *device)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
{
libnm: simplify NMDevice:state-reason The NMDevice:state-reason property was awkward to work with since it was a tuple of two values (state and reason), and likewise, although we had nm_device_get_state() to return just the state, there was no corresponding function to get just the reason. Fix this; make the state-reason property contain just the NMDeviceStateReason, and make nm_device_get_state_reason() return just that.
2014-08-25 10:51:26 -04:00
g_return_val_if_fail (NM_IS_DEVICE (device), NM_DEVICE_STATE_REASON_UNKNOWN);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return NM_DEVICE_GET_PRIVATE (device)->state_reason;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_active_connection:
* @device: a #NMDevice
*
* Gets the #NMActiveConnection object which owns this device during activation.
*
* Returns: (transfer none): the #NMActiveConnection or %NULL if the device is
* not part of an active connection
**/
NMActiveConnection *
nm_device_get_active_connection (NMDevice *device)
{
libnm: add assertion for object returned by nm_device_get_active_connection() I have a coredump that seems to indicate that nm_device_get_active_connection() did not return a valid object. Let's add an assertion, trying to identify the issue earlier. Aside from that, this change isn't useful, but an nm_assert() shouldn't hurt anyway.
2019-11-28 12:40:55 +01:00
NMActiveConnection *ac;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: add assertion for object returned by nm_device_get_active_connection() I have a coredump that seems to indicate that nm_device_get_active_connection() did not return a valid object. Let's add an assertion, trying to identify the issue earlier. Aside from that, this change isn't useful, but an nm_assert() shouldn't hurt anyway.
2019-11-28 12:40:55 +01:00
ac = nml_dbus_property_o_get_obj (&NM_DEVICE_GET_PRIVATE (device)->property_o[PROPERTY_O_IDX_ACTIVE_CONNECTION]);
nm_assert (!ac || NM_IS_ACTIVE_CONNECTION (ac));
return ac;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_available_connections:
* @device: a #NMDevice
*
* Gets the #NMRemoteConnections currently known to the daemon that could
* be activated on @device.
*
* Returns: (element-type NMRemoteConnection): the #GPtrArray
* containing #NMRemoteConnections. This is the internal copy used by
* the connection, and must not be modified.
**/
const GPtrArray *
nm_device_get_available_connections (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return nml_dbus_property_ao_get_objs_as_ptrarray (&NM_DEVICE_GET_PRIVATE (device)->available_connections);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
libnm/device: (trivial) move some function upwards
2018-02-23 14:52:06 +01:00
static const char *
get_type_name (NMDevice *device)
{
switch (nm_device_get_device_type (device)) {
case NM_DEVICE_TYPE_ETHERNET:
return _("Ethernet");
case NM_DEVICE_TYPE_WIFI:
return _("Wi-Fi");
case NM_DEVICE_TYPE_BT:
return _("Bluetooth");
case NM_DEVICE_TYPE_OLPC_MESH:
return _("OLPC Mesh");
case NM_DEVICE_TYPE_OVS_INTERFACE:
contrib/rpm: Use Open vSwitch instead of OpenVSwitch The correct naming is Open vSwitch so use it instead of OpenVSwitch [lkundrak@v3.sk: added some more cases of the same] https://github.com/NetworkManager/NetworkManager/pull/150 Fixes: 830a5a14cb29ca00b73a9623c1ea7c5cd92f4d00
2018-06-27 16:53:05 +02:00
return _("Open vSwitch Interface");
libnm/device: (trivial) move some function upwards
2018-02-23 14:52:06 +01:00
case NM_DEVICE_TYPE_OVS_PORT:
contrib/rpm: Use Open vSwitch instead of OpenVSwitch The correct naming is Open vSwitch so use it instead of OpenVSwitch [lkundrak@v3.sk: added some more cases of the same] https://github.com/NetworkManager/NetworkManager/pull/150 Fixes: 830a5a14cb29ca00b73a9623c1ea7c5cd92f4d00
2018-06-27 16:53:05 +02:00
return _("Open vSwitch Port");
libnm/device: (trivial) move some function upwards
2018-02-23 14:52:06 +01:00
case NM_DEVICE_TYPE_OVS_BRIDGE:
contrib/rpm: Use Open vSwitch instead of OpenVSwitch The correct naming is Open vSwitch so use it instead of OpenVSwitch [lkundrak@v3.sk: added some more cases of the same] https://github.com/NetworkManager/NetworkManager/pull/150 Fixes: 830a5a14cb29ca00b73a9623c1ea7c5cd92f4d00
2018-06-27 16:53:05 +02:00
return _("Open vSwitch Bridge");
libnm/device: (trivial) move some function upwards
2018-02-23 14:52:06 +01:00
case NM_DEVICE_TYPE_WIMAX:
return _("WiMAX");
case NM_DEVICE_TYPE_MODEM:
return _("Mobile Broadband");
case NM_DEVICE_TYPE_INFINIBAND:
return _("InfiniBand");
case NM_DEVICE_TYPE_BOND:
return _("Bond");
case NM_DEVICE_TYPE_TEAM:
return _("Team");
case NM_DEVICE_TYPE_BRIDGE:
return _("Bridge");
case NM_DEVICE_TYPE_VLAN:
return _("VLAN");
case NM_DEVICE_TYPE_ADSL:
return _("ADSL");
case NM_DEVICE_TYPE_MACVLAN:
return _("MACVLAN");
case NM_DEVICE_TYPE_VXLAN:
return _("VXLAN");
case NM_DEVICE_TYPE_IP_TUNNEL:
return _("IPTunnel");
case NM_DEVICE_TYPE_TUN:
return _("Tun");
case NM_DEVICE_TYPE_VETH:
return _("Veth");
case NM_DEVICE_TYPE_MACSEC:
return _("MACsec");
case NM_DEVICE_TYPE_DUMMY:
return _("Dummy");
case NM_DEVICE_TYPE_PPP:
return _("PPP");
devices: add NMDeviceWpan
2018-03-09 16:26:25 +01:00
case NM_DEVICE_TYPE_WPAN:
return _("IEEE 802.15.4");
devices: add NMDevice6Lowpan
2018-05-22 16:25:54 +02:00
case NM_DEVICE_TYPE_6LOWPAN:
return _("6LoWPAN");
core: introduce NMDeviceWireGuard For now, the device only exposes partial link status (not including peers). It cannot create new links.
2018-03-13 13:42:38 +00:00
case NM_DEVICE_TYPE_WIREGUARD:
return _("WireGuard");
core,wifi-p2p: Fix Wi-Fi P2P device type The device type was set to the GType rather than a new value in the NMDeviceType enum. Add the corresponding enum entry, fix the device type and set the routing priority to the same value as generic devices. (cherry picked from commit 8d9365a973025976a15e5e7adb26a6f791957b7c)
2019-03-07 21:20:28 +01:00
case NM_DEVICE_TYPE_WIFI_P2P:
return _("Wi-Fi P2P");
core,libnm: add VRF support Add VRF support to the daemon. When the device we are activating is a VRF or a VRF's slave, put routes in the table specified by the VRF connection. Also, introduce a VRF device type in libnm.
2019-12-05 10:36:54 +01:00
case NM_DEVICE_TYPE_VRF:
return _("VRF");
libnm/device: (trivial) move some function upwards
2018-02-23 14:52:06 +01:00
case NM_DEVICE_TYPE_GENERIC:
case NM_DEVICE_TYPE_UNUSED1:
case NM_DEVICE_TYPE_UNUSED2:
case NM_DEVICE_TYPE_UNKNOWN:
break;
}
return _("Unknown");
}
static char *
get_device_type_name_with_iface (NMDevice *device)
{
const char *type_name = get_type_name (device);
switch (nm_device_get_device_type (device)) {
case NM_DEVICE_TYPE_BOND:
case NM_DEVICE_TYPE_TEAM:
case NM_DEVICE_TYPE_BRIDGE:
case NM_DEVICE_TYPE_VLAN:
return g_strdup_printf ("%s (%s)", type_name, nm_device_get_iface (device));
default:
return g_strdup (type_name);
}
}
static char *
get_device_generic_type_name_with_iface (NMDevice *device)
{
switch (nm_device_get_device_type (device)) {
case NM_DEVICE_TYPE_ETHERNET:
case NM_DEVICE_TYPE_INFINIBAND:
return g_strdup (_("Wired"));
default:
return get_device_type_name_with_iface (device);
}
}
static const char *
get_bus_name (NMDevice *device)
{
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (device);
struct udev_device *udevice;
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
const char *ifname;
const char *bus;
libnm/device: (trivial) move some function upwards
2018-02-23 14:52:06 +01:00
if (priv->bus_name)
goto out;
if (!priv->udev)
return NULL;
ifname = nm_device_get_iface (device);
if (!ifname)
return NULL;
udevice = udev_device_new_from_subsystem_sysname (priv->udev, "net", ifname);
if (!udevice) {
udevice = udev_device_new_from_subsystem_sysname (priv->udev, "tty", ifname);
if (!udevice)
return NULL;
}
bus = udev_device_get_property_value (udevice, "ID_BUS");
if (!g_strcmp0 (bus, "pci"))
priv->bus_name = g_strdup (_("PCI"));
else if (!g_strcmp0 (bus, "usb"))
priv->bus_name = g_strdup (_("USB"));
else {
/* Use "" instead of NULL so we can tell later that we've
* already tried.
*/
priv->bus_name = g_strdup ("");
}
udev_device_unref (udevice);
out:
if (*priv->bus_name)
return priv->bus_name;
else
return NULL;
}
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
static char *
_get_udev_property (NMDevice *device,
const char *enc_prop, /* ID_XXX_ENC */
const char *db_prop) /* ID_XXX_FROM_DATABASE */
{
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (device);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
struct udev_device *udev_device;
struct udev_device *tmpdev;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
const char *ifname;
guint32 count = 0;
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
char *enc_value = NULL;
char *db_value = NULL;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm/udev: cache and reuse udev instance for NMDevice No need to create a separate NMUdevClient instance for all devices. Instead, have one "struct udev" instance in NMClient and pass it down during object construction.
2017-03-20 12:27:17 +01:00
if (!priv->udev)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return NULL;
ifname = nm_device_get_iface (device);
if (!ifname)
return NULL;
libnm/udev: cache and reuse udev instance for NMDevice No need to create a separate NMUdevClient instance for all devices. Instead, have one "struct udev" instance in NMClient and pass it down during object construction.
2017-03-20 12:27:17 +01:00
udev_device = udev_device_new_from_subsystem_sysname (priv->udev, "net", ifname);
udev: drop libgudev in favor of libudev libgudev is just a wrapper around libudev. We can use libudev directly and drop the dependency for libgudev.
2017-03-12 15:54:02 +01:00
if (!udev_device) {
libnm/udev: cache and reuse udev instance for NMDevice No need to create a separate NMUdevClient instance for all devices. Instead, have one "struct udev" instance in NMClient and pass it down during object construction.
2017-03-20 12:27:17 +01:00
udev_device = udev_device_new_from_subsystem_sysname (priv->udev, "tty", ifname);
udev: drop libgudev in favor of libudev libgudev is just a wrapper around libudev. We can use libudev directly and drop the dependency for libgudev.
2017-03-12 15:54:02 +01:00
if (!udev_device)
return NULL;
}
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/* Walk up the chain of the device and its parents a few steps to grab
* vendor and device ID information off it.
*/
udev: drop libgudev in favor of libudev libgudev is just a wrapper around libudev. We can use libudev directly and drop the dependency for libgudev.
2017-03-12 15:54:02 +01:00
tmpdev = udev_device;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
while ((count++ < 3) && tmpdev && !enc_value) {
if (!enc_value)
udev: add and use nm_udev_utils_property_decode() function DRY.
2017-03-20 16:04:29 +01:00
enc_value = nm_udev_utils_property_decode_cp (udev_device_get_property_value (tmpdev, enc_prop));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
if (!db_value)
udev: drop libgudev in favor of libudev libgudev is just a wrapper around libudev. We can use libudev directly and drop the dependency for libgudev.
2017-03-12 15:54:02 +01:00
db_value = g_strdup (udev_device_get_property_value (tmpdev, db_prop));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
udev: drop libgudev in favor of libudev libgudev is just a wrapper around libudev. We can use libudev directly and drop the dependency for libgudev.
2017-03-12 15:54:02 +01:00
tmpdev = udev_device_get_parent (tmpdev);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
udev: drop libgudev in favor of libudev libgudev is just a wrapper around libudev. We can use libudev directly and drop the dependency for libgudev.
2017-03-12 15:54:02 +01:00
udev_device_unref (udev_device);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm/device: prefer FROM_DATABASE udev properties The database has a chance to fix up over garbage strings from the device properties and we're now known to be reasonably good at fixing up crap there.
2018-02-23 14:52:05 +01:00
/* Prefer the hwdata database value over what comes directly
* from the device. */
if (db_value) {
g_free (enc_value);
return db_value;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
libnm/device: prefer FROM_DATABASE udev properties The database has a chance to fix up over garbage strings from the device properties and we're now known to be reasonably good at fixing up crap there.
2018-02-23 14:52:05 +01:00
return enc_value;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
static char *
_get_udev_property_utf8safe (NMDevice *device,
const char *enc_prop, /* ID_XXX_ENC */
const char *db_prop) /* ID_XXX_FROM_DATABASE */
{
return nm_utils_str_utf8safe_escape_take (_get_udev_property (device,
enc_prop,
db_prop),
NM_UTILS_STR_UTF8_SAFE_FLAG_ESCAPE_CTRL);
}
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* nm_device_get_product:
* @device: a #NMDevice
*
* Gets the product string of the #NMDevice.
*
* Returns: the product name of the device. This is the internal string used by the
* device, and must not be modified.
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
*
* The string is backslash escaped (C escaping) for invalid characters. The escaping
* can be reverted with g_strcompress(), however the result may not be valid UTF-8.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
const char *
nm_device_get_product (NMDevice *device)
{
NMDevicePrivate *priv;
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
priv = NM_DEVICE_GET_PRIVATE (device);
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
if (!priv->product) {
priv->product = _get_udev_property_utf8safe (device, "ID_MODEL_ENC", "ID_MODEL_FROM_DATABASE");
libnm,device: don't notify of property change when getting vendor & product from udev I have no idea what was the purpose, however this causes an infinite loop if udev has not product & vendor and the notify handler gets the property.
2015-03-26 17:52:04 +01:00
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
/* Sometimes ID_PRODUCT_FROM_DATABASE is used? */
if (!priv->product)
priv->product = _get_udev_property_utf8safe (device, "ID_MODEL_ENC", "ID_PRODUCT_FROM_DATABASE");
libnm,device: don't notify of property change when getting vendor & product from udev I have no idea what was the purpose, however this causes an infinite loop if udev has not product & vendor and the notify handler gets the property.
2015-03-26 17:52:04 +01:00
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
if (!priv->product)
priv->product = g_strdup ("");
}
libnm,device: don't notify of property change when getting vendor & product from udev I have no idea what was the purpose, however this causes an infinite loop if udev has not product & vendor and the notify handler gets the property.
2015-03-26 17:52:04 +01:00
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return priv->product;
}
/**
* nm_device_get_vendor:
* @device: a #NMDevice
*
* Gets the vendor string of the #NMDevice.
*
* Returns: the vendor name of the device. This is the internal string used by the
* device, and must not be modified.
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
*
* The string is backslash escaped (C escaping) for invalid characters. The escaping
* can be reverted with g_strcompress(), however the result may not be valid UTF-8.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
const char *
nm_device_get_vendor (NMDevice *device)
{
NMDevicePrivate *priv;
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
priv = NM_DEVICE_GET_PRIVATE (device);
libnm,device: don't notify of property change when getting vendor & product from udev I have no idea what was the purpose, however this causes an infinite loop if udev has not product & vendor and the notify handler gets the property.
2015-03-26 17:52:04 +01:00
if (!priv->vendor)
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
priv->vendor = _get_udev_property_utf8safe (device, "ID_VENDOR_ENC", "ID_VENDOR_FROM_DATABASE");
libnm,device: don't notify of property change when getting vendor & product from udev I have no idea what was the purpose, however this causes an infinite loop if udev has not product & vendor and the notify handler gets the property.
2015-03-26 17:52:04 +01:00
if (!priv->vendor)
priv->vendor = g_strdup ("");
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return priv->vendor;
}
static void
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
ensure_description (NMDevice *device)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
{
NMDevicePrivate *priv = NM_DEVICE_GET_PRIVATE (device);
libnm/libnm-glib: use Bluetooth device name as description (bgo #592819) Abuse the 'name' property for this, for now, so we don't have to grab a free slot from NMDeviceClass. https://bugzilla.gnome.org/show_bug.cgi?id=592819
2016-06-15 16:40:49 -05:00
GParamSpec *name_prop;
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
gs_free char *short_product = NULL;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm/utils: split out vendor fixup The hwdb generally contains the strings of rather poor quality, especially when it comes to sensibly presenting them to the user and they need various cleanups. While the following patches add fixups, this one splits out vendor fixups, because it turns out that a different set of fixups is needed than for products.
2018-02-23 14:52:00 +01:00
priv->short_vendor = nm_str_realloc (nm_utils_fixup_vendor_string (nm_device_get_vendor (device)));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm/libnm-glib: use Bluetooth device name as description (bgo #592819) Abuse the 'name' property for this, for now, so we don't have to grab a free slot from NMDeviceClass. https://bugzilla.gnome.org/show_bug.cgi?id=592819
2016-06-15 16:40:49 -05:00
/* Grab device's preferred name, if any */
name_prop = g_object_class_find_property (G_OBJECT_GET_CLASS (G_OBJECT (device)), "name");
if (name_prop) {
g_object_get (device, "name", &priv->description, NULL);
if (priv->description && priv->description[0])
return;
all: use nm_clear_g_free() instead of g_clear_pointer() I think it's preferable to use nm_clear_g_free() instead of g_clear_pointer(, g_free). The reasons are not very strong, but I think it is overall preferable to have a shorthand for this frequently used functionality. sed 's/\<g_clear_pointer *(\([^;]*\), *\(g_free\) *)/nm_clear_g_free (\1)/g' $(git grep -l g_clear_pointer) -i
2020-03-23 11:00:43 +01:00
nm_clear_g_free (&priv->description);
libnm/libnm-glib: use Bluetooth device name as description (bgo #592819) Abuse the 'name' property for this, for now, so we don't have to grab a free slot from NMDeviceClass. https://bugzilla.gnome.org/show_bug.cgi?id=592819
2016-06-15 16:40:49 -05:00
}
libnm/device: use the type name for description if the product is empty It's very likely that the product said something that was filtered out by the fixup, such as "PCI Ethernet" or "Wi-Fi Adapter". Use a generic type name in place of it.
2018-02-23 14:52:06 +01:00
if (!priv->short_vendor) {
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
priv->description = g_strdup (nm_device_get_iface (device) ?: "");
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return;
}
libnm/device: use the type name for description if the product is empty It's very likely that the product said something that was filtered out by the fixup, such as "PCI Ethernet" or "Wi-Fi Adapter". Use a generic type name in place of it.
2018-02-23 14:52:06 +01:00
short_product = nm_utils_fixup_product_string (nm_device_get_product (device));
if (short_product == NULL)
short_product = g_strdup (get_type_name (device));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/* Another quick hack; if all of the fixed up vendor string
* is found in product, ignore the vendor.
*/
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
{
gs_free char *pdown = g_ascii_strdown (short_product, -1);
gs_free char *vdown = g_ascii_strdown (priv->short_vendor, -1);
if (!strstr (pdown, vdown))
priv->description = g_strconcat (priv->short_vendor, " ", short_product, NULL);
else
priv->description = g_steal_pointer (&short_product);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
}
static const char *
get_short_vendor (NMDevice *device)
{
NMDevicePrivate *priv;
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
priv = NM_DEVICE_GET_PRIVATE (device);
if (!priv->description)
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
ensure_description (device);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return priv->short_vendor;
}
/**
* nm_device_get_description:
* @device: an #NMDevice
*
libnm, docs: docs fixes Update the docs build to include and exclude the correct files. Fill in some missing documentation, and fix problems in the existing docs. (In particular, "<" can't appear as a literal in documentation, so change it to "&lt;". Also, "PKCS#12" has to be written as "PKCS#<!-- -->12", or gtk-doc will think "#12" is a reference to a type named "12".)
2014-11-07 12:38:17 -05:00
* Gets a description of @device, based on its vendor and product names.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
* Returns: a description of @device. If either the vendor or the
* product name is unknown, this returns the interface name.
*/
const char *
nm_device_get_description (NMDevice *device)
{
NMDevicePrivate *priv;
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
priv = NM_DEVICE_GET_PRIVATE (device);
if (!priv->description)
libnm: UTF-8 sanitize strings from UDev in NMDevice
2017-05-17 11:48:53 +02:00
ensure_description (device);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return priv->description;
}
static gboolean
find_duplicates (char **names,
gboolean *duplicates,
int num_devices)
{
int i, j;
gboolean found_any = FALSE;
memset (duplicates, 0, num_devices * sizeof (gboolean));
for (i = 0; i < num_devices; i++) {
if (duplicates[i])
continue;
for (j = i + 1; j < num_devices; j++) {
if (duplicates[j])
continue;
if (!strcmp (names[i], names[j]))
duplicates[i] = duplicates[j] = found_any = TRUE;
}
}
return found_any;
}
/**
* nm_device_disambiguate_names:
* @devices: (array length=num_devices): an array of #NMDevice
* @num_devices: length of @devices
*
* Generates a list of short-ish unique presentation names for the
* devices in @devices.
*
* Returns: (transfer full) (array zero-terminated=1): the device names
*/
char **
nm_device_disambiguate_names (NMDevice **devices,
int num_devices)
{
char **names;
gboolean *duplicates;
int i;
names = g_new (char *, num_devices + 1);
duplicates = g_new (gboolean, num_devices);
/* Generic device name */
for (i = 0; i < num_devices; i++)
names[i] = get_device_generic_type_name_with_iface (devices[i]);
if (!find_duplicates (names, duplicates, num_devices))
goto done;
/* Try specific names (eg, "Ethernet" and "InfiniBand" rather
* than "Wired")
*/
for (i = 0; i < num_devices; i++) {
if (duplicates[i]) {
g_free (names[i]);
names[i] = get_device_type_name_with_iface (devices[i]);
}
}
if (!find_duplicates (names, duplicates, num_devices))
goto done;
/* Try prefixing bus name (eg, "PCI Ethernet" vs "USB Ethernet") */
for (i = 0; i < num_devices; i++) {
if (duplicates[i]) {
const char *bus = get_bus_name (devices[i]);
char *name;
if (!bus)
continue;
g_free (names[i]);
name = get_device_type_name_with_iface (devices[i]);
all: unify spelling of translators hint in source code Use the same form everywhere: "TRANSLATORS" instead of "Translators". The manual also seems to prefer the upper-case form [1]. $ sed 's/\<Translators\>: /TRANSLATORS: /g' $(git grep -l Translators) -i [1] https://www.gnu.org/software/gettext/manual/gettext.html
2017-12-27 14:17:34 +01:00
/* TRANSLATORS: the first %s is a bus name (eg, "USB") or
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
* product name, the second is a device type (eg,
* "Ethernet"). You can change this to something like
* "%2$s (%1$s)" if there's no grammatical way to combine
* the strings otherwise.
*/
names[i] = g_strdup_printf (C_("long device name", "%s %s"),
bus, name);
g_free (name);
}
}
if (!find_duplicates (names, duplicates, num_devices))
goto done;
/* Try prefixing vendor name */
for (i = 0; i < num_devices; i++) {
if (duplicates[i]) {
const char *vendor = get_short_vendor (devices[i]);
char *name;
if (!vendor)
continue;
g_free (names[i]);
name = get_device_type_name_with_iface (devices[i]);
names[i] = g_strdup_printf (C_("long device name", "%s %s"),
vendor,
get_type_name (devices[i]));
g_free (name);
}
}
if (!find_duplicates (names, duplicates, num_devices))
goto done;
Currently if we have two paired bluetooth devices, nm_device_disambiguate_names() will disambiguate them by their hardware address. This is not very helpful, so detect this case and use the Bluetooth device name instead. This function is used by System Settings when showing the list of network devices. https://bugzilla.gnome.org/show_bug.cgi?id=740553 (Port of a libnm-gtk patch written by Ryan Lortie.)
2015-01-27 17:07:49 -05:00
/* If dealing with Bluetooth devices, try to distinguish them by
* device name.
*/
for (i = 0; i < num_devices; i++) {
if (duplicates[i] && NM_IS_DEVICE_BT (devices[i])) {
const char *devname = nm_device_bt_get_name (NM_DEVICE_BT (devices[i]));
device: free temporary typename with iface variable To create a disambiguated name for some Bluetooth devices we use its type name with iface, however this value is allocated but never free'd when passed to g_strdup_printf. So use instead a temporary variable and free it once done. https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/208 Fixes: 8bbda5cdff0f7f04d3b5863edfdba335dab4c7f8
2019-07-21 17:01:43 +02:00
char *name;
Currently if we have two paired bluetooth devices, nm_device_disambiguate_names() will disambiguate them by their hardware address. This is not very helpful, so detect this case and use the Bluetooth device name instead. This function is used by System Settings when showing the list of network devices. https://bugzilla.gnome.org/show_bug.cgi?id=740553 (Port of a libnm-gtk patch written by Ryan Lortie.)
2015-01-27 17:07:49 -05:00
if (!devname)
continue;
g_free (names[i]);
device: free temporary typename with iface variable To create a disambiguated name for some Bluetooth devices we use its type name with iface, however this value is allocated but never free'd when passed to g_strdup_printf. So use instead a temporary variable and free it once done. https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/208 Fixes: 8bbda5cdff0f7f04d3b5863edfdba335dab4c7f8
2019-07-21 17:01:43 +02:00
name = get_device_type_name_with_iface (devices[i]);
names[i] = g_strdup_printf ("%s (%s)", name, devname);
g_free (name);
Currently if we have two paired bluetooth devices, nm_device_disambiguate_names() will disambiguate them by their hardware address. This is not very helpful, so detect this case and use the Bluetooth device name instead. This function is used by System Settings when showing the list of network devices. https://bugzilla.gnome.org/show_bug.cgi?id=740553 (Port of a libnm-gtk patch written by Ryan Lortie.)
2015-01-27 17:07:49 -05:00
}
}
if (!find_duplicates (names, duplicates, num_devices))
goto done;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/* We have multiple identical network cards, so we have to differentiate
* them by interface name.
*/
for (i = 0; i < num_devices; i++) {
if (duplicates[i]) {
const char *interface = nm_device_get_iface (devices[i]);
if (!interface)
continue;
g_free (names[i]);
names[i] = g_strdup_printf ("%s (%s)",
get_type_name (devices[i]),
interface);
}
}
done:
g_free (duplicates);
names[num_devices] = NULL;
return names;
}
/**
* nm_device_get_physical_port_id:
* @device: a #NMDevice
*
* Gets the physical port ID of the #NMDevice. If non-%NULL, this is
* an opaque string that can be used to recognize when
* seemingly-unrelated #NMDevices are actually just different virtual
* ports on a single physical port. (Eg, NPAR / SR-IOV.)
*
* Returns: the physical port ID of the device, or %NULL if the port
* ID is unknown. This is the internal string used by the device and
* must not be modified.
**/
const char *
nm_device_get_physical_port_id (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: add and use _nml_coerce_property_*() Our NMObject implementations should behave in a similar manner. For example, string properties should be coerced the a consistent manner. Add functions _nml_coerce_property_*() for that. Of course, they are trivial. Their value is not that they encapsulate some complex implementation, but that they convey a general concept of how we want to handle certain properties in NMClient's object cache.
2019-10-22 16:51:09 +02:00
return _nml_coerce_property_str_not_empty (NM_DEVICE_GET_PRIVATE (device)->physical_port_id);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_mtu:
* @device: a #NMDevice
*
* Gets the MTU of the #NMDevice.
*
doc: add units to nm_device_get_mtu()
2018-05-23 16:08:01 +02:00
* Returns: the MTU of the device in bytes.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
guint32
nm_device_get_mtu (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), 0);
return NM_DEVICE_GET_PRIVATE (device)->mtu;
}
core,libnm: add 'metered' property to NMDevice
2015-04-29 16:34:38 +02:00
/**
* nm_device_get_metered:
* @device: a #NMDevice
*
* Gets the metered setting of a #NMDevice.
*
* Returns: the metered setting.
*
Revert "all: change "Since: 1.2" to "Since: 1.0.4"/"Since: 1.0.6" for backported API" API should be added with "Since:" of the next release on the same branch. That means, new API on 1.1 branch (development), should be "Since: 1.2" and new API on 1.0 branch (stable) will be "Since: 1.0.x". Similarly, new API on master is NM_AVAILABLE_IN_1_2 and will be added with the linker version libnl_1_2 -- never the versions of minor releases. It is also strongly advised that for the 1.0 branch, we only add API that was previously formerly added on master. IOW, that we only do true backports of API that already exists on master. API that gets backported, must also be added to master via NM_BACKPORT_SYMBOL(). That gives ABI compatibility and an application that was build against 1.0.x will work with 1.y.z version (y > 0) without need for recompiling -- provided that 1.y.z also contains that API. There is one important caveat: if a major branch (e.g. current master) has a linker section of backported APIs (e.g. libnm_1_0_6), we must do the minor release (1.0.6) before the next major release (1.2). The reason is that after the major release, the linker section (libnm_1_0_6) must not be extended and thus the minor release (1.0.6) must be already released at that point. In general, users should avoid using backported API because it limits the ability to upgrade to arbitrary later versions. But together with the previous point (that we only backport API to minor releases), a user that uses backported API can be sure that a 1.y.z version is ABI compatible with 1.0.x, if the 1.y.z release date was after the release date of 1.0.x. This reverts commit 02a136682c749a0fd27853c0152d36c44635151f.
2015-08-22 00:57:30 +02:00
* Since: 1.2
core,libnm: add 'metered' property to NMDevice
2015-04-29 16:34:38 +02:00
**/
NMMetered
nm_device_get_metered (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), NM_METERED_UNKNOWN);
return NM_DEVICE_GET_PRIVATE (device)->metered;
}
libnm: backport symbols for metered connections support to 1.0.6 Backport to 1.0.6 the following symbols: - nm_device_get_metered - nm_metered_get_type - nm_setting_connection_get_metered
2015-07-21 19:02:12 +02:00
NM_BACKPORT_SYMBOL (libnm_1_0_6, NMMetered, nm_device_get_metered, (NMDevice *device), (device));
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
/**
* nm_device_get_lldp_neighbors:
* @device: a #NMDevice
*
* Gets the list of neighbors discovered through LLDP.
*
* Returns: (element-type NMLldpNeighbor) (transfer none): the #GPtrArray
* containing #NMLldpNeighbor<!-- -->s. This is the internal copy used by the
libnm: document that some return values are immutable Document that the GPtrArray returned by - nm_device_get_lldp_neighbors() - nm_ip_config_get_addresses() - nm_ip_config_get_routes() is immutable and can be used by callers without the need of a copy.
2015-10-09 14:51:09 +02:00
* device and must not be modified. The library never modifies the returned
* array and thus it is safe for callers to reference and keep using it.
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
*
* Since: 1.2
**/
GPtrArray *
nm_device_get_lldp_neighbors (NMDevice *device)
{
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
NMDevicePrivate *priv;
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
priv = NM_DEVICE_GET_PRIVATE (device);
if (!priv->lldp_neighbors)
priv->lldp_neighbors = g_ptr_array_new_with_free_func ((GDestroyNotify) nm_lldp_neighbor_unref);
return priv->lldp_neighbors;
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
}
libnm-glib/libnm: add support for "real" NMDevice property
2014-10-06 11:34:02 -05:00
/**
* nm_device_is_real:
* @device: a #NMDevice
*
* Returns: %TRUE if the device exists, or %FALSE if it is a placeholder device
* that could be automatically created by NetworkManager if one of its
* #NMDevice:available-connections was activated.
*
* Since: 1.2
**/
gboolean
nm_device_is_real (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), FALSE);
return NM_DEVICE_GET_PRIVATE (device)->real;
}
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* nm_device_is_software:
* @device: a #NMDevice
*
* Whether the device is a software device.
*
* Returns: %TRUE if @device is a software device, %FALSE if it is a hardware device.
**/
gboolean
nm_device_is_software (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), FALSE);
return !!(NM_DEVICE_GET_PRIVATE (device)->capabilities & NM_DEVICE_CAP_IS_SOFTWARE);
}
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
/**
* nm_device_reapply:
* @device: a #NMDevice
libnm: add (allow-none) annotations to nm_device_reapply() The @connection argument can be NULL; add the (allow-none) annotation otherwise calling the API with a NULL argument through GObject introspection fails with: Argument 1 does not allow None as a value Fixes: 278fd4fb0fde3f290e366dab91fb6a49f9ff186c
2018-07-04 13:56:25 +02:00
* @connection: (allow-none): the #NMConnection to replace the applied
* settings with or %NULL to reuse existing
* @version_id: zero or the expected version id of the applied connection.
* If specified and the version id mismatches, the call fails without
* modification. This allows to catch concurrent accesses.
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
* @flags: always set this to zero
* @cancellable: a #GCancellable, or %NULL
* @error: location for a #GError, or %NULL
*
* Attempts to update device with changes to the currently active connection
* made since it was last applied.
*
* Returns: %TRUE on success, %FALSE on error, in which case @error will be set.
libnm: add versioning comments and macros to nm_device_reapply*() Fixes: 278fd4fb0fde3f290e366dab91fb6a49f9ff186c
2016-01-11 09:36:47 +01:00
*
* Since: 1.2
libnm: deprecate synchronous/blocking API in libnm Note that D-Bus is fundamentally asynchronous. Doing blocking calls on top of D-Bus is odd, especially for libnm's NMClient. That is because NMClient essentially is a client-side cache of the objects from the D-Bus interface. This cache should be filled exclusively by (asynchronous) D-Bus events (PropertiesChanged). So, making a blocking D-Bus call means to wait for a response and return it, while queuing all messages that are received in the meantime. Basically there are three ways how a synchronous API on NMClient could behave: 1) the call just calls g_dbus_connection_call_sync(). This means that libnm sends a D-Bus request via GDBusConnection, and blockingly waits for the response. All D-Bus messages that get received in the meantime are queued in the GMainContext that belongs to NMClient. That means, none of these D-Bus events are processed until we iterate the GMainContext after the call returns. The effect is, that NMClient (and all cached objects in there) are unaffected by the D-Bus request. Most of the synchronous API calls in libnm are of this kind. The problem is that the strict ordering of D-Bus events gets violated. For some API this is not an immediate problem. Take for example nm_device_wifi_request_scan(). The call merely blockingly tells NetworkManager to start scanning, but since NetworkManager's D-Bus API does not directly expose any state that tells whether we are currently scanning, this out of order processing of the D-Bus request is a small issue. The problem is more obvious for nm_client_networking_set_enabled(). After calling it, NM_CLIENT_NETWORKING_ENABLED is still unaffected and unchanged, because the PropertiesChanged signal from D-Bus is not yet processed. This means, while you make such a blocking call, NMClient's state does not change. But usually you perform the synchronous call to change some state. In this form, the blocking call is not useful, because NMClient only changes the state after iterating the GMainContext, and not after the blocking call returns. 2) like 1), but after making the blocking g_dbus_connection_call_sync(), update the NMClient cache artificially. This is what nm_manager_check_connectivity() does, to "fix" bgo#784629. This also has the problem of out-of-order events, but it kinda solves the problem of not changing the state during the blocking call. But it does so by hacking the state of the cache. I think this is really wrong because the state should only be updated from the ordered stream of D-Bus messages (PropertiesChanged signal and similar). When libnm decides to modify the state, there may be already D-Bus messages queued that affect this very state. 3) instead of calling g_dbus_connection_call_sync(), use the asynchronous g_dbus_connection_call(). If we would use a sepaate GMainContext for all D-Bus related calls, we could ensure that while we block for the response, we iterate that internal main context. This might be nice, because all events are processed in order and after the blocking call returns, the NMClient state is up to date. The are problems however: current blocking API does not do this, so it's a significant change in behavior. Also, it might be unexpected to the user that during the blocking call the entire content of NMClient's cache might change and all pointers to the cache might be invalidated. Also, of course NMClient would invoke signals for all the changes that happen. Another problem is that this would be more effort to implement and it involves a small performance overhead for all D-Bus related calls (because we have to serialize all events in an internal GMainContext first and then invoke them on the caller's context). Also, if the users wants this behavior, they could implement it themself by running libnm in their own GMainContext. Note that libnm might have bugs to make that really working, but that should be fixed instead of adding such synchrnous API behavior. Read also [1], for why blocking calls are wrong. [1] https://smcv.pseudorandom.co.uk/2008/11/nonblocking/ So, all possible behaviors for synchronous API have severe behavioural issues. Mark all this API as deprecated. Also, this serves the purpose of identifying blocking D-Bus calls in libnm. Note that "deprecated" here does not really mean that the API is going to be removed. We don't break API. The user may: - continue to use this API. It's deprecated, awkward and discouraged, but if it works, by all means use it. - use asynchronous API. That's the only sensible way to use D-Bus. If libnm lacks a certain asynchronous counterpart, it should be added. - use GDBusConnection directly. There really isn't anything wrong with D-Bus or GDBusConnection. This deprecated API is just a wrapper around g_dbus_connection_call_sync(). You may call it directly without feeling dirty. --- The only other remainging API is the synchronous GInitable call for NMClient. That is an entirely separate beast and not particularly wrong (from an API point of view). Note that synchronous API in NMSecretAgentOld, NMVpnPluginOld and NMVpnServicePlugin as not deprecated here. These types are not part of the D-Bus cache and while they have similar issues, it's less severe because they have less state.
2019-09-04 13:58:43 +02:00
*
libnm/doc: fix gtk-doc for deprecated markers in libnm
2020-03-13 19:45:09 +01:00
* Deprecated: 1.22: Use nm_device_reapply_async() or GDBusConnection.
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
**/
gboolean
nm_device_reapply (NMDevice *device,
NMConnection *connection,
all: add version-id argument to device's Reapply method This breaks API and ABI for the functions related to Reapply, which got introduced in the current 1.1 development phase. The version-id is here to allow users to error out if the connection on the device was changed by a concurrent action. https://bugzilla.gnome.org/show_bug.cgi?id=761714
2016-02-11 09:04:58 +01:00
guint64 version_id,
device: fix signature for @flags argument of impl_device_reapply() Thereby, also adjust the type for libnm's wrapper function -- as we already broke ABI.
2016-02-12 14:02:53 +01:00
guint32 flags,
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
GCancellable *cancellable,
GError **error)
{
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
GVariant *arg_connection = NULL;
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
g_return_val_if_fail (NM_IS_DEVICE (device), FALSE);
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
g_return_val_if_fail (!connection || NM_IS_CONNECTION (connection), FALSE);
g_return_val_if_fail (!cancellable || G_IS_CANCELLABLE (cancellable), FALSE);
g_return_val_if_fail (!error || !*error, FALSE);
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
if (connection)
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
arg_connection = nm_connection_to_dbus (connection, NM_CONNECTION_SERIALIZE_ALL);
if (!arg_connection)
arg_connection = g_variant_new_array (G_VARIANT_TYPE ("{sa{sv}}"), NULL, 0);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return _nm_client_dbus_call_sync_void (_nm_object_get_client (device),
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
cancellable,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_object_get_path (device),
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
NM_DBUS_INTERFACE_DEVICE,
"Reapply",
g_variant_new ("(@a{sa{sv}}tu)",
arg_connection,
version_id,
flags),
G_DBUS_CALL_FLAGS_NONE,
NM_DBUS_DEFAULT_TIMEOUT_MSEC,
TRUE,
error);
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
}
/**
* nm_device_reapply_async:
* @device: a #NMDevice
libnm: add (allow-none) annotations to nm_device_reapply() The @connection argument can be NULL; add the (allow-none) annotation otherwise calling the API with a NULL argument through GObject introspection fails with: Argument 1 does not allow None as a value Fixes: 278fd4fb0fde3f290e366dab91fb6a49f9ff186c
2018-07-04 13:56:25 +02:00
* @connection: (allow-none): the #NMConnection to replace the applied
* settings with or %NULL to reuse existing
* @version_id: zero or the expected version id of the applied
* connection. If specified and the version id mismatches, the call
* fails without modification. This allows to catch concurrent
* accesses.
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
* @flags: always set this to zero
* @cancellable: a #GCancellable, or %NULL
* @callback: callback to be called when the reapply operation completes
* @user_data: caller-specific data passed to @callback
*
* Asynchronously begins an attempt to update device with changes to the
* currently active connection made since it was last applied.
libnm: add versioning comments and macros to nm_device_reapply*() Fixes: 278fd4fb0fde3f290e366dab91fb6a49f9ff186c
2016-01-11 09:36:47 +01:00
*
* Since: 1.2
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
**/
void
nm_device_reapply_async (NMDevice *device,
NMConnection *connection,
all: add version-id argument to device's Reapply method This breaks API and ABI for the functions related to Reapply, which got introduced in the current 1.1 development phase. The version-id is here to allow users to error out if the connection on the device was changed by a concurrent action. https://bugzilla.gnome.org/show_bug.cgi?id=761714
2016-02-11 09:04:58 +01:00
guint64 version_id,
device: fix signature for @flags argument of impl_device_reapply() Thereby, also adjust the type for libnm's wrapper function -- as we already broke ABI.
2016-02-12 14:02:53 +01:00
guint32 flags,
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
GCancellable *cancellable,
GAsyncReadyCallback callback,
gpointer user_data)
{
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
GVariant *arg_connection = NULL;
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
g_return_if_fail (NM_IS_DEVICE (device));
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
g_return_if_fail (!connection || NM_IS_CONNECTION (connection));
g_return_if_fail (!cancellable || G_IS_CANCELLABLE (cancellable));
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
if (connection)
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
arg_connection = nm_connection_to_dbus (connection, NM_CONNECTION_SERIALIZE_ALL);
if (!arg_connection)
arg_connection = g_variant_new_array (G_VARIANT_TYPE ("{sa{sv}}"), NULL, 0);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_client_dbus_call (_nm_object_get_client (device),
device,
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
nm_device_reapply_async,
cancellable,
callback,
user_data,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_object_get_path (device),
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
NM_DBUS_INTERFACE_DEVICE,
"Reapply",
g_variant_new ("(@a{sa{sv}}tu)",
arg_connection,
version_id,
flags),
G_VARIANT_TYPE ("()"),
G_DBUS_CALL_FLAGS_NONE,
NM_DBUS_DEFAULT_TIMEOUT_MSEC,
nm_dbus_connection_call_finish_void_strip_dbus_error_cb);
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
}
/**
* nm_device_reapply_finish:
* @device: a #NMDevice
* @result: the result passed to the #GAsyncReadyCallback
* @error: location for a #GError, or %NULL
*
* Gets the result of a call to nm_device_reapply_async().
*
* Returns: %TRUE on success, %FALSE on error, in which case @error
* will be set.
libnm: add versioning comments and macros to nm_device_reapply*() Fixes: 278fd4fb0fde3f290e366dab91fb6a49f9ff186c
2016-01-11 09:36:47 +01:00
*
* Since: 1.2
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
**/
gboolean
nm_device_reapply_finish (NMDevice *device,
GAsyncResult *result,
GError **error)
{
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
g_return_val_if_fail (NM_IS_DEVICE (device), FALSE);
g_return_val_if_fail (nm_g_task_is_valid (result, device, nm_device_reapply_async), FALSE);
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
libnm: implement nm_device_reapply()/nm_device_reapply_async() by using GDBusConnection directly
2019-10-06 22:26:21 +02:00
return g_task_propagate_boolean (G_TASK (result), error);
libnm: add nm_device_reapply() Client support for O.FD.NM.Device.Reapply().
2016-01-05 15:43:48 +01:00
}
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
/*****************************************************************************/
/**
* nm_device_get_applied_connection:
* @device: a #NMDevice
all: fix typo in man pages There should be a comma after 'Otherwise' and 'Currently'. https://bugzilla.redhat.com/show_bug.cgi?id=1852452 https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/560
2020-07-01 17:20:40 -04:00
* @flags: the flags argument. Currently, this value must always be zero.
nm: Fix syntax on introspection annotations Various annotations were added using multiple colons, while only one has to be added or g-ir-introspect will consider them part of the description https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/94 (cherry picked from commit 73005fcf5b957a4cf7f8244da85ade0214db7606)
2019-03-06 20:04:50 +01:00
* @version_id: (out) (allow-none): returns the current version id of
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
* the applied connection
* @cancellable: a #GCancellable, or %NULL
* @error: location for a #GError, or %NULL
*
* Fetch the currently applied connection on the device.
*
libnm,core: fix syntax of 'transfer' annotation
2016-02-24 17:12:39 +01:00
* Returns: (transfer full): a %NMConnection with the currently applied settings
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
* or %NULL on error.
*
libnm: add code comments to hint that NMConnection might not validate
2016-03-23 17:36:28 +01:00
* The connection is as received from D-Bus and might not validate according
* to nm_connection_verify().
*
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
* Since: 1.2
libnm: deprecate synchronous/blocking API in libnm Note that D-Bus is fundamentally asynchronous. Doing blocking calls on top of D-Bus is odd, especially for libnm's NMClient. That is because NMClient essentially is a client-side cache of the objects from the D-Bus interface. This cache should be filled exclusively by (asynchronous) D-Bus events (PropertiesChanged). So, making a blocking D-Bus call means to wait for a response and return it, while queuing all messages that are received in the meantime. Basically there are three ways how a synchronous API on NMClient could behave: 1) the call just calls g_dbus_connection_call_sync(). This means that libnm sends a D-Bus request via GDBusConnection, and blockingly waits for the response. All D-Bus messages that get received in the meantime are queued in the GMainContext that belongs to NMClient. That means, none of these D-Bus events are processed until we iterate the GMainContext after the call returns. The effect is, that NMClient (and all cached objects in there) are unaffected by the D-Bus request. Most of the synchronous API calls in libnm are of this kind. The problem is that the strict ordering of D-Bus events gets violated. For some API this is not an immediate problem. Take for example nm_device_wifi_request_scan(). The call merely blockingly tells NetworkManager to start scanning, but since NetworkManager's D-Bus API does not directly expose any state that tells whether we are currently scanning, this out of order processing of the D-Bus request is a small issue. The problem is more obvious for nm_client_networking_set_enabled(). After calling it, NM_CLIENT_NETWORKING_ENABLED is still unaffected and unchanged, because the PropertiesChanged signal from D-Bus is not yet processed. This means, while you make such a blocking call, NMClient's state does not change. But usually you perform the synchronous call to change some state. In this form, the blocking call is not useful, because NMClient only changes the state after iterating the GMainContext, and not after the blocking call returns. 2) like 1), but after making the blocking g_dbus_connection_call_sync(), update the NMClient cache artificially. This is what nm_manager_check_connectivity() does, to "fix" bgo#784629. This also has the problem of out-of-order events, but it kinda solves the problem of not changing the state during the blocking call. But it does so by hacking the state of the cache. I think this is really wrong because the state should only be updated from the ordered stream of D-Bus messages (PropertiesChanged signal and similar). When libnm decides to modify the state, there may be already D-Bus messages queued that affect this very state. 3) instead of calling g_dbus_connection_call_sync(), use the asynchronous g_dbus_connection_call(). If we would use a sepaate GMainContext for all D-Bus related calls, we could ensure that while we block for the response, we iterate that internal main context. This might be nice, because all events are processed in order and after the blocking call returns, the NMClient state is up to date. The are problems however: current blocking API does not do this, so it's a significant change in behavior. Also, it might be unexpected to the user that during the blocking call the entire content of NMClient's cache might change and all pointers to the cache might be invalidated. Also, of course NMClient would invoke signals for all the changes that happen. Another problem is that this would be more effort to implement and it involves a small performance overhead for all D-Bus related calls (because we have to serialize all events in an internal GMainContext first and then invoke them on the caller's context). Also, if the users wants this behavior, they could implement it themself by running libnm in their own GMainContext. Note that libnm might have bugs to make that really working, but that should be fixed instead of adding such synchrnous API behavior. Read also [1], for why blocking calls are wrong. [1] https://smcv.pseudorandom.co.uk/2008/11/nonblocking/ So, all possible behaviors for synchronous API have severe behavioural issues. Mark all this API as deprecated. Also, this serves the purpose of identifying blocking D-Bus calls in libnm. Note that "deprecated" here does not really mean that the API is going to be removed. We don't break API. The user may: - continue to use this API. It's deprecated, awkward and discouraged, but if it works, by all means use it. - use asynchronous API. That's the only sensible way to use D-Bus. If libnm lacks a certain asynchronous counterpart, it should be added. - use GDBusConnection directly. There really isn't anything wrong with D-Bus or GDBusConnection. This deprecated API is just a wrapper around g_dbus_connection_call_sync(). You may call it directly without feeling dirty. --- The only other remainging API is the synchronous GInitable call for NMClient. That is an entirely separate beast and not particularly wrong (from an API point of view). Note that synchronous API in NMSecretAgentOld, NMVpnPluginOld and NMVpnServicePlugin as not deprecated here. These types are not part of the D-Bus cache and while they have similar issues, it's less severe because they have less state.
2019-09-04 13:58:43 +02:00
*
libnm/doc: fix gtk-doc for deprecated markers in libnm
2020-03-13 19:45:09 +01:00
* Deprecated: 1.22: Use nm_device_get_applied_connection_async() or GDBusConnection.
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
**/
NMConnection *
nm_device_get_applied_connection (NMDevice *device,
guint32 flags,
guint64 *version_id,
GCancellable *cancellable,
GError **error)
{
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
gs_unref_variant GVariant *ret = NULL;
gs_unref_variant GVariant *v_connection = NULL;
guint64 v_version_id;
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
NMConnection *connection;
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
g_return_val_if_fail (!cancellable || G_IS_CANCELLABLE (cancellable), NULL);
g_return_val_if_fail (!error || !*error, NULL);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
ret = _nm_client_dbus_call_sync (_nm_object_get_client (device),
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
cancellable,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_object_get_path (device),
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
NM_DBUS_INTERFACE_DEVICE,
"GetAppliedConnection",
g_variant_new ("(u)", flags),
G_VARIANT_TYPE ("(a{sa{sv}}t)"),
G_DBUS_CALL_FLAGS_NONE,
NM_DBUS_DEFAULT_TIMEOUT_MSEC,
TRUE,
error);
if (!ret)
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
return NULL;
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
g_variant_get (ret,
"(@a{sa{sv}}t)",
&v_connection,
&v_version_id);
connection = _nm_simple_connection_new_from_dbus (v_connection, NM_SETTING_PARSE_FLAGS_BEST_EFFORT, error);
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
if (!connection)
return NULL;
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
NM_SET_OUT (version_id, v_version_id);
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
return connection;
}
/**
* nm_device_get_applied_connection_async:
* @device: a #NMDevice
all: fix typo in man pages There should be a comma after 'Otherwise' and 'Currently'. https://bugzilla.redhat.com/show_bug.cgi?id=1852452 https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/560
2020-07-01 17:20:40 -04:00
* @flags: the flags argument. Currently, this value must always be zero.
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
* @cancellable: a #GCancellable, or %NULL
* @callback: callback to be called when the reapply operation completes
* @user_data: caller-specific data passed to @callback
*
all: fix typos in documentation, translated strings and comments https://bugzilla.gnome.org/show_bug.cgi?id=783173
2017-05-28 17:34:31 +03:00
* Asynchronously begins and gets the currently applied connection.
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
*
* Since: 1.2
**/
void
nm_device_get_applied_connection_async (NMDevice *device,
guint32 flags,
GCancellable *cancellable,
GAsyncReadyCallback callback,
gpointer user_data)
{
g_return_if_fail (NM_IS_DEVICE (device));
g_return_if_fail (!cancellable || G_IS_CANCELLABLE (cancellable));
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_client_dbus_call (_nm_object_get_client (device),
device,
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
nm_device_get_applied_connection_async,
cancellable,
callback,
user_data,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_object_get_path (device),
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
NM_DBUS_INTERFACE_DEVICE,
"GetAppliedConnection",
g_variant_new ("(u)", flags),
G_VARIANT_TYPE ("(a{sa{sv}}t)"),
G_DBUS_CALL_FLAGS_NONE,
NM_DBUS_DEFAULT_TIMEOUT_MSEC,
nm_dbus_connection_call_finish_variant_strip_dbus_error_cb);
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
}
/**
* nm_device_get_applied_connection_finish:
* @device: a #NMDevice
* @result: the result passed to the #GAsyncReadyCallback
nm: Fix syntax on introspection annotations Various annotations were added using multiple colons, while only one has to be added or g-ir-introspect will consider them part of the description https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/94 (cherry picked from commit 73005fcf5b957a4cf7f8244da85ade0214db7606)
2019-03-06 20:04:50 +01:00
* @version_id: (out) (allow-none): the current version id of the applied
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
* connection.
* @error: location for a #GError, or %NULL
*
* Gets the result of a call to nm_device_get_applied_connection_async().
*
libnm,core: fix syntax of 'transfer' annotation
2016-02-24 17:12:39 +01:00
* Returns: (transfer full): a currently applied %NMConnection or %NULL in case
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
* of error.
*
libnm: add code comments to hint that NMConnection might not validate
2016-03-23 17:36:28 +01:00
* The connection is as received from D-Bus and might not validate according
* to nm_connection_verify().
*
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
* Since: 1.2
**/
NMConnection *
nm_device_get_applied_connection_finish (NMDevice *device,
GAsyncResult *result,
guint64 *version_id,
GError **error)
{
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
gs_unref_variant GVariant *ret = NULL;
gs_unref_variant GVariant *v_connection = NULL;
guint64 v_version_id;
NMConnection *connection;
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
g_return_val_if_fail (NM_IS_DEVICE (device), NULL);
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
g_return_val_if_fail (nm_g_task_is_valid (result, device, nm_device_get_applied_connection_async), NULL);
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
g_return_val_if_fail (!error || !*error, NULL);
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
ret = g_task_propagate_pointer (G_TASK (result), error);
if (!ret)
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
return NULL;
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
g_variant_get (ret,
"(@a{sa{sv}}t)",
&v_connection,
&v_version_id);
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
libnm: implement nm_device_get_applied_connection()/nm_device_get_applied_connection_async() by using GDBusConnection directly
2019-10-06 22:36:04 +02:00
connection = _nm_simple_connection_new_from_dbus (v_connection, NM_SETTING_PARSE_FLAGS_BEST_EFFORT, error);
if (!connection)
return NULL;
NM_SET_OUT (version_id, v_version_id);
return connection;
libnm: add nm_device_get_applied_connection() function
2016-02-11 12:00:30 +01:00
}
/*****************************************************************************/
libnm: add some missing sync/async method variants Add the missing variant in most places in the API where previously there was either only a synchronous version or only an asynchronous version. There is not yet a synchronous nm_client_activate_connection(), nm_client_add_and_activate_connection(), or nm_remote_settings_add_connection(), because the existing async code depends on waiting for other asynchronous events, so making them run synchronously is slightly more complicated. But these APIs can be added later.
2014-09-11 16:27:13 -04:00
/**
* nm_device_disconnect:
* @device: a #NMDevice
* @cancellable: a #GCancellable, or %NULL
* @error: location for a #GError, or %NULL
*
* Disconnects the device if currently connected, and prevents the device from
* automatically connecting to networks until the next manual network connection
* request.
*
* Returns: %TRUE on success, %FALSE on error, in which case @error will be set.
libnm: deprecate synchronous/blocking API in libnm Note that D-Bus is fundamentally asynchronous. Doing blocking calls on top of D-Bus is odd, especially for libnm's NMClient. That is because NMClient essentially is a client-side cache of the objects from the D-Bus interface. This cache should be filled exclusively by (asynchronous) D-Bus events (PropertiesChanged). So, making a blocking D-Bus call means to wait for a response and return it, while queuing all messages that are received in the meantime. Basically there are three ways how a synchronous API on NMClient could behave: 1) the call just calls g_dbus_connection_call_sync(). This means that libnm sends a D-Bus request via GDBusConnection, and blockingly waits for the response. All D-Bus messages that get received in the meantime are queued in the GMainContext that belongs to NMClient. That means, none of these D-Bus events are processed until we iterate the GMainContext after the call returns. The effect is, that NMClient (and all cached objects in there) are unaffected by the D-Bus request. Most of the synchronous API calls in libnm are of this kind. The problem is that the strict ordering of D-Bus events gets violated. For some API this is not an immediate problem. Take for example nm_device_wifi_request_scan(). The call merely blockingly tells NetworkManager to start scanning, but since NetworkManager's D-Bus API does not directly expose any state that tells whether we are currently scanning, this out of order processing of the D-Bus request is a small issue. The problem is more obvious for nm_client_networking_set_enabled(). After calling it, NM_CLIENT_NETWORKING_ENABLED is still unaffected and unchanged, because the PropertiesChanged signal from D-Bus is not yet processed. This means, while you make such a blocking call, NMClient's state does not change. But usually you perform the synchronous call to change some state. In this form, the blocking call is not useful, because NMClient only changes the state after iterating the GMainContext, and not after the blocking call returns. 2) like 1), but after making the blocking g_dbus_connection_call_sync(), update the NMClient cache artificially. This is what nm_manager_check_connectivity() does, to "fix" bgo#784629. This also has the problem of out-of-order events, but it kinda solves the problem of not changing the state during the blocking call. But it does so by hacking the state of the cache. I think this is really wrong because the state should only be updated from the ordered stream of D-Bus messages (PropertiesChanged signal and similar). When libnm decides to modify the state, there may be already D-Bus messages queued that affect this very state. 3) instead of calling g_dbus_connection_call_sync(), use the asynchronous g_dbus_connection_call(). If we would use a sepaate GMainContext for all D-Bus related calls, we could ensure that while we block for the response, we iterate that internal main context. This might be nice, because all events are processed in order and after the blocking call returns, the NMClient state is up to date. The are problems however: current blocking API does not do this, so it's a significant change in behavior. Also, it might be unexpected to the user that during the blocking call the entire content of NMClient's cache might change and all pointers to the cache might be invalidated. Also, of course NMClient would invoke signals for all the changes that happen. Another problem is that this would be more effort to implement and it involves a small performance overhead for all D-Bus related calls (because we have to serialize all events in an internal GMainContext first and then invoke them on the caller's context). Also, if the users wants this behavior, they could implement it themself by running libnm in their own GMainContext. Note that libnm might have bugs to make that really working, but that should be fixed instead of adding such synchrnous API behavior. Read also [1], for why blocking calls are wrong. [1] https://smcv.pseudorandom.co.uk/2008/11/nonblocking/ So, all possible behaviors for synchronous API have severe behavioural issues. Mark all this API as deprecated. Also, this serves the purpose of identifying blocking D-Bus calls in libnm. Note that "deprecated" here does not really mean that the API is going to be removed. We don't break API. The user may: - continue to use this API. It's deprecated, awkward and discouraged, but if it works, by all means use it. - use asynchronous API. That's the only sensible way to use D-Bus. If libnm lacks a certain asynchronous counterpart, it should be added. - use GDBusConnection directly. There really isn't anything wrong with D-Bus or GDBusConnection. This deprecated API is just a wrapper around g_dbus_connection_call_sync(). You may call it directly without feeling dirty. --- The only other remainging API is the synchronous GInitable call for NMClient. That is an entirely separate beast and not particularly wrong (from an API point of view). Note that synchronous API in NMSecretAgentOld, NMVpnPluginOld and NMVpnServicePlugin as not deprecated here. These types are not part of the D-Bus cache and while they have similar issues, it's less severe because they have less state.
2019-09-04 13:58:43 +02:00
*
libnm/doc: fix gtk-doc for deprecated markers in libnm
2020-03-13 19:45:09 +01:00
* Deprecated: 1.22: Use nm_device_disconnect_async() or GDBusConnection.
libnm: add some missing sync/async method variants Add the missing variant in most places in the API where previously there was either only a synchronous version or only an asynchronous version. There is not yet a synchronous nm_client_activate_connection(), nm_client_add_and_activate_connection(), or nm_remote_settings_add_connection(), because the existing async code depends on waiting for other asynchronous events, so making them run synchronously is slightly more complicated. But these APIs can be added later.
2014-09-11 16:27:13 -04:00
**/
gboolean
nm_device_disconnect (NMDevice *device,
GCancellable *cancellable,
GError **error)
{
g_return_val_if_fail (NM_IS_DEVICE (device), FALSE);
libnm: implement nm_device_disconnect()/nm_device_disconnect_async() by using GDBusConnection directly
2019-10-06 22:44:25 +02:00
g_return_val_if_fail (!cancellable || G_IS_CANCELLABLE (cancellable), FALSE);
g_return_val_if_fail (!error || !*error, FALSE);
libnm: add some missing sync/async method variants Add the missing variant in most places in the API where previously there was either only a synchronous version or only an asynchronous version. There is not yet a synchronous nm_client_activate_connection(), nm_client_add_and_activate_connection(), or nm_remote_settings_add_connection(), because the existing async code depends on waiting for other asynchronous events, so making them run synchronously is slightly more complicated. But these APIs can be added later.
2014-09-11 16:27:13 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return _nm_client_dbus_call_sync_void (_nm_object_get_client (device),
libnm: implement nm_device_disconnect()/nm_device_disconnect_async() by using GDBusConnection directly
2019-10-06 22:44:25 +02:00
cancellable,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_object_get_path (device),
libnm: implement nm_device_disconnect()/nm_device_disconnect_async() by using GDBusConnection directly
2019-10-06 22:44:25 +02:00
NM_DBUS_INTERFACE_DEVICE,
"Disconnect",
g_variant_new ("()"),
G_DBUS_CALL_FLAGS_NONE,
NM_DBUS_DEFAULT_TIMEOUT_MSEC,
TRUE,
error);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
* nm_device_disconnect_async:
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
* @device: a #NMDevice
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
* @cancellable: a #GCancellable, or %NULL
* @callback: callback to be called when the disconnect operation completes
* @user_data: caller-specific data passed to @callback
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
* Asynchronously begins disconnecting the device if currently connected, and
* prevents the device from automatically connecting to networks until the next
* manual network connection request.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
void
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
nm_device_disconnect_async (NMDevice *device,
GCancellable *cancellable,
GAsyncReadyCallback callback,
gpointer user_data)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
{
g_return_if_fail (NM_IS_DEVICE (device));
libnm: implement nm_device_disconnect()/nm_device_disconnect_async() by using GDBusConnection directly
2019-10-06 22:44:25 +02:00
g_return_if_fail (!cancellable || G_IS_CANCELLABLE (cancellable));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_client_dbus_call (_nm_object_get_client (device),
device,
libnm: implement nm_device_disconnect()/nm_device_disconnect_async() by using GDBusConnection directly
2019-10-06 22:44:25 +02:00
nm_device_disconnect_async,
cancellable,
callback,
user_data,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_object_get_path (device),
libnm: implement nm_device_disconnect()/nm_device_disconnect_async() by using GDBusConnection directly
2019-10-06 22:44:25 +02:00
NM_DBUS_INTERFACE_DEVICE,
"Disconnect",
g_variant_new ("()"),
G_VARIANT_TYPE ("()"),
G_DBUS_CALL_FLAGS_NONE,
NM_DBUS_DEFAULT_TIMEOUT_MSEC,
nm_dbus_connection_call_finish_void_strip_dbus_error_cb);
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
}
/**
* nm_device_disconnect_finish:
* @device: a #NMDevice
* @result: the result passed to the #GAsyncReadyCallback
* @error: location for a #GError, or %NULL
*
* Gets the result of a call to nm_device_disconnect_async().
*
* Returns: %TRUE on success, %FALSE on error, in which case @error
* will be set.
**/
gboolean
nm_device_disconnect_finish (NMDevice *device,
GAsyncResult *result,
GError **error)
{
libnm: implement nm_device_disconnect()/nm_device_disconnect_async() by using GDBusConnection directly
2019-10-06 22:44:25 +02:00
g_return_val_if_fail (NM_IS_DEVICE (device), FALSE);
g_return_val_if_fail (nm_g_task_is_valid (result, device, nm_device_disconnect_async), FALSE);
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
libnm: implement nm_device_disconnect()/nm_device_disconnect_async() by using GDBusConnection directly
2019-10-06 22:44:25 +02:00
return g_task_propagate_boolean (G_TASK (result), error);
libnm: port to GDBus Port libnm-core/libnm to GDBus. The NetworkManager daemon continues to use dbus-glib; the previously-added connection hash/variant conversion methods are now moved to NetworkManagerUtils (along with a few other utilities that are now only needed by the daemon code).
2014-09-10 13:51:53 -04:00
}
libnm: add some missing sync/async method variants Add the missing variant in most places in the API where previously there was either only a synchronous version or only an asynchronous version. There is not yet a synchronous nm_client_activate_connection(), nm_client_add_and_activate_connection(), or nm_remote_settings_add_connection(), because the existing async code depends on waiting for other asynchronous events, so making them run synchronously is slightly more complicated. But these APIs can be added later.
2014-09-11 16:27:13 -04:00
/**
* nm_device_delete:
* @device: a #NMDevice
* @cancellable: a #GCancellable, or %NULL
* @error: location for a #GError, or %NULL
*
* Deletes the software device. Hardware devices can't be deleted.
*
* Returns: %TRUE on success, %FALSE on error, in which case @error
* will be set.
libnm: deprecate synchronous/blocking API in libnm Note that D-Bus is fundamentally asynchronous. Doing blocking calls on top of D-Bus is odd, especially for libnm's NMClient. That is because NMClient essentially is a client-side cache of the objects from the D-Bus interface. This cache should be filled exclusively by (asynchronous) D-Bus events (PropertiesChanged). So, making a blocking D-Bus call means to wait for a response and return it, while queuing all messages that are received in the meantime. Basically there are three ways how a synchronous API on NMClient could behave: 1) the call just calls g_dbus_connection_call_sync(). This means that libnm sends a D-Bus request via GDBusConnection, and blockingly waits for the response. All D-Bus messages that get received in the meantime are queued in the GMainContext that belongs to NMClient. That means, none of these D-Bus events are processed until we iterate the GMainContext after the call returns. The effect is, that NMClient (and all cached objects in there) are unaffected by the D-Bus request. Most of the synchronous API calls in libnm are of this kind. The problem is that the strict ordering of D-Bus events gets violated. For some API this is not an immediate problem. Take for example nm_device_wifi_request_scan(). The call merely blockingly tells NetworkManager to start scanning, but since NetworkManager's D-Bus API does not directly expose any state that tells whether we are currently scanning, this out of order processing of the D-Bus request is a small issue. The problem is more obvious for nm_client_networking_set_enabled(). After calling it, NM_CLIENT_NETWORKING_ENABLED is still unaffected and unchanged, because the PropertiesChanged signal from D-Bus is not yet processed. This means, while you make such a blocking call, NMClient's state does not change. But usually you perform the synchronous call to change some state. In this form, the blocking call is not useful, because NMClient only changes the state after iterating the GMainContext, and not after the blocking call returns. 2) like 1), but after making the blocking g_dbus_connection_call_sync(), update the NMClient cache artificially. This is what nm_manager_check_connectivity() does, to "fix" bgo#784629. This also has the problem of out-of-order events, but it kinda solves the problem of not changing the state during the blocking call. But it does so by hacking the state of the cache. I think this is really wrong because the state should only be updated from the ordered stream of D-Bus messages (PropertiesChanged signal and similar). When libnm decides to modify the state, there may be already D-Bus messages queued that affect this very state. 3) instead of calling g_dbus_connection_call_sync(), use the asynchronous g_dbus_connection_call(). If we would use a sepaate GMainContext for all D-Bus related calls, we could ensure that while we block for the response, we iterate that internal main context. This might be nice, because all events are processed in order and after the blocking call returns, the NMClient state is up to date. The are problems however: current blocking API does not do this, so it's a significant change in behavior. Also, it might be unexpected to the user that during the blocking call the entire content of NMClient's cache might change and all pointers to the cache might be invalidated. Also, of course NMClient would invoke signals for all the changes that happen. Another problem is that this would be more effort to implement and it involves a small performance overhead for all D-Bus related calls (because we have to serialize all events in an internal GMainContext first and then invoke them on the caller's context). Also, if the users wants this behavior, they could implement it themself by running libnm in their own GMainContext. Note that libnm might have bugs to make that really working, but that should be fixed instead of adding such synchrnous API behavior. Read also [1], for why blocking calls are wrong. [1] https://smcv.pseudorandom.co.uk/2008/11/nonblocking/ So, all possible behaviors for synchronous API have severe behavioural issues. Mark all this API as deprecated. Also, this serves the purpose of identifying blocking D-Bus calls in libnm. Note that "deprecated" here does not really mean that the API is going to be removed. We don't break API. The user may: - continue to use this API. It's deprecated, awkward and discouraged, but if it works, by all means use it. - use asynchronous API. That's the only sensible way to use D-Bus. If libnm lacks a certain asynchronous counterpart, it should be added. - use GDBusConnection directly. There really isn't anything wrong with D-Bus or GDBusConnection. This deprecated API is just a wrapper around g_dbus_connection_call_sync(). You may call it directly without feeling dirty. --- The only other remainging API is the synchronous GInitable call for NMClient. That is an entirely separate beast and not particularly wrong (from an API point of view). Note that synchronous API in NMSecretAgentOld, NMVpnPluginOld and NMVpnServicePlugin as not deprecated here. These types are not part of the D-Bus cache and while they have similar issues, it's less severe because they have less state.
2019-09-04 13:58:43 +02:00
*
libnm/doc: fix gtk-doc for deprecated markers in libnm
2020-03-13 19:45:09 +01:00
* Deprecated: 1.22: Use nm_device_delete_async() or GDBusConnection.
libnm: add some missing sync/async method variants Add the missing variant in most places in the API where previously there was either only a synchronous version or only an asynchronous version. There is not yet a synchronous nm_client_activate_connection(), nm_client_add_and_activate_connection(), or nm_remote_settings_add_connection(), because the existing async code depends on waiting for other asynchronous events, so making them run synchronously is slightly more complicated. But these APIs can be added later.
2014-09-11 16:27:13 -04:00
**/
gboolean
nm_device_delete (NMDevice *device,
GCancellable *cancellable,
GError **error)
{
g_return_val_if_fail (NM_IS_DEVICE (device), FALSE);
libnm: implement nm_device_delete()/nm_device_delete_async() by using GDBusConnection directly
2019-10-06 22:50:29 +02:00
g_return_val_if_fail (!cancellable || G_IS_CANCELLABLE (cancellable), FALSE);
g_return_val_if_fail (!error || !*error, FALSE);
libnm: add some missing sync/async method variants Add the missing variant in most places in the API where previously there was either only a synchronous version or only an asynchronous version. There is not yet a synchronous nm_client_activate_connection(), nm_client_add_and_activate_connection(), or nm_remote_settings_add_connection(), because the existing async code depends on waiting for other asynchronous events, so making them run synchronously is slightly more complicated. But these APIs can be added later.
2014-09-11 16:27:13 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return _nm_client_dbus_call_sync_void (_nm_object_get_client (device),
libnm: implement nm_device_delete()/nm_device_delete_async() by using GDBusConnection directly
2019-10-06 22:50:29 +02:00
cancellable,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_object_get_path (device),
libnm: implement nm_device_delete()/nm_device_delete_async() by using GDBusConnection directly
2019-10-06 22:50:29 +02:00
NM_DBUS_INTERFACE_DEVICE,
"Delete",
g_variant_new ("()"),
G_DBUS_CALL_FLAGS_NONE,
NM_DBUS_DEFAULT_TIMEOUT_MSEC,
TRUE,
error);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
* nm_device_delete_async:
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
* @device: a #NMDevice
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
* @cancellable: a #GCancellable, or %NULL
* @callback: callback to be called when delete operation completes
* @user_data: caller-specific data passed to @callback
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
docs: misc. typos pt2 Remainder of typos found using `codespell -q 3 --skip="./shared,./src/systemd,*.po" -I ../NetworkManager-word-whitelist.txt` whereby whitelist consists of: ``` ans busses cace cna conexant crasher iff liftime creat nd sav technik uint ``` https://github.com/NetworkManager/NetworkManager/pull/205
2018-09-15 07:20:54 -04:00
* Asynchronously begins deleting the software device. Hardware devices can't
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
* be deleted.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
void
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
nm_device_delete_async (NMDevice *device,
GCancellable *cancellable,
GAsyncReadyCallback callback,
gpointer user_data)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
{
g_return_if_fail (NM_IS_DEVICE (device));
libnm: implement nm_device_delete()/nm_device_delete_async() by using GDBusConnection directly
2019-10-06 22:50:29 +02:00
g_return_if_fail (!cancellable || G_IS_CANCELLABLE (cancellable));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_client_dbus_call (_nm_object_get_client (device),
device,
libnm: implement nm_device_delete()/nm_device_delete_async() by using GDBusConnection directly
2019-10-06 22:50:29 +02:00
nm_device_delete_async,
cancellable,
callback,
user_data,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_object_get_path (device),
libnm: implement nm_device_delete()/nm_device_delete_async() by using GDBusConnection directly
2019-10-06 22:50:29 +02:00
NM_DBUS_INTERFACE_DEVICE,
libnm: fix nm_device_delete_async() after GDBusConnection rework Fixes: c1046dd803ed ('libnm: implement nm_device_delete()/nm_device_delete_async() by using GDBusConnection directly')
2019-10-17 15:06:17 +02:00
"Delete",
libnm: implement nm_device_delete()/nm_device_delete_async() by using GDBusConnection directly
2019-10-06 22:50:29 +02:00
g_variant_new ("()"),
G_VARIANT_TYPE ("()"),
G_DBUS_CALL_FLAGS_NONE,
NM_DBUS_DEFAULT_TIMEOUT_MSEC,
nm_dbus_connection_call_finish_void_strip_dbus_error_cb);
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
}
/**
* nm_device_delete_finish:
* @device: a #NMDevice
* @result: the result passed to the #GAsyncReadyCallback
* @error: location for a #GError, or %NULL
*
* Gets the result of a call to nm_device_delete_async().
*
* Returns: %TRUE on success, %FALSE on error, in which case @error
* will be set.
**/
gboolean
nm_device_delete_finish (NMDevice *device,
GAsyncResult *result,
GError **error)
{
libnm: implement nm_device_delete()/nm_device_delete_async() by using GDBusConnection directly
2019-10-06 22:50:29 +02:00
g_return_val_if_fail (NM_IS_DEVICE (device), FALSE);
g_return_val_if_fail (nm_g_task_is_valid (result, device, nm_device_delete_async), FALSE);
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
libnm: implement nm_device_delete()/nm_device_delete_async() by using GDBusConnection directly
2019-10-06 22:50:29 +02:00
return g_task_propagate_boolean (G_TASK (result), error);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_connection_valid:
* @device: an #NMDevice to validate @connection against
* @connection: an #NMConnection to validate against @device
*
* Validates a given connection for a given #NMDevice object and returns
* whether the connection may be activated with the device. For example if
* @device is a Wi-Fi device that supports only WEP encryption, the connection
* will only be valid if it is a Wi-Fi connection which describes a WEP or open
* network, and will not be valid if it describes a WPA network, or if it is
* an Ethernet, Bluetooth, WWAN, etc connection that is incompatible with the
* device.
*
* Returns: %TRUE if the connection may be activated with this device, %FALSE
* if is incompatible with the device's capabilities and characteristics.
**/
gboolean
nm_device_connection_valid (NMDevice *device, NMConnection *connection)
{
return nm_device_connection_compatible (device, connection, NULL);
}
libnm: declare internal function as static
2016-03-17 11:23:05 +01:00
static gboolean
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
connection_compatible (NMDevice *device, NMConnection *connection, GError **error)
{
const char *config_iface, *device_iface;
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
GError *local = NULL;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
if (!nm_connection_verify (connection, &local)) {
g_set_error (error, NM_DEVICE_ERROR, NM_DEVICE_ERROR_INVALID_CONNECTION,
_("The connection was not valid: %s"), local->message);
g_error_free (local);
return FALSE;
}
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
config_iface = nm_connection_get_interface_name (connection);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
device_iface = nm_device_get_iface (device);
if (config_iface && g_strcmp0 (config_iface, device_iface) != 0) {
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
g_set_error (error, NM_DEVICE_ERROR, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION,
_("The interface names of the device and the connection didn't match."));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return FALSE;
}
return TRUE;
}
/**
* nm_device_connection_compatible:
* @device: an #NMDevice to validate @connection against
* @connection: an #NMConnection to validate against @device
* @error: return location for a #GError, or %NULL
*
* Validates a given connection for a given #NMDevice object and returns
* whether the connection may be activated with the device. For example if
* @device is a Wi-Fi device that supports only WEP encryption, the connection
* will only be valid if it is a Wi-Fi connection which describes a WEP or open
* network, and will not be valid if it describes a WPA network, or if it is
* an Ethernet, Bluetooth, WWAN, etc connection that is incompatible with the
* device.
*
* This function does the same as nm_device_connection_valid(), i.e. checking
* compatibility of the given device and connection. But, in addition, it sets
* GError when FALSE is returned.
*
* Returns: %TRUE if the connection may be activated with this device, %FALSE
* if is incompatible with the device's capabilities and characteristics.
**/
gboolean
nm_device_connection_compatible (NMDevice *device, NMConnection *connection, GError **error)
{
g_return_val_if_fail (NM_IS_DEVICE (device), FALSE);
g_return_val_if_fail (NM_IS_CONNECTION (connection), FALSE);
g_return_val_if_fail (error == NULL || *error == NULL, FALSE);
return NM_DEVICE_GET_CLASS (device)->connection_compatible (device, connection, error);
}
/**
* nm_device_filter_connections:
* @device: an #NMDevice to filter connections for
libnm: change GSList to GPtrArray in libnm methods libnm mostly used GPtrArrays in its APIs, except that arrays of connections were usually GSLists. Fix this and make them GPtrArrays too (and rename nm_client_list_connections() to nm_client_get_connections() to match everything else).
2014-10-22 12:32:46 -04:00
* @connections: (element-type NMConnection): an array of #NMConnections to filter
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: change GSList to GPtrArray in libnm methods libnm mostly used GPtrArrays in its APIs, except that arrays of connections were usually GSLists. Fix this and make them GPtrArrays too (and rename nm_client_list_connections() to nm_client_get_connections() to match everything else).
2014-10-22 12:32:46 -04:00
* Filters a given array of connections for a given #NMDevice object and returns
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
* connections which may be activated with the device. For example if @device
libnm: change GSList to GPtrArray in libnm methods libnm mostly used GPtrArrays in its APIs, except that arrays of connections were usually GSLists. Fix this and make them GPtrArrays too (and rename nm_client_list_connections() to nm_client_get_connections() to match everything else).
2014-10-22 12:32:46 -04:00
* is a Wi-Fi device that supports only WEP encryption, the returned array will
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
* contain any Wi-Fi connections in @connections that allow connection to
libnm: change GSList to GPtrArray in libnm methods libnm mostly used GPtrArrays in its APIs, except that arrays of connections were usually GSLists. Fix this and make them GPtrArrays too (and rename nm_client_list_connections() to nm_client_get_connections() to match everything else).
2014-10-22 12:32:46 -04:00
* unencrypted or WEP-enabled SSIDs. The returned array will not contain
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
* Ethernet, Bluetooth, Wi-Fi WPA connections, or any other connection that is
* incompatible with the device. To get the full list of connections see
libnm: change GSList to GPtrArray in libnm methods libnm mostly used GPtrArrays in its APIs, except that arrays of connections were usually GSLists. Fix this and make them GPtrArrays too (and rename nm_client_list_connections() to nm_client_get_connections() to match everything else).
2014-10-22 12:32:46 -04:00
* nm_client_get_connections().
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
libnm: fix GObject Introspection annotations for functions returning a GPtrArray The GPtrArray owns the element so it is a (transfer full). https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/46
2018-11-08 18:50:49 +01:00
* Returns: (transfer full) (element-type NMConnection): an array of
libnm: change GSList to GPtrArray in libnm methods libnm mostly used GPtrArrays in its APIs, except that arrays of connections were usually GSLists. Fix this and make them GPtrArrays too (and rename nm_client_list_connections() to nm_client_get_connections() to match everything else).
2014-10-22 12:32:46 -04:00
* #NMConnections that could be activated with the given @device. The array
* should be freed with g_ptr_array_unref() when it is no longer required.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
libnm: change GSList to GPtrArray in libnm methods libnm mostly used GPtrArrays in its APIs, except that arrays of connections were usually GSLists. Fix this and make them GPtrArrays too (and rename nm_client_list_connections() to nm_client_get_connections() to match everything else).
2014-10-22 12:32:46 -04:00
GPtrArray *
nm_device_filter_connections (NMDevice *device, const GPtrArray *connections)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
{
libnm: change GSList to GPtrArray in libnm methods libnm mostly used GPtrArrays in its APIs, except that arrays of connections were usually GSLists. Fix this and make them GPtrArrays too (and rename nm_client_list_connections() to nm_client_get_connections() to match everything else).
2014-10-22 12:32:46 -04:00
GPtrArray *filtered;
int i;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: change GSList to GPtrArray in libnm methods libnm mostly used GPtrArrays in its APIs, except that arrays of connections were usually GSLists. Fix this and make them GPtrArrays too (and rename nm_client_list_connections() to nm_client_get_connections() to match everything else).
2014-10-22 12:32:46 -04:00
filtered = g_ptr_array_new_with_free_func (g_object_unref);
for (i = 0; i < connections->len; i++) {
NMConnection *candidate = connections->pdata[i];
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/* Connection applies to this device */
if (nm_device_connection_valid (device, candidate))
libnm: change GSList to GPtrArray in libnm methods libnm mostly used GPtrArrays in its APIs, except that arrays of connections were usually GSLists. Fix this and make them GPtrArrays too (and rename nm_client_list_connections() to nm_client_get_connections() to match everything else).
2014-10-22 12:32:46 -04:00
g_ptr_array_add (filtered, g_object_ref (candidate));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
libnm: change GSList to GPtrArray in libnm methods libnm mostly used GPtrArrays in its APIs, except that arrays of connections were usually GSLists. Fix this and make them GPtrArrays too (and rename nm_client_list_connections() to nm_client_get_connections() to match everything else).
2014-10-22 12:32:46 -04:00
return filtered;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_get_setting_type:
* @device: an #NMDevice
*
* Gets the (primary) #NMSetting subtype associated with connections
* that can be used on @device.
*
* Returns: @device's associated #NMSetting type
*/
GType
nm_device_get_setting_type (NMDevice *device)
{
g_return_val_if_fail (NM_IS_DEVICE (device), G_TYPE_INVALID);
g_return_val_if_fail (NM_DEVICE_GET_CLASS (device)->get_setting_type != NULL, G_TYPE_INVALID);
return NM_DEVICE_GET_CLASS (device)->get_setting_type (device);
}
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
libnm/lldp: cleanup asserting for valid NMLldpNeighbor agument At a few places we checked whether neighbor->attrs was non-NULL. That is not necessary, unless we'd like to catch some dangling/invalid pointers. The attrs hash is always set otherwise. Instead of just dropping the check, add a NM_IS_LLDP_NEIGHBOR() macro (inline function).
2019-01-30 07:42:33 +01:00
/*****************************************************************************/
static gboolean
NM_IS_LLDP_NEIGHBOR (const NMLldpNeighbor *self)
{
nm_assert ( !self
|| ( self->refcount > 0
&& self->attrs));
return self
&& self->refcount > 0;
}
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
/**
libnm: define NMLldpNeighbor as boxed type GLib introspection requires all types returned by public functions to be GObjects, basic types or boxed types in order to correctly manage resources. NMLldpNeighbor was a plain struct and GI complained with: Warning: NM: nm_lldp_neighbor_new: return value: Invalid non-constant return of bare structure or union; register as boxed type or (skip) To fix this define NMLldpNeighbor as a boxed type. Fixes: d3d2b494008322a4f1452cf873393ca85166022b
2015-10-16 16:11:42 +02:00
* nm_lldp_neighbor_new:
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
*
* Creates a new #NMLldpNeighbor object.
*
* Returns: (transfer full): the new #NMLldpNeighbor object.
*
* Since: 1.2
**/
NMLldpNeighbor *
nm_lldp_neighbor_new (void)
{
NMLldpNeighbor *neigh;
neigh = g_new0 (NMLldpNeighbor, 1);
neigh->refcount = 1;
all: use nm_str_hash() instead of g_str_hash() We also do this for libnm and libnm-core, where it causes visible changes in behavior. But if somebody would rely on the hashing implementation for hash tables, it would be seriously flawed.
2017-11-15 16:06:43 +01:00
neigh->attrs = g_hash_table_new_full (nm_str_hash, g_str_equal, g_free,
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
(GDestroyNotify) g_variant_unref);
return neigh;
}
libnm: define NMLldpNeighbor as boxed type GLib introspection requires all types returned by public functions to be GObjects, basic types or boxed types in order to correctly manage resources. NMLldpNeighbor was a plain struct and GI complained with: Warning: NM: nm_lldp_neighbor_new: return value: Invalid non-constant return of bare structure or union; register as boxed type or (skip) To fix this define NMLldpNeighbor as a boxed type. Fixes: d3d2b494008322a4f1452cf873393ca85166022b
2015-10-16 16:11:42 +02:00
static NMLldpNeighbor *
nm_lldp_neighbor_dup (NMLldpNeighbor *neighbor)
{
NMLldpNeighbor *copy;
libnm/lldp: fix leak and bug in nm_lldp_neighbor_dup() For one, just reassigning copy->attrs leaks the previous hash table. Fix that. Also, NMLldpNeighbor instances are not immutable. I think that is an uglyness, and it would be preferable that they can be sealed. A sealed object could safely share/ref the internal hash-table. However, as it is, we cannot just have two NMLldpNeighbor instances share the same hash-table. Do a full copy.
2019-01-30 07:57:28 +01:00
GHashTableIter iter;
const char *key;
GVariant *value;
libnm: define NMLldpNeighbor as boxed type GLib introspection requires all types returned by public functions to be GObjects, basic types or boxed types in order to correctly manage resources. NMLldpNeighbor was a plain struct and GI complained with: Warning: NM: nm_lldp_neighbor_new: return value: Invalid non-constant return of bare structure or union; register as boxed type or (skip) To fix this define NMLldpNeighbor as a boxed type. Fixes: d3d2b494008322a4f1452cf873393ca85166022b
2015-10-16 16:11:42 +02:00
copy = nm_lldp_neighbor_new ();
libnm/lldp: fix leak and bug in nm_lldp_neighbor_dup() For one, just reassigning copy->attrs leaks the previous hash table. Fix that. Also, NMLldpNeighbor instances are not immutable. I think that is an uglyness, and it would be preferable that they can be sealed. A sealed object could safely share/ref the internal hash-table. However, as it is, we cannot just have two NMLldpNeighbor instances share the same hash-table. Do a full copy.
2019-01-30 07:57:28 +01:00
g_hash_table_iter_init (&iter, neighbor->attrs);
while (g_hash_table_iter_next (&iter, (gpointer *) &key, (gpointer *) &value))
g_hash_table_insert (copy->attrs, g_strdup (key), g_variant_ref (value));
libnm: define NMLldpNeighbor as boxed type GLib introspection requires all types returned by public functions to be GObjects, basic types or boxed types in order to correctly manage resources. NMLldpNeighbor was a plain struct and GI complained with: Warning: NM: nm_lldp_neighbor_new: return value: Invalid non-constant return of bare structure or union; register as boxed type or (skip) To fix this define NMLldpNeighbor as a boxed type. Fixes: d3d2b494008322a4f1452cf873393ca85166022b
2015-10-16 16:11:42 +02:00
return copy;
}
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
/**
libnm: define NMLldpNeighbor as boxed type GLib introspection requires all types returned by public functions to be GObjects, basic types or boxed types in order to correctly manage resources. NMLldpNeighbor was a plain struct and GI complained with: Warning: NM: nm_lldp_neighbor_new: return value: Invalid non-constant return of bare structure or union; register as boxed type or (skip) To fix this define NMLldpNeighbor as a boxed type. Fixes: d3d2b494008322a4f1452cf873393ca85166022b
2015-10-16 16:11:42 +02:00
* nm_lldp_neighbor_ref:
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
* @neighbor: the #NMLldpNeighbor
*
* Increases the reference count of the object.
*
* Since: 1.2
**/
void
nm_lldp_neighbor_ref (NMLldpNeighbor *neighbor)
{
libnm/lldp: cleanup asserting for valid NMLldpNeighbor agument At a few places we checked whether neighbor->attrs was non-NULL. That is not necessary, unless we'd like to catch some dangling/invalid pointers. The attrs hash is always set otherwise. Instead of just dropping the check, add a NM_IS_LLDP_NEIGHBOR() macro (inline function).
2019-01-30 07:42:33 +01:00
g_return_if_fail (NM_IS_LLDP_NEIGHBOR (neighbor));
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
neighbor->refcount++;
}
/**
* nm_lldp_neighbor_unref:
* @neighbor: the #NMLldpNeighbor
*
* Decreases the reference count of the object. If the reference count
* reaches zero, the object will be destroyed.
*
* Since: 1.2
**/
void
nm_lldp_neighbor_unref (NMLldpNeighbor *neighbor)
{
libnm/lldp: cleanup asserting for valid NMLldpNeighbor agument At a few places we checked whether neighbor->attrs was non-NULL. That is not necessary, unless we'd like to catch some dangling/invalid pointers. The attrs hash is always set otherwise. Instead of just dropping the check, add a NM_IS_LLDP_NEIGHBOR() macro (inline function).
2019-01-30 07:42:33 +01:00
g_return_if_fail (NM_IS_LLDP_NEIGHBOR (neighbor));
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
if (--neighbor->refcount == 0) {
g_hash_table_unref (neighbor->attrs);
g_free (neighbor);
}
}
/**
* nm_lldp_neighbor_get_attr_names:
* @neighbor: the #NMLldpNeighbor
*
* Gets an array of attribute names available for @neighbor.
*
* Returns: (transfer full): a %NULL-terminated array of attribute names.
*
* Since: 1.2
**/
char **
nm_lldp_neighbor_get_attr_names (NMLldpNeighbor *neighbor)
{
GHashTableIter iter;
const char *key;
GPtrArray *names;
libnm/lldp: cleanup asserting for valid NMLldpNeighbor agument At a few places we checked whether neighbor->attrs was non-NULL. That is not necessary, unless we'd like to catch some dangling/invalid pointers. The attrs hash is always set otherwise. Instead of just dropping the check, add a NM_IS_LLDP_NEIGHBOR() macro (inline function).
2019-01-30 07:42:33 +01:00
g_return_val_if_fail (NM_IS_LLDP_NEIGHBOR (neighbor), NULL);
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
names = g_ptr_array_new ();
g_hash_table_iter_init (&iter, neighbor->attrs);
while (g_hash_table_iter_next (&iter, (gpointer *) &key, NULL))
g_ptr_array_add (names, g_strdup (key));
g_ptr_array_add (names, NULL);
return (char **) g_ptr_array_free (names, FALSE);
}
/**
* nm_lldp_neighbor_get_attr_string_value:
* @neighbor: the #NMLldpNeighbor
* @name: the attribute name
* @out_value: (out) (allow-none) (transfer none): on return, the attribute value
*
* Gets the string value of attribute with name @name on @neighbor
*
* Returns: %TRUE if a string attribute with name @name was found, %FALSE otherwise
*
* Since: 1.2
**/
gboolean
libnm: make attribute name const in LLDP API functions Add the const qualifier to the attribute name in LLDP API functions so that const strings and string literals are accepted. This change is backwards compatible for existing users of the API.
2019-01-04 09:36:41 +01:00
nm_lldp_neighbor_get_attr_string_value (NMLldpNeighbor *neighbor, const char *name,
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
const char **out_value)
{
GVariant *variant;
libnm/lldp: cleanup asserting for valid NMLldpNeighbor agument At a few places we checked whether neighbor->attrs was non-NULL. That is not necessary, unless we'd like to catch some dangling/invalid pointers. The attrs hash is always set otherwise. Instead of just dropping the check, add a NM_IS_LLDP_NEIGHBOR() macro (inline function).
2019-01-30 07:42:33 +01:00
g_return_val_if_fail (NM_IS_LLDP_NEIGHBOR (neighbor), FALSE);
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
g_return_val_if_fail (name && name[0], FALSE);
variant = g_hash_table_lookup (neighbor->attrs, name);
if (variant && g_variant_is_of_type (variant, G_VARIANT_TYPE_STRING)) {
if (out_value)
*out_value = g_variant_get_string (variant, NULL);
return TRUE;
} else
return FALSE;
}
/**
* nm_lldp_neighbor_get_attr_uint_value:
* @neighbor: the #NMLldpNeighbor
* @name: the attribute name
libnm,libnm-core: fix GTK-Doc warnings
2015-10-16 15:20:45 +02:00
* @out_value: (out) (allow-none): on return, the attribute value
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
*
* Gets the uint value of attribute with name @name on @neighbor
*
* Returns: %TRUE if a uint attribute with name @name was found, %FALSE otherwise
*
* Since: 1.2
**/
gboolean
libnm: make attribute name const in LLDP API functions Add the const qualifier to the attribute name in LLDP API functions so that const strings and string literals are accepted. This change is backwards compatible for existing users of the API.
2019-01-04 09:36:41 +01:00
nm_lldp_neighbor_get_attr_uint_value (NMLldpNeighbor *neighbor, const char *name,
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
guint *out_value)
{
GVariant *variant;
libnm/lldp: cleanup asserting for valid NMLldpNeighbor agument At a few places we checked whether neighbor->attrs was non-NULL. That is not necessary, unless we'd like to catch some dangling/invalid pointers. The attrs hash is always set otherwise. Instead of just dropping the check, add a NM_IS_LLDP_NEIGHBOR() macro (inline function).
2019-01-30 07:42:33 +01:00
g_return_val_if_fail (NM_IS_LLDP_NEIGHBOR (neighbor), FALSE);
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
g_return_val_if_fail (name && name[0], FALSE);
variant = g_hash_table_lookup (neighbor->attrs, name);
if (variant && g_variant_is_of_type (variant, G_VARIANT_TYPE_UINT32)) {
if (out_value)
*out_value = g_variant_get_uint32 (variant);
return TRUE;
} else
return FALSE;
}
libnm: add nm_lldp_neighbor_get_attr_value() The function provides access to the GVariant representing a LLDP attribute.
2019-01-21 11:53:48 +01:00
/**
* nm_lldp_neighbor_get_attr_value:
* @neighbor: the #NMLldpNeighbor
* @name: the attribute name
*
* Gets the value (as a GVariant) of attribute with name @name on @neighbor
*
* Returns: (transfer none): the value or %NULL if the attribute with @name was
* not found.
*
* Since: 1.18
**/
GVariant *
nm_lldp_neighbor_get_attr_value (NMLldpNeighbor *neighbor, const char *name)
{
libnm/lldp: cleanup asserting for valid NMLldpNeighbor agument At a few places we checked whether neighbor->attrs was non-NULL. That is not necessary, unless we'd like to catch some dangling/invalid pointers. The attrs hash is always set otherwise. Instead of just dropping the check, add a NM_IS_LLDP_NEIGHBOR() macro (inline function).
2019-01-30 07:42:33 +01:00
g_return_val_if_fail (NM_IS_LLDP_NEIGHBOR (neighbor), FALSE);
libnm: add nm_lldp_neighbor_get_attr_value() The function provides access to the GVariant representing a LLDP attribute.
2019-01-21 11:53:48 +01:00
g_return_val_if_fail (name && name[0], FALSE);
return g_hash_table_lookup (neighbor->attrs, name);
}
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
/**
* nm_lldp_neighbor_get_attr_type:
* @neighbor: the #NMLldpNeighbor
* @name: the attribute name
*
* Get the type of an attribute.
*
* Returns: the #GVariantType of the attribute with name @name
*
* Since: 1.2
**/
const GVariantType *
libnm: make attribute name const in LLDP API functions Add the const qualifier to the attribute name in LLDP API functions so that const strings and string literals are accepted. This change is backwards compatible for existing users of the API.
2019-01-04 09:36:41 +01:00
nm_lldp_neighbor_get_attr_type (NMLldpNeighbor *neighbor, const char *name)
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
{
GVariant *variant;
libnm/lldp: cleanup asserting for valid NMLldpNeighbor agument At a few places we checked whether neighbor->attrs was non-NULL. That is not necessary, unless we'd like to catch some dangling/invalid pointers. The attrs hash is always set otherwise. Instead of just dropping the check, add a NM_IS_LLDP_NEIGHBOR() macro (inline function).
2019-01-30 07:42:33 +01:00
g_return_val_if_fail (NM_IS_LLDP_NEIGHBOR (neighbor), NULL);
libnm: add LLDP support Add functions to libnm for retrieving a list of LLDP neighbors and accessor functions to get their attributes.
2015-10-07 11:48:44 +02:00
g_return_val_if_fail (name && name[0], NULL);
variant = g_hash_table_lookup (neighbor->attrs, name);
if (variant)
return g_variant_get_type (variant);
else
return NULL;
}
Reference in a new issue Copy permalink