fdo-mirrors/NetworkManager
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/NetworkManager/NetworkManager.git synced 2025-12-23 00:20:15 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
NetworkManager/libnm/nm-device-wifi.c

743 lines
25 KiB
C
Raw Normal View History

all: SPDX header conversion $ find * -type f |xargs perl contrib/scripts/spdx.pl $ git rm contrib/scripts/spdx.pl
2019-09-10 11:19:01 +02:00
// SPDX-License-Identifier: LGPL-2.1+
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/*
all: unify format of our Copyright source code comments ```bash readarray -d '' FILES < <( git ls-files -z \ ':(exclude)po' \ ':(exclude)shared/c-rbtree' \ ':(exclude)shared/c-list' \ ':(exclude)shared/c-siphash' \ ':(exclude)shared/c-stdaux' \ ':(exclude)shared/n-acd' \ ':(exclude)shared/n-dhcp4' \ ':(exclude)src/systemd/src' \ ':(exclude)shared/systemd/src' \ ':(exclude)m4' \ ':(exclude)COPYING*' ) sed \ -e 's/^\(--\|#\| \*\) *\(([cC]) *\)\?Copyright \+\(\(([cC])\) \+\)\?\(\(20\|19\)[0-9][0-9]\) *[-–] *\(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/\1 C1pyright#\5 - \7#\9/' \ -e 's/^\(--\|#\| \*\) *\(([cC]) *\)\?Copyright \+\(\(([cC])\) \+\)\?\(\(20\|19\)[0-9][0-9]\) *[,] *\(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/\1 C2pyright#\5, \7#\9/' \ -e 's/^\(--\|#\| \*\) *\(([cC]) *\)\?Copyright \+\(\(([cC])\) \+\)\?\(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/\1 C3pyright#\5#\7/' \ -e 's/^Copyright \(\(20\|19\)[0-9][0-9]\) \+\([^ ].*\)$/C4pyright#\1#\3/' \ -i \ "${FILES[@]}" echo ">>> untouched Copyright lines" git grep Copyright "${FILES[@]}" echo ">>> Copyright lines with unusual extra" git grep '\<C[0-9]pyright#' "${FILES[@]}" | grep -i reserved sed \ -e 's/\<C[0-9]pyright#\([^#]*\)#\(.*\)$/Copyright (C) \1 \2/' \ -i \ "${FILES[@]}" ``` https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/298
2019-10-01 09:20:35 +02:00
* Copyright (C) 2007 - 2008 Novell, Inc.
* Copyright (C) 2007 - 2014 Red Hat, Inc.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*/
build: cleanup default includes - "gsystem-local-alloc.h" and <gio/gio.h> are already included via "nm-default.h". No need to include them separately. - include "nm-macros-internal.h" via "nm-default.h" and drop all explict includes. - in the modified files, ensure that we always include "config.h" and "nm-default.h" first. As second, include the header file for the current source file (if applicable). Then follow external includes and finally internal nm includes. - include nm headers inside source code files with quotes - internal header files don't need to include default headers. They can savely assume that "nm-default.h" is already included and with it glib, nm-glib.h, nm-macros-internal.h, etc.
2016-02-12 14:44:52 +01:00
#include "nm-default.h"
all: cleanup includes and let "nm-default.h" include "config.h" - All internal source files (except "examples", which are not internal) should include "config.h" first. As also all internal source files should include "nm-default.h", let "config.h" be included by "nm-default.h" and include "nm-default.h" as first in every source file. We already wanted to include "nm-default.h" before other headers because it might contains some fixes (like "nm-glib.h" compatibility) that is required first. - After including "nm-default.h", we optinally allow for including the corresponding header file for the source file at hand. The idea is to ensure that each header file is self contained. - Don't include "config.h" or "nm-default.h" in any header file (except "nm-sd-adapt.h"). Public headers anyway must not include these headers, and internal headers are never included after "nm-default.h", as of the first previous point. - Include all internal headers with quotes instead of angle brackets. In practice it doesn't matter, because in our public headers we must include other headers with angle brackets. As we use our public headers also to compile our interal source files, effectively the result must be the same. Still do it for consistency. - Except for <config.h> itself. Include it with angle brackets as suggested by https://www.gnu.org/software/autoconf/manual/autoconf.html#Configuration-Headers
2016-02-19 14:57:48 +01:00
build: cleanup default includes - "gsystem-local-alloc.h" and <gio/gio.h> are already included via "nm-default.h". No need to include them separately. - include "nm-macros-internal.h" via "nm-default.h" and drop all explict includes. - in the modified files, ensure that we always include "config.h" and "nm-default.h" first. As second, include the header file for the current source file (if applicable). Then follow external includes and finally internal nm includes. - include nm headers inside source code files with quotes - internal header files don't need to include default headers. They can savely assume that "nm-default.h" is already included and with it glib, nm-glib.h, nm-macros-internal.h, etc.
2016-02-12 14:44:52 +01:00
#include "nm-device-wifi.h"
libnm: implement nm_device_wifi_request_scan_async() by using GDBusConnection directly No longer keep track if a request is already pending. Previously, in that case, the function would return success in an idle handler. Instead, just always issue the new request. It's up to the caller to rate-limit the requests and trigger a scan whenever appropriate. libnm should not add additional logic to that.
2019-10-06 11:26:09 +02:00
#include "nm-glib-aux/nm-dbus-aux.h"
build: cleanup default includes - "gsystem-local-alloc.h" and <gio/gio.h> are already included via "nm-default.h". No need to include them separately. - include "nm-macros-internal.h" via "nm-default.h" and drop all explict includes. - in the modified files, ensure that we always include "config.h" and "nm-default.h" first. As second, include the header file for the current source file (if applicable). Then follow external includes and finally internal nm includes. - include nm headers inside source code files with quotes - internal header files don't need to include default headers. They can savely assume that "nm-default.h" is already included and with it glib, nm-glib.h, nm-macros-internal.h, etc.
2016-02-12 14:44:52 +01:00
#include "nm-setting-connection.h"
#include "nm-setting-wireless.h"
#include "nm-setting-wireless-security.h"
#include "nm-utils.h"
libnm: add nm-types.h, to avoid include loops Add nm-types.h defining all the type structs, to avoid future include loops. Clean up the includes in all of the installed headers.
2014-10-09 09:12:42 -04:00
#include "nm-access-point.h"
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
#include "nm-object-private.h"
libnm: drop NM_TYPE_OBJECT_ARRAY, use G_TYPE_PTR_ARRAY Use G_TYPE_PTR_ARRAY for GPtrArray-of-NMObject-valued properties, because it has better introspection/bindings support. As with the strdict change in libnm-core, we need to manually copy the array in get_property() implementations, to preserve the standard semantics that get_property() returns a copy, not the internal array. (This patch also changes those properties so that they are always non-NULL until dispose(); previously some of them could be either NULL or 0-length at different times.)
2014-08-26 08:31:04 -04:00
#include "nm-core-internal.h"
libnm: port to GDBus Port libnm-core/libnm to GDBus. The NetworkManager daemon continues to use dbus-glib; the previously-added connection hash/variant conversion methods are now moved to NetworkManagerUtils (along with a few other utilities that are now only needed by the daemon code).
2014-09-10 13:51:53 -04:00
#include "nm-dbus-helpers.h"
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
/*****************************************************************************/
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
NM_GOBJECT_PROPERTIES_DEFINE_BASE (
PROP_PERM_HW_ADDRESS,
PROP_MODE,
PROP_BITRATE,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
PROP_ACCESS_POINTS,
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
PROP_ACTIVE_ACCESS_POINT,
PROP_WIRELESS_CAPABILITIES,
PROP_LAST_SCAN,
);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
typedef struct {
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
NMLDBusPropertyAO access_points;
NMLDBusPropertyO active_access_point;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
char *perm_hw_address;
wifi: expose LastScan as milliseconds not seconds This doesn't wrap around in 68 years of uptime and is consistent with o.fd.NM.Checkpoint.Created.
2018-06-15 13:57:30 +02:00
gint64 last_scan;
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
guint32 mode;
guint32 bitrate;
guint32 wireless_capabilities;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
} NMDeviceWifiPrivate;
enum {
ACCESS_POINT_ADDED,
ACCESS_POINT_REMOVED,
LAST_SIGNAL
};
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
static guint signals[LAST_SIGNAL] = { 0 };
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
struct _NMDeviceWifi {
NMDevice parent;
NMDeviceWifiPrivate _priv;
};
struct _NMDeviceWifiClass {
NMDeviceClass parent;
};
G_DEFINE_TYPE (NMDeviceWifi, nm_device_wifi, NM_TYPE_DEVICE)
#define NM_DEVICE_WIFI_GET_PRIVATE(self) _NM_GET_PRIVATE(self, NMDeviceWifi, NM_IS_DEVICE_WIFI, NMObject, NMDevice)
/*****************************************************************************/
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* nm_device_wifi_get_hw_address:
* @device: a #NMDeviceWifi
*
* Gets the actual hardware (MAC) address of the #NMDeviceWifi
*
* Returns: the actual hardware address. This is the internal string used by the
* device, and must not be modified.
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
*
libnm/doc: fix gtk-doc for deprecated markers in libnm
2020-03-13 19:45:09 +01:00
* Deprecated: 1.24: Use nm_device_get_hw_address() instead.
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
**/
const char *
nm_device_wifi_get_hw_address (NMDeviceWifi *device)
{
g_return_val_if_fail (NM_IS_DEVICE_WIFI (device), NULL);
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
return nm_device_get_hw_address (NM_DEVICE (device));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_wifi_get_permanent_hw_address:
* @device: a #NMDeviceWifi
*
* Gets the permanent hardware (MAC) address of the #NMDeviceWifi
*
* Returns: the permanent hardware address. This is the internal string used by the
* device, and must not be modified.
**/
const char *
nm_device_wifi_get_permanent_hw_address (NMDeviceWifi *device)
{
g_return_val_if_fail (NM_IS_DEVICE_WIFI (device), NULL);
libnm: add and use _nml_coerce_property_*() Our NMObject implementations should behave in a similar manner. For example, string properties should be coerced the a consistent manner. Add functions _nml_coerce_property_*() for that. Of course, they are trivial. Their value is not that they encapsulate some complex implementation, but that they convey a general concept of how we want to handle certain properties in NMClient's object cache.
2019-10-22 16:51:09 +02:00
return _nml_coerce_property_str_not_empty (NM_DEVICE_WIFI_GET_PRIVATE (device)->perm_hw_address);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_wifi_get_mode:
* @device: a #NMDeviceWifi
*
* Gets the #NMDeviceWifi mode.
*
* Returns: the mode
**/
NM80211Mode
nm_device_wifi_get_mode (NMDeviceWifi *device)
{
g_return_val_if_fail (NM_IS_DEVICE_WIFI (device), 0);
return NM_DEVICE_WIFI_GET_PRIVATE (device)->mode;
}
/**
* nm_device_wifi_get_bitrate:
* @device: a #NMDeviceWifi
*
* Gets the bit rate of the #NMDeviceWifi in kbit/s.
*
* Returns: the bit rate (kbit/s)
**/
guint32
nm_device_wifi_get_bitrate (NMDeviceWifi *device)
{
NMDeviceState state;
g_return_val_if_fail (NM_IS_DEVICE_WIFI (device), 0);
state = nm_device_get_state (NM_DEVICE (device));
switch (state) {
case NM_DEVICE_STATE_IP_CONFIG:
case NM_DEVICE_STATE_IP_CHECK:
case NM_DEVICE_STATE_SECONDARIES:
case NM_DEVICE_STATE_ACTIVATED:
case NM_DEVICE_STATE_DEACTIVATING:
break;
default:
return 0;
}
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return NM_DEVICE_WIFI_GET_PRIVATE (device)->bitrate;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_wifi_get_capabilities:
* @device: a #NMDeviceWifi
*
* Gets the Wi-Fi capabilities of the #NMDeviceWifi.
*
* Returns: the capabilities
**/
NMDeviceWifiCapabilities
nm_device_wifi_get_capabilities (NMDeviceWifi *device)
{
g_return_val_if_fail (NM_IS_DEVICE_WIFI (device), 0);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return NM_DEVICE_WIFI_GET_PRIVATE (device)->wireless_capabilities;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_wifi_get_active_access_point:
* @device: a #NMDeviceWifi
*
* Gets the active #NMAccessPoint.
*
* Returns: (transfer none): the access point or %NULL if none is active
**/
NMAccessPoint *
nm_device_wifi_get_active_access_point (NMDeviceWifi *device)
{
g_return_val_if_fail (NM_IS_DEVICE_WIFI (device), NULL);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return nml_dbus_property_o_get_obj (&NM_DEVICE_WIFI_GET_PRIVATE (device)->active_access_point);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_wifi_get_access_points:
* @device: a #NMDeviceWifi
*
* Gets all the scanned access points of the #NMDeviceWifi.
*
* Returns: (element-type NMAccessPoint): a #GPtrArray containing all the
* scanned #NMAccessPoints.
* The returned array is owned by the client and should not be modified.
**/
const GPtrArray *
nm_device_wifi_get_access_points (NMDeviceWifi *device)
{
g_return_val_if_fail (NM_IS_DEVICE_WIFI (device), NULL);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return nml_dbus_property_ao_get_objs_as_ptrarray (&NM_DEVICE_WIFI_GET_PRIVATE (device)->access_points);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
/**
* nm_device_wifi_get_access_point_by_path:
* @device: a #NMDeviceWifi
* @path: the object path of the access point
*
* Gets a #NMAccessPoint by path.
*
* Returns: (transfer none): the access point or %NULL if none is found.
**/
NMAccessPoint *
nm_device_wifi_get_access_point_by_path (NMDeviceWifi *device,
const char *path)
{
const GPtrArray *aps;
int i;
NMAccessPoint *ap = NULL;
g_return_val_if_fail (NM_IS_DEVICE_WIFI (device), NULL);
g_return_val_if_fail (path != NULL, NULL);
aps = nm_device_wifi_get_access_points (device);
if (!aps)
return NULL;
for (i = 0; i < aps->len; i++) {
NMAccessPoint *candidate = g_ptr_array_index (aps, i);
if (!strcmp (nm_object_get_path (NM_OBJECT (candidate)), path)) {
ap = candidate;
break;
}
}
return ap;
}
wifi: expose the LastScan property This is the time when the last Wi-Fi scan finished. It will help clients decide whether the AP list is fresh enough.
2018-02-25 14:53:44 +01:00
/**
* nm_device_wifi_get_last_scan:
* @device: a #NMDeviceWifi
*
wifi: expose LastScan as milliseconds not seconds This doesn't wrap around in 68 years of uptime and is consistent with o.fd.NM.Checkpoint.Created.
2018-06-15 13:57:30 +02:00
* Returns the timestamp (in CLOCK_BOOTTIME milliseconds) for the last finished
wifi: expose the LastScan property This is the time when the last Wi-Fi scan finished. It will help clients decide whether the AP list is fresh enough.
2018-02-25 14:53:44 +01:00
* network scan. A value of -1 means the device never scanned for access points.
*
libnm: add nm_utils_get_timestamp_msec()
2018-06-15 14:15:27 +02:00
* Use nm_utils_get_timestamp_msec() to obtain current time value suitable for
* comparing to this value.
*
wifi: expose the LastScan property This is the time when the last Wi-Fi scan finished. It will help clients decide whether the AP list is fresh enough.
2018-02-25 14:53:44 +01:00
* Returns: the last scan time in seconds
*
* Since: 1.12
**/
wifi: expose LastScan as milliseconds not seconds This doesn't wrap around in 68 years of uptime and is consistent with o.fd.NM.Checkpoint.Created.
2018-06-15 13:57:30 +02:00
gint64
wifi: expose the LastScan property This is the time when the last Wi-Fi scan finished. It will help clients decide whether the AP list is fresh enough.
2018-02-25 14:53:44 +01:00
nm_device_wifi_get_last_scan (NMDeviceWifi *device)
{
wifi: expose LastScan as milliseconds not seconds This doesn't wrap around in 68 years of uptime and is consistent with o.fd.NM.Checkpoint.Created.
2018-06-15 13:57:30 +02:00
g_return_val_if_fail (NM_IS_DEVICE_WIFI (device), -1);
wifi: expose the LastScan property This is the time when the last Wi-Fi scan finished. It will help clients decide whether the AP list is fresh enough.
2018-02-25 14:53:44 +01:00
wifi: expose LastScan as milliseconds not seconds This doesn't wrap around in 68 years of uptime and is consistent with o.fd.NM.Checkpoint.Created.
2018-06-15 13:57:30 +02:00
return NM_DEVICE_WIFI_GET_PRIVATE (device)->last_scan;
wifi: expose the LastScan property This is the time when the last Wi-Fi scan finished. It will help clients decide whether the AP list is fresh enough.
2018-02-25 14:53:44 +01:00
}
libnm: add some missing sync/async method variants Add the missing variant in most places in the API where previously there was either only a synchronous version or only an asynchronous version. There is not yet a synchronous nm_client_activate_connection(), nm_client_add_and_activate_connection(), or nm_remote_settings_add_connection(), because the existing async code depends on waiting for other asynchronous events, so making them run synchronously is slightly more complicated. But these APIs can be added later.
2014-09-11 16:27:13 -04:00
/**
* nm_device_wifi_request_scan:
* @device: a #NMDeviceWifi
* @cancellable: a #GCancellable, or %NULL
* @error: location for a #GError, or %NULL
*
* Request NM to scan for access points on @device. Note that the function
* returns immediately after requesting the scan, and it may take some time
* after that for the scan to complete.
*
* Returns: %TRUE on success, %FALSE on error, in which case @error will be
* set.
libnm: deprecate synchronous/blocking API in libnm Note that D-Bus is fundamentally asynchronous. Doing blocking calls on top of D-Bus is odd, especially for libnm's NMClient. That is because NMClient essentially is a client-side cache of the objects from the D-Bus interface. This cache should be filled exclusively by (asynchronous) D-Bus events (PropertiesChanged). So, making a blocking D-Bus call means to wait for a response and return it, while queuing all messages that are received in the meantime. Basically there are three ways how a synchronous API on NMClient could behave: 1) the call just calls g_dbus_connection_call_sync(). This means that libnm sends a D-Bus request via GDBusConnection, and blockingly waits for the response. All D-Bus messages that get received in the meantime are queued in the GMainContext that belongs to NMClient. That means, none of these D-Bus events are processed until we iterate the GMainContext after the call returns. The effect is, that NMClient (and all cached objects in there) are unaffected by the D-Bus request. Most of the synchronous API calls in libnm are of this kind. The problem is that the strict ordering of D-Bus events gets violated. For some API this is not an immediate problem. Take for example nm_device_wifi_request_scan(). The call merely blockingly tells NetworkManager to start scanning, but since NetworkManager's D-Bus API does not directly expose any state that tells whether we are currently scanning, this out of order processing of the D-Bus request is a small issue. The problem is more obvious for nm_client_networking_set_enabled(). After calling it, NM_CLIENT_NETWORKING_ENABLED is still unaffected and unchanged, because the PropertiesChanged signal from D-Bus is not yet processed. This means, while you make such a blocking call, NMClient's state does not change. But usually you perform the synchronous call to change some state. In this form, the blocking call is not useful, because NMClient only changes the state after iterating the GMainContext, and not after the blocking call returns. 2) like 1), but after making the blocking g_dbus_connection_call_sync(), update the NMClient cache artificially. This is what nm_manager_check_connectivity() does, to "fix" bgo#784629. This also has the problem of out-of-order events, but it kinda solves the problem of not changing the state during the blocking call. But it does so by hacking the state of the cache. I think this is really wrong because the state should only be updated from the ordered stream of D-Bus messages (PropertiesChanged signal and similar). When libnm decides to modify the state, there may be already D-Bus messages queued that affect this very state. 3) instead of calling g_dbus_connection_call_sync(), use the asynchronous g_dbus_connection_call(). If we would use a sepaate GMainContext for all D-Bus related calls, we could ensure that while we block for the response, we iterate that internal main context. This might be nice, because all events are processed in order and after the blocking call returns, the NMClient state is up to date. The are problems however: current blocking API does not do this, so it's a significant change in behavior. Also, it might be unexpected to the user that during the blocking call the entire content of NMClient's cache might change and all pointers to the cache might be invalidated. Also, of course NMClient would invoke signals for all the changes that happen. Another problem is that this would be more effort to implement and it involves a small performance overhead for all D-Bus related calls (because we have to serialize all events in an internal GMainContext first and then invoke them on the caller's context). Also, if the users wants this behavior, they could implement it themself by running libnm in their own GMainContext. Note that libnm might have bugs to make that really working, but that should be fixed instead of adding such synchrnous API behavior. Read also [1], for why blocking calls are wrong. [1] https://smcv.pseudorandom.co.uk/2008/11/nonblocking/ So, all possible behaviors for synchronous API have severe behavioural issues. Mark all this API as deprecated. Also, this serves the purpose of identifying blocking D-Bus calls in libnm. Note that "deprecated" here does not really mean that the API is going to be removed. We don't break API. The user may: - continue to use this API. It's deprecated, awkward and discouraged, but if it works, by all means use it. - use asynchronous API. That's the only sensible way to use D-Bus. If libnm lacks a certain asynchronous counterpart, it should be added. - use GDBusConnection directly. There really isn't anything wrong with D-Bus or GDBusConnection. This deprecated API is just a wrapper around g_dbus_connection_call_sync(). You may call it directly without feeling dirty. --- The only other remainging API is the synchronous GInitable call for NMClient. That is an entirely separate beast and not particularly wrong (from an API point of view). Note that synchronous API in NMSecretAgentOld, NMVpnPluginOld and NMVpnServicePlugin as not deprecated here. These types are not part of the D-Bus cache and while they have similar issues, it's less severe because they have less state.
2019-09-04 13:58:43 +02:00
*
libnm/doc: fix gtk-doc for deprecated markers in libnm
2020-03-13 19:45:09 +01:00
* Deprecated: 1.22: Use nm_device_wifi_request_scan_async() or GDBusConnection.
libnm: add some missing sync/async method variants Add the missing variant in most places in the API where previously there was either only a synchronous version or only an asynchronous version. There is not yet a synchronous nm_client_activate_connection(), nm_client_add_and_activate_connection(), or nm_remote_settings_add_connection(), because the existing async code depends on waiting for other asynchronous events, so making them run synchronously is slightly more complicated. But these APIs can be added later.
2014-09-11 16:27:13 -04:00
**/
gboolean
nm_device_wifi_request_scan (NMDeviceWifi *device,
GCancellable *cancellable,
GError **error)
{
libnm: implement nm_device_wifi_request_scan() by using GDBusConnection directly
2019-10-06 11:26:09 +02:00
return nm_device_wifi_request_scan_options (device, NULL, cancellable, error);
libnm: add new functions allowing passing options to RequestScan() D-Bus call nm_device_wifi_request_scan_options() nm_device_wifi_request_scan_options_async()
2015-07-14 09:46:55 +02:00
}
libnm: add some missing sync/async method variants Add the missing variant in most places in the API where previously there was either only a synchronous version or only an asynchronous version. There is not yet a synchronous nm_client_activate_connection(), nm_client_add_and_activate_connection(), or nm_remote_settings_add_connection(), because the existing async code depends on waiting for other asynchronous events, so making them run synchronously is slightly more complicated. But these APIs can be added later.
2014-09-11 16:27:13 -04:00
libnm: add new functions allowing passing options to RequestScan() D-Bus call nm_device_wifi_request_scan_options() nm_device_wifi_request_scan_options_async()
2015-07-14 09:46:55 +02:00
/**
* nm_device_wifi_request_scan_options:
* @device: a #NMDeviceWifi
* @options: dictionary with options for RequestScan(), or %NULL
* @cancellable: a #GCancellable, or %NULL
* @error: location for a #GError, or %NULL
*
* Request NM to scan for access points on @device. Note that the function
* returns immediately after requesting the scan, and it may take some time
* after that for the scan to complete.
* This is the same as @nm_device_wifi_request_scan except it accepts @options
* for the scanning. The argument is the dictionary passed to RequestScan()
docs: misc. typos pt2 Remainder of typos found using `codespell -q 3 --skip="./shared,./src/systemd,*.po" -I ../NetworkManager-word-whitelist.txt` whereby whitelist consists of: ``` ans busses cace cna conexant crasher iff liftime creat nd sav technik uint ``` https://github.com/NetworkManager/NetworkManager/pull/205
2018-09-15 07:20:54 -04:00
* D-Bus call. Valid options inside the dictionary are:
libnm: add new functions allowing passing options to RequestScan() D-Bus call nm_device_wifi_request_scan_options() nm_device_wifi_request_scan_options_async()
2015-07-14 09:46:55 +02:00
* 'ssids' => array of SSIDs (saay)
*
* Returns: %TRUE on success, %FALSE on error, in which case @error will be
* set.
*
Revert "all: change "Since: 1.2" to "Since: 1.0.4"/"Since: 1.0.6" for backported API" API should be added with "Since:" of the next release on the same branch. That means, new API on 1.1 branch (development), should be "Since: 1.2" and new API on 1.0 branch (stable) will be "Since: 1.0.x". Similarly, new API on master is NM_AVAILABLE_IN_1_2 and will be added with the linker version libnl_1_2 -- never the versions of minor releases. It is also strongly advised that for the 1.0 branch, we only add API that was previously formerly added on master. IOW, that we only do true backports of API that already exists on master. API that gets backported, must also be added to master via NM_BACKPORT_SYMBOL(). That gives ABI compatibility and an application that was build against 1.0.x will work with 1.y.z version (y > 0) without need for recompiling -- provided that 1.y.z also contains that API. There is one important caveat: if a major branch (e.g. current master) has a linker section of backported APIs (e.g. libnm_1_0_6), we must do the minor release (1.0.6) before the next major release (1.2). The reason is that after the major release, the linker section (libnm_1_0_6) must not be extended and thus the minor release (1.0.6) must be already released at that point. In general, users should avoid using backported API because it limits the ability to upgrade to arbitrary later versions. But together with the previous point (that we only backport API to minor releases), a user that uses backported API can be sure that a 1.y.z version is ABI compatible with 1.0.x, if the 1.y.z release date was after the release date of 1.0.x. This reverts commit 02a136682c749a0fd27853c0152d36c44635151f.
2015-08-22 00:57:30 +02:00
* Since: 1.2
libnm: deprecate synchronous/blocking API in libnm Note that D-Bus is fundamentally asynchronous. Doing blocking calls on top of D-Bus is odd, especially for libnm's NMClient. That is because NMClient essentially is a client-side cache of the objects from the D-Bus interface. This cache should be filled exclusively by (asynchronous) D-Bus events (PropertiesChanged). So, making a blocking D-Bus call means to wait for a response and return it, while queuing all messages that are received in the meantime. Basically there are three ways how a synchronous API on NMClient could behave: 1) the call just calls g_dbus_connection_call_sync(). This means that libnm sends a D-Bus request via GDBusConnection, and blockingly waits for the response. All D-Bus messages that get received in the meantime are queued in the GMainContext that belongs to NMClient. That means, none of these D-Bus events are processed until we iterate the GMainContext after the call returns. The effect is, that NMClient (and all cached objects in there) are unaffected by the D-Bus request. Most of the synchronous API calls in libnm are of this kind. The problem is that the strict ordering of D-Bus events gets violated. For some API this is not an immediate problem. Take for example nm_device_wifi_request_scan(). The call merely blockingly tells NetworkManager to start scanning, but since NetworkManager's D-Bus API does not directly expose any state that tells whether we are currently scanning, this out of order processing of the D-Bus request is a small issue. The problem is more obvious for nm_client_networking_set_enabled(). After calling it, NM_CLIENT_NETWORKING_ENABLED is still unaffected and unchanged, because the PropertiesChanged signal from D-Bus is not yet processed. This means, while you make such a blocking call, NMClient's state does not change. But usually you perform the synchronous call to change some state. In this form, the blocking call is not useful, because NMClient only changes the state after iterating the GMainContext, and not after the blocking call returns. 2) like 1), but after making the blocking g_dbus_connection_call_sync(), update the NMClient cache artificially. This is what nm_manager_check_connectivity() does, to "fix" bgo#784629. This also has the problem of out-of-order events, but it kinda solves the problem of not changing the state during the blocking call. But it does so by hacking the state of the cache. I think this is really wrong because the state should only be updated from the ordered stream of D-Bus messages (PropertiesChanged signal and similar). When libnm decides to modify the state, there may be already D-Bus messages queued that affect this very state. 3) instead of calling g_dbus_connection_call_sync(), use the asynchronous g_dbus_connection_call(). If we would use a sepaate GMainContext for all D-Bus related calls, we could ensure that while we block for the response, we iterate that internal main context. This might be nice, because all events are processed in order and after the blocking call returns, the NMClient state is up to date. The are problems however: current blocking API does not do this, so it's a significant change in behavior. Also, it might be unexpected to the user that during the blocking call the entire content of NMClient's cache might change and all pointers to the cache might be invalidated. Also, of course NMClient would invoke signals for all the changes that happen. Another problem is that this would be more effort to implement and it involves a small performance overhead for all D-Bus related calls (because we have to serialize all events in an internal GMainContext first and then invoke them on the caller's context). Also, if the users wants this behavior, they could implement it themself by running libnm in their own GMainContext. Note that libnm might have bugs to make that really working, but that should be fixed instead of adding such synchrnous API behavior. Read also [1], for why blocking calls are wrong. [1] https://smcv.pseudorandom.co.uk/2008/11/nonblocking/ So, all possible behaviors for synchronous API have severe behavioural issues. Mark all this API as deprecated. Also, this serves the purpose of identifying blocking D-Bus calls in libnm. Note that "deprecated" here does not really mean that the API is going to be removed. We don't break API. The user may: - continue to use this API. It's deprecated, awkward and discouraged, but if it works, by all means use it. - use asynchronous API. That's the only sensible way to use D-Bus. If libnm lacks a certain asynchronous counterpart, it should be added. - use GDBusConnection directly. There really isn't anything wrong with D-Bus or GDBusConnection. This deprecated API is just a wrapper around g_dbus_connection_call_sync(). You may call it directly without feeling dirty. --- The only other remainging API is the synchronous GInitable call for NMClient. That is an entirely separate beast and not particularly wrong (from an API point of view). Note that synchronous API in NMSecretAgentOld, NMVpnPluginOld and NMVpnServicePlugin as not deprecated here. These types are not part of the D-Bus cache and while they have similar issues, it's less severe because they have less state.
2019-09-04 13:58:43 +02:00
*
libnm/doc: fix gtk-doc for deprecated markers in libnm
2020-03-13 19:45:09 +01:00
* Deprecated: 1.22: Use nm_device_wifi_request_scan_options_async() or GDBusConnection.
libnm: add new functions allowing passing options to RequestScan() D-Bus call nm_device_wifi_request_scan_options() nm_device_wifi_request_scan_options_async()
2015-07-14 09:46:55 +02:00
**/
gboolean
nm_device_wifi_request_scan_options (NMDeviceWifi *device,
GVariant *options,
GCancellable *cancellable,
GError **error)
{
libnm: implement nm_device_wifi_request_scan() by using GDBusConnection directly
2019-10-06 11:26:09 +02:00
g_return_val_if_fail (NM_IS_DEVICE_WIFI (device), FALSE);
g_return_val_if_fail (!options || g_variant_is_of_type (options, G_VARIANT_TYPE_VARDICT), FALSE);
g_return_val_if_fail (!cancellable || G_IS_CANCELLABLE (cancellable), FALSE);
g_return_val_if_fail (!error || !*error, FALSE);
if (!options)
options = g_variant_new_array (G_VARIANT_TYPE ("{sv}"), NULL, 0);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
return _nm_client_dbus_call_sync_void (_nm_object_get_client (device),
libnm: implement nm_device_wifi_request_scan() by using GDBusConnection directly
2019-10-06 11:26:09 +02:00
cancellable,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_object_get_path (device),
libnm: implement nm_device_wifi_request_scan() by using GDBusConnection directly
2019-10-06 11:26:09 +02:00
NM_DBUS_INTERFACE_DEVICE_WIRELESS,
"RequestScan",
g_variant_new ("(@a{sv})", options),
G_DBUS_CALL_FLAGS_NONE,
NM_DBUS_DEFAULT_TIMEOUT_MSEC,
TRUE,
error);
libnm: add some missing sync/async method variants Add the missing variant in most places in the API where previously there was either only a synchronous version or only an asynchronous version. There is not yet a synchronous nm_client_activate_connection(), nm_client_add_and_activate_connection(), or nm_remote_settings_add_connection(), because the existing async code depends on waiting for other asynchronous events, so making them run synchronously is slightly more complicated. But these APIs can be added later.
2014-09-11 16:27:13 -04:00
}
libnm: symbols for request scanning backported to 1.0.6 Backport to 1.0.6 the following symbols: - nm_device_wifi_request_scan_options - nm_device_wifi_request_scan_options_async Backported by commit 91c0555afaac57234744f42024d95bde46ba170e
2015-08-14 14:24:16 +02:00
NM_BACKPORT_SYMBOL (libnm_1_0_6, gboolean, nm_device_wifi_request_scan_options,
(NMDeviceWifi *device, GVariant *options, GCancellable *cancellable, GError **error),
(device, options, cancellable, error));
libnm: add new functions allowing passing options to RequestScan() D-Bus call nm_device_wifi_request_scan_options() nm_device_wifi_request_scan_options_async()
2015-07-14 09:46:55 +02:00
/**
* nm_device_wifi_request_scan_async:
* @device: a #NMDeviceWifi
* @cancellable: a #GCancellable, or %NULL
* @callback: callback to be called when the scan has been requested
* @user_data: caller-specific data passed to @callback
*
* Request NM to scan for access points on @device. Note that @callback will be
* called immediately after requesting the scan, and it may take some time after
* that for the scan to complete.
**/
void
nm_device_wifi_request_scan_async (NMDeviceWifi *device,
GCancellable *cancellable,
GAsyncReadyCallback callback,
gpointer user_data)
{
libnm: implement nm_device_wifi_request_scan_async() by using GDBusConnection directly No longer keep track if a request is already pending. Previously, in that case, the function would return success in an idle handler. Instead, just always issue the new request. It's up to the caller to rate-limit the requests and trigger a scan whenever appropriate. libnm should not add additional logic to that.
2019-10-06 11:26:09 +02:00
nm_device_wifi_request_scan_options_async (device, NULL, cancellable, callback, user_data);
libnm: add new functions allowing passing options to RequestScan() D-Bus call nm_device_wifi_request_scan_options() nm_device_wifi_request_scan_options_async()
2015-07-14 09:46:55 +02:00
}
/**
* nm_device_wifi_request_scan_options_async:
* @device: a #NMDeviceWifi
* @options: dictionary with options for RequestScan(), or %NULL
* @cancellable: a #GCancellable, or %NULL
* @callback: callback to be called when the scan has been requested
* @user_data: caller-specific data passed to @callback
*
* Request NM to scan for access points on @device. Note that @callback will be
* called immediately after requesting the scan, and it may take some time after
* that for the scan to complete.
* This is the same as @nm_device_wifi_request_scan_async except it accepts @options
* for the scanning. The argument is the dictionary passed to RequestScan()
docs: misc. typos pt2 Remainder of typos found using `codespell -q 3 --skip="./shared,./src/systemd,*.po" -I ../NetworkManager-word-whitelist.txt` whereby whitelist consists of: ``` ans busses cace cna conexant crasher iff liftime creat nd sav technik uint ``` https://github.com/NetworkManager/NetworkManager/pull/205
2018-09-15 07:20:54 -04:00
* D-Bus call. Valid options inside the dictionary are:
libnm: add new functions allowing passing options to RequestScan() D-Bus call nm_device_wifi_request_scan_options() nm_device_wifi_request_scan_options_async()
2015-07-14 09:46:55 +02:00
* 'ssids' => array of SSIDs (saay)
*
libnm: implement nm_device_wifi_request_scan_async() by using GDBusConnection directly No longer keep track if a request is already pending. Previously, in that case, the function would return success in an idle handler. Instead, just always issue the new request. It's up to the caller to rate-limit the requests and trigger a scan whenever appropriate. libnm should not add additional logic to that.
2019-10-06 11:26:09 +02:00
* To complete the request call nm_device_wifi_request_scan_finish().
*
Revert "all: change "Since: 1.2" to "Since: 1.0.4"/"Since: 1.0.6" for backported API" API should be added with "Since:" of the next release on the same branch. That means, new API on 1.1 branch (development), should be "Since: 1.2" and new API on 1.0 branch (stable) will be "Since: 1.0.x". Similarly, new API on master is NM_AVAILABLE_IN_1_2 and will be added with the linker version libnl_1_2 -- never the versions of minor releases. It is also strongly advised that for the 1.0 branch, we only add API that was previously formerly added on master. IOW, that we only do true backports of API that already exists on master. API that gets backported, must also be added to master via NM_BACKPORT_SYMBOL(). That gives ABI compatibility and an application that was build against 1.0.x will work with 1.y.z version (y > 0) without need for recompiling -- provided that 1.y.z also contains that API. There is one important caveat: if a major branch (e.g. current master) has a linker section of backported APIs (e.g. libnm_1_0_6), we must do the minor release (1.0.6) before the next major release (1.2). The reason is that after the major release, the linker section (libnm_1_0_6) must not be extended and thus the minor release (1.0.6) must be already released at that point. In general, users should avoid using backported API because it limits the ability to upgrade to arbitrary later versions. But together with the previous point (that we only backport API to minor releases), a user that uses backported API can be sure that a 1.y.z version is ABI compatible with 1.0.x, if the 1.y.z release date was after the release date of 1.0.x. This reverts commit 02a136682c749a0fd27853c0152d36c44635151f.
2015-08-22 00:57:30 +02:00
* Since: 1.2
libnm: add new functions allowing passing options to RequestScan() D-Bus call nm_device_wifi_request_scan_options() nm_device_wifi_request_scan_options_async()
2015-07-14 09:46:55 +02:00
**/
void
nm_device_wifi_request_scan_options_async (NMDeviceWifi *device,
GVariant *options,
GCancellable *cancellable,
GAsyncReadyCallback callback,
gpointer user_data)
{
libnm: implement nm_device_wifi_request_scan_async() by using GDBusConnection directly No longer keep track if a request is already pending. Previously, in that case, the function would return success in an idle handler. Instead, just always issue the new request. It's up to the caller to rate-limit the requests and trigger a scan whenever appropriate. libnm should not add additional logic to that.
2019-10-06 11:26:09 +02:00
g_return_if_fail (NM_IS_DEVICE_WIFI (device));
g_return_if_fail (!options || g_variant_is_of_type (options, G_VARIANT_TYPE_VARDICT));
g_return_if_fail (!cancellable || G_IS_CANCELLABLE (cancellable));
if (!options)
options = g_variant_new_array (G_VARIANT_TYPE ("{sv}"), NULL, 0);
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_client_dbus_call (_nm_object_get_client (device),
device,
libnm: implement nm_device_wifi_request_scan_async() by using GDBusConnection directly No longer keep track if a request is already pending. Previously, in that case, the function would return success in an idle handler. Instead, just always issue the new request. It's up to the caller to rate-limit the requests and trigger a scan whenever appropriate. libnm should not add additional logic to that.
2019-10-06 11:26:09 +02:00
nm_device_wifi_request_scan_async,
cancellable,
callback,
user_data,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nm_object_get_path (device),
libnm: implement nm_device_wifi_request_scan_async() by using GDBusConnection directly No longer keep track if a request is already pending. Previously, in that case, the function would return success in an idle handler. Instead, just always issue the new request. It's up to the caller to rate-limit the requests and trigger a scan whenever appropriate. libnm should not add additional logic to that.
2019-10-06 11:26:09 +02:00
NM_DBUS_INTERFACE_DEVICE_WIRELESS,
"RequestScan",
g_variant_new ("(@a{sv})", options),
G_VARIANT_TYPE ("()"),
G_DBUS_CALL_FLAGS_NONE,
NM_DBUS_DEFAULT_TIMEOUT_MSEC,
nm_dbus_connection_call_finish_void_strip_dbus_error_cb);
libnm: add new functions allowing passing options to RequestScan() D-Bus call nm_device_wifi_request_scan_options() nm_device_wifi_request_scan_options_async()
2015-07-14 09:46:55 +02:00
}
libnm: symbols for request scanning backported to 1.0.6 Backport to 1.0.6 the following symbols: - nm_device_wifi_request_scan_options - nm_device_wifi_request_scan_options_async Backported by commit 91c0555afaac57234744f42024d95bde46ba170e
2015-08-14 14:24:16 +02:00
NM_BACKPORT_SYMBOL (libnm_1_0_6, void, nm_device_wifi_request_scan_options_async,
(NMDeviceWifi *device, GVariant *options, GCancellable *cancellable, GAsyncReadyCallback callback, gpointer user_data),
(device, options, cancellable, callback, user_data));
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
/**
* nm_device_wifi_request_scan_finish:
* @device: a #NMDeviceWifi
* @result: the result passed to the #GAsyncReadyCallback
* @error: location for a #GError, or %NULL
*
libnm: implement nm_device_wifi_request_scan_async() by using GDBusConnection directly No longer keep track if a request is already pending. Previously, in that case, the function would return success in an idle handler. Instead, just always issue the new request. It's up to the caller to rate-limit the requests and trigger a scan whenever appropriate. libnm should not add additional logic to that.
2019-10-06 11:26:09 +02:00
* Gets the result of a call to nm_device_wifi_request_scan_async() and
* nm_device_wifi_request_scan_options_async().
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
*
* Returns: %TRUE on success, %FALSE on error, in which case @error will be
* set.
**/
gboolean
nm_device_wifi_request_scan_finish (NMDeviceWifi *device,
GAsyncResult *result,
GError **error)
{
libnm: implement nm_device_wifi_request_scan_async() by using GDBusConnection directly No longer keep track if a request is already pending. Previously, in that case, the function would return success in an idle handler. Instead, just always issue the new request. It's up to the caller to rate-limit the requests and trigger a scan whenever appropriate. libnm should not add additional logic to that.
2019-10-06 11:26:09 +02:00
g_return_val_if_fail (NM_IS_DEVICE_WIFI (device), FALSE);
g_return_val_if_fail (nm_g_task_is_valid (result, device, nm_device_wifi_request_scan_async), FALSE);
libnm: make sync/async APIs more GLib-like Make synchronous APIs take GCancellables, and make asynchronous APIs use GAsyncReadyCallbacks and have names ending in "_async", with "_finish" functions to retrieve the results. Also, make nm_client_activate_connection_finish(), nm_client_add_and_activate_finish(), and nm_remote_settings_add_connection_finish() be (transfer full) rather than (transfer none), because the refcounting semantics become slightly confusing in some edge cases otherwise.
2014-09-11 16:27:13 -04:00
libnm: implement nm_device_wifi_request_scan_async() by using GDBusConnection directly No longer keep track if a request is already pending. Previously, in that case, the function would return success in an idle handler. Instead, just always issue the new request. It's up to the caller to rate-limit the requests and trigger a scan whenever appropriate. libnm should not add additional logic to that.
2019-10-06 11:26:09 +02:00
return g_task_propagate_boolean (G_TASK (result), error);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
#define WPA_CAPS (NM_WIFI_DEVICE_CAP_CIPHER_TKIP | \
NM_WIFI_DEVICE_CAP_CIPHER_CCMP | \
NM_WIFI_DEVICE_CAP_WPA | \
NM_WIFI_DEVICE_CAP_RSN)
#define RSN_CAPS (NM_WIFI_DEVICE_CAP_CIPHER_CCMP | NM_WIFI_DEVICE_CAP_RSN)
static gboolean
has_proto (NMSettingWirelessSecurity *s_wsec, const char *proto)
{
int i;
for (i = 0; i < nm_setting_wireless_security_get_num_protos (s_wsec); i++) {
if (g_strcmp0 (proto, nm_setting_wireless_security_get_proto (s_wsec, i)) == 0)
return TRUE;
}
return FALSE;
}
static gboolean
connection_compatible (NMDevice *device, NMConnection *connection, GError **error)
{
NMSettingWireless *s_wifi;
NMSettingWirelessSecurity *s_wsec;
libnm-core: change all mac-address properties to G_TYPE_STRING Make all mac-address properties (including NMSettingBluetooth:bdaddr, NMSettingOlpcMesh:dhcp-anycast-addr, and NMSettingWireless:bssid) be strings, using _nm_setting_class_transform_property() to handle translating to/from binary form when dealing with D-Bus. Update everything accordingly for the change, and also add a test for transformed setting properties to test-general.
2014-07-30 10:57:45 -04:00
const char *hwaddr, *setting_hwaddr;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
NMDeviceWifiCapabilities wifi_caps;
const char *key_mgmt;
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
if (!NM_DEVICE_CLASS (nm_device_wifi_parent_class)->connection_compatible (device, connection, error))
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return FALSE;
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
if (!nm_connection_is_type (connection, NM_SETTING_WIRELESS_SETTING_NAME)) {
g_set_error_literal (error, NM_DEVICE_ERROR, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION,
_("The connection was not a Wi-Fi connection."));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return FALSE;
}
/* Check MAC address */
libnm-core: change all mac-address properties to G_TYPE_STRING Make all mac-address properties (including NMSettingBluetooth:bdaddr, NMSettingOlpcMesh:dhcp-anycast-addr, and NMSettingWireless:bssid) be strings, using _nm_setting_class_transform_property() to handle translating to/from binary form when dealing with D-Bus. Update everything accordingly for the change, and also add a test for transformed setting properties to test-general.
2014-07-30 10:57:45 -04:00
hwaddr = nm_device_wifi_get_permanent_hw_address (NM_DEVICE_WIFI (device));
if (hwaddr) {
if (!nm_utils_hwaddr_valid (hwaddr, ETH_ALEN)) {
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
g_set_error_literal (error, NM_DEVICE_ERROR, NM_DEVICE_ERROR_FAILED,
_("Invalid device MAC address."));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return FALSE;
}
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
s_wifi = nm_connection_get_setting_wireless (connection);
libnm-core: change all mac-address properties to G_TYPE_STRING Make all mac-address properties (including NMSettingBluetooth:bdaddr, NMSettingOlpcMesh:dhcp-anycast-addr, and NMSettingWireless:bssid) be strings, using _nm_setting_class_transform_property() to handle translating to/from binary form when dealing with D-Bus. Update everything accordingly for the change, and also add a test for transformed setting properties to test-general.
2014-07-30 10:57:45 -04:00
setting_hwaddr = nm_setting_wireless_get_mac_address (s_wifi);
if (setting_hwaddr && !nm_utils_hwaddr_matches (setting_hwaddr, -1, hwaddr, -1)) {
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
g_set_error_literal (error, NM_DEVICE_ERROR, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION,
_("The MACs of the device and the connection didn't match."));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return FALSE;
}
}
/* Check device capabilities; we assume all devices can do WEP at least */
s_wsec = nm_connection_get_setting_wireless_security (connection);
if (s_wsec) {
/* Connection has security, verify it against the device's capabilities */
key_mgmt = nm_setting_wireless_security_get_key_mgmt (s_wsec);
wifi: drop support for wpa-none key-mgmt NM didn't support wpa-none for years because kernel drivers used to be broken. Note that it wasn't even possible to *add* a connection with wpa-none because it was rejected in nm_settings_add_connection_dbus(). Given that wpa-none is also deprecated in wpa_supplicant and is considered insecure, drop altogether any reference to it.
2019-08-19 15:32:42 +02:00
if ( !g_strcmp0 (key_mgmt, "wpa-psk")
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
|| !g_strcmp0 (key_mgmt, "wpa-eap")) {
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
wifi_caps = nm_device_wifi_get_capabilities (NM_DEVICE_WIFI (device));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/* Is device only WEP capable? */
if (!(wifi_caps & WPA_CAPS)) {
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
g_set_error_literal (error, NM_DEVICE_ERROR, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION,
_("The device is lacking WPA capabilities required by the connection."));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return FALSE;
}
/* Make sure WPA2/RSN-only connections don't get chosen for WPA-only cards */
if (has_proto (s_wsec, "rsn") && !has_proto (s_wsec, "wpa") && !(wifi_caps & RSN_CAPS)) {
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
g_set_error_literal (error, NM_DEVICE_ERROR, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION,
_("The device is lacking WPA2/RSN capabilities required by the connection."));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
return FALSE;
}
}
}
libnm: merge device-type-specific errors into NMDeviceError As with the settings, each device type was defining its own error type, containing either redundant or non-useful error codes. Drop all of the subtype-specific errors, and reduce things to just NM_DEVICE_ERROR_FAILED, NM_DEVICE_ERROR_INCOMPATIBLE_CONNECTION, and NM_DEVICE_ERROR_INVALID_CONNECTION. The device-type-specific errors were only returned from their nm_device_connection_compatible() implementations, so this is also a good opportunity to simplify those, by moving duplicated functionality into the base NMDevice implementation, and then allowing the subclasses to assume that the connection has already been validated in their own code. Most of the implementations now just check that the connection has the correct type for the device (which can't be done at the NMDevice level since some device types (eg, Ethernet) support multiple connection types.) Also, make sure that all of the error messages are localized.
2014-10-11 14:44:10 -04:00
return TRUE;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
static GType
get_setting_type (NMDevice *device)
{
return NM_TYPE_SETTING_WIRELESS;
}
all: modify line separator comments to be 80 chars wide sed 's#^/\*\{5\}\*\+/$#/*****************************************************************************/#' $(git grep -l '\*\{5\}' | grep '\.[hc]$') -i
2016-10-02 18:22:50 +02:00
/*****************************************************************************/
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
static void
_property_ao_notify_changed_access_points_cb (NMLDBusPropertyAO *pr_ao,
NMClient *client,
NMObject *nmobj,
gboolean is_added /* or else removed */)
{
_nm_client_notify_event_queue_emit_obj_signal (client,
G_OBJECT (pr_ao->owner_dbobj->nmobj),
nmobj,
is_added,
10,
is_added
? signals[ACCESS_POINT_ADDED]
: signals[ACCESS_POINT_REMOVED]);
}
/*****************************************************************************/
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
static void
nm_device_wifi_init (NMDeviceWifi *device)
{
libnm: further NULL-vs-empty-array fixes In some cases, code may look at the value of an array-valued property during object initialization, before NMObject has set it to its actual initial value. So ensure that we initialize all such properties to an empty array, rather than leaving them NULL. Also fix another bug in NMClient that could result in priv->active_connections being NULL during certain signal emissions, and fix nm_client_get_active_connections() to not return NULL when NM was not running.
2014-10-06 21:16:00 -04:00
NMDeviceWifiPrivate *priv = NM_DEVICE_WIFI_GET_PRIVATE (device);
wifi: expose the LastScan property This is the time when the last Wi-Fi scan finished. It will help clients decide whether the AP list is fresh enough.
2018-02-25 14:53:44 +01:00
priv->last_scan = -1;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
static void
get_property (GObject *object,
guint prop_id,
GValue *value,
GParamSpec *pspec)
{
NMDeviceWifi *self = NM_DEVICE_WIFI (object);
switch (prop_id) {
case PROP_PERM_HW_ADDRESS:
g_value_set_string (value, nm_device_wifi_get_permanent_hw_address (self));
break;
case PROP_MODE:
libnm: make use of GParamSpecFlags and GParamSpecEnum Make enum- and flags-valued properties use GParamSpecEnum and GParamSpecFlags, for better introspectability/bindability. This requires no changes outside libnm-core/libnm since the expected data size is still the same with g_object_get()/g_object_set(), and GLib will internally convert between int/uint and enum/flags GValues when using g_object_get_property()/g_object_set_property().
2014-06-26 16:47:46 -04:00
g_value_set_enum (value, nm_device_wifi_get_mode (self));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
break;
case PROP_BITRATE:
g_value_set_uint (value, nm_device_wifi_get_bitrate (self));
break;
case PROP_ACTIVE_ACCESS_POINT:
g_value_set_object (value, nm_device_wifi_get_active_access_point (self));
break;
case PROP_WIRELESS_CAPABILITIES:
libnm: make use of GParamSpecFlags and GParamSpecEnum Make enum- and flags-valued properties use GParamSpecEnum and GParamSpecFlags, for better introspectability/bindability. This requires no changes outside libnm-core/libnm since the expected data size is still the same with g_object_get()/g_object_set(), and GLib will internally convert between int/uint and enum/flags GValues when using g_object_get_property()/g_object_set_property().
2014-06-26 16:47:46 -04:00
g_value_set_flags (value, nm_device_wifi_get_capabilities (self));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
break;
case PROP_ACCESS_POINTS:
libnm: drop NM_TYPE_OBJECT_ARRAY, use G_TYPE_PTR_ARRAY Use G_TYPE_PTR_ARRAY for GPtrArray-of-NMObject-valued properties, because it has better introspection/bindings support. As with the strdict change in libnm-core, we need to manually copy the array in get_property() implementations, to preserve the standard semantics that get_property() returns a copy, not the internal array. (This patch also changes those properties so that they are always non-NULL until dispose(); previously some of them could be either NULL or 0-length at different times.)
2014-08-26 08:31:04 -04:00
g_value_take_boxed (value, _nm_utils_copy_object_array (nm_device_wifi_get_access_points (self)));
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
break;
wifi: expose the LastScan property This is the time when the last Wi-Fi scan finished. It will help clients decide whether the AP list is fresh enough.
2018-02-25 14:53:44 +01:00
case PROP_LAST_SCAN:
wifi: expose LastScan as milliseconds not seconds This doesn't wrap around in 68 years of uptime and is consistent with o.fd.NM.Checkpoint.Created.
2018-06-15 13:57:30 +02:00
g_value_set_int64 (value, nm_device_wifi_get_last_scan (self));
wifi: expose the LastScan property This is the time when the last Wi-Fi scan finished. It will help clients decide whether the AP list is fresh enough.
2018-02-25 14:53:44 +01:00
break;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
static void
finalize (GObject *object)
{
NMDeviceWifiPrivate *priv = NM_DEVICE_WIFI_GET_PRIVATE (object);
g_free (priv->perm_hw_address);
G_OBJECT_CLASS (nm_device_wifi_parent_class)->finalize (object);
}
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
const NMLDBusMetaIface _nml_dbus_meta_iface_nm_device_wireless = NML_DBUS_META_IFACE_INIT_PROP (
NM_DBUS_INTERFACE_DEVICE_WIRELESS,
nm_device_wifi_get_type,
NML_DBUS_META_INTERFACE_PRIO_INSTANTIATE_HIGH,
NML_DBUS_META_IFACE_DBUS_PROPERTIES (
nm-device: expose via D-Bus the 'hw-address' property Drop device-specific 'hw-address' GObject properties which are now redundant. https://bugzilla.redhat.com/show_bug.cgi?id=1786937
2020-03-05 18:56:07 +01:00
NML_DBUS_META_PROPERTY_INIT_AO_PROP ("AccessPoints", PROP_ACCESS_POINTS, NMDeviceWifi, _priv.access_points, nm_access_point_get_type, .notify_changed_ao = _property_ao_notify_changed_access_points_cb ),
NML_DBUS_META_PROPERTY_INIT_O_PROP ("ActiveAccessPoint", PROP_ACTIVE_ACCESS_POINT, NMDeviceWifi, _priv.active_access_point, nm_access_point_get_type ),
NML_DBUS_META_PROPERTY_INIT_U ("Bitrate", PROP_BITRATE, NMDeviceWifi, _priv.bitrate ),
NML_DBUS_META_PROPERTY_INIT_FCN ("HwAddress", 0, "s", _nm_device_notify_update_prop_hw_address ),
NML_DBUS_META_PROPERTY_INIT_X ("LastScan", PROP_LAST_SCAN, NMDeviceWifi, _priv.last_scan ),
NML_DBUS_META_PROPERTY_INIT_U ("Mode", PROP_MODE, NMDeviceWifi, _priv.mode ),
NML_DBUS_META_PROPERTY_INIT_S ("PermHwAddress", PROP_PERM_HW_ADDRESS, NMDeviceWifi, _priv.perm_hw_address ),
NML_DBUS_META_PROPERTY_INIT_U ("WirelessCapabilities", PROP_WIRELESS_CAPABILITIES, NMDeviceWifi, _priv.wireless_capabilities ),
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
),
);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
static void
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
nm_device_wifi_class_init (NMDeviceWifiClass *klass)
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
{
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
GObjectClass *object_class = G_OBJECT_CLASS (klass);
NMObjectClass *nm_object_class = NM_OBJECT_CLASS (klass);
NMDeviceClass *device_class = NM_DEVICE_CLASS (klass);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
object_class->get_property = get_property;
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
object_class->finalize = finalize;
libnm: add init_dbus() virtual method to NMObject Rather than having each object type override constructed() to call _nm_object_register_properties(), have NMObject call a virtual method on the subclass to ask it to register them. Move some code around in nm-client.c and nm-object.c so that all D-Bus-related initialization happens in init_dbus(), and non-D-Bus-related stuff stays in construct(). (This simplifies the next commit.)
2014-05-15 14:24:56 -04:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_NM_OBJECT_CLASS_INIT_PRIV_PTR_DIRECT (nm_object_class, NMDeviceWifi);
_NM_OBJECT_CLASS_INIT_PROPERTY_O_FIELDS_1 (nm_object_class, NMDeviceWifiPrivate, active_access_point);
_NM_OBJECT_CLASS_INIT_PROPERTY_AO_FIELDS_1 (nm_object_class, NMDeviceWifiPrivate, access_points);
libnm: add init_dbus() virtual method to NMObject Rather than having each object type override constructed() to call _nm_object_register_properties(), have NMObject call a virtual method on the subclass to ask it to register them. Move some code around in nm-client.c and nm-object.c so that all D-Bus-related initialization happens in init_dbus(), and non-D-Bus-related stuff stays in construct(). (This simplifies the next commit.)
2014-05-15 14:24:56 -04:00
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
device_class->connection_compatible = connection_compatible;
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
device_class->get_setting_type = get_setting_type;
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDeviceWifi:perm-hw-address:
*
* The hardware (MAC) address of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_PERM_HW_ADDRESS] =
g_param_spec_string (NM_DEVICE_WIFI_PERMANENT_HW_ADDRESS, "", "",
NULL,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDeviceWifi:mode:
*
* The mode of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_MODE] =
g_param_spec_enum (NM_DEVICE_WIFI_MODE, "", "",
NM_TYPE_802_11_MODE,
NM_802_11_MODE_UNKNOWN,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDeviceWifi:bitrate:
*
* The bit rate of the device in kbit/s.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_BITRATE] =
g_param_spec_uint (NM_DEVICE_WIFI_BITRATE, "", "",
0, G_MAXUINT32, 0,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDeviceWifi:active-access-point:
*
* The active #NMAccessPoint of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_ACTIVE_ACCESS_POINT] =
g_param_spec_object (NM_DEVICE_WIFI_ACTIVE_ACCESS_POINT, "", "",
NM_TYPE_ACCESS_POINT,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDeviceWifi:wireless-capabilities:
*
* The wireless capabilities of the device.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_WIRELESS_CAPABILITIES] =
g_param_spec_flags (NM_DEVICE_WIFI_CAPABILITIES, "", "",
NM_TYPE_DEVICE_WIFI_CAPABILITIES,
NM_WIFI_DEVICE_CAP_NONE,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
gobject-introspection: made several fixes to the annotations https://bugzilla.gnome.org/show_bug.cgi?id=794658
2018-03-24 15:18:21 +00:00
* NMDeviceWifi:access-points: (type GPtrArray(NMAccessPoint))
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
*
* List of all Wi-Fi access points the device can see.
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_ACCESS_POINTS] =
g_param_spec_boxed (NM_DEVICE_WIFI_ACCESS_POINTS, "", "",
G_TYPE_PTR_ARRAY,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
wifi: expose the LastScan property This is the time when the last Wi-Fi scan finished. It will help clients decide whether the AP list is fresh enough.
2018-02-25 14:53:44 +01:00
/**
* NMDeviceWifi:last-scan:
*
* The timestamp (in CLOCK_BOOTTIME seconds) for the last finished
* network scan. A value of -1 means the device never scanned for
* access points.
*
* Since: 1.12
**/
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
obj_properties[PROP_LAST_SCAN] =
g_param_spec_int64 (NM_DEVICE_WIFI_LAST_SCAN, "", "",
-1, G_MAXINT64, -1,
G_PARAM_READABLE |
G_PARAM_STATIC_STRINGS);
wifi: expose the LastScan property This is the time when the last Wi-Fi scan finished. It will help clients decide whether the AP list is fresh enough.
2018-02-25 14:53:44 +01:00
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
_nml_dbus_meta_class_init_with_properties (object_class, &_nml_dbus_meta_iface_nm_device_wireless);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDeviceWifi::access-point-added:
* @device: the Wi-Fi device that received the signal
* @ap: the new access point
*
* Notifies that a #NMAccessPoint is added to the Wi-Fi device.
**/
signals[ACCESS_POINT_ADDED] =
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
g_signal_new ("access-point-added",
G_OBJECT_CLASS_TYPE (object_class),
G_SIGNAL_RUN_FIRST,
libnm: hide GObject structs from public API and embed private data These types are all subclasses of NMObject. These instances are commonly created by NMClient itself. It makes no sense that a user would instantiate the type. Much less does it make sense to subclass them. Hide the object and class structures from public API. This is an API and ABI break, but of something that is very likely unused. This is mainly done to embed the private structure in the object itself. This has benefits for performance and debugability. But most importantly, we can obtain a static offset where to access the private data. That means, we can use the information to access the data pointer generically, as we will need later. This is not done for the internal types NMManager, NMRemoteSettings, and NMDnsManager. These types will be dropped later.
2019-10-18 08:12:01 +02:00
0, NULL, NULL,
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
g_cclosure_marshal_VOID__OBJECT,
G_TYPE_NONE, 1,
G_TYPE_OBJECT);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
/**
* NMDeviceWifi::access-point-removed:
* @device: the Wi-Fi device that received the signal
* @ap: the removed access point
*
* Notifies that a #NMAccessPoint is removed from the Wi-Fi device.
**/
signals[ACCESS_POINT_REMOVED] =
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
g_signal_new ("access-point-removed",
G_OBJECT_CLASS_TYPE (object_class),
G_SIGNAL_RUN_FIRST,
libnm: refactor caching of D-Bus objects in NMClient No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes for the NMClient cache. Instead, use GDBusConnection directly and a custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager data. CHANGES ------- - This is a complete rework. I think the previous implementation was difficult to understand. There were unfixed bugs and nobody understood the code well enough to fix them. Maybe somebody out there understood the code, but I certainly did not. At least nobody provided patches to fix those issues. I do believe that this implementation is more straightforward and easier to understand. It removes a lot of layers of code. Whether this claim of simplicity is true, each reader must decide for himself/herself. Note that it is still fairly complex. - There was a lingering performance issue with large number of D-Bus objects. The patch tries hard that the implementation scales well. Of course, when we cache N objects that have N-to-M references to other, we still are fundamentally O(N*M) for runtime and memory consumption (with M being the number of references between objects). But each part should behave efficiently and well. - Play well with GMainContext. libnm code (NMClient) is generally not thread safe. However, it should work to use multiple instances in parallel, as long as each access to a NMClient is through the caller's GMainContext. This follows glib's style and effectively allows to use NMClient in a multi threaded scenario. This implies to stick to a main context upon construction and ensure that callbacks are only invoked when iterating that context. Also, NMClient itself shall never iterate the caller's context. This also means, libnm must never use g_idle_add() or g_timeout_add(), as those enqueue sources in the g_main_context_default() context. - Get ordering of messages right. All events are consistently enqueued in a GMainContext and processed strictly in order. For example, previously "nm-object.c" tried to combine signals and emit them on an idle handler. That is wrong, signals must be emitted in the right order and when they happen. Note that when using GInitable's synchronous initialization to initialize the NMClient instance, NMClient internally still operates fully asynchronously. In that case NMClient has an internal main context. - NMClient takes over most of the functionality. When using D-Bus' ObjectManager interface, one needs to handle basically the entire state of the D-Bus interface. That cannot be separated well into distinct parts, and even if you try, you just end up having closely related code in different source files. Spreading related code does not make it easier to understand, on the contrary. That means, NMClient is inherently complex as it contains most of the logic. I think that is not avoidable, but it's not as bad as it sounds. - NMClient processes D-Bus messages and state changes in separate steps. First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and keeps track of the changed data. Then we update the GObject instances (_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally, we emit all signals and notifications that were collected (_dbus_handle_changes_commit()). Note that for example during the initial GetManagedObjects() reply, NMClient receive a large amount of state at once. But we first apply all the changes to our GObject instances before emitting any signals. The result is that signals are always emitted in a moment when the cache is consistent. The unavoidable downside is that when you receive a property changed signal, possibly many other properties changed already and more signals are about to be emitted. - NMDeviceWifi no longer modifies the content of the cache from client side during poke_wireless_devices_with_rf_status(). The content of the cache should be determined by D-Bus alone and follow what NetworkManager service exposes. Local modifications should be avoided. - This aims to bring no API/ABI change, though it does of course bring various subtle changes in behavior. Those should be all for the better, but the goal is not to break any existing clients. This does change internal (albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER property and NMObject no longer implementing GInitableIface and GAsyncInitableIface. - Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin and NMSecretAgentOld. These are independent of NMClient/NMObject and should be reworked separately. - While we no longer use generated classes from gdbus-codegen, we don't need more glue code than before. Also before we constructed NMPropertiesInfo and a had large amount of code to propagate properties from NMDBus* to NMObject. That got completely reworked, but did not fundamentally change. You still need about the same effort to create the NMLDBusMetaIface. Not using generated bindings did not make anything worse (which tells about the usefulness of generated code, at least in the way it was used). - NMLDBusMetaIface and other meta data is static and immutable. This avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U() have compile time checks to ensure the property types matches. It's pretty hard to misuse them because it won't compile. - The meta data now explicitly encodes the expected D-Bus types and makes sure never to accept wrong data. That would only matter when the server (accidentally or intentionally) exposes unexpected types on D-Bus. I don't think that was previously ensured in all cases. For example, demarshal_generic() only cared about the GObject property type, it didn't know the expected D-Bus type. - Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those probably indicated real bugs. In any case, it prevented us from running CI with G_DEBUG=fatal-warnings, because there would be just too many unrelated crashes. Now we log debug messages that can be enabled with "LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error. Together with G_DEBUG=fatal-warnings, this turns them into assertions. Note that such "assertion failures" might also happen because of a server bug (or change). Thus these are not common assertions that indicate a bug in libnm and are thus not armed unless explicitly requested. In our CI we should now always run with LIBNM_CLIENT_DEBUG=warning,error and G_DEBUG=fatal-warnings and to catch bugs. Note that currently NetworkManager has bugs in this regard, so enabling this will result in assertion failures. That should be fixed first. - Note that this changes the order in which we emit "notify:devices" and "device-added" signals. I think it makes the most sense to emit first "device-removed", then "notify:devices", and finally "device-added" signals. This changes behavior for commit 52ae28f6e5bf ('libnm: queue added/removed signals and suppress uninitialized notifications'), but I don't think that users should actually rely on the order. Still, the new order makes the most sense to me. - In NetworkManager, profiles can be invisible to the user by setting "connection.permissions". Such profiles would be hidden by NMClient's nm_client_get_connections() and their "connection-added"/"connection-removed" signals. Note that NMActiveConnection's nm_active_connection_get_connection() and NMDevice's nm_device_get_available_connections() still exposes such hidden NMRemoteConnection instances. This behavior was preserved. NUMBERS ------- I compared 3 versions of libnm. [1] 962297f9085d, current tip of nm-1-20 branch [2] 4fad8c7c642e, current master, immediate parent of this patch [3] this patch All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31. The libraries were build with $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug Note that RPM build already stripped the library. --- N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but in these tests it had large (and bogus) ELF notes. Anyway, the point is to show the relative sizes, so it doesn't matter). [1] 4075552 (102.7%) [2] 3969624 (100.0%) [3] 3705208 ( 93.3%) --- N2) `size /usr/lib64/libnm.so.0.1.0`: text data bss dec hex filename [1] 1314569 (102.0%) 69980 ( 94.8%) 10632 ( 80.4%) 1395181 (101.4%) 1549ed /usr/lib64/libnm.so.0.1.0 [2] 1288410 (100.0%) 73796 (100.0%) 13224 (100.0%) 1375430 (100.0%) 14fcc6 /usr/lib64/libnm.so.0.1.0 [3] 1229066 ( 95.4%) 65248 ( 88.4%) 13400 (101.3%) 1307714 ( 95.1%) 13f442 /usr/lib64/libnm.so.0.1.0 --- N3) Performance test with test-client.py. With checkout of [2], run ``` prepare_checkout() { rm -rf /tmp/nm-test && \ git checkout -B test 4fad8c7c642e && \ git clean -fdx && \ ./autogen.sh --prefix=/tmp/nm-test && \ make -j 5 install && \ make -j 5 check-local-clients-tests-test-client } prepare_test() { NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v } do_test() { for i in {1..10}; do NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1 done echo "done!" } prepare_checkout prepare_test time do_test ``` [1] real 2m14.497s (101.3%) user 5m26.651s (100.3%) sys 1m40.453s (101.4%) [2] real 2m12.800s (100.0%) user 5m25.619s (100.0%) sys 1m39.065s (100.0%) [3] real 1m54.915s ( 86.5%) user 4m18.585s ( 79.4%) sys 1m32.066s ( 92.9%) --- N4) Performance. Run NetworkManager from build [2] and setup a large number of profiles (551 profiles and 515 devices, mostly unrealized). This setup is already at the edge of what NetworkManager currently can handle. Of course, that is a different issue. Here we just check how long plain `nmcli` takes on the system. ``` do_cleanup() { for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do nmcli connection delete uuid "$UUID" done for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do nmcli device delete "$DEVICE" done } do_setup() { do_cleanup for i in {1..30}; do nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore for j in $(seq $i 30); do nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i ipv4.method disabled ipv6.method ignore done done systemctl restart NetworkManager.service sleep 5 } do_test() { perf stat -r 50 -B nmcli 1>/dev/null } do_test ``` [1] Performance counter stats for 'nmcli' (50 runs): 456.33 msec task-clock:u # 1.093 CPUs utilized ( +- 0.44% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,900 page-faults:u # 0.013 M/sec ( +- 0.02% ) 1,408,675,453 cycles:u # 3.087 GHz ( +- 0.48% ) 1,594,741,060 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 368,744,018 branches:u # 808.061 M/sec ( +- 0.02% ) 4,566,058 branch-misses:u # 1.24% of all branches ( +- 0.76% ) 0.41761 +- 0.00282 seconds time elapsed ( +- 0.68% ) [2] Performance counter stats for 'nmcli' (50 runs): 477.99 msec task-clock:u # 1.088 CPUs utilized ( +- 0.36% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 5,948 page-faults:u # 0.012 M/sec ( +- 0.03% ) 1,471,133,482 cycles:u # 3.078 GHz ( +- 0.36% ) 1,655,275,369 instructions:u # 1.13 insn per cycle ( +- 0.02% ) 382,595,152 branches:u # 800.433 M/sec ( +- 0.02% ) 4,746,070 branch-misses:u # 1.24% of all branches ( +- 0.49% ) 0.43923 +- 0.00242 seconds time elapsed ( +- 0.55% ) [3] Performance counter stats for 'nmcli' (50 runs): 352.36 msec task-clock:u # 1.027 CPUs utilized ( +- 0.32% ) 0 context-switches:u # 0.000 K/sec 0 cpu-migrations:u # 0.000 K/sec 4,790 page-faults:u # 0.014 M/sec ( +- 0.26% ) 1,092,341,186 cycles:u # 3.100 GHz ( +- 0.26% ) 1,209,045,283 instructions:u # 1.11 insn per cycle ( +- 0.02% ) 281,708,462 branches:u # 799.499 M/sec ( +- 0.01% ) 3,101,031 branch-misses:u # 1.10% of all branches ( +- 0.61% ) 0.34296 +- 0.00120 seconds time elapsed ( +- 0.35% ) --- N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`: [1] Command being timed: "nmcli" User time (seconds): 0.42 System time (seconds): 0.04 Percent of CPU this job got: 107% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34456 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6128 Voluntary context switches: 1298 Involuntary context switches: 1106 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [2] Command being timed: "nmcli" User time (seconds): 0.44 System time (seconds): 0.04 Percent of CPU this job got: 108% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 34452 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 6169 Voluntary context switches: 1849 Involuntary context switches: 142 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 [3] Command being timed: "nmcli" User time (seconds): 0.32 System time (seconds): 0.02 Percent of CPU this job got: 102% Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34 Average shared text size (kbytes): 0 Average unshared data size (kbytes): 0 Average stack size (kbytes): 0 Average total size (kbytes): 0 Maximum resident set size (kbytes): 29196 Average resident set size (kbytes): 0 Major (requiring I/O) page faults: 0 Minor (reclaiming a frame) page faults: 5059 Voluntary context switches: 919 Involuntary context switches: 685 Swaps: 0 File system inputs: 0 File system outputs: 0 Socket messages sent: 0 Socket messages received: 0 Signals delivered: 0 Page size (bytes): 4096 Exit status: 0 --- N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for the RSS size. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND [1] me 1492900 21.0 0.2 461348 33248 pts/10 Sl+ 15:02 0:00 nmcli monitor [2] me 1490721 5.0 0.2 461496 33548 pts/10 Sl+ 15:00 0:00 nmcli monitor [3] me 1495801 16.5 0.1 459476 28692 pts/10 Sl+ 15:04 0:00 nmcli monitor
2019-10-30 11:42:58 +01:00
0, NULL, NULL,
libnm: use obj_properties array in libnm and cleanup This is not merely cosmetic. I will need the obj_properties array to lookup GParamSpec by their PROP_* enum value. The alternative would be lookup by name, which is more expensive.
2019-10-10 17:57:30 +02:00
g_cclosure_marshal_VOID__OBJECT,
G_TYPE_NONE, 1,
G_TYPE_OBJECT);
libnm: add libnm/libnm-core (part 1) This commit begins creating the new "libnm", which will replace libnm-util and libnm-glib. The main reason for the libnm-util/libnm-glib split is that the daemon needs to link to libnm-util (to get NMSettings, NMConnection, etc), but can't link to libnm-glib (because it uses many of the same type names as the NetworkManager daemon. eg, NMDevice). So the daemon links to only libnm-util, but basically all clients link to both. With libnm, there will be only a single client-visible library, and NetworkManager will internally link against a private "libnm-core" containing the parts that used to be in libnm-util. (The "libnm-core" parts still need to be in their own directory so that the daemon can see those header files without also seeing the ones in libnm/ that conflict with its own headers.) [This commit just copies the source code from libnm-util/ to libnm-core/, and libnm-glib/ to libnm/: mkdir -p libnm-core/tests/ mkdir -p libnm/tests/ cp libnm-util/*.[ch] libnm-util/nm-version.h.in libnm-core/ rm -f libnm-core/nm-version.h libnm-core/nm-setting-template.[ch] libnm-core/nm-utils-enum-types.[ch] cp libnm-util/tests/*.[ch] libnm-core/tests/ cp libnm-glib/*.[ch] libnm/ rm -f libnm/libnm_glib.[ch] libnm/libnm-glib-test.c libnm/nm-glib-enum-types.[ch] cp libnm-glib/tests/*.[ch] libnm/tests/ ]
2014-07-24 08:53:33 -04:00
}
Reference in a new issue Copy permalink