Commit graph

874 commits

Author SHA1 Message Date
Beniamino Galvani
4bcdc3c1eb n-dhcp4: allow calling listen() on already listening connection
When the client enters the INIT state, it calls listen() on the
connection connection to create the packet socket. However, if the
client is coming from the REBOOTING state after a NAK, the connection
is already in the listening state; do nothing in such case.
2020-01-09 09:04:08 +01:00
Thomas Haller
d964decbbd libnm/keyfile: build keyfile code as separate GPL licensed internal library
Keyfile support was initially added under GPL-2.0+ license as part of
core. It was moved to "libnm-core" in commit 59eb5312a5 ('keyfile: merge
branch 'th/libnm-keyfile-bgo744699'').

"libnm-core" is statically linked with by core and "libnm". In
the former case under terms of GPL-2.0+ (good) and in the latter case
under terms of LGPL-2.1+ (bad).

In fact, to this day, "libnm" doesn't actually use the code. The linker
will probably remove all the GPL-2.0+ symbols when compiled with
gc-sections or LTO. Still, linking them together in the first place
makes "libnm" only available under GPL code (despite the code
not actually being used).

Instead, move the GPL code to a separate static library
"shared/nm-keyfile/libnm-keyfile.la" and only link it to the part
that actually uses the code (and which is GPL licensed too).

This fixes the license violation.

Eventually, it would be very useful to be able to expose keyfile
handling via "libnm". However that is not straight forward due to the
licensing conflict.

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/381
2020-01-07 13:17:47 +01:00
Thomas Haller
cb7cfc3f12 shared: don't allow NULL arguments with g_hash_table_steal_extended() compat implementation
We cannot know the key/value free functions, hence, our compat implementation
cannot free the values if they are not requested. The "solution" is to require
the caller to fetch all values, always.
2019-12-31 02:13:45 +01:00
Antonio Cardace
1e45865e4f shared: nm-auth-subject: add unix-session type 2019-12-24 10:13:51 +01:00
Antonio Cardace
0f7994328d shared: move nm-dbus-auth-subject to shared/nm-libnm-core-intern
Move it to shared as it's useful for clients as well.

Move and rename nm_dbus_manager_new_auth_subject_from_context() and
nm_dbus_manager_new_auth_subject_from_message() in nm-dbus-manager.c
as they're needed there.
2019-12-24 10:13:51 +01:00
Antonio Cardace
c0f1a657c3 shared: add io-util to read data from a fd into a GString 2019-12-24 10:13:51 +01:00
Beniamino Galvani
36f8822c9b n-dhcp4: handle invalid return codes gracefully
Instead of terminating the program when the dispatch function returns
an invalid return code, log an error message and convert the error
code to a valid, generic one.

https://bugs.archlinux.org/task/64880
2019-12-23 16:19:35 +01:00
Beniamino Galvani
f860e929c0 n-dhcp4: use packet socket in rebinding state
After t1, the client tries to renew the lease by contacting via the
udp socket the server specified in the server-id option. If this
fails, after t2 it tries to contact any server using broadcast. For
this to work, the packet socket must be used.
2019-12-23 15:42:09 +01:00
Beniamino Galvani
af03b77980 n-dhcp4: support init-reboot state
Currently the client always starts from the INIT state (i.e. sending a
discover message). If a requested-ip was specified by the caller, it
is added as an option in the discover.

It was reported that some DHCP servers don't respond to discover
messages with the requested-ip option set [1][2].

The RFC allows to skip the discover by entering the INIT-REBOOT state
and starting directly with a broadcast request message containing the
requested IP address. Implement that.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1781856
[2] https://gitlab.freedesktop.org/NetworkManager/NetworkManager/issues/310
2019-12-23 15:42:09 +01:00
Beniamino Galvani
30798e0af4 n-dhcp4: fix logging broadcast messages
Log the broadcast address instead of the server IP as destination when
needed.
2019-12-23 15:42:09 +01:00
Thomas Haller
27d51c3250 shared/glib: add compat implementation for g_hash_table_steal_extended() 2019-12-21 12:30:37 +01:00
Thomas Haller
785da51d83 core: add and indicate NM_CAPABILITY_OVS capability on D-Bus
https://bugzilla.redhat.com/show_bug.cgi?id=1785147
2019-12-21 11:33:59 +01:00
Beniamino Galvani
0de4fd6ebc shared: add missing va_end() to _nm_dbus_error_is()
Found by covscan:

   NetworkManager-1.22.0/shared/nm-glib-aux/nm-dbus-aux.c:361:
   missing_va_end: va_end was not called for "ap".

Fixes: ce36494c0a ('shared: add nm_dbus_error_is() helper')
2019-12-18 09:54:08 +01:00
Thomas Haller
1b00fd2fd2 shared: implement _LOGx() macros using log levels that are themself defines
"nm-glib-aux/nm-logging-fwd.h" provides macros like _LOGD() to be reused
by various parts which implement logging (by defining _NMLOG() accordingly).

libnm also has logging, however it uses different logging levels
aside LOGD_DEBUG.

Instead, implement _LOGD() using a define _LOGL_DEBUG, so that libnm can
redefine thos _LOGL_DEBUG defines and use the _LOGD() macro.
2019-12-16 18:37:47 +01:00
Thomas Haller
ce36494c0a shared: add nm_dbus_error_is() helper 2019-12-16 18:16:12 +01:00
Thomas Haller
4a3ca7115a all: fix wrong "gs_free GError *" declarations
This is a bug and leads either to a leak or a crash.
2019-12-16 17:42:23 +01:00
Thomas Haller
9e02a67619 systemd: merge branch systemd into master 2019-12-16 10:22:09 +01:00
Thomas Haller
0d155d1821 systemd: update code from upstream (2019-12-13)
This is a direct dump from systemd git.

======

SYSTEMD_DIR=../systemd
COMMIT=c8bf87b3399a3dd0b17fd0003b9797635b161ee0

(
  cd "$SYSTEMD_DIR"
  git checkout "$COMMIT"
  git reset --hard
  git clean -fdx
)

git ls-files -z :/src/systemd/src/ \
                :/shared/systemd/src/ \
                :/shared/nm-std-aux/unaligned.h | \
  xargs -0 rm -f

nm_copy_sd_shared() {
    mkdir -p "./shared/systemd/$(dirname "$1")"
    cp "$SYSTEMD_DIR/$1" "./shared/systemd/$1"
}

nm_copy_sd_core() {
    mkdir -p "./src/systemd/$(dirname "$1")"
    cp "$SYSTEMD_DIR/$1" "./src/systemd/$1"
}

nm_copy_sd_stdaux() {
    mkdir -p "./shared/nm-std-aux/"
    cp "$SYSTEMD_DIR/$1" "./shared/nm-std-aux/${1##*/}"
}

nm_copy_sd_core "src/libsystemd-network/arp-util.c"
nm_copy_sd_core "src/libsystemd-network/arp-util.h"
nm_copy_sd_core "src/libsystemd-network/dhcp-identifier.c"
nm_copy_sd_core "src/libsystemd-network/dhcp-identifier.h"
nm_copy_sd_core "src/libsystemd-network/dhcp-internal.h"
nm_copy_sd_core "src/libsystemd-network/dhcp-lease-internal.h"
nm_copy_sd_core "src/libsystemd-network/dhcp-network.c"
nm_copy_sd_core "src/libsystemd-network/dhcp-option.c"
nm_copy_sd_core "src/libsystemd-network/dhcp-packet.c"
nm_copy_sd_core "src/libsystemd-network/dhcp-protocol.h"
nm_copy_sd_core "src/libsystemd-network/dhcp6-internal.h"
nm_copy_sd_core "src/libsystemd-network/dhcp6-lease-internal.h"
nm_copy_sd_core "src/libsystemd-network/dhcp6-network.c"
nm_copy_sd_core "src/libsystemd-network/dhcp6-option.c"
nm_copy_sd_core "src/libsystemd-network/dhcp6-protocol.h"
nm_copy_sd_core "src/libsystemd-network/lldp-internal.h"
nm_copy_sd_core "src/libsystemd-network/lldp-neighbor.c"
nm_copy_sd_core "src/libsystemd-network/lldp-neighbor.h"
nm_copy_sd_core "src/libsystemd-network/lldp-network.c"
nm_copy_sd_core "src/libsystemd-network/lldp-network.h"
nm_copy_sd_core "src/libsystemd-network/network-internal.c"
nm_copy_sd_core "src/libsystemd-network/network-internal.h"
nm_copy_sd_core "src/libsystemd-network/sd-dhcp-client.c"
nm_copy_sd_core "src/libsystemd-network/sd-dhcp-lease.c"
nm_copy_sd_core "src/libsystemd-network/sd-dhcp6-client.c"
nm_copy_sd_core "src/libsystemd-network/sd-dhcp6-lease.c"
nm_copy_sd_core "src/libsystemd-network/sd-ipv4acd.c"
nm_copy_sd_core "src/libsystemd-network/sd-ipv4ll.c"
nm_copy_sd_core "src/libsystemd-network/sd-lldp.c"
nm_copy_sd_core "src/libsystemd/sd-event/event-source.h"
nm_copy_sd_core "src/libsystemd/sd-event/event-util.c"
nm_copy_sd_core "src/libsystemd/sd-event/event-util.h"
nm_copy_sd_core "src/libsystemd/sd-event/sd-event.c"
nm_copy_sd_core "src/libsystemd/sd-id128/id128-util.c"
nm_copy_sd_core "src/libsystemd/sd-id128/id128-util.h"
nm_copy_sd_core "src/libsystemd/sd-id128/sd-id128.c"
nm_copy_sd_core "src/systemd/_sd-common.h"
nm_copy_sd_core "src/systemd/sd-dhcp-client.h"
nm_copy_sd_core "src/systemd/sd-dhcp-lease.h"
nm_copy_sd_core "src/systemd/sd-dhcp-option.h"
nm_copy_sd_core "src/systemd/sd-dhcp6-client.h"
nm_copy_sd_core "src/systemd/sd-dhcp6-lease.h"
nm_copy_sd_core "src/systemd/sd-event.h"
nm_copy_sd_core "src/systemd/sd-id128.h"
nm_copy_sd_core "src/systemd/sd-ipv4acd.h"
nm_copy_sd_core "src/systemd/sd-ipv4ll.h"
nm_copy_sd_core "src/systemd/sd-lldp.h"
nm_copy_sd_core "src/systemd/sd-ndisc.h"
nm_copy_sd_shared "src/basic/alloc-util.c"
nm_copy_sd_shared "src/basic/alloc-util.h"
nm_copy_sd_shared "src/basic/async.h"
nm_copy_sd_shared "src/basic/env-file.c"
nm_copy_sd_shared "src/basic/env-file.h"
nm_copy_sd_shared "src/basic/env-util.c"
nm_copy_sd_shared "src/basic/env-util.h"
nm_copy_sd_shared "src/basic/errno-util.h"
nm_copy_sd_shared "src/basic/escape.c"
nm_copy_sd_shared "src/basic/escape.h"
nm_copy_sd_shared "src/basic/ether-addr-util.c"
nm_copy_sd_shared "src/basic/ether-addr-util.h"
nm_copy_sd_shared "src/basic/extract-word.c"
nm_copy_sd_shared "src/basic/extract-word.h"
nm_copy_sd_shared "src/basic/fd-util.c"
nm_copy_sd_shared "src/basic/fd-util.h"
nm_copy_sd_shared "src/basic/fileio.c"
nm_copy_sd_shared "src/basic/fileio.h"
nm_copy_sd_shared "src/basic/format-util.c"
nm_copy_sd_shared "src/basic/format-util.h"
nm_copy_sd_shared "src/basic/fs-util.c"
nm_copy_sd_shared "src/basic/fs-util.h"
nm_copy_sd_shared "src/basic/hash-funcs.c"
nm_copy_sd_shared "src/basic/hash-funcs.h"
nm_copy_sd_shared "src/basic/hashmap.c"
nm_copy_sd_shared "src/basic/hashmap.h"
nm_copy_sd_shared "src/basic/hexdecoct.c"
nm_copy_sd_shared "src/basic/hexdecoct.h"
nm_copy_sd_shared "src/basic/hostname-util.c"
nm_copy_sd_shared "src/basic/hostname-util.h"
nm_copy_sd_shared "src/basic/in-addr-util.c"
nm_copy_sd_shared "src/basic/in-addr-util.h"
nm_copy_sd_shared "src/basic/io-util.c"
nm_copy_sd_shared "src/basic/io-util.h"
nm_copy_sd_shared "src/basic/list.h"
nm_copy_sd_shared "src/basic/log.h"
nm_copy_sd_shared "src/basic/macro.h"
nm_copy_sd_shared "src/basic/memory-util.c"
nm_copy_sd_shared "src/basic/memory-util.h"
nm_copy_sd_shared "src/basic/mempool.c"
nm_copy_sd_shared "src/basic/mempool.h"
nm_copy_sd_shared "src/basic/missing_fcntl.h"
nm_copy_sd_shared "src/basic/missing_random.h"
nm_copy_sd_shared "src/basic/missing_socket.h"
nm_copy_sd_shared "src/basic/missing_stat.h"
nm_copy_sd_shared "src/basic/missing_syscall.h"
nm_copy_sd_shared "src/basic/missing_type.h"
nm_copy_sd_shared "src/basic/parse-util.c"
nm_copy_sd_shared "src/basic/parse-util.h"
nm_copy_sd_shared "src/basic/path-util.c"
nm_copy_sd_shared "src/basic/path-util.h"
nm_copy_sd_shared "src/basic/prioq.c"
nm_copy_sd_shared "src/basic/prioq.h"
nm_copy_sd_shared "src/basic/process-util.c"
nm_copy_sd_shared "src/basic/process-util.h"
nm_copy_sd_shared "src/basic/random-util.c"
nm_copy_sd_shared "src/basic/random-util.h"
nm_copy_sd_shared "src/basic/set.h"
nm_copy_sd_shared "src/basic/signal-util.c"
nm_copy_sd_shared "src/basic/signal-util.h"
nm_copy_sd_shared "src/basic/siphash24.h"
nm_copy_sd_shared "src/basic/socket-util.c"
nm_copy_sd_shared "src/basic/socket-util.h"
nm_copy_sd_shared "src/basic/sort-util.h"
nm_copy_sd_shared "src/basic/sparse-endian.h"
nm_copy_sd_shared "src/basic/stat-util.c"
nm_copy_sd_shared "src/basic/stat-util.h"
nm_copy_sd_shared "src/basic/stdio-util.h"
nm_copy_sd_shared "src/basic/string-table.c"
nm_copy_sd_shared "src/basic/string-table.h"
nm_copy_sd_shared "src/basic/string-util.c"
nm_copy_sd_shared "src/basic/string-util.h"
nm_copy_sd_shared "src/basic/strv.c"
nm_copy_sd_shared "src/basic/strv.h"
nm_copy_sd_shared "src/basic/strxcpyx.c"
nm_copy_sd_shared "src/basic/strxcpyx.h"
nm_copy_sd_shared "src/basic/time-util.c"
nm_copy_sd_shared "src/basic/time-util.h"
nm_copy_sd_shared "src/basic/tmpfile-util.c"
nm_copy_sd_shared "src/basic/tmpfile-util.h"
nm_copy_sd_shared "src/basic/umask-util.h"
nm_copy_sd_shared "src/basic/utf8.c"
nm_copy_sd_shared "src/basic/utf8.h"
nm_copy_sd_shared "src/basic/util.c"
nm_copy_sd_shared "src/basic/util.h"
nm_copy_sd_shared "src/shared/dns-domain.c"
nm_copy_sd_shared "src/shared/dns-domain.h"
nm_copy_sd_stdaux "src/basic/unaligned.h"
2019-12-15 15:14:48 +01:00
Thomas Haller
a1771c738d client: fallback to CLOCK_MONOTONIC for timerfd
RHEL7 supports clock_gettime(CLOCK_BOOTIME), but it does not support
timerfd_create(CLOCK_BOOTIME). Creating a timerfd will fail with EINVAL.
Fallback to CLOCK_MONOTONIC.

Compare this to n-acd which also has compatibility code to fallback to
CLOCK_MONOTONIC. However when n-acd falls back to CLOCK_MONOTONIC, it uses
monotonic clock also for clock_gettime().

For n-dhcp4, the timestamps are also exposed in the public API
(n_dhcp4_client_lease_get_lifetime()). Hence, for timestamps n-dhcp4
still uses and requires clock_gettime(CLOCK_BOOTIME). Only the internal
timeout handling with the timerfd falls back to CLOCK_MONOTONIC.

https://github.com/nettools/n-dhcp4/pull/13
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/merge_requests/362
2019-12-15 13:50:45 +01:00
Thomas Haller
bd9b253540 all: rename time related function to spell out nsec/usec/msec/sec
The abbreviations "ns" and "ms" seem not very clear to me. Spell them
out to nsec/msec. Also, in parts we already used the longer abbreviations,
so it wasn't consistent.
2019-12-13 16:54:40 +01:00
Thomas Haller
f5b0713651 shared/glib: reimplement g_atomic_pointer_compare_and_exchange() macro
With glib 2.63.2 and clang 9.0.0 (Fedora 32) we get compile errors:

  ../clients/cloud-setup/nmcs-provider-ec2.c:51:8: error: incompatible pointer types passing 'typeof ((((void *)0))) *' (aka 'void **') to parameter of type 'const char **' [-Werror,-Wincompatible-pointer-types]
                  if (!g_atomic_pointer_compare_and_exchange (&base_cached, NULL, base))
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /usr/include/glib-2.0/glib/gatomic.h:192:44: note: expanded from macro 'g_atomic_pointer_compare_and_exchange'
      __atomic_compare_exchange_n ((atomic), &gapcae_oldval, (newval), FALSE, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST) ? TRUE : FALSE; \
                                             ^~~~~~~~~~~~~~
  ../src/devices/bluetooth/nm-bluez-manager.c:2836:2: error: incompatible pointer types passing 'typeof ((((void *)0))) *' (aka 'void **') to parameter of type 'const NMBtVTableNetworkServer **' (aka 'const struct _NMBtVTableNetworkServer **') [-Werror,-Wincompatible-pointer-types]
          g_atomic_pointer_compare_and_exchange (&nm_bt_vtable_network_server, NULL, &priv->vtable_network_server);
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /usr/include/glib-2.0/glib/gatomic.h:192:44: note: expanded from macro 'g_atomic_pointer_compare_and_exchange'
      __atomic_compare_exchange_n ((atomic), &gapcae_oldval, (newval), FALSE, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST) ? TRUE : FALSE; \
                                             ^~~~~~~~~~~~~~
  ../src/devices/bluetooth/nm-bluez-manager.c:2853:2: error: passing 'typeof ((&priv->vtable_network_server)) *' (aka 'struct _NMBtVTableNetworkServer **') to parameter of type 'const NMBtVTableNetworkServer **' (aka 'const struct _NMBtVTableNetworkServer **') discards qualifiers in nested pointer types [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
          g_atomic_pointer_compare_and_exchange (&nm_bt_vtable_network_server, &priv->vtable_network_server, NULL);
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /usr/include/glib-2.0/glib/gatomic.h:192:44: note: expanded from macro 'g_atomic_pointer_compare_and_exchange'
      __atomic_compare_exchange_n ((atomic), &gapcae_oldval, (newval), FALSE, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST) ? TRUE : FALSE; \
                                             ^~~~~~~~~~~~~~
  ../src/devices/nm-device.c:8857:8: error: incompatible pointer types passing 'typeof ((((void *)0))) *' (aka 'void **') to parameter of type 'GBytes **' (aka 'struct _GBytes **') [-Werror,-Wincompatible-pointer-types]
                  if (!g_atomic_pointer_compare_and_exchange (&global_duid, NULL, p)) {
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /usr/include/glib-2.0/glib/gatomic.h:192:44: note: expanded from macro 'g_atomic_pointer_compare_and_exchange'
      __atomic_compare_exchange_n ((atomic), &gapcae_oldval, (newval), FALSE, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST) ? TRUE : FALSE; \
                                             ^~~~~~~~~~~~~~

The issue happens because glib passes the "atomic" argument to

  __atomic_compare_exchange_n ((atomic), &gapcae_oldval, (newval), FALSE, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST)

without cast, and clang 9 seems to be picky about const pointers.
Add our own version of the macro that does better casts while also having
better compile time checks for valid arguments.
2019-12-13 13:46:37 +01:00
Thomas Haller
53b74bc614 n-dhcp4/socket: use SO_REUSEADDR on UDP socket
Otherwise, other applications cannot bind to port 0.0.0.0:68 at the same time.
This is for example what dhclient wants to do. So even when running
dhclient on another, unrelated interface, it would fail to bind the UDP
socket and quit.

Note that also systemd-networkd's DHCPv4 client sets this socket option.
Presumably for the same reasons.

Signed-off-by: Thomas Haller <thaller@redhat.com>

https://github.com/nettools/n-dhcp4/pull/12
2019-12-11 09:24:14 +01:00
Thomas Haller
cb755afa24 clients: in debug builds randomly use sync init of NMClient for testing
This affects nmtui and nm-cloud-setup, which use nmc_client_new_waitsync().
2019-12-10 09:17:17 +01:00
Thomas Haller
9e10b4f699 shared: add nmc_client_new_waitsync() and nmc_client_new_async*() helpers
Benefits:

- nmc_client_new_async*() allows to set properties on the NMClient
  instance before calling g_async_initable_init_async().
  It also allows to subscribe to any signals (like NM_CLIENT_DEVICE_ADDED)
  before actually iterating the GMainContext. This is a sensible and
  supported thing to do!

- nmc_client_new_waitsync() iterates the GMainContext until the (async)
  initialization is complete. That is different from synchronous nm_client_new(),
  which does not iterate the caller's GMainContext, and hence needs an
  internal context to ensure the order of events is honored.

- nmc_client_new_waitsync() always returns the NMClient instance, even
  if initialization fails.
  That is useful if you need the nm_client_get_context_busy_watcher() instance
  to ensure all pending messages are completed.
2019-12-10 09:17:17 +01:00
Thomas Haller
3843e0c87d shared: add "shared/nm-libnm-aux" static library
We have "shared/nm-libnm-core-aux", which is shared code that can be used
by anybody (including libnm-core, src, libnm and clients).

We have "clients/common", which are helper function for clients. But
that implies that the code is inside "clients". I think it would be
useful to have auxiliary code that extends libnm, but is not only
usable by code in "clients". In other words, "shared/nm-libnm-aux"
is a better place than "clients/common", and I think most of the
functionality form "clients/common" should move there.
2019-12-10 09:17:17 +01:00
Thomas Haller
44c5331e29 shared: move "shared/nm-utils/tests/test-shared-general" to "shared/nm-glib-aux/tests"
"shared/nm-utils" got long renamed and split into separate parts. The remaining
tests are really to test nm-std-aux and nm-glib-aux (no libnm dependencies). Move
the tests to the appropriate place.
2019-12-10 09:17:17 +01:00
Thomas Haller
dab61a919b shared: add nm_client_permission_result_to_string() helper 2019-12-10 07:53:25 +01:00
Thomas Haller
b7462b1910 libnm,shared: move nm_permission_result_to_client() to shared's nm_client_permission_result_from_string() 2019-12-10 07:53:25 +01:00
Thomas Haller
09e17888f7 libnm: add mapping functions between string and NMClientPermission enum 2019-12-10 07:53:25 +01:00
Thomas Haller
8fbf67d138 shared: add nm_utils_parse_inaddr_bin_full() to support legacy IPv4 formats as inet_aton()
inet_aton() also supports IPv4 addresses in octal (with a leading '0')
or where not all 4 digits of the address are present.

Add nm_utils_parse_inaddr_bin_full() to optionally fallback to
parse the address with inet_aton().

Note taht inet_aton() also supports all crazy formats, including
ignoring trailing garbage after a whitespace. We don't want to accept
that in general.

Note that even in legacy format we:

  - accept everything that inet_pton() would accept

  - additionally, we also accept some forms which inet_aton() would
    accept, but not all.

That means, the legacy format that we accept is a superset of
inet_pton() and a subset of inet_aton(). Which is desirable.
2019-12-05 12:36:13 +01:00
Thomas Haller
06a976358b shared: add nm_utils_addr_family_from_size() helper 2019-12-05 12:36:13 +01:00
Thomas Haller
0c06a0f368 release: bump version to 1.23.0 (development) 2019-11-29 15:46:22 +01:00
Thomas Haller
d345c4fd69 systemd: merge branch systemd into master
Do another import, shortly before re-release.

There are no actual changes, but as always: to find out
that there are no changes requires large part of the work of
just doing the reimport.

Also, systemd import branch was rebased recently, that means
git-merge does not get this reimport right automatically (because
it thinks that the changes on master should be reverted). Hence,
this reimport required more care. Do it while there are few
changes.
2019-11-29 13:14:05 +01:00
Thomas Haller
71adddb116 systemd: update code from upstream (2019-11-29)
This is a direct dump from systemd git.

======

SYSTEMD_DIR=../systemd
COMMIT=dd08aa6488543727375d7377505a5333bb9e6047

(
  cd "$SYSTEMD_DIR"
  git checkout "$COMMIT"
  git reset --hard
  git clean -fdx
)

git ls-files -z :/src/systemd/src/ \
                :/shared/systemd/src/ \
                :/shared/nm-std-aux/unaligned.h | \
  xargs -0 rm -f

nm_copy_sd_shared() {
    mkdir -p "./shared/systemd/$(dirname "$1")"
    cp "$SYSTEMD_DIR/$1" "./shared/systemd/$1"
}

nm_copy_sd_core() {
    mkdir -p "./src/systemd/$(dirname "$1")"
    cp "$SYSTEMD_DIR/$1" "./src/systemd/$1"
}

nm_copy_sd_stdaux() {
    mkdir -p "./shared/nm-std-aux/"
    cp "$SYSTEMD_DIR/$1" "./shared/nm-std-aux/${1##*/}"
}

nm_copy_sd_core "src/libsystemd-network/arp-util.c"
nm_copy_sd_core "src/libsystemd-network/arp-util.h"
nm_copy_sd_core "src/libsystemd-network/dhcp-identifier.c"
nm_copy_sd_core "src/libsystemd-network/dhcp-identifier.h"
nm_copy_sd_core "src/libsystemd-network/dhcp-internal.h"
nm_copy_sd_core "src/libsystemd-network/dhcp-lease-internal.h"
nm_copy_sd_core "src/libsystemd-network/dhcp-network.c"
nm_copy_sd_core "src/libsystemd-network/dhcp-option.c"
nm_copy_sd_core "src/libsystemd-network/dhcp-packet.c"
nm_copy_sd_core "src/libsystemd-network/dhcp-protocol.h"
nm_copy_sd_core "src/libsystemd-network/dhcp6-internal.h"
nm_copy_sd_core "src/libsystemd-network/dhcp6-lease-internal.h"
nm_copy_sd_core "src/libsystemd-network/dhcp6-network.c"
nm_copy_sd_core "src/libsystemd-network/dhcp6-option.c"
nm_copy_sd_core "src/libsystemd-network/dhcp6-protocol.h"
nm_copy_sd_core "src/libsystemd-network/lldp-internal.h"
nm_copy_sd_core "src/libsystemd-network/lldp-neighbor.c"
nm_copy_sd_core "src/libsystemd-network/lldp-neighbor.h"
nm_copy_sd_core "src/libsystemd-network/lldp-network.c"
nm_copy_sd_core "src/libsystemd-network/lldp-network.h"
nm_copy_sd_core "src/libsystemd-network/network-internal.c"
nm_copy_sd_core "src/libsystemd-network/network-internal.h"
nm_copy_sd_core "src/libsystemd-network/sd-dhcp-client.c"
nm_copy_sd_core "src/libsystemd-network/sd-dhcp-lease.c"
nm_copy_sd_core "src/libsystemd-network/sd-dhcp6-client.c"
nm_copy_sd_core "src/libsystemd-network/sd-dhcp6-lease.c"
nm_copy_sd_core "src/libsystemd-network/sd-ipv4acd.c"
nm_copy_sd_core "src/libsystemd-network/sd-ipv4ll.c"
nm_copy_sd_core "src/libsystemd-network/sd-lldp.c"
nm_copy_sd_core "src/libsystemd/sd-event/event-source.h"
nm_copy_sd_core "src/libsystemd/sd-event/event-util.c"
nm_copy_sd_core "src/libsystemd/sd-event/event-util.h"
nm_copy_sd_core "src/libsystemd/sd-event/sd-event.c"
nm_copy_sd_core "src/libsystemd/sd-id128/id128-util.c"
nm_copy_sd_core "src/libsystemd/sd-id128/id128-util.h"
nm_copy_sd_core "src/libsystemd/sd-id128/sd-id128.c"
nm_copy_sd_core "src/systemd/_sd-common.h"
nm_copy_sd_core "src/systemd/sd-dhcp-client.h"
nm_copy_sd_core "src/systemd/sd-dhcp-lease.h"
nm_copy_sd_core "src/systemd/sd-dhcp-option.h"
nm_copy_sd_core "src/systemd/sd-dhcp6-client.h"
nm_copy_sd_core "src/systemd/sd-dhcp6-lease.h"
nm_copy_sd_core "src/systemd/sd-event.h"
nm_copy_sd_core "src/systemd/sd-id128.h"
nm_copy_sd_core "src/systemd/sd-ipv4acd.h"
nm_copy_sd_core "src/systemd/sd-ipv4ll.h"
nm_copy_sd_core "src/systemd/sd-lldp.h"
nm_copy_sd_core "src/systemd/sd-ndisc.h"
nm_copy_sd_shared "src/basic/alloc-util.c"
nm_copy_sd_shared "src/basic/alloc-util.h"
nm_copy_sd_shared "src/basic/async.h"
nm_copy_sd_shared "src/basic/env-file.c"
nm_copy_sd_shared "src/basic/env-file.h"
nm_copy_sd_shared "src/basic/env-util.c"
nm_copy_sd_shared "src/basic/env-util.h"
nm_copy_sd_shared "src/basic/errno-util.h"
nm_copy_sd_shared "src/basic/escape.c"
nm_copy_sd_shared "src/basic/escape.h"
nm_copy_sd_shared "src/basic/ether-addr-util.c"
nm_copy_sd_shared "src/basic/ether-addr-util.h"
nm_copy_sd_shared "src/basic/extract-word.c"
nm_copy_sd_shared "src/basic/extract-word.h"
nm_copy_sd_shared "src/basic/fd-util.c"
nm_copy_sd_shared "src/basic/fd-util.h"
nm_copy_sd_shared "src/basic/fileio.c"
nm_copy_sd_shared "src/basic/fileio.h"
nm_copy_sd_shared "src/basic/format-util.c"
nm_copy_sd_shared "src/basic/format-util.h"
nm_copy_sd_shared "src/basic/fs-util.c"
nm_copy_sd_shared "src/basic/fs-util.h"
nm_copy_sd_shared "src/basic/hash-funcs.c"
nm_copy_sd_shared "src/basic/hash-funcs.h"
nm_copy_sd_shared "src/basic/hashmap.c"
nm_copy_sd_shared "src/basic/hashmap.h"
nm_copy_sd_shared "src/basic/hexdecoct.c"
nm_copy_sd_shared "src/basic/hexdecoct.h"
nm_copy_sd_shared "src/basic/hostname-util.c"
nm_copy_sd_shared "src/basic/hostname-util.h"
nm_copy_sd_shared "src/basic/in-addr-util.c"
nm_copy_sd_shared "src/basic/in-addr-util.h"
nm_copy_sd_shared "src/basic/io-util.c"
nm_copy_sd_shared "src/basic/io-util.h"
nm_copy_sd_shared "src/basic/list.h"
nm_copy_sd_shared "src/basic/log.h"
nm_copy_sd_shared "src/basic/macro.h"
nm_copy_sd_shared "src/basic/memory-util.c"
nm_copy_sd_shared "src/basic/memory-util.h"
nm_copy_sd_shared "src/basic/mempool.c"
nm_copy_sd_shared "src/basic/mempool.h"
nm_copy_sd_shared "src/basic/missing_fcntl.h"
nm_copy_sd_shared "src/basic/missing_random.h"
nm_copy_sd_shared "src/basic/missing_socket.h"
nm_copy_sd_shared "src/basic/missing_stat.h"
nm_copy_sd_shared "src/basic/missing_type.h"
nm_copy_sd_shared "src/basic/parse-util.c"
nm_copy_sd_shared "src/basic/parse-util.h"
nm_copy_sd_shared "src/basic/path-util.c"
nm_copy_sd_shared "src/basic/path-util.h"
nm_copy_sd_shared "src/basic/prioq.c"
nm_copy_sd_shared "src/basic/prioq.h"
nm_copy_sd_shared "src/basic/process-util.c"
nm_copy_sd_shared "src/basic/process-util.h"
nm_copy_sd_shared "src/basic/random-util.c"
nm_copy_sd_shared "src/basic/random-util.h"
nm_copy_sd_shared "src/basic/set.h"
nm_copy_sd_shared "src/basic/signal-util.h"
nm_copy_sd_shared "src/basic/siphash24.h"
nm_copy_sd_shared "src/basic/socket-util.c"
nm_copy_sd_shared "src/basic/socket-util.h"
nm_copy_sd_shared "src/basic/sort-util.h"
nm_copy_sd_shared "src/basic/sparse-endian.h"
nm_copy_sd_shared "src/basic/stat-util.c"
nm_copy_sd_shared "src/basic/stat-util.h"
nm_copy_sd_shared "src/basic/stdio-util.h"
nm_copy_sd_shared "src/basic/string-table.c"
nm_copy_sd_shared "src/basic/string-table.h"
nm_copy_sd_shared "src/basic/string-util.c"
nm_copy_sd_shared "src/basic/string-util.h"
nm_copy_sd_shared "src/basic/strv.c"
nm_copy_sd_shared "src/basic/strv.h"
nm_copy_sd_shared "src/basic/strxcpyx.c"
nm_copy_sd_shared "src/basic/strxcpyx.h"
nm_copy_sd_shared "src/basic/time-util.c"
nm_copy_sd_shared "src/basic/time-util.h"
nm_copy_sd_shared "src/basic/tmpfile-util.c"
nm_copy_sd_shared "src/basic/tmpfile-util.h"
nm_copy_sd_shared "src/basic/umask-util.h"
nm_copy_sd_shared "src/basic/utf8.c"
nm_copy_sd_shared "src/basic/utf8.h"
nm_copy_sd_shared "src/basic/util.c"
nm_copy_sd_shared "src/basic/util.h"
nm_copy_sd_shared "src/shared/dns-domain.c"
nm_copy_sd_shared "src/shared/dns-domain.h"
nm_copy_sd_stdaux "src/basic/unaligned.h"
2019-11-29 13:05:13 +01:00
Thomas Haller
2b6f5a305c shared: add nm_utils_error_new() and nm_utils_error_new_cancelled() helper 2019-11-28 19:20:33 +01:00
Thomas Haller
40012e2aa8 shared: move log level info from core to "nm-logging-base.h"
We have our NM specific logging and log levels. Maybe we should
not have that, and instead only rely on syslog (like systemd)
or glog(). Anyway, currently we have one way and it makes sense
that this is also used outside from "src".

Move the helper function to parse log levels from string to
"nm-logging-base.h" so that we can use the same logging levels
outside of core.

This moves code that is currently GPL2+ licensed to
LGPL2.1+. However as far as I see, this code was entirely written
by Red Hat employees who would not object with this change. Also,
it's as obvious and trivial as it gets.
2019-11-28 19:20:33 +01:00
Thomas Haller
32d3a3f7ef shared: cleanup include guard for nm-logging-fwd.h 2019-11-28 19:20:33 +01:00
Thomas Haller
41d81e6893 shared/logging: add "nm-logging-base.h"
We have "nm-logging-fwd.h", which (as the name implies) is header-only.
Add instead a "nm-logging-base.c", which also contains implementation for
logging functions that are not only useful under "src/nm-logging.c"
2019-11-28 19:20:33 +01:00
Beniamino Galvani
316ff68bfa dhcp: support FQDN flags in the dhclient backend
Make the dhclient backend honor the ipv4.dhcp-fqdn-flags setting.
2019-11-28 17:56:35 +01:00
Thomas Haller
ce0e898fb4 libnm: refactor caching of D-Bus objects in NMClient
No longer use GDBusObjectMangaerClient and gdbus-codegen generated classes
for the NMClient cache. Instead, use GDBusConnection directly and a
custom implementation (NMLDBusObject) for caching D-Bus' ObjectManager
data.

CHANGES
-------

- This is a complete rework. I think the previous implementation was
difficult to understand. There were unfixed bugs and nobody understood
the code well enough to fix them. Maybe somebody out there understood the
code, but I certainly did not. At least nobody provided patches to fix those
issues. I do believe that this implementation is more straightforward and
easier to understand. It removes a lot of layers of code. Whether this claim
of simplicity is true, each reader must decide for himself/herself. Note
that it is still fairly complex.

- There was a lingering performance issue with large number of D-Bus
objects. The patch tries hard that the implementation scales well. Of
course, when we cache N objects that have N-to-M references to other,
we still are fundamentally O(N*M) for runtime and memory consumption (with
M being the number of references between objects). But each part should behave
efficiently and well.

- Play well with GMainContext. libnm code (NMClient) is generally not
thread safe. However, it should work to use multiple instances in
parallel, as long as each access to a NMClient is through the caller's
GMainContext. This follows glib's style and effectively allows to use NMClient
in a multi threaded scenario. This implies to stick to a main context
upon construction and ensure that callbacks are only invoked when
iterating that context. Also, NMClient itself shall never iterate the
caller's context. This also means, libnm must never use g_idle_add() or
g_timeout_add(), as those enqueue sources in the g_main_context_default()
context.

- Get ordering of messages right. All events are consistently enqueued
in a GMainContext and processed strictly in order. For example,
previously "nm-object.c" tried to combine signals and emit them on an
idle handler. That is wrong, signals must be emitted in the right order
and when they happen. Note that when using GInitable's synchronous initialization
to initialize the NMClient instance, NMClient internally still operates fully
asynchronously. In that case NMClient has an internal main context.

- NMClient takes over most of the functionality. When using D-Bus'
ObjectManager interface, one needs to handle basically the entire state
of the D-Bus interface. That cannot be separated well into distinct
parts, and even if you try, you just end up having closely related code
in different source files. Spreading related code does not make it
easier to understand, on the contrary. That means, NMClient is
inherently complex as it contains most of the logic. I think that is
not avoidable, but it's not as bad as it sounds.

- NMClient processes D-Bus messages and state changes in separate steps.
First NMClient unpacks the message (e.g. _dbus_handle_properties_changed()) and
keeps track of the changed data. Then we update the GObject instances
(_dbus_handle_obj_changed_dbus()) without emitting any signals yet. Finally,
we emit all signals and notifications that were collected
(_dbus_handle_changes_commit()). Note that for example during the initial
GetManagedObjects() reply, NMClient receive a large amount of state at once.
But we first apply all the changes to our GObject instances before
emitting any signals. The result is that signals are always emitted in a moment
when the cache is consistent. The unavoidable downside is that when you receive
a property changed signal, possibly many other properties changed
already and more signals are about to be emitted.

- NMDeviceWifi no longer modifies the content of the cache from client side
during poke_wireless_devices_with_rf_status(). The content of the cache
should be determined by D-Bus alone and follow what NetworkManager
service exposes. Local modifications should be avoided.

- This aims to bring no API/ABI change, though it does of course bring
various subtle changes in behavior. Those should be all for the better, but the
goal is not to break any existing clients. This does change internal
(albeit externally visible) API, like dropping NM_OBJECT_DBUS_OBJECT_MANAGER
property and NMObject no longer implementing GInitableIface and GAsyncInitableIface.

- Some uses of gdbus-codegen classes remain in NMVpnPluginOld, NMVpnServicePlugin
and NMSecretAgentOld. These are independent of NMClient/NMObject and
should be reworked separately.

- While we no longer use generated classes from gdbus-codegen, we don't
need more glue code than before. Also before we constructed NMPropertiesInfo and
a had large amount of code to propagate properties from NMDBus* to NMObject.
That got completely reworked, but did not fundamentally change. You still need
about the same effort to create the NMLDBusMetaIface. Not using
generated bindings did not make anything worse (which tells about the
usefulness of generated code, at least in the way it was used).

- NMLDBusMetaIface and other meta data is static and immutable. This
avoids copying them around. Also, macros like NML_DBUS_META_PROPERTY_INIT_U()
have compile time checks to ensure the property types matches. It's pretty hard
to misuse them because it won't compile.

- The meta data now explicitly encodes the expected D-Bus types and
makes sure never to accept wrong data. That would only matter when the
server (accidentally or intentionally) exposes unexpected types on
D-Bus. I don't think that was previously ensured in all cases.
For example, demarshal_generic() only cared about the GObject property
type, it didn't know the expected D-Bus type.

- Previously GDBusObjectManager would sometimes emit warnings (g_log()). Those
probably indicated real bugs. In any case, it prevented us from running CI
with G_DEBUG=fatal-warnings, because there would be just too many
unrelated crashes. Now we log debug messages that can be enabled with
"LIBNM_CLIENT_DEBUG=trace". Some of these messages can also be turned
into g_warning()/g_critical() by setting LIBNM_CLIENT_DEBUG=warning,error.
Together with G_DEBUG=fatal-warnings, this turns them into assertions.
Note that such "assertion failures" might also happen because of a server
bug (or change). Thus these are not common assertions that indicate a bug
in libnm and are thus not armed unless explicitly requested. In our CI we
should now always run with LIBNM_CLIENT_DEBUG=warning,error and
G_DEBUG=fatal-warnings and to catch bugs. Note that currently
NetworkManager has bugs in this regard, so enabling this will result in
assertion failures. That should be fixed first.

- Note that this changes the order in which we emit "notify:devices" and
"device-added" signals. I think it makes the most sense to emit first
"device-removed", then "notify:devices", and finally "device-added"
signals.
This changes behavior for commit 52ae28f6e5 ('libnm: queue
added/removed signals and suppress uninitialized notifications'),
but I don't think that users should actually rely on the order. Still,
the new order makes the most sense to me.

- In NetworkManager, profiles can be invisible to the user by setting
"connection.permissions". Such profiles would be hidden by NMClient's
nm_client_get_connections() and their "connection-added"/"connection-removed"
signals.
Note that NMActiveConnection's nm_active_connection_get_connection()
and NMDevice's nm_device_get_available_connections() still exposes such
hidden NMRemoteConnection instances. This behavior was preserved.

NUMBERS
-------

I compared 3 versions of libnm.

  [1] 962297f908, current tip of nm-1-20 branch
  [2] 4fad8c7c64, current master, immediate parent of this patch
  [3] this patch

All tests were done on Fedora 31, x86_64, gcc 9.2.1-1.fc31.
The libraries were build with

  $ ./contrib/fedora/rpm/build_clean.sh -g -w test -W debug

Note that RPM build already stripped the library.

---

N1) File size of libnm.so.0.1.0 in bytes. There currently seems to be a issue
  on Fedora 31 generating wrong ELF notes. Usually, libnm is smaller but
  in these tests it had large (and bogus) ELF notes. Anyway, the point
  is to show the relative sizes, so it doesn't matter).

  [1] 4075552 (102.7%)
  [2] 3969624 (100.0%)
  [3] 3705208 ( 93.3%)

---

N2) `size /usr/lib64/libnm.so.0.1.0`:

          text             data              bss                dec               hex   filename
  [1]  1314569 (102.0%)   69980 ( 94.8%)   10632 ( 80.4%)   1395181 (101.4%)   1549ed   /usr/lib64/libnm.so.0.1.0
  [2]  1288410 (100.0%)   73796 (100.0%)   13224 (100.0%)   1375430 (100.0%)   14fcc6   /usr/lib64/libnm.so.0.1.0
  [3]  1229066 ( 95.4%)   65248 ( 88.4%)   13400 (101.3%)   1307714 ( 95.1%)   13f442   /usr/lib64/libnm.so.0.1.0

---

N3) Performance test with test-client.py. With checkout of [2], run

```
prepare_checkout() {
    rm -rf /tmp/nm-test && \
    git checkout -B test 4fad8c7c64 && \
    git clean -fdx && \
    ./autogen.sh --prefix=/tmp/nm-test && \
    make -j 5 install && \
    make -j 5 check-local-clients-tests-test-client
}
prepare_test() {
    NM_TEST_REGENERATE=1 NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v
}
do_test() {
  for i in {1..10}; do
      NM_TEST_CLIENT_BUILDDIR="/data/src/NetworkManager" NM_TEST_CLIENT_NMCLI_PATH=/usr/bin/nmcli python3 ./clients/tests/test-client.py -v || return -1
  done
  echo "done!"
}
prepare_checkout
prepare_test
time do_test
```

  [1]  real 2m14.497s (101.3%)     user 5m26.651s (100.3%)     sys  1m40.453s (101.4%)
  [2]  real 2m12.800s (100.0%)     user 5m25.619s (100.0%)     sys  1m39.065s (100.0%)
  [3]  real 1m54.915s ( 86.5%)     user 4m18.585s ( 79.4%)     sys  1m32.066s ( 92.9%)

---

N4) Performance. Run NetworkManager from build [2] and setup a large number
of profiles (551 profiles and 515 devices, mostly unrealized). This
setup is already at the edge of what NetworkManager currently can
handle. Of course, that is a different issue. Here we just check how
long plain `nmcli` takes on the system.

```
do_cleanup() {
    for UUID in $(nmcli -g NAME,UUID connection show | sed -n 's/^xx-c-.*:\([^:]\+\)$/\1/p'); do
        nmcli connection delete uuid "$UUID"
    done
    for DEVICE in $(nmcli -g DEVICE device status | grep '^xx-i-'); do
        nmcli device delete "$DEVICE"
    done
}

do_setup() {
    do_cleanup
    for i in {1..30}; do
        nmcli connection add type bond autoconnect no con-name xx-c-bond-$i ifname xx-i-bond-$i ipv4.method disabled ipv6.method ignore
        for j in $(seq $i 30); do
            nmcli connection add type vlan autoconnect no con-name xx-c-vlan-$i-$j vlan.id $j ifname xx-i-vlan-$i-$j vlan.parent xx-i-bond-$i  ipv4.method disabled ipv6.method ignore
        done
    done
    systemctl restart NetworkManager.service
    sleep 5
}

do_test() {
    perf stat -r 50 -B nmcli 1>/dev/null
}

do_test
```

  [1]

   Performance counter stats for 'nmcli' (50 runs):

              456.33 msec task-clock:u              #    1.093 CPUs utilized            ( +-  0.44% )
                   0      context-switches:u        #    0.000 K/sec
                   0      cpu-migrations:u          #    0.000 K/sec
               5,900      page-faults:u             #    0.013 M/sec                    ( +-  0.02% )
       1,408,675,453      cycles:u                  #    3.087 GHz                      ( +-  0.48% )
       1,594,741,060      instructions:u            #    1.13  insn per cycle           ( +-  0.02% )
         368,744,018      branches:u                #  808.061 M/sec                    ( +-  0.02% )
           4,566,058      branch-misses:u           #    1.24% of all branches          ( +-  0.76% )

             0.41761 +- 0.00282 seconds time elapsed  ( +-  0.68% )

  [2]

   Performance counter stats for 'nmcli' (50 runs):

              477.99 msec task-clock:u              #    1.088 CPUs utilized            ( +-  0.36% )
                   0      context-switches:u        #    0.000 K/sec
                   0      cpu-migrations:u          #    0.000 K/sec
               5,948      page-faults:u             #    0.012 M/sec                    ( +-  0.03% )
       1,471,133,482      cycles:u                  #    3.078 GHz                      ( +-  0.36% )
       1,655,275,369      instructions:u            #    1.13  insn per cycle           ( +-  0.02% )
         382,595,152      branches:u                #  800.433 M/sec                    ( +-  0.02% )
           4,746,070      branch-misses:u           #    1.24% of all branches          ( +-  0.49% )

             0.43923 +- 0.00242 seconds time elapsed  ( +-  0.55% )

  [3]

   Performance counter stats for 'nmcli' (50 runs):

              352.36 msec task-clock:u              #    1.027 CPUs utilized            ( +-  0.32% )
                   0      context-switches:u        #    0.000 K/sec
                   0      cpu-migrations:u          #    0.000 K/sec
               4,790      page-faults:u             #    0.014 M/sec                    ( +-  0.26% )
       1,092,341,186      cycles:u                  #    3.100 GHz                      ( +-  0.26% )
       1,209,045,283      instructions:u            #    1.11  insn per cycle           ( +-  0.02% )
         281,708,462      branches:u                #  799.499 M/sec                    ( +-  0.01% )
           3,101,031      branch-misses:u           #    1.10% of all branches          ( +-  0.61% )

             0.34296 +- 0.00120 seconds time elapsed  ( +-  0.35% )

---

N5) same setup as N4), but run `PAGER= /bin/time -v nmcli`:

  [1]

        Command being timed: "nmcli"
        User time (seconds): 0.42
        System time (seconds): 0.04
        Percent of CPU this job got: 107%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.43
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 34456
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 6128
        Voluntary context switches: 1298
        Involuntary context switches: 1106
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

  [2]
        Command being timed: "nmcli"
        User time (seconds): 0.44
        System time (seconds): 0.04
        Percent of CPU this job got: 108%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.44
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 34452
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 6169
        Voluntary context switches: 1849
        Involuntary context switches: 142
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

  [3]

        Command being timed: "nmcli"
        User time (seconds): 0.32
        System time (seconds): 0.02
        Percent of CPU this job got: 102%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.34
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 29196
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 5059
        Voluntary context switches: 919
        Involuntary context switches: 685
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

---

N6) same setup as N4), but run `nmcli monitor` and look at `ps aux` for
  the RSS size.

      USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
  [1] me     1492900 21.0  0.2 461348 33248 pts/10   Sl+  15:02   0:00 nmcli monitor
  [2] me     1490721  5.0  0.2 461496 33548 pts/10   Sl+  15:00   0:00 nmcli monitor
  [3] me     1495801 16.5  0.1 459476 28692 pts/10   Sl+  15:04   0:00 nmcli monitor
2019-11-25 15:08:00 +01:00
Thomas Haller
4fad8c7c64 shared: add nm_utils_g_main_context_create_integrate_source() for integrating a GMainContext in another
We will rework NMClient entirely. Then, the synchronous initialization will also use
the asynchronous code paths. The difference will be that with synchronous initialization,
all D-Bus interaction will be done with an internal GMainContext as current thread default,
and that internal context will run until initialization completes.

Note that even after initialization completes, it cannot be swapped back to the user's
(outer) GMainContext. That is because contexts are essentially the queue for our
D-Bus events, and we cannot swap from one queue to the other in a race
free manner (or a full resync). In other words, the two contexts are not in sync,
so after using the internal context NMClient needs to stick to that (at least, until
the name owner gets lost, which gives an opportunity to resync and switch back to the
user's main context).

We thus need to hook the internal (inner) GMainContext with the user's (outer) context,
so when the user iterates the outer context, events on the inner context get dispatched.

Add nm_utils_g_main_context_create_integrate_source() to create such a GSource for
integrating two contexts.

Note that the use-case here is limited: the integrated, inner main context must
not be explicitly iterated except from being dispatched by the integrating
source. Otherwise, you'd get recursive runs, possible deadlocks and general
ugliness. NMClient must show restrain how to use the inner context while it is
integrated.
2019-11-25 12:58:33 +01:00
Thomas Haller
45c9f3c39b shared: make NM_STRUCT_OFFSET_ENSURE_TYPE() work with arrays
Some compilers don't convert arrays as _Generic() type selectors to
their pointer type. That means, for those compilers the generic type
would be an array and not a pointer. Work around that by adding zero
to the pointer/array argument.

Also, I cannot get this to work with "clang-3.4.2-9.el7". Disable it
for that compiler. The value of the generic check is anyway that it only
needs to work with some compiler combinations. That will trigger a
compilation failure and we can fix the implementation also for compilers
that don't support the macro.

See-also: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1930.htm
2019-11-25 12:58:33 +01:00
Thomas Haller
b8b8ef7600 shared: mark _notify() function as "_nm_unused"
There are two macros: NM_GOBJECT_PROPERTIES_DEFINE_BASE() and
NM_GOBJECT_PROPERTIES_DEFINE(). The former just defines the
property enums and the obj_properties array. The latter also
defines the functions _notify() and _nm_gobject_notify_together_impl().

That means, depending on whether you actually use _notify(), you have
to choose one of the macros. I think that is unnecessarily cumbersome.
Let's mark the function as _nm_unused so that the compiler doesn't
complain about the unused function. I don't think it's a problem
to use NM_GOBJECT_PROPERTIES_DEFINE() even if you don't actually use
_notify().
2019-11-25 12:58:33 +01:00
Thomas Haller
58811e019d shared: add nm_auto_unref_gmaincontext macro 2019-11-25 12:58:33 +01:00
Thomas Haller
bc9baf0fd5 shared: add nm_pint_hash()/nm_pint_equals() utils 2019-11-25 12:58:33 +01:00
Thomas Haller
6d7270e222 build/meson: cleanup configuration_data() for paths
We don't need such data duplicated. The build setup should
have only one configuration_data() for patching such values.

Now we only have one global, immutable data_conf dictionary with
configuration values. Note that none of the users of data_conf uses all
entries, but as the entries are basically only dependent on the
meson/configure option and valid for the entire project, this simplifies
to handling.
2019-11-22 15:59:31 +01:00
Thomas Haller
033c2b82c2 core: move _LOG*() macros to "shared/nm-glib-aux/nm-logging-fwd.h"
We preferably should use our convenience macros like _LOGD().
Since those macros expand to _NMLOG() (which needs to be defined
separately), we can move it to "nm-logging-fwd.h" and reuse.
2019-11-22 15:32:52 +01:00
Thomas Haller
ec868916c8 shared: move nm_utils_ip._address_clear_host_address() helpers to shared 2019-11-22 15:32:52 +01:00
Thomas Haller
2ef5014f98 shared: add nm_clear_g_source_inst()
glib really likes the numeric source IDs. That is, g_idle_add(), g_timeout_add(),
etc. return those IDs, that can then be destroyed with g_remove_source() (or
nm_clear_g_source()).

I think these numeric IDs are really not great.

- API like g_idle_add() and g_remove_source() only works with the g_main_context_get_default()
  instance. That means, you cannot use this API for any other contexts. If you'd insist on using
  numeric IDs, you'd need to call g_main_context_find_source_by_id() on the right context
  first (but you'd also have to track the context alongside the ID).
- g_remove_source() requires first a call to g_main_context_find_source_by_id(). This involves
  taking a mutex and doing an extra hash lookup.

Instead, it often seems preferable to use the GSource instance directly. It works
with any context, it can be referenced and unreferenced, and it can be destroyed, and
avoids the overhead of g_main_context_find_source_by_id().

The only downside really is that keeping a GSource pointer takes one pointer size, while
the guint source ID is usually only 4 bytes.

Anyway, I think we should deal more with GSource instances directly. Hence, add this
convenience macro, that works like nm_clear_g_source().
2019-11-22 15:32:52 +01:00
Thomas Haller
c40ff42ae6 shared: add nm_g_*_source_new() and nm_g_source_attach() helpers
Small utilities to make is more convenient to create and attach GSource
instances.
2019-11-22 15:32:52 +01:00