core: limit number of exported addresses/routes on D-Bus to 100

It doesn't scale to export all addresses/routes on D-Bus as properties.
In particular not combined with PropertiesChanged signal. On a busy
system, this causes severe performance issues. It also doesn't seem very
useful. Routes and addresses are complex things (e.g. policy routing).
If you want to do anything serious, you must check netlink (or find
another way to get the information).

Note that NMPlatform already ignores routes of certain protocols
(ip_route_is_alive()). It also does not expose most route attributes,
making the output only useful for very limited cases (e.g. displaying to
the user for information).

Limit the number of exported entries to 100.

Try adding 100K routes one-by-one. Run a `nmcli monitor` instance.
Re-nice the nmcli process and/or keep the CPUs busy. Then start a script
that adds 100k routes. Observe. Glib's D-Bus worker thread receives the
messages and queues them for the main thread. The main thread is too
slow to process them, the memory consumption grows very quickly in Giga
bytes. Afterwards, the memory also is not returned to the operation
system, either because of fragmentation or because the libc allocator
does anyway not return heap memory.

It doesn't work to expose an unlimited number of objects on D-Bus.  At
least not with an API, that sends the full list of all routes, whenever
a route changes. Nobody can use that feature either, because the only
use is a quick overview in `nmcli` output or a GUI. If you see 100+
routes there, that becomes unmanageable anyway. Instead use netlink if
you want to handle the full list of addresses/routes (or some other
API).
This commit is contained in:
Thomas Haller 2023-11-20 12:38:08 +01:00
parent 623012c14a
commit 4bb3ad0977
No known key found for this signature in database
GPG key ID: 29C2366E4DFC5728
2 changed files with 20 additions and 1 deletions

3
NEWS
View file

@ -16,6 +16,9 @@ Overview of changes since NetworkManager-1.44
when set to "org.freedesktop.NetworkManager".
* Drop build support with Python2. Python3 is now required.
* nmcli: limit number of printed addresses/routes in `nmcli` overview to 10.
* Limit number of exported IP addresses/routes on D-Bus to 100 to reduce
performance cost. Also, D-Bus updates for addresses/routes are now rate
limited to 3 per second.
=============================================
NetworkManager-1.44

View file

@ -1558,7 +1558,8 @@ nm_utils_ip_addresses_to_dbus(int addr_family,
char addr_str[NM_INET_ADDRSTRLEN];
NMDedupMultiIter iter;
const NMPObject *obj;
guint i;
const gsize MAX_ADDRESSES = 100;
gsize i;
nm_assert_addr_family(addr_family);
@ -1580,6 +1581,11 @@ nm_utils_ip_addresses_to_dbus(int addr_family,
nm_platform_dedup_multi_iter_next_obj(&iter, &obj, NMP_OBJECT_TYPE_IP_ADDRESS(IS_IPv4))) {
const NMPlatformIPXAddress *address = NMP_OBJECT_CAST_IPX_ADDRESS(obj);
if (i > MAX_ADDRESSES) {
/* Limited. The rest is hidden. */
break;
}
if (out_address_data) {
GVariantBuilder addr_builder;
gconstpointer p;
@ -1669,6 +1675,8 @@ nm_utils_ip_routes_to_dbus(int addr_family,
GVariantBuilder builder_data;
GVariantBuilder builder_legacy;
char addr_str[NM_INET_ADDRSTRLEN];
const gsize MAX_ROUTES = 100;
gsize i;
nm_assert_addr_family(addr_family);
@ -1681,6 +1689,7 @@ nm_utils_ip_routes_to_dbus(int addr_family,
g_variant_builder_init(&builder_legacy, G_VARIANT_TYPE("a(ayuayu)"));
}
i = 0;
nm_dedup_multi_iter_init(&iter, head_entry);
while (nm_platform_dedup_multi_iter_next_obj(&iter, &obj, NMP_OBJECT_TYPE_IP_ROUTE(IS_IPv4))) {
const NMPlatformIPXRoute *r = NMP_OBJECT_CAST_IPX_ROUTE(obj);
@ -1699,6 +1708,13 @@ nm_utils_ip_routes_to_dbus(int addr_family,
if (r->rx.type_coerced != nm_platform_route_type_coerce(RTN_UNICAST))
continue;
if (i >= MAX_ROUTES) {
/* Limited. The rest is hidden. */
break;
}
i++;
if (out_route_data) {
GVariantBuilder route_builder;
gconstpointer gateway;