Now that unit tests no longer dynamically link against libnm/libnm.la,
and statically against the same code, we can enable this assertion
again.
This reverts commit 7a0e347b38.
libnm/tests/test-general statically links against libnm/libnm-utils.la
and dynamically against libnm/libnm.so. Hence, _nm_utils_init() is invoked
twice, failing the assertion.
That is a bug that must be fixed. For now, just don't assert.
Kernel calls the netlink attribute TCA_FQ_CODEL_MEMORY_LIMIT. Likewise,
iproute2 calls this "memory_limit".
Rename because TC parameters are inherrently tied to the kernel
implementation and we should use the familiar name.
iproute2 uses the special value ~0u to indicate not to set
TCA_FQ_CODEL_CE_THRESHOLD in RTM_NEWQDISC. When not explicitly
setting the value, kernel treats the threshold as disabled.
However note that 0xFFFFFFFFu is not an invalid threshold (as far as
kernel is concerned). Thus, we should not use that as value to indicate
that the value is unset. Note that iproute2 uses the special value ~0u
only internally thereby making it impossible to set the threshold to
0xFFFFFFFFu). But kernel does not have this limitation.
Maybe the cleanest way would be to add another field to NMPlatformQDisc:
guint32 ce_threshold;
bool ce_threshold_set:1;
that indicates whether the threshold is enable or not.
But note that kernel does:
static void codel_params_init(struct codel_params *params)
{
...
params->ce_threshold = CODEL_DISABLED_THRESHOLD;
static int fq_codel_change(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
...
if (tb[TCA_FQ_CODEL_CE_THRESHOLD]) {
u64 val = nla_get_u32(tb[TCA_FQ_CODEL_CE_THRESHOLD]);
q->cparams.ce_threshold = (val * NSEC_PER_USEC) >> CODEL_SHIFT;
}
static int fq_codel_dump(struct Qdisc *sch, struct sk_buff *skb)
{
...
if (q->cparams.ce_threshold != CODEL_DISABLED_THRESHOLD &&
nla_put_u32(skb, TCA_FQ_CODEL_CE_THRESHOLD,
codel_time_to_us(q->cparams.ce_threshold)))
goto nla_put_failure;
This means, kernel internally uses the special value 0x83126E97u to indicate
that the threshold is disabled (WTF). That is because
(((guint64) 0x83126E97u) * NSEC_PER_USEC) >> CODEL_SHIFT == CODEL_DISABLED_THRESHOLD
So in kernel API this value is reserved (and has a special meaning
to indicate that the threshold is disabled). So, instead of adding a
ce_threshold_set flag, use the same value that kernel anyway uses.
The memory-limit is an unsigned integer. It is ugly (if not wrong) to compare unsigned
values with "-1". When comparing with the default value we must also use an u32 type.
Instead add a define NM_PLATFORM_FQ_CODEL_MEMORY_LIMIT_UNSET.
Note that like iproute2 we treat NM_PLATFORM_FQ_CODEL_MEMORY_LIMIT_UNSET
to indicate to not set TCA_FQ_CODEL_MEMORY_LIMIT in RTM_NEWQDISC. This
special value is entirely internal to NetworkManager (or iproute2) and
kernel will then choose a default memory limit (of 32MB). So setting
NM_PLATFORM_FQ_CODEL_MEMORY_LIMIT_UNSET means to leave it to kernel to
choose a value (which then chooses 32MB).
See kernel's net/sched/sch_fq_codel.c:
static int fq_codel_init(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
...
q->memory_limit = 32 << 20; /* 32 MBytes */
static int fq_codel_change(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
...
if (tb[TCA_FQ_CODEL_MEMORY_LIMIT])
q->memory_limit = min(1U << 31, nla_get_u32(tb[TCA_FQ_CODEL_MEMORY_LIMIT]));
Note that not having zero as default value is problematic. In fields like
"NMPlatformIP4Route.table_coerced" and "NMPlatformRoutingRule.suppress_prefixlen_inverse"
we avoid this problem by storing a coerced value in the structure so that zero is still
the default. We don't do that here for memory-limit, so the caller must always explicitly
set the value.
I think initializing structs should (almost) be always done with designated
initializers, because otherwise it's easy to get the order wrong. The
problem is that otherwise the order of fields gets additional meaning
not only for the memory layout, but also for the code that initialize
the structs.
Add a macro NM_VARIANT_ATTRIBUTE_SPEC_DEFINE() that replaces the other
(duplicate) macros. This macro also gets it right to mark the struct as
const.
This actually allows the compiler/linker to mark the memory as read-only and any
modification will cause a segmentation fault.
I would also think that it allows the compiler to put the structure directly
beside the outer constant array (in which this pointer is embedded). That is good
locality-wise.
- g_ascii_strtoll() accepts leading spaces, but it leaves
the end pointer at the first space after the digit. That means,
we accepted "1: 0" but not "1 :0". We should either consistently
accept spaces around the digits/colon or reject it.
- g_ascii_strtoll() accepts "\v" as a space (just like `man 3 isspace`
comments that "\v" is a space in C and POSIX locale.
For some reasons (unknown to me) g_ascii_isspace() does not treat
"\v" as space. And neither does NM_ASCII_SPACES and
nm_str_skip_leading_spaces().
We should be consistent about what we consider spaces and what not.
It's already odd to accept '\n' as spaces here, but well, lets do
it for the sake of consistency (so that it matches with our
understanding of ASCII spaces, albeit not POSIX's).
- don't use bogus error domains in "g_set_error (error, 1, 0, ..."
That is a bug and we have NM_UTILS_ERROR exactly for error instances
with unspecified domain and code.
- as before, accept a trailing ":" with omitted minor number.
- reject all unexpected characters. strtoll() accepts '+' / '-'
and a "0x" prefix of the numbers (and leading POSIX spaces). Be
strict here and only accepts NM_ASCII_SPACES, ':', and hexdigits.
In particular, don't accept the "0x" prefix.
This parsing would be significantly simpler to implement, if we could
just strdup() the string, split the string at the colon delimiter and
use _nm_utils_ascii_str_to_int64() which gets leading/trailing spaces
right. But let's save the "overhead" of an additional alloc.
Merge the function pointer get_func() into to_dbus_fcn().
Previously, get_func() as handled separately from to_dbus_fnc()
(formerly synth_func()). The notion was that synth-func would syntetize
properties that are D-Bus only. But that distinction does not seem
very helpful to me.
Instaed, we want to convert a property to D-Bus. Period. The
implementation should be handled uniformly. Hence, now that is
all done by property_to_dbus().
Note that property_to_dbus() is also called as default implementation
for compare-property. At least, for properties that are backed by a
GObject property.
The naming was not very clear. How does get_func(), synth_func()
and to_dbus() relate? What does synth_func() do anyway?
Answers:
- get_func() and synth_func() do very similar. They should be merged
in a next step.
synth_func() has the notion of "synthetize" a property for
D-Bus. As such, these properties are a bit unusual in that they
don't have a backing GObject property in the setting. But it'd
rather treat such properties like other properties. The step
in that direction will be to merge the to-dbus functions.
- to_dbus() converts a GValue of the GObject property go GVariant.
It's a simplified form of get_func()/synth_func() and a better name
is gprop_to_dbus_fcn().
The same for gprop_from_dbus_fcn().
For now, just rename.
We have certain artificial properties that not only depend on one
property alone or that depend on a property in another(!) setting.
For that, we have synth_func.
Other than that, synth_func and get_func are really fundamentally
similar and should be merged. That is because the distinction whether a
property value is "synthetized" or just based on a plain property is
minor. It's better to have the general concept of "convert property to
GVariant" in one form only.
Note that compare_property() is by default implemented based
on get_func. Hence, if get_func and synth_func get merged,
compare_property() will also require access to the NMConnection.
Also it makes some sense: some properties are artificial and actually
stored in "another" setting of the connection. But still, the property
descriptor for the property is in this setting. The example is the
"bond.interface-name" which only exists on D-Bus. It's stored as
"connection.interface-name".
I don't really like to say "exists on D-Bus only". It's still a valid
property, despite in NMSetting it's stored somehow differently (or not
at all). So, this is also just a regular property for which we have a
property-info vtable.
Does it make sense to compare such properties? Maybe. But the point is that
compare_property() function needs sometimes access to the entire
connection. So add the argument.
Always properly set NMSettInfoProperty.dbus_type, instead of leaving it
unspecified for GObject property based properties, and detect it each
time anew with variant_type_for_gtype().
Instead, autodetect and remember the dbus-type during _properties_override_add_struct().
For types that need special handling (GBytes, enums and flags) set a to_dbus() function.
This allows us to handle properties uniformly by either calling the to_dbus() function
or g_dbus_gvalue_to_gvariant().
- use cleanup attribute in get_property_for_dbus() and return early.
- use NM_FLAGS_HAS() macro in _nm_setting_to_dbus().
- in nm_setting_get_dbus_property_type() use g_return*() asserts
instead of crash or hard asserts.
- return early from variant_type_for_gtype().
- the previous implementation of nm_setting_wired_get_s390_option()
returned the elements in an arbitrary order (because it just iterated
idx times over the unsorted hash table).
- the API for "s390-options" suggests both accessing by index and by
name. Storing the options in a hash-table is not optimal for lookup
by index. It also requires us to sort the elements over and over
again.
Use instead a sorted array. Note that add/remove of course requires to
move the elements (and has thus O(n)).
- "s390-options" are very seldomly set. We shouldn't pay the price in every
NMSettingWired to allocate a GHashTable and deal with it.
- don't assert in nm_setting_wired_add_s390_option() and
nm_setting_wired_remove_s390_option() that the key is valid.
ifcfg-rh reader understandably does not want to implement additional
logic to pre-validate the key, so any invalid keys would trigger an
assertion failure. We have verify() for this purpose.
Currently, nm_setting_wired_get_s390_option() returns the key
in an undefined order. Hence, the keyfile writer and the test
need to awkwardly sort the keys first. That will be solved better
in the next commit, when nm_setting_wired_get_s390_option() returns
the items sorted by key.
Most of the caller won't require a deep-clone of the attribute
names. Likely, the fetch the name, so they can lookup the attributes.
In that common case, there is no need to clone the strings themself.
"libnm-core" implements common functionality for "NetworkManager" and
"libnm".
Note that clients like "nmcli" cannot access the internal API provided
by "libnm-core". So, if nmcli wants to do something that is also done by
"libnm-core", , "libnm", or "NetworkManager", the code would have to be
duplicated.
Instead, such code can be in "libnm-libnm-core-{intern|aux}.la".
Note that:
0) "libnm-libnm-core-intern.la" is used by libnm-core itsself.
On the other hand, "libnm-libnm-core-aux.la" is not used by
libnm-core, but provides utilities on top of it.
1) they both extend "libnm-core" with utlities that are not public
API of libnm itself. Maybe part of the code should one day become
public API of libnm. On the other hand, this is code for which
we may not want to commit to a stable interface or which we
don't want to provide as part of the API.
2) "libnm-libnm-core-intern.la" is statically linked by "libnm-core"
and thus directly available to "libnm" and "NetworkManager".
On the other hand, "libnm-libnm-core-aux.la" may be used by "libnm"
and "NetworkManager".
Both libraries may be statically linked by libnm clients (like
nmcli).
3) it must only use glib, libnm-glib-aux.la, and the public API
of libnm-core.
This is important: it must not use "libnm-core/nm-core-internal.h"
nor "libnm-core/nm-utils-private.h" so the static library is usable
by nmcli which couldn't access these.
Note that "shared/nm-meta-setting.c" is an entirely different case,
because it behaves differently depending on whether linking against
"libnm-core" or the client programs. As such, this file must be compiled
twice.
From the files under "shared/nm-utils" we build an internal library
that provides glib-based helper utilities.
Move the files of that basic library to a new subdirectory
"shared/nm-glib-aux" and rename the helper library "libnm-core-base.la"
to "libnm-glib-aux.la".
Reasons:
- the name "utils" is overused in our code-base. Everything's an
"utils". Give this thing a more distinct name.
- there were additional files under "shared/nm-utils", which are not
part of this internal library "libnm-utils-base.la". All the files
that are part of this library should be together in the same
directory, but files that are not, should not be there.
- the new name should better convey what this library is and what is isn't:
it's a set of utilities and helper functions that extend glib with
funcitonality that we commonly need.
There are still some files left under "shared/nm-utils". They have less
a unifying propose to be in their own directory, so I leave them there
for now. But at least they are separate from "shared/nm-glib-aux",
which has a very clear purpose.
We built (among others) two libraries from the sources in "shared/nm-utils":
"libnm-utils-base.la" and "libnm-utils-udev.la".
It's confusing. Instead use directories so there is a direct
correspondence between these internal libraries and the source files.
"shared/nm-utils" contains general purpose utility functions that only
depend on glib (and extend glib with some helper functions).
We will also add code that does not use glib, hence it would be good
if the part of "shared/nm-utils" that does not depend on glib, could be
used by these future projects.
Also, we use the term "utils" everywhere. While that covers the purpose
and content well, having everything called "nm-something-utils" is not
great. Instead, call this "nm-std-aux", inspired by "c-util/c-stdaux".
CC libnm-core/tests/libnm_core_tests_test_general-test-general.o
In file included from ../shared/nm-default.h:280:0,
from ../libnm-core/tests/test-general.c:24:
../libnm-core/tests/test-general.c: In function _sock_addr_endpoint:
../libnm-core/tests/test-general.c:5911:18: error: logical not is only applied to the left hand side of comparison [-Werror=logical-not-parentheses]
g_assert (!host == (port == -1));
^
../shared/nm-utils/nm-macros-internal.h:1793:7: note: in definition of macro __NM_G_BOOLEAN_EXPR_IMPL
if (expr) \
^
/usr/include/glib-2.0/glib/gmacros.h:376:43: note: in expansion of macro _G_BOOLEAN_EXPR
#define G_LIKELY(expr) (__builtin_expect (_G_BOOLEAN_EXPR((expr)), 1))
^
/usr/include/glib-2.0/glib/gtestutils.h:116:49: note: in expansion of macro G_LIKELY
if G_LIKELY (expr) ; else \
^
../libnm-core/tests/test-general.c:5911:2: note: in expansion of macro g_assert
g_assert (!host == (port == -1));
^
Fixes: 713e879d76 ('libnm: add NMSockAddrEndpoint API')
- if there is only one vlan in the list, then we can return success
early. That is, because one NMBridgeVlan instance is always valid
due to the way how users must use the API to construct the element.
- the implementation for check_normalizable is only correct, if there
are no duplicate or overlapping ranges. Assert for that. In fact,
all callers first check for errors and then for normalizable errors.
- avoid duplicate calls to nm_bridge_vlan_get_vid_range(). There are
duplicate assertions that we don't need.
- only check for pvid once per range.
- combine calls to g_hash_table_contains() and g_hash_table_add().
In some cases it is convenient to specify ranges of bridge vlans, as
already supported by iproute2 and natively by kernel. With this commit
it becomes possible to add a range in this way:
nmcli connection modify eth0-slave +bridge-port.vlans "100-200 untagged"
vlan ranges can't be PVIDs because only one PVID vlan can exist.
https://bugzilla.redhat.com/show_bug.cgi?id=1652910
Replace nm_utils_str_simpletokens_extract_next() by
nm_utils_escaped_tokens_split().
nm_utils_escaped_tokens_split() should become our first choice for
parsing and tokenizing.
Note that both nm_utils_str_simpletokens_extract_next() and
nm_utils_escaped_tokens_split() need to strdup the string once,
and tokenizing takes O(n). So, they are roughtly the same performance
wise. The only difference is, that as we iterate through the tokens,
we might abort early on error with nm_utils_str_simpletokens_extract_next()
and not parse the entire string. But that is a small benefit, since we
anyway always strdup() the string (being O(n) already).
Note that to-string will no longer escape ',' and ';'. This is a change
in behavior, of unreleased API. Also note, that escaping these is no
longer necessary, because nmcli soon will also use nm_utils_escaped_tokens_*().
Another change in behavior is that nm_utils_str_simpletokens_extract_next()
treated invalid escape sequences (backslashes followed by an arbitrary
character), buy stripping the backslash. nm_utils_escaped_tokens_*()
leaves such backslashes as is, and only honors them if they are followed
by a whitespace (the delimiter) or another backslash. The disadvantage
of the new approach is that backslashes are treated differently
depending on the following character. The benefit is, that most
backslashes can now be written verbatim, not requiring them to escape
them with a double-backslash.
Yes, there is a problem with these nested escape schemes:
- the caller may already need to escape backslash in shell.
- then nmcli will use backslash escaping to split the rules at ','.
- then nm_ip_routing_rule_from_string() will honor backslash escaping
for spaces.
- then iifname and oifname use backslash escaping for nm_utils_buf_utf8safe_escape()
to express non-UTF-8 characters (because interface names are not
necessarily UTF-8).
This is only redeamed because escaping is really only necessary for very
unusual cases, if you want to embed a backslash, a space, a comma, or a
non-UTF-8 character. But if you have to, now you will be able to express
that.
The other upside of these layers of escaping is that they become all
indendent from each other:
- shell can accept quoted/escaped arguments and will unescape them.
- nmcli can do the tokenizing for ',' (and escape the content
unconditionally when converting to string).
- nm_ip_routing_rule_from_string() can do its tokenizing without
special consideration of utf8safe escaping.
- NMIPRoutingRule takes iifname/oifname as-is and is not concerned
about nm_utils_buf_utf8safe_escape(). However, before configuring
the rule in kernel, this utf8safe escape will be unescaped to get
the interface name (which is non-UTF8 binary).
This removes libnm-glib, libnm-glib-vpn, and libnm-util for good.
The it has been replaced with libnm since NetworkManager 1.0, disabled
by default since 1.12 and no up-to-date distributions ship it for years
now.
Removing the libraries allows us to:
* Remove the horrible hacks that were in place to deal with accidental use
of both the new and old library in a single process.
* Relief the translators of maintenance burden of similar yet different
strings.
* Get rid of known bad code without chances of ever getting fixed
(libnm-glib/nm-object.c and libnm-glib/nm-object-cache.c)
* Generally lower the footprint of the releases and our workspace
If there are some really really legacy users; they can just build
libnm-glib and friends from the NetworkManager-1.16 distribution. The
D-Bus API is stable and old libnm-glib will keep working forever.
https://github.com/NetworkManager/NetworkManager/pull/308
The call to nm_utils_parse_variant_attributes() is useless. The following
_tc_read_common_opts() call does the same thing. This was probably left
in place by accident.
Before:
# nmcli c modify eth666 tc.qdiscs root
Error: failed to modify tc.qdiscs: '(null)' is not a valid kind The valid syntax is: '[root | parent <handle>] [handle <handle>] <qdisc>'.
After:
# nmcli c modify eth666 tc.qdiscs root
Error: failed to modify tc.qdiscs: kind is missing. The valid syntax is: '[root | parent <handle>] [handle <handle>] <kind>'.
==16725==ERROR: AddressSanitizer: global-buffer-overflow on address 0x0000005a159f at pc 0x00000046fc1b bp 0x7fff6038f900 sp 0x7fff6038f8f0
READ of size 1 at 0x0000005a159f thread T0
#0 0x46fc1a in _do_test_unescape_spaces libnm-core/tests/test-general.c:7791
#1 0x46fe5b in test_nm_utils_unescape_spaces libnm-core/tests/test-general.c:7810
#2 0x7f4ac5fe7fc9 in test_case_run gtestutils.c:2318
#3 0x7f4ac5fe7fc9 in g_test_run_suite_internal gtestutils.c:2403
#4 0x7f4ac5fe7e83 in g_test_run_suite_internal gtestutils.c:2415
#5 0x7f4ac5fe7e83 in g_test_run_suite_internal gtestutils.c:2415
#6 0x7f4ac5fe8281 in g_test_run_suite gtestutils.c:2490
#7 0x7f4ac5fe82a4 in g_test_run (/lib64/libglib-2.0.so.0+0x772a4)
#8 0x48240d in main libnm-core/tests/test-general.c:7994
#9 0x7f4ac5dc9412 in __libc_start_main (/lib64/libc.so.6+0x24412)
#10 0x423ffd in _start (/home/bgalvani/work/NetworkManager/libnm-core/tests/test-general+0x423ffd)
0x0000005a159f is located 49 bytes to the right of global variable '*.LC370' defined in 'libnm-core/tests/test-general.c' (0x5a1560) of size 14
'*.LC370' is ascii string 'nick-5, green'
0x0000005a159f is located 1 bytes to the left of global variable '*.LC371' defined in 'libnm-core/tests/test-general.c' (0x5a15a0) of size 1
'*.LC371' is ascii string ''
SUMMARY: AddressSanitizer: global-buffer-overflow libnm-core/tests/test-general.c:7791 in _do_test_unescape_spaces
We already have code that parses exactly this kinds of string:
nm_utils_parse_inaddr_prefix_bin(). Use it.
Also, it doesn't use g_strsplit_set() to separate a string at the first
'/'. Total overkill.
Previously, nm_utils_strsplit_set_full() would always remove empty
tokens. Add a flag NM_UTILS_STRSPLIT_SET_FLAGS_PRESERVE_EMPTY to avoid
that.
This makes nm_utils_strsplit_set_full() return the same result as
g_strsplit_set() and a direct replacement for it -- except for "",
where we return %NULL.
