For each artifical team property we need to track whether it was
explicitly set (i.e., present in JSON/GVariant or set by the user
via NMSettingTeam/NMSettingTeamPort API).
--
As a plus, libnm is now no longer concerned with the underling default values
that teamd uses. For example, the effective default value for "notify_peers.count"
depends on the selected runner. But libnm does not need to care, it only cares
wheher the property is set in JSON or not. This also means that the default (e.g. as
interesting to `nmcli -o con show $PROFILE`) is independent from other properties
(like the runner).
Also change the default value for the GObject properties of
NMSettingTeam and NMSettingTeamPort to indicate the "unset" value.
For most properties, the default value is a special value that is
not a valid configuration itself.
For some properties the default value is itself a valid value, namely,
"runner.active", "runner.fast_rate", "port.sticky" and "port.prio".
As far as NMTeamSetting is concerned, it distinguishes between unset
value and set value (including the default value). That means,
when it parses a JSON or GVariant, it will remember whether the property
was present or not.
When using API of NMSettingTeam/NMSettingTeamPort to set a property to the
default value, it marks the property as unset. For example, setting
NM_SETTING_TEAM_RUNNER_ACTIVE to TRUE (the default), means that the
value will not be serialized to JSON/GVariant. For the above 4
properties (where the default value is itself a valid value) this is a
limitation of libnm API, as it does not allow to explicitly set
'"runner": { "active": true }'. See SET_FIELD_MODE_SET_UNLESS_DEFAULT,
Note that changing the default value for properties of NMSetting is problematic,
because it changes behavior for how settings are parsed from keyfile/GVariant.
For team settings that's not the case, because if a JSON "config" is
present, all other properties are ignore. Also, we serialize properties
to JSON/GVariant depending on whether it's marked as present, and not
whether the value is set to the default (_nm_team_settings_property_to_dbus()).
--
While at it, sticter validate the settings. Note that if a setting is
initialized from JSON, the strict validation is not not performed. That
means, such a setting will always validate, regardless whether the values
in JSON are invalid according to libnm. Only when using the extended
properties, strict validation is turned on.
Note that libnm serializes the properties to GVariant both as JSON "config"
and extended properties. Since when parsing a setting from GVariant will
prefer the "config" (if present), in most cases also validation is
performed.
Likewise, settings plugins (keyfile, ifcfg-rh) only persist the JSON
config to disk. When loading a setting from file, strict validation is
also not performed.
The stricter validation only happens if as last operation one of the
artificial properties was set, or if the setting was created from a
GVariant that has no "config" field.
--
This is a (another) change in behavior.
The order of the fields in the JSON object does not really matter.
Note that with the recent rework the order changed. Before it was
arbitrarily, now it still is arbitrary.
Reorder again, to follow the same order as `man teamd.conf`.
Commit d6ec009afd ('team: normalize invalid configuration during
load') let's keyfile reader ignore JSON configs that cannot be parsed.
Keep doing that, but don't parse the JSON twice for that.
Just set the JSON, and if the setting afterwards does not verify, reset
it to NULL. We also get a better error message and in most cases we
don't need to parse twice.
- re-use the existing meta-data. That is, don't duplicate the
dbus-names, the types, or the default value. Instead, parse the
settings according to these flags.
- notice and return if there are any suspicious/wrong keys in the
variant. For strict parsing, we should not silently ignore them.
- previously, the code used g_variant_lookup() to search for the
expected keys. As the number of keys that we look up is fixed, this
still had O(n). But if we want to detect and reject invalid keys,
we cannot use g_variant_lookup() but need to iterate the entire variant
once.
Completely refactor the team/JSON handling in libnm's NMSettingTeam and
NMSettingTeamPort.
- team handling was added as rh#1398925. The goal is to have a more
convenient way to set properties than constructing JSON. This requires
libnm to implement the hard task of parsing JSON (and exposing well-understood
properties) and generating JSON (based on these "artificial" properties).
But not only libnm. In particular nmcli and the D-Bus API must make this
"simpler" API accessible.
- since NMSettingTeam and NMSettingTeamPort are conceptually the same,
add "libnm-core/nm-team-utils.h" and NMTeamSetting that tries to
handle the similar code side-by-sdie.
The setting classes now just delegate for everything to NMTeamSetting.
- Previously, there was a very fuzzy understanding of the provided
JSON config. Tighten that up, when setting a JSON config it
regenerates/parses all other properties and tries to make the
best of it. When modifying any abstraction property, the entire
JSON config gets regenerated. In particular, don't try to merge
existing JSON config with the new fields. If the user uses the
abstraction API, then the entire JSON gets replaced.
For example note that nm_setting_team_add_link_watcher() would not
be reflected in the JSON config (a bug). That only accidentally worked
because client would serializing the changed link watcher to
GVariant/D-Bus, then NetworkManager would set it via g_object_set(),
which would renerate the JSON, and finally persist it to disk. But
as far as libnm is concerned, nm_setting_team_add_link_watcher() would
bring the settings instance in an inconsistent state where JSON and
the link watcher property disagree. Setting any property must
immediately update both the JSON and the abstraction API.
- when constucting a team setting from D-Bus, we would previously parse
both "config" and abstraction properties. That is wrong. Since our
settings plugins only support JSON, all information must be present
in the JSON config anyway. So, when "config" is present, only the JSON
must be parsed. In the best case, the other information is redudant and
contributes nothing. In the worse case, they information differs
(which might happen if the client version differs from the server
version). As the settings plugin only supports JSON, it's wrong to
consider redundant, differing information from D-Bus.
- we now only convert string to JSON or back when needed. Previously,
setting a property resulted in parsing several JSON multiple times
(per property). All operations should now scale well and be reasonably
efficient.
- also the property-changed signals are now handled correctly. Since
NMTeamSetting knows the current state of all attributes, it can emit
the exact property changed signals for what changed.
- we no longer use libjansson to generate the JSON. JSON is supposed
to be a machine readable exchange format, hence a major goal is
to be easily handled by applications. While parsing JSON is not so
trivial, writing a well-known set of values to JSON is.
The advantage is that when you build libnm without libjansson support,
then we still can convert the artificial properties to JSON.
- Requiring libjansson in libnm is a burden, because most of the time
it is not needed (as most users don't create team configurations). With
this change we only require it to parse the team settings (no longer to
write them). It should be reasonably simple to use a more minimalistic
JSON parser that is sufficient for us, so that we can get rid of the
libjansson dependency (for libnm). This also avoids the pain that we have
due to the symbol collision of libjansson and libjson-glib.
https://bugzilla.redhat.com/show_bug.cgi?id=1691619
NMTeamLinkWatcher are immutable and ref-counted.
As such, there is little reason to ever duplicate them. Just increase
the ref-count.
For this change to be safe in all cicumstances, make ref/unref of the
link watcher thread-safe. After all, NMTeamLinkWatcher is public API,
and while libnm generally is not thead-safe, the cost of this is small.
Don't rely on the same memory layout for the NMTeamLinkWatcher union to
hold data for the watcher types. It looks wrong and is error prone.
Also, as we no longer require the full union, we can allocate a smaller
structure for ethtool. And while at it, we can embed the strings
directly instead of allocating them separately.
The point is to have a more efficient representation (memory wise).
Add internal nm_team_link_watcher_cmp() function.
We can implement nm_team_link_watcher_equal() based on that.
This was we can sort link watchers so that nm_team_link_watchers_equal()
with ignore_order is O(n*log(n)) instead of O(n^2).
In general, as basic API cmp() functions are as much effort to implement
as equal(), but they can also be used for sorting.
In nm_team_link_watcher_cmp(), only compare the fields that are set
according to the link watcher type.
NMTeamLinkWatcher is immutable, meaning there is no way to change an
existing instance (aside increasing/decreasing the ref-count).
Hence, all API is implicitly const.
Still, mark the arguments as const.
We should not just disable tests with an #if.
Instead, mark them as skipped. This way, we still compile them, and we
even run them (showing a message why they are skipped).
Now that unit tests no longer dynamically link against libnm/libnm.la,
and statically against the same code, we can enable this assertion
again.
This reverts commit 7a0e347b38.
libnm/tests/test-general statically links against libnm/libnm-utils.la
and dynamically against libnm/libnm.so. Hence, _nm_utils_init() is invoked
twice, failing the assertion.
That is a bug that must be fixed. For now, just don't assert.
Kernel calls the netlink attribute TCA_FQ_CODEL_MEMORY_LIMIT. Likewise,
iproute2 calls this "memory_limit".
Rename because TC parameters are inherrently tied to the kernel
implementation and we should use the familiar name.
iproute2 uses the special value ~0u to indicate not to set
TCA_FQ_CODEL_CE_THRESHOLD in RTM_NEWQDISC. When not explicitly
setting the value, kernel treats the threshold as disabled.
However note that 0xFFFFFFFFu is not an invalid threshold (as far as
kernel is concerned). Thus, we should not use that as value to indicate
that the value is unset. Note that iproute2 uses the special value ~0u
only internally thereby making it impossible to set the threshold to
0xFFFFFFFFu). But kernel does not have this limitation.
Maybe the cleanest way would be to add another field to NMPlatformQDisc:
guint32 ce_threshold;
bool ce_threshold_set:1;
that indicates whether the threshold is enable or not.
But note that kernel does:
static void codel_params_init(struct codel_params *params)
{
...
params->ce_threshold = CODEL_DISABLED_THRESHOLD;
static int fq_codel_change(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
...
if (tb[TCA_FQ_CODEL_CE_THRESHOLD]) {
u64 val = nla_get_u32(tb[TCA_FQ_CODEL_CE_THRESHOLD]);
q->cparams.ce_threshold = (val * NSEC_PER_USEC) >> CODEL_SHIFT;
}
static int fq_codel_dump(struct Qdisc *sch, struct sk_buff *skb)
{
...
if (q->cparams.ce_threshold != CODEL_DISABLED_THRESHOLD &&
nla_put_u32(skb, TCA_FQ_CODEL_CE_THRESHOLD,
codel_time_to_us(q->cparams.ce_threshold)))
goto nla_put_failure;
This means, kernel internally uses the special value 0x83126E97u to indicate
that the threshold is disabled (WTF). That is because
(((guint64) 0x83126E97u) * NSEC_PER_USEC) >> CODEL_SHIFT == CODEL_DISABLED_THRESHOLD
So in kernel API this value is reserved (and has a special meaning
to indicate that the threshold is disabled). So, instead of adding a
ce_threshold_set flag, use the same value that kernel anyway uses.
The memory-limit is an unsigned integer. It is ugly (if not wrong) to compare unsigned
values with "-1". When comparing with the default value we must also use an u32 type.
Instead add a define NM_PLATFORM_FQ_CODEL_MEMORY_LIMIT_UNSET.
Note that like iproute2 we treat NM_PLATFORM_FQ_CODEL_MEMORY_LIMIT_UNSET
to indicate to not set TCA_FQ_CODEL_MEMORY_LIMIT in RTM_NEWQDISC. This
special value is entirely internal to NetworkManager (or iproute2) and
kernel will then choose a default memory limit (of 32MB). So setting
NM_PLATFORM_FQ_CODEL_MEMORY_LIMIT_UNSET means to leave it to kernel to
choose a value (which then chooses 32MB).
See kernel's net/sched/sch_fq_codel.c:
static int fq_codel_init(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
{
...
q->memory_limit = 32 << 20; /* 32 MBytes */
static int fq_codel_change(struct Qdisc *sch, struct nlattr *opt,
struct netlink_ext_ack *extack)
...
if (tb[TCA_FQ_CODEL_MEMORY_LIMIT])
q->memory_limit = min(1U << 31, nla_get_u32(tb[TCA_FQ_CODEL_MEMORY_LIMIT]));
Note that not having zero as default value is problematic. In fields like
"NMPlatformIP4Route.table_coerced" and "NMPlatformRoutingRule.suppress_prefixlen_inverse"
we avoid this problem by storing a coerced value in the structure so that zero is still
the default. We don't do that here for memory-limit, so the caller must always explicitly
set the value.
I think initializing structs should (almost) be always done with designated
initializers, because otherwise it's easy to get the order wrong. The
problem is that otherwise the order of fields gets additional meaning
not only for the memory layout, but also for the code that initialize
the structs.
Add a macro NM_VARIANT_ATTRIBUTE_SPEC_DEFINE() that replaces the other
(duplicate) macros. This macro also gets it right to mark the struct as
const.
This actually allows the compiler/linker to mark the memory as read-only and any
modification will cause a segmentation fault.
I would also think that it allows the compiler to put the structure directly
beside the outer constant array (in which this pointer is embedded). That is good
locality-wise.
- g_ascii_strtoll() accepts leading spaces, but it leaves
the end pointer at the first space after the digit. That means,
we accepted "1: 0" but not "1 :0". We should either consistently
accept spaces around the digits/colon or reject it.
- g_ascii_strtoll() accepts "\v" as a space (just like `man 3 isspace`
comments that "\v" is a space in C and POSIX locale.
For some reasons (unknown to me) g_ascii_isspace() does not treat
"\v" as space. And neither does NM_ASCII_SPACES and
nm_str_skip_leading_spaces().
We should be consistent about what we consider spaces and what not.
It's already odd to accept '\n' as spaces here, but well, lets do
it for the sake of consistency (so that it matches with our
understanding of ASCII spaces, albeit not POSIX's).
- don't use bogus error domains in "g_set_error (error, 1, 0, ..."
That is a bug and we have NM_UTILS_ERROR exactly for error instances
with unspecified domain and code.
- as before, accept a trailing ":" with omitted minor number.
- reject all unexpected characters. strtoll() accepts '+' / '-'
and a "0x" prefix of the numbers (and leading POSIX spaces). Be
strict here and only accepts NM_ASCII_SPACES, ':', and hexdigits.
In particular, don't accept the "0x" prefix.
This parsing would be significantly simpler to implement, if we could
just strdup() the string, split the string at the colon delimiter and
use _nm_utils_ascii_str_to_int64() which gets leading/trailing spaces
right. But let's save the "overhead" of an additional alloc.
Merge the function pointer get_func() into to_dbus_fcn().
Previously, get_func() as handled separately from to_dbus_fnc()
(formerly synth_func()). The notion was that synth-func would syntetize
properties that are D-Bus only. But that distinction does not seem
very helpful to me.
Instaed, we want to convert a property to D-Bus. Period. The
implementation should be handled uniformly. Hence, now that is
all done by property_to_dbus().
Note that property_to_dbus() is also called as default implementation
for compare-property. At least, for properties that are backed by a
GObject property.
The naming was not very clear. How does get_func(), synth_func()
and to_dbus() relate? What does synth_func() do anyway?
Answers:
- get_func() and synth_func() do very similar. They should be merged
in a next step.
synth_func() has the notion of "synthetize" a property for
D-Bus. As such, these properties are a bit unusual in that they
don't have a backing GObject property in the setting. But it'd
rather treat such properties like other properties. The step
in that direction will be to merge the to-dbus functions.
- to_dbus() converts a GValue of the GObject property go GVariant.
It's a simplified form of get_func()/synth_func() and a better name
is gprop_to_dbus_fcn().
The same for gprop_from_dbus_fcn().
For now, just rename.
We have certain artificial properties that not only depend on one
property alone or that depend on a property in another(!) setting.
For that, we have synth_func.
Other than that, synth_func and get_func are really fundamentally
similar and should be merged. That is because the distinction whether a
property value is "synthetized" or just based on a plain property is
minor. It's better to have the general concept of "convert property to
GVariant" in one form only.
Note that compare_property() is by default implemented based
on get_func. Hence, if get_func and synth_func get merged,
compare_property() will also require access to the NMConnection.
Also it makes some sense: some properties are artificial and actually
stored in "another" setting of the connection. But still, the property
descriptor for the property is in this setting. The example is the
"bond.interface-name" which only exists on D-Bus. It's stored as
"connection.interface-name".
I don't really like to say "exists on D-Bus only". It's still a valid
property, despite in NMSetting it's stored somehow differently (or not
at all). So, this is also just a regular property for which we have a
property-info vtable.
Does it make sense to compare such properties? Maybe. But the point is that
compare_property() function needs sometimes access to the entire
connection. So add the argument.
Always properly set NMSettInfoProperty.dbus_type, instead of leaving it
unspecified for GObject property based properties, and detect it each
time anew with variant_type_for_gtype().
Instead, autodetect and remember the dbus-type during _properties_override_add_struct().
For types that need special handling (GBytes, enums and flags) set a to_dbus() function.
This allows us to handle properties uniformly by either calling the to_dbus() function
or g_dbus_gvalue_to_gvariant().
- use cleanup attribute in get_property_for_dbus() and return early.
- use NM_FLAGS_HAS() macro in _nm_setting_to_dbus().
- in nm_setting_get_dbus_property_type() use g_return*() asserts
instead of crash or hard asserts.
- return early from variant_type_for_gtype().
- the previous implementation of nm_setting_wired_get_s390_option()
returned the elements in an arbitrary order (because it just iterated
idx times over the unsorted hash table).
- the API for "s390-options" suggests both accessing by index and by
name. Storing the options in a hash-table is not optimal for lookup
by index. It also requires us to sort the elements over and over
again.
Use instead a sorted array. Note that add/remove of course requires to
move the elements (and has thus O(n)).
- "s390-options" are very seldomly set. We shouldn't pay the price in every
NMSettingWired to allocate a GHashTable and deal with it.
- don't assert in nm_setting_wired_add_s390_option() and
nm_setting_wired_remove_s390_option() that the key is valid.
ifcfg-rh reader understandably does not want to implement additional
logic to pre-validate the key, so any invalid keys would trigger an
assertion failure. We have verify() for this purpose.
Currently, nm_setting_wired_get_s390_option() returns the key
in an undefined order. Hence, the keyfile writer and the test
need to awkwardly sort the keys first. That will be solved better
in the next commit, when nm_setting_wired_get_s390_option() returns
the items sorted by key.
Most of the caller won't require a deep-clone of the attribute
names. Likely, the fetch the name, so they can lookup the attributes.
In that common case, there is no need to clone the strings themself.
"libnm-core" implements common functionality for "NetworkManager" and
"libnm".
Note that clients like "nmcli" cannot access the internal API provided
by "libnm-core". So, if nmcli wants to do something that is also done by
"libnm-core", , "libnm", or "NetworkManager", the code would have to be
duplicated.
Instead, such code can be in "libnm-libnm-core-{intern|aux}.la".
Note that:
0) "libnm-libnm-core-intern.la" is used by libnm-core itsself.
On the other hand, "libnm-libnm-core-aux.la" is not used by
libnm-core, but provides utilities on top of it.
1) they both extend "libnm-core" with utlities that are not public
API of libnm itself. Maybe part of the code should one day become
public API of libnm. On the other hand, this is code for which
we may not want to commit to a stable interface or which we
don't want to provide as part of the API.
2) "libnm-libnm-core-intern.la" is statically linked by "libnm-core"
and thus directly available to "libnm" and "NetworkManager".
On the other hand, "libnm-libnm-core-aux.la" may be used by "libnm"
and "NetworkManager".
Both libraries may be statically linked by libnm clients (like
nmcli).
3) it must only use glib, libnm-glib-aux.la, and the public API
of libnm-core.
This is important: it must not use "libnm-core/nm-core-internal.h"
nor "libnm-core/nm-utils-private.h" so the static library is usable
by nmcli which couldn't access these.
Note that "shared/nm-meta-setting.c" is an entirely different case,
because it behaves differently depending on whether linking against
"libnm-core" or the client programs. As such, this file must be compiled
twice.
