Commit graph

88 commits

Author SHA1 Message Date
Wen Liang
a9e6aa663e aliyun: reuse ipv4 gateway address returned by metadata server
The default ipv4 gateway address of the VPC in Aliyun cloud is not the
first IP address in the CIDR subnet block, we should instead use the
ipv4 gateway address retrieved from the metadata server in
`_nmc_mangle_connection()`.

https://bugzilla.redhat.com/show_bug.cgi?id=1823315

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/958

Signed-off-by: Wen Liang <liangwen12year@gmail.com>
(cherry picked from commit 778e1f8493)
(cherry picked from commit 59633dbe11)
2021-10-05 09:35:48 +02:00
Thomas Haller
661da869b3 cloud-setup: limit number of supported interfaces to avoid overlapping table numbers
The table number is chosen as 30400 + iface_idx. That is, the range is
limited and we shouldn't handle more than 100 devices. Add a check for
that and error out.

(cherry picked from commit b68d694b78)
(cherry picked from commit 292233e16e)
2021-10-05 09:35:48 +02:00
Thomas Haller
67a83b54cd cloud-setup: process iface-datas in sorted order
The routes/rules that are configured are independent of the
order in which we process the devices. That is, because they
use the "iface_idx" for cases where there is ambiguity.

Still, it feels nicer to always process them in a defined order.

(cherry picked from commit a95ea0eb29)
(cherry picked from commit 6302cd416d)
2021-10-05 09:35:48 +02:00
Thomas Haller
48e79fb4b2 cloud-setup: track sorted list of NMCSProviderGetConfigIfaceData
Sorted by iface_idx. The iface_idx is probably something useful and
stable, provided by the provider. E.g. it's the order in which
interfaces are exposed on the meta data.

(cherry picked from commit 1c5cb9d3c2)
(cherry picked from commit 0a2ed62703)
2021-10-05 09:35:48 +02:00
Thomas Haller
c36e42dbd9 cloud-setup: add "hwaddr" to NMCSProviderGetConfigIfaceData struct
get-config() gives a NMCSProviderGetConfigResult structure, and the
main part of data is the GHashTable of MAC addresses and
NMCSProviderGetConfigIfaceData instances.

Let NMCSProviderGetConfigIfaceData also have a reference to the MAC
address. This way, I'll be able to create a (sorted) list of interface
datas, that also contain the MAC address.

(cherry picked from commit ec56fe60fb)
(cherry picked from commit cc289e5369)
2021-10-05 09:35:48 +02:00
Thomas Haller
8bad924931 cloud-setup/trivial: rename variables in Azure's _get_config_fetch_done_cb()
The previous name seem not very expressive/fitting. Naming is hard, but
I think these are better names.

(cherry picked from commit 89f3267859)
2021-10-05 09:35:48 +02:00
Thomas Haller
91ddd5f5ba cloud-setup: use _nm_utils_ascii_str_to_int64_bin() in Azure's _get_config_fetch_done_cb()
(cherry picked from commit a2fded3cee)
2021-10-05 09:35:48 +02:00
Thomas Haller
2e4b73c7fe cloud-setup: skip configuring policy routing if there is only one interface/address
nm-cloud-setup automatically configures the network. That may conflict
with what the user wants. In case the user configures some specific
setup, they are encouraged to disable nm-cloud-setup (and its
automatism).

Still, what we do by default matters, and should play as well with
user's expectations. Configuring policy routing and a higher priority
table (30400+) that hijacks the traffic can cause problems.

If the system only has one IPv4 address and one interface, then there
is no point in configuring policy routing at all. Detect that, and skip
the change in that case.

Note that of course we need to handle the case where previously multiple
IP addresses were configured and an update gives only one address. In
that case we need to clear the previously configured rules/routes. The
patch achieves this.

(cherry picked from commit 5f047968d7)
(cherry picked from commit 8bc8a0f56b)
2021-10-05 09:35:48 +02:00
Thomas Haller
2b96e9314c cloud-setup: preserve IPv4 addresses/routes/rules from profile
nm-cloud-setup automatically detects routes, addresses and rules and configures them
on the device using the emphermal Reapply() API. That is, it does not modify the
existing profile (on disk), but changes the runtime configuration only.

As such, it used to wipe otherwise statically configured IP addresses, routes and
rules. That seems unnecessary. Let's keep the configuration from the (persistent)
configuration.

There is of course the problem that nm-cloud-setup doesn't really
understand the existing IP configuration, and it can only hope that
it can be meaningfully combined with what nm-cloud-setup wants to
configure. This should cover most simple cases, for more complex setups,
the user probably should disable nm-cloud-setup and configure the
network explicitly to their liking.

https://bugzilla.redhat.com/show_bug.cgi?id=1971527

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/893
(cherry picked from commit 4201ee5119)
(cherry picked from commit 9541b0bea4)
2021-10-05 09:35:48 +02:00
Thomas Haller
ef7a97977a cloud-setup: count numbers of valid IPv4 addresses in get-config result
Will be used next.

(cherry picked from commit 7969ae1a82)
(cherry picked from commit ae504433f1)
2021-10-05 09:35:48 +02:00
Thomas Haller
7fcc89db6e cloud-setup: cache number of valid interfaces in get-config result
Now that we return a struct from get_config(), we can have system-wide
properties returned.

Let it count and cache the number of valid iface_datas.

Currently that is not yet used, but it will be.

(cherry picked from commit a3cd66d3fa)
(cherry picked from commit e74375fc3b)
2021-10-05 09:35:48 +02:00
Thomas Haller
b2ed9e7d5d cloud-setup: return structure for get_config() result instead of generic hash table
Returning a struct seems easier to understand, because then the result
is typed.

Also, we might return additional results, which are system wide and not
per-interface.

(cherry picked from commit 323e182768)
(cherry picked from commit c94b1c43d4)
2021-10-05 09:35:48 +02:00
Thomas Haller
d0552e00d0
cloud-setup/tests: add test for nmcs_setting_ip_replace_ipv4_addresses() 2021-02-10 18:35:13 +01:00
Thomas Haller
7e8809a9c5
cloud-setup/tests: add unit test for nm-cloud-setup 2021-02-10 18:23:44 +01:00
Thomas Haller
bbd36be44a
cloud-setup: fix replacing IPv4 addresses during update
If previously the profile would track two addresses ("10.116.1.130/24",
"10.116.1.65/24"), and during an update the second address was removed
(leaving "10.116.1.130/24"), then the addresses of the profile were
wrongly not changed.

The effect is that removing a secondary IP address might not take
effect.

Fix that.

https://bugzilla.redhat.com/show_bug.cgi?id=1920838

Fixes: 69f048bf0c ('cloud-setup: add tool for automatic IP configuration in cloud')
2021-02-10 18:23:44 +01:00
Thomas Haller
94a75c8cd6
cloud-setup: build intermediate library "libnm-cloud-setup-core.a"
Next we'll add unit tests, hence we need to link against the code
but we don't want to compile it twice.
2021-02-10 18:23:44 +01:00
Thomas Haller
606612ea59
all: add "libnm/nm-default-client.h" as replacement for "nm-default.h" 2021-02-09 12:38:17 +01:00
Thomas Haller
5f47a2d44d
build/meson: cleanup clients/***/meson.build 2021-02-09 12:38:16 +01:00
Thomas Haller
684019d6f1
cloud-setup: add Documentation link to "nm-cloud-setup.service" 2021-02-02 18:13:02 +01:00
Thomas Haller
150c18130e
cloud-setup: add comment to "nm-cloud-setup.service" about drop-in files 2021-02-02 18:09:27 +01:00
Thomas Haller
8128f791c9
cloud-setup: assign incremental iface_idx for Azure,GCP get_config() result
We use the iface_idx for example to determine the routing table,
by using table 30400+iface_idx.

While the HTTP API for Azure has a index, it does not mean that we
should use that index as-is for our purpose.

Instead, treat those indexes separately and ensure that the
iface_idx that we return is numbering the interfaces starting
from zero.
2021-01-08 16:32:50 +01:00
Thomas Haller
e81b442d8b
cloud-setup: fail get_config() for Azure on invalid prefix
While it's not clear whether we should be strict or forgiving
when fetching the HTTP meta data, we should be consistent.

On a parse error of the IP addresses we fail. Hence also
fail on a parse error for the subnet.
2021-01-08 16:32:49 +01:00
Thomas Haller
fc8315cd94
cloud-setup: use nmcs_utils_ipaddr_normalize_bin() in Azure provider 2021-01-08 16:32:49 +01:00
Thomas Haller
288b38e719
cloud-setup: use nmcs_utils_ipaddr_normalize_gbytes() in GCP provider 2021-01-08 16:32:49 +01:00
Thomas Haller
30e1f73347
cloud-setup: add nmcs_utils_ipaddr_normalize*() helper 2021-01-08 16:32:49 +01:00
Thomas Haller
f0faf2e1a1
cloud-setup: handle unknown interaces in get_config() for GCP/Azure
The API of mcs_provider_get_config() allows to explicitly request
for certain interfaces (MAC addresses), but it also allows to fetch
any.

That means, the result dictionary will be pre-populated with the
MAC addresses that were requested, but if we encounter an unknown
interface, then that is not a reason to fail.
2021-01-08 16:32:48 +01:00
Thomas Haller
511b4ab411
cloud-setup: add and use nmcs_utils_hwaddr_normalize_gbytes()
Previously we would call

    nmcs_utils_hwaddr_normalize(g_bytes_get_data(response, NULL), -1);

which treats the data in response as NUL terminated. That is not
entirely wrong, because the HTTP request's response is guaranteed
to have a NUL termination at the end. However, it doesn't seam to good
either.

For one, we already have the length. Use it. But also, if the response
contains any NUL bytes in the middle, then this would wrongly only
consider the first line. We should not accept "00:11:22:33:44:55\0bogus"
as valid.

While at it, reject NUL characters from nmcs_utils_hwaddr_normalize() --
except one NUL at the end.
2021-01-08 16:32:48 +01:00
Thomas Haller
2ba984a80a
cloud-setup: strip whitespace from nmcs_utils_hwaddr_normalize()
This function should be accepting, and not reject leading/trailing
white space.
2021-01-08 16:32:48 +01:00
Thomas Haller
494819bbbf
cloud-setup: move common code for get_config() to base class and improve cancellation
First note that all three provider implementations are very similar.
That is why NMCSProvider's implementation does already some work that
is common to all implementations. For example, it provides the
NMCSProviderGetConfigTaskData structure to help tracking the data of
the request.

Also note that the GCP/Azure implementations didn't handle the
cancellation correctly. They always would pass

    g_task_get_cancellable(get_config_data->task)

to the asynchronous requests. That is the GCancellable provider by the
caller. That is fine when there is only one async operation ongoing. But
that is not the case, we have parallel HTTP requests.
Then, when an error happened, the overall get_config() operations fails
and the still pending requests should all be aborted. However, we must
not cancel the GCancellable of the user (because that is not owned by us).
The correct solution is to use an internal cancellable in those cases.

Anyway. Since all of this is similar, we can extend the base class
to handle things for us. This also gets the cancellation right by having
a "get_config_data->intern_cancellable".
2021-01-08 16:32:48 +01:00
Thomas Haller
5fb2f7e717
cloud-setup/trivial: rename "response_data" variable
We have three implementations of providers, that all do something similar.
Name the variable with the HTTP response GBytes the same everywhere.
2021-01-08 15:24:43 +01:00
Thomas Haller
9afdbb97ea
cloud-setup/trivial: rename variables "config_data" to "get_config_data"
The code is not entirely straight forward. Consistent naming
is hence important.

In "nmcs-provider-ec2.c", variables of this kind are called
"get_config_data". That also matches to the type of the data
(NMCSProviderGetConfigTaskData).

Rename the variables to make naming consistent. Also, I find the
longer name to be clearer.
2021-01-08 15:24:43 +01:00
Thomas Haller
977ea352a0
all: update deprecated SPDX license identifiers
These SPDX license identifiers are deprecated ([1]). Update them.

[1] https://spdx.org/licenses/

  sed \
     -e '1 s%^/\* SPDX-License-Identifier: \(GPL-2.0\|LGPL-2.1\)+ \*/$%/* SPDX-License-Identifier: \1-or-later */%' \
     -e '1,2 s%^\(--\|#\|//\) SPDX-License-Identifier: \(GPL-2.0\|LGPL-2.1\)+$%\1 SPDX-License-Identifier: \2-or-later%' \
     -i \
     $(git grep -l SPDX-License-Identifier -- \
         ':(exclude)shared/c-*/' \
         ':(exclude)shared/n-*/' \
         ':(exclude)shared/systemd/src' \
         ':(exclude)src/systemd/src')
2021-01-05 09:46:21 +01:00
Thomas Haller
0fca809bfd
all: explicit include <linux/if_{ether,infiniband,vlan}.h> as needed
Currently libnm headers include <linux/if_{ether,infiniband,vlan}.h>.
These are public headers, that means we drag in the linux header to all
users of <NetworkManager.h>.

Often the linux headers work badly together with certain headers from libc.
Depending on the libc version, you have to order linux headers in the right
order with respect to libc headers.

We should do better about libnm headers. As a first step, assume that
the linux headers don't get included by libnm, and explicitly include
them where they are needed.
2020-12-22 16:33:33 +01:00
Thomas Haller
be8a3f9902
cloud-setup: simplify cancellation in _get_config_fetch_cancelled_cb()
If we call g_cancellable_connect() on a GCancellable that is already
cancelled, then the callback is invoked synchronously. We need to
handle that.

However, we can slightly simplify the code. There is no change in
behavior, but we can always let the cancelled callback return the
result.
2020-12-11 17:36:37 +01:00
Thomas Haller
422ab25626
cloud-setup: in EC2's _get_config_task_maybe_return() cancel internal requests on any error
"iface_data->cancellable" is an internal cancellable for the parallel
HTTP requests. Once we encounter a failure, those requests are all
obsolete and must be cancelled.
2020-12-11 17:36:37 +01:00
Thomas Haller
399c04e810
cloud-setup: fix handling cancellation of internal GET operation for EC2 provider
There are two GCancellable at work: one is provided by the user
during nmcs_provider_get_config(), and one is used internally for the
individual HTTP GET requests.

In _get_config_fetch_done_cb(), if the error reason is "cancelled",
then it means that our internal iface_data->cancellable was cancelled.
Probably because an error happend (like a timeout or the user cancelled
the external GCancellable).

In that case, we must not report that the task completed with a
cancellation, because we need to preserve the error that was the
original cause.
2020-12-11 17:36:37 +01:00
Thomas Haller
88071abb43
all: unify comment style for SPDX-License-Identifier tag
Our coding style recommends C style comments (/* */) instead of C++
(//). Also, systemd (which we partly fork) uses C style comments for
the SPDX-License-Identifier.

Unify the style.

  $ sed -i '1 s#// SPDX-License-Identifier: \([^ ]\+\)$#/* SPDX-License-Identifier: \1 */#' -- $(git ls-files -- '*.[hc]' '*.[hc]pp')
2020-09-29 16:50:53 +02:00
Thomas Haller
740b092fda
format: replace tabs for indentation in code comments
sed -i \
     -e 's/^'$'\t'' \*/     */g' \
     -e 's/^'$'\t\t'' \*/         */g' \
     -e 's/^'$'\t\t\t'' \*/             */g' \
     -e 's/^'$'\t\t\t\t'' \*/                 */g' \
     -e 's/^'$'\t\t\t\t\t'' \*/                     */g' \
     -e 's/^'$'\t\t\t\t\t\t'' \*/                         */g' \
     -e 's/^'$'\t\t\t\t\t\t\t'' \*/                             */g' \
     $(git ls-files -- '*.[hc]')
2020-09-28 16:07:52 +02:00
Antonio Cardace
328fb90f3e
all: reformat all with new clang-format style
Run:

    ./contrib/scripts/nm-code-format.sh -i
    ./contrib/scripts/nm-code-format.sh -i

Yes, it needs to run twice because the first run doesn't yet produce the
final result.

Signed-off-by: Antonio Cardace <acardace@redhat.com>
2020-09-28 16:07:51 +02:00
Thomas Haller
2d360d8293
cloud-setup: add code comment to nmcs_utils_hwaddr_normalize() 2020-09-23 13:57:38 +02:00
Thomas Haller
e8dd19bb01
shared: extend nm_utils_hexstr2bin_full() to require hexdigits in pairs
nm_utils_hexstr2bin_full() is our general hexstr to binary parsing
method. It uses (either mandatory or optional) delimiters. Before,
if delimiters are in use, it would accept individual hexdigits.
E.g. "a:b" would be accepted as "0a:0b:.

Add an argument that prevents accepting such single digits.
2020-09-22 17:40:41 +02:00
Thomas Haller
70971d1141
all: avoid wrong compiler warning about uninitalized variables with LTO
Seems with LTO the compiler can sometimes think that thes variables are
uninitialized. Usually those code paths are only after an assertion was
hit (g_return*()), but we still need to workaround the warning.
2020-08-17 15:18:02 +02:00
Thomas Haller
4c94a9736b
cloud-setup: rework error handling and completion for EC2 provider
Make the error handling similar to the other provider implementations.

- only actually return once all callbacks completed.

- cache the first error and report it.
2020-07-30 09:18:40 +02:00
Thomas Haller
9887ea5b61
cloud-setup: cleanup error handling in Azure provider
- drop AzureData.success field. It is redundant to have AzureData.error set.
  Also it was actually unused.

- ensure that we keep the first error passed during
  _get_config_maybe_task_return(). Once we set an error, that error gets
  returned. There is a twist here, that we prefer cancellation errors
  over other error reasons.
2020-07-30 09:18:40 +02:00
Thomas Haller
2582694b46
cloud-setup: cleanup error handling in GCP provider
- drop GCPData.success field. It is redundant to have GCPData.error set.
  Also, it's meaningless to indicate failure, if we don't have an error
  at hand.

- ensure that we keep the first error passed during
  _get_config_maybe_task_return(). Once we set an error, that error gets
  returned. There is a twist here, that we prefer cancellation errors
  over other error reasons.

- in _get_config_fip_cb(), ensure to call _get_config_maybe_task_return()
  even if we are not yet ready. That is useful to record a potential
  error.
2020-07-30 09:18:40 +02:00
Sayed Shah
df6cdd4315
cloud-setup: fix the host address for azure
Fixes: e7ac7290bd ('cloud-setup: add tool for automatic IP configuration in cloud')
2020-07-30 09:16:42 +02:00
Sayed Shah
e7ac7290bd
cloud-setup: add tool for automatic IP configuration in cloud
This is a tool for automatically configuring networking in azure
cloud environment.

This add a provider implementation for Azure that when detected fetches
the private ip addressess and the subnet prefix of configured internal
load balancers.

Once this information is fetched from the metadata server, it instructs
NetworkManager to add private ip addressess and subnet prefix for each
interface detected.

It is inspired by SuSE's cloud-netconfig ([1], [2]) and Azure Instance Metadata service [3].

[1] https://www.suse.com/c/multi-nic-cloud-netconfig-ec2-azure/
[2] https://github.com/SUSE-Enceladus/cloud-netconfig
[3] https://docs.microsoft.com/en-us/azure/virtual-machines/linux/instance-metadata-service

It is also intended to work without configuration. The main point is
that you boot an image with NetworkManager and nm-cloud-setup enabled,
and it just works.

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/572
2020-07-29 15:56:15 +02:00
Thomas Haller
e73bd2cf5f
cloud-setup: always replace addresses, routes and rules in _nmc_mangle_connection()
If the list of addresses, routes and rules is empty, we still want to mangle
the applied connection, to also have an empty list.

nm-cloud-setup has certain expectations. For example, that the static addresses,
routes and rules of the active connection is entirely under the control of the
tool. For example, so it usually replaces the lists entirely. It also should do
that, if the new list is empty.

Maybe, one day there could be more complex merging strategies, where the user could
also add static addresses, routes, or rules to the profile, and nm-cloud-setup
would preserve them. However, that is not implemented, nor is it clear how exactly
that would work.
2020-07-20 19:02:57 +02:00
Sayed Shah
e3cd3b73ed
cloud-setup: don't require delimiter in 'nmcs_utils_hwaddr_normalize'
Azure gives MAC address without delimiter. So, make the normalize function
more liberal so that it can accept this format.

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/576
2020-07-14 19:10:29 +02:00
Frazer Clews
16abfca78a
cloud-setup: fix nmcs_utils_poll argument ordering
the order of the arguments in the header and C file did not match

Fixes: 69f048bf0c ('cloud-setup: add tool for automatic IP configuration in cloud')

https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/574
2020-07-13 13:13:58 +02:00