These containers are ancient. Also, when we update ci-templates
they will no longer build (because then a different container hub
will be used, which doesn't contain those images). Drop them.
"nm-code-format.sh" is going to change the default behavior from "-n" to
"-i", that is, from check-only to reformat. Explicitly pass "-n" where
we want it.
It fails to install the container. Disable it, until it is more
stable.
...
Install 363 Packages
Total download size: 275 M
Installed size: 1.1 G
Downloading Packages:
python3: allocatestack.c:191: advise_stack_range: Assertion `freesize < size' failed.
./contrib/fedora/REQUIRED_PACKAGES: line 17: 815 Aborted $NM_INSTALL "$@"
subprocess exited with status 134
subprocess exited with status 134
exit status 134
...
File "/usr/local/lib/python3.9/site-packages/jinja2/environment.py", line 1361, in generate
yield self.environment.handle_exception()
File "/usr/local/lib/python3.9/site-packages/jinja2/environment.py", line 925, in handle_exception
raise rewrite_traceback_stack(source=source)
File ".gitlab-ci/ci.template", line 178, in top-level template code
{% if not version in distro.always and (distro.name != pages_build.name or version != pages_build.version) %}
jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'always'
Ups, we actually still require libuuid. Actually, we only need to
to build the example script `examples/C/glib/add-connection-gdbus.c`.
The proper solution would be to make this an optional dependency.
So far this was not yet done. Also, libuuid is really an ubiquitous
dependency on Linux, so it's not really a problem to have this build
dependency, even if it's just to build the examples.
This reverts commit c0a3947ff9.
- the container that is also "pages_build" should always
run automatically. This can replace the "always" tag.
- comment out the "always: 33" part, because we no longer need
it. It was also wrong, because by now we should run Fedora 34
automatically.
On Fedora 33, we get it automatically because "clang" package
has an indirect (weak) dependency for clang-tools-extra. On
Fedora 34, that is no loger the case.
We need to explicitly install it.
- the python-black version on Fedora 33 was updated, and this formats
our python code differently. The black version that is used by our
gitlab-ci is the authoritative version that should be used. Update
it by regenerating the containers.
- especially Fedora:rawhide and Debian:sid often introduce changes that
will cause a break of our build. Update the containers to test latest
versions.
Alpine is especially interesting because it uses musl as libc.
The build does not yet succeed. There are several issues that
need to be fixed.
However, it will be simpler to fix things, if we have tests
in place -- even if at the moment they are known to be broken.
See-also: https://git.alpinelinux.org/aports/tree/community/networkmanager?h=master
Ubuntu/Debian and CentOS/Fedora are sufficiently similar that it's
better that we have only one variant of ".gitlab-ci/*-install.sh"
and "contrib/*/REQUIRED_PACKAGES".
This was already the case, however, we used to symlink
".gitlab-ci/centos-install.sh" to "fedora-install.sh". That
worked, but it didn't scale very well. For example, if we would follow
that pattern, we would also need a symlink "contrib/centos/REQUIRED_PACKAGES"
Or should "contrib/centos" symlink to "contrib/fedora"? That seems even
more wrong.
We already had the "distro.base_type" variable for that. Make use of
that also for the install script.
It's not clear that ci-fairy has a stable API. Instead, we pin the
version by specifying the git commit sha.
In the build script, install the pinned version.
ci-templates builds and caches the test containers. When the build
scripts, the ci-template or "config.yml" changes, we need to bump
the tag so that the containers get rebuild.
Partly automate this. The tag now gets generated by the template and
contains a checksum of certain build files. Thus, if you change
any build files, then `ci-fairy generate-template` would generate a
different tag. You can not miss that, because we have tests that ensure
that our ".gitlab-ci.yml" is up to date. Also, you no longer need to
manually bump the tag when a build script changes, just regenerate
".gitlab-ci.yml" with `ci-fairy generate-template`.
See also: https://gitlab.freedesktop.org/freedesktop/ci-templates/-/merge_requests/54
The goal is to run most distros only manually. However, it would be nice
to avoid (manually) clicking twice to start the tests for one distro:
once for the container preparation, and once for the actual test.
Previously, the container prep part was set to manual and the actual
test automatic. It worked almost as desired, except that this leads
to the entire gitlab-ci pipeline be be in running state indefinitely.
To fix that, always run the container prep steps. If the container is
cached, this is supposed to be fast and cheap. Now only the actual tests
are marked as "manual".
It seems "pages" test does not get properly triggered, if only
t_fedora:33 completes. It should, because the other distros are
optional. Try to set "needs" to fix that.
Certain parts of the code are entirely generated or must follow
a certain format that can be enforced by a tool. These invariants
must never fail:
- ci-fairy generate-template (check-ci-script)
- black python formatting
- clang-format C formatting
- msgfmt -vs
On the other hand, we also have a checkpatch script that checks
the current patch for common errors. These are heuristics and
only depend on the current patch (contrary to the previous type
that depend on the entire source tree).
Refactor the gitlab-ci tests:
- split "checkpatch" into "check-patch" and "check-tree".
- merge the "check-ci-script" test into "check-tree".
On one image we do extra work, like generating documentation (gitlab-pages).
The same image is currently also used for the "checkpatch" step. That step
checks code formatting using clang-format. The formatting depends on the
clang version, and we currently choose Fedora 33 as the desired version
for formatting.
It means, the "checkpatch" step requires Fedora 33. We could choose a
different image for generating pages and run check patch. However, that
might not be best. Just also generate the pages using Fedora 33.
All the steps of "checkpatch" test (except the last) check
the current tree for consistency. Those checks must always
pass.
Only the last step calls the "checkpatch-feature-branch.sh".
That script checks for common patterns, like avoiding g_assert()
(in favor of other assertion types). That last check only checks
the current patch, and there are many cases where the test is
known to fail (because these are just heuristics). As such, the
step that may fail should be called as last.
ci-templates encourages building specific containers that can be re-used:
- containers are re-used across pipelines, producing consistent results
- containers are re-used by contributors since they will use the upstream
containers for their MR, thus guaranteeing the same results.
Containers are automatically rebuild whenever the respective
FDO_DISTRIBUTION_TAG changes. This is particularly interesting now that
Docker Hub will introduce pull limits.
This CI script consists of a config file and a jinja2 template, simply
running 'ci-fairy generate-template' produces the .gitlab-ci.yml.
ci-fairy is part of the freedesktop.org ci-templates and can be pip
installed, see the check-ci-script job.
Functional changes to the previous script:
- new job: check-ci-script, verifies that our gitlab-ci.yml is the one
generated by the sources
- Added distributions:
- Fedora 33
- The actual work is now down by a set of scripts in .gitlab-ci/,
specifically:
- .gitlab-ci/build.sh is the previous do_build job
- .gitlab-ci/{fedora|debian}-install.sh are the previous {fedora|debian}_install jobs
symlinks are in place for centos and ubuntu
Why the scripts instead of steps in the CI? Easer to reading and
reproduce. With the containers being static, it's easy to pull one
locally and re-run the CI job to reproduce an issue. Having everything in a
single script makes that trivial.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/664
There is no need spawn up two containers (and install similar packages, doing so is
resource intensive), to run two different tests. We can run them in the same
container.
At this point Fedora 33 is not yet fully released. But it will happen soon, so the
image is stable enough for these kinds of tests (and will only get more
stable in the following weeks).
Also no longer collect an artifact with the test output. The test output
is already nicely visible in the gitlab-ci UI. No need to collect it
separately.
