The user can set "$NMTST_LIBTOOL" to empty or the path to libtool.
If unset, we want to detect it. However, previously we would always
use "$SCRIPT_PATH/../libtool", even if that file doesn't exit. Improve
that.
For example, when running on Ubuntu with the ".gitlab-ci/debian-install.sh"
script, we don't have libtool. Entering such a container and running the
"run-nm-test.sh" script will fail as "$SCRIPT_PATH/../libtool" doesn't
exist.
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1800
This is important. We must not swallow 77, which has the meaning
that the test was skipped.
Fixes: f65747f6e9 ('tests: let "run-nm-test.sh" fail with exit code 1 on failure')
(cherry picked from commit aff40f736c)
`git bisect run` is peculiar about the exit code:
error: bisect run failed: exit code 134 from '...' is < 0 or >= 128
If we just "exec" the test, it usually will fail on an assert. That results
in SIGABRT or exit code 134. So out of the box that is annoying with
git-bisect.
Work around that and let the test wrapper always coerce any test failure
to exit code 1.
(cherry picked from commit f65747f6e9)
Why? Because I often use a command line like
$ ./tools/run-nm-test.sh -m src/libnm-client-impl/tests/test-nm-client -p /libnm/device-connection-compatibility
or even alias it to a one character command `x`.
Usually I want to do the rebuild, and as `make` is so slow, it
adds noticeable time running the command. Thus, sometimes I want
to modify the command, for which I have to edit the command from the
history, or toggle two separate commands.
Add a `-M` flag that can reverse the effect of an earlier `-m`.
An "enable" flag in general should just also have a "disable" flag.
valgrind might log warnings about syscalls that it doesn't implement.
When we run valgrind tests, we check that the command exits with
success, but we also check that there is no unexpected content in the
valgrind log.
Those warnings are not relevant for us. We don't unit-tests valgrind, we
unit tests NetworkManager. Let's always remove such warnings with `sed`.
We already did that previously, but only for a explicit list of tests.
Now do it for all tests.
This is currently relevant on Fedora 35 and Ubuntu devel, where the
"close_range" syscall is used by libc, but not supported by valgrind.
While at it, rework the confusing logic of "HAS_ERRORS" variable.
Currently "src/" mostly contains the source code of the daemon.
I say mostly, because that is not true, there are also the device,
settings, wwan, ppp plugins, the initrd generator, the pppd and dhcp
helper, and probably more.
Also we have source code under libnm-core/, libnm/, clients/, and
shared/ directories. That is all confusing.
We should have one "src" directory, that contains subdirectories. Those
subdirectories should contain individual parts (libraries or
applications), that possibly have dependencies on other subdirectories.
There should be a flat hierarchy of directories under src/, which
contains individual modules.
As the name "src/" is already taken, that prevents any sensible
restructuring of the code.
As a first step, move "src/" to "src/core/". This gives space to
reorganize the code better by moving individual components into "src/".
For inspiration, look at systemd's "src/" directory.
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/743
When "tools/run-nm-test.sh" is called from the build scripts,
it has as first argument "--called-from-make". Then all arguments
must follow in a well defined order, which autotools/meson understand
and follow.
Another main use is however to call "tools/run-nm-test.sh" form the command
line. In that case, we want to have the command line parsing convenient.
Some of the parameters to the script are interpreted by the script, and
some are passed on to the test. The user can use "--" to separate the
parameters:
./tools/run-nm-test.sh -m shared/nm-glib-aux/tests/test-shared-general -- -p /general/test_strv_cmp
Otherwise, on the first unknown argument "tools/run-nm-test.sh" would
assume all following arguments are for the test. So this worked too:
./tools/run-nm-test.sh -m shared/nm-glib-aux/tests/test-shared-general -p /general/test_strv_cmp
However, if you now want to run the test with valgrind, you need to edit
the command line before the test arguments, like
./tools/run-nm-test.sh -m shared/nm-glib-aux/tests/test-shared-general -v -p /general/test_strv_cmp
That is inconvenient because I call the script from the shell history and
the cursor is at the end of the line. Instead, assume that all unknown parameters
are for the test (until "--" is encountered).
Now this works:
./tools/run-nm-test.sh -m shared/nm-glib-aux/tests/test-shared-general -p /general/test_strv_cmp -v
Arguably, now also
./tools/run-nm-test.sh -m shared/nm-glib-aux/tests/test-shared-general -p -v /general/test_strv_cmp
works, which is a bid odd.
On Fedora rawhide (34), valgrind gives a lot of warnings like:
./src/platform/tests/test-cleanup-linux.valgrind-log:--48279-- WARNING: unhandled amd64-linux syscall: 439
./src/platform/tests/test-cleanup-linux.valgrind-log:--48279-- You may be able to write your own handler.
./src/platform/tests/test-cleanup-linux.valgrind-log:--48279-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
./src/platform/tests/test-cleanup-linux.valgrind-log:--48279-- Nevertheless we consider this a bug. Please report
./src/platform/tests/test-cleanup-linux.valgrind-log:--48279-- it at http://valgrind.org/support/bug_reports.html.
Ignore them.
Otherwise, we get test failures with valgrind on fedora:rawhide
(valgrind-3.15.0-18.fc33.x86_64.rpm, gcc-10.0.1-0.8.fc33.x86_64,
glib2-devel-2.63.5-3.fc33.x86_64):
>>>> PRINT VALGRIND LOGS (valgrind test) (start)
+ find -name '*.valgrind-log' -print0
+ xargs -0 grep -H '^'
./src/devices/wwan/tests/test-service-providers.valgrind-log:--95634-- WARNING: unhandled amd64-linux syscall: 315
./src/devices/wwan/tests/test-service-providers.valgrind-log:--95634-- You may be able to write your own handler.
./src/devices/wwan/tests/test-service-providers.valgrind-log:--95634-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
./src/devices/wwan/tests/test-service-providers.valgrind-log:--95634-- Nevertheless we consider this a bug. Please report
./src/devices/wwan/tests/test-service-providers.valgrind-log:--95634-- it at http://valgrind.org/support/bug_reports.html.
./libnm/tests/test-remote-settings-client.valgrind-log:--95245-- WARNING: unhandled amd64-linux syscall: 315
./libnm/tests/test-remote-settings-client.valgrind-log:--95245-- You may be able to write your own handler.
./libnm/tests/test-remote-settings-client.valgrind-log:--95245-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
./libnm/tests/test-remote-settings-client.valgrind-log:--95245-- Nevertheless we consider this a bug. Please report
./libnm/tests/test-remote-settings-client.valgrind-log:--95245-- it at http://valgrind.org/support/bug_reports.html.
./libnm/tests/test-secret-agent.valgrind-log:--95280-- WARNING: unhandled amd64-linux syscall: 315
./libnm/tests/test-secret-agent.valgrind-log:--95280-- You may be able to write your own handler.
./libnm/tests/test-secret-agent.valgrind-log:--95280-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
./libnm/tests/test-secret-agent.valgrind-log:--95280-- Nevertheless we consider this a bug. Please report
./libnm/tests/test-secret-agent.valgrind-log:--95280-- it at http://valgrind.org/support/bug_reports.html.
./libnm/tests/test-nm-client.valgrind-log:--95208-- WARNING: unhandled amd64-linux syscall: 315
./libnm/tests/test-nm-client.valgrind-log:--95208-- You may be able to write your own handler.
./libnm/tests/test-nm-client.valgrind-log:--95208-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
./libnm/tests/test-nm-client.valgrind-log:--95208-- Nevertheless we consider this a bug. Please report
./libnm/tests/test-nm-client.valgrind-log:--95208-- it at http://valgrind.org/support/bug_reports.html.
+ echo '>>>> PRINT VALGRIND LOGS (valgrind test) (done)'
>>>> PRINT VALGRIND LOGS (valgrind test) (done)
Otherwise, the script tries to run
dbus-run-session -- exec ...
which fails (because `exec` is a shell command, not a program).
After the failure, the code falls through to run the test under
valgrind.
Fixes: 6a58c55ca4 ('run-nm-test: Just use exec instead of running and exiting')
This removes libnm-glib, libnm-glib-vpn, and libnm-util for good.
The it has been replaced with libnm since NetworkManager 1.0, disabled
by default since 1.12 and no up-to-date distributions ship it for years
now.
Removing the libraries allows us to:
* Remove the horrible hacks that were in place to deal with accidental use
of both the new and old library in a single process.
* Relief the translators of maintenance burden of similar yet different
strings.
* Get rid of known bad code without chances of ever getting fixed
(libnm-glib/nm-object.c and libnm-glib/nm-object-cache.c)
* Generally lower the footprint of the releases and our workspace
If there are some really really legacy users; they can just build
libnm-glib and friends from the NetworkManager-1.16 distribution. The
D-Bus API is stable and old libnm-glib will keep working forever.
https://github.com/NetworkManager/NetworkManager/pull/308
This removes libnm-glib, libnm-glib-vpn, and libnm-util for good.
The it has been replaced with libnm since NetworkManager 1.0, disabled
by default since 1.12 and no up-to-date distributions ship it for years
now.
Removing the libraries allows us to:
* Remove the horrible hacks that were in place to deal with accidental use
of both the new and old library in a single process.
* Relief the translators of maintenance burden of similar yet different
strings.
* Get rid of known bad code without chances of ever getting fixed
(libnm-glib/nm-object.c and libnm-glib/nm-object-cache.c)
* Generally lower the footprint of the releases and our workspace
If there are some really really legacy users; they can just build
libnm-glib and friends from the NetworkManager-1.16 distribution. The
D-Bus API is stable and old libnm-glib will keep working forever.
https://github.com/NetworkManager/NetworkManager/pull/308
Under valgrind, we cannot create an NAcd instance.
--10916-- WARNING: unhandled amd64-linux syscall: 321
--10916-- You may be able to write your own handler.
--10916-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
--10916-- Nevertheless we consider this a bug. Please report
--10916-- it at http://valgrind.org/support/bug_reports.html.
This limitation already poses a problem, because running NetworkManager
under valgrind might fail. However, for tests it doesn't matter and we
can just skip them.
With autotools, we use libtool so that the right libraries are
automatically found. Still, we won't find the right GI typelib.
Add a mechanism so that when make/meson invokes the run-nm-test.sh
runner, it passes the build-root directory.
Also, try to autodetect when invoked manually.
Previously, the test runner would only accept leading options,
an optional "--" separator, followed by "$TEST" and optional arguments
for the test.
Like
./tools/run-nm-test.sh -m src/settings/plugins/ifcfg-rh/tests/test-ifcfg-rh
That is annoying, because to toggle an option you have to seek the
curser in the before the test name.
Now, accept a "--test" option, so that the above can be done with trialing
arguments:
./tools/run-nm-test.sh -t src/settings/plugins/ifcfg-rh/tests/test-ifcfg-rh -m
However, the arguments for the tests still must come last:
./tools/run-nm-test.sh -m [--] src/settings/plugins/ifcfg-rh/tests/test-ifcfg-rh -p /settings/plugins/ifcfg-rh/bridge/write-master
./tools/run-nm-test.sh -t src/settings/plugins/ifcfg-rh/tests/test-ifcfg-rh -m [--] -p /settings/plugins/ifcfg-rh/bridge/write-master
./tools/run-nm-test.sh -m src/settings/plugins/ifcfg-rh/tests/test-ifcfg-rh
makes the test before running it. However, that failed if the
test didn't exist already. Reorder the code so that we always
try to make the test before trying to run it.
- also when called from makefile, allow enabling valgrind even if it was not
enabled via configure option. That is, even if you configured --without-valgrind,
you can run tests via `NMTST_USE_VALGRIND=1 make check`. Previously, there
was no way to run on valgrind during `make check` unless you also had
configured --with-valgrind.
- Use $NMTST_VALGRIND variable to override the valgrind path. This now
always takes precedence. For `make check`, the path can be determined
by the configure script.
If all unspecified, as last fallback "valgrind" is searched in the current
$PATH.
- Allow to specify the suppressions file via $NMTST_SUPPRESSIONS.
If unset, fall back to the default. The default during `make check`
is determined by the configure options. The default for manual
invocation is our one valgrind.suppressions file.
To use no suppressions file, set NMTST_SUPPRESSIONS to empty.
Now, regardless of what you enabled during ./configure, you can
overwrite it via:
$ NMTST_USE_VALGRIND=1 \
NMTST_VALGRIND=~/bin/valgrind \
NMTST_SUPPRESSIONS=my-suppressions \
make check
This adds 0.4 seconds to the build time.
You can disable it by setting $NM_BUILD_NO_CREATE_EXPORTS environment
variable. This is useful in the unexpected case that the script
is broken.
Or, if you just want to use a different, non-generated version-script.
Or, if you want to save 0.4 seconds build-time.
With --make-first|-m we first call `make` on the test.
However, the make path must be a relative path rooted
in the top directory.
Make sure we `cd` into the parent directory first and pass
the proper make path.
cd src
../tools/run-nm-test.sh -m settings/plugins/ifcfg-rh/tests/test-ifcfg-rh
No need to have two test-runners. Combine them, and call tests always
via "tools/run-nm-test.sh".
Yes, this brings an overhead, that we now always invoke the test with
a test wrapper script, also --without-vagrind. Previously, that was only
necessary for libnm tests that require their own D-Bus session.
Later we will do non-recursive Makefiles, thus all tests should have the
same LOG_COMPILER.
2016-10-19 15:26:30 +02:00
Renamed from tools/run-test-valgrind.sh (Browse further)
