litest supports ranged tests but they are not enough, doubly so with
tests where we want to parametrize across multiple options.
This patch adds support for just that, in clunky C style.
The typical invocation for a test is by giving the test parameter
a name, a number of values and then the values themselves:
struct litest_parameters *params = litest_parameters_new("axis", 's', 2, "ABS_X", "ABS_Y",
"enabled", 'b', '2', true, false,
"number", 'u', '2', 10, 11,
NULL);
litest_add_parametrized(sometest, LITEST_ANY, LITEST_ANY, params);
litest_parameters_unref(params);
Currently supported are u (uint32), i (int32), d (double), b (bool),
c (char) and s (string).
In the test itself, the `test_env->params` variable is available and
retrieval of the parameters works like this:
const char *axis;
uint32_t number;
bool enabled;
litest_test_param_fetch(test_env->params,
"axis", &axis,
"enabled", &enabled,
"number", &number,
NULL);
Note that since this is an effectively internal test-suite only
functionality we don't do type-checking here, it's assumed that if you
write the code to pass parameters into a test you remember the type
of said params when you write the test code.
Because we don't have hashmaps or anything useful other than lists the
implementation is a bit clunky: we copy the parameter into the test
during litest_add_*, permutate it for our test list which gives us yet
another linked list C struct, and finally copy the actual value into
the test and test environment as it's executed. Not pretty, but it
works.
A few tests are switched as simple demonstration. The name of the
test has the parameters with their names and values appended now, e.g.:
"pointer:pointer_scroll_wheel_hires_send_only_lores:ms-surface-cover:axis:ABS_X"
"pointer:pointer_motion_relative_min_decel:mouse-roccat:direction:NW"
Filtering by parameters can be done via globs of their string
representation:
libinput-test-suite --filter-params="axis:ABS_*,enabled:true,number:10*"
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1109>
This replaces check. The code is a copy of pwtest which I wrote years
ago for pipewire but adjusted for us here the last few days.
There are a few advantages over check:
- Ability to SKIP tests or mark them as NOT_APPLICABLE, the latter
of which is used for early checks if a device doesn't meet
requirements.
- it captures stdout/stderr separately
- colors!
- YAML output format makes it a lot easier to read the results and
eventually parse them for e.g. "restart failed tests"
Less abstraction: we set up the tests, pass them to the runner and run
them with the given number of forks. This is an improvement over before
where we forked into N test suites which each called check which then
forked again. Since we're now keeping track of those processes
ourselves we can also write tests that are expected to fail with
signals.
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
This is the first step in switching away from the check framework.
Our litest macros already do almost exactly the same anyway so most of
this is a simple sed with a few compiler fixes where things mismatch
(nonnull -> notnull) and (_tol -> _epsilon).
This now generates a whole bunch of integer mismatch warnings: check
casts everything to intmax_t whereas we use typeof, so lots of warnings
especially for enums.
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>
All minor:
- execdir does not need initialization, it's not used until written to
- 'newest' could be NULL
- zalloc(-1) confuses coverity
- 't' is never used in that test
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
This isn't technically needed since those tests aren't in the valgrind test
suite anymore. But let's have it here anyway.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
This header is intended to be included in the project, so let's do that and
have proper runtime detection of the valgrind environment.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
The ssize_t cast upsets coverity for some reason but we can be a lot more
restrictive here anyway. Quick analysis of the zalloc calls in the test suite
show the largest allocation is 9204 bytes.
Let's put a cap on for one MB, anything above that is likely some memory
corruption and should be caught early.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Reviewed-by: Matheus Santana <embs@cin.ufpe.br>
Nothing in libinput needs large buffers, so if we ever get something that
large, we probably passed a negative number to zalloc.
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
litest-selftest.c: In function ‘litest_ptr_eq_notrigger’:
litest-selftest.c:172:10: warning: initialization makes integer from pointer
without a cast [-Wint-conversion]
int c = NULL;
^
litest-selftest.c:173:10: warning: initialization makes integer from pointer
without a cast [-Wint-conversion]
int d = NULL;
^
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
Adds the macros ck_assert_double_{eq,ne,lt,gt,le,ge} to compare double
values using a fixed tolerance value. The tolerance value is
picked based on the range of values to be expected by the libinput API.
Signed-off-by: Andreas Pokorny <andreas.pokorny@canonical.com>
Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net>
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
The check unit test framework isn't particularly suited to having ck_assert*
calls in helper functions. A failed assertion in a helper function or the
litest framework merely gives us a the failed line in litest.c.
which doesn't tell us which test actually failed.
Add a backtracing facility with litest_backtrace(). And since this requires
wrapping all ck_assert macros with litest_assert() this patch ended up
replacing/duplicating a bunch of ck_assert_* bits. So rather than
ck_assert_int_eq() we now use litest_assert_int_eq(), etc. in the litest
framework itself.
The int comparison macros are more type-safe than ck_assert()'s macros which
just cast all the ints to intmax_t.
Backtrace is spewed to stderr, which is good enough for debugging. Example
backtrace:
FAILED COMPARISON: status != expected
Resolved to: 0 != 0
in disable_button_scrolling() (pointer.c:115)
Backtrace:
0: ./test/test-pointer (litest_fail_comparison_int+0xab) [0x40973b]
1: ./test/test-pointer (disable_button_scrolling+0x174) [0x40421b]
2: ./test/test-pointer (middlebutton_middleclick+0x40) [0x40829c]
3: /lib64/libcheck.so.0 (srunner_run+0x7f5) [0x7f0e8a277025]
4: ./test/test-pointer (litest_run+0x107) [0x40a42b]
5: ./test/test-pointer (main+0x2fa) [0x4090e7]
6: /lib64/libc.so.6 (__libc_start_main+0xf0) [0x7f0e88f5e790]
7: ./test/test-pointer (_start+0x29) [0x403ce9]
8: ? (?+0x29) [0x29]
litest_backtrace() itself is copied from xserver/os/backtrace.c which git
blame attributes to Marcin.
CC: Marcin Slusarz <marcin.slusarz@gmail.com>
Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
