Commit graph

601 commits

Author SHA1 Message Date
Peter Hutterer
fefeab4efe test: a boolean parameter is always true/false
Don't require the caller to provide the values, they're always the
same.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1123>
2025-01-20 12:08:46 +00:00
Peter Hutterer
86b9142a02 test: highlight the interesting part of the backtrace
This makes it easier to immediately see where things are breaking since
the full backtrace has >70% noise.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1124>
2025-01-20 11:30:01 +00:00
Peter Hutterer
6759a2f754 test: honor NO_COLOR and FORCE_COLOR for output colorization
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1124>
2025-01-20 11:30:01 +00:00
Peter Hutterer
2c0c4a6516 Replace strneq() with hardcoded lengths with strstartswith()
Slightly less efficient but easier to read and it's not possible to
accidentally provide the wrong length. Plus it handles null pointers
correctly so get to skip the checks (which weren't needed for strneq()
either, but still).

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1121>
2025-01-20 10:29:47 +00:00
Peter Hutterer
d9de017d6c test: fix --filter-params handling
The previous implementation skipped parameters that were filtered, so
our test cases got called with parameters missing. Fix this by filtering
any test case that has a negative fnmatch on any parameter.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1120>
2025-01-08 10:18:31 +00:00
Peter Hutterer
1e445f3f84 test: implement support for parametrizing tests
litest supports ranged tests but they are not enough, doubly so with
tests where we want to parametrize across multiple options.

This patch adds support for just that, in clunky C style.
The typical invocation for a test is by giving the test parameter
a name, a number of values and then the values themselves:

	struct litest_parameters *params = litest_parameters_new("axis", 's', 2, "ABS_X", "ABS_Y",
	                                                         "enabled", 'b', '2', true, false,
	                                                         "number", 'u', '2', 10, 11,
	                                                         NULL);
	litest_add_parametrized(sometest, LITEST_ANY, LITEST_ANY, params);
	litest_parameters_unref(params);

Currently supported are u (uint32), i (int32), d (double), b (bool),
c (char) and s (string).

In the test itself, the `test_env->params` variable is available and
retrieval of the parameters works like this:

	const char *axis;
	uint32_t number;
	bool enabled;
	litest_test_param_fetch(test_env->params,
	                        "axis", &axis,
	                        "enabled", &enabled,
	                        "number", &number,
	                        NULL);

Note that since this is an effectively internal test-suite only
functionality we don't do type-checking here, it's assumed that if you
write the code to pass parameters into a test you remember the type
of said params when you write the test code.

Because we don't have hashmaps or anything useful other than lists the
implementation is a bit clunky: we copy the parameter into the test
during litest_add_*, permutate it for our test list which gives us yet
another linked list C struct, and finally copy the actual value into
the test and test environment as it's executed. Not pretty, but it
works.

A few tests are switched as simple demonstration. The name of the
test has the parameters with their names and values appended now, e.g.:
   "pointer:pointer_scroll_wheel_hires_send_only_lores:ms-surface-cover:axis:ABS_X"
   "pointer:pointer_motion_relative_min_decel:mouse-roccat:direction:NW"

Filtering by parameters can be done via globs of their string
representation:
   libinput-test-suite --filter-params="axis:ABS_*,enabled:true,number:10*"

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1109>
2025-01-07 02:02:37 +00:00
Peter Hutterer
7efec8f2e7 test: move a switch case up for better grouping
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1109>
2025-01-07 02:02:37 +00:00
Peter Hutterer
5e5799a319 test: log litest_checkpoint to stderr
Otherwise the logs are detached from libinput's logs (which are printed
to stderr) which makes the checkpoint function mostly useless since it
doesn't actually group the messages as expected.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1116>
2025-01-07 11:40:00 +10:00
Peter Hutterer
266ce4218b test: fix strip values being passed to ring functions
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1070>
2024-10-31 10:28:08 +10:00
Peter Hutterer
4a98e273a4 test: add --output-file to print test logs to a file
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
1f8a4df58c test: drop the --xml-prefix option
litest-runner outputs in YAML and no-one ever looks at the XML files
anyway. They are used to provide nice pipeline summaries but again,
no-one looks at those because they're not very useful.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
a80678c241 test: add --filter-rangeval to only run tests with a range value
For ranged tests (e.g. gesture tests) it's convenient to be able to run
those with a value of N (e.g. 3 fingers) instead ov having to run all of
them all the time.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
3c2e92d169 test: add ability to collect test results from a nonforking test
Use longjmp to try to get out of a test error. This will only work for
errors triggered by our own test suite (i.e. not for SIGSEGV and
friends) but that's good enough for most things.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
c1f744a6e5 test: move the quirks setup into the global setup func
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
1fd3cefc60 test: the created files list doesn't need to be a global
This list was used in the signal handler but that is no longer the case,
so let's localize this.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
c68c0f593b test: move the rest of the custom initialization to litest_run
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
d4c6c47677 test: move tty disabling closer to the actual running of the tests
And move the litest-specific condition out into the caller so
disable_tty() only cares about the TTY, not any surrounding conditions.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
b81cc0ceb7 test: only force gdb to no-fork, valgrind can fork
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
27fa85a884 test: move litest_free_test_list to inside ifndef NO_MAIN
This is now only called from one spot, so let's move it so it is
ifdef'd out.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
d84efcb4b6 test: add --exit-first to the litest test suite runner
Exits upon the first failed test.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
9928de7fa3 test: move the quirks context init into litest_run
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
41c08f0816 test: add litest-runner as test suite runner
This replaces check. The code is a copy of pwtest which I wrote years
ago for pipewire but adjusted for us here the last few days.

There are a few advantages over check:
- Ability to SKIP tests or mark them as NOT_APPLICABLE, the latter
  of which is used for early checks if a device doesn't meet
  requirements.
- it captures stdout/stderr separately
- colors!
- YAML output format makes it a lot easier to read the results and
  eventually parse them for e.g. "restart failed tests"

Less abstraction: we set up the tests, pass them to the runner and run
them with the given number of forks. This is an improvement over before
where we forked into N test suites which each called check which then
forked again. Since we're now keeping track of those processes
ourselves we can also write tests that are expected to fail with
signals.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
d1575fbce5 test: move the deviceless check into ltest_add_tcase
This is out of place here, check this when we are about
to add the test, not later when the tests are about to be run.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
efd3dd4427 test: filter out duplicate name/device combos in --list
This doesn't have an effect yet because ranged tests are handled inside
check. But in the future these tests will be split up individually so
de-duplicating is useful here.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
e3b5c0da5f test: make litest_backtrace() available from outside litest.c
Prep-work for adding a new runner

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
b67ab51f76 test: append, not insert the suites
list_append() came later than list_insert() and there's an argument to
be made that tests added later should be run first since they're less
likely to succeed. But it's a lot harder to read test logs when they are
in reverse order, and with the TEST_COLLECTION() macro the order of the
test suite is not obvious anyway.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>
2024-10-30 23:20:42 +00:00
Peter Hutterer
e0f671126d util: add the backtrace printing function as separate util
Might as well make this easier to re-use since it doesn't do anything
specific to litest.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1066>
2024-10-23 10:08:12 +10:00
Peter Hutterer
0135b0b41c test: detach the suite handling from the file names
Instead of extracting the suite name from the test's file name use the
current suite that is being parsed. This way we pave the way for
multiple suites in the same file.

This uses a global because otherwise we'd have to redo all the
litest_add() functions but it does the job here.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1065>
2024-10-18 10:49:47 +10:00
Peter Hutterer
fb4c5dd7c6 test: use a litest helper instead of return code comparison
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>
2024-10-16 18:03:04 +10:00
Peter Hutterer
d010c6a170 test: rename a variable to indicate suites, not tests
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>
2024-10-16 18:03:04 +10:00
Peter Hutterer
cd957b4e79 test: rename all_tests to all_test_suites
This is a a list of struct suite with the various tests inside that
suite.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>
2024-10-16 18:03:04 +10:00
Peter Hutterer
cb0575b473 test: check for verbosity immediately in main()
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>
2024-10-16 18:03:04 +10:00
Peter Hutterer
75ce537342 test: check for empty tests immediately, not later when we're running
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>
2024-10-16 18:03:03 +10:00
Peter Hutterer
6d26d83f00 test: init the global devices/test lists immediately
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>
2024-10-16 18:03:03 +10:00
Peter Hutterer
50f6e4f195 test: add restore_tty helper and use it
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>
2024-10-16 18:03:03 +10:00
Peter Hutterer
1a1a8f5378 test: init the device list via argument
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>
2024-10-16 18:03:03 +10:00
Peter Hutterer
f7cc8e05f7 test: add a helper for deleting a created file
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>
2024-10-16 18:03:03 +10:00
Peter Hutterer
ecb2664e05 test: fix litest_abort_msg() not starting a new line
Most callers of litest_abort_msg() don't add '\n' so the output was
mangled.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>
2024-10-16 18:03:03 +10:00
Peter Hutterer
f89fd6d924 test: switch ck_assert over to litest_assert
This is the first step in switching away from the check framework.
Our litest macros already do almost exactly the same anyway so most of
this is a simple sed with a few compiler fixes where things mismatch
(nonnull -> notnull) and (_tol -> _epsilon).

This now generates a whole bunch of integer mismatch warnings: check
casts everything to intmax_t whereas we use typeof, so lots of warnings
especially for enums.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>
2024-10-15 12:44:27 +10:00
Peter Hutterer
0bc2bf60d2 test: add litest string comparison macros
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>
2024-10-15 12:44:27 +10:00
Peter Hutterer
3ed70e864f test: add litest_assert_event_type_not_one_of
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>
2024-10-15 12:44:27 +10:00
Peter Hutterer
67061c1af0 test: replace ck_abort with litest_abort_msg()
Weirdly, that also required initializing two variables to NULL to stop a
compiler warning.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>
2024-10-15 12:44:27 +10:00
Peter Hutterer
7316a627d6 test: add a missing va_end
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>
2024-10-15 12:44:27 +10:00
Peter Hutterer
aecfcf3d1b test: fix uinput creation for the slotted devices with too many slots
Kernel commit 206f533a0a7c
       "Input: uinput - reject requests with unreasonable number of slots"
limits the number of slots to 99 - let's manually adjust that so we can
keep creating uinput devices.

Since these are just a test device and we don't use the slots here
anyway (they're all fake MT devices) we can manually work around this.
The real devices won't be affected by this since this is a limitation
in uinput, not the input subsystem.

Also move the comment one line up in the ms-surface device, the previous
comment referred to the wrong event code.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1061>
2024-10-15 09:56:19 +10:00
Peter Hutterer
c123a76b0d test: make litest-test-suite --list YAML-compatible
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1060>
2024-10-14 08:56:07 +00:00
Peter Hutterer
5b5776b148 test: move the test devices into the test_device_section
Better naming this way

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1060>
2024-10-14 08:56:07 +00:00
Peter Hutterer
3577369805 test: switch unused argc/argv to the list of tests
This means we don't rely on the "all_tests" global here though it also
means we need to move the cleanup into the caller.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1060>
2024-10-14 08:56:07 +00:00
Peter Hutterer
86c47be816 test: add litest_dispatch() for better test debugging
Wraps libinput_dispatch() with a location which will make things a bit
easier to track. Output (in --verbose) is something like:

  gestures_swipe_3fg_unaccel_fn():1346 - dispatching

Which makes it easier to associate the various calls to libinput
dispatch with the other output from libinput.

This patch switches all uses of libinput_dispatch() in test cases over
but not the litest functions that may call dispatch too. Remains to be
seen if that is necessary.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1048>
2024-09-19 15:21:04 +10:00
Peter Hutterer
27e3897420 test: abort on libinput bugs again
This was always intended but a bug prevented the actual abort.
strstr returns NULL when we cannot find the substring so we always
triggered the first noop condition on bugs.

Fixes: bd7b91065b ("evdev: warn if our event processing lags by 10ms or more")
Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1048>
2024-09-19 15:07:50 +10:00
Peter Hutterer
596d86ecc1 test: improve event debugging a bit
Add a litest_checkpoint macro and convert a few of the litest_assert
macros to make use of that - this gives us a printf of the call site in
case it fails.

Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1048>
2024-09-19 15:07:44 +10:00