1692 commits

Author	SHA1	Message	Date
Peter Hutterer	a80678c241	test: add --filter-rangeval to only run tests with a range value ... For ranged tests (e.g. gesture tests) it's convenient to be able to run those with a value of N (e.g. 3 fingers) instead ov having to run all of them all the time. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	3c2e92d169	test: add ability to collect test results from a nonforking test ... Use longjmp to try to get out of a test error. This will only work for errors triggered by our own test suite (i.e. not for SIGSEGV and friends) but that's good enough for most things. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	c1f744a6e5	test: move the quirks setup into the global setup func ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	b21fd3cfe7	test: add runner support for a global setup/teardown func ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	1fd3cefc60	test: the created files list doesn't need to be a global ... This list was used in the signal handler but that is no longer the case, so let's localize this. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	c68c0f593b	test: move the rest of the custom initialization to litest_run ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	d4c6c47677	test: move tty disabling closer to the actual running of the tests ... And move the litest-specific condition out into the caller so disable_tty() only cares about the TTY, not any surrounding conditions. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	b81cc0ceb7	test: only force gdb to no-fork, valgrind can fork ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	16e5bdabac	test: intercept and collect valgrind errors ... This works because we always invoke with --exit-errorcode=3 from the test suite. It won't work for manual invocations but oh well. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	27fa85a884	test: move litest_free_test_list to inside ifndef NO_MAIN ... This is now only called from one spot, so let's move it so it is ifdef'd out. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	0cfde0f95f	test: make the utils test a "pure" litest-runner test ... Now that we have the runner let's use it directly for this set of simple test. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	d84efcb4b6	test: add --exit-first to the litest test suite runner ... Exits upon the first failed test. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	7b69fb2d63	test: remove now-unused check-double-macros.h ... We no longer use those functions, so let's drop them. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	9928de7fa3	test: move the quirks context init into litest_run ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	41c08f0816	test: add litest-runner as test suite runner ... This replaces check. The code is a copy of pwtest which I wrote years ago for pipewire but adjusted for us here the last few days. There are a few advantages over check: - Ability to SKIP tests or mark them as NOT_APPLICABLE, the latter of which is used for early checks if a device doesn't meet requirements. - it captures stdout/stderr separately - colors! - YAML output format makes it a lot easier to read the results and eventually parse them for e.g. "restart failed tests" Less abstraction: we set up the tests, pass them to the runner and run them with the given number of forks. This is an improvement over before where we forked into N test suites which each called check which then forked again. Since we're now keeping track of those processes ourselves we can also write tests that are expected to fail with signals. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	d94407f810	test: fix utils test for types for litest assertions ... check always typecasts to intmax_t so a lot of type mismatches were hidden. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	5cbb37a3d2	test: convert utils test to litest_assert ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	6cac0d4d2f	test: make the utils test another litest ... This doesn't really gain us anything for now but it's prep work for dropping check and the custom invocations of check. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	4fa5ea268e	util: add a generic stringbuffer with a few helper functions ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	817dc42381	util: add the range handling as separate header file ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	d1575fbce5	test: move the deviceless check into ltest_add_tcase ... This is out of place here, check this when we are about to add the test, not later when the tests are about to be run. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	efd3dd4427	test: filter out duplicate name/device combos in --list ... This doesn't have an effect yet because ranged tests are handled inside check. But in the future these tests will be split up individually so de-duplicating is useful here. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	e3b5c0da5f	test: make litest_backtrace() available from outside litest.c ... Prep-work for adding a new runner Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	b67ab51f76	test: append, not insert the suites ... list_append() came later than list_insert() and there's an argument to be made that tests added later should be run first since they're less likely to succeed. But it's a lot harder to read test logs when they are in reverse order, and with the TEST_COLLECTION() macro the order of the test suite is not obvious anyway. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1067>	2024-10-30 23:20:42 +00:00
Peter Hutterer	e0f671126d	util: add the backtrace printing function as separate util ... Might as well make this easier to re-use since it doesn't do anything specific to litest. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1066>	2024-10-23 10:08:12 +10:00
Peter Hutterer	b34010bf4e	test: split the touchpad tests into a palm and a dwt test collection ... dwt and palm tests have a lot of timeouts to wait for so let's split those out. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1065>	2024-10-18 10:49:47 +10:00
Peter Hutterer	9e00d09b30	test: split the tablet left-handed tests out into a separate collection ... These aren't complicated but there's a lot of them so let's run them separately to make the overall tablet test shorter. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1065>	2024-10-18 10:49:47 +10:00
Peter Hutterer	a67dbefc0f	test: split out the touchpad tap tests into multiple collections ... These take a long time and have a reasonable high chance of failure due to the timing constraints. Let's split them up so they don't hog the runners for that long and in case they fail, we only need to re-run a short test. Before: one test running approx 21 min, now 3 tests running approx 7 + 11 + 4 min. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1065>	2024-10-18 10:49:47 +10:00
Peter Hutterer	3390f2e647	test: make the TEST_COLLECTION() macro re-usable in the same file ... Concat the line number to the generated variable names, this way we can have more than one TEST_COLLECTION() in the same file. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1065>	2024-10-18 10:49:47 +10:00
Peter Hutterer	0135b0b41c	test: detach the suite handling from the file names ... Instead of extracting the suite name from the test's file name use the current suite that is being parsed. This way we pave the way for multiple suites in the same file. This uses a global because otherwise we'd have to redo all the litest_add() functions but it does the job here. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1065>	2024-10-18 10:49:47 +10:00
Peter Hutterer	dba296d290	test: match the tablet's pad test collection with the file name ... This is currently a requirement, so let's match this before we run into issues here re-structuring stuff. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1065>	2024-10-18 10:49:47 +10:00
Peter Hutterer	fb4c5dd7c6	test: use a litest helper instead of return code comparison ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>	2024-10-16 18:03:04 +10:00
Peter Hutterer	d010c6a170	test: rename a variable to indicate suites, not tests ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>	2024-10-16 18:03:04 +10:00
Peter Hutterer	cd957b4e79	test: rename all_tests to all_test_suites ... This is a a list of struct suite with the various tests inside that suite. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>	2024-10-16 18:03:04 +10:00
Peter Hutterer	cb0575b473	test: check for verbosity immediately in main() ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>	2024-10-16 18:03:04 +10:00
Peter Hutterer	75ce537342	test: check for empty tests immediately, not later when we're running ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>	2024-10-16 18:03:03 +10:00
Peter Hutterer	6d26d83f00	test: init the global devices/test lists immediately ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>	2024-10-16 18:03:03 +10:00
Peter Hutterer	50f6e4f195	test: add restore_tty helper and use it ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>	2024-10-16 18:03:03 +10:00
Peter Hutterer	1a1a8f5378	test: init the device list via argument ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>	2024-10-16 18:03:03 +10:00
Peter Hutterer	f7cc8e05f7	test: add a helper for deleting a created file ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>	2024-10-16 18:03:03 +10:00
Peter Hutterer	57a3313d33	test: switch a bunch of tests from abort() to one with a message ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>	2024-10-16 18:03:03 +10:00
Peter Hutterer	ecb2664e05	test: fix litest_abort_msg() not starting a new line ... Most callers of litest_abort_msg() don't add '\n' so the output was mangled. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1062>	2024-10-16 18:03:03 +10:00
Peter Hutterer	bd62da11e4	test: switch the tests to use enum comparisons ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>	2024-10-15 12:44:27 +10:00
Peter Hutterer	151c2feae6	test: add macros to compare enum values ... This requires switching a lot of int_eq/int_ne over to enum_eq/enum_ne because the compiler doesn't infer the right type from a harcoded enum value - it just defaults to int. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>	2024-10-15 12:44:27 +10:00
Peter Hutterer	5f4f72dca4	test: switch a bunch of integer comparisons to right sizes ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>	2024-10-15 12:44:27 +10:00
Peter Hutterer	f89fd6d924	test: switch ck_assert over to litest_assert ... This is the first step in switching away from the check framework. Our litest macros already do almost exactly the same anyway so most of this is a simple sed with a few compiler fixes where things mismatch (nonnull -> notnull) and (_tol -> _epsilon). This now generates a whole bunch of integer mismatch warnings: check casts everything to intmax_t whereas we use typeof, so lots of warnings especially for enums. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>	2024-10-15 12:44:27 +10:00
Peter Hutterer	0bc2bf60d2	test: add litest string comparison macros ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>	2024-10-15 12:44:27 +10:00
Peter Hutterer	9a5f888795	test: replace manual "almost equal" with the new double epsilon ... Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>	2024-10-15 12:44:27 +10:00
Peter Hutterer	237b15b401	test: fix some checks using ints to compare doubles ... Theoretically we should be using ck_assert_double_eq here for consistency but this patch is part of a series eventually replacing those calls, so let's jump to litest_assert_double directly to avoid further rebase conflicts. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>	2024-10-15 12:44:27 +10:00
Peter Hutterer	825e521d9b	test: fix a touch test comparing ints to doubles ... We were checking doubles for integers but better to check that we're close to the maximum range without actually being over. This worked because check typecasts to uint_max_t but let's be explicit here. Part-of: <https://gitlab.freedesktop.org/libinput/libinput/-/merge_requests/1059>	2024-10-15 12:44:27 +10:00