libinput/doc/test-suite.dox
Peter Hutterer eb8e1d4a13 test: add a make-like job control to run tests in parallel
Add a make-like -j/--jobs option to split the number of parallel test
processes. Defaults to 8 if not specified, future patches will default this to
1 for special cases where filters are specified or gdb is detected.

Each subprocess overwrites argv[0] to be easier identifiable in the ps
output when we're trying to figure out which tests are still running.

A -j1 is equivalent to the previous functionality, i.e. we don't fork.

One quirk needed for check: any test case not part of a test runner will not
be freed and thus triggers valgrind. We do test filtering by splitting
up the tests across multiple forks (i.e. each process has several tests that
are in the list but not added to the runner). Thus we need to mark those we
expect check to free as used.
Then on cleanup we traverse the test list, add all the unused one to a
test runner and free that test runner (without actually running it). This
cleans up both the filtered tests in each subprocess and the whole test list
in the parent process which doesn't run a test itself.

Signed-off-by: Peter Hutterer <peter.hutterer@who-t.net>
2016-08-02 11:26:58 +10:00

98 lines
3.4 KiB
Text

/**
@page test-suite libinput test suite
The libinput test suite is based on
[Check](http://check.sourceforge.net/doc/check_html/) and runs automatically
during `make check`. Check itself is wrapped into a libinput-specific test
suite called *litest*. Tests are found in `$srcdir/test/`, the main test
suite is `libinput-test-suite-runner`.
The test suite has a make-like job control enabled by the `-j` or `--jobs`
flag and will fork off as many parallel processes as given by this flag. The
default if unspecified is 8. When debugging a specific test case failure it
is recommended to employ test filtures (see @ref test-filtering) and disable
parallel tests.
@section test-config X.Org config to avoid interference
uinput devices created by the test suite are usually recognised by X as
input devices. All events sent through these devices will generate X events
and interfere with your desktop.
Copy the file `$srcdir/test/50-litest.conf` into your `/etc/X11/xorg.conf.d`
and restart X. This will ignore any litest devices and thus not interfere
with your desktop.
@section test-root Permissions required to run tests
Most tests require the creation of uinput devices and access to the
resulting `/dev/input/eventX` nodes. Some tests require temporary udev rules.
<b>This usually requires the tests to be run as root</b>.
@section test-filtering Selective running of tests
litest's tests are grouped by test groups and devices. A test group is e.g.
"touchpad:tap" and incorporates all tapping-related tests for touchpads.
Each test function is (usually) run with one or more specific devices.
The `--list` commandline argument shows the list of suites and tests.
@code
$ ./test/libinput-test-suite-runner --list
device:wheel:
wheel only
blackwidow
device:invalid devices:
no device
device:group:
no device
logitech trackball
MS surface cover
mouse_roccat
wheel only
blackwidow
...
@endcode
In the above example, the "device:wheel" suite is run for the "wheel only" and
the "blackwidow" device. Both devices are automatically instantiated through
uinput by litest. The "no device" entry signals that litest does not
instantiate a uinput device for a specific test (though the test itself may
instantiate one).
The `--filter-test` argument enables selective running of tests through
basic shell-style function name matching. For example:
@code
$ ./test/libinput-test-suite-runner --filter-test="*1fg_tap*"
@endcode
The `--filter-device` argument enables selective running of tests through
basic shell-style device name matching. The device names matched are the
litest-specific shortnames, see the output of `--list`. For example:
@code
$ ./test/libinput-test-suite-runner --filter-device="synaptics*"
@endcode
The `--filter-group` argument enables selective running of test groups
through basic shell-style test group matching. The test groups matched are
litest-specific test groups, see the output of `--list`. For example:
@code
$ ./test/libinput-test-suite-runner --filter-group="touchpad:*hover*"
@endcode
The `--filter-device` and `--filter-group` arguments can be combined with
`--list` to show which groups and devices will be affected.
@section test-verbosity Controlling test output
Each test supports the `--verbose` commandline option to enable debugging
output, see libinput_log_set_priority() for details. The `LITEST_VERBOSE`
environment variable, if set, also enables verbose mode.
@code
$ ./test/libinput-test-suite-runner --verbose
$ LITEST_VERBOSE=1 make check
@endcode
*/