test: run pytest with xdist where available

A lot of the protocol tests have enforced timeouts because we need to
wait for the server to do something (or not). Running the protocol tests
with xdist means we can run those in parallel, which on my local box
roughly halves the time required to run all protocol tests.

Of course, this is all terrible. The only way to tell pytest to use
xdist is with `pytest -n <jobcount>`. But -n is only available if xdist
is installed. Dynamically adding commandline options is supposed to
be possible with `pytest_load_initial_conftests` in conftest.py, but
that doesn't actually work unless we're configuring a setuptool plugin.
Which we don't. What would work is `pytest_cmdline_preparse` but that
has been deprecated.

So we work around this by having meson check if xdist is available and
append `-n auto` for us.

In the CI let's be nice and only use the FDI_CI_CONCURRENT number of
jobs rather than hogging everything available. And, continuation from
"all is terrible": pytest complains if you have a hook that's unknown.
So if xdist is not available, you must not have the hook in conftest.py.
Which means we have to do it within an import + ImportError clause.

But yay, now we're saving literally seconds!
This commit is contained in:
Peter Hutterer 2023-03-02 11:09:53 +10:00
parent bbc64b2eff
commit 9700a47473
2 changed files with 21 additions and 3 deletions

11
test/conftest.py Normal file
View file

@ -0,0 +1,11 @@
import os
try:
import xdist # noqa: F401
# Otherwise we get unknown hook 'pytest_xdist_auto_num_workers'
def pytest_xdist_auto_num_workers(config):
return os.getenv("FDO_CI_CONCURRENT", None)
except ImportError:
pass

View file

@ -44,6 +44,13 @@ if build_oeffis
endif
pymod.find_installation('python3', modules: required_python_modules)
pytest = find_program('pytest-3', 'pytest')
pytest_args = ['--verbose', '--log-level=DEBUG']
optional_python_modules = ['xdist']
if pymod.find_installation('python3', modules: optional_python_modules, required: false).found()
pytest_args += ['-n', 'auto']
configure_file(input: 'conftest.py', output: '@PLAINNAME@', copy: true)
endif
if build_oeffis
test('unit-tests-oeffis',
@ -57,7 +64,7 @@ if build_oeffis
env = environment()
env.set('LD_LIBRARY_PATH', meson.project_build_root())
test('oeffis-pytest', pytest,
args: ['--verbose', '--log-level=DEBUG'],
args: pytest_args,
suite: 'python',
workdir: meson.current_source_dir(),
env: env,
@ -132,7 +139,7 @@ configure_file(input: 'test_protocol.py',
configuration: protocol_test_config)
test('protocol-test', pytest,
args: ['--verbose', '--log-level=DEBUG'],
args: pytest_args,
suite: 'python',
workdir: meson.project_build_root(),
)
@ -140,7 +147,7 @@ if valgrind.found()
env = environment()
env.set('LIBEI_USE_VALGRIND', '1')
test('protocol-test-valgrind', pytest,
args: ['--verbose', '--log-level=DEBUG'],
args: pytest_args,
suite: 'python',
workdir: meson.project_build_root(),
env: env