mirror of
https://gitlab.freedesktop.org/cairo/cairo.git
synced 2025-12-24 20:40:10 +01:00
973d3a3d14
- Remove cairo_test_expect_failure. cairo-test.c now checks env var CAIRO_XFAIL_TESTS to see if the running test is expected to fail. The reason for expected failure is appended to the test description. - Test description is written out. - Failed/crashed tests also write a line out to stderr (in red), so one can now redirect stdout to /dev/null to only see failures. - cairo_test() has been changed to not take the draw function anymore, instead, draw function is now part of the test struct. - "make check" doesn't allow limiting backends to test using env var anymore. To limit backends to test, one should use the TARGETS variable on the make command line. - "make check-valgrind" now writes its log to valgrind-log instead of valgrind.log, to not interfere with test log file processing.
88 lines
2.9 KiB
Text
88 lines
2.9 KiB
Text
Regression test suite for cairo.
|
|
|
|
Using this test should be as simple as running:
|
|
|
|
make check
|
|
|
|
assuming that the cairo distribution in the directory above has been
|
|
configured and built. The test suite here goes through some effort to
|
|
run against the locally compiled library rather than any installed
|
|
version, but those efforts may fall short depending the level of your
|
|
libtool madness.
|
|
|
|
The test suite needs to be run before any code is committed and before
|
|
any release. Here are the rules governing the use of the suite:
|
|
|
|
|
|
Tailoring tests running
|
|
-----------------------
|
|
|
|
There are some mechanisms to limit the tests run during "make check".
|
|
These come very handy when doing development, but should not be used
|
|
to circumvent the "pass" requirements listed below.
|
|
|
|
To limit the backends that the tests are run against, use the
|
|
TARGETS make variable, that can also be passed to make.
|
|
It should contain a (space-, comma-, etc-separate) list of backends to test.
|
|
To limit the tests run, use the TESTS make variable, which should be a
|
|
space-separated list of tests to run. For example:
|
|
|
|
make check TARGETS=image,ps TESTS="zero-alpha"
|
|
|
|
|
|
Before committing
|
|
-----------------
|
|
|
|
All tests should return a result of PASS or XFAIL. The XFAIL results
|
|
indicate known bugs. The final message should be one of the following:
|
|
|
|
All XX tests behaved as expected (YY expected failures)
|
|
All XX tests passed
|
|
|
|
If any tests have a status of FAIL, then the new code has caused a
|
|
regression error which should be fixed before the code is committed.
|
|
|
|
|
|
When a new bug is found
|
|
-----------------------
|
|
A new test case should be added by imitating the style of an existing
|
|
test. This means adding the following files:
|
|
|
|
new-bug.c
|
|
new-bug-ref.png
|
|
|
|
Where new-bug.c is a minimal program to demonstrate the bug, following
|
|
the style of existing tests. The new-bug-ref.png image should contain
|
|
the desired result of new-bug.c if the bug were fixed.
|
|
|
|
Makefile.am should be edited, adding new-bug.c to both the TESTS and
|
|
XFAIL_TESTS lists and new-bug-ref.png to EXTRA_DIST, add new-bug to
|
|
.gitignore, and last but not least, don't forget to "git add" the new
|
|
files.
|
|
|
|
|
|
When a new feature is added
|
|
---------------------------
|
|
It's important for the regression suite to keep pace with development
|
|
of the library. So a new test should be added for each new feature.
|
|
The work involved is similar the work described above for new bugs.
|
|
The only distinction is that the test is expected to pass so it
|
|
should not be added to the XFAIL_TESTS list.
|
|
|
|
|
|
When a bug is fixed
|
|
-------------------
|
|
The fix should be verified by running the test suite which should
|
|
result in an "unexpected pass" for the test of interest. Rejoice as
|
|
appropriate, then remove the relevant file name from the XFAIL_TESTS
|
|
variable in Makefile.am.
|
|
|
|
|
|
Before releasing
|
|
----------------
|
|
All tests should return a result of PASS for all supported (those enabled by
|
|
default) backends, meaning all known bugs are fixed, resulting in the happy
|
|
message:
|
|
|
|
All XX tests passed
|
|
|