Compare the current output against a previous run to determine if there
has been any change since last time, and only run through imagediff if
there has been. For the vector surfaces, we can check the vector output
first and potentially skip the rasterisation. On my machine this reduces
the time for a second run from 6 minutes to 2m30s. As most of the time,
most test output will remain unchanged, so this seems to be a big win. On
unix systems, hard linking is used to reduce the amount of storage space
required - others will see about a three-fold increase in the amount of
disk used. The directory continues to be a stress test for file selectors.
In order to reduce the changes between runs, the current time is no longer
written to the PNG files (justified by that it only exists as a debugging
aid) and the boilerplate tweaks the PS surface so that the creation date
is fixed. To fully realise the benefits here, we need to strip the
creation time from all the reference images...
The biggest problem with using the caches is that different runs of the
test suite can go through different code paths, introducing potential
Heisenbergs. If you suspect that caching is interfering with the test
results, use 'make -C test clean-caches check'.
This is justified by the previously mentioned bug on poppler
gradients. Note that this test only passes with a patch
to fix a bug in poppler (not yet upstream). Here it is:
PATCH: Keep 'cairo_shape' and 'cairo' consistent
https://bugs.freedesktop.org/show_bug.cgi?id=14593
This is justified by the following new bug report:
Poppler should paint images with CAIRO_EXTEND_PAD
https://bugs.freedesktop.org/show_bug.cgi?id=14578
The four affected tests are:
paint-source-alpha
paint-with-alpha
rotate-image-surface-paint
scale-source-surface-paint
Allow using a previous test output directory as a source of
reference images. To make use of this, set the environment
variable 'CAIRO_REF_DIR' to point at an old test directory,
relative to the current test directory.
This is useful for testing backends when reference images haven't
been created yet, or which the current reference image structure
can't accomodate, like multiple font backends.
- Remove cairo_test_expect_failure. cairo-test.c now checks
env var CAIRO_XFAIL_TESTS to see if the running test is
expected to fail. The reason for expected failure is
appended to the test description.
- Test description is written out.
- Failed/crashed tests also write a line out to stderr (in red),
so one can now redirect stdout to /dev/null to only see failures.
- cairo_test() has been changed to not take the draw function
anymore, instead, draw function is now part of the test struct.
- "make check" doesn't allow limiting backends to test using env
var anymore. To limit backends to test, one should use the
TARGETS variable on the make command line.
- "make check-valgrind" now writes its log to valgrind-log instead
of valgrind.log, to not interfere with test log file processing.
Don't AC_SUBST a dozen different FOO_CFLAGS and FOO_LIBS. Instead, incrementally build up just CAIRO_CFLAGS and CAIRO_LIBS.
Don't list flags that should get picked up via dependency information through Requires.
Add description of move_to_show_surface.
