Instead of tagging the sources, which is insensitive to changes, track the
known failure modes by recording the current fail as an xfail.png
reference. (We also introduce a new.png to track a fresh error, so that
they are not lost in the noise of the old XFAILs and hopefully do not
cause everyone to fret).
As we have removed the XFAIL tagging we find, surprise surprise, that some
tests are now working -- so review all the reference images (as also some
.ref.png now should be .xfail.png).
Note: I've only checked image,pdf,ps,svg. The test surfaces report some
failures that probably need to addressed in source. I've not correct the
changes for win32 and quartz. Nor fixed up the experimental backends.
Avoid calling libtool to link every single test case, by building just one
binary from all the sources.
This binary is then given the task of choosing tests to run (based on user
selection and individual test requirement), forking each test into its own
process and accumulating the results.
In order to run under memfault, the framework is first extended to handle
running concurrent tests - i.e. multi-threading. (Not that this is a
requirement for memfault, instead it shares a common goal of storing
per-test data). To that end all the global data is moved into a per-test
context and the targets are adjusted to avoid overlap on shared, global
resources (such as output files and frame buffers). In order to preserve
the simplicity of the standard draw routines, the context is not passed
explicitly as a parameter to the routines, but is instead attached to the
cairo_t via the user_data.
For the masochist, to enable the tests to be run across multiple threads
simply set the environment variable CAIRO_TEST_NUM_THREADS to the desired
number.
In the long run, we can hope the need for memfault (runtime testing of
error paths) will be mitigated by static analysis. A promising candidate
for this task would appear to be http://hal.cs.berkeley.edu/cil/.
cairo_get_target() returns the original surface passed to
cairo_create(), and not the current destination as required when
testing drawing to the same surface using multiple contexts.
For completeness we also use the group target when creating similar
surfaces within the tests (to check that similar surfaces of similar
surfaces also work).
We update the test suite reference images where needed, (pdiff
avoided a few, but most still needed updating). We take advantage
of the need for new reference images to shrink some of the giant
tests to speed them up a bit.
This optimization provides a 2x improvement in linear gradient
generation performance (numbers from an x86 laptop):
image-rgb paint_linear_rgba_source-512 26.13 -> 11.13: 2.35x speedup
█▍
image-rgb paint_linear_rgba_source-256 6.47 -> 2.76: 2.34x speedup
█▍
image-rgba paint_linear_rgb_over-256 6.51 -> 2.86: 2.28x speedup
█▎
image-rgb paint_linear_rgba_over-512 28.62 -> 13.70: 2.09x speedup
█▏
image-rgba fill_linear_rgb_over-256 3.24 -> 1.94: 1.66x speedup
▋
image-rgb stroke_linear_rgba_over-256 5.68 -> 4.10: 1.39x speedup
▍
This is the same fix as in the recent commit (166dffc96a)
but for the same idiom that appears in the operator-clear,
operator-source, and unbounded-operator tests. This fixes
the failures that were introduced by the recent switch to
_cairo_lround.
- Remove cairo_test_expect_failure. cairo-test.c now checks
env var CAIRO_XFAIL_TESTS to see if the running test is
expected to fail. The reason for expected failure is
appended to the test description.
- Test description is written out.
- Failed/crashed tests also write a line out to stderr (in red),
so one can now redirect stdout to /dev/null to only see failures.
- cairo_test() has been changed to not take the draw function
anymore, instead, draw function is now part of the test struct.
- "make check" doesn't allow limiting backends to test using env
var anymore. To limit backends to test, one should use the
TARGETS variable on the make command line.
- "make check-valgrind" now writes its log to valgrind-log instead
of valgrind.log, to not interfere with test log file processing.
CLEAR and SOURCE are now bounded.
Assert that SOURCE and CLEAR aren't passed to these functions.
Assert that SOURCE and CLEAR aren't passed to these functions when there is a mask.
Do fixups for SOURCE and CLEAR as well as unbounded operators, since in the absence of a mask, we need SOURCE to work correctly (don't care about CLEAR)
_cairo_ft_font_show_glyphs) Consistently use CLEAR/TRANSPARENT (source doesn't matter) rather than SOURCE/TRANSPARENT when clearing rectangles.
src/cairo-xlib-surface.c src/cairo-surface.c: Use IN rather than SOURCE as an example of an unbounded operator in docs.
Remove CLEAR/SOURCE columns since they are no longer unbounded.
test/operator-clear.c test/operator-source Makefile.am: Add targetted tests of CLEAR/SOURCE.
