The variable XFAIL_TESTS is not used anymore since commit e90073f7dd.
The description for the known failures are moved into the respective tests as
comments.
The following descriptions were dropped:
- surface-pattern-big: Didn't really explain the failure
- big-line: Test isn't failing any more
- self-intersecting: Only XFAIL on quartz, but description doesn't match this
The following tests don't have a xfail reference image and seem to fail just
because of not having a reference image at all (I kept their description for
now):
big-trap, long-lines, self-copy-overlap
Signed-off-by: Uli Schlachter <psychon@znc.in>
Instead of tagging the sources, which is insensitive to changes, track the
known failure modes by recording the current fail as an xfail.png
reference. (We also introduce a new.png to track a fresh error, so that
they are not lost in the noise of the old XFAILs and hopefully do not
cause everyone to fret).
As we have removed the XFAIL tagging we find, surprise surprise, that some
tests are now working -- so review all the reference images (as also some
.ref.png now should be .xfail.png).
Note: I've only checked image,pdf,ps,svg. The test surfaces report some
failures that probably need to addressed in source. I've not correct the
changes for win32 and quartz. Nor fixed up the experimental backends.
Avoid calling libtool to link every single test case, by building just one
binary from all the sources.
This binary is then given the task of choosing tests to run (based on user
selection and individual test requirement), forking each test into its own
process and accumulating the results.
In order to run under memfault, the framework is first extended to handle
running concurrent tests - i.e. multi-threading. (Not that this is a
requirement for memfault, instead it shares a common goal of storing
per-test data). To that end all the global data is moved into a per-test
context and the targets are adjusted to avoid overlap on shared, global
resources (such as output files and frame buffers). In order to preserve
the simplicity of the standard draw routines, the context is not passed
explicitly as a parameter to the routines, but is instead attached to the
cairo_t via the user_data.
For the masochist, to enable the tests to be run across multiple threads
simply set the environment variable CAIRO_TEST_NUM_THREADS to the desired
number.
In the long run, we can hope the need for memfault (runtime testing of
error paths) will be mitigated by static analysis. A promising candidate
for this task would appear to be http://hal.cs.berkeley.edu/cil/.
We'll need to rewrite this test (and the way cairo is compiled for
testing) before it can exercise the bug of interest. In the
meantime, let's just give it an "expected failure" status.
This test fills a single path (a trapezoid, even) that's larger than the
16.16 precision used in pixman and other code. It leads to memory
scribbling and a crash. Note that there is no crash if a clip is not set.
