As Behdad suggested, we can dramatically speed up the test suite by
short-circuiting the write to a png file, only to then immediately read it
back in. So for the raster based surfaces, we avoid the round-trip through
libpng by implementing a new boilerplate method to directly extract the image
buffer from the test result. A secondary speedup is achieved by caching the
most recent reference image.
In order to run under memfault, the framework is first extended to handle
running concurrent tests - i.e. multi-threading. (Not that this is a
requirement for memfault, instead it shares a common goal of storing
per-test data). To that end all the global data is moved into a per-test
context and the targets are adjusted to avoid overlap on shared, global
resources (such as output files and frame buffers). In order to preserve
the simplicity of the standard draw routines, the context is not passed
explicitly as a parameter to the routines, but is instead attached to the
cairo_t via the user_data.
For the masochist, to enable the tests to be run across multiple threads
simply set the environment variable CAIRO_TEST_NUM_THREADS to the desired
number.
In the long run, we can hope the need for memfault (runtime testing of
error paths) will be mitigated by static analysis. A promising candidate
for this task would appear to be http://hal.cs.berkeley.edu/cil/.
The interface of the various buffer/image_diff functions is improved to
provide the maximum pixel difference in addition to the number of pixels
that differ. This value can then be used to compare against a per-backend
tolerance.
Currently I've set the SVG backend's tolerance to 1 to handle some issues
we're currently seeing of single-bit differences on different systems, (but
we're not exactly sure why yet).
Also I improved the image_diff routines to properly report a status value
on failure rather than the bogus value of -1 for pixels_changed.
This patch was produced with the following (GNU) sed script:
sed -i -r -e 's/[ \t]+$//'
run on all *.[ch] files within cairo.
Note that the above script would have also created all the changes
from the previous commits to remove trailing whitespace.
This patch was produced with the following (GNU) sed script:
sed -i -r -e '/^[ \t]*\/?\*/ s/[ \t]+$//'
run on all *.[ch] files within cairo, (though I manually excluded
src/cairo-atsui-font.c which has a code line that appears as a comment
to this script).
[With this change, two tests with gradients exhibit subtle differences
under device offsets. I don't think we actually care about them
though. -cworth]
In several cases the -rgb24 reference images were identical to the -ref images.
So we change the logic to find the latter if the former doesn't exist, and then
we remove the redundant images.
Add new testsvg script and accompanying imagediff program, (for interim SVG-based test suites while we wait for the standard cairo test suite to mature).
Split buffer_diff out into its own file for the purpose of imagediff.
