The ARRAY_LENGTH macro is used by many tests, although sometimes it is
named ARRAY_SIZE. Define it just once in cairo-test.h and reuse it.
In a similar way, MAX() and MIN() are currently defined in some
specific tests, while they could be reused.
The old radial gradient tests were actually drawing the same gradients
with different operators (OVER/SOURCE) and operations (paint/mask).
It is possible to refactor them to share the gradient construction
code. This makes it easy to change the gradient shape in all of them
to test more interesting combinations.
Instead of testing 16 generic positions we are now testing just 7
cases, which correspond to every possible combination of the relative
size and position of the two circles defining the gradient. In
particular we are now testing a constant radius gradient and gradients
with tangent circles.
Avoid calling libtool to link every single test case, by building just one
binary from all the sources.
This binary is then given the task of choosing tests to run (based on user
selection and individual test requirement), forking each test into its own
process and accumulating the results.
In order to run under memfault, the framework is first extended to handle
running concurrent tests - i.e. multi-threading. (Not that this is a
requirement for memfault, instead it shares a common goal of storing
per-test data). To that end all the global data is moved into a per-test
context and the targets are adjusted to avoid overlap on shared, global
resources (such as output files and frame buffers). In order to preserve
the simplicity of the standard draw routines, the context is not passed
explicitly as a parameter to the routines, but is instead attached to the
cairo_t via the user_data.
For the masochist, to enable the tests to be run across multiple threads
simply set the environment variable CAIRO_TEST_NUM_THREADS to the desired
number.
In the long run, we can hope the need for memfault (runtime testing of
error paths) will be mitigated by static analysis. A promising candidate
for this task would appear to be http://hal.cs.berkeley.edu/cil/.
The 5 additional bugs that will be shipped with 1.4 are
ft-text-vertical-layout-type1
radial-gradient
surface-pattern
surface-pattern-scale-down
surface-pattern-scale-up
Most of these are non-issues, (unbundled font for
ft-text-vertical-layout-type1), or very minor issues (radial-gradient
and surface-pattern). The only things in here that look like a real
bug are the surface-pattern-scale-down and surface-pattern-scale-up
tests where the xlib backend results have some non-1.0 alpha that is
very unexpected.
Embarrassingly enough, the test suite previously never called
into cairo_pattern_create_radial at all. Unsurprisingly, this
has led to bugs creeping into the radial gradient implementation.
