MacOS X 10.7 (and maybe some previous versions, too) can fork()
processes which use CoreGraphics. This makes it possible for
cairo-test-suite to withstand a test crash without killing the whole
suite.
The old behavior is still available using the '-f' (foreground)
option.
Instead of embedding the pointer in the test structure, consistently
use the cairo_test_list_t structure for test lists.
This cleans up the code as the reverse-list operation can be reused.
Moreover this makes the code clearer, because each test list is now
independent and has no way to know about other test lists.
Keep the option flags in alphabetical order. This makes it easier to
check for collisions or missing handlers.
Avoids an internal error when passing flags -c, -r or -v to
cairo-analyse-trace.
If CAIRO_TESTS is set, argv will be replaced by a xmaloc'ed variable. We
need to free it.
This can be easily detected by running:
CAIRO_TESTS=user-font make check-valgrind TARGETS=image
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
When I ran "CAIRO_TESTS=a1-bug make test", no test executed because of a
bug in append_argv(). The "olen" variable was assuming that we always
only append a single argument to argv and the resulting argc was also
wrong.
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
warning: cannot optimize possibly infinite loops
gcc does not detect that the "infinite" loops are actually just one or
two iterations, depending on the has_similar value being FALSE or
TRUE. It realizes it if the iteration variable and the iteration stop
value are both enum values.
Specifying individual tests to run is more common than using a group
chosen from a keyword, so change the default matching mode and introduce
'-k' to select by keyword.
Hitting an error in a test case is almost as bad as crashing, and the
severity may be lost amidst "normal" failures. So introduce a new class
of ERROR so that we can immediately spot these during a test run, and
appropriately log them afterwards.
Set the CAIRO_TEST_FORCE_PASS environment variable to run through the
test suite and ignore errors. Useful for forcing distcheck to continue
past a broken test suite.
A common mistake is to forget to pass the foreground mode to
cairo-test-suite when launching it under the debugger, resulting in the
debugger not attaching to the children and missing the error you were
trying to capture. Under linux, we can inspect the path to our parent's
executable and if that looks like gdb, we assume it is and disable forking
of traces.
Add a command line option to the test suite to cause it to exit after the
first failure. The purpose of this is for integration into 'git bisect run',
where the failing test is unknown and we are looking for any failure. For
example, for use in a regression script to find commits in the midst of as
series that need a refresh of a reference image (or fixing!).
Instead of tagging the sources, which is insensitive to changes, track the
known failure modes by recording the current fail as an xfail.png
reference. (We also introduce a new.png to track a fresh error, so that
they are not lost in the noise of the old XFAILs and hopefully do not
cause everyone to fret).
As we have removed the XFAIL tagging we find, surprise surprise, that some
tests are now working -- so review all the reference images (as also some
.ref.png now should be .xfail.png).
Note: I've only checked image,pdf,ps,svg. The test surfaces report some
failures that probably need to addressed in source. I've not correct the
changes for win32 and quartz. Nor fixed up the experimental backends.
The configure test for __attribute__((constructor)) succeeds when
compiling with Sun Studio 12 on OpenSolaris but the attribute
is just ignored and has no actual effect. This causes the test
suite to not run any tests at all. With this patch we revert to
always using make-cairo-test-constructors.pl.
The test suite depends upon many external factors and in order to achieve
a pass, your system must match that upon which the reference images were
generated. Add a note to read test/README in case of failures so that the
casual user is not unduly alarmed by cairo reportedly failing.
When the cairo-test-suite forks CoreFoundation complains with:
"The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
When forked so avoid doing it in the test suite for now. In the future we should investigate
the possibility of a work around.
The attached patch makes the SDL tests compile under Mac OS X. The
problem is:
1) that <SDL_main.h> should be included in files that define the main
function for SDL Mac OS X programs (this is not true with the upcoming
SDL 1.3 release).
2) that -lSDLmain, because it is statically linked, needs the Cocoa
framework in the LDADD of the main program. Again, 1.3 will not require
this.
Avoid calling libtool to link every single test case, by building just one
binary from all the sources.
This binary is then given the task of choosing tests to run (based on user
selection and individual test requirement), forking each test into its own
process and accumulating the results.
