Now, if you pass exactly two performance reports on the command line
you'll get the traditional report style again, (so the tool remains
backwards compatible). If you really want the new style with two
reports you can get it by adding /dev/null as a third argument.
This support is intended to compare the identical backends across multiple
reports from several different configurations, (of one sort or another).
The configuration names used in the report are taken from the filenames
of the report files, (which will format most nicely if 8 characters or
less).
The traditional two-input report mode, (showing one line perdiff with
all speedups before all slowdowns), is removed with this commit, but
is intended to return again shortly.
We terminate the iteration by adding a final report with a NULL
name. This will simplify future code that iterates over more than
two reports simultaneously.
An upcoming change will allow cairo-perf-diff-files to generate
a report based on more than two files. This prepares for that by
moving the minimum-change support from the 3rd positional argument
to a new --min-change option.
Previously only "--" would givethe usage, and not "--help"
as intended. The new approach has been tested with ash, bash,
and dash, (hopefully that's good enough).
Although cairo-perf is not written to perform explicit failure testing of
cairo, it does generate long sequences of cairo operations which often
trigger unexpected errors. By including it with make check, it becomes
even easier for the programmer to check that one has not broken cairo in
terms of expected behaviour or performance.
Ideally, the minimum value would indicate the best possible performance,
but I've encountered situations where a bogus minimum value gets lodged
into the cached results for some reason, (and yet doesn't also get
discarded as an outlier). The downside of that situation is that running
more iterations never changes the result, so it's hard to fix the problem,
(resulting in cairo-perf-diff feeling just plain broken as more runs
change nothing).
So let's try using the median time instead.
The text perf-case tries to fill the region with a single text string,
but fails to detect when the current point does not advance due to an
error. This causes the perf-case to enter an infinite loop, so we break
out when the cairo_status() has been set.
cairo-perf and the X server should be bound to CPUs (either the same
or separate) on SMP systems. Not doing so causes random results when
the X server is moved to or from cairo-perf's CPU during the
benchmarks.
When choosing the samples that are not outliers we use a half-open interval
(outlier_min <= x < outlier_max). This causes all of the samples to be
discarded when the interquartile range is 0 because none of them are less
than outlier_max. Fix the problem and make the test more consistent by
using a closed interval (outliner_min <= x <= outlier_max).
This patch adds options to disable the UTF-8 change bars and replace
them with ASCII '*' gfx. The motivation is that UTF-8 really does some
damage to some terminals, and some always forget to pipe stuff through
a pager to make it safe. The new options for cairo-perf-diff-files are:
--no-utf
--no-bars
This test shows that drawing a 100x100 single-pixel wide box outline is
currently 5 to 16 times slower when using the natural cairo_stroke() as
compared to a rather awkward cairo_fill() of two rectangles.
[ # ] backend-content test-size min(ticks) min(ms) median(ms) stddev. iterations
[ 0] image-rgba box-outline-stroke-100 301321 0.218 0.219 0.39% 5
[ 1] image-rgba box-outline-fill-100 18178 0.013 0.013 0.43% 5
[ 0] xlib-rgba box-outline-stroke-100 379177 0.275 0.276 1.39% 6
[ 1] xlib-rgba box-outline-fill-100 83355 0.060 0.060 0.17% 5
