This causes the graph coloring allocator to use the optimistic
coloring codepath for all nodes whose total Q value exceeds the
threshold of 96 GRFs, in order to do a better job at minimizing the
register requirement of programs even when they are trivially
colorable. At the threshold of 96 GRFs the number of threads
available per EU starts decreasing as the number of register blocks
requested by the program increases, so decreasing the number of
registers can increase performance.
That showed up in some test cases as a performance inversion from the
enabling of VRT, since the extension of the register set to 256 GRFs
has the side effect of making some non-trivially colorable programs
trivially colorable, which would cause the register allocator to do a
worse job at ordering the (trivial) allocations due to the optimistic
coloring path being skipped, leading to increased register use and
reduced performance.
The following Traci test cases improve significantly as a result of
this change (4 iterations, 5% significance):
MetroExodus-trace-dx11-2160p-ultra: 1.90% ±0.85%
BaldursGate3-trace-dx11-1440p-ultra: 1.47% ±0.38%
Palworld-trace-dx11-1080p-med: 1.01% ±0.09%
TerminatorResistance-trace-dx11-2160p-ultra: 0.95% ±0.29%
Control-trace-dx11-1440p-high: 0.87% ±0.50%
Even though lowering the P value threshold is expected to have a cost
in compile time theoretically due to the increased use of the slower
optimistic path of the graph coloring allocator, this doesn't actually
show up in my numbers, my shader-db and fossil-db compile-time numbers
don't show any statistically significant change (13 iterations, 5%
significance).
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36618>
This is helpful for the driver to have the option to provide a custom
threshold for the PQ test performed by the graph coloring algorithm.
A threshold lower than the physical number of registers is helpful on
platforms where the number of registers used can impose a limit on the
thread parallelism of the program. In such platforms even though a
passing PQ test guarantees that the node can be pushed onto the stack
and neglected while coloring the remaining nodes, the ordering in
which this happen can have a dramatic effect in the register pressure
of the resulting shader and therefore also on the thread parallelism
of the program.
Setting a P value threshold lower than the real P value will cause
nodes with Q value above the threshold to use the existing optimistic
coloring heuristic that takes the effort of ordering nodes in the
stack by Q value, in order to do a better job at minimizing the total
register requirement of the program. Even though this causes us to
hit the optimistic codepaths for trivially colorable nodes the
interference graph is still guaranteed to be trivially colorable if it
was trivially colorable without the override.
The use of a threshold lower than the real P value will come at a
compile-time performance cost, the specific trade-off between
compile-time and run-time can be adjusted by the driver based on the
number of registers available to each thread without causing a hit to
thread parallelism.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36618>
This defines a new pre-RA scheduling mode similar to BRW_SCHEDULE_PRE
but more aggressive at optimizing for minimum latency rather than
minimum register usage. The main motivation is that on recent xe3
platforms we use a register allocation heuristic that packs variables
more tightly at the bottom of the register file instead of the
round-robin heuristic we used on previous platforms, since as a result
of VRT there is a parallelism penalty when a program uses more GRF
registers than necessary. Unfortunately the xe3 tight-packing
heuristic severely constrains the work of the post-RA scheduler due to
the false dependencies introduced during register allocation, so we
can do a better job by making the scheduler aware of instruction
latencies before the register allocator introduces any false
dependencies.
This can lead to higher register pressure, but only when the scheduler
decides it could save cycles by extending a live range. It makes
sense to preserve the preexisting BRW_SCHEDULE_PRE as a separate mode
since some workloads can still benefit from neglecting latencies
pre-RA due to the trade-off mentioned between parallelism and GRF use,
a future commit will introduce a more accurate estimate of the
expected relative performance of BRW_SCHEDULE_PRE
vs. BRW_SCHEDULE_PRE_LATENCY taking into account this trade-off.
In theory this could also be helpful on earlier pre-xe3 platforms, but
the benefit should be significantly smaller due to the different RA
heuristic so it hasn't been tested extensively pre-xe3.
The following Traci tests are improved significantly by this change on
PTL (nearly all tests that run on my system are affected positively):
Ghostrunner2-trace-dx11-1440p-ultra: 7.12% ±0.36%
SpaceEngineers-trace-dx11-2160p-high: 5.77% ±0.43%
HogwartsLegacy-trace-dx12-1080p-ultra: 4.40% ±0.03%
Naraka-trace-dx11-1440p-highest: 3.06% ±0.43%
MetroExodus-trace-dx11-2160p-ultra: 2.26% ±0.60%
Fortnite-trace-dx11-2160p-epix: 2.12% ±0.53%
Nba2K23-trace-dx11-2160p-ultra: 1.98% ±0.30%
Control-trace-dx11-1440p-high: 1.93% ±0.36%
GodOfWar-trace-dx11-2160p-ultra: 1.62% ±0.47%
TotalWarPharaoh-trace-dx11-1440p-ultra: 1.55% ±0.18%
MountAndBlade2-trace-dx11-1440p-veryhigh: 1.51% ±0.37%
Destiny2-trace-dx11-1440p-highest: 1.44% ±0.34%
GtaV-trace-dx11-2160p-ultra: 1.26% ±0.27%
ShadowTombRaider-trace-dx11-2160p-ultra: 1.10% ±0.58%
Borderlands3-trace-dx11-2160p-ultra: 0.95% ±0.43%
TerminatorResistance-trace-dx11-2160p-ultra: 0.87% ±0.22%
BaldursGate3-trace-dx11-1440p-ultra: 0.84% ±0.28%
CitiesSkylines2-trace-dx11-1440p-high: 0.82% ±0.22%
PubG-trace-dx11-1440p-ultra: 0.72% ±0.37%
Palworld-trace-dx11-1080p-med: 0.71% ±0.26%
Superposition-trace-dx11-2160p-extreme: 0.69% ±0.19%
The compile-time cost of shader-db increases significantly by 1.85%
after this commit (14 iterations, 5% significance), the compile-time
of fossil-db doesn't change significantly in my setup.
v2: Addressed interaction with 81594d0db1,
since the code that calculates deps, delays and exits is no longer
mode-independent after this change. Instead of reverting that
commit (which is non-trivial and would have a greater compile-time
hit) simply reconstruct the scheduler object during the transition
between BRW_SCHEDULE_PRE_LATENCY and any other PRE mode that
doesn't require instruction latencies.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36618>
We were currently treating explicit flag writes and reads as a full
scheduler barrier, which is unnecessary since the tracking we already
do handles explicit flag access correctly so there is no reason for
taking a possibly large performance hit from add_barrier_deps().
Found by inspection while trying to understand the poor scheduling of
some fragment shaders. Improves performance by a small but
statistically significant amount (4 iterations, 5% significance) for
the following Traci tests in combination with a subsequent commit that
makes the pre-RA scheduler sensitive to instruction latencies:
SpaceEngineers-trace-dx11-2160p-high: 0.66% ±0.30%
MountAndBlade2-trace-dx11-1440p-veryhigh: 0.62% ±0.23%
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36618>
We weren't handling the SHADER_OPCODE_SEND_GATHER instruction in the
instruction scheduler and this was leading to reduced performance in
many programs since SEND instructions have the longest latency and
tend to be among the most critical to schedule efficiently. Handle
SENDG similarly to SEND since the timings of both instructions are
mostly bound by the shared function which doesn't care if the message
was sent by SEND or SENDG.
Improves performance significantly in the following Traci traces (4
iterations, 5% significance), most of them regressions from SENDG
being enabled:
MetroExodus-trace-dx11-2160p-ultra: 1.99% ±0.88%
HogwartsLegacy-trace-dx12-1080p-ultra: 1.33% ±0.20%
GtaV-trace-dx11-2160p-ultra: 1.12% ±0.19%
Borderlands3-trace-dx11-2160p-ultra: 1.00% ±0.58%
TerminatorResistance-trace-dx11-2160p-ultra: 0.98% ±0.27%
Control-trace-dx11-1440p-high: 0.91% ±0.36%
Naraka-trace-dx11-1440p-highest: 0.90% ±0.30%
Ghostrunner2-trace-dx11-1440p-ultra: 0.87% ±0.38%
Palworld-trace-dx11-1080p-med: 0.71% ±0.17%
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36618>
Not required since we've disabled maintenance8 support.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: d39e443ef8 ("anv: add infrastructure for common vk_pipeline")
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37242>
refcounting uses atomics, which are a significant source of CPU overhead
in many applications. by adding a method to inform the driver that
the frontend has released ownership of a buffer, all other refcounting
for the buffer can be eliminated
see MR for more details
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36296>
A barrier between two lds/vmem instructions needs to ensure that the
second starts after the first finishes, which means that we can't just
skip workgroup-scope vmem barriers if there is a lds instruction later.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36491>
For code like:
if (cond) {
val = load()
}
use(val)
The "use(val)" now has a similar cost to a use inside the IF.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36491>
* optimize not(compare(a,b)), nir_opt_algebraic does this only if the
comparison result is used only once, but on a vector arch we still get
an advantage when doing this, because it reduces dependencies.
* optimize b2f32(compare(a,b)), this is r600 specific
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37205>
This was incorrect (it also lowered int64 reductions/scans), and the only
user can just use the general callback to precisely only lower what it wants.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37164>
This was added with the goal to eventually replace the per
pass subgroup/ballot size options, but that won't work because
some backends don't have a fixed subgroup size across the compilation
process.
It was also mostly added to hack around mesa state tracker behavior,
and we have a better solution there now.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37164>
We skip iterations with ifs.
These can be optimized aways after the subgroup size is known.
Every driver should do that because applications depend on it anyway.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37164>
If the offset is iadd(iadd(iadd(a, 1), b), -1), try_extract_const_addition
will create a dead iadd(a, b) and claim that it didn't modify the shader.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36370>
Commit d2f7b6d5 changed the BLEND_STATE update process so that only
the used render targets will be updated. This mostly works fine, but
in cases when the Dual Source Blending was used previously, we still
must turn it off to avoid the undefined behavior that leads to hangs.
Fixes: d2f7b6d5 ("anv: implement VK_KHR_dynamic_rendering_local_read")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13675
Signed-off-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37246>
