we can convert f16->f32 for free on read, so we can compact constants to reduce
register pressure. this makes constant promotion more effective.
this saves a few instructions in "wall and chimney".
total instructions in shared programs: 2039709 -> 2039862 (<.01%)
instructions in affected programs: 12585 -> 12738 (1.22%)
helped: 0
HURT: 3
total bytes in shared programs: 14111800 -> 14112726 (<.01%)
bytes in affected programs: 102778 -> 103704 (0.90%)
helped: 7
HURT: 4
Inconclusive result (value mean confidence interval includes 0).
total uniforms in shared programs: 1533232 -> 1532271 (-0.06%)
uniforms in affected programs: 60255 -> 59294 (-1.59%)
helped: 481
HURT: 0
Uniforms are helped.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
Add an optimization pass to promote constants loaded in the shader to dedicated
uniform registers preloaded before the shader. This is beneficial for two
reasons:
* fewer mov_imm instructions
* less GPR pressure (uniforms have dedicated registers)
The latter can significantly improve occupancy since we don't remat constants
for occupancy. We do remat to avoid spilling so it won't affect spilling,
although it can still be a win by reducing remat when a shader would otherwise
spill.
The problem is that we have limited uniform registers so can't promote
everything that we would want to. We model this as a 0-1 knapsack problem and
use the well-known heuristic to prioritize frequently used constants. This is
not optimal but works quite well in practice.
This gives a nice fps win in some complex shaders, including:
* Dolphin ubers from 10.25fps to 10.85fps at 4K in MMG.
* "Wall and chimney" shadertoy from 24.8fps to 29.5fps at 1188x658.
shader-db results are excellent as well.
total instructions in shared programs: 2088290 -> 2039709 (-2.33%)
instructions in affected programs: 1478061 -> 1429480 (-3.29%)
helped: 8246
HURT: 85
Instructions are helped.
total bytes in shared programs: 14321004 -> 14111800 (-1.46%)
bytes in affected programs: 10108742 -> 9899538 (-2.07%)
helped: 7999
HURT: 1416
Bytes are helped.
total regs in shared programs: 602415 -> 590371 (-2.00%)
regs in affected programs: 92177 -> 80133 (-13.07%)
helped: 1887
HURT: 209
Regs are helped.
total uniforms in shared programs: 1457531 -> 1533232 (5.19%)
uniforms in affected programs: 835522 -> 911223 (9.06%)
helped: 0
HURT: 11042
Uniforms are HURT.
total threads in shared programs: 20325824 -> 20329216 (0.02%)
threads in affected programs: 29632 -> 33024 (11.45%)
helped: 41
HURT: 0
Threads are helped.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
don't bother the compiler proper about it. this now counts NIR scratch access as
spills/fills, which I think is probably the right call
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
ssa_to_reg is necessarily sparse, and since it's allocated per block, it's
tremendously memory intensive for shaders with thousands of blocks (which
can easily happen with if-ladders)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
spilling and SSA repair will generate piles of dead SSA defs. add a reindexing
pass to keep memory usag emanagable on large shaders.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
This massively improves our coalescing of phis by considering not just single
phi instructions but entire webs of phi-related SSA values. We do this with a
union-find data structure, which is effectively constant time thanks to union by
rank and path compression. Phi related SSA values are unioned and we try to
assign the same register to everything in the union. Boissinot might be better
but this is delightfully simple.
total instructions in shared programs: 2910655 -> 2883792 (-0.92%)
instructions in affected programs: 1295671 -> 1268808 (-2.07%)
helped: 1129
HURT: 34
Instructions are helped.
total bytes in shared programs: 19417970 -> 19255234 (-0.84%)
bytes in affected programs: 8790112 -> 8627376 (-1.85%)
helped: 1129
HURT: 34
Bytes are helped.
total halfregs in shared programs: 517813 -> 517867 (0.01%)
halfregs in affected programs: 751 -> 805 (7.19%)
helped: 2
HURT: 15
Halfregs are HURT.
total spills in shared programs: 135918 -> 134070 (-1.36%)
spills in affected programs: 135918 -> 134070 (-1.36%)
helped: 6
HURT: 0
total fills in shared programs: 343204 -> 341356 (-0.54%)
fills in affected programs: 343204 -> 341356 (-0.54%)
helped: 6
HURT: 0
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
this requires some special handling but closes the last soundness gap (I hope)
in our RA. with later patches in this series, we actually hit this (50+ tests on
the CTS even) so I can be sure this actually works ^^
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
ASAN saves the day! Was stuck on this for hours.
==37495==ERROR: AddressSanitizer: stack-buffer-overflow on address 0xfffff29ecdbc at pc 0xffff7c0751f4 bp 0xfffff29eca30 sp 0xfffff29eca48
READ of size 4 at 0xfffff29ecdbc thread T0
#0 0xffff7c0751f0 in __bitset_set_range ../src/util/bitset.h:249
#1 0xffff7c0751f0 in find_regs ../src/asahi/compiler/agx_register_allocate.c:642
#2 0xffff7c077d2c in pick_regs ../src/asahi/compiler/agx_register_allocate.c:1008
#3 0xffff7c077d2c in agx_ra_assign_local ../src/asahi/compiler/agx_register_allocate.c:1096
#4 0xffff7c077d2c in agx_ra ../src/asahi/compiler/agx_register_allocate.c:1353
#5 0xffff7c03b6c4 in agx_compile_function_nir ../src/asahi/compiler/agx_compile.c:2840
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
