Natalie Vock
e8f1dc687c
aco: Use parameter assignment hints for any-hit shaders
...
Query the signature of the traversal function stored in the any-hit
shader and make the parameter locations between the two match up, to
remove unnecessary movs inside the traversal loop.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39314 >
2026-01-20 21:49:55 +00:00
Natalie Vock
a32709674a
aco: Add parameter assignment hints
...
Parameter assignment hints allow to influence parameter register
assignment logic with user-specified affinities. If there is an affinity
declared for a parameter, the assignment logic will try to match the
registers a parameter and its affinity are assigned.
It also allows to hint that certain registers are not suitable for
assigning parameters to and should be avoided.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39314 >
2026-01-20 21:49:55 +00:00
Natalie Vock
2d6ecf303a
aco: Add and use nir_abi_to_aco helper
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39314 >
2026-01-20 21:49:55 +00:00
Natalie Vock
a03e9287c3
radv/rt: Compile ahit/isec shaders to asm
...
We can express any-hit/intersection shaders as functions, too.
Any-hit/Intersection shaders need the usual parameters like launch
IDs/descriptor data/ray properties, origin, direction/etc., but also
some special parameters related to traversal state. Any-hit/intersection
shaders need to return whether the hit was accepted and/or traversal
should be terminated, as well as the intersection T value (for
intersection shaders). Both any-hit and intersection shaders also need
to be passed hit attributes via parameters. Closest-Hit shaders need
those too, but we pass them out-of-band via LDS. LDS is used for the
traversal stack when any-hit/intersection shaders, so we need to pass
them via parameters.
Hit attributes are similar to ray payloads in the sense that they're
dynamically sized depending on how much space the application uses.
However, unlike ray payloads, hit attribute sizes have a strict upper
bound of 8 dwords. To make managing parameters easier, we put all hit
attributes in a single vector parameter with 0-8 components. This
prevents having a function with two sets of arbitrary numbers of
parameters.
This commit sets up ahit/isec function signatures and implements
lowering for ahit/isec-specific intrinsics in the context of these
functions. Subsequent commits will merely have to call into these
functions to execute a separate-compiled any-hit/intersection shader.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39314 >
2026-01-20 21:49:55 +00:00
Natalie Vock
b52adac42c
aco: Tweak ABI register param limits
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39314 >
2026-01-20 21:49:54 +00:00
Natalie Vock
7a2f050daa
aco: Put boolean parameters inside SGPRs
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39314 >
2026-01-20 21:49:54 +00:00
Georg Lehmann
939b4a6476
aco/optimizer: apply v_cvt_pkrtz_f16_f32 as fma_mix to operands
...
Foz-DB Navi21:
Totals from 2085 (2.14% of 97591) affected shaders:
Instrs: 4880879 -> 4882355 (+0.03%); split: -0.04%, +0.07%
CodeSize: 26869332 -> 26881744 (+0.05%); split: -0.02%, +0.06%
VGPRs: 93944 -> 94160 (+0.23%); split: -0.06%, +0.29%
Latency: 40035558 -> 40035595 (+0.00%); split: -0.02%, +0.02%
InvThroughput: 10333800 -> 10329093 (-0.05%); split: -0.06%, +0.01%
VClause: 139147 -> 139148 (+0.00%)
Copies: 454527 -> 454656 (+0.03%); split: -0.00%, +0.03%
VALU: 3214838 -> 3211105 (-0.12%)
Foz-DB Navi48:
Totals from 2349 (2.41% of 97637) affected shaders:
Instrs: 6471998 -> 6471817 (-0.00%); split: -0.05%, +0.05%
CodeSize: 34793372 -> 34808748 (+0.04%); split: -0.02%, +0.06%
VGPRs: 141804 -> 142560 (+0.53%)
Latency: 45225910 -> 45226000 (+0.00%); split: -0.01%, +0.01%
InvThroughput: 9152634 -> 9149850 (-0.03%); split: -0.04%, +0.01%
VClause: 148536 -> 148537 (+0.00%)
Copies: 527206 -> 527336 (+0.02%); split: -0.01%, +0.03%
VALU: 3491701 -> 3487347 (-0.12%); split: -0.12%, +0.00%
VOPD: 669 -> 683 (+2.09%); split: +2.69%, -0.60%
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38815 >
2026-01-20 14:48:23 +00:00
Georg Lehmann
c6b74705dd
aco/optimizer: support fma_mix with rtz
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38815 >
2026-01-20 14:48:23 +00:00
Georg Lehmann
6b9d28ab9b
aco/insert_fp_mode: insert fp mode in reverse
...
This allows us to skip the mode set by changing
the initial mode in the command stream.
Foz-DB Navi48:
Totals from 14 (0.02% of 82405) affected shaders:
Latency: 79417 -> 79438 (+0.03%)
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38815 >
2026-01-20 14:48:22 +00:00
Georg Lehmann
7212a75c5e
aco/insert_fp_mode: exclude some instructions that will never round
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38815 >
2026-01-20 14:48:22 +00:00
Georg Lehmann
d6356191b9
aco: add fma_mix opcodes with rtz fp16 rounding
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38815 >
2026-01-20 14:48:22 +00:00
Rhys Perry
f0f53e624c
aco/tests: remove vcc definitions from p_call
...
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
The version of instruction selection that got merged doesn't have vcc
definitions, so this shouldn't either.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39390 >
2026-01-20 13:33:16 +00:00
Rhys Perry
adf5c7cba4
aco: remove dead p_call code in live_var_analysis
...
The version of instruction selection that got merged doesn't have vcc
definitions.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39390 >
2026-01-20 13:33:16 +00:00
Rhys Perry
ba798120c6
aco/ra: split blocking vectors if needed when handling fixed operands
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39390 >
2026-01-20 13:33:16 +00:00
Rhys Perry
5ebefceb42
aco/ra: move split_blocking_vectors higher
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39390 >
2026-01-20 13:33:16 +00:00
Rhys Perry
24fe4a5b58
aco/ra: copy precolor affinities to p_create_vector/p_split_vector
...
fossil-db (navi31):
Totals from 7 (0.01% of 84369) affected shaders:
Instrs: 2742 -> 2704 (-1.39%); split: -1.82%, +0.44%
CodeSize: 15300 -> 15052 (-1.62%); split: -1.93%, +0.31%
VGPRs: 516 -> 504 (-2.33%)
Latency: 12478 -> 12504 (+0.21%); split: -0.24%, +0.45%
InvThroughput: 2350 -> 2300 (-2.13%)
Copies: 350 -> 272 (-22.29%)
VALU: 1626 -> 1592 (-2.09%)
VOPD: 280 -> 236 (-15.71%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39387 >
2026-01-20 10:53:18 +00:00
Rhys Perry
893e9d7595
aco: don't spill no-op copies of input parameters in preserved registers
...
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:53 +00:00
Rhys Perry
8a3cc7200e
aco/sched: don't use previously unused preserved registers
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:53 +00:00
Rhys Perry
da1dca28a7
aco/ra: consider already-used preserved registers to be free
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:53 +00:00
Rhys Perry
103160ff2f
aco/ra: prefer clobbered registers in get_reg_specified()
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:53 +00:00
Rhys Perry
2b20d568e0
aco/ra: prefer clobbered registers in callees
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:53 +00:00
Rhys Perry
21b1118a08
aco/ra: refactor get_reg_impl and get_regs_for_copies using tuples
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:52 +00:00
Rhys Perry
61919b93fa
aco/ra: always abort loop in get_regs_for_copies() if candidate is worse
...
fossil-db (navi31):
Totals from 2 (0.00% of 79825) affected shaders:
Instrs: 58162 -> 58178 (+0.03%)
CodeSize: 299132 -> 299212 (+0.03%)
Latency: 72864 -> 72861 (-0.00%)
VClause: 977 -> 984 (+0.72%)
Copies: 7433 -> 7428 (-0.07%)
Branches: 1602 -> 1603 (+0.06%)
VALU: 37100 -> 37095 (-0.01%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:51 +00:00
Rhys Perry
137f71895d
aco/ra: always prefer earlier regs in get_reg_impl() if costs are the same
...
Before, this would have updated best_win if it encounters an assignment
with the same num_moves and num_vars but is aligned, even if the previous
best assignment was also aligned.
fossil-db (navi31):
Totals from 7693 (9.64% of 79825) affected shaders:
Instrs: 9767559 -> 9766370 (-0.01%); split: -0.11%, +0.09%
CodeSize: 51008244 -> 51002396 (-0.01%); split: -0.09%, +0.08%
VGPRs: 497224 -> 497212 (-0.00%); split: -0.00%, +0.00%
Latency: 80435654 -> 80373798 (-0.08%); split: -0.20%, +0.12%
InvThroughput: 12426170 -> 12414490 (-0.09%); split: -0.15%, +0.06%
VClause: 217725 -> 217438 (-0.13%); split: -0.59%, +0.46%
SClause: 230555 -> 230551 (-0.00%); split: -0.00%, +0.00%
Copies: 758396 -> 757988 (-0.05%); split: -0.81%, +0.75%
Branches: 200669 -> 200662 (-0.00%); split: -0.01%, +0.01%
VALU: 6040415 -> 6039979 (-0.01%); split: -0.10%, +0.09%
SALU: 960327 -> 960671 (+0.04%); split: -0.03%, +0.07%
VOPD: 426 -> 458 (+7.51%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:50 +00:00
Rhys Perry
0a744ba735
aco: increase max_reg_demand to help avoid preserved VGPRs
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:50 +00:00
Rhys Perry
bf56f1efd3
aco: move update(fixed_reg_demand) into update_vgpr_sgpr_demand
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:50 +00:00
Rhys Perry
a932bc40d3
aco/ra: don't use update_vgpr_sgpr_demand in increase_register_file
...
In a future commit, update_vgpr_sgpr_demand() should only be given the
register demand calculated from the number of live temporaries, not the
maximum register used.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:49 +00:00
Rhys Perry
6e618ac327
aco: don't try to preserve SCC in callees
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:48 +00:00
Rhys Perry
88ee78c6b7
aco: track number of post-RA spilled vgprs/sgprs
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:47 +00:00
Rhys Perry
d2a9122cfa
aco/tests: add function call regalloc tests
...
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
aa09335485
aco/ra: emit p_split_vector after p_parallelcopy
...
This fixes situations where the variable being split is a parallelcopy
definition.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
7caa0ab897
aco/ra: fix split_blocking_vectors with some subdword vectors
...
For example, v6b at byte=2 should be split into v2b,v1 instead of v1,v2b.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
667771950d
aco/ra: update register file when recreating blocking vectors
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
5cf991ecfb
aco/ra: use original name for blocking vectors rename
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
829e3a6a7d
aco/ra: fix operands when recreating blocking vectors
...
It is likely that these have been moved.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
6e6e40ab5c
aco/ra: discard tmp_file after get_regs_for_copies fails
...
get_regs_for_copies() might change tmp_file, so we need to create a new
one.
Recursively handle_call() with an modified register_file instead of
repeating get_regs_for_copies().
This also updates register_file for the caller.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
159154a741
aco/ra: remove dead code in split_blocking_vectors
...
collect_vars_from_bitset() from the caller should have already cleared
these.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
89d06f7db5
aco: add RA validation for p_call
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
d0801c9eb7
aco: always run RA validation during tests
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
bf30a57440
aco/ra: omit renaming when necessary when moving copy definitions
...
This should resolve the FIXME.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:41 +00:00
Rhys Perry
8dcfce3587
aco/ra: refactor update_renames slightly
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:39 +00:00
Emma Anholt
ed8676dc28
nir: Rename the unit_test_*_amd intrinics to be un-vendored.
...
We'll reuse these from the nir_opt_algebraic_pattern_test.
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076 >
2026-01-15 19:09:37 +00:00
Natalie Vock
cc81c7de23
nir,aco: Clean up useless lowering of sbt_base_amd
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:07 +00:00
Natalie Vock
0a1911b220
radv,aco: Use function call structure for RT programs
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:07 +00:00
Natalie Vock
c5d796c902
radv/rt: Use function call structure in NIR lowering
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:06 +00:00
Natalie Vock
06c2e90e35
aco: Note if a parameter needs to be explicitly preserved
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Natalie Vock
2bf403ccec
aco: Remove bypass_reg_preservation
...
Not needed anymore
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Natalie Vock
8525993d02
aco/ra: Move register preservation logic in last block to p_return
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Natalie Vock
eb86ae3b8c
aco/spill_preserved: Fix spilled VGPR overflow handling
...
We modified the spill offset inside the function, which made subsequent
calls to emit_vgpr_spills_reloads reload from the wrong offset.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Rhys Perry
7a09e4a740
aco: use correct addition opcodes in gfx6-8 RT prolog
...
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: 60dd9d797e ("aco: Swizzle ray launch IDs in the RT prolog")
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39232 >
2026-01-14 11:23:42 +00:00