Georg Lehmann
6b9d28ab9b
aco/insert_fp_mode: insert fp mode in reverse
...
This allows us to skip the mode set by changing
the initial mode in the command stream.
Foz-DB Navi48:
Totals from 14 (0.02% of 82405) affected shaders:
Latency: 79417 -> 79438 (+0.03%)
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38815 >
2026-01-20 14:48:22 +00:00
Georg Lehmann
7212a75c5e
aco/insert_fp_mode: exclude some instructions that will never round
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38815 >
2026-01-20 14:48:22 +00:00
Georg Lehmann
d6356191b9
aco: add fma_mix opcodes with rtz fp16 rounding
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38815 >
2026-01-20 14:48:22 +00:00
Rhys Perry
f0f53e624c
aco/tests: remove vcc definitions from p_call
...
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
The version of instruction selection that got merged doesn't have vcc
definitions, so this shouldn't either.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39390 >
2026-01-20 13:33:16 +00:00
Rhys Perry
adf5c7cba4
aco: remove dead p_call code in live_var_analysis
...
The version of instruction selection that got merged doesn't have vcc
definitions.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39390 >
2026-01-20 13:33:16 +00:00
Rhys Perry
ba798120c6
aco/ra: split blocking vectors if needed when handling fixed operands
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39390 >
2026-01-20 13:33:16 +00:00
Rhys Perry
5ebefceb42
aco/ra: move split_blocking_vectors higher
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39390 >
2026-01-20 13:33:16 +00:00
Rhys Perry
24fe4a5b58
aco/ra: copy precolor affinities to p_create_vector/p_split_vector
...
fossil-db (navi31):
Totals from 7 (0.01% of 84369) affected shaders:
Instrs: 2742 -> 2704 (-1.39%); split: -1.82%, +0.44%
CodeSize: 15300 -> 15052 (-1.62%); split: -1.93%, +0.31%
VGPRs: 516 -> 504 (-2.33%)
Latency: 12478 -> 12504 (+0.21%); split: -0.24%, +0.45%
InvThroughput: 2350 -> 2300 (-2.13%)
Copies: 350 -> 272 (-22.29%)
VALU: 1626 -> 1592 (-2.09%)
VOPD: 280 -> 236 (-15.71%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39387 >
2026-01-20 10:53:18 +00:00
Rhys Perry
893e9d7595
aco: don't spill no-op copies of input parameters in preserved registers
...
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:53 +00:00
Rhys Perry
8a3cc7200e
aco/sched: don't use previously unused preserved registers
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:53 +00:00
Rhys Perry
da1dca28a7
aco/ra: consider already-used preserved registers to be free
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:53 +00:00
Rhys Perry
103160ff2f
aco/ra: prefer clobbered registers in get_reg_specified()
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:53 +00:00
Rhys Perry
2b20d568e0
aco/ra: prefer clobbered registers in callees
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:53 +00:00
Rhys Perry
21b1118a08
aco/ra: refactor get_reg_impl and get_regs_for_copies using tuples
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:52 +00:00
Rhys Perry
61919b93fa
aco/ra: always abort loop in get_regs_for_copies() if candidate is worse
...
fossil-db (navi31):
Totals from 2 (0.00% of 79825) affected shaders:
Instrs: 58162 -> 58178 (+0.03%)
CodeSize: 299132 -> 299212 (+0.03%)
Latency: 72864 -> 72861 (-0.00%)
VClause: 977 -> 984 (+0.72%)
Copies: 7433 -> 7428 (-0.07%)
Branches: 1602 -> 1603 (+0.06%)
VALU: 37100 -> 37095 (-0.01%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:51 +00:00
Rhys Perry
137f71895d
aco/ra: always prefer earlier regs in get_reg_impl() if costs are the same
...
Before, this would have updated best_win if it encounters an assignment
with the same num_moves and num_vars but is aligned, even if the previous
best assignment was also aligned.
fossil-db (navi31):
Totals from 7693 (9.64% of 79825) affected shaders:
Instrs: 9767559 -> 9766370 (-0.01%); split: -0.11%, +0.09%
CodeSize: 51008244 -> 51002396 (-0.01%); split: -0.09%, +0.08%
VGPRs: 497224 -> 497212 (-0.00%); split: -0.00%, +0.00%
Latency: 80435654 -> 80373798 (-0.08%); split: -0.20%, +0.12%
InvThroughput: 12426170 -> 12414490 (-0.09%); split: -0.15%, +0.06%
VClause: 217725 -> 217438 (-0.13%); split: -0.59%, +0.46%
SClause: 230555 -> 230551 (-0.00%); split: -0.00%, +0.00%
Copies: 758396 -> 757988 (-0.05%); split: -0.81%, +0.75%
Branches: 200669 -> 200662 (-0.00%); split: -0.01%, +0.01%
VALU: 6040415 -> 6039979 (-0.01%); split: -0.10%, +0.09%
SALU: 960327 -> 960671 (+0.04%); split: -0.03%, +0.07%
VOPD: 426 -> 458 (+7.51%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:50 +00:00
Rhys Perry
0a744ba735
aco: increase max_reg_demand to help avoid preserved VGPRs
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:50 +00:00
Rhys Perry
bf56f1efd3
aco: move update(fixed_reg_demand) into update_vgpr_sgpr_demand
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:50 +00:00
Rhys Perry
a932bc40d3
aco/ra: don't use update_vgpr_sgpr_demand in increase_register_file
...
In a future commit, update_vgpr_sgpr_demand() should only be given the
register demand calculated from the number of live temporaries, not the
maximum register used.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:49 +00:00
Rhys Perry
6e618ac327
aco: don't try to preserve SCC in callees
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:48 +00:00
Rhys Perry
88ee78c6b7
aco: track number of post-RA spilled vgprs/sgprs
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:47 +00:00
Rhys Perry
d2a9122cfa
aco/tests: add function call regalloc tests
...
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
aa09335485
aco/ra: emit p_split_vector after p_parallelcopy
...
This fixes situations where the variable being split is a parallelcopy
definition.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
7caa0ab897
aco/ra: fix split_blocking_vectors with some subdword vectors
...
For example, v6b at byte=2 should be split into v2b,v1 instead of v1,v2b.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
667771950d
aco/ra: update register file when recreating blocking vectors
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
5cf991ecfb
aco/ra: use original name for blocking vectors rename
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
829e3a6a7d
aco/ra: fix operands when recreating blocking vectors
...
It is likely that these have been moved.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
6e6e40ab5c
aco/ra: discard tmp_file after get_regs_for_copies fails
...
get_regs_for_copies() might change tmp_file, so we need to create a new
one.
Recursively handle_call() with an modified register_file instead of
repeating get_regs_for_copies().
This also updates register_file for the caller.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
159154a741
aco/ra: remove dead code in split_blocking_vectors
...
collect_vars_from_bitset() from the caller should have already cleared
these.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
89d06f7db5
aco: add RA validation for p_call
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
d0801c9eb7
aco: always run RA validation during tests
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
bf30a57440
aco/ra: omit renaming when necessary when moving copy definitions
...
This should resolve the FIXME.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:41 +00:00
Rhys Perry
8dcfce3587
aco/ra: refactor update_renames slightly
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:39 +00:00
Emma Anholt
ed8676dc28
nir: Rename the unit_test_*_amd intrinics to be un-vendored.
...
We'll reuse these from the nir_opt_algebraic_pattern_test.
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076 >
2026-01-15 19:09:37 +00:00
Natalie Vock
cc81c7de23
nir,aco: Clean up useless lowering of sbt_base_amd
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:07 +00:00
Natalie Vock
0a1911b220
radv,aco: Use function call structure for RT programs
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:07 +00:00
Natalie Vock
c5d796c902
radv/rt: Use function call structure in NIR lowering
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:06 +00:00
Natalie Vock
06c2e90e35
aco: Note if a parameter needs to be explicitly preserved
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Natalie Vock
2bf403ccec
aco: Remove bypass_reg_preservation
...
Not needed anymore
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Natalie Vock
8525993d02
aco/ra: Move register preservation logic in last block to p_return
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Natalie Vock
eb86ae3b8c
aco/spill_preserved: Fix spilled VGPR overflow handling
...
We modified the spill offset inside the function, which made subsequent
calls to emit_vgpr_spills_reloads reload from the wrong offset.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Rhys Perry
7a09e4a740
aco: use correct addition opcodes in gfx6-8 RT prolog
...
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: 60dd9d797e ("aco: Swizzle ray launch IDs in the RT prolog")
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39232 >
2026-01-14 11:23:42 +00:00
Rhys Perry
da728d5a1a
aco: micro-optimize ray launch ID swizzling
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39232 >
2026-01-14 11:23:42 +00:00
Natalie Vock
473cf6046a
aco/spill_preserved: Preserve linear VGPRs even if they aren't p_spill operands
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
1ef2691221
aco/spill: Fix preserved reload operand update
...
p_logical_end is actually after p_reload_preserved, so this didn't do
anything.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
548062f10e
aco/insert_waitcnt: Don't determine linearity by reg number
...
VGPRs can be linear too, and RT function calls will add VMEM
instructions acting on linear VGPRs. Using the linear VGPR in a block
with only linear preds will cause the pass to incorrectly skip inserting
s_waitcnt.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
7c12603933
aco/lower_to_hw_instr: Preserve linearity of lowered linear VGPRs
...
So subsequent passes like waitcnt insertion can know these are linear.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
0d93e8ce54
aco: Don't insert p_reload_preserved in loops
...
This can't work.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
c816f699b2
aco/spill_preserved: Only reload linear VGPRs at end
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
897c95c37e
aco: Include arbitrarily fixed registers in max_reg_demand
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00