Rhys Perry
bf56f1efd3
aco: move update(fixed_reg_demand) into update_vgpr_sgpr_demand
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:50 +00:00
Rhys Perry
a932bc40d3
aco/ra: don't use update_vgpr_sgpr_demand in increase_register_file
...
In a future commit, update_vgpr_sgpr_demand() should only be given the
register demand calculated from the number of live temporaries, not the
maximum register used.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:49 +00:00
Rhys Perry
6e618ac327
aco: don't try to preserve SCC in callees
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:48 +00:00
Rhys Perry
88ee78c6b7
aco: track number of post-RA spilled vgprs/sgprs
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679 >
2026-01-19 09:17:47 +00:00
Rhys Perry
d2a9122cfa
aco/tests: add function call regalloc tests
...
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
aa09335485
aco/ra: emit p_split_vector after p_parallelcopy
...
This fixes situations where the variable being split is a parallelcopy
definition.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
7caa0ab897
aco/ra: fix split_blocking_vectors with some subdword vectors
...
For example, v6b at byte=2 should be split into v2b,v1 instead of v1,v2b.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
667771950d
aco/ra: update register file when recreating blocking vectors
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
5cf991ecfb
aco/ra: use original name for blocking vectors rename
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
829e3a6a7d
aco/ra: fix operands when recreating blocking vectors
...
It is likely that these have been moved.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
6e6e40ab5c
aco/ra: discard tmp_file after get_regs_for_copies fails
...
get_regs_for_copies() might change tmp_file, so we need to create a new
one.
Recursively handle_call() with an modified register_file instead of
repeating get_regs_for_copies().
This also updates register_file for the caller.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
159154a741
aco/ra: remove dead code in split_blocking_vectors
...
collect_vars_from_bitset() from the caller should have already cleared
these.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
89d06f7db5
aco: add RA validation for p_call
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
d0801c9eb7
aco: always run RA validation during tests
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:43 +00:00
Rhys Perry
bf30a57440
aco/ra: omit renaming when necessary when moving copy definitions
...
This should resolve the FIXME.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:41 +00:00
Rhys Perry
8dcfce3587
aco/ra: refactor update_renames slightly
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811 >
2026-01-16 10:05:39 +00:00
Emma Anholt
ed8676dc28
nir: Rename the unit_test_*_amd intrinics to be un-vendored.
...
We'll reuse these from the nir_opt_algebraic_pattern_test.
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076 >
2026-01-15 19:09:37 +00:00
Natalie Vock
cc81c7de23
nir,aco: Clean up useless lowering of sbt_base_amd
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:07 +00:00
Natalie Vock
0a1911b220
radv,aco: Use function call structure for RT programs
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:07 +00:00
Natalie Vock
c5d796c902
radv/rt: Use function call structure in NIR lowering
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:06 +00:00
Natalie Vock
06c2e90e35
aco: Note if a parameter needs to be explicitly preserved
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Natalie Vock
2bf403ccec
aco: Remove bypass_reg_preservation
...
Not needed anymore
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Natalie Vock
8525993d02
aco/ra: Move register preservation logic in last block to p_return
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Natalie Vock
eb86ae3b8c
aco/spill_preserved: Fix spilled VGPR overflow handling
...
We modified the spill offset inside the function, which made subsequent
calls to emit_vgpr_spills_reloads reload from the wrong offset.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580 >
2026-01-14 14:19:05 +00:00
Rhys Perry
7a09e4a740
aco: use correct addition opcodes in gfx6-8 RT prolog
...
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: 60dd9d797e ("aco: Swizzle ray launch IDs in the RT prolog")
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39232 >
2026-01-14 11:23:42 +00:00
Rhys Perry
da728d5a1a
aco: micro-optimize ray launch ID swizzling
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39232 >
2026-01-14 11:23:42 +00:00
Natalie Vock
473cf6046a
aco/spill_preserved: Preserve linear VGPRs even if they aren't p_spill operands
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
1ef2691221
aco/spill: Fix preserved reload operand update
...
p_logical_end is actually after p_reload_preserved, so this didn't do
anything.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
548062f10e
aco/insert_waitcnt: Don't determine linearity by reg number
...
VGPRs can be linear too, and RT function calls will add VMEM
instructions acting on linear VGPRs. Using the linear VGPR in a block
with only linear preds will cause the pass to incorrectly skip inserting
s_waitcnt.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
7c12603933
aco/lower_to_hw_instr: Preserve linearity of lowered linear VGPRs
...
So subsequent passes like waitcnt insertion can know these are linear.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
0d93e8ce54
aco: Don't insert p_reload_preserved in loops
...
This can't work.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
c816f699b2
aco/spill_preserved: Only reload linear VGPRs at end
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Natalie Vock
897c95c37e
aco: Include arbitrarily fixed registers in max_reg_demand
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39157 >
2026-01-12 21:46:50 +00:00
Georg Lehmann
daf235c607
aco/tests: don't destroy vk_device if it was never created
...
Happens if you only run one test that doesn't need a vk_device.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39268 >
2026-01-12 16:16:54 +00:00
Georg Lehmann
fad95030a7
aco/tests: test VALUMaskWriteHazard with v_cmpx
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39252 >
2026-01-12 15:48:39 +00:00
Georg Lehmann
1d85552745
aco/tests: test VALUReadSGPRHazard with v_cmpx
...
To avoid regressing this in a future rework.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39252 >
2026-01-12 15:48:39 +00:00
Georg Lehmann
3e10ab34e1
aco/insert_NOPs: explicitly wait for sa_sdst to resolve SALU -> VALU hazards
...
The assumption that these waits are not required has been proven incorrect
in at least some cases.
Totals from 190 (0.24% of 79825) affected shaders: (Navi31)
Instrs: 499718 -> 500491 (+0.15%)
CodeSize: 2658228 -> 2661916 (+0.14%)
Latency: 5964632 -> 5965453 (+0.01%); split: -0.00%, +0.01%
InvThroughput: 794221 -> 794289 (+0.01%)
Totals from 17093 (21.41% of 79839) affected shaders: (Navi48)
Instrs: 22805214 -> 22854313 (+0.22%)
CodeSize: 121240428 -> 121432904 (+0.16%); split: -0.00%, +0.16%
Latency: 166500300 -> 166530529 (+0.02%); split: -0.00%, +0.02%
InvThroughput: 28770053 -> 28772870 (+0.01%); split: -0.00%, +0.01%
Fixes: 018f45f981 ("aco/insert_NOPs: remove redundant VALUReadSGPRHazard waits")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/14516
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39252 >
2026-01-12 15:48:38 +00:00
Konstantin Seurer
39d58a55a7
aco: Add support to f2f16 with rtpi/rtni
...
Those rounding modes are needed when computing 16-bit bounding boxes
since the bounding box must not get smaller.
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37883 >
2026-01-10 11:34:12 +01:00
Natalie Vock
60dd9d797e
aco: Swizzle ray launch IDs in the RT prolog
...
This converts from 1D workgroups to 2D ray launch IDs entirely via
shader ALU, including handling partial/cut-off workgroups optimally.
Doing this entirely in-shader means it Just Works(TM) with indirect
dispatches as well. Previous approaches manipulating various things on
CPU depending on the dispatch size couldn't handle indirect dispatches.
The swizzle implemented here also swizzles with a recursive Z-order
pattern, which should be a little more optimal than arranging
invocations linearly within the wave.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39142 >
2026-01-08 19:49:55 +01:00
Natalie Vock
1f6ac3fa93
radv/rt,aco: Always dispatch 1D workgroups for RT
...
We will swizzle the workgroups ourselves in the next commit.
Removes the need for 1D dispatch workarounds.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39142 >
2026-01-08 19:49:54 +01:00
Georg Lehmann
eb4737a1dd
nir: add nir_alu_instr_is_exact helper
...
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39103 >
2026-01-07 09:40:57 +00:00
Daniel Schürmann
2d0d5fc104
aco/validate: validate constant bus limit after register allocation based on PhysReg
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39107 >
2026-01-05 14:54:00 +00:00
Daniel Schürmann
eb16f701a6
aco/tests: Add new test to pack 2x16 SGPRs into VGPR
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39107 >
2026-01-05 14:54:00 +00:00
Daniel Schürmann
61c1ec541d
aco/tests: Add test for subdword extraction from SGPR
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39107 >
2026-01-05 14:54:00 +00:00
Daniel Schürmann
0674c9d30e
aco/validate: Validate correct RegisterClasses after lowering to HW instructions
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39107 >
2026-01-05 14:53:59 +00:00
Daniel Schürmann
b087bf2fbf
aco/lower_to_hw: Fix SGPR Operand RegClasses for pack_2x16
...
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39107 >
2026-01-05 14:53:59 +00:00
Daniel Schürmann
9f5996ae8a
aco/lower_to_hw: Don't use 2 SGPR operands before GFX10 in a single VOP3 instruction in do_pack_2x16()
...
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39107 >
2026-01-05 14:53:58 +00:00
Daniel Schürmann
d8481fd7cc
aco/lower_to_hw: Fix SGPR Operand RegClasses of subdword copies
...
Extracting from an SGPR could cause a wrong RegClass on
the operand which could later lead to selecting VOPD
instructions which falsely operate on the corresponding
VGPR.
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39107 >
2026-01-05 14:53:58 +00:00
Georg Lehmann
0c42141299
aco: allow opsel for last v_alignbyte/bit operand
...
For completeness' sake.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13285
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39061 >
2025-12-31 08:58:24 +00:00
Daniel Schürmann
7b1f6fa6fc
aco: remove radeon_family from aco::Program
...
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38701 >
2025-12-22 07:34:48 +00:00