fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2026-05-18 07:18:06 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
205267 commits 66 branches 935 tags 942 MiB
Commit graph

3680 commits

Author SHA1 Message Date
Georg Lehmann
345bf8a2f2 aco/optimizer: remove label_vop3p 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
 2025-05-09 14:23:23 +00:00
Georg Lehmann
6667ee66d5 aco/optimizer: remove label_vopc 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
 2025-05-09 14:23:23 +00:00
Georg Lehmann
f5a5905e37 aco: support nir_op_bfdot2_bfadd 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
 2025-05-09 11:20:26 +00:00
Georg Lehmann
5ca98bf99e aco: support bf16 wmma 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
 2025-05-09 11:20:25 +00:00
Georg Lehmann
e8f5c335ff radv,aco,nir: keep the A and B base type for cmat_muladd_amd 
With bfloat16, and the two fp8 formats in the future, using just the bit size
to identify the types is no longer possible.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
 2025-05-09 11:20:25 +00:00
Samuel Pitoiset
dae45adc9d aco: adjust an assertion in select_trap_handler_shader()
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
Details
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Details 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34840>
 2025-05-09 09:04:57 +00:00
Samuel Pitoiset
ae6d3df139 radv,aco: dump more SQ_WAVE registers from the trap handler on GFX12 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34840>
 2025-05-09 09:04:57 +00:00
Samuel Pitoiset
50a01a6559 radv: fix save/restore SCC in the trap handler on GFX12 
SQ_WAVE_STATE_PRIV now contains SCC instead of SQ_WAVE_STATUS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34840>
 2025-05-09 09:04:57 +00:00
Rhys Perry
18a53230eb aco: don't check dst_bitsize in apply_load_extract 
I don't think this is necessary.

fossil-db (gfx1201):
Totals from 12 (0.02% of 79377) affected shaders:
Instrs: 73041 -> 72669 (-0.51%); split: -0.51%, +0.00%
CodeSize: 417376 -> 413852 (-0.84%); split: -0.85%, +0.00%
Latency: 1301862 -> 1301533 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 599874 -> 599723 (-0.03%)
VClause: 1344 -> 1346 (+0.15%)
Copies: 15855 -> 15832 (-0.15%); split: -0.37%, +0.23%
VALU: 42138 -> 41883 (-0.61%); split: -0.61%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34162>
 2025-05-08 13:30:50 +00:00
Rhys Perry
eb95f7cc0e aco: support sign extension in apply_load_extract 
fossil-db (gfx1201):
Totals from 10 (0.01% of 79377) affected shaders:
Instrs: 28954 -> 28938 (-0.06%)
CodeSize: 164552 -> 164472 (-0.05%)
Latency: 1249341 -> 1247037 (-0.18%)
InvThroughput: 297077 -> 296618 (-0.15%)
VALU: 15951 -> 15941 (-0.06%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34162>
 2025-05-08 13:30:50 +00:00
Rhys Perry
0de0fd38b4 aco: support more opcodes in apply_ds_extract 
fossil-db (gfx1201):
Totals from 320 (0.40% of 79377) affected shaders:
Instrs: 3439754 -> 3432384 (-0.21%)
CodeSize: 18008696 -> 17973180 (-0.20%); split: -0.20%, +0.00%
VGPRs: 16016 -> 15404 (-3.82%)
Latency: 20246168 -> 20295740 (+0.24%); split: -0.08%, +0.33%
InvThroughput: 4462916 -> 4478546 (+0.35%); split: -0.08%, +0.43%
VClause: 87123 -> 87099 (-0.03%)
Copies: 261779 -> 261948 (+0.06%); split: -0.05%, +0.12%
Branches: 94611 -> 94601 (-0.01%); split: -0.01%, +0.00%
VALU: 1870695 -> 1865738 (-0.26%)
SALU: 488351 -> 487557 (-0.16%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34162>
 2025-05-08 13:30:50 +00:00
Rhys Perry
4fa1c92862 aco/gfx12: allow 8/16-bit smem loads 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34162>
 2025-05-08 13:30:50 +00:00
Rhys Perry
75efc218f5 aco: support 8/16-bit loads in smem_combine() 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34162>
 2025-05-08 13:30:50 +00:00
Rhys Perry
13b0131edc aco/gfx12: select dwordx3 smem loads 
fossil-db (gfx1201):
Totals from 47814 (60.24% of 79377) affected shaders:
MaxWaves: 1436871 -> 1436855 (-0.00%); split: +0.00%, -0.00%
Instrs: 36653621 -> 36649461 (-0.01%); split: -0.07%, +0.06%
CodeSize: 194102884 -> 194076060 (-0.01%); split: -0.06%, +0.04%
VGPRs: 2267944 -> 2269648 (+0.08%); split: -0.01%, +0.08%
SpillSGPRs: 8301 -> 8295 (-0.07%); split: -0.08%, +0.01%
Latency: 249627561 -> 249631829 (+0.00%); split: -0.04%, +0.04%
InvThroughput: 40004042 -> 40003575 (-0.00%); split: -0.02%, +0.02%
VClause: 680488 -> 680429 (-0.01%); split: -0.08%, +0.07%
SClause: 1062835 -> 1066206 (+0.32%); split: -0.20%, +0.52%
Copies: 2393981 -> 2393607 (-0.02%); split: -0.23%, +0.22%
Branches: 728117 -> 728113 (-0.00%); split: -0.00%, +0.00%
VALU: 20358269 -> 20358585 (+0.00%); split: -0.01%, +0.01%
SALU: 4737317 -> 4736411 (-0.02%); split: -0.07%, +0.06%
SMEM: 1712349 -> 1710075 (-0.13%); split: -0.13%, +0.00%
VOPD: 5808 -> 5813 (+0.09%); split: +0.12%, -0.03%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34162>
 2025-05-08 13:30:50 +00:00
Rhys Perry
90a5c93ea5 aco: prepare for dwordx3 smem loads 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34162>
 2025-05-08 13:30:50 +00:00
Rhys Perry
208d62430f aco/gfx12: use s_load_dwordx3 to load ray launch sizes 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34162>
 2025-05-08 13:30:49 +00:00
Rhys Perry
cbd718506b aco: add smem opcode helper 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34162>
 2025-05-08 13:30:49 +00:00
Marek Olšák
b960137ebf aco: remove unused aco_shader_info::tcs_offchip_layout 
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34863>
 2025-05-08 02:54:13 +00:00
Rhys Perry
9ca71b52aa aco: swap the correct v_mov_b32 if there are two of them 
Previously, this function tried to swap the instruction which is not
v_mov_b32, so that it doesn't introduce any new OPY-only instructions. If
both were v_mov_b32, it swapped Y. Since this makes Y opy-only, this can't
be done if X is also opy-only.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: 408fa33c09 ("aco/gfx12: don't use second VALU for VOPD's OPX if there is a WaR")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13101
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34841>
 2025-05-06 21:04:28 +00:00
Rhys Perry
752f5f317e aco: replace max_const_offset_plus_one with max_const_offset
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
Details
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Details 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34766>
 2025-05-01 09:19:02 +00:00
Rhys Perry
a85ebe16b3 aco: fix max_const_offset_plus_one overflow 
smem_offset_max is UINT32_MAX on GFX7 and setting
max_const_offset_plus_one to 0 causes divisions by zero later.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: c26851b80b ("aco: increase max_const_offset_plus_one for SMEM load_global")
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Tested-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34766>
 2025-05-01 09:19:02 +00:00
Rhys Perry
6338ed44c5 aco/gfx12: increase maximum vbuffer offset 
fossil-db (gfx1201):
Totals from 301 (0.38% of 79377) affected shaders:
Instrs: 2734478 -> 2728816 (-0.21%); split: -0.21%, +0.00%
CodeSize: 14347476 -> 14306568 (-0.29%)
Latency: 15508055 -> 15502202 (-0.04%); split: -0.04%, +0.00%
InvThroughput: 2846419 -> 2842387 (-0.14%); split: -0.14%, +0.00%
VClause: 68286 -> 68101 (-0.27%); split: -0.30%, +0.03%
SClause: 49487 -> 49500 (+0.03%)
Copies: 207179 -> 206093 (-0.52%); split: -0.57%, +0.04%
Branches: 72941 -> 72942 (+0.00%); split: -0.00%, +0.00%
VALU: 1549156 -> 1544727 (-0.29%); split: -0.29%, +0.00%
SALU: 339620 -> 338989 (-0.19%); split: -0.19%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34730>
 2025-04-29 17:44:41 +00:00
Rhys Perry
d987d5e341 aco/gfx12: increase maximum global/scratch offset 
fossil-db (gfx1201):
Totals from 29 (0.04% of 79377) affected shaders:
Instrs: 1439082 -> 1439070 (-0.00%)
CodeSize: 7641688 -> 7641608 (-0.00%)
Latency: 9836296 -> 9836280 (-0.00%)
VALU: 799504 -> 799496 (-0.00%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34730>
 2025-04-29 17:44:41 +00:00
Rhys Perry
02d193f058 aco/gfx12: increase maximum smem offset 
fossil-db (gfx1201):
Totals from 55 (0.07% of 79377) affected shaders:
Instrs: 3203775 -> 3200809 (-0.09%)
CodeSize: 16817140 -> 16813440 (-0.02%); split: -0.04%, +0.02%
Latency: 17838315 -> 17829658 (-0.05%)
InvThroughput: 3352905 -> 3351689 (-0.04%)
SClause: 57377 -> 57273 (-0.18%)
Copies: 231006 -> 230941 (-0.03%)
SALU: 436900 -> 435234 (-0.38%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34730>
 2025-04-29 17:44:41 +00:00
Rhys Perry
c26851b80b aco: increase max_const_offset_plus_one for SMEM load_global 
fossil-db (gfx1201):
Totals from 1115 (1.40% of 79377) affected shaders:
Instrs: 1473805 -> 1467571 (-0.42%); split: -0.43%, +0.01%
CodeSize: 7852972 -> 7819656 (-0.42%); split: -0.44%, +0.02%
SpillSGPRs: 1632 -> 1460 (-10.54%); split: -11.27%, +0.74%
Latency: 11975762 -> 11971915 (-0.03%); split: -0.05%, +0.02%
InvThroughput: 2496961 -> 2496448 (-0.02%); split: -0.03%, +0.01%
VClause: 25213 -> 25218 (+0.02%); split: -0.00%, +0.02%
SClause: 28822 -> 28565 (-0.89%); split: -1.41%, +0.52%
Copies: 106377 -> 105715 (-0.62%); split: -1.23%, +0.61%
Branches: 27497 -> 27473 (-0.09%)
PreSGPRs: 52071 -> 51310 (-1.46%)
VALU: 871051 -> 870694 (-0.04%); split: -0.04%, +0.00%
SALU: 186090 -> 181811 (-2.30%); split: -2.32%, +0.02%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34730>
 2025-04-29 17:44:41 +00:00
Rhys Perry
f390893a64 aco/gfx12: use s_sub_u64 
fossil-db (gfx1201):
Totals from 2 (0.00% of 79377) affected shaders:
Instrs: 243999 -> 243993 (-0.00%)
CodeSize: 1288176 -> 1288152 (-0.00%)
Latency: 1894093 -> 1894091 (-0.00%)
InvThroughput: 378819 -> 378818 (-0.00%)
SALU: 33048 -> 33044 (-0.01%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34730>
 2025-04-29 17:44:41 +00:00
Rhys Perry
5b4813c4f0 aco/gfx12: use s_add_u64 
fossil-db (gfx1201):
Totals from 122 (0.15% of 79377) affected shaders:
Instrs: 3640138 -> 3637577 (-0.07%); split: -0.07%, +0.00%
CodeSize: 19133796 -> 19120080 (-0.07%); split: -0.08%, +0.01%
SpillSGPRs: 666 -> 650 (-2.40%)
SpillVGPRs: 2147 -> 2159 (+0.56%)
Scratch: 254208 -> 255232 (+0.40%)
Latency: 21529337 -> 21522317 (-0.03%); split: -0.04%, +0.00%
InvThroughput: 4048519 -> 4047233 (-0.03%); split: -0.03%, +0.00%
VClause: 90453 -> 90455 (+0.00%)
SClause: 67846 -> 67674 (-0.25%); split: -0.28%, +0.03%
Copies: 287449 -> 287476 (+0.01%); split: -0.04%, +0.05%
Branches: 104526 -> 104530 (+0.00%); split: -0.00%, +0.01%
PreSGPRs: 9795 -> 9723 (-0.74%); split: -0.78%, +0.04%
VALU: 2004219 -> 2003031 (-0.06%); split: -0.06%, +0.00%
SALU: 492651 -> 491737 (-0.19%); split: -0.19%, +0.00%
VMEM: 161317 -> 161341 (+0.01%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34730>
 2025-04-29 17:44:41 +00:00
Rhys Perry
20279c28c8 aco/tests: add pseudo-scalar transcendental and fallback path RA tests
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
Details
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Details 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34343>
 2025-04-29 15:15:11 +00:00
Rhys Perry
96e49b7904 aco/ra: add ra_test_policy::use_compact_relocate 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34343>
 2025-04-29 15:15:10 +00:00
Rhys Perry
3c1dbc1d9b aco/ra: cleanup compact_relocate_vars fallback path 
We don't need to duplicate these loops.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34343>
 2025-04-29 15:15:10 +00:00
Rhys Perry
a780345e01 aco: fix compact_relocate_vars fallback with scc/exec/m0 precolored regs 
This probably doesn't fix anything in practice. I don't think this path is
ever taken for SGPRs except for pseudo-scalar transcendental instructions,
and those don't have any precolored operands/definitions.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34343>
 2025-04-29 15:15:10 +00:00
Rhys Perry
f6581b41c4 aco/ra: don't require alignment for NPOT SGPR temporaries 
Aligning these can create situations where register allocation is
impossible. Only pseudo-instructions can use these, and they don't require
any alignment.

I'm not sure if these temporaries actually happen in practice. This
probably only affects the get_reg()'s compact_relocate_vars fallback path,
which doesn't usually happen for SGPRs.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34343>
 2025-04-29 15:15:10 +00:00
Rhys Perry
623230a6ef aco/ra: change sorting in compact_relocate_vars 
D16 MIMG or pseudo-scalar transcendental instructions might give lower
limits to the maximum register available for their definitions, so just
try to place them earlier.

This is also part of fixing compact_relocate_vars with aligned NPOT
def/killed-op space (the second part is the later commit which changes
get_stride()).

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34343>
 2025-04-29 15:15:10 +00:00
Rhys Perry
de896234d8 aco: improve spilling of clobbered operands
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
Details
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Details 
We can ignore live_changes for these.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34699>
 2025-04-28 10:04:43 +00:00
Rhys Perry
7fe84024cb aco: fix get_temp_reg_changes with clobbered operands 
The spiller might have tried to spill a live-through first or second
s_fmac_f32 operand, but this wouldn't have reduced the SGPRs if the third
operand wasn't killed

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13038
Fixes: d6cb45dbb0 ("aco/spill: Allow spilling live-through operands")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34699>
 2025-04-28 10:04:43 +00:00
Rhys Perry
3c021b79b4 aco/ra: use a correct stride for subdword get_reg_impl
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
Details
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Details 
fossil-db (gfx1201):
Totals from 3 (0.00% of 79377) affected shaders:
Instrs: 1312 -> 1308 (-0.30%)
CodeSize: 7112 -> 7096 (-0.22%)
Latency: 5381 -> 5382 (+0.02%)
InvThroughput: 753 -> 752 (-0.13%)
Copies: 69 -> 68 (-1.45%)
VALU: 836 -> 835 (-0.12%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34679>
 2025-04-25 14:43:41 +00:00
Rhys Perry
ae6d4f1195 aco/ra: update_renames() before add_subdword_definition() 
The register file tests here should be done after update_renames().

Normally, get_reg() wouldn't have to move anything to make space for a 1-3
byte definition. This was encountered with skip_optimistic_path=true and a
get_reg_impl() bug (fixed in a later commit) which caused suboptimal
register assignment.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34679>
 2025-04-25 14:43:41 +00:00
Rhys Perry
b03e071583 aco/gfx11: create waitcnt for workgroup vmem barriers
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
Details
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Details 
It seems this is necessary on GFX11.

Similar to 576a2e798c

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Backport-to: 25.0
Backport-to: 25.1
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34634>
 2025-04-25 10:41:52 +00:00
Georg Lehmann
65411350ac aco/insert_exec: disable empty quads when leaving divergent control, even if not top level 
We don't restore exec after uniform top level branches, so nothing disabled
empty quads after a demote in divergent control flow in a uniform branch.

Foz-DB Navi31:
Totals from 17 (0.02% of 79789) affected shaders:
Instrs: 34573 -> 34572 (-0.00%)
CodeSize: 186876 -> 186872 (-0.00%)
Latency: 324145 -> 324141 (-0.00%)
Copies: 1467 -> 1458 (-0.61%)
PreSGPRs: 802 -> 800 (-0.25%)
SALU: 2531 -> 2530 (-0.04%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34670>
 2025-04-24 15:43:09 +00:00
Rhys Perry
62e50de5d0 aco: use v_perm_b32 for byte swaps within a VGPR on gfx10 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34636>
 2025-04-23 18:23:18 +00:00
Rhys Perry
a43783fd76 aco: use v_perm_b32 for do_pack_2x16 on gfx10+ 
fossil-db (gfx1201);
Totals from 93 (0.12% of 79377) affected shaders:
Instrs: 373212 -> 372761 (-0.12%)
CodeSize: 2062752 -> 2063704 (+0.05%); split: -0.00%, +0.05%
Latency: 4172059 -> 4171993 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 1299144 -> 1299093 (-0.00%)
Copies: 51268 -> 50831 (-0.85%)
Branches: 10980 -> 10979 (-0.01%)
VALU: 220192 -> 219756 (-0.20%)
VOPD: 48 -> 47 (-2.08%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34636>
 2025-04-23 18:23:18 +00:00
Georg Lehmann
dd3e1190a2 aco/insert_exec: reset temporary when recreating wqm mask from exact mask 
The old, now incorrect temporary was still used for invert blocks and loop masks.

Foz-DB Navi31:
Totals from 379 (0.48% of 79789) affected shaders:
Instrs: 399471 -> 399897 (+0.11%); split: -0.00%, +0.11%
CodeSize: 2197292 -> 2198908 (+0.07%); split: -0.00%, +0.08%
Latency: 2500636 -> 2500895 (+0.01%); split: -0.00%, +0.01%
SClause: 7912 -> 7918 (+0.08%); split: -0.04%, +0.11%
Copies: 25687 -> 26068 (+1.48%); split: -0.04%, +1.53%
PreSGPRs: 15648 -> 15562 (-0.55%)
SALU: 35125 -> 35517 (+1.12%)

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12901
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13019
Fixes: b872ff6ef2 ("aco/insert_exec_mask: if applicable, use s_wqm to restore exec after divergent CF")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34659>
 2025-04-23 09:37:50 +00:00
Georg Lehmann
13f6be262a aco/insert_exec: only restore wqm mask after control flow if necessary 
The next commit will make this not free, so we should avoid it if possible.

Foz-DB Navi31:
Totals from 3933 (4.93% of 79789) affected shaders:
Instrs: 5726914 -> 5727295 (+0.01%); split: -0.00%, +0.01%
CodeSize: 31307100 -> 31308884 (+0.01%); split: -0.00%, +0.01%
SpillSGPRs: 1797 -> 1793 (-0.22%); split: -0.33%, +0.11%
Latency: 58973929 -> 58974343 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 8591893 -> 8591911 (+0.00%); split: -0.00%, +0.00%
SClause: 209074 -> 209115 (+0.02%); split: -0.00%, +0.02%
Copies: 423965 -> 432420 (+1.99%)
Branches: 149976 -> 149979 (+0.00%); split: -0.00%, +0.00%
PreSGPRs: 200175 -> 200663 (+0.24%)
VALU: 3440165 -> 3440156 (-0.00%); split: -0.00%, +0.00%
SALU: 555727 -> 556143 (+0.07%); split: -0.00%, +0.08%

Fixes: b872ff6ef2 ("aco/insert_exec_mask: if applicable, use s_wqm to restore exec after divergent CF")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34659>
 2025-04-23 09:37:50 +00:00
Georg Lehmann
8f3489f351 aco/isel: create WMMA with constant C matrix if possible 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34396>
 2025-04-22 16:08:57 +00:00
Georg Lehmann
4fa3fb87c7 aco/insert_NOPs: allow WMMA with constant C matrix 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34396>
 2025-04-22 16:08:56 +00:00
Georg Lehmann
6d7e67d986 nir,amd: add neg_lo/hi modifiers to cmat_matmul_amd 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34396>
 2025-04-22 16:08:55 +00:00
Georg Lehmann
b0c8f31600 aco: set opsel_hi to 1 for WMMA 
This is ignored by the hardware but LLVM requires it to disassemble GFX12 WMMA.

Cc: mesa-stable
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34396>
 2025-04-22 16:08:54 +00:00
Eric Engestrom
2bcb55f3f6 aco: help clang 20 do some additions and subtractions 
clang 20 complains:

    ../src/amd/compiler/aco_assembler.cpp:837:28: error: writing 1 byte into a region of size 0 [-Werror=stringop-overflow=]
      837 |       vaddr[num_vaddr + i] = reg(ctx, instr->operands.back(), 8) + i + 1;
          |       ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ../src/amd/compiler/aco_assembler.cpp:832:12: note: at offset 5 into destination object ‘vaddr’ of size 5
      832 |    uint8_t vaddr[5] = {0, 0, 0, 0, 0};
          |            ^~~~~
    ../src/amd/compiler/aco_assembler.cpp:837:28: error: writing 1 byte into a region of size 0 [-Werror=stringop-overflow=]
      837 |       vaddr[num_vaddr + i] = reg(ctx, instr->operands.back(), 8) + i + 1;
          |       ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ../src/amd/compiler/aco_assembler.cpp:832:12: note: at offset 6 into destination object ‘vaddr’ of size 5
      832 |    uint8_t vaddr[5] = {0, 0, 0, 0, 0};
          |            ^~~~~
    ../src/amd/compiler/aco_assembler.cpp:837:28: error: writing 1 byte into a region of size 0 [-Werror=stringop-overflow=]
      837 |       vaddr[num_vaddr + i] = reg(ctx, instr->operands.back(), 8) + i + 1;
          |       ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ../src/amd/compiler/aco_assembler.cpp:832:12: note: at offset 7 into destination object ‘vaddr’ of size 5
      832 |    uint8_t vaddr[5] = {0, 0, 0, 0, 0};
          |            ^~~~~

But `i < MIN2(instr->operands.back().size() - 1, 5 - num_vaddr)` means `i` is
at most `5 - num_vaddr - 1`, which means `vaddr[num_vaddr + i]` =>
`vaddr[num_vaddr + 5 - num_vaddr - 1]` => `vaddr[5 - 1]` => `vaddr[4]` which
is within the valid indices.

For some reason, using signed `int` instead allows clang to figure this
out, so let's do that since we don't need the extra range.

While at it, use ARRAY_SIZE(vaddr) instead of hard-coding the same `5`
in several places.

Backport-to: 25.0
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34625>
 2025-04-21 15:16:02 +00:00
Konstantin Seurer
978e9b670e aco,nir: Add support for new GFX12 ray tracing instructions 
Adds image_bvh_dual_intersect_ray and image_bvh8_intersect_ray which can
handle the new BVH format. Both instructions write up to 10 VGPRs so
they need to use a vec16 definition in nir.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
 2025-04-17 20:20:40 +00:00
Natalie Vock
ee0f784858 aco/ra: Don't consider precolored ops/defs in get_reg_impl 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
 2025-04-17 20:20:40 +00:00