Commit graph

17537 commits

Author SHA1 Message Date
Boyuan Zhang
e1f9dde5c0 radeonsi/vcn: add drm keyblob for cenc
Add new message buffer for drm keyblob which contains local policy
for cenc

Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34854>
2025-05-09 17:54:14 +00:00
Boyuan Zhang
2b6ecc8269 radeonsi/vcn: add drm key for cenc
Add new function to handle drm message buffer for cenc, which is
different than legacy mode.

Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34854>
2025-05-09 17:54:14 +00:00
Boyuan Zhang
d6e2438113 radeonsi/vcn: add subsample for cenc
Create new buffer for subsample parameters and submit it for cenc decryption

Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34854>
2025-05-09 17:54:14 +00:00
Daniel Schürmann
83fcd0e908 aco/insert_exec_mask: don't create unnecessary loop-header and -exit phis after demote
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
No fossil changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33479>
2025-05-09 17:20:29 +00:00
Daniel Schürmann
7adad4fc0e aco/isel: assert that terminate intrinsics don't appear in loops
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33479>
2025-05-09 17:20:29 +00:00
Daniel Schürmann
46f6c73d36 aco/isel: remove check for empty exec mask on uniform continues
This could only happen after terminate_if inside loops.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33479>
2025-05-09 17:20:29 +00:00
Daniel Schürmann
2b0536e921 aco: remove block_kind_continue_or_break workaround and tests
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33479>
2025-05-09 17:20:29 +00:00
Daniel Schürmann
fa4eb37bf6 radv: move terminate{_if} out of loops.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33479>
2025-05-09 17:20:29 +00:00
Eric Engestrom
5fc98ce169 amd/ci: set kernel for amdgpu jobs in .ci-tron-test-amdgpu
This fixes the kernel used by `radeonsi-vangogh-glcts`.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34898>
2025-05-09 15:56:18 +00:00
Eric Engestrom
1d902e3372 amd/ci: split .ci-tron-test-amdgpu from .ci-tron-test-radv
To be reused by the other ci-tron amdgpu jobs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34898>
2025-05-09 15:56:18 +00:00
Georg Lehmann
f1c066b353 aco/optimizer: remove instr_usedef_labels completely
Some changes because omod/clamp/insert no longer remove label_extract.

Foz-DB GFX1201:
Totals from 15 (0.02% of 79789) affected shaders:
Instrs: 15328 -> 15299 (-0.19%)
CodeSize: 76416 -> 76292 (-0.16%)
Latency: 62306 -> 62288 (-0.03%)
InvThroughput: 6165 -> 6148 (-0.28%)
Copies: 710 -> 709 (-0.14%)
VALU: 7908 -> 7892 (-0.20%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:25 +00:00
Georg Lehmann
eb6f5202ac aco/optimizer: remove label_usedef
No Foz-DB changes on NAvi21 and GFX1201.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:25 +00:00
Georg Lehmann
ffe11c0235 aco/optimizer: create fma without label_usedef
Foz-DB GFX1201:
Totals from 1 (0.00% of 79789) affected shaders:
Instrs: 1492 -> 1485 (-0.47%)
CodeSize: 7436 -> 7384 (-0.70%)
Latency: 2738 -> 2743 (+0.18%)
InvThroughput: 364 -> 365 (+0.27%)
SClause: 50 -> 56 (+12.00%)
Copies: 101 -> 100 (-0.99%)
PreSGPRs: 32 -> 31 (-3.12%)
VALU: 700 -> 701 (+0.14%)
SALU: 351 -> 350 (-0.28%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:25 +00:00
Georg Lehmann
177c98f95a aco/optimizer: apply dpp without label_usedef
Less interference with output modifier labels.

Foz-DB GFX1201:
Totals from 109 (0.14% of 79789) affected shaders:
Instrs: 161005 -> 160890 (-0.07%); split: -0.08%, +0.01%
CodeSize: 859472 -> 859720 (+0.03%); split: -0.03%, +0.06%
VGPRs: 6420 -> 6432 (+0.19%)
Latency: 1453659 -> 1452201 (-0.10%); split: -0.14%, +0.04%
InvThroughput: 171403 -> 171135 (-0.16%); split: -0.21%, +0.06%
VClause: 2746 -> 2743 (-0.11%)
SClause: 5256 -> 5257 (+0.02%)
Copies: 11085 -> 11008 (-0.69%); split: -0.89%, +0.20%
Branches: 3758 -> 3756 (-0.05%)
VALU: 90903 -> 90793 (-0.12%); split: -0.14%, +0.02%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:25 +00:00
Georg Lehmann
94ea9fd999 aco/optimizer: label p_extract as usedef
Helpful for combine_add_or_then_and_lshl on gfx11+.

Foz-DB GFX1201:
Totals from 478 (0.60% of 79789) affected shaders:
Instrs: 3844774 -> 3843719 (-0.03%); split: -0.03%, +0.00%
CodeSize: 20268684 -> 20268120 (-0.00%); split: -0.01%, +0.00%
Latency: 27005820 -> 27002460 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 5438686 -> 5436935 (-0.03%); split: -0.03%, +0.00%
Copies: 312458 -> 312469 (+0.00%); split: -0.00%, +0.01%
VALU: 2141236 -> 2140406 (-0.04%)
SALU: 527973 -> 527922 (-0.01%); split: -0.01%, +0.00%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:25 +00:00
Georg Lehmann
edfba93f4c aco/optimizer: apply f2f32 without label_usedef
This means we apply it to instructions where an omod label removed label_usedef.

Foz-DB Navi21:
Totals from 52 (0.07% of 79789) affected shaders:
Instrs: 144945 -> 144304 (-0.44%); split: -0.45%, +0.01%
CodeSize: 835956 -> 833992 (-0.23%); split: -0.30%, +0.06%
Latency: 2850900 -> 2850018 (-0.03%); split: -0.05%, +0.02%
InvThroughput: 1299486 -> 1299116 (-0.03%); split: -0.05%, +0.02%
Copies: 29638 -> 29803 (+0.56%); split: -0.16%, +0.71%
VALU: 94971 -> 94329 (-0.68%); split: -0.69%, +0.01%
SALU: 29272 -> 29274 (+0.01%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:25 +00:00
Georg Lehmann
c0e88c376a aco/optimizer: validate context data
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:25 +00:00
Georg Lehmann
906b7dbcec aco: replace novalidateir with novalidate debug option
The next commits will add more validation that's enabled by default.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:25 +00:00
Georg Lehmann
1540db244b aco/optimizer: store parent_instr for all temps
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:25 +00:00
Georg Lehmann
918359b41e aco/optimizer: add semantic aliases for info.instr
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:25 +00:00
Georg Lehmann
c62d7e680c aco/optimizer: remove label_mul
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:24 +00:00
Georg Lehmann
f773860a23 aco/optimizer: remove label_bitwise
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:24 +00:00
Georg Lehmann
cf3ec4a28f aco/optimizer: remove label_split
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:24 +00:00
Georg Lehmann
907e86e8fb aco/optimizer: remove label_vec
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:24 +00:00
Georg Lehmann
2c0a924521 aco/optimizer: remove label_minmax
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:24 +00:00
Georg Lehmann
dca8a7981d aco/optimizer: remove label_f2f32
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:24 +00:00
Georg Lehmann
17a973c6fa aco/optimizer: remove label_dpp8 and label_dpp16
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:24 +00:00
Georg Lehmann
dfa7e56f23 aco/optimizer: remove label_add_sub
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:23 +00:00
Georg Lehmann
345bf8a2f2 aco/optimizer: remove label_vop3p
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:23 +00:00
Georg Lehmann
6667ee66d5 aco/optimizer: remove label_vopc
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34858>
2025-05-09 14:23:23 +00:00
Georg Lehmann
6f4e26e54d radv/gfx12+: enable VK_KHR_shader_bfloat16
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
GFX11 seems to have precision issues, so don't enable the extension there for now.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
2025-05-09 11:20:26 +00:00
Georg Lehmann
a2209547db ac/nir: enable nir_op_bfdot2_bfadd
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
2025-05-09 11:20:26 +00:00
Georg Lehmann
44be05cc45 ac/llvm: support nir_op_bfdot2_bfadd
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
2025-05-09 11:20:26 +00:00
Georg Lehmann
f5a5905e37 aco: support nir_op_bfdot2_bfadd
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
2025-05-09 11:20:26 +00:00
Georg Lehmann
f364303084 ac/nir: set lower_bfloat16_conversions
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
2025-05-09 11:20:26 +00:00
Georg Lehmann
7716e63cd6 radv/nir/lower_cmat: handle bf16 conversions
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
2025-05-09 11:20:25 +00:00
Georg Lehmann
78524837c1 radv/nir/opt_cmat: support bfloat16
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
2025-05-09 11:20:25 +00:00
Georg Lehmann
5ca98bf99e aco: support bf16 wmma
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
2025-05-09 11:20:25 +00:00
Georg Lehmann
e8f5c335ff radv,aco,nir: keep the A and B base type for cmat_muladd_amd
With bfloat16, and the two fp8 formats in the future, using just the bit size
to identify the types is no longer possible.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34768>
2025-05-09 11:20:25 +00:00
Konstantin Seurer
c21e1776b3 radv: Use build flags instead of defines
Using the meta framework makes managing shader variants much easier.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34594>
2025-05-09 09:55:32 +00:00
Konstantin Seurer
33ac143779 vulkan: Introduce VK_BUILD_FLAG for specializing BVH build shaders
The advantage of using spec constants is that we do not have to include
multiple spirv binaries for multiple variants of a build stage.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34594>
2025-05-09 09:55:32 +00:00
Samuel Pitoiset
dae45adc9d aco: adjust an assertion in select_trap_handler_shader()
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34840>
2025-05-09 09:04:57 +00:00
Samuel Pitoiset
ae6d3df139 radv,aco: dump more SQ_WAVE registers from the trap handler on GFX12
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34840>
2025-05-09 09:04:57 +00:00
Samuel Pitoiset
0e73c85424 radv: fix configuring TRAP_PRESENT for compute shaders on GFX12
It no longer exists and it's been replaced by DYNAMIC_VGPR.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34840>
2025-05-09 09:04:57 +00:00
Samuel Pitoiset
50a01a6559 radv: fix save/restore SCC in the trap handler on GFX12
SQ_WAVE_STATE_PRIV now contains SCC instead of SQ_WAVE_STATUS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34840>
2025-05-09 09:04:57 +00:00
Samuel Pitoiset
effa563bb0 radv: adjust computing the PC from the trap handler on GFX12
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34840>
2025-05-09 09:04:57 +00:00
Valentine Burley
34012d5af3 ci: Remove EXTERNAL_KERNEL_TAG variable
The EXTERNAL_KERNEL_TAG variable is no longer needed.
For LAVA and bare-metal, we can override the KERNEL_TAG variable to fetch
both the kernel image and modules from a different tag than the default
mainline gfx-ci/linux kernel.

For LAVA, this also avoids the issue where jobs using EXTERNAL_KERNEL_TAG
would still have mainline kernel modules downloaded by the LAVA overlay.

Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34873>
2025-05-09 07:18:53 +00:00
Samuel Pitoiset
4b76d04f7f radv: ignore conditional rendering with vkCmdTraceRays*
CmdTraceRays is neither a dispatch or a draw command which means it
shouldn't be affected by conditional rendering.

Fixes recent VKCTS coverage.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34868>
2025-05-08 19:08:10 +00:00
Samuel Pitoiset
b7d2cdd2b4 radv: ignore radv_disable_dcc_stores on GFX12
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
It's not necessary because DCC is completely transparent to the
userspace driver. Also it's causing issues with scanout.

This fixes rendering issues with scanout in Indiana Jones.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12924
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34859>
2025-05-08 17:17:28 +02:00
Rhys Perry
2704a30df0 radv: perform nir_opt_access before the first radv_optimize_nir
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Two lowered loads might not be CSE'd after nir_lower_explicit_io if one of
them is shrinked. This doesn't happen for deref loads, but it needs the
CAN_REORDER flag first.

fossil-db (gfx1201):
Totals from 556 (0.70% of 79377) affected shaders:
MaxWaves: 14936 -> 14940 (+0.03%); split: +0.05%, -0.03%
Instrs: 2140334 -> 2140942 (+0.03%); split: -0.07%, +0.10%
CodeSize: 11137948 -> 11145416 (+0.07%); split: -0.07%, +0.13%
SpillSGPRs: 2385 -> 2527 (+5.95%); split: -0.34%, +6.29%
Latency: 12310570 -> 12305011 (-0.05%); split: -0.08%, +0.04%
InvThroughput: 2136142 -> 2135516 (-0.03%); split: -0.06%, +0.03%
VClause: 47419 -> 47420 (+0.00%); split: -0.01%, +0.01%
SClause: 58423 -> 58290 (-0.23%); split: -0.36%, +0.14%
Copies: 160626 -> 161321 (+0.43%); split: -0.25%, +0.68%
Branches: 69693 -> 69710 (+0.02%); split: -0.04%, +0.06%
PreSGPRs: 34824 -> 34945 (+0.35%); split: -0.24%, +0.58%
PreVGPRs: 28682 -> 28649 (-0.12%); split: -0.36%, +0.24%
VALU: 1080800 -> 1081171 (+0.03%); split: -0.04%, +0.08%
SALU: 353112 -> 353770 (+0.19%); split: -0.15%, +0.34%
SMEM: 81587 -> 81364 (-0.27%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34162>
2025-05-08 13:30:50 +00:00