Commit graph

13373 commits

Author SHA1 Message Date
Georg Lehmann
178c046b9e aco: add missing scc def for SALU quad broadcast
Cc: mesa-stable

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26163>
(cherry picked from commit 509ce19643)
2023-11-15 21:21:24 +00:00
Samuel Pitoiset
0960a0c5dd radv: fix registering queues for RGP with compute only
This crashes if the graphics queue isn't created.

Fixes: 930e77e903 ("radv/sqtt: add support for queue info")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10136
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26183>
(cherry picked from commit 627d593443)
2023-11-15 21:21:24 +00:00
Rhys Perry
f30bd0074e radv: disable gs_fast_launch=2 by default
This currently causes severe performance problems in Remnant 2 and
Alan Wake 2.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: 28ebe62af2 ("radv: enable mesh shader gs_fast_launch=2 and multi-row export")
Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10071
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26090>
(cherry picked from commit 2011b678c5)
2023-11-08 18:18:59 +00:00
Rhys Perry
a8e266b7b3 radv: skip radv_remove_varyings for mesh shaders
Fixes compilation of a Talos Principle 2 shader.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: 9fa9782c17 ("radv: stop compiling a noop FS when the application doesn't provide a FS")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25659>
(cherry picked from commit 0a418561da)
2023-11-04 14:15:53 +00:00
Rhys Perry
9e45151243 radv: call lower_array_deref_of_vec before lower_io_arrays_to_elements
nir_lower_io_arrays_to_elements does not support array derefs of vectors,
even when nir_deref_instr_is_known_out_of_bounds is fixed.

They can occur with mesh shaders.

Found by inspection.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25659>
(cherry picked from commit ed12be533e)
2023-11-04 14:15:52 +00:00
Samuel Pitoiset
2b66b23045 radv: emit COMPUTE_PIPELINESTAT_ENABLE for CS invocations on ACE
This register seems needed to enable compute shader shader invocations
on GFX7. On GFX8+ it's working fine without emitting this register but
I think it doesn't hurt.

This fixes dEQP-VK.query_pool.statistics_query.*_cq on GFX7.

Fixes: a9945216ba ("radv: fix COMPUTE_SHADER_INVOCATIONS query on compute queue")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25957>
(cherry picked from commit 17daa08dff)
2023-11-01 14:52:55 +00:00
Samuel Pitoiset
18397e51ad radv: fix compute shader invocations query on compute queue on GFX6
Looks like GFX6 always writes the number of compute shader invocations
at offset 0 when used on compute queue.

This fixes dEQP-VK.query_pool.statistics_query.*_cq on GFX6.

Fixes: a9945216ba ("radv: fix COMPUTE_SHADER_INVOCATIONS query on compute queue")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25957>
(cherry picked from commit 9a0a77cb53)
2023-11-01 14:52:54 +00:00
Samuel Pitoiset
3cb8275141 radv: bind the non-dynamic graphics state from the pipeline unconditionally
The following sequence is valid (although weird) but many other drivers
(including RADV) were broken:
    - bind pipeline with some static state
    - set state command for that static state (to a bad value)
    - bind the same pipeline again
    - draw

Fixes new dEQP-VK.dynamic_state.*.double_static_bind.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25954>
(cherry picked from commit e4a1bc70dd)
2023-11-01 12:48:16 +00:00
Samuel Pitoiset
c03932b6e6 radv: fix a synchronization issue with primitives generated query on RDNA1-2
Only RDNA1-2 are affected because RADV needs to handle the legacy vs
NGG path for this query, and the NGG results are stored with 2 extra
64-bit values.

Fixes flakes with
dEQP-VK.transform_feedback.primitives_generated_query.* since VKCTS
1.3.7.0.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25862>
(cherry picked from commit 7d96fe853e)
2023-10-30 15:49:44 +00:00
Mike Blumenkrantz
2ee4ef998a radv: correctly return oom from the device when failing to create a cs
cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25927>
(cherry picked from commit c4283e32e3)
2023-10-30 15:48:29 +00:00
Karol Herbst
77eb71a612 radv: fix buffers in vkGetDescriptorEXT with size not aligned to 4
The range alignment didn't happen through GetDescriptorEXT as it called
write_buffer_descriptor directly. So simply move the align
from write_buffer_descriptor_impl into write_buffer_descriptor.

Fixes: 46e0c77582 ("radv: implement VK_EXT_descriptor_buffer")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25837>
(cherry picked from commit 1c619b668d)
2023-10-30 15:48:27 +00:00
Samuel Pitoiset
e3bef4dfa3 ac/gpu_info: remove bogus assertion about number of COMPUTE/SDMA queues
For example, my polaris10 GPU now returns 3 compute queues.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25876>
(cherry picked from commit 4f8a225387)
2023-10-30 15:47:09 +00:00
Samuel Pitoiset
1febb6f762 radv: report the last GPUVM fault when a device lost is detected
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>
2023-10-25 15:29:23 +00:00
Samuel Pitoiset
5a6208ad72 radv: query and report the last GPUVM fault with RADV_DEBUG=hang
Tested with a small use-after-free Vulkan test. It's pretty basic
for now but I think I will add status decoding support to report
more useful information.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>
2023-10-25 15:29:22 +00:00
Samuel Pitoiset
5eb8b3e9b1 radv/amdgpu: add support quering the last GPUVM fault
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>
2023-10-25 15:29:22 +00:00
Samuel Pitoiset
bdc62108be amd: add has_gpuvm_fault_query
Whether the kernel allows to query the last GPUVM fault.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>
2023-10-25 15:29:22 +00:00
Samuel Pitoiset
1b0fbcbe1c radv: remove outdated RADV_DEBUG=vmfaults support
It's been a very long time since this was useless because dmesg
required sudo access for a while.

This will be replaced by the new GPUVM fault interface which allows
to query from the kernel directly instead of trying to parse dmesg.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>
2023-10-25 15:29:22 +00:00
David Rosca
9b4fccf4b3 radeonsi/vcn: Add encode support for H264 B-frames
Tested with ffmpeg and gstreamer vah264enc.

References to B-frames (b_depth/b-pyramid) not supported.

Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25565>
2023-10-25 01:03:58 +00:00
Timur Kristóf
9a79c5f1e5 radv: Support SDMA in si_cs_emit_write_event_eop.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25833>
2023-10-24 23:12:45 +00:00
Timur Kristóf
b442de86c1 radv: Support SDMA in radv_cp_wait_mem.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25833>
2023-10-24 23:12:45 +00:00
Timur Kristóf
04a3ed8a92 radv: Support SDMA in radv_cs_write_data_head.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25833>
2023-10-24 23:12:45 +00:00
Rhys Perry
28ebe62af2 radv: enable mesh shader gs_fast_launch=2 and multi-row export
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:07 +00:00
Rhys Perry
3531136949 radv: implement mesh shader multi-row export
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:07 +00:00
Rhys Perry
773d35d25e ac/nir: implement mesh shader multi-row export
Unlike AMDVLK, this has separate loops for attribute stores and exports,
so that the stores from different rows can overlap.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:07 +00:00
Rhys Perry
c29d8a9e68 ac/nir,radv: pass workgroup size to ac_nir_lower_ngg_ms
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
c300a496ea ac/nir: add emit_ms_outputs helper
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
23cb0b00db radv: implement mesh shader gs_fast_launch=2
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
c63ac28014 ac/nir: optimize mesh shader local_invocation_index
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
cddbe9a4c2 ac/nir: implement mesh shader gs_fast_launch=2
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
75bc2e7149 ac/nir: refactor mesh vertex/primitive export
To prepare for row export, move them into helpers with index and row
parameters.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
24431419ce ac/nir: remove dead code
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
6ec72806b7 ac/nir: add row parameter to helpers
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
4c3677094e aco,nir: add export_row_amd intrinsic
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Samuel Pitoiset
13cac671fd ac/gpu_info: fix querying the maximum number of IBs per ring
The kernel uses 32-bit and Mesa was using 8-bit...

Fixes: 96345ae2ed ("ac/gpu_info: query the maximum number of IBs per submit from the kernel")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25867>
2023-10-24 19:53:08 +00:00
Timur Kristóf
eff4db3718 radv: Add gang follower semaphore functions.
These functions implement the possibility to have the
gang leader wait for its follower.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Timur Kristóf
b5e0a04d4e radv: Slightly refactor gang semaphore functions.
This is in preparation for adding some new ones.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Timur Kristóf
cbbb85c907 radv: Allow gang submit use cases other than task shaders.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Timur Kristóf
67ce3f11e8 radv: Simplify gang CS and semaphore initialization.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Samuel Pitoiset
56a05b9251 radv: fix missing predicate bit for WRITE_DATA helper
Fixes: 12a753f8d2 ("radv: Use new WRITE_DATA helper in more places.")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25850>
2023-10-24 14:53:04 +00:00
Samuel Pitoiset
f9cc2c5ba4 Revert "radv: fix finding shaders by PC"
Shaders are no longer allocated contiguously in memory and the next
pointer is now the shader.

This fixes using the trap handler on GFX8.

This reverts commit 2dcd12f38b.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25852>
2023-10-24 14:19:13 +00:00
Bas Nieuwenhuizen
653bcbcb09 radv: Expose VK_KHR_cooperative_matrix.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Bas Nieuwenhuizen
9df4703fbb radv: Add cooperative matrix lowering.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Bas Nieuwenhuizen
31863aa48c radv: Don't transparently use wave32 with cooperative matrices.
The instruction has different regsizes for wave32 vs. wave64.

To ensure cases with cooperative matrix load/store without any
actual wmma instructions get handled correctly, also require
full subgroups if subgroup invocation/id are used. Not sure
those could be transparently changed anyway.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Bas Nieuwenhuizen
d8458c0559 aco: Make RA understand WMMA instructions.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Bas Nieuwenhuizen
5e7c828c0e aco: Add WMMA instructions.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Samuel Pitoiset
96345ae2ed ac/gpu_info: query the maximum number of IBs per submit from the kernel
When the query fails (unsupported by the current kernel), we fallback
to some rough estimate.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10014
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25803>
2023-10-24 11:05:26 +00:00
Rhys Perry
fc342c897f Revert "radv: pre-init surface info"
This reverts commit e15a4e6e1a.

The change wasn't thread safe.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9955
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10021
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25830>
2023-10-24 10:11:34 +00:00
Samuel Pitoiset
aba815c041 radv: initialize video decoder for GFX11.5
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25304>
2023-10-24 08:20:47 +02:00
Samuel Pitoiset
50845f6fa4 radv: set ENABLE_PING_PONG_BIN_ORDER for GFX11.5
Ported from RadeonSI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25304>
2023-10-24 08:20:47 +02:00
Lang Yu
f9629fa729 amd/common: add missing stuff for gfx11.5
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25836>
2023-10-24 01:07:12 +00:00