Commit graph

7868 commits

Author SHA1 Message Date
Samuel Pitoiset
e4a1bc70dd radv: bind the non-dynamic graphics state from the pipeline unconditionally
The following sequence is valid (although weird) but many other drivers
(including RADV) were broken:
    - bind pipeline with some static state
    - set state command for that static state (to a bad value)
    - bind the same pipeline again
    - draw

Fixes new dEQP-VK.dynamic_state.*.double_static_bind.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25954>
2023-11-01 09:44:13 +00:00
Samuel Pitoiset
a97160cad8 radv: add RADV_DEBUG=nomeshshader
This option will be used to disable VK_EXT_mesh_shader in Mesa CI
for GFX11 because running task shader tests in parallel can hang the
GPU.

Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10051
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25969>
2023-11-01 08:49:35 +01:00
Bas Nieuwenhuizen
da7e6f303b radv: Add some initial graphics DGC preprocessing support.
Just the bits that obviously need no adjustment in the DGC
preprocessing code.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25835>
2023-11-01 00:06:10 +00:00
Bas Nieuwenhuizen
c4fb827441 radv: Add compute DGC preprocessing support.
This should reduce the overhead due to reduced syncs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25835>
2023-11-01 00:06:10 +00:00
Bas Nieuwenhuizen
108227a84e radv: Add DGC preprocessing barrier support.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25835>
2023-11-01 00:06:09 +00:00
Samuel Pitoiset
5176f75e0d radv: remove unnecessary VS_PARTIAL_FLUSH for NGG streamout
This used to be a PS_PARTIAL_FLUSH to fix synchronization issues with
NGG streamout on RDNA2 but it's no longer needed for RDNA3. It's
already synchronized in CmdEndTransformFeedbackEXT().

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25903>
2023-10-30 08:51:51 +00:00
Samuel Pitoiset
eb47e07782 radv: remove NGG streamout support for RDNA1-2
This was useful for experimenting it on RDNA2 and during RNDA3 bringup,
but now the support is rock solid on RDNA3 and it's useless to keep the
RADV_PERFTEST=ngg_streamout option.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25903>
2023-10-30 08:51:51 +00:00
Samuel Pitoiset
7beddd4f5c radv: use the GPUVM fault protection status helper
To print more useful information when a fault happens.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25855>
2023-10-30 08:10:23 +00:00
Samuel Pitoiset
7d96fe853e radv: fix a synchronization issue with primitives generated query on RDNA1-2
Only RDNA1-2 are affected because RADV needs to handle the legacy vs
NGG path for this query, and the NGG results are stored with 2 extra
64-bit values.

Fixes flakes with
dEQP-VK.transform_feedback.primitives_generated_query.* since VKCTS
1.3.7.0.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25862>
2023-10-30 07:33:06 +00:00
Mike Blumenkrantz
c4283e32e3 radv: correctly return oom from the device when failing to create a cs
cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25927>
2023-10-27 15:29:32 +00:00
Samuel Pitoiset
7ec2544a9b radv: dump the pipeline hash to the gpu hang report
It can be useful to verify that a specific pipeline causes a hang.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25908>
2023-10-27 12:10:51 +00:00
Karol Herbst
1c619b668d radv: fix buffers in vkGetDescriptorEXT with size not aligned to 4
The range alignment didn't happen through GetDescriptorEXT as it called
write_buffer_descriptor directly. So simply move the align
from write_buffer_descriptor_impl into write_buffer_descriptor.

Fixes: 46e0c77582 ("radv: implement VK_EXT_descriptor_buffer")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25837>
2023-10-27 10:52:55 +00:00
Faith Ekstrand
d5c310899a nir: Split nir_lower_subgroup_options::lower_vote_eq into two bits
On NVIDIA, we can do a vote_ieq on bool in one hardware op so we don't
want that lowered.  We do want to lower vote_feq and other vote_ieq,
though.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894>
2023-10-26 23:05:44 +00:00
Samuel Pitoiset
fb64f7c676 radv: advertise VK_EXT_image_compression_control
This will be used by vkd3d-proton to disable eg. DCC per image in order
to workaround game bugs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25681>
2023-10-26 13:04:08 +00:00
Samuel Pitoiset
8a25a9f4dd radv: implement VK_EXT_image_compression_control
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25681>
2023-10-26 13:04:08 +00:00
Samuel Pitoiset
8ec0a4c89c radv: move RADV_DEBUG_NO_HIZ check in radv_use_htile_for_image()
To match radv_use_dcc_for_image().

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25681>
2023-10-26 13:04:08 +00:00
Samuel Pitoiset
1febb6f762 radv: report the last GPUVM fault when a device lost is detected
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>
2023-10-25 15:29:23 +00:00
Samuel Pitoiset
5a6208ad72 radv: query and report the last GPUVM fault with RADV_DEBUG=hang
Tested with a small use-after-free Vulkan test. It's pretty basic
for now but I think I will add status decoding support to report
more useful information.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>
2023-10-25 15:29:22 +00:00
Samuel Pitoiset
5eb8b3e9b1 radv/amdgpu: add support quering the last GPUVM fault
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>
2023-10-25 15:29:22 +00:00
Samuel Pitoiset
1b0fbcbe1c radv: remove outdated RADV_DEBUG=vmfaults support
It's been a very long time since this was useless because dmesg
required sudo access for a while.

This will be replaced by the new GPUVM fault interface which allows
to query from the kernel directly instead of trying to parse dmesg.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23238>
2023-10-25 15:29:22 +00:00
Timur Kristóf
9a79c5f1e5 radv: Support SDMA in si_cs_emit_write_event_eop.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25833>
2023-10-24 23:12:45 +00:00
Timur Kristóf
b442de86c1 radv: Support SDMA in radv_cp_wait_mem.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25833>
2023-10-24 23:12:45 +00:00
Timur Kristóf
04a3ed8a92 radv: Support SDMA in radv_cs_write_data_head.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25833>
2023-10-24 23:12:45 +00:00
Rhys Perry
28ebe62af2 radv: enable mesh shader gs_fast_launch=2 and multi-row export
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:07 +00:00
Rhys Perry
3531136949 radv: implement mesh shader multi-row export
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:07 +00:00
Rhys Perry
c29d8a9e68 ac/nir,radv: pass workgroup size to ac_nir_lower_ngg_ms
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
23cb0b00db radv: implement mesh shader gs_fast_launch=2
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
cddbe9a4c2 ac/nir: implement mesh shader gs_fast_launch=2
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Samuel Pitoiset
13cac671fd ac/gpu_info: fix querying the maximum number of IBs per ring
The kernel uses 32-bit and Mesa was using 8-bit...

Fixes: 96345ae2ed ("ac/gpu_info: query the maximum number of IBs per submit from the kernel")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25867>
2023-10-24 19:53:08 +00:00
Timur Kristóf
eff4db3718 radv: Add gang follower semaphore functions.
These functions implement the possibility to have the
gang leader wait for its follower.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Timur Kristóf
b5e0a04d4e radv: Slightly refactor gang semaphore functions.
This is in preparation for adding some new ones.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Timur Kristóf
cbbb85c907 radv: Allow gang submit use cases other than task shaders.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Timur Kristóf
67ce3f11e8 radv: Simplify gang CS and semaphore initialization.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Samuel Pitoiset
56a05b9251 radv: fix missing predicate bit for WRITE_DATA helper
Fixes: 12a753f8d2 ("radv: Use new WRITE_DATA helper in more places.")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25850>
2023-10-24 14:53:04 +00:00
Samuel Pitoiset
f9cc2c5ba4 Revert "radv: fix finding shaders by PC"
Shaders are no longer allocated contiguously in memory and the next
pointer is now the shader.

This fixes using the trap handler on GFX8.

This reverts commit 2dcd12f38b.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25852>
2023-10-24 14:19:13 +00:00
Bas Nieuwenhuizen
653bcbcb09 radv: Expose VK_KHR_cooperative_matrix.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Bas Nieuwenhuizen
9df4703fbb radv: Add cooperative matrix lowering.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Bas Nieuwenhuizen
31863aa48c radv: Don't transparently use wave32 with cooperative matrices.
The instruction has different regsizes for wave32 vs. wave64.

To ensure cases with cooperative matrix load/store without any
actual wmma instructions get handled correctly, also require
full subgroups if subgroup invocation/id are used. Not sure
those could be transparently changed anyway.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Rhys Perry
fc342c897f Revert "radv: pre-init surface info"
This reverts commit e15a4e6e1a.

The change wasn't thread safe.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9955
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10021
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25830>
2023-10-24 10:11:34 +00:00
Samuel Pitoiset
aba815c041 radv: initialize video decoder for GFX11.5
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25304>
2023-10-24 08:20:47 +02:00
Samuel Pitoiset
50845f6fa4 radv: set ENABLE_PING_PONG_BIN_ORDER for GFX11.5
Ported from RadeonSI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25304>
2023-10-24 08:20:47 +02:00
Rhys Perry
9e3b014d4f radv: fix signed integer overflow
Fixes UBSan error:
src/amd/vulkan/radv_shader_info.c:78:41: runtime error: left shift of 15 by 28 places cannot be represented in type 'int'

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25432>
2023-10-23 22:16:27 +00:00
Rhys Perry
99214f0481 radv: skip zero-sized memcpy
Fixes UBSan:
src/amd/vulkan/radv_pipeline.c:1181:4: runtime error: null pointer passed as argument 2, which is declared to never be null
src/amd/vulkan/radv_shader.c:2243:4: runtime error: null pointer passed as argument 2, which is declared to never be null

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25432>
2023-10-23 22:16:26 +00:00
Friedrich Vock
16e1163651 radv/rmv: Recognize LPDDR memory
Fixes: 8d0e6c02 ("radv: Add RMV tracing utilities")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25845>
2023-10-23 18:08:02 +00:00
Friedrich Vock
5e734a7cbf radv/rra: Recognize LPDDR memory
Also move away from using unnamed constants for memory types.

Fixes: aef7ea868f ("ac/gpu_info: handle LPDDR4 and 5 in ac_memory_ops_per_clock")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25845>
2023-10-23 18:08:02 +00:00
Marek Olšák
6835257246 amd/common: update DCC for gfx11.5
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25278>
2023-10-20 07:32:34 +00:00
Timur Kristóf
4752b188dc radv: Wait for bottom of pipe in ACE gang wait postamble.
This makes sure that the semaphore is only written when all work
on the compute queue is finished.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
ff6c585121 radv: Add queue family argument to some functions.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
12a753f8d2 radv: Use new WRITE_DATA helper in more places.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
107473162e radv: Refactor WRITE_DATA helper function.
Create a version of this function that takes a CS and queue family.
move it to radv_cs.h so it can be called from multiple other files.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00