Commit graph

13547 commits

Author SHA1 Message Date
Rhys Perry
23cb0b00db radv: implement mesh shader gs_fast_launch=2
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
c63ac28014 ac/nir: optimize mesh shader local_invocation_index
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
cddbe9a4c2 ac/nir: implement mesh shader gs_fast_launch=2
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
75bc2e7149 ac/nir: refactor mesh vertex/primitive export
To prepare for row export, move them into helpers with index and row
parameters.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
24431419ce ac/nir: remove dead code
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
6ec72806b7 ac/nir: add row parameter to helpers
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Rhys Perry
4c3677094e aco,nir: add export_row_amd intrinsic
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Samuel Pitoiset
13cac671fd ac/gpu_info: fix querying the maximum number of IBs per ring
The kernel uses 32-bit and Mesa was using 8-bit...

Fixes: 96345ae2ed ("ac/gpu_info: query the maximum number of IBs per submit from the kernel")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25867>
2023-10-24 19:53:08 +00:00
Timur Kristóf
eff4db3718 radv: Add gang follower semaphore functions.
These functions implement the possibility to have the
gang leader wait for its follower.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Timur Kristóf
b5e0a04d4e radv: Slightly refactor gang semaphore functions.
This is in preparation for adding some new ones.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Timur Kristóf
cbbb85c907 radv: Allow gang submit use cases other than task shaders.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Timur Kristóf
67ce3f11e8 radv: Simplify gang CS and semaphore initialization.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25796>
2023-10-24 17:58:12 +00:00
Samuel Pitoiset
56a05b9251 radv: fix missing predicate bit for WRITE_DATA helper
Fixes: 12a753f8d2 ("radv: Use new WRITE_DATA helper in more places.")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25850>
2023-10-24 14:53:04 +00:00
Samuel Pitoiset
f9cc2c5ba4 Revert "radv: fix finding shaders by PC"
Shaders are no longer allocated contiguously in memory and the next
pointer is now the shader.

This fixes using the trap handler on GFX8.

This reverts commit 2dcd12f38b.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25852>
2023-10-24 14:19:13 +00:00
Bas Nieuwenhuizen
653bcbcb09 radv: Expose VK_KHR_cooperative_matrix.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Bas Nieuwenhuizen
9df4703fbb radv: Add cooperative matrix lowering.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Bas Nieuwenhuizen
31863aa48c radv: Don't transparently use wave32 with cooperative matrices.
The instruction has different regsizes for wave32 vs. wave64.

To ensure cases with cooperative matrix load/store without any
actual wmma instructions get handled correctly, also require
full subgroups if subgroup invocation/id are used. Not sure
those could be transparently changed anyway.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Bas Nieuwenhuizen
d8458c0559 aco: Make RA understand WMMA instructions.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Bas Nieuwenhuizen
5e7c828c0e aco: Add WMMA instructions.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Samuel Pitoiset
96345ae2ed ac/gpu_info: query the maximum number of IBs per submit from the kernel
When the query fails (unsupported by the current kernel), we fallback
to some rough estimate.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10014
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25803>
2023-10-24 11:05:26 +00:00
Rhys Perry
fc342c897f Revert "radv: pre-init surface info"
This reverts commit e15a4e6e1a.

The change wasn't thread safe.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9955
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10021
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25830>
2023-10-24 10:11:34 +00:00
Samuel Pitoiset
aba815c041 radv: initialize video decoder for GFX11.5
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25304>
2023-10-24 08:20:47 +02:00
Samuel Pitoiset
50845f6fa4 radv: set ENABLE_PING_PONG_BIN_ORDER for GFX11.5
Ported from RadeonSI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25304>
2023-10-24 08:20:47 +02:00
Lang Yu
f9629fa729 amd/common: add missing stuff for gfx11.5
Signed-off-by: Lang Yu <Lang.Yu@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25836>
2023-10-24 01:07:12 +00:00
Rhys Perry
9e3b014d4f radv: fix signed integer overflow
Fixes UBSan error:
src/amd/vulkan/radv_shader_info.c:78:41: runtime error: left shift of 15 by 28 places cannot be represented in type 'int'

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25432>
2023-10-23 22:16:27 +00:00
Rhys Perry
f440e82679 ac/nir: fix out-of-bounds access in ac_nir_export_position
These accesses (and similar) in ac_nir_exprot_position were out-of-bounds:
if (!outputs[VARYING_SLOT_PSIZ][0])
   outputs_written &= ~VARYING_BIT_PSIZ;

I don't think this caused any real issue, but this silences ASan:
==40091==ERROR: AddressSanitizer: stack-use-after-return on address 0x7f8ffa6b1cb8 at pc 0x7f900da99068 bp 0x7f8ffb8871d0 sp 0x7f8ffb8871c8

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25432>
2023-10-23 22:16:26 +00:00
Rhys Perry
99214f0481 radv: skip zero-sized memcpy
Fixes UBSan:
src/amd/vulkan/radv_pipeline.c:1181:4: runtime error: null pointer passed as argument 2, which is declared to never be null
src/amd/vulkan/radv_shader.c:2243:4: runtime error: null pointer passed as argument 2, which is declared to never be null

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25432>
2023-10-23 22:16:26 +00:00
Friedrich Vock
16e1163651 radv/rmv: Recognize LPDDR memory
Fixes: 8d0e6c02 ("radv: Add RMV tracing utilities")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25845>
2023-10-23 18:08:02 +00:00
Friedrich Vock
5e734a7cbf radv/rra: Recognize LPDDR memory
Also move away from using unnamed constants for memory types.

Fixes: aef7ea868f ("ac/gpu_info: handle LPDDR4 and 5 in ac_memory_ops_per_clock")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25845>
2023-10-23 18:08:02 +00:00
Saleemkhan Jamadar
5f19452b8c radeonsi/vcn: set jpeg reg version for gfx 1150
select appropriate jpeg register version for gfx1150

Signed-off-by: Saleemkhan Jamadar <saleemkhan.jamadar@amd.com>
Reviewed-by: Leo Liu <Leo.Liu@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25278>
2023-10-20 07:32:35 +00:00
Marek Olšák
6835257246 amd/common: update DCC for gfx11.5
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25278>
2023-10-20 07:32:34 +00:00
Pierre-Eric Pelloux-Prayer
74978fd8e9 amd, radeonsi: Add code to enable gfx11.5
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25278>
2023-10-20 07:32:34 +00:00
Pierre-Eric Pelloux-Prayer
eabd2232e8 ac/nir: extract must_wait_attr_ring helper
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25278>
2023-10-20 07:32:34 +00:00
Pierre-Eric Pelloux-Prayer
b44a886b84 amd/common: add registers for gfx11.5
Built from amd-staging-drm-next c5a7d38c2c7fc + Alex D
"drm/amdgpu: update to the latest GC 11.5 header" patch.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25278>
2023-10-20 07:32:34 +00:00
Pierre-Eric Pelloux-Prayer
3b4424a4a6 amd/common: update addrlib for gfx11.5
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25278>
2023-10-20 07:32:34 +00:00
Timur Kristóf
4752b188dc radv: Wait for bottom of pipe in ACE gang wait postamble.
This makes sure that the semaphore is only written when all work
on the compute queue is finished.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
ff6c585121 radv: Add queue family argument to some functions.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
12a753f8d2 radv: Use new WRITE_DATA helper in more places.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
107473162e radv: Refactor WRITE_DATA helper function.
Create a version of this function that takes a CS and queue family.
move it to radv_cs.h so it can be called from multiple other files.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
1b988af0ad radv: Move radv_cp_wait_mem to radv_cs.h and add queue family argument.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
cd41d9e96d radv: Only call si_cp_dma_wait_for_idle on GFX and ACE queues.
This command is only supported on GFX and ACE.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
614bb47cbf radv: Use const on vi_alpha_is_on_msb arguments.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
ff39212e28 radv: Use const device argument in radv_sdma_copy_buffer.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
320e20db68 radv: Rename SDMA file to radv_sdma.c
It already contains more than just the buffer image copies.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25770>
2023-10-19 14:35:00 +00:00
Timur Kristóf
d7f5ff402a ac: Add rudimentary implementation of printing SDMA IBs.
This isn't meant to be exhaustive and only prints the packets
and fields which were interesting to me during debugging the
RADV transfer queue implementation.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25769>
2023-10-18 16:29:01 +00:00
Timur Kristóf
90455ac864 ac: Print IP type for IBs.
This makes it clearer what is what when reading IBs
for debugging purposes.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25769>
2023-10-18 16:29:01 +00:00
Timur Kristóf
05d74265dc ac: Rename ac_do_parse_ib to parse_pkt3_ib.
This function only parses PKT3 packets, not other types.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25769>
2023-10-18 16:29:01 +00:00
Timur Kristóf
dfcb8d4784 ac: Add amd_ip_type argument to ac_parse_ib and ac_parse_ib_chunk.
The current parser only supports PKT3 packets, but not all HW IP
types use this format. This makes it possible to support more
than one format depending on the IP type.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25769>
2023-10-18 16:29:01 +00:00
Timur Kristóf
b44a616253 ac: Clarify SDMA opcode defines.
Make the naming of these defines consistent with others.
And add a few new opcodes and related definitions.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25769>
2023-10-18 16:29:01 +00:00
Timur Kristóf
375a14234f ac/gpu_info: Add some SDMA related information.
SDMA only supports sparse resources on GFX9+.
SDMA only supports DCC and HTILE on GFX10+.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25769>
2023-10-18 16:29:01 +00:00