Commit graph

10167 commits

Author SHA1 Message Date
Georg Lehmann
31a3430570 radv/nir/lower_cmat: use radv_nir_cmat_bits consistently
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34382>
2025-04-24 06:37:44 +00:00
Georg Lehmann
c3964e87f8 radv: apply fneg/fabs modifiers to wmma
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34396>
2025-04-22 16:08:55 +00:00
Georg Lehmann
6d7e67d986 nir,amd: add neg_lo/hi modifiers to cmat_matmul_amd
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34396>
2025-04-22 16:08:55 +00:00
Marek Olšák
4a51089f30 radv: fix incorrect patch_outputs_read for TCS with dynamic state
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Fixes: 8c2f9f0665 - radv: switch to the new TCS LDS/offchip size computation

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34544>
2025-04-19 22:55:00 -04:00
Marek Olšák
be8977811b ac/nir: remove shader_info parameter from ac_nir_compute_tess_wg_info
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34544>
2025-04-19 22:55:00 -04:00
Marek Olšák
d82eda72a1 ac/gpu_info: move HS info into radeon_info
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34544>
2025-04-19 22:55:00 -04:00
Marek Olšák
ea294349bd radv: move the tess factor ring after the tess offchip ring
to match radeonsi

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34544>
2025-04-19 22:54:59 -04:00
Marek Olšák
c057d9105f ac/gpu_info: add total_tess_ring_size
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34544>
2025-04-19 22:54:59 -04:00
Samuel Pitoiset
792c30dd32 radv/meta: remove redundant parameter to blit_surf_for_image_level_layer()
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34558>
2025-04-18 17:21:24 +02:00
Samuel Pitoiset
a3f2c5f05e radv/meta: remove unnecessary radv_meta_blit2d_buffer::bs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34558>
2025-04-18 17:21:24 +02:00
Samuel Pitoiset
78c2feed00 radv/meta: rename more buffer->memory for fill/copy/update operations
Recently, I renamed most of the helpers for future work but I forgot
few things like meta keys, etc.

This is for consistency.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34558>
2025-04-18 17:21:24 +02:00
Samuel Pitoiset
43c8cb1ae2 radv/meta: remove unused functions/prototypes
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34558>
2025-04-18 17:21:24 +02:00
Samuel Pitoiset
78f03dcf70 radv/meta: simplify dealing with image layouts for blits/resolves
This doesn't do anything useful.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34558>
2025-04-18 17:21:24 +02:00
Konstantin Seurer
76031ba53d radv: Optimize the gfx12 encode shader
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
97f6287827 radv: Use the BVH8 format on GFX12
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
95e7343a7d radv/bvh: Add helpers for encoding
The build and update paths can use the same code.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
3af19f336c radv/bvh: Document GFX12 BVH encoding
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
2942e3affb radv/rra: Set rra_accel_struct_header::rtip_level
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
fa99eeb2b4 radv/rra: Move gfx10_3 specific code to a new file
gfx12 needs completely different code and having them in different files
is cleaner.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
9d157173b2 radv: Refactor create_bvh_descriptor
Make it a bit more extendable since GFX12 introduced more fields.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
978e9b670e aco,nir: Add support for new GFX12 ray tracing instructions
Adds image_bvh_dual_intersect_ray and image_bvh8_intersect_ray which can
handle the new BVH format. Both instructions write up to 10 VGPRs so
they need to use a vec16 definition in nir.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
2dee1117b7 vulkan: Add a vk_device parameter to get_encode_key
Useful for selecting different encoding options based on hardware
generation.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Caio Oliveira
fd0a7efb5a spirv, nir: Delay calculation of shared_size when using explicit layout
Move the calculation to nir_lower_vars_to_explicit_types().  This
consolidates the check of shader_info::shared_memory_explicit_layout
in a single place instead of in all drivers.

This is motivated by SPV_KHR_untyped_pointers.  Before that extension
we had essentially two modes for shared memory variables

- No layout decorations in the SPIR-V, and both internal layout and
  driver location was _given by the driver_.

- Explicitly laid out, i.e. they are blocks, and decorated with Aliased.
  Because they all alias, we could assign them driver location directly
  to the start of the shared memory.

With the untyped pointers extension, there's a third option, to be added
by a later commit

- Explicitly laid out, i.e. they are blocks, and NOT decorated
  with Aliased.  Driver location is _given by the driver_.  Blocks
  with and without Aliased can be mixed.

The driver location of multiple blocks that don't alias depend on
alignment that is driver-specific, which we can more easily do from
the nir_lower_vars_to_explicit_types() that already has access to
a function to obtain such value.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> (hk)
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> (v3dv)
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (anv/hasvk)
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (panvk)
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> (radv)
Reviewed-by: Rob Clark <robdclark@gmail.com> (tu)
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34139>
2025-04-17 19:13:17 +00:00
Samuel Pitoiset
209a0ede98 radv: add a function to emit meshlet registers on GFX11+
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34518>
2025-04-17 12:49:47 +00:00
Samuel Pitoiset
836757bec3 radv: tidy up radv_emit_ps_epilog_state()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34518>
2025-04-17 12:49:47 +00:00
Samuel Pitoiset
dca35b7226 radv: tidy up radv_emit_geometry_shader()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34518>
2025-04-17 12:49:47 +00:00
Samuel Pitoiset
d999afeb7a radv: tidy up radv_emit_vertex_shader()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34518>
2025-04-17 12:49:47 +00:00
Samuel Pitoiset
85fdf69027 radv: simplify combining TES/VS+GS config registers
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34518>
2025-04-17 12:49:47 +00:00
Samuel Pitoiset
0dd9833348 radv: remove redundant assertion when emitting PS epilog state
It's already checked by radv_emit_32bit_pointer().

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34518>
2025-04-17 12:49:47 +00:00
Samuel Pitoiset
a230d2daa3 radv: use radeon_set_sh_reg() for only 1 DWORD
It's just shorter to write.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34518>
2025-04-17 12:49:47 +00:00
Samuel Pitoiset
11e8a96495 radv: use common scratch tmpring size programming
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
No logical changes.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34549>
2025-04-17 10:35:40 +00:00
Samuel Pitoiset
710d7ea8b8 radv: compute the optimal scratch wavesize
This might increase the scratch BO sizes but it's supposed to be
faster because scratch waves would be distributed among memory channels.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34549>
2025-04-17 10:35:40 +00:00
Samuel Pitoiset
d94f8b4460 ac/gpu_info,radv: add scratch_wavesize_granularity info
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34549>
2025-04-17 10:35:40 +00:00
Samuel Pitoiset
e616761fb2 radv: re-introduce the compute vs CP DMA heuristic for copy/fill operations
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
This caused a -5% performance regression in Control because using
compute always eats resources.

This new approach introduces a flag called RADV_COPY_FLAGS_DEVICE_LOCAL
which can be used to indicate if the underlying memory is device local.
This should also help for future work.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12639
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34556>
2025-04-17 08:59:58 +00:00
Samuel Pitoiset
5e2508e7c4 radv: simplify radv_fill_xxx() helpers
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34556>
2025-04-17 08:59:58 +00:00
Samuel Pitoiset
8ba94d8263 radv: add radv_fill_image() helper
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34556>
2025-04-17 08:59:58 +00:00
Samuel Pitoiset
0fa43b5bfb radv: use radv_fill_memory() in the accel struct path
It's now possible to remove the NULL BO check.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34556>
2025-04-17 08:59:58 +00:00
Caio Oliveira
d5ad798140 spirv, radv, intel: Add NIR intrinsic for cmat conversion
A cooperative matrix conversion operation was represented in NIR by the
cmat_unary_op intrinsic with an nir_alu_op as extra parameter,
that was already lowered to a specific conversion operation
based on the matrix types.

Instead of that, add a new intrinsic `cmat_convert` that is specific
for that conversion.  In addition to the src/dst matrix descriptions
already available, also include the signedness information in the
intrinsic (reuse nir_cmat_signed for that).  This is needed because
different Convert operations define different interpretations for
integers, regardless their original type.

In this patch, both radv and intel were changed to use the same logic
that was previously used to pick the lowered ALU op.

This change will help represent cmat conversions involving BFloat16,
because it avoids having to create new NIR ALU ops for all the
combinations involving BFloat16.

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34511>
2025-04-16 23:13:36 +00:00
Samuel Pitoiset
b4940255ed radv/sdma: add support for compression on GFX12
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Similar to previous generations that support compression, except that
the driver don't need to configure a meta VA because DCC is completely
transparent to the userspace.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34517>
2025-04-16 06:57:00 +00:00
Samuel Pitoiset
efa0b16bb2 radv/sdma: add a new flag to know if the surface is compressed
On GFX12, DCC is transparent to the driver and there is no meta VA.
Adding a new flag to know if the SDMA surface is compressed is needed.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34517>
2025-04-16 06:57:00 +00:00
Samuel Pitoiset
03671ccf9e radv/sdma: use the correct helper to get the number type field
This wasn't technically incorrect because V_028C70_BU_NUM_xxx values
are similar to V_028C70_NUMBER_xxx but it's better to use the corect
helper.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34517>
2025-04-16 06:57:00 +00:00
Samuel Pitoiset
b44dc98cde radv/sdma: remove redundant check for compression when getting metadata
It's already checked by the caller.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34517>
2025-04-16 06:57:00 +00:00
Samuel Pitoiset
d3d5d2fe86 radv/sdma: use SDMA5_DCC_xxx bitfields
It's cleaner.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34517>
2025-04-16 06:57:00 +00:00
Samuel Pitoiset
f44342199a radv/sdma: simplify configuring the number of uncompressed DCC blocks
SDMA doesn't support MSAA, so the value can be
V_028C78_MAX_BLOCK_SIZE_256B.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34517>
2025-04-16 06:57:00 +00:00
Samuel Pitoiset
c42d43e8eb radv: print more error messages during SPM initialization
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34524>
2025-04-16 06:35:33 +00:00
Marek Olšák
d7c903f258 ac/gpu_info: add payload_entry_size into ac_task_info
to stop causing full RADV recompiles when it's changed.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34432>
2025-04-16 06:08:48 +00:00
Samuel Pitoiset
e86e0fc525 radv: allocate the SPM BO in GTT for faster readback
Reading VRAM from CPU is very slow.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34467>
2025-04-15 06:30:38 +00:00
Samuel Pitoiset
410f7f9f6e radv: only enable DCC for invisible VRAM on GFX12
DCC should only be allowed on invisible VRAM, otherwise the CPU could
read the data and it will read garbage if it's compressed.

This also caused GPU hangs after suspend/resume probably because
some buffers were compressed when moved back from GTT to VRAM.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12962
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12922
Fixes: 9af11bf306 ("radv: add initial DCC support on GFX12")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34347>
2025-04-14 07:39:33 +00:00
Samuel Pitoiset
75be860eec radv: use paired context regs when optimal on GFX12
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
CP is very slow on GFX12 and parsing the packet header is the main
bottleneck. Using paired context regs reduce the number of packet
headers and it should be more optimal.

It doesn't seem worth when only one context reg is emitted (one packet
header and same number of DWORDS) or when consecutive context regs are
emitted (would increase the number of DWORDS).

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34421>
2025-04-14 06:18:13 +00:00
Samuel Pitoiset
f92f50c58a radv: add macros for paired context registers on GFX12
Imported from RadeonSI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34421>
2025-04-14 06:18:13 +00:00