Commit graph

2429 commits

Author SHA1 Message Date
Konstantin Seurer
ae84d41d48 radv/meta: Rework saving/restoring state
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
The current approach of explicitly saving/restoring some states is
unnecessarily complicated and inefficient. For example, some meta OPs
that use memory fills/copies will have nested save/restores. This patch
is the first step towards avoiding unnecessary state re-emits around
meta OPs.

The changes are:
- Move radv_meta_saved_state to radv_cmd_buffer::state
- Add radv_meta_begin/end helpers that initialize radv_meta_saved_state
and restore states used by the meta OP
- Remove all explicit saves/restores, use the new helpers

radv_meta_begin/end is called inside the entrypoint and not some nested
helper function which means that state is only restored once per meta
OP.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39774>
2026-02-18 09:37:55 +01:00
Rhys Perry
f9c11a8e15 radv: add ngg_wave_id_en to radv_shader_info
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39720>
2026-02-16 14:33:57 +00:00
Samuel Pitoiset
c3cc6fd051 radv: cleanup barriers after a depth/stencil expand
Synchronize in radv_expand_depth_stencil() is more robust.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39805>
2026-02-12 20:17:19 +00:00
Samuel Pitoiset
02a2451e1f radv: rename radv_image_use_dcc_image_stores()
To radv_image_compress_dcc_on_image_stores() because it seems more
informative.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39803>
2026-02-12 15:18:26 +00:00
Samuel Pitoiset
e58ef1b3bc radv: do not set the resume rendering flag for custom resolves
It's not a resume operation, it's a complete new rendering pass.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39782>
2026-02-12 07:12:56 +00:00
Samuel Pitoiset
cbf981e99a radv: do not resolve when rendering is suspended
The Vulkan spec says:
    "Store and resolve operations are only performed at the end of a
     render pass instance that does not specify the
     VK_RENDERING_SUSPENDING_BIT_KHR flag."

VK_RENDERING_SUSPENDING_BIT is also illegal with custom resolves.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39782>
2026-02-12 07:12:56 +00:00
Samuel Pitoiset
c1c031ca91 radv: make sure rendering isn't already active in CmdBeginRendering()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39782>
2026-02-12 07:12:56 +00:00
Samuel Pitoiset
99344bdfe5 radv: clear rendering state before performing resolves
This is mostly for not calling CmdBeginRendering() while rendering
is already active in order to catch potential driver issues. This
requires a small refactoring of how the rendering info is passed for
resolves though.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39782>
2026-02-12 07:12:55 +00:00
Samuel Pitoiset
4c18a36765 radv: pass VkSampleLocationsInfoEXT for depth/stencil expand
Instead of using an intermediate structure.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39782>
2026-02-12 07:12:55 +00:00
Samuel Pitoiset
17bbd45d59 radv: emit the framebuffer state when rendering begins
Much better.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39731>
2026-02-09 09:43:02 +00:00
Samuel Pitoiset
e178382fb8 radv: add a new dirty bit for the GFX12 HiZ workaround
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39731>
2026-02-09 09:43:02 +00:00
Samuel Pitoiset
a010c2694a radv: move {depth,stencil}_compress_disable to the image view extra info
Doesn't have to be a pipeline parameter.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39731>
2026-02-09 09:43:01 +00:00
Samuel Pitoiset
e488085942 radv/meta: remove unused saving/restoring rendering state logic
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39729>
2026-02-09 08:41:07 +00:00
Samuel Pitoiset
04d5077b00 radv: emit late decompressions for fbfetch slightly earlier
Right after "normal" layout transitions and just before the rendering
state is set, mostly because it doesn't need to be saved/restored
either.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39729>
2026-02-09 08:41:07 +00:00
Samuel Pitoiset
04f6bfae51 radv: only pass custom sample locations when relevant
Custom sample locations are only needed for depth decompression.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39729>
2026-02-09 08:41:07 +00:00
Samuel Pitoiset
ce3539b54f radv: fix late decompressions for fbfetch with more corner cases
With layers, or custom sample locations for depth.
Found this by inspection.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39729>
2026-02-09 08:41:06 +00:00
Samuel Pitoiset
f2d7d998a2 radv: track redundant PA_SC_VRS_OVERRIDE_CNTL register writes
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39675>
2026-02-06 07:15:10 +00:00
Samuel Pitoiset
f8153a7c20 radv: emit the VRS surface as part of the framebuffer state on GFX11+
No need to delay this and it slightly better for CPU overhead in the
hot draw path.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39675>
2026-02-06 07:15:10 +00:00
Samuel Pitoiset
65c1c66412 radv: stop delaying decompression passes for feedback loops with DRLR
This was an oversight of VK_KHR_dynamic_rendering_local_read which has
been addressed by VK_KHR_maintenance10 which introduced new flags to
give more information to implementations.

The Vulkan spec says:
    "VK_RENDERING_ATTACHMENT_INPUT_ATTACHMENT_FEEDBACK_BIT_KHR is
     intended to give implementations similar information as a subpass
     where an attachment could be used as both a color attachment and
     input attachment. Some implementations require extra work to make
     this scenario work beyond just considering the image layouts.
     Implementations which have no such considerations may treat this
     flag as a noop. The primary use case for this flag is to enable
     feedback loops inside a single shader."

    "Applications are encouraged to use
     VK_RENDERING_LOCAL_READ_CONCURRENT_ACCESS_CONTROL_BIT_KHR if
     maintenance10 is available and they use feedback loops with
     VK_KHR_dynamic_rendering_local_read. Feedback loops are still
     allowed when not using the rendering flag, but the performance
     implication was an oversight in the original definition of
     VK_KHR_dynamic_rendering_local_read."

Because it's clearly defined by the Vulkan spec, let's just pessimize
always to avoid relying on some shaders state which require to do very
late decompression passes. This will allow us to do more cleanups and
optimizations related to the framebuffer. Also note that DRLR is still
a niche feature.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39538>
2026-02-05 10:55:42 +00:00
Samuel Pitoiset
13c9e529bd radv: emit pending flushes after late decompressions with fbfetch
If the rendering state is inherited in the secondary, otherwise nothing
wait for the pending flushes after a decompression pass. One more
argument to stop delaying this.

Fixes
dEQP-VK.renderpasses.dynamic_rendering.partial_secondary_cmd_buff.local_read.*

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39678>
2026-02-04 13:49:47 +00:00
Marek Olšák
edffb2d76d ac: add FMASK codes
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39631>
2026-02-03 17:10:32 +00:00
Marek Olšák
e0c7c642f4 ac: unify and demystify CMASK clear codes
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39631>
2026-02-03 17:10:32 +00:00
Marek Olšák
43e9af713b radv: use DCC definitions more
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39631>
2026-02-03 17:10:31 +00:00
Samuel Pitoiset
9ad02b5724 radv: fix tracking of pipelines used in secondaries
This is just wrong if the secondary uses ESO because the emitted
pipelines would be NULL in the secondary, but if the app re-binds
the same pipeline in the primary it would consider it as already
emitted. A sequence like this would break:

CmdBindPipeline(compute)
CmdDispatch()
CmdExecuteCommands() --> with ESO compute
CmdBindPipeline(compute)
CmdDispatch()

This tracking is probably useless anyways because it's unlikely that
apps will rebind the same pipeline right after CmdExecuteCommands() but
let's keep it because this is a bugfix.

Fixes
dEQP-VK.api.command_buffers.pipeline_shader_object_mix_with_secondaries.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39587>
2026-01-30 08:09:39 +00:00
Samuel Pitoiset
96829d6c5e radv/meta: return the flush bits from radv_clear_hiz()
Similar to other functions.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39433>
2026-01-27 14:37:01 +00:00
Samuel Pitoiset
c91ed27582 radv: use the SQTT enable bit for PKT3_DISPATCH_TASKMESH_INDIRECT_MULTI_ACE
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39425>
2026-01-26 08:10:53 +00:00
Samuel Pitoiset
e272c8062d radv: use the SQTT enable bit for PKT3_DISPATCH_MESH_INDIRECT_MULTI
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39425>
2026-01-26 08:10:53 +00:00
Samuel Pitoiset
c7da19e2bf radv: use the SQTT enable bit for PKT3_DRAW_{INDEX}_INDIRECT_MULTI
This reports more info in RGP.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39425>
2026-01-26 08:10:52 +00:00
Samuel Pitoiset
e5982496f6 radv: move emitting SQTT markers closer to the draw/dispatch packets
Some packets already include a SQTT enable bit.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39425>
2026-01-26 08:10:52 +00:00
Timur Kristóf
fc0827126f radv: Remove previous mitigation of CS regalloc hang bug
Now that all larger workgroup sizes are lowered to 256,
the old workaround is not needed anymore.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39288>
2026-01-21 17:24:57 +00:00
Qiang Yu
4708eb85d7 radv: fix primitive restart gpu hang for pre gfx10
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
PAL always set WD_SWITCH_ON_EOP for pre gfx10 when primitive
restart is enabled to prevent gpu hang.

It only happens when specific index stream with primitive
restart. Since we don't know what's the exact problem,
just follow PAL to disable 4x primitive rate when primitive
restart is enabled.

GFX10+ does not use this function.

Cc: mesa-stable
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39292>
2026-01-21 02:38:26 +00:00
Marek Olšák
f8f4c02c97 radv: rename hiz_his to gfx12_*hiz
Only GFX12 will ever have this.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39260>
2026-01-19 16:58:17 +00:00
Marek Olšák
482c410f41 ac: remove never enabled gfx12 HiS
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39260>
2026-01-19 16:58:17 +00:00
Samuel Pitoiset
82d00a4963 radv: fix missing SQTT markers for task+mesh draws
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39178>
2026-01-15 09:43:00 +01:00
Samuel Pitoiset
aee006efcd radv/sqtt: emit userdata in the gang CS when needed
For task shaders.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39178>
2026-01-15 09:43:00 +01:00
Natalie Vock
c5d796c902 radv/rt: Use function call structure in NIR lowering
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580>
2026-01-14 14:19:06 +00:00
Samuel Pitoiset
edb730f647 radv: fix flushing gang semaphore with SDMA/ACE
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
If the main CS is SDMA and the gang CS is ACE, this would emit a
SDMA_FENCE packet on ACE which just hangs.

Fixes: b1938901d0 ("radv: Use SDMA fence packet when flushing gang semaphores")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39211>
2026-01-09 09:07:45 +00:00
Natalie Vock
1f6ac3fa93 radv/rt,aco: Always dispatch 1D workgroups for RT
We will swizzle the workgroups ourselves in the next commit.
Removes the need for 1D dispatch workarounds.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39142>
2026-01-08 19:49:54 +01:00
Marek Olšák
bd9206192d radv: use ac_set_sx_downconvert_state_for_mrt
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39093>
2026-01-02 16:46:20 +00:00
Marek Olšák
e64e41f69e radv: fix halved pixel throughput for a few non-blended 16bpp/32bpp formats
Fixed formats:
* R16_SFLOAT
* R16G16_SFLOAT
* R5G5B5A1_UNORM
* A2B10G10R10_UINT

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39093>
2026-01-02 16:46:20 +00:00
Timur Kristóf
0638fa5156 radv: Use compute for transfer operations unsupported by SDMA
For transfer queue operations that aren't supported by SDMA,
implement them with ACE (Async Compute Engine) using the pre-
existing compute copy functions.

Add a helper radv_get_pm4_cs that returns the ACE gang CS for
transfer command buffers and the main CS for graphics/compute
command buffers. Use radv_get_pm4_cs to make sure to emit the
compute commands to the correct command stream.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25594>
2026-01-02 04:32:06 +00:00
Samuel Pitoiset
b3c983b8dd amd,radv,radeonsi: add a new function to update windowed perf counters
Some checks failed
macOS-CI / macOS-CI (dri) (push) Has been cancelled
macOS-CI / macOS-CI (xlib) (push) Has been cancelled
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39065>
2025-12-24 07:20:01 +00:00
Samuel Pitoiset
47366527ce radv: fix capturing performance counters with SPM
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/14333
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39065>
2025-12-24 07:20:01 +00:00
Timur Kristóf
cc5190829f radv: Declare some gang submit functions in radv private header.
They will be called from the transfer copy functions.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39057>
2025-12-23 12:14:59 +00:00
Timur Kristóf
b1938901d0 radv: Use SDMA fence packet when flushing gang semaphores
Add back the SDMA fence packet to radv_flush_gang_semaphore.
This was regressed by 9666bd1245.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39057>
2025-12-23 12:14:59 +00:00
Timur Kristóf
d71a05dffa radv: Implement gang semaphores for transfer queues.
We need to use gang semaphores in the following two scenarios:

1. Leader to follower semaphore:
Increment the leader to follower semaphore when the leader wants
to block the follower: a transfer operation on ACE needs to wait
for a previous operation on SDMA.

2. Follower to leader semaphore:
Increment the follower to leader semaphore when the follower wants
to block the leader: a transfer operation on SDMA needs to wait
for a previous operation on ACE.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39057>
2025-12-23 12:14:58 +00:00
Timur Kristóf
4d0975dc83 radv: Update comments for gang semaphores
Change the explanation to use "leader" and "follower" terminology.
Explain better how it is used with GFX/ACE and SDMA/ACE.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39057>
2025-12-23 12:14:58 +00:00
Daniel Schürmann
1e8d367537 amd: add and use ac_cu_info::has_vtx_format_alpha_adjust_bug
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38701>
2025-12-22 07:34:48 +00:00
Samuel Pitoiset
045b778ed6 radv: add the SQTT relocated shaders BO to the cmdbuf list
Found this while debugging another thing with amdgpu.debug_mask=0x1 (VM).

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39002>
2025-12-22 07:13:06 +00:00
Samuel Pitoiset
c580fc667f ac,radv: add ac_cmdbuf::context_roll and use it
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38740>
2025-12-17 15:09:26 +00:00