Commit graph

10900 commits

Author SHA1 Message Date
Samuel Pitoiset
a47952d495 radv: upload and emit dynamic descriptors separately from push constants
Dynamic descriptors are rarely used and this will allow to do more
optimizations for push constants, like gathering the size from shaders
themselves instead of using the pipeline layout.

fossils-db (GFX1201):
Totals from 21740 (27.30% of 79646) affected shaders:
Instrs: 11186407 -> 11192061 (+0.05%); split: -0.05%, +0.10%
CodeSize: 59842068 -> 59864412 (+0.04%); split: -0.04%, +0.08%
Latency: 56333136 -> 56325208 (-0.01%); split: -0.03%, +0.02%
InvThroughput: 8576452 -> 8576516 (+0.00%); split: -0.00%, +0.00%
SClause: 279186 -> 279713 (+0.19%); split: -0.06%, +0.25%
Copies: 577854 -> 581735 (+0.67%); split: -0.28%, +0.95%
PreSGPRs: 867163 -> 866409 (-0.09%)
SALU: 1391187 -> 1395055 (+0.28%); split: -0.12%, +0.39%

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37768>
2025-10-14 15:34:43 +00:00
Samuel Pitoiset
bc32286e5b radv: declare a new user SGPR for dynamic descriptors
To move them out of push constants.

fossils-db (GFX1201):
Totals from 20700 (25.99% of 79646) affected shaders:
Instrs: 14375624 -> 14370051 (-0.04%); split: -0.07%, +0.03%
CodeSize: 76746128 -> 76723772 (-0.03%); split: -0.05%, +0.02%
Latency: 74103586 -> 74113651 (+0.01%); split: -0.01%, +0.02%
InvThroughput: 11908817 -> 11908798 (-0.00%); split: -0.00%, +0.00%
VClause: 249605 -> 249607 (+0.00%); split: -0.00%, +0.00%
SClause: 337914 -> 337772 (-0.04%); split: -0.08%, +0.04%
Copies: 843585 -> 839233 (-0.52%); split: -0.62%, +0.10%
PreSGPRs: 836283 -> 837260 (+0.12%)
SALU: 1790713 -> 1786374 (-0.24%); split: -0.29%, +0.05%

Co-authored-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37768>
2025-10-14 15:34:43 +00:00
Timur Kristóf
772b9ce411 radv: Remove qf from radv_spm/sqtt/perfcounter where applicable
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:20 +00:00
Timur Kristóf
8501c8cd43 radv: Remove unneeded forward declaration of qf from dgc header
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:19 +00:00
Timur Kristóf
4462def21b radv: Remove qf argument from radv_cs_write_data (and _head)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:18 +00:00
Timur Kristóf
2bd1beac4c radv: Remove qf argument from radv_cs_emit_cache_flush
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:18 +00:00
Timur Kristóf
5f901928e8 radv: Remove qf argument from radv_cp_wait_mem
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:16 +00:00
Timur Kristóf
bc1f438f2a radv: Remove qf argument from radv_cs_emit_write_event_eop
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:15 +00:00
Timur Kristóf
a0104c5bf6 radv: Add amd_ip_type to radv_cmd_stream
Specify an amd_ip_type when initializing radv_cmd_stream
so that we know which packets are valid for the HW IP type
of a given command stream.

This field should be used instead of radv_cmd_buffer::qf when
emitting packets to a command stream.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:13 +00:00
Timur Kristóf
a57419f96b radv: Clarify image and image/buffer copy helper functions
Refactor the functions to make it clear what they do:
copy_memory_to_image -> gfx_or_compute_copy_memory_to_image
copy_image_to_memory -> compute_copy_image_to_memory
copy_image -> gfx_or_compute_copy_image

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:12 +00:00
Timur Kristóf
db4a9aaf29 radv: Call transfer copy functions from API functions, not helpers
Improves code readability.
No functional changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:12 +00:00
Timur Kristóf
21232ffd12 radv: Pass correct queue family in radv_emit_cache_flush
Probably a leftover from a past refactor.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:12 +00:00
Timur Kristóf
50f0912f54 radv: Pass correct queue family to radv_cs_emit_write_event_eop
Probably a leftover from a previous code refactor.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:12 +00:00
Samuel Pitoiset
a566a933ca radv: always return optimalDeviceAccess=TRUE for block-compressed formats
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Required by the spec.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37817>
2025-10-14 10:14:45 +00:00
Samuel Pitoiset
a157faee42 radv: only return identicalMemoryLayout for linear images
This isn't true for tiled images because the swizzling can change.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37817>
2025-10-14 10:14:45 +00:00
Benjamin Cheng
c8093e6cb1 radv: Output requested encode query results only
Video encode feedback queries have a configurable set of feedback bits,
specified in VkQueryPoolVideoEncodeFeedbackCreateInfoKHR::encodeFeedbackFlags.
Only the bits specified should be output when retrieving results.

Fixes: 1d74661dfd ("radv: add encoder queue support pieces and encoder queries.")
Reviewed-by: David Rosca <david.rosca@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37774>
2025-10-14 09:11:25 +00:00
Samuel Pitoiset
26c7f2fd6a radv: enable the global BO list by default
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
vkd3d-proton (DX12) and Zink (GL) have always been enabling features
that require the global BO list to be enabled.

Since DXVK 2.7+ (August 2025), it's also always enabled by default for
DX9-11 games (because it requires BDA now).

The global BO list used to decrease performance in the past mostly
because of bad memory management in AMDGPU, but it seems the situation
slightly improved since. Though, there might still some workloads that
hit the issue, but I think it should be mostly good overall.

This introduces RADV_DEBUG=nobolist to disable the global BO list
when no features require it.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6957
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2331
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35075>
2025-10-14 08:12:36 +00:00
Samuel Pitoiset
df269714ef radv/meta: remove radv_cmd_buffer_resolve_rendering_{hw,cs,fs}
Just call the other functions directly.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37792>
2025-10-14 07:46:13 +00:00
Samuel Pitoiset
a81f01bc96 radv/meta: pass iview formats for subpass resolves
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37792>
2025-10-14 07:46:13 +00:00
Samuel Pitoiset
d3e716f1fb radv/meta: re-use radv_meta_resolve_{fragment,hardware}_image() for subpass resolves
Similar to compute.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37792>
2025-10-14 07:46:13 +00:00
Samuel Pitoiset
ac3c21f130 radv/meta: pass image formats to radv_meta_resolve_{hardware,fragment}_image()
Similar to the compute function.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37792>
2025-10-14 07:46:12 +00:00
Samuel Pitoiset
e982f6e2c8 radv: fix shaders memleak when importing pipeline binaries with GPL
The implementation must use the data in VkPipelineBinaryInfoKHR when
provided instead of importing binaries from libraries.

This fixes a memleak with shaders found with ASAN.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37844>
2025-10-14 06:54:02 +00:00
Samuel Pitoiset
876e6a3bfe radv/rt: fix memory leak in lower_rt_instructions_monolithic()
Found with ASAN.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37844>
2025-10-14 06:54:02 +00:00
Emma Anholt
f7cbc7b1c5 radv: Allocate BOs as implicit sync even if the WSI is doing implicit sync.
As noted, the flag we allocate with controls whether *anyone* can implicit
sync on the BO through amdgpu interfaces, not just whether our fd does.
This restores radv to the behavior before the regressing commit.

Fixes: 4dcf32c56e ("wsi/drm: Don't request implicit sync if we're doing implicit sync ourselves.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37772>
2025-10-10 19:17:04 +00:00
Emma Anholt
38ac55ebff radv: Restore marking WSI image's mem->buffer as uncached.
Prior to 4dcf32c56e, radv was getting a request for implicit sync, even
when we were doing the work to do implicit sync in the WSI.  Once that was
turned off, we incidentally dropped flagging WSI's mem->buffer as
uncached, due to it being under the wrong condition.

Fixes: 4dcf32c56e ("wsi/drm: Don't request implicit sync if we're doing implicit sync ourselves.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37772>
2025-10-10 19:17:04 +00:00
Hans-Kristian Arntzen
2848901722 radv: Actually fail custom border color sampler creation.
Signed-off-by: Hans-Kristian Arntzen <post@arntzen-software.no>
Fixes: a52483d9e7 ("radv: fix capture/replay with sampler border color")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37787>
2025-10-10 14:25:54 +00:00
Samuel Pitoiset
183ed8046c radv: allow VK_FORMAT_S8_UINT with host image copy
Depth/stencil formats still need to be properly implemented.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37748>
2025-10-10 13:46:51 +00:00
Samuel Pitoiset
d063072182 radv: rename radv_mark_descriptor_sets_dirty()
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Descriptor heaps will be marked as dirty in this function too.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37786>
2025-10-10 13:22:05 +00:00
Samuel Pitoiset
34b3dae3b6 radv: make radv_descriptor_get_va() a static function
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37786>
2025-10-10 13:22:05 +00:00
Samuel Pitoiset
08dbab0600 radv: rename shader arg descriptor_sets to descriptors
It's more generic and descriptor heaps will use it too.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37786>
2025-10-10 13:22:03 +00:00
Samuel Pitoiset
609ae4e647 radv: rename indirect_descriptor_sets to indirect_descriptors
With descriptor heap the driver will also have to emit indirect
descriptor heaps in some cases.

Rename couple of things to make them more generic.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37786>
2025-10-10 13:22:03 +00:00
Samuel Pitoiset
0ff1ce4ac5 radv: use force_indirect_desc_sets when creating RT prologs
This is cleaner and this field has been added exactly for that.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37786>
2025-10-10 13:22:02 +00:00
Samuel Pitoiset
055b10a75c radv: do not initialize HiZ on transfer queue on RDNA4
Emitting compute dispatches on SDMA would just hang.

This fixes pending depth/stencil copy tests on transfer queue with
RADV_PERFTEST=transfer_queue.

Fixes: e6c485afb0 ("radv: initialize HiZ metadata during image layout transitions")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37795>
2025-10-10 12:50:02 +00:00
Samuel Pitoiset
aeec53f020 radv,radeonsi: use new ac_cmdbuf macros
But keep them behind existing macros for consistency until all macros
are moved to common code.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36292>
2025-10-08 18:00:15 +00:00
Samuel Pitoiset
902f5a8618 radv: replace radeon_cmdbuf by ac_cmdbuf completely
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36292>
2025-10-08 18:00:15 +00:00
Rhys Perry
c63c695149 radv: move nir_opt_algebraic loop for NGG culling earlier
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Totals from 6913 (8.66% of 79825) affected shaders: (Navi21)
Instrs: 5373319 -> 5358717 (-0.27%); split: -0.30%, +0.03%
CodeSize: 27448536 -> 27345464 (-0.38%); split: -0.41%, +0.03%
SpillSGPRs: 982 -> 998 (+1.63%)
Latency: 22998827 -> 23011602 (+0.06%); split: -0.13%, +0.19%
InvThroughput: 4663749 -> 4664809 (+0.02%); split: -0.00%, +0.03%
VClause: 120845 -> 120461 (-0.32%); split: -0.49%, +0.17%
SClause: 119068 -> 116064 (-2.52%); split: -2.71%, +0.18%
Copies: 456590 -> 456450 (-0.03%); split: -0.19%, +0.16%
Branches: 145555 -> 145559 (+0.00%); split: -0.00%, +0.01%
PreSGPRs: 300465 -> 301154 (+0.23%); split: -0.01%, +0.24%
VALU: 3064127 -> 3064210 (+0.00%); split: -0.00%, +0.00%
SALU: 891257 -> 886368 (-0.55%); split: -0.71%, +0.16%
SMEM: 190500 -> 184624 (-3.08%); split: -3.11%, +0.02%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36936>
2025-10-08 08:54:11 +00:00
Daniel Schürmann
2622a3bc47 radv,radeonsi: call ac_nir_lower_global_access and nir_lower_int64 for gs copy shaders
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36936>
2025-10-08 08:54:08 +00:00
Daniel Schürmann
50fcfe6bd8 radv: delay nir_opt_shrink_vectors
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36936>
2025-10-08 08:54:05 +00:00
Daniel Schürmann
c82d70d3ec radv: delay lowering int64
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36936>
2025-10-08 08:53:53 +00:00
Daniel Schürmann
eda8fc1e90 radv: delay lowering global access
Totals from 21 (0.03% of 79839) affected shaders: (Navi48)

Instrs: 30258 -> 30249 (-0.03%); split: -0.05%, +0.02%
CodeSize: 159660 -> 159552 (-0.07%); split: -0.07%, +0.01%
Latency: 188154 -> 188131 (-0.01%); split: -0.02%, +0.00%
SClause: 251 -> 252 (+0.40%)
SMEM: 619 -> 598 (-3.39%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36936>
2025-10-08 08:53:53 +00:00
Timur Kristóf
c473b0b551 radv/amdgpu: Allow IB2 when primary CS isn't chained
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
The primary CS doesn't need to use chaining in order to use IB2.
Allow using IB2 packets when chaining is disabled.

Rationale for this patch:
When chaining is enabled (the default), this patch removes a
useless check.
When chaining is disabled (by noibchaining), this patch allows us
to use IB2 without chaining.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37280>
2025-10-07 15:49:02 +00:00
Timur Kristóf
503963c08c radv/amdgpu: Support IB2 without chaining, enable on GFX6
GFX6 supports IB2 but not chaining within an IB2.

To use IB2 on GFX6, disable chaining in secondary CS,
and emit an IB2 packet for each secondary IB.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37280>
2025-10-07 15:49:02 +00:00
Timur Kristóf
2091db2461 radv/amdgpu: Small cleanup of counting submitted IBs
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37280>
2025-10-07 15:49:01 +00:00
Timur Kristóf
fd5c50664e radv/amdgpu: Emit a single 4 dword NOP in chainable CS buffers
This is a small optimization that should slightly reduce the CP
overhead for all GPUs as we now only emit a single NOP packet
instead of 4.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37280>
2025-10-07 15:49:01 +00:00
Timur Kristóf
e6a1355bd5 radv/amdgpu: Add a helper function to emit NOP packets
No functional changes, just make the code a bit easier to read.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37280>
2025-10-07 15:49:00 +00:00
Timur Kristóf
e20080315b radv/amdgpu: Don't assert chaining match when copying secondary IB
This assertion is useless.

In this code path it is not relevant whether or not the primary
CS support chaining. And it is already handled when the secondary
has chaining.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37280>
2025-10-07 15:49:00 +00:00
Timur Kristóf
df58cac660 radv: Rename RADV_DEBUG=noibs to noibchaining
Clarify what it actually means.
Also fix the documentation in envvars.rst to better describe it.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37280>
2025-10-07 15:48:59 +00:00
Timur Kristóf
3902cffab7 radv/amdgpu: Rename use_ib to chain_ib
All CS always use IBs, so the naming was confusing.

Rename these fields to chain_ib to better reflect
what it actually means, which is enabling chaining:
radv_amdgpu_winsys::use_ib_bos
radv_amdgpu_cs::chain_ib

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37280>
2025-10-07 15:48:59 +00:00
Samuel Pitoiset
c177bf81b4 radv: fix expected disk cache size for meta shaders
Math can go wrong.

If the disk cache size is too small, buckets are evicted and this
might cause stuttering when starting applications.

Fixes: 4fc856af98 ("radv: fix caching on-demand meta shaders")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13930
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37718>
2025-10-07 12:50:41 +00:00
Samuel Pitoiset
08ddf2f878 radv: lower embedded/immutable samplers earlier
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Lowering them earlier right after VTN would allow us to implement
embedded samplers for descriptor heap properly for merged shaders.

Non-immediate samplers are still lowered in
radv_nir_apply_pipeline_layout because they require shader arguments.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37688>
2025-10-07 09:25:28 +00:00