Commit graph

8017 commits

Author SHA1 Message Date
Tatsuyuki Ishi
c3c3a8926a radv/amdgpu: Separate the concept of residency from use_global_list.
A BO can be always resident by two ways:
1. Through kernel bookkeeping. The BO is created with
   AMDGPU_GEM_CREATE_VM_ALWAYS_VALID and bo->is_local gets set to true.
2. Through the driver global BO list. On every submission, the global
   BO list is added to the CS's BO list.

Until now, use_global_list reflected either 1. or 2. . This commit
changes it to reflect 2. only, and update callsites that checks for
residency to use a new helper.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26591>
2023-12-13 12:03:20 +00:00
Tatsuyuki Ishi
63120a55b8 radv/amdgpu: Remove virtual bo dump logic.
Virtual BOs cannot go into the global bo list. Accessing bo_handle is
also invalid for virtual BOs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26592>
2023-12-13 11:42:35 +00:00
Samuel Pitoiset
f576ce3340 radv: advertise VK_KHR_vertex_attribute_divisor
This is a promotion from the EXT, except the new property
supportsNonZeroFirstInstance which should already be supported.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26595>
2023-12-13 08:08:21 +00:00
Samuel Pitoiset
0605631094 radv: stop clearing FMASK_COMPRESS_1FRAG_ONLY for TC-compat CMASK images
TC-compat CMASK means Fmask decompression isn't needed because the hw
can read it directly from shaders, so this shouldn't have any effects.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26575>
2023-12-13 07:48:51 +00:00
Samuel Pitoiset
c70c269b16 radv: remove useless check for TC-compat CMASK images during fb emission
The FMASK decompression only happens for images with FMASK and without
TC-compat CMASK, so both can never be TRUE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26575>
2023-12-13 07:48:51 +00:00
Friedrich Vock
f1817ab7e0 radv,vtn,driconf: Add and use radv_rt_ssbo_non_uniform workaround for Crysis 2/3 Remastered
Crysis 2 and 3 Remastered's RT shaders non-uniformly index into SSBO
descriptor arrays without specifying the NonUniformEXT qualifier on the
relevant access chains/load ops. This leads to artifacts around objects.

To add insult to injury, the game fails to provide a meaningful
applicationName/engineName in the Vulkan part of the DX11-Vulkan interop
solution used for RT. Both of these fields are set to "nvpro-sample"
(perhaps the code has been copied from NVIDIA's sample applications).
Therefore, fall back to executable name matching.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9883
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26391>
2023-12-12 21:16:39 +00:00
Daniel Schürmann
dd7b6898e6 radv: fix number of physical SGPRs on GFX10+
This change has no effect.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26521>
2023-12-11 10:39:51 +00:00
Daniel Schürmann
f5bdc46a57 amd: rename max_wave64_per_simd -> max_waves_per_simd
These are hard limits and don't depend on wave size.
Accordingly, also update the usage in order to avoid
reporting unreasonable occupancy.

Totals from 192 (0.24% of 79330) affected shaders:

MaxWaves: 5814 -> 3072 (-47.16%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26521>
2023-12-11 10:39:50 +00:00
Samuel Pitoiset
ac20c70e9d radv: promote EXT_calibrated_timestamps to KHR
All functionality are similar.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26596>
2023-12-11 08:26:20 +01:00
Timur Kristóf
8e77da1f21 radv: Implement vkCmdUpdateBuffer on transfer queues.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26580>
2023-12-09 01:49:13 +00:00
Timur Kristóf
635c81f723 radv: Implement buffer copies on transfer queues.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26580>
2023-12-09 01:49:13 +00:00
Timur Kristóf
a632024700 radv: Implement vkCmdWriteBufferMarker2AMD on transfer queues.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26580>
2023-12-09 01:49:13 +00:00
Timur Kristóf
7a4f535145 radv: Implement vkCmdWriteTimestamp2 on transfer queues.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26580>
2023-12-09 01:49:13 +00:00
Timur Kristóf
32dd77ea2d radv: Implement vkCmdFillBuffer on transfer queues.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26580>
2023-12-09 01:49:13 +00:00
Timur Kristóf
23590a4e98 radv: Implement barriers for transfer queues.
The current flush flags in RADV don't really match the SDMA HW,
so just always emit a NOP packet, for now.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26580>
2023-12-09 01:49:13 +00:00
Timur Kristóf
1c8c3e5a7a radv: Don't retile DCC on transfer queues.
Instead, the retile will be executed on another queue type
when the image is transitioned to another queue.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25834>
2023-12-08 14:46:17 +00:00
Timur Kristóf
5c30d462b9 radv: Disable HTILE on exclusive images with transfer queues when SDMA doesn't support it.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25834>
2023-12-08 14:46:17 +00:00
Timur Kristóf
1764259ba8 radv: Disable DCC on exclusive images with transfer queue when SDMA doesn't support it.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25834>
2023-12-08 14:46:17 +00:00
Timur Kristóf
89a6b08cba radv: disable HTILE/DCC for concurrent images with transfer queue if unsupported.
DCC and HTILE are only supported by SDMA on GFX10+ (unless disabled by a workaround).

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25834>
2023-12-08 14:46:16 +00:00
Chia-I Wu
ad6b6673be radv: convert a check in radv_get_memory_fd to assert
VUID-VkBindImageMemoryInfo-memory-02628 and
VUID-VkBindImageMemoryInfo-memory-02629 make sure the memory offset is 0
for dedicated allocations.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25964>
2023-12-08 14:21:42 +00:00
Chia-I Wu
8aa62ba240 radv: fix asserts for radv_init_metadata
radv_init_metadata hits several assert failures when the image is
multi-planar.  Make sure we use plane 0.

This change should make no difference in practice.  Also, this is done
only to follow radeonsi.  Since the opaque metadata is mainly for
validations and DCC, and we don't enable DCC for multi-planar images, we
probably don't need to call radv_query_opaque_metadata at all.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25964>
2023-12-08 14:21:42 +00:00
Chia-I Wu
035cf7ab97 radv: fix a typo in radv_image_view_make_descriptor
Only GFX8 and before have legacy_surf_level.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25964>
2023-12-08 14:21:42 +00:00
Chia-I Wu
07f575a8a6 radv: fix VkSubresourceLayout2KHR for multi-planar formats with modifiers
Memory planes and format planes are equivalent for multi-planar formats
with modifiers.  Do not return the DCC info of plane 0.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25964>
2023-12-08 14:21:42 +00:00
Chia-I Wu
8f60ccf969 radv: fix VkDrmFormatModifierProperties2EXT for multi-planar formats
Do not report DCC modifiers for multi-planar formats.  We don't support
DCC for them and drmFormatModifierPlaneCount had incorrect values.

Fix vkGetImageSubresourceLayout for multi-planar images with modifiers.
In that case, memory planes and format planes are equivalent.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25964>
2023-12-08 14:21:42 +00:00
Samuel Pitoiset
90dda31901 radv: simplify disabling MRT compaction for PS epilogs
If the fragment shader isn't compiled, the PS epilog key isn't used
at all with GPL.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26563>
2023-12-08 13:52:40 +00:00
Juston Li
dc7c1d989b radv: enable stippledBresenhamLines on GFX9 chips
This isn't supposed to work nor does it match radeonsi but setting
AUTO_RESET_CNTL=0 by default for GFX9 chips is what gets it passing
linestrip CTS tests:

dEQP-VK.rasterization.primitives.dynamic_stipple.bresenham_line_strip
dEQP-VK.rasterization.primitives.dynamic_stipple_and_topology.bresenham_line_strip
dEQP-VK.rasterization.primitives.dynamic_stipple_and_topology.bresenham_line_strip_wide
dEQP-VK.rasterization.primitives.static_stipple.bresenham_line_strip

Signed-off-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24623>
2023-12-07 19:10:15 +00:00
Tatsuyuki Ishi
b2e8095d6d radv: Add workaround to allow sparse binding on gfx queues.
For working around improper usage of sparse in DOOM Eternal.

When fully explicit sync sparse binding is implemented, this path will
remain implicit sync to also deal with the improper semaphore usage.
radv_queue_submit_bind_sparse_memory will likely get a bool parameter to
control explicit / implicit sync in that case.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26464>
2023-12-07 11:40:38 +00:00
Faith Ekstrand
09fc5e1c4d nir: Split has_[su]dot_4x8 bits into regular and _sat versions
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26533>
2023-12-06 23:15:33 +00:00
Samuel Pitoiset
47f666b64b radv: enable extendedDynamicState3AlphaToCoverageEnable on GFX11
This is now supported and this allows Zink to enable full ds3 mode for
even less stuttering. DXVK also uses dynamic alpha to coverage
sometimes.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26250>
2023-12-06 16:36:16 +00:00
Samuel Pitoiset
bca4ebc3c4 radv: export MRTZ via PS epilogs when alpha to coverage is dynamic on GFX11
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26250>
2023-12-06 16:36:16 +00:00
Samuel Pitoiset
79cf2b6824 radv: determine and emit SPI_SHADER_Z_FORMAT for PS epilogs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26413>
2023-12-06 11:49:31 +00:00
Samuel Pitoiset
4248d0b03a radv,aco: declare PS epilog VGPR arguments for depth/stencil/samplemask
Depth/stencil/samplemask inputs are first to match
create_fs_jump_to_epilog().

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26413>
2023-12-06 11:49:31 +00:00
Samuel Pitoiset
8b87c985b0 radv: prepare the PS epilog key for exporting MRTZ on RDNA3
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26413>
2023-12-06 11:49:31 +00:00
Samuel Pitoiset
5b01285cfb radv: determine if MRTZ needs to be exported via PS epilogs
For GFX11 only.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26413>
2023-12-06 11:49:31 +00:00
Samuel Pitoiset
a3b3a7d36a radv: remove unused code for compiling PS epilogs as part of pipelines
Since we switched to the "on-demand" path for GPL, this is dead code.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26398>
2023-12-06 08:01:46 +00:00
Samuel Pitoiset
725fc0ec03 radv: switch to on-demand PS epilogs for GPL
RADV currently has two paths for PS epilogs:
- the first one is mostly used by GPL to compile fragment shader epilogs
  as part of the graphics pipeline. It's supposed to be optimal because
  fragment shader epilogs are compiled in the pipeline and eventually
  cached.
- the second one (the "on-demand" path) is required when some dynamic
  states are used because otherwise it's just impossible to compile the
  fragment shader. These epilogs are compiled during cmdbuf recording
  when all needed info are known, they are also cached in memory. This
  is the main path for Zink.

Having two different paths isn't ideal for maintenance but there is
another problem. On RDNA3, alpha to coverage needs to be exported as
part of MRTZ when either depth/stencil/samplemask are exported. The
problem being that with GPL, the PSO multisample state can be NULL when
the frag shader lib is created, which means that we can't know if atc
needs to be exported or not, even if it's static. The solution seems to
to always use on-demand fragment shader epilogs for GPL on RDNA3.

So far, I think that switching to on-demand PS epilogs unconditionally
for GPL shouldn't hurt performance and that will simplify a lot of
things.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26398>
2023-12-06 08:01:45 +00:00
Samuel Pitoiset
3678482f1e radv: set combinedImageSamplerDescriptorCount to 1 for multi-planar formats
RADV compacts all descriptors for multi-planar images into one
combined image sampler, so it should be 96, and not eg. 192 for a two
planes format.

Fixes new CTS
dEQP-VK.binding_model.descriptor_buffer.ycbcr_sampler.*array.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26498>
2023-12-05 22:27:19 +00:00
Samuel Pitoiset
13dcbdcdf3 radv: make sure to reset the stipple line state when it's disabled
In a scenario like:
CmdBindPipeline(stipple line enabled)
CmdDraw()
CmdBindPipeline(stipple line disabled)
CmdDraw()

The second draw wasn't resetting the stipple line state and this might
have caused issues, though it's uncovered by VK CTS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26429>
2023-12-05 18:29:30 +00:00
Samuel Pitoiset
e88973fd02 radv: change the reset stipple pattern mode for adjacent lines
Ported from RadeonSI. This isn't covered by VK CTS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26429>
2023-12-05 18:29:29 +00:00
Samuel Pitoiset
16d5ffc3ee radv: update the reset stipple pattern mode
PAL recently changed the mode. This doesn't fix anything known.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26429>
2023-12-05 18:29:29 +00:00
Samuel Pitoiset
949fdc9a73 radv: advertise VK_EXT_depth_clamp_zero_one
RADV already implements this behavior, so enabling this extension is
a no-op.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26443>
2023-12-05 18:10:31 +00:00
Samuel Pitoiset
338319741c radv: add DGC support for mesh shader only
This only implements mesh shaders with DGC because task shaders are
really tricky. I will address them later.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25890>
2023-12-05 14:17:39 +00:00
Samuel Pitoiset
eb3e1bdfe6 radv: only initialize the VTX base SGPR if non-zero with DGC
Otherwise, its value is incorrect.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25890>
2023-12-05 14:17:39 +00:00
Samuel Pitoiset
1deedc70db radv: only initialize the VBO reg if VBOs are bound with DGC
With mesh shader there is no VBO at all.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25890>
2023-12-05 14:17:39 +00:00
Samuel Pitoiset
400cfa0eba radv: remove never used binds_state for DGC
This has been removed a while ago.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25890>
2023-12-05 14:17:39 +00:00
Yonggang Luo
83a5fb9faf util: Fixes note: the alignment of ‘_Atomic long long int’ fields changed in GCC 11.
This is a improve of https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22121

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23961>
2023-12-05 09:26:08 +00:00
Friedrich Vock
d6d68ceda1 radv: Enable compute dispatch tunneling
Compute tunneling can considerably lower the latency of high-priority
compute work. Enabling it is beneficial in cases where high-priority
work is dispatched while the GPU is already busy with other work (e.g.
rendering on GFX). This is the case in VR compositors that dispatch
latency-sensitive compositing work to ACE while GFX is busy rendering
the next frame.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26462>
2023-12-04 12:32:47 +00:00
Eric Engestrom
778000ec7f radv: update symbols that have become aliases for newer ones
All of these have been renamed in the spec (usually by being promoted);
renamed them in our code too.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26490>
2023-12-04 10:45:48 +00:00
Felix bridault
059391b631 radv: use 32bit va range for sparse descriptor buffers
Fixes: 5c5735fd68 ("radv: advertise VK_EXT_descriptor_buffer")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26482>
2023-12-04 09:59:29 +00:00
Samuel Pitoiset
9027c6d8ca radv: adjust assertions for multi-layer resolves with the HW/FS paths
Only compute supports layers.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26316>
2023-12-04 08:12:16 +00:00