Commit graph

391 commits

Author SHA1 Message Date
Timur Kristóf
292460670a ac/gpu_info: Fix determining when CP DMA supports sparse
Change has_cp_dma_with_null_prt_bug to cp_dma_supports_sparse
to know when CP DMA supports sparse. CP DMA doesn't support
sparse on any gfx6-9 chip.

Sources:
- d2669628 already documented this on gfx6 in 2018
- e259f405 added a radeonsi workaround for gfx9 in 2023
- 235f70e4 added a radv workaround for Polaris in 2025

Now RADV will use compute copy and fill for sparse resources
on all gfx6-9 chips (previously only did on Polaris and newer).

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38553>
2025-11-25 10:38:45 +01:00
Samuel Pitoiset
0dba538643 radv/meta: fuse depth/stencil aspects copy with the GFX path
Depth/stencil copies on graphics are twice as fast now.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38139>
2025-11-12 07:35:33 +00:00
Samuel Pitoiset
332f881375 radv/meta: simplify aspect/formats in radv_gfx_copy_image()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38139>
2025-11-12 07:35:32 +00:00
Samuel Pitoiset
cd59db45f9 radv/meta: simplify radv_gfx_copy_memory_to_image() even more
Selecting formats can be simplified.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38139>
2025-11-12 07:35:32 +00:00
Samuel Pitoiset
ed05c3fc31 radv/meta: remove multiple aspects in radv_gfx_copy_memory_to_image()
Only one aspect at any time is valid.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38139>
2025-11-12 07:35:31 +00:00
Samuel Pitoiset
a1884dc737 radv/meta: remove radv_meta_blit2d_rect
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38139>
2025-11-12 07:35:31 +00:00
Samuel Pitoiset
1319b2bef6 radv/meta: split radv_meta_blit2d() into two separate functions
It's more code but it's definitely easier to read and it will allow us
to do more cleanups/optimizations.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38139>
2025-11-12 07:35:30 +00:00
Samuel Pitoiset
bb3f69fefe radv/meta: remove useless blit2d_src_temps
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38139>
2025-11-12 07:35:29 +00:00
Samuel Pitoiset
968fb06a94 radv,vulkan: replace VK_RENDERING_INPUT_ATTACHMENT_NO_CONCURRENT_WRITES_BIT_MESA
The new flag from maintenance10 has similar meaning.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38043>
2025-10-31 07:51:23 +00:00
Samuel Pitoiset
14639898d0 radv: add support for controlling sRGB transfer function with resolves
Just need to use UNORM image views.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38043>
2025-10-31 07:51:22 +00:00
Samuel Pitoiset
d3924f5bd6 radv: add support for depth/stencil resolves with vkCmdResolve2()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38043>
2025-10-31 07:51:20 +00:00
Samuel Pitoiset
d5d2a4ad07 radv: implement vkCmdEndRendering2KHR()
Common runtime code already does CmdEndRendering()->CmdEndRendering2().

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38043>
2025-10-31 07:51:19 +00:00
Konstantin Seurer
c18a7d0e2b radv: Emit compressed primitive nodes on GFX12
The normal encode pass writes batches to a section in build scratch
memory. Those batches contain information about the internal node and
the primitive nodes. The encoder is split to avoid the register
pressure of the compressor and maximize occupancy.

The compressor works in two passes because one pass can not guarantee
that every primitive node (except) has at least two triangles. This
guarantee is used to advertise a smaller acceleration structure size to
the application.

During compression, every invocation processes at most two triangles.
Groups of 8 invocations are used to support the maximum triangle count
of 16 that the hardware supports.

The first step of compression is loading the triangle(s). Shared
vertices are deduplicated early to avoid doing it in the compression
loop. The compression loop tries to add triangles to a list of triangles
until the computed node size needed for storing the triangles reaches
the hardware node size. For this, each invocation first deduplicates
vertices with the triangles that have already been picked. It then
computes the node size of the picked triangles plus the candidate
triangles of the current invocation. The invocation that computed the
smallest size is added to the list.

Because it may not be possible to fit every triangle into the same node,
there can be multiple hardware nodes which are written in parallel for
optimal performance. If there are no nodes with only one triangle, all
nodes are written. If there is, compression of the batch is aborted and
the index of the batch is written to build scratch memory. The second
compression pass will repeat the steps above but only for those aborted
batches. The nodes with only one triangle can and are now merged.

It can not be determined during box node encode which triangles will be
compressed together so the encoder also has to fix up the parent box
node's child infos.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36965>
2025-10-21 19:32:55 +00:00
Samuel Pitoiset
4989b6e6b9 amd,radv,radeonsi: add ac_emit_cp_write_data_{head}()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37881>
2025-10-21 13:31:20 +02:00
Timur Kristóf
a57419f96b radv: Clarify image and image/buffer copy helper functions
Refactor the functions to make it clear what they do:
copy_memory_to_image -> gfx_or_compute_copy_memory_to_image
copy_image_to_memory -> compute_copy_image_to_memory
copy_image -> gfx_or_compute_copy_image

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:12 +00:00
Timur Kristóf
db4a9aaf29 radv: Call transfer copy functions from API functions, not helpers
Improves code readability.
No functional changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37775>
2025-10-14 12:33:12 +00:00
Samuel Pitoiset
df269714ef radv/meta: remove radv_cmd_buffer_resolve_rendering_{hw,cs,fs}
Just call the other functions directly.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37792>
2025-10-14 07:46:13 +00:00
Samuel Pitoiset
a81f01bc96 radv/meta: pass iview formats for subpass resolves
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37792>
2025-10-14 07:46:13 +00:00
Samuel Pitoiset
d3e716f1fb radv/meta: re-use radv_meta_resolve_{fragment,hardware}_image() for subpass resolves
Similar to compute.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37792>
2025-10-14 07:46:13 +00:00
Samuel Pitoiset
ac3c21f130 radv/meta: pass image formats to radv_meta_resolve_{hardware,fragment}_image()
Similar to the compute function.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37792>
2025-10-14 07:46:12 +00:00
Samuel Pitoiset
8d991c2572 radv/meta: remove useless assertion when choosing resolve method
The destination image layout is used for depth/stencil resolves and
asserting isn't very useful.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37300>
2025-09-15 18:52:55 +00:00
Samuel Pitoiset
c8f6b27964 radv/meta: simplify calling depth/stencil resolve helpers
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37300>
2025-09-15 18:52:55 +00:00
Samuel Pitoiset
39725fc935 radv/meta: simplify barriers for resolves
This is equivalent.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37300>
2025-09-15 18:52:54 +00:00
Samuel Pitoiset
e673ccfcb5 radv/meta: remove useless VK_ACCESS_2_SHADER_WRITE_BIT for subpass resolves
This doesn't do anything.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37300>
2025-09-15 18:52:54 +00:00
Samuel Pitoiset
704fbbb108 radv/meta: rework depth/stencil resolves using graphics
This adds a new helper that doesn't depend on the rendering info.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37300>
2025-09-15 18:52:53 +00:00
Samuel Pitoiset
141beaee4e radv/meta: rework depth/stencil resolves using compute
This adds a new helper that doesn't depend on the rendering info.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37300>
2025-09-15 18:52:53 +00:00
Samuel Pitoiset
2207d1e732 radv/meta: fix saving push constants for depth/stensil resolves on compute
Found by inspection.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37300>
2025-09-15 18:52:52 +00:00
Faith Ekstrand
1897d5d9c9 radv: Use VK_IMAGE_VIEW_CREATE_DRIVER_INTERNAL_BIT_MESA
This does mean having to set the flag everywhere, which is a bit
annoying, but I don't think I missed any.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36957>
2025-09-05 23:34:12 +00:00
Samuel Pitoiset
1b6aad9def radv/meta: use radv_CmdDispatchBase() directly for ASTC decode
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37141>
2025-09-05 09:21:25 +00:00
Konstantin Seurer
be4be884e1 radv: Rename radv_printf files to radv_debug_nir
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34392>
2025-08-15 10:32:34 +00:00
Samuel Pitoiset
3de108da66 radv/meta: update HiZ metadata after depth/stencil image clears
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36739>
2025-08-12 13:48:10 +00:00
Samuel Pitoiset
297cf6f1aa radv/meta: add a pass to clear HiZ surfaces
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36739>
2025-08-12 13:48:09 +00:00
Samuel Pitoiset
3ccb48ec46 radv: switch to radv_cmd_stream everywhere
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36314>
2025-08-08 11:49:23 +00:00
Antonio Ospite
ddf2aa3a4d build: avoid redefining unreachable() which is standard in C23
In the C23 standard unreachable() is now a predefined function-like
macro in <stddef.h>

See https://android.googlesource.com/platform/bionic/+/HEAD/docs/c23.md#is-now-a-predefined-function_like-macro-in

And this causes build errors when building for C23:

-----------------------------------------------------------------------
In file included from ../src/util/log.h:30,
                 from ../src/util/log.c:30:
../src/util/macros.h:123:9: warning: "unreachable" redefined
  123 | #define unreachable(str)    \
      |         ^~~~~~~~~~~
In file included from ../src/util/macros.h:31:
/usr/lib/gcc/x86_64-linux-gnu/14/include/stddef.h:456:9: note: this is the location of the previous definition
  456 | #define unreachable() (__builtin_unreachable ())
      |         ^~~~~~~~~~~
-----------------------------------------------------------------------

So don't redefine it with the same name, but use the name UNREACHABLE()
to also signify it's a macro.

Using a different name also makes sense because the behavior of the
macro was extending the one of __builtin_unreachable() anyway, and it
also had a different signature, accepting one argument, compared to the
standard unreachable() with no arguments.

This change improves the chances of building mesa with the C23 standard,
which for instance is the default in recent AOSP versions.

All the instances of the macro, including the definition, were updated
with the following command line:

  git grep -l '[^_]unreachable(' -- "src/**" | sort | uniq | \
  while read file; \
  do \
    sed -e 's/\([^_]\)unreachable(/\1UNREACHABLE(/g' -i "$file"; \
  done && \
  sed -e 's/#undef unreachable/#undef UNREACHABLE/g' -i src/intel/isl/isl_aux_info.c

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36437>
2025-07-31 17:49:42 +00:00
Konstantin Seurer
d59c22b6e1 radv/rt: Implement null acceleration structure in shader code
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
The previous approach is broken with descriptor buffer capture/replay
because the address off the dummy VA used can randomly change.

Totals from 78 (20.58% of 379) affected shaders:

Instrs: 3837275 -> 3839653 (+0.06%); split: -0.01%, +0.07%
CodeSize: 20235104 -> 20251744 (+0.08%); split: -0.01%, +0.09%
SpillSGPRs: 997 -> 1007 (+1.00%)
Latency: 22305937 -> 22331551 (+0.11%); split: -0.03%, +0.15%
InvThroughput: 4232313 -> 4237341 (+0.12%); split: -0.03%, +0.15%
VClause: 97043 -> 97027 (-0.02%); split: -0.02%, +0.01%
SClause: 72169 -> 72416 (+0.34%); split: -0.00%, +0.35%
Copies: 321578 -> 322126 (+0.17%); split: -0.11%, +0.28%
Branches: 110163 -> 110444 (+0.26%); split: -0.00%, +0.26%
PreSGPRs: 7879 -> 7942 (+0.80%)
VALU: 2155040 -> 2156425 (+0.06%); split: -0.02%, +0.09%
SALU: 502292 -> 503078 (+0.16%); split: -0.00%, +0.16%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36034>
2025-07-19 21:02:42 +00:00
Samuel Pitoiset
ea742877f6 radv: re-run clang-format
For style consistency.

$ clang-format -i $(find src/amd/vulkan/ -name "*.h" -o -name "*.c" -o -name "*.cpp")

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36118>
2025-07-16 09:10:33 +02:00
Samuel Pitoiset
36e4b52e9f radv: do not perform a per-pixel copy for BCn formats with mips on GFX12+
This is unnecessary because GFX12 isn't affected by this clamping issue
when NO_EDGE_CLAMP is correctly configured.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36043>
2025-07-11 05:46:50 +00:00
Samuel Pitoiset
71397a8162 radv/meta: stop allocating sampler for blit operations
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35790>
2025-07-01 08:58:03 +02:00
Samuel Pitoiset
5e2fcdfea2 radv/meta: add a helper to determine if clearing is a full rect
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35603>
2025-06-20 06:36:19 +00:00
Samuel Pitoiset
203aacf064 radv/meta: use radv_get_copy_flags_from_bo() more
Cleanups.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35550>
2025-06-17 06:16:07 +00:00
Samuel Pitoiset
ee200cc0d1 radv: stop using vk_common entrypoints when not necessary
For less indirections.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35359>
2025-06-11 07:10:02 +00:00
Samuel Pitoiset
f3578973d7 radv/meta: fix using the wrong pipeline layout for ASTC decoding
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35359>
2025-06-11 07:10:01 +00:00
Marek Olšák
c3034fa82c amd: replace most u_bit_consecutive* with BITFIELD_MASK/RANGE
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35346>
2025-06-04 17:46:38 +00:00
Samuel Pitoiset
25eb836eec radv: fix CP DMA with NULL PRT pages on GFX8-9
On GFX8-9 (starting from Polaris10), CP DMA is broken with NULL PRT
pages. It doesn't read 0 and doesn't discard writes which can cause
GPU hangs.

Fix that by always using the compute path when a BO is sparse.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12828
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35071>
2025-05-21 09:41:23 +00:00
Samuel Pitoiset
7ce7009ee4 radv/meta: move and rename get_r32g32b32_format()
For future work.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34971>
2025-05-20 13:30:07 +00:00
Samuel Pitoiset
b7ce612743 radv: add vk_format_is_96bit()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34971>
2025-05-20 13:30:07 +00:00
Samuel Pitoiset
3ca2f71f3d radv: fix conditional rendering with DGC and non native 32-bit predicate
When the hardware doesn't natively support 32-bit predication, the
driver has a fallback which allocates a 64-bit predicate to the upload
BO in order to copy the original value.

But when conditional rendering is enabled in the stateCommandBuffer
which is used by preprocess() and the execute() is recorded also in the
stateCommandBuffer. If the preprocess() is recorded in a different
cmdbuf which is submitted before the cmdbuf that contains execute(),
the fallback (ie. alloc + COPY_DATA) will be performed after. This would
cause the predicate value to be always 0.

To fix that, keep track of the user predication VA which is the only
VA that needs to be used by DGC because it reads 32-bit from the shader.

This fixes a very weird corner case with vkd3d-proton.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13143
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34953>
2025-05-15 05:51:04 +00:00
Konstantin Seurer
c6fdf11303 radv: Make radv_update_memory non-static
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34601>
2025-05-12 17:45:25 +02:00
Konstantin Seurer
c21e1776b3 radv: Use build flags instead of defines
Using the meta framework makes managing shader variants much easier.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34594>
2025-05-09 09:55:32 +00:00
Samuel Pitoiset
0684dc5fa8 radv: fix GPU hangs with image copies for ASTC/ETC2 formats on transfer queue
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Emitting compute dispatches on SDMA just hangs. It might be needed
to switch to gang submit for these to work but fixing the GPU hang is
more important for now.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34805>
2025-05-05 13:50:25 +00:00