Commit graph

177 commits

Author SHA1 Message Date
Samuel Pitoiset
78c2feed00 radv/meta: rename more buffer->memory for fill/copy/update operations
Recently, I renamed most of the helpers for future work but I forgot
few things like meta keys, etc.

This is for consistency.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34558>
2025-04-18 17:21:24 +02:00
Konstantin Seurer
76031ba53d radv: Optimize the gfx12 encode shader
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
97f6287827 radv: Use the BVH8 format on GFX12
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
2dee1117b7 vulkan: Add a vk_device parameter to get_encode_key
Useful for selecting different encoding options based on hardware
generation.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Samuel Pitoiset
e616761fb2 radv: re-introduce the compute vs CP DMA heuristic for copy/fill operations
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
This caused a -5% performance regression in Control because using
compute always eats resources.

This new approach introduces a flag called RADV_COPY_FLAGS_DEVICE_LOCAL
which can be used to indicate if the underlying memory is device local.
This should also help for future work.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12639
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34556>
2025-04-17 08:59:58 +00:00
Samuel Pitoiset
5e2508e7c4 radv: simplify radv_fill_xxx() helpers
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34556>
2025-04-17 08:59:58 +00:00
Samuel Pitoiset
0fa43b5bfb radv: use radv_fill_memory() in the accel struct path
It's now possible to remove the NULL BO check.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34556>
2025-04-17 08:59:58 +00:00
Konstantin Seurer
676e26aed5 radv: Fix rayTracingPositionFetch with multiple geometies
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
The fix adds more indirections to avoid increasing register pressure by
tracking the primitive address.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34460>
2025-04-11 22:26:08 +00:00
Natalie Vock
c1e1d86bd1 radv/rt: Flush CP writes from the common BVH framework with INV_L2 on GFX12
a1b05991 ("radv/rt: Flush L2 after writing internal node offset on GFX12")
did this for radv-internal CP writes - we also need to do this for PLOC
sync data initialization which is done in the common framework.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34178>
2025-03-28 23:07:17 +00:00
Lionel Landwerlin
602843a881 vulkan/runtime: rely on vk_buffer::device_address
With all the RT-enabled driver setting this field, we can now have the
runtime use it instead of calling into the driver's vfunc.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34026>
2025-03-13 14:44:06 +00:00
Natalie Vock
a1b0599105 radv/rt: Flush L2 after writing internal node offset on GFX12
Otherwise the encoder can read a stale value and make internal nodes
point into leaf space (if 0 is read).

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33985>
2025-03-10 17:42:05 +00:00
Samuel Pitoiset
0323ed1022 radv: use radv_copy_memory() for accel structure updates
VK_KHR_acceleration_structure requires the BDA feature to be enabled
and this enables the global BO list.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33599>
2025-02-19 10:30:36 +00:00
Samuel Pitoiset
361a598b4c radv: use radv_CmdDispatchIndirect() in the accel struct path
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33599>
2025-02-19 10:30:36 +00:00
Natalie Vock
f01623ea75 radv/bvh: Add custom leaf node builder
This custom builder implements fine-grained instance node bounds
calculation by looking at all AABBs at tree depth 2.

Shaves off 0.3ms in the start scene for Indiana Jones: The Great Circle
on Deck (roughly 29.1ms->28.7ms).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32797>
2025-02-18 13:00:53 +00:00
Timur Kristóf
666f10c174 radv: Add missing includes and remove unnecessary NIR includes.
RADV won't compile without the added includes after we
stop including the full nir.h from the VK common functions.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33439>
2025-02-12 22:33:07 +01:00
Samuel Pitoiset
8df1ffaa78 radv: use radv_buffer::addr more
And remove radv_buffer:offset.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33475>
2025-02-11 15:12:34 +00:00
Samuel Pitoiset
f70af40c5d radv: pass addr to radv_copy_buffer()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33475>
2025-02-11 15:12:34 +00:00
Connor Abbott
47a85815b0 radv: Delete acceleration structure stubs
These are now provided by common code.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33153>
2025-01-23 05:16:58 +00:00
Samuel Pitoiset
3be1e9ee4d radv: add support for VkMemoryBarrierAccessFlags3KHR
There is no flags yet.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33091>
2025-01-21 07:28:14 +00:00
Feng Jiang
701600fb11 radv/rt: Fix memleak in radv_init_header()
Fixes: f8b584d ("vulkan/runtime,radv: Add shared BVH building framework")
Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32887>
2025-01-07 09:49:56 +00:00
Konstantin Seurer
407186994c radv: Fix encoding empty acceleration structures
There will always be at least one internal node so the expectation that
there are more leaf nodes than internal nodes dows not hold when the
acceleration structure has no leaf nodes.

Fixes: f8b584d ("vulkan/runtime,radv: Add shared BVH building framework")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12302
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32820>
2025-01-02 13:07:33 +01:00
Konstantin
815ca049cd vulkan: Fix the argument order of update_as
Also moves the src argument before dst which is more consistent.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32488>
2024-12-12 11:15:08 +00:00
Samuel Pitoiset
310e03c8d3 radv/meta: move vk_meta_device_init() to radv_device_init_meta()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32571>
2024-12-11 07:25:29 +00:00
Lionel Landwerlin
877a296b2c vulkan: make acceleration structure debug markers virtual
Not every implementation supports VK_EXT_debug_marker.
VK_EXT_debug_utils is also pretty similar, it would be nice to plug
into whatever is available.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kevin Chuang <kaiwenjon23@gmail.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32483>
2024-12-09 14:45:00 +00:00
Samuel Pitoiset
54cd43f93e radv: promote VK_KHR_maintenance5 to core 1.4 API
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32432>
2024-12-03 10:21:55 +00:00
Connor Abbott
f8b584d6a5 vulkan/runtime,radv: Add shared BVH building framework
This is mostly adapted from radv's BVH building. This defines a common
"IR" for BVH trees, two algorithms for constructing it, and a callback
that the driver implements for encoding. The framework takes care of
parallelizing the different passes, so the driver just has to split the
encoding process into "stages" and implement just one part for each
stage.

The runtime changes are:
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
The radv changes are;

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31433>
2024-12-01 16:08:06 +00:00
Samuel Pitoiset
7a3a65c0c4 radv: pass the image subresource range to radv_{src,dst}_access_flush()
This will allow us to optimize the pipe misaligned special case for
GFX11 because only the first mip in the mip-tail needs the L2 cache
invalidation.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31921>
2024-11-12 17:27:39 +00:00
Konstantin Seurer
3acab3dfff radv: Use vk_expand_(src|dst)_access_flags2
Simplifies access flags handling since the driver doesn't have to worry
about VK_ACCESS_2_MEMORY_READ_BIT and friends.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29051>
2024-06-28 10:41:49 +00:00
Konstantin Seurer
14f7b077c8 radv: Remove dead access bits
READ access bits are dead as radv_src_access_flush arguments and WRITE
access bits are dead as radv_dst_access_flush arguments.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29780>
2024-06-21 12:52:39 +00:00
Konstantin Seurer
03483ecb11 radv: Destroy leaf_updateable_pipeline
The pipeline was never destroyed. Fixes an assert in ac_sqtt_finish.

Fixes: 217072d ("radv/rt: Force active leaves for every updateable accel struct")
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28557>
2024-04-10 08:23:14 +00:00
Samuel Pitoiset
87fde606cf radv: use SPDX-License-Identifier
Only files under src/amd/vulkan/** are concerned.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28599>
2024-04-08 07:17:31 +00:00
Samuel Pitoiset
69b911bc53 radv: remove radv_private.h
This file was a giant mess and I'm very happy to remove it entirely.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28568>
2024-04-04 21:57:46 +00:00
Samuel Pitoiset
0388df3d08 radv: replace RADV_FROM_HANDLE by VK_FROM_HANDLE
It was exactly the same thing.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28568>
2024-04-04 21:57:45 +00:00
Samuel Pitoiset
c1414a9799 radv: add radv_sqtt.h
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28551>
2024-04-04 16:35:15 +00:00
Samuel Pitoiset
675d7f4bd0 radv: remove radv_cmd_buffer::device
Get the logical device object using the base object.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28500>
2024-04-01 16:35:09 +00:00
Samuel Pitoiset
896c9cf486 radv: remove radv_device::physical_device
Get the logical device object using the base object.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28500>
2024-04-01 16:35:09 +00:00
Konstantin Seurer
d902b6d805 radv: Skip more acceleration structure build markers
We should skip even more stuff when using updates only.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28228>
2024-03-19 19:05:25 +00:00
Friedrich Vock
217072d25f radv/rt: Force active leaves for every updateable accel struct
We can't rely on games getting updates right. To avoid adding
workarounds for tons of games, be more robust by default.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28077>
2024-03-11 17:08:09 +00:00
Konstantin Seurer
6d2926c54e radv: Skip unused acceleration structure build paths
If the whole batch doesn't use a path, we can skip binding the pipeline
and writing an user event.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26697>
2024-01-31 10:24:20 +01:00
Konstantin Seurer
93b1eec09b radv: Emit user events during acceleration structure builds
This way, we can see exactly which dispatches belong to acceleration
structure builds and which dispatches were caused by the app. We loose
the red coloring in RGP for this, but we get way more information.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26697>
2024-01-31 10:24:15 +01:00
Konstantin Seurer
f43099ecf5 radv: Reduce the amount of radv_device_to_handle calls
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26697>
2024-01-31 10:24:06 +01:00
Friedrich Vock
a9831caa14 radv/rt: Add workaround to make leaves always active
DOOM Eternal builds acceleration structures with inactive primitives and
tries to make them active in later AS updates. This is disallowed by the
spec and triggers a GPU hang. Fix the hang by working around the bug.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27034>
2024-01-17 15:14:48 +00:00
Friedrich Vock
0b55a3cf64 radv/rt: Acceleration structure updates
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26729>
2024-01-09 18:47:45 +00:00
Friedrich Vock
62fe4f0b1b radv/rt: Move per-geometry build info into a geometry_data struct
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26729>
2024-01-09 18:47:45 +00:00
Konstantin Seurer
2e4951d3fb radv: Remove the BVH depth heuristics
It only helps Quake II RTX and hurts everything else.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26481>
2024-01-09 09:00:24 +00:00
Konstantin Seurer
719619c477 radv: Use PLOC for TLAS builds
Improves control performance by about 1%.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26481>
2024-01-09 09:00:24 +00:00
Samuel Pitoiset
744cb98bc6 radv: add support for version 2 of all descriptor binding commands
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26757>
2024-01-03 11:24:27 +00:00
Konstantin Seurer
662f86c533 radv/bvh: Stop emitting leaf nodes inside the encoder
Avoids unnecessary copies.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26725>
2023-12-18 19:17:21 +00:00
Konstantin Seurer
8fca54a754 radv: Add more offsets acceleration_structure_layout
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26725>
2023-12-18 19:17:21 +00:00
Konstantin Seurer
ca92a10800 radv: Do not sync after radv_update_buffer_cp
WR_CONFIRM is set.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24846>
2023-09-30 11:49:14 +00:00