Commit graph

130 commits

Author SHA1 Message Date
Konstantin Seurer
906b541567 radv/bvh: Copy parent_id during updates on GFX12
Fixes: cc0dc4b5 ("radv: Store parent node IDs inside nodes on GFX12")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13567
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36898>
2025-08-24 13:45:29 +00:00
Konstantin Seurer
cc0dc4b566 radv: Store parent node IDs inside nodes on GFX12
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Saves some space.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36691>
2025-08-15 13:00:32 +00:00
Konstantin Seurer
c4b18c689f radv: Emit compressed primitive nodes on GFX12
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Emits two triangles per node whenever possible. The nir code will
revisit the triangle node to handle the second triangle only if both
triangles are interescted by the ray.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35734>
2025-08-07 20:23:15 +00:00
Konstantin Seurer
48d15c3cf8 radv/bvh: Specialize the update shader for geometryCount==1
The geometry data can be loaded from push constants in that case.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:13 +00:00
Konstantin Seurer
b20ab07e4a radv/bvh: Update leaf nodes before refitting
This should reduce latency between refitting nodes and their parent
nodes.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:13 +00:00
Konstantin Seurer
33a694fe9b radv: Initialize base IDs when doing a BVH update with src!=dst
Fixes: 2d48b2c ("radv: Use subgroup OPs for BVH updates on GFX12")
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:12 +00:00
Konstantin Seurer
4a4251dc16 radv/bvh: Use a fixed indices midpoint on GFX12
This saves a couple of loads inside the update shader.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:12 +00:00
Konstantin Seurer
7ad02416f6 radv/bvh: Fix flush in bit_writer_skip_to
If temp is not cleared, the next flushed dword will contain data from
the previous one.

Fixes: 97f6287 ("radv: Use the BVH8 format on GFX12")
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:11 +00:00
Konstantin Seurer
6201e24307 radv: Only write leaf node offsets when required
They are only used for serialization and position fetch which makes them
unnecessary most of the times.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:11 +00:00
Konstantin Seurer
df44b353ad radv: Optimize ray tracing position fetch
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Gets rid of a lot of indirection when fetching triangle positions.
Storing the primitive address increases register pressure by a bit but
the traversal shader which should have the highest register demand
should not be affected when position fetch is not used.

Totals:
Instrs: 4021686 -> 4022435 (+0.02%); split: -0.01%, +0.03%
CodeSize: 21235812 -> 21235832 (+0.00%); split: -0.02%, +0.02%
Latency: 23402275 -> 23412110 (+0.04%); split: -0.04%, +0.09%
InvThroughput: 4352818 -> 4352206 (-0.01%); split: -0.04%, +0.02%
VClause: 101906 -> 102058 (+0.15%); split: -0.03%, +0.18%
Copies: 342210 -> 342368 (+0.05%); split: -0.09%, +0.14%
Branches: 114988 -> 114993 (+0.00%)
PreVGPRs: 26551 -> 27111 (+2.11%)
VALU: 2249366 -> 2249524 (+0.01%); split: -0.01%, +0.02%
SALU: 529828 -> 529808 (-0.00%); split: -0.01%, +0.00%

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35533>
2025-07-19 16:07:59 +00:00
Samuel Pitoiset
ea742877f6 radv: re-run clang-format
For style consistency.

$ clang-format -i $(find src/amd/vulkan/ -name "*.h" -o -name "*.c" -o -name "*.cpp")

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36118>
2025-07-16 09:10:33 +02:00
Samuel Pitoiset
6111e40a55 radv/bvh: remove redundant definition of DIV_ROUND_UP
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36118>
2025-07-16 09:09:30 +02:00
Natalie Vock
e978f6e247 radv/rt: Use ds_bvh_stack_push8_pop1_rtn_b32
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35269>
2025-07-15 21:34:40 +00:00
Natalie Vock
f0aa383e09 radv/rt: Use ds_bvh_stack_rtn
Improves Quake 2 RTX performance by 5% on RDNA3.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35269>
2025-07-15 21:34:40 +00:00
Natalie Vock
e82717a5cf radv: Use common helper to set BLAS node pointer flags on gfx11+
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32417>
2025-06-28 10:31:38 +00:00
Natalie Vock
06a06bbe09 radv: Encode child opaqueness information in box nodes
Also, use one reserved field from the header to store the root node's
opaqueness flags. This is used to propagate opaqueness info across the
BLAS/TLAS boundary.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32417>
2025-06-28 10:31:37 +00:00
Natalie Vock
3b1f94d00d radv: Encode child opaqueness information in triangle nodes
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32417>
2025-06-28 10:31:37 +00:00
Natalie Vock
6628ac8ad9 radv/rt: Avoid encoding infinities in box node coords
On Navi33, certain box sorting modes combined with infinity/-infinity in
the child AABBs cause image_bvh64_intersect_ray to return garbage node
pointers.

To avoid this, convert infinity to the maximum representable
floating-point value, which will still intersect with any non-inf ray.

Fixes consistent hangs in DOOM: The Dark Ages.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35254>
2025-06-02 19:33:18 +00:00
Konstantin Seurer
36c9b66ee2 radv/bvh: Fix updating empty bvhs
valid_child_count_minus_one is 15 for box nodes without child so every
child was considered valid which made the code read invalid data and use
that for addressing.

Fixes: 2d48b2c ("radv: Use subgroup OPs for BVH updates on GFX12")
Closes: #13217
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35119>
2025-05-26 12:03:21 +00:00
Konstantin Seurer
97f71420df radv/bvh: Fix comment
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34938>
2025-05-19 14:08:33 +00:00
Konstantin Seurer
100616859e radv/bvh: Remove some unused variables
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34938>
2025-05-19 14:08:33 +00:00
Konstantin Seurer
f00b25331a radv/bvh: Make sure the AABB is written before internal_ready_count
Otherwise, the next stage can read garbage. Fixes flickering in The
Witcher 3.

Closes: #13145
Closes: #13196
Fixes: 2d48b2c ("radv: Use subgroup OPs for BVH updates on GFX12")
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34938>
2025-05-19 14:08:33 +00:00
Konstantin Seurer
2d48b2cb47 radv: Use subgroup OPs for BVH updates on GFX12
This patch changes the update code to launch 8 invocations for every
internal node. The internal nodes update their child leaf nodes using
the geometry index and primitive index stored inside the primitive node.

Processing 8 child nodes in parallel is faster than looping over them.
Moving to one dispatch that updates all nodes in one go lets us get rid
of atomics and will also enable updatable BVHs to use pair compression.

Improves Elden Ring (high settings, max RT settings, 1080p) by around
10%.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34601>
2025-05-12 17:45:31 +02:00
Konstantin Seurer
b2aa0647d5 radv: Use a specialized shader for in place updates
If src == dst, we only need to update aabbs for the internal nodes.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34601>
2025-05-12 17:45:00 +02:00
Konstantin Seurer
c21e1776b3 radv: Use build flags instead of defines
Using the meta framework makes managing shader variants much easier.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34594>
2025-05-09 09:55:32 +00:00
Konstantin Seurer
33ac143779 vulkan: Introduce VK_BUILD_FLAG for specializing BVH build shaders
The advantage of using spec constants is that we do not have to include
multiple spirv binaries for multiple variants of a build stage.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34594>
2025-05-09 09:55:32 +00:00
Konstantin Seurer
76031ba53d radv: Optimize the gfx12 encode shader
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
97f6287827 radv: Use the BVH8 format on GFX12
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
95e7343a7d radv/bvh: Add helpers for encoding
The build and update paths can use the same code.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
3af19f336c radv/bvh: Document GFX12 BVH encoding
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
676e26aed5 radv: Fix rayTracingPositionFetch with multiple geometies
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
The fix adds more indirections to avoid increasing register pressure by
tracking the primitive address.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34460>
2025-04-11 22:26:08 +00:00
Natalie Vock
cdadda2d51 radv/rt: Guard leaf encoding by leaf node count
For empty BVHs we shouldn't emit any leaf nodes, but there is one
invocation to encode the root node. Guard leaf node encoding so that
invocation doesn't try writing any leaves.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33985>
2025-03-10 17:42:05 +00:00
Natalie Vock
f01623ea75 radv/bvh: Add custom leaf node builder
This custom builder implements fine-grained instance node bounds
calculation by looking at all AABBs at tree depth 2.

Shaves off 0.3ms in the start scene for Indiana Jones: The Great Circle
on Deck (roughly 29.1ms->28.7ms).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32797>
2025-02-18 13:00:53 +00:00
Natalie Vock
90c3450621 radv/bvh: Prefix RADV-specific node functions with radv_
Avoids naming conflicts when including both the common leaf shader and
RADV's build_helpers.h.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32797>
2025-02-18 13:00:53 +00:00
Natalie Vock
444bd02255 radv/bvh: Remove unused build_instance helper
This is in common code now.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32797>
2025-02-18 13:00:53 +00:00
Natalie Vock
b1f6d3b6b7 radv/bvh, vulkan/bvh: Move AccelerationStructureInstance to vk_build_helpers
Remove duplications.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32797>
2025-02-18 13:00:52 +00:00
Connor Abbott
8fe3674df8 vulkan/runtime,radv: Add shared BVH building framework
This is mostly adapted from radv's BVH building. This defines a common
"IR" for BVH trees, two algorithms for constructing it, and a callback
that the driver implements for encoding. The framework takes care of
parallelizing the different passes, so the driver just has to split the
encoding process into "stages" and implement just one part for each
stage.

The runtime changes are:
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
The radv changes are;

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31433>
2024-12-01 20:08:35 +01:00
Connor Abbott
f8b584d6a5 vulkan/runtime,radv: Add shared BVH building framework
This is mostly adapted from radv's BVH building. This defines a common
"IR" for BVH trees, two algorithms for constructing it, and a callback
that the driver implements for encoding. The framework takes care of
parallelizing the different passes, so the driver just has to split the
encoding process into "stages" and implement just one part for each
stage.

The runtime changes are:
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
The radv changes are;

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31433>
2024-12-01 16:08:06 +00:00
Friedrich Vock
70fc5987d4 radv/rt: Don't atomicAdd local prefix sums
This only gets written to by one invocation at a time.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30483>
2024-08-09 18:12:52 +00:00
Friedrich Vock
a3df3ebab4 radv/rt: Only do ploc atomicCompSwap once per workgroup
There is no need to do this for every invocation in the wave.

Improves GravityMark scores by 5%.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30483>
2024-08-09 18:12:52 +00:00
David Heidelberg
68215332a8 build: pass licensing information in SPDX form
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Dylan Baker <dylan.c.baker@intel.com>
Acked-by: Eric Engestrom <eric@igalia.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: David Heidelberg <david@ixit.cz>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29972>
2024-06-29 12:42:49 -07:00
Dylan Baker
46644ba371 meson: use glslang --depfile argument when possible
This reduces the amount of manual dependency tracking developers need to
do. This is turned on if glslang >= 11.3.0 is used, or 11.9.0 on
Windows, but otherwise the status quo is maintained. This means I have
not removed any use of `depend_files`. We could make make these hard
requirements and remove the use of `depend_files` too.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28329>
2024-05-20 17:34:17 +00:00
Samuel Pitoiset
87fde606cf radv: use SPDX-License-Identifier
Only files under src/amd/vulkan/** are concerned.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28599>
2024-04-08 07:17:31 +00:00
Samuel Pitoiset
b409936e97 radv: add missing endif comment for some headers
For consistency.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28568>
2024-04-04 21:57:45 +00:00
Friedrich Vock
d4c094dfb4 radv/rt: Optimize update shader VGPR usage
Brings VGPR allocation down from 72 (absolutely insane) to 32.

We can now reach the theoretical maximum occupancy of 16 waves per SIMD.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27472>
2024-02-05 21:14:45 +01:00
Friedrich Vock
f66055a6a6 radv/rt: Write inactive node data in ALWAYS_ACTIVE workaround
Fixes: a9831caa ("radv/rt: Add workaround to make leaves always active")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27340>
2024-02-01 05:42:59 +00:00
Friedrich Vock
a9831caa14 radv/rt: Add workaround to make leaves always active
DOOM Eternal builds acceleration structures with inactive primitives and
tries to make them active in later AS updates. This is disallowed by the
spec and triggers a GPU hang. Fix the hang by working around the bug.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27034>
2024-01-17 15:14:48 +00:00
Friedrich Vock
0b55a3cf64 radv/rt: Acceleration structure updates
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26729>
2024-01-09 18:47:45 +00:00
Friedrich Vock
62fe4f0b1b radv/rt: Move per-geometry build info into a geometry_data struct
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26729>
2024-01-09 18:47:45 +00:00
Konstantin Seurer
2e4951d3fb radv: Remove the BVH depth heuristics
It only helps Quake II RTX and hurts everything else.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26481>
2024-01-09 09:00:24 +00:00