Commit graph

138 commits

Author SHA1 Message Date
Konstantin Seurer
7809af5e46 radv: Always use compact bvh encoding
The compact encoding will make it possible to allocate less space for
internal nodes

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37830>
2025-10-24 21:17:10 +00:00
Konstantin Seurer
d423554e9e radv/bvh: Pair compress triangles in more cases
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36965>
2025-10-21 19:32:55 +00:00
Konstantin Seurer
c18a7d0e2b radv: Emit compressed primitive nodes on GFX12
The normal encode pass writes batches to a section in build scratch
memory. Those batches contain information about the internal node and
the primitive nodes. The encoder is split to avoid the register
pressure of the compressor and maximize occupancy.

The compressor works in two passes because one pass can not guarantee
that every primitive node (except) has at least two triangles. This
guarantee is used to advertise a smaller acceleration structure size to
the application.

During compression, every invocation processes at most two triangles.
Groups of 8 invocations are used to support the maximum triangle count
of 16 that the hardware supports.

The first step of compression is loading the triangle(s). Shared
vertices are deduplicated early to avoid doing it in the compression
loop. The compression loop tries to add triangles to a list of triangles
until the computed node size needed for storing the triangles reaches
the hardware node size. For this, each invocation first deduplicates
vertices with the triangles that have already been picked. It then
computes the node size of the picked triangles plus the candidate
triangles of the current invocation. The invocation that computed the
smallest size is added to the list.

Because it may not be possible to fit every triangle into the same node,
there can be multiple hardware nodes which are written in parallel for
optimal performance. If there are no nodes with only one triangle, all
nodes are written. If there is, compression of the batch is aborted and
the index of the batch is written to build scratch memory. The second
compression pass will repeat the steps above but only for those aborted
batches. The nodes with only one triangle can and are now merged.

It can not be determined during box node encode which triangles will be
compressed together so the encoder also has to fix up the parent box
node's child infos.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36965>
2025-10-21 19:32:55 +00:00
Konstantin Seurer
2ee8bfefe6 radv/bvh: Add radv_first_active_invocation
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36965>
2025-10-21 19:32:53 +00:00
Natalie Vock
52c7b0d20c radv/bvh: Encode empty AS bounds as NaN
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
If there are no leaves, the root node bounds still span -inf/inf.
Making empty BLASs infinite-sized guarantees ray traversal needs to
enter the BLAS (and immediately exit because it's empty). Remove the
BLAS from the BVH entirely by marking its bounds as NaN. As a bonus,
this works around RADV encountering issues in Silent Hill 2 on RDNA4 due
to infinite-sized BVHs.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37492>
2025-10-01 14:27:15 +00:00
Konstantin Seurer
ea51a67996 vulkan/bvh: Enable glsl extensions in meson
Having a list of all enabled/used extensions in meson allows us to get
rid of a lot of boilerplate in every bvh build shader.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35326>
2025-09-16 20:18:01 +00:00
Christian Gmeiner
1492de1bc3 radv: re-format using clang-format
No manual changes here, this is simply running
$ ninja -C build/ clang-format

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37226>
2025-09-09 05:48:56 +00:00
Konstantin Seurer
9a93f794cd radv/bvh: Do not write pointer flag related data on GFX103
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
On GFX103, those fields are unused. VK_BUILD_FLAG_PROPAGATE_CULL_FLAGS
is set if the fields are used so it can be used to skip writing them.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37021>
2025-09-08 12:47:18 +00:00
Konstantin Seurer
906b541567 radv/bvh: Copy parent_id during updates on GFX12
Fixes: cc0dc4b5 ("radv: Store parent node IDs inside nodes on GFX12")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13567
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36898>
2025-08-24 13:45:29 +00:00
Konstantin Seurer
cc0dc4b566 radv: Store parent node IDs inside nodes on GFX12
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Saves some space.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36691>
2025-08-15 13:00:32 +00:00
Konstantin Seurer
c4b18c689f radv: Emit compressed primitive nodes on GFX12
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Emits two triangles per node whenever possible. The nir code will
revisit the triangle node to handle the second triangle only if both
triangles are interescted by the ray.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35734>
2025-08-07 20:23:15 +00:00
Konstantin Seurer
48d15c3cf8 radv/bvh: Specialize the update shader for geometryCount==1
The geometry data can be loaded from push constants in that case.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:13 +00:00
Konstantin Seurer
b20ab07e4a radv/bvh: Update leaf nodes before refitting
This should reduce latency between refitting nodes and their parent
nodes.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:13 +00:00
Konstantin Seurer
33a694fe9b radv: Initialize base IDs when doing a BVH update with src!=dst
Fixes: 2d48b2c ("radv: Use subgroup OPs for BVH updates on GFX12")
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:12 +00:00
Konstantin Seurer
4a4251dc16 radv/bvh: Use a fixed indices midpoint on GFX12
This saves a couple of loads inside the update shader.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:12 +00:00
Konstantin Seurer
7ad02416f6 radv/bvh: Fix flush in bit_writer_skip_to
If temp is not cleared, the next flushed dword will contain data from
the previous one.

Fixes: 97f6287 ("radv: Use the BVH8 format on GFX12")
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:11 +00:00
Konstantin Seurer
6201e24307 radv: Only write leaf node offsets when required
They are only used for serialization and position fetch which makes them
unnecessary most of the times.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35445>
2025-07-25 09:05:11 +00:00
Konstantin Seurer
df44b353ad radv: Optimize ray tracing position fetch
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Gets rid of a lot of indirection when fetching triangle positions.
Storing the primitive address increases register pressure by a bit but
the traversal shader which should have the highest register demand
should not be affected when position fetch is not used.

Totals:
Instrs: 4021686 -> 4022435 (+0.02%); split: -0.01%, +0.03%
CodeSize: 21235812 -> 21235832 (+0.00%); split: -0.02%, +0.02%
Latency: 23402275 -> 23412110 (+0.04%); split: -0.04%, +0.09%
InvThroughput: 4352818 -> 4352206 (-0.01%); split: -0.04%, +0.02%
VClause: 101906 -> 102058 (+0.15%); split: -0.03%, +0.18%
Copies: 342210 -> 342368 (+0.05%); split: -0.09%, +0.14%
Branches: 114988 -> 114993 (+0.00%)
PreVGPRs: 26551 -> 27111 (+2.11%)
VALU: 2249366 -> 2249524 (+0.01%); split: -0.01%, +0.02%
SALU: 529828 -> 529808 (-0.00%); split: -0.01%, +0.00%

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35533>
2025-07-19 16:07:59 +00:00
Samuel Pitoiset
ea742877f6 radv: re-run clang-format
For style consistency.

$ clang-format -i $(find src/amd/vulkan/ -name "*.h" -o -name "*.c" -o -name "*.cpp")

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36118>
2025-07-16 09:10:33 +02:00
Samuel Pitoiset
6111e40a55 radv/bvh: remove redundant definition of DIV_ROUND_UP
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36118>
2025-07-16 09:09:30 +02:00
Natalie Vock
e978f6e247 radv/rt: Use ds_bvh_stack_push8_pop1_rtn_b32
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35269>
2025-07-15 21:34:40 +00:00
Natalie Vock
f0aa383e09 radv/rt: Use ds_bvh_stack_rtn
Improves Quake 2 RTX performance by 5% on RDNA3.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35269>
2025-07-15 21:34:40 +00:00
Natalie Vock
e82717a5cf radv: Use common helper to set BLAS node pointer flags on gfx11+
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32417>
2025-06-28 10:31:38 +00:00
Natalie Vock
06a06bbe09 radv: Encode child opaqueness information in box nodes
Also, use one reserved field from the header to store the root node's
opaqueness flags. This is used to propagate opaqueness info across the
BLAS/TLAS boundary.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32417>
2025-06-28 10:31:37 +00:00
Natalie Vock
3b1f94d00d radv: Encode child opaqueness information in triangle nodes
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32417>
2025-06-28 10:31:37 +00:00
Natalie Vock
6628ac8ad9 radv/rt: Avoid encoding infinities in box node coords
On Navi33, certain box sorting modes combined with infinity/-infinity in
the child AABBs cause image_bvh64_intersect_ray to return garbage node
pointers.

To avoid this, convert infinity to the maximum representable
floating-point value, which will still intersect with any non-inf ray.

Fixes consistent hangs in DOOM: The Dark Ages.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35254>
2025-06-02 19:33:18 +00:00
Konstantin Seurer
36c9b66ee2 radv/bvh: Fix updating empty bvhs
valid_child_count_minus_one is 15 for box nodes without child so every
child was considered valid which made the code read invalid data and use
that for addressing.

Fixes: 2d48b2c ("radv: Use subgroup OPs for BVH updates on GFX12")
Closes: #13217
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35119>
2025-05-26 12:03:21 +00:00
Konstantin Seurer
97f71420df radv/bvh: Fix comment
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34938>
2025-05-19 14:08:33 +00:00
Konstantin Seurer
100616859e radv/bvh: Remove some unused variables
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34938>
2025-05-19 14:08:33 +00:00
Konstantin Seurer
f00b25331a radv/bvh: Make sure the AABB is written before internal_ready_count
Otherwise, the next stage can read garbage. Fixes flickering in The
Witcher 3.

Closes: #13145
Closes: #13196
Fixes: 2d48b2c ("radv: Use subgroup OPs for BVH updates on GFX12")
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34938>
2025-05-19 14:08:33 +00:00
Konstantin Seurer
2d48b2cb47 radv: Use subgroup OPs for BVH updates on GFX12
This patch changes the update code to launch 8 invocations for every
internal node. The internal nodes update their child leaf nodes using
the geometry index and primitive index stored inside the primitive node.

Processing 8 child nodes in parallel is faster than looping over them.
Moving to one dispatch that updates all nodes in one go lets us get rid
of atomics and will also enable updatable BVHs to use pair compression.

Improves Elden Ring (high settings, max RT settings, 1080p) by around
10%.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34601>
2025-05-12 17:45:31 +02:00
Konstantin Seurer
b2aa0647d5 radv: Use a specialized shader for in place updates
If src == dst, we only need to update aabbs for the internal nodes.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34601>
2025-05-12 17:45:00 +02:00
Konstantin Seurer
c21e1776b3 radv: Use build flags instead of defines
Using the meta framework makes managing shader variants much easier.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34594>
2025-05-09 09:55:32 +00:00
Konstantin Seurer
33ac143779 vulkan: Introduce VK_BUILD_FLAG for specializing BVH build shaders
The advantage of using spec constants is that we do not have to include
multiple spirv binaries for multiple variants of a build stage.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34594>
2025-05-09 09:55:32 +00:00
Konstantin Seurer
76031ba53d radv: Optimize the gfx12 encode shader
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
97f6287827 radv: Use the BVH8 format on GFX12
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
95e7343a7d radv/bvh: Add helpers for encoding
The build and update paths can use the same code.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
3af19f336c radv/bvh: Document GFX12 BVH encoding
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
676e26aed5 radv: Fix rayTracingPositionFetch with multiple geometies
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
The fix adds more indirections to avoid increasing register pressure by
tracking the primitive address.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34460>
2025-04-11 22:26:08 +00:00
Natalie Vock
cdadda2d51 radv/rt: Guard leaf encoding by leaf node count
For empty BVHs we shouldn't emit any leaf nodes, but there is one
invocation to encode the root node. Guard leaf node encoding so that
invocation doesn't try writing any leaves.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33985>
2025-03-10 17:42:05 +00:00
Natalie Vock
f01623ea75 radv/bvh: Add custom leaf node builder
This custom builder implements fine-grained instance node bounds
calculation by looking at all AABBs at tree depth 2.

Shaves off 0.3ms in the start scene for Indiana Jones: The Great Circle
on Deck (roughly 29.1ms->28.7ms).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32797>
2025-02-18 13:00:53 +00:00
Natalie Vock
90c3450621 radv/bvh: Prefix RADV-specific node functions with radv_
Avoids naming conflicts when including both the common leaf shader and
RADV's build_helpers.h.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32797>
2025-02-18 13:00:53 +00:00
Natalie Vock
444bd02255 radv/bvh: Remove unused build_instance helper
This is in common code now.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32797>
2025-02-18 13:00:53 +00:00
Natalie Vock
b1f6d3b6b7 radv/bvh, vulkan/bvh: Move AccelerationStructureInstance to vk_build_helpers
Remove duplications.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32797>
2025-02-18 13:00:52 +00:00
Connor Abbott
8fe3674df8 vulkan/runtime,radv: Add shared BVH building framework
This is mostly adapted from radv's BVH building. This defines a common
"IR" for BVH trees, two algorithms for constructing it, and a callback
that the driver implements for encoding. The framework takes care of
parallelizing the different passes, so the driver just has to split the
encoding process into "stages" and implement just one part for each
stage.

The runtime changes are:
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
The radv changes are;

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31433>
2024-12-01 20:08:35 +01:00
Connor Abbott
f8b584d6a5 vulkan/runtime,radv: Add shared BVH building framework
This is mostly adapted from radv's BVH building. This defines a common
"IR" for BVH trees, two algorithms for constructing it, and a callback
that the driver implements for encoding. The framework takes care of
parallelizing the different passes, so the driver just has to split the
encoding process into "stages" and implement just one part for each
stage.

The runtime changes are:
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
The radv changes are;

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31433>
2024-12-01 16:08:06 +00:00
Friedrich Vock
70fc5987d4 radv/rt: Don't atomicAdd local prefix sums
This only gets written to by one invocation at a time.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30483>
2024-08-09 18:12:52 +00:00
Friedrich Vock
a3df3ebab4 radv/rt: Only do ploc atomicCompSwap once per workgroup
There is no need to do this for every invocation in the wave.

Improves GravityMark scores by 5%.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30483>
2024-08-09 18:12:52 +00:00
David Heidelberg
68215332a8 build: pass licensing information in SPDX form
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Dylan Baker <dylan.c.baker@intel.com>
Acked-by: Eric Engestrom <eric@igalia.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: David Heidelberg <david@ixit.cz>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29972>
2024-06-29 12:42:49 -07:00
Dylan Baker
46644ba371 meson: use glslang --depfile argument when possible
This reduces the amount of manual dependency tracking developers need to
do. This is turned on if glslang >= 11.3.0 is used, or 11.9.0 on
Windows, but otherwise the status quo is maintained. This means I have
not removed any use of `depend_files`. We could make make these hard
requirements and remove the use of `depend_files` too.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28329>
2024-05-20 17:34:17 +00:00