If invalid childrens don't consume space in memory, we don't have to
increment the block count. HW unit just look at the bounding boxes and
reject them in intersection test.
Also, this patch handles invalid children type encoding.
Fixes: 198537039a ("anv/rt: reduce writes to block_incr_and_start_prim")
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Felix DeGrood <felix.j.degrood@intel.com>
Tested-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38717>
Rewrite ANV's encode.comp, the final intel-specific raytracing shader
used for bvh-build. Performance is greatly improved for this shader
by adding the following features:
1) Find children early. All threads speculative find their children
before they know if they are valid (not collapsed). This makes more
work overall but reduces latency for propagating valid nodes from
root to leaves. Nodes find out if they are valid faster if all nodes
know who their children are upfront.
2) Hoist code used for intra-thread communication. Communicate
to children as soon as possible, minimizing wait time for later
threads.
3) Multithread encoding. Still launching 1 simd lane per node, same
as before, but encoding of nodes and children are parallelized across
multiple lanes. This works well because most nodes are collapsed
without any encode work required.
4) Hash globalID. Reduce chance that the thread processing a node
will also need to process node's children, which was found to
degrade performance, particularly for root node processing.
Measured RT game speedups:
* Hitman3 +48%
* F1'22 +10%
* Indiana Jones +8%
* GravityMark +2.5%
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36937>
Having a list of all enabled/used extensions in meson allows us to get
rid of a lot of boilerplate in every bvh build shader.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35326>
Adding a boolean "enable_64b_rt" in anv_accel_struct_header for the
interpret.py to properly decode anv_instance_leaf
Signed-off-by: Kevin Chuang <kaiwenjon23@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33047>
This change separate the encode, header, and copy shader into versions
for Xe1/2 and Xe3+, including adding compile options and handling 64bit
version of instance leaf for Xe3+.
Signed-off-by: Kevin Chuang <kaiwenjon23@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33047>
Fixes: 692b5fa9f2 ("anv: Add shader to copy acceleration structures")
This commit fixes the future test "sparse_binding_structures" for
"header_bottom_address" for ray tracing pipeline.
Even on 48-bit ray tracing (Xe1/2), the software-defined part
instance_leaf_part1.bvh_ptr has to be in canonical form for copy.comp
to deference a bvh, which means we have to preserve the upper 16bits.
This is especially relevant in cases where the acceleration structure buffer
is located high, such as sparse buffer.
Signed-off-by: Kevin Chuang <kaiwenjon23@gmail.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33745>
Fixes: 2fe57947e3 ("anv: Implement encode shader to fit in ANV BVH")
This commit resolves the failures in the future tests
"sparse_binding_structures" for rayquery. Sparse buffers' heaps are
located high, and since it's in canonical form, the higher 16bits are
all set to 1. However, the existing encoder did not expect any non-zero
values at the higher 16bits. As a result, the instance flags got
corrupted, causing most triangle tests to fail.
Thanks for Paulo providing insights about sparse buffer properties.
Co-developed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: Kevin Chuang <kaiwenjon23@gmail.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33745>
Modified the BVH dumping mechanism to synchronously wait for the command
buffer to complete before saving BVH data to files. This approach is
more robust compared to the previous method of dumping during
acceleration strucutre destruction.
Note: if DEBUG_BVH_ANY is enabled but intel-rt is disabled, we will wait
for nothing.
Signed-off-by: Kevin Chuang <kaiwenjon23@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32585>
This commit allows you to dump different regions of memory related to
bvh building. An additional script to decode the memory dump is also
added, and you're able to view the built bvh in 3D view in html. See the
included README.md for usage.
Rework:
- you can now view the actual child_coord in internalNode in html
- change exponent to be int8_t in the interpreter
- fix the actual coordinates using an updated formula
- now you can have 3D view of the bvh
- blockIncr could be 2 and vk_aabb should be first
- Now, if any bvh dump is enabled, we will zero out tlas, to prevent gpu
hang caused by incorrect tlas traversal
- rootNodeOffset is back to the beginning
- Add INTEL_DEBUG=bvh_no_build.
- Fix type of dump_size
- add assertion for a 4B alignment
- when clearing out bvh, only clear out everything after
(header+bvh_offset)
- TODO: instead of dumping on destory, track in the command buffer
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31588>
Rework: (Kevin)
- Properly setup bvh_layout
Our bvh resides in contiguous memory and can be divided into two sections:
1. anv_accel_struct_header, tightly followed by
2. actual bvh, which starts with root node, followed by interleaving
leaves or internal nodes.
- Update comments for some fields for BVH and nodes.
- Properly populate the UUIDs in serialization header
- separate header func into completely two paths based on compaction bit
- Encode rt_uuid at second VK_UUID_SIZE.
- Write query result at correct slot
- add assertion for a 4B alignment
- move bvh_layout to anv_bvh
- Use meson option to decide which files to compile
- The alignment of serialization size is not needed
- Change static_assert to STATIC_ASSERT and move them inside functions
Rework (Sagar)
- Use anv_cmd_buffer_update_buffer instead of MI to copy data
Rework (Lionel)
- Remove flush after builds, and add flush in copy before dispatch
- Handle the flushes in CmdWriteAccelerationStructuresPropertiesKHR properly
Co-authored-by: Kevin Chuang <kaiwenjon23@gmail.com>
Co-authored-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31588>
This shader gets called and will construct ANV BVH from IR BVH. More
specifically, each invocation will take care of one internal node. The
internal nodes get processed starting from root node all the way to the
bottom leaves.
During processing, we keep track of the destination of
where the internal node should be encoded (tracked in
vk_ir_box.bvh_offset), and where its leaves should be encoded (tracked
in vk_ir_header.dst_node_offset).
The processed bvh is in contiguous memory, which starts with header,
followed by interleaving internal nodes and leaves. The nodes
information are also populated.
Rework: (Sagar)
- Return out of bounds threads early
- Mimic GRL internal node encoding
- Handle node mask
- Fix block_incr_and_start_prim
- Fix shader_index_and_geom_mask for instance node
- Fix instance flag
- Fix block_incr and instance_contribution_and_geom_flags initialized to be zero
- Fix lower_x and upper_x to be properly flipped for invalid child
- For invalid node, clear blockIncr and set startPrim to INVALID
- Calculated things upfront and assign, cutting down more than ~200
instructions
Co-authored-by: Kevin Chuang <kaiwenjon23@gmail.com>
Co-authored-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31588>
Rework (Kevin)
- encode the address of anv_instance_leaf after header in order to
handle serialization and deserialization part.
- draw serialized data layout and explanation
Co-authored-by: Kevin Chuang <kaiwenjon23@gmail.com>
Co-authored-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31588>
This commit adds build interface and helper header for ANV BVH.
Rework: (Kevin)
- Use block_size macro to represent bvh node/leaf size
- Rename BVH-related node/leaf size macros for clarity
- Updated comments for some fields for bvh and nodes.
- move bvh_layout to anv_bvh.h
- Draw anv_bvh layout
- rename child_offset to child_block_offset
Co-authored-by: Kevin Chuang <kaiwenjon23@gmail.com>
Co-authored-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31588>
