We already did the work of transforming the ray data, no need to do it
multiple times.
Should theoretically be a lot better. However, none of the fossils
appear to use object-space ray data in anyhit/intersection shaders. :(
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38809>
This splits up radv_nir_rt_shader.c into several parts.
The first part is all ray traversal lowering for RT pipelines, located
at radv_nir_rt_traversal_shader.c. It implements building the traversal
loop, including inlined any-hit/intersection shaders (optionally as a
completely separate shader).
The second part is lowering for individual RT stages (right now,
monolithic vs. CPS-style separate compilation). Each lowering technique
lives in its own file (radv_nir_rt_stage_{monolithic,cps}.c).
Code shared between RT lowering techniques (shader inlining helpers and
storage lowering passes) gets moved into radv_nir_rt_stage_common.c.
One header, radv_nir_rt_stage.h, is the public interface for RT pipeline
stage lowering. Functions exposed to users (really just
radv_pipeline_rt.c) go there. The header for internal shared helpers is
radv_nir_rt_stage_common.c.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38809>
This add support to the lowering the reduction operations.
Thanks to Georg Lehmann for a lot of the ideas and optimising in
this.
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38389>
We add a bunch of new helpers to avoid the need to touch >parent_instr,
including the full set of:
* nir_def_is_*
* nir_def_as_*_or_null
* nir_def_as_* [assumes the right instr type]
* nir_src_is_*
* nir_src_as_*
* nir_scalar_is_*
* nir_scalar_as_*
Plus nir_def_instr() where there's no more suitable helper.
Also an existing helper is renamed to unify all the names, while we're
churning the tree:
* nir_src_as_alu_instr -> nir_src_as_alu
..and then we port the tree to use the helpers as much as possible, using
nir_def_instr() where that does not work.
Acked-by: Marek Olšák <maraeo@gmail.com>
---
To eliminate nir_def::parent_instr we need to churn the tree anyway, so I'm
taking this opportunity to clean up a lot of NIR patterns.
Co-authored-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38313>
The size and stage parameters are left-overs from history. Originally,
the function acted on a list and so it needed an explicit stage and size
output. Now that it takes a NIR shader and a mode, we can just take the
stage from the shader and set num_(in|out)puts.
The one caller that actually used the explicit output parameter was
turnip. However, given that the helper sorts and re-numbers all the I/O
variables, it's not like changing num_(in|out)puts instead of writing it
to some other location is that big of a deal.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38297>
The code for lowering get_ssbo_size will be different in future chips,
so do it in common code to reduce duplication in the future.
Lower get_ssbo_size to ssbo_descriptor_amd + nir_channel.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38097>
When NIR_DEBUG=serialize or NIR_DEBUG=clone is used, NIR_PASS recreates
nir_function_impl and nir_variable objects, causing use-after-free since
insert_rt_case() keeps pointers to those in local variables and var_remap.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37573>
This doesn't work with NIR_DEBUG=serialize or NIR_DEBUG=clone.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37573>
When NIR_DEBUG=serialize or NIR_DEBUG=clone is used, NIR_PASS recreates
nir_function_impl and nir_variable objects, causing use-after-free since
radv_nir_lower_rt_abi() keeps pointers to those in local variables.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37573>
With descriptor heap the driver will also have to emit indirect
descriptor heaps in some cases.
Rename couple of things to make them more generic.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37786>
Lowering them earlier right after VTN would allow us to implement
embedded samplers for descriptor heap properly for merged shaders.
Non-immediate samplers are still lowered in
radv_nir_apply_pipeline_layout because they require shader arguments.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37688>
Only create nir_load_rt_arg_scratch_offset_amd if needed.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35069>
radv_cmd_buffer_upload_alloc_aligned is used with alignment=0, which
guarantees that the alignment is at least 4.
Fixes: 9e16ed7a13 - ac/nir: switch nir_load_smem_amd uses to ac_nir_load_smem wrapper
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37345>
This means we can actually implement varying subgroup size correctly.
It also means that we implement the implicit SPIR-V 1.6 full subgroups
requirement in compute shaders with cswave32/rtwave32.
In the future it will also allow more optimizations that use the subgroup size.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
The only somewhat complex case here is GFX10 geometry shaders, if gewave32 is
used. We then only know the subgroup size when is_ngg is decided, as legacy
GS doesn't support wave32.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37294>
All shaders are allocated in the 32-bit addr space. To avoid an issue
with alignment, and also for future work, there is an unused user SGPR.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37133>
