Commit graph

7907 commits

Author SHA1 Message Date
Yevhenii Kolesnikov
c05434cd6d radv: Use a common vk_command_buffer structure
Switch to using common structure.

Signed-off-by: Yevhenii Kolesnikov <yevhenii.kolesnikov@globallogic.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13000>
2021-09-23 15:57:46 +00:00
Timur Kristóf
611d583e64 radv/llvm: Don't read edge flags anymore.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12998>
2021-09-23 16:58:15 +02:00
Timur Kristóf
a7f2faea46 ac/nir: Emit edge flag instructions conditionally.
They are not needed by RADV but will be needed by RadeonSI.

Fossil DB results on Sienna Cichlid (with NGGC on):
Totals from 56917 (44.24% of 128647) affected shaders:
VGPRs: 1982664 -> 1975936 (-0.34%); split: -0.43%, +0.09%
CodeSize: 152790880 -> 149510316 (-2.15%); split: -2.15%, +0.00%
MaxWaves: 1617984 -> 1621900 (+0.24%)
Instrs: 29272825 -> 28907038 (-1.25%); split: -1.26%, +0.01%
Latency: 128744182 -> 127565678 (-0.92%); split: -1.14%, +0.22%
InvThroughput: 20125915 -> 19805168 (-1.59%); split: -1.63%, +0.03%
VClause: 521312 -> 519804 (-0.29%); split: -0.77%, +0.48%
SClause: 688861 -> 688897 (+0.01%); split: -0.04%, +0.05%
Copies: 3205421 -> 3177799 (-0.86%); split: -1.68%, +0.82%
Branches: 1181457 -> 1183147 (+0.14%); split: -0.03%, +0.17%
PreVGPRs: 1626681 -> 1595406 (-1.92%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12998>
2021-09-23 16:57:56 +02:00
Timur Kristóf
cb05c85abd radv: Disable HW generated edge flags for NGG shaders.
Vulkan doesn't support user edge flags, and also doesn't have
any decomposed primitives where drawing internal edges should
be disallowed. Hence, we don't need this.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12998>
2021-09-23 15:52:26 +02:00
Joshua Ashton
8564a4c584 radv: Push box traversal results onto stack in correct order
Currently, it's pushing from lowest distance to highest distance, so when these are popped off the stack, we start at the highest distance to lowest.

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12960>
2021-09-23 09:08:52 +00:00
Samuel Pitoiset
cd64535aff radv: remove useless shader variant key copies for VS+TCS
It's unnecessary.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12976>
2021-09-23 06:50:40 +00:00
Christian Gmeiner
3d65cea6ee util/bitset: s/BITSET_SET_RANGE/BITSET_SET_RANGE_INSIDE_WORD
Prep work for the next commit.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11321>
2021-09-21 20:25:31 +00:00
Samuel Pitoiset
af740f2c35 radv: do not store meta shaders to the default shader disk cache
Meta shaders are already stored in a different shader cache file.
Storing them in two places wastes disk space and they are never
loaded from the default shader disk cache anyways.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12937>
2021-09-21 12:21:41 +00:00
Samuel Pitoiset
f6d5cb2339 radv: do not use a different disk cache key for LLVM
The driver already adds a pipeline hash for LLVM which is redundant.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12938>
2021-09-21 11:34:42 +00:00
Samuel Pitoiset
b9cf327787 radv: remove useless assertions in the SQTT path
The driver aborts when the chip class is older than GFX8.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12854>
2021-09-21 09:08:48 +02:00
Samuel Pitoiset
8846f65c4f radv: make the SQTT BO a resident buffer
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12854>
2021-09-21 09:08:47 +02:00
Samuel Pitoiset
c80ac276bf radv: replicate THREAD_TRACE_CTRL config when stopping SQTT
This seems missed and it might be important.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12854>
2021-09-21 09:08:45 +02:00
Bas Nieuwenhuizen
f1095260a4 radv: Experimentally enable RT extensions.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
ca2d96db51 radv: Add caching for RT pipelines.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
a22a4162d9 radv: Add support for setting a dynamic stack size.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
063d0c90c8 radv: Combine all the parts together with a main loop for an RT pipeline.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
85580faa4b radv: Add ray traversal loop.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
c3d82a9622 radv: Add pass to lower anyhit shader into an intersection shader.
So we avoid having yet another shader calling loop. Such a thing
is not needed since neither shader types do recursion.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
ae5bea3125 radv: Add helper to parse raytracing stages.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
207ce6d658 radv: Add helper to inline shaders into the main shader.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
dcb02dbe73 radv: Add main loop variables.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
701803e340 radv: Add scaffolding for RT pipeline compilation incl libraries.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
f087b457e5 radv: Make some pipeline functions non-static.
Want to put the rt stuff in its own file. radv_pipeline.c is pretty
large already.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
8312b2232b radv: Add raytracing pipeline properties.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
0f090a51b3 radv: Add group info to pipeline.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
080400e43c radv: Add pipeline type.
I want to keep pointers that need to be freed in the union and need
to figure out the type a destruction time. This seems more reliable
than checking the shader array in case we destroy mid-creation (i.e.
on failure).

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
f5362742b9 radv: Add RT cache flushes.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
8ca54b4d38 radv: Support nir_intrinsic_load_global_constant.
SPIR-V parsing can result in some direct constant usage for shader
records. Lower this early to a global based intrinsic so that it
doesn't interfere with the later 32-bit offset based constants
for scratch usage.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
c299968988 aco: Add support for ray launch size.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
1ca4fd31e6 radv: Add support for ray launch size.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
148ea4375c radv: Implement NULL accel struct descriptor write.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
2070d36cf3 radv: Do more meta shader lowering.
Need this to clean up our generated RT pipeline.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
b1074fd51d radv: Refactor some nir_channels usage to use nir_channel.
cleanup, nir_channels wasn't needed as these were only accessing a
single channel.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
817553c052 aco: Implement call scope.
Since we do no repacking yet, just use invocation.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
b6be96a2bd radv: Modify load_sbt_amd intrinsic to get the descriptor.
That way we can get the address to the entry, which is needed for
some nir builtins because extra data in the entry can be used as
shader input.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
2abf44cf18 radv: Add bvh node definitions to a header.
So that we can avoid some magic numbers in the pipeline creation.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
a99ccd353b radv: Add optimized CPU BVH builds.
This trivial sorting helps speed up bad cases (like in CTS significantly).

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
848d175c76 radv: Add more acceleration structure formats.
These are required ...

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
506757ea86 radv: Add accel struct build support for the object-to-world matrix.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
3e059dc1e0 radv: Fix arrayOfPointers for instances in accel struct build.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
58ec4f3069 radv: Fix CPU AABB build.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Timur Kristóf
13e467a147 ac/nir: Fix match_mask to work correctly for VS outputs.
match_mask checks the intrinsic type and decides whether it's
per-patch or not. VS don't have per-patch outputs,
so this causes wrong behaviour there.

Found using the GCC undefined behavior sanitizer.
Fixes the following error:

runtime error:
shift exponent 18446744073709551584 is too large
for 64-bit type 'long unsigned int'

Closes: #5319
Fixes: bf966d1c1d
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12719>
2021-09-20 18:08:16 +00:00
Samuel Pitoiset
0be0ad102e radv: keep depth/stencil images compressed for TRANSFER_DST on compute
Only if the image is TC-compat HTILE because it can be decompressed
on compute for partial copies.

This should remove few depth/stencil decompressions for RAGE2 and Red
Dead Redemption 2 because they declare all images as concurrent but
never use the compute queue for them.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>
2021-09-20 15:17:42 +00:00
Samuel Pitoiset
2b805b8a6c radv: add support for copying compressed depth/stencil images on compute
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>
2021-09-20 15:17:42 +00:00
Samuel Pitoiset
0ac8731b6c radv: implement depth/stencil expand on compute
This works as long as the image is TC-compatible HTILE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>
2021-09-20 15:17:42 +00:00
Samuel Pitoiset
966b780992 radv: rename radv_decompress_depth_stencil()
To radv_expand_depth_stencil() for consistency.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>
2021-09-20 15:17:42 +00:00
Timur Kristóf
75dbb40439 ac/nir: Remove byte permute from prefix sum of the repack sequence.
The byte-permute instruction v_perm_b32 is not exposed by older
LLVM releases (only available on LLVM 13 and later), therefore a new
sequence is needed which we can use with these LLVM versions too.

The prefix sum is replaced by two alternatives:

1. For GPUs that support v_dot, we shift 0x01 to the wanted byte
positions and then use v_dot to sum the results.

2. For older GPUs (Navi 10), we simply shift out the unwanted bytes
and use v_sad_u8 to produce the sum.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12786>
2021-09-20 12:39:03 +02:00
Timur Kristóf
966cff9cfa aco/isel: Fix emit_vop2_instruction to apply 16/24-bit flags properly.
Previously it used a builder function but didn't use the return
value from that function, so the flags were not applied.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12786>
2021-09-20 12:39:03 +02:00
Timur Kristóf
f2e41eda9e aco: Add ability to optimize v_lshl + v_sub into v_mad_i32_i24.
Also change combine_add_lshl to use check_vop3_operands instead
of its own checks of the operands.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12786>
2021-09-20 12:39:03 +02:00
Filip Gawin
6b5e9352ef aco: cleanup assignment of unique_ptrs
Reviewed-by: Joshua Ashton <joshua@froggi.es>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12903>
2021-09-18 11:09:24 +00:00