Commit graph

4506 commits

Author SHA1 Message Date
Bas Nieuwenhuizen
f087b457e5 radv: Make some pipeline functions non-static.
Want to put the rt stuff in its own file. radv_pipeline.c is pretty
large already.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
8312b2232b radv: Add raytracing pipeline properties.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
0f090a51b3 radv: Add group info to pipeline.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
080400e43c radv: Add pipeline type.
I want to keep pointers that need to be freed in the union and need
to figure out the type a destruction time. This seems more reliable
than checking the shader array in case we destroy mid-creation (i.e.
on failure).

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
f5362742b9 radv: Add RT cache flushes.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
8ca54b4d38 radv: Support nir_intrinsic_load_global_constant.
SPIR-V parsing can result in some direct constant usage for shader
records. Lower this early to a global based intrinsic so that it
doesn't interfere with the later 32-bit offset based constants
for scratch usage.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
1ca4fd31e6 radv: Add support for ray launch size.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
148ea4375c radv: Implement NULL accel struct descriptor write.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
2070d36cf3 radv: Do more meta shader lowering.
Need this to clean up our generated RT pipeline.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
b1074fd51d radv: Refactor some nir_channels usage to use nir_channel.
cleanup, nir_channels wasn't needed as these were only accessing a
single channel.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
b6be96a2bd radv: Modify load_sbt_amd intrinsic to get the descriptor.
That way we can get the address to the entry, which is needed for
some nir builtins because extra data in the entry can be used as
shader input.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
2abf44cf18 radv: Add bvh node definitions to a header.
So that we can avoid some magic numbers in the pipeline creation.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
a99ccd353b radv: Add optimized CPU BVH builds.
This trivial sorting helps speed up bad cases (like in CTS significantly).

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
848d175c76 radv: Add more acceleration structure formats.
These are required ...

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
506757ea86 radv: Add accel struct build support for the object-to-world matrix.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
3e059dc1e0 radv: Fix arrayOfPointers for instances in accel struct build.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Bas Nieuwenhuizen
58ec4f3069 radv: Fix CPU AABB build.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Samuel Pitoiset
0be0ad102e radv: keep depth/stencil images compressed for TRANSFER_DST on compute
Only if the image is TC-compat HTILE because it can be decompressed
on compute for partial copies.

This should remove few depth/stencil decompressions for RAGE2 and Red
Dead Redemption 2 because they declare all images as concurrent but
never use the compute queue for them.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>
2021-09-20 15:17:42 +00:00
Samuel Pitoiset
2b805b8a6c radv: add support for copying compressed depth/stencil images on compute
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>
2021-09-20 15:17:42 +00:00
Samuel Pitoiset
0ac8731b6c radv: implement depth/stencil expand on compute
This works as long as the image is TC-compatible HTILE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>
2021-09-20 15:17:42 +00:00
Samuel Pitoiset
966b780992 radv: rename radv_decompress_depth_stencil()
To radv_expand_depth_stencil() for consistency.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12452>
2021-09-20 15:17:42 +00:00
Joshua Ashton
592aa9375c radv: Expose modifiers that support DCC image stores with STORAGE_IMAGE_BIT
Some games, ie. Doom Eternal, present from compute following compute post-fx and would benefit from having DCC image stores available.

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12862>
2021-09-18 00:01:01 +00:00
Rhys Perry
dfa8ef723b radv: fix pipeline caching with robust buffer access
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12900>
2021-09-17 13:49:46 +00:00
Samuel Pitoiset
c952655693 ac/rgp, radv: report wave size for shaders
Fills the "Wave mode" in "Pipelines" for GPUs that supports Wave32.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12896>
2021-09-17 08:05:36 +00:00
Samuel Pitoiset
d29c381c64 ac/rgp, radv: report scratch memory size for shaders
Fills the "Scatch Mem" with "Yes/No" in "Pipelines", this requires
instruction timing to be enabled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12896>
2021-09-17 08:05:36 +00:00
Timur Kristóf
92e1981a80 radv: Remove PSIZ output when it isn't needed.
PSIZ output is only needed when:
1. There is a next stage and it reads it.
2. Primitive topology is point list, in the last vertex pipeline stage.

Zink always adds this output in its vertex (and other) shaders,
because it helps Zink avoid recompiling shader variants.

However, this has a performance impact for RADV because
it needs a scalar memory load. That becomes noticeable
at high primitive rates.

The Fossil stats are unremarkable because our DB doesn't include any
shaders from Zink or D9VK, but there are a few affected shaders.

Note that there may be an increase in LDS use in some GS. This is
because with PSIZ removed the ES per-vertex LDS size is smaller, so
we can squeeze more GS threads in the same workgroup.

Fossil DB stats on Sienna Cichlid:

Totals from 14 (0.01% of 128647) affected shaders:
CodeSize: 119884 -> 119732 (-0.13%)
LDS: 235008 -> 228864 (-2.61%); split: -2.83%, +0.22%
Instrs: 23076 -> 23048 (-0.12%)
Latency: 71667 -> 71625 (-0.06%)
InvThroughput: 19155 -> 18870 (-1.49%)
Copies: 1586 -> 1572 (-0.88%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10725>
2021-09-16 11:06:05 +00:00
Samuel Pitoiset
3bb5082900 radv: advertise EXT_shader_atomic_float2 with LLVM 14+
The intrinsics aren't exposed on earlier LLVM versions.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12716>
2021-09-15 14:10:42 +00:00
Samuel Pitoiset
3a26dfe761 radv/llvm: fix vertex input fetches with 16-bit floats
Not ideal but ac/llvm and RADV works with integers, so passing a
16-bit float type would break more than it helps.

Fixes a few CTS with 16-bit float IO.

Fixes: 3fb229e010 ("ac,radeonsi: load VS inputs at the call site of nir_intrinsic_load_input")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12835>
2021-09-15 11:48:01 +00:00
Samuel Pitoiset
5eda8e5fe9 radv/llvm: fix using Wave32
The Wave32 pass manager has been removed a while ago.

Fixes: 94a1f45e15 ("ac/llvm: set target features per function instead of per target machine")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12833>
2021-09-15 12:31:32 +02:00
Samuel Pitoiset
9bf977ffa5 radv: reduce SQTT traffic when instruction timing is disabled
Some tokens can be excluded without instruction timing. This reduces
RGP capture sizes significantly.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12853>
2021-09-15 10:48:58 +02:00
Joshua Ashton
fe96cea2f3 radv: Disable DCC on storage images that cannot support DCC image stores
Additional work is needed for storage images with DCC without DCC image stores to not be broken.

Fixes black screens in Doom Eternal.

Fixes: #5345
Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12818>
2021-09-15 00:21:04 +00:00
Emma Anholt
aed4c0b5a9 nir: Drop the unused instr arg for src/dest copy functions.
Now that we don't use ralloc, we don't need this arg to get at the right
ralloc ctx.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11776>
2021-09-14 17:53:06 +00:00
Marek Olšák
b330c7cb2a radeonsi: use a trick to extract and pack edgeflags using fewer instructions
This removes 4 instructions from the prim export packing.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12343>
2021-09-14 15:24:11 +00:00
Simon Ser
3c18e69078 vulkan/wsi/x11: add driconf option to not wait under Xwayland
By default, Mesa's X11 Vulkan WSI will wait for buffers to be ready
before submitting them to Xwayland when the swapchain is created
with the IMMEDIATE mode.

This is undesirable when the Wayland compositor already monitors
fences. A Wayland compositor may want to know the delay between
the buffer submition and the end of the GPU work, this is impossible
to measure if the WSI waits for the buffer to be ready before
submission.

Since most compositors don't monitor fences, let's introduce a driconf
option for this for now. We can reconsider once more compositors
have better support for fences.

Signed-off-by: Simon Ser <contact@emersion.fr>
Acked-by: Michel Dänzer <mdaenzer@redhat.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11290>
2021-09-13 19:38:13 +00:00
Samuel Pitoiset
b269f59ec5 radv: enable comp-to-single for MSAA images
This allows fast clears for concurrent MSAA images.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12483>
2021-09-13 16:43:07 +00:00
Samuel Pitoiset
de8865df56 radv: add MSAA support to the comp-to-single fast clear path
Clearing the first sample is enough as long as CMASK is also cleared
to indicate that other samples are also cleared.

I verified that the first sample is always at the beginning of 256B
blocks.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12483>
2021-09-13 16:43:07 +00:00
Bas Nieuwenhuizen
afee571d2b radv: Expose a bufferImageGranularity of 1.
It shouldn't matter as image & buffer alignment are enough and we
don't need any alignment between the two.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5213
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12765>
2021-09-13 13:12:41 +00:00
Joshua Ashton
138463d3c3 radv: Fix DCC image store check
Doesn't seem to be causing any issues right now but could with modifiers potentially.

Matches what is in RadeonSI where the comment is also shamelessly stolen from.

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12811>
2021-09-11 05:58:06 +00:00
Marek Olšák
69e96cfc0d ac,radv: remove unused inputs array and VS input code
The previous commit stopped using "inputs".

"load_layer_id" has always been broken and it was probably unused anyway.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Marek Olšák
bce7c7f3fc ac/llvm: implement nir_intrinsic_elect
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Marek Olšák
1e178f7a37 ac: make ac_shader_abi::inputs an array instead of a pointer
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Marek Olšák
6df5f268db ac: remove needless parameters from ac_shader_abi::emit_outputs
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Marek Olšák
a33602b1f9 ac/llvm: remove load_tess_coord callback
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Rhys Perry
522f135d06 radv: expose VK_KHR_shader_integer_dot_product
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>
2021-09-03 13:21:28 +00:00
Rhys Perry
4dd420f76d radv,aco: implement iadd_sat
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>
2021-09-03 13:21:28 +00:00
Rhys Perry
44be450dc1 radv: refactor handling of nir_options
Make it easier to change them depending on chip_class and family.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>
2021-09-03 13:21:28 +00:00
Rhys Perry
d6619d0a01 ac/llvm,radv: implement uadd_sat/iadd_sat
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>
2021-09-03 13:21:27 +00:00
Samuel Pitoiset
ad878856e6 radv/llvm: rework VS input loads and implement the callback
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12693>
2021-09-03 08:14:51 +00:00
Samuel Pitoiset
1402c17e4f radv: optimize VRS when no depth stencil attachment is bound
This is allowed by the Vulkan spec and we have to handle this situation
internally. We used to create and bind a 4096x4096 image to copy the
VRS rates but this wasted too much VRAM (~33MiB). Now, the driver only
allocates a HTILE buffer (~1MiB) and bind it to the framebuffer.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12243>
2021-09-02 19:39:04 +00:00
Samuel Pitoiset
ab635b024b radv: pass the HTILE buffer to radv_copy_vrs_htile()
Will be used to use a global HTILE buffer without an image.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12243>
2021-09-02 19:39:04 +00:00