Commit graph

11286 commits

Author SHA1 Message Date
Timur Kristóf
2c40215ab9 aco/optimizer: Change v_cmp with subgroup invocation to constant.
When a shader has a comparison with the subgroup invocation id,
we can use a constant instead, saving a VALU instruction.
When the constant can't be represented as a 64-bit literal,
use the s_bfm_b64 instruction to generate it instead, which
is still a win.

Fossil DB stats on GFX11:
Totals from 300 (0.22% of 134913) affected shaders:
CodeSize: 2223052 -> 2214336 (-0.39%); split: -0.43%, +0.04%
Instrs: 430216 -> 429882 (-0.08%); split: -0.14%, +0.06%
Latency: 5881180 -> 5878181 (-0.05%); split: -0.05%, +0.00%
InvThroughput: 731846 -> 729293 (-0.35%)
Copies: 31662 -> 31847 (+0.58%); split: -0.03%, +0.61%
Branches: 8241 -> 8100 (-1.71%)
PreVGPRs: 15788 -> 15786 (-0.01%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20843>
2023-02-18 21:16:58 +01:00
Chia-I Wu
4459668b6e radv: add a size check in radv_create_buffer for Android
This is to make dEQP-VK.api.buffer.basic.size_max_uint64 pass on
android.

The test creates a buffer of size UINT64_MAX and makes sure the memory
requirement for the buffer is sane.  It fails because our memory
requirement is "align64(UINT64_MAX, 16)" which is 0 after overflow.

The test checks maintenance4's maxBufferSize and is skipped normally.
But the extension can be disabled on an android build.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21346>
2023-02-17 18:14:34 +00:00
Timur Kristóf
e13074d763 radv: Call nir_lower_array_deref_of_vec in radv_lower_io_to_scalar_early.
This fixes an issue when a vector component of an arrayed output has a deref.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8197
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21028>
2023-02-17 17:47:47 +00:00
Konstantin Seurer
c76060c253 radv: Advertise ray query support with LLVM
What could go wrong?

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21268>
2023-02-17 17:04:47 +00:00
Konstantin Seurer
3966fbdc51 radv: Pre-compile BVH build shaders if there is a cache
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21268>
2023-02-17 17:04:47 +00:00
Konstantin Seurer
40f246e3e9 radv: Force ACO for BVH build shaders
They hang with LLVM.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21268>
2023-02-17 17:04:47 +00:00
Konstantin Seurer
2ef5acedc2 radv: Make accel struct meta state initialization thread safe
Fixes: 0d5570b ("radv: Always compile accel structure shaders on demand.")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21268>
2023-02-17 17:04:47 +00:00
Konstantin Seurer
0f709510f4 ac/llvm: Implement bvh64_intersect_ray_amd
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21268>
2023-02-17 17:04:47 +00:00
Samuel Pitoiset
c8495dbee3 radv: stop using a PS epilog when the FS doesn't write any color outputs
This is a small optimization for fragment shaders that only write
depth/stencil/sample mask without any color outputs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21341>
2023-02-17 13:19:54 +00:00
Samuel Pitoiset
78c0fae08c radv: only skip emitting the pipeline blend state if the FS uses an epilog
The blend state is emitted from the command buffer when the FS uses
an epilog (either compiled from a lib with GPL or compiled on-demand).

This shouldn't change anything but it will allow to disable using a
PS epilog when the fragment shader doesn't write any color outputs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21341>
2023-02-17 13:19:54 +00:00
Samuel Pitoiset
6d73841d34 radv/amdgpu: only set a new pstate if the current one is different
AMDGPU pstate is per context but if there is multiple AMDGPU contexts
in flight, the kernel can return -EBUSY.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21222>
2023-02-17 11:36:06 +00:00
Samuel Pitoiset
663877e894 Revert "radv: acquire pstate on-demand when capturing with RGP"
This change is wrong for two reasons:
- it hangs most of the time maybe, because changing PSTATE when the
  application is running is broken somehow
- it increases the time between triggering and generating the capture
  considerably, because there is a delay for changing PSTATE

This restores previous logic where PSTATE is set to profile_peak at
logical device creation. Though, it also re-introduces an issue with
multiple logical devices (kernel returns -EBUSY) but this will be
fixed in the next commit.

This fixes GPU hangs when trying to record RGP captures on my NAVI21.
Note that profile_peak is only required for some RDNA2 chips (including
VanGogh).

Cc: mesa-stable
This reverts commit 923a864d94.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21222>
2023-02-17 11:36:06 +00:00
Samuel Pitoiset
685f08f91d radv: add support for rectangularLines
dEQP-VK.*rectangular_line* pass on NAVI21.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21287>
2023-02-17 07:38:10 +00:00
Samuel Pitoiset
3368c0e6f2 radv: reduce maximum line width to 8.0
Using 8191.875 seems to big for the hardware to correctly render wide
rectangular lines. This can also be reproduced with AMDVLK by forcing
rectangularLines = True, and fixed by reducing the maximum size as well.

Other drivers seem to expose that value.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21287>
2023-02-17 07:38:10 +00:00
Daniel Schürmann
f6251b21f9 radv/rt: don't hash maxPipelineRayRecursionDepth
The stack size has no effect on the generated shader anymore.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>
2023-02-16 19:37:25 +00:00
Daniel Schürmann
8e718c5b63 radv/rt: use dynamic_callable_stack_base also for static stack_sizes
This patch also removes rt_pipeline->dynamic_stack_size and replaces
it by checking for rt_pipeline->stack_size == -1u.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>
2023-02-16 19:37:25 +00:00
Daniel Schürmann
2649a1f272 radv/rt: introduce and set rt_pipeline->stack_size
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>
2023-02-16 19:37:25 +00:00
Daniel Schürmann
b338d59047 radv: unconditionally enable scratch for RT shaders
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>
2023-02-16 19:37:25 +00:00
Daniel Schürmann
aa362b4b6f radv: rename shader_info->cs.uses_sbt -> shader_info->cs.is_rt_shader
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>
2023-02-16 19:37:25 +00:00
Konstantin Seurer
72d9604db0 radv: Clean up dynamic RT stack allocation
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>
2023-02-16 19:37:25 +00:00
Timur Kristóf
084d10a702 aco: Remove MTBUF zero operand.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21363>
2023-02-16 17:16:34 +00:00
Timur Kristóf
afdacf4dcc aco: Don't set scalar offset on buffer load instructions when it's zero.
This helps generate slightly more optimal instructions.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21363>
2023-02-16 17:16:34 +00:00
Timur Kristóf
4621ffdec1 aco: Get rid of redundant load_vmem_mubuf function.
Call emit_load directly from visit_load_buffer instead.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21358>
2023-02-16 15:29:37 +00:00
Timur Kristóf
74f1b77046 radv: Move VS input lowering to new file: radv_nir_lower_vs_inputs.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21358>
2023-02-16 15:29:37 +00:00
Timur Kristóf
450e173de0 ac/llvm: Change ac_build_tbuffer_load to take format and channel type.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21358>
2023-02-16 15:29:37 +00:00
Timur Kristóf
0ae778ca59 ac/llvm: Fix ac_build_buffer_load to work with more than 4 channels.
LLVM is unable to select instructions for num_channels > 4, so we
workaround that by manually splitting larger buffer loads.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21358>
2023-02-16 15:29:37 +00:00
Timur Kristóf
a2755fc203 ac/llvm: Fix buffer_load_amd with larger than 32-bit channel sizes.
LLVM is unable to select instructions for larger than 32-bit channel types.
Workaround by using i32 and casting to the correct type later.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21358>
2023-02-16 15:29:37 +00:00
Timur Kristóf
b5b0ded4c1 ac/llvm: Remove "structurized" argument and instead check vindex.
Change ac_build_buffer_load_common and ac_build_tbuffer_load so
the use structurized load when the vindex argument is not NULL.
Adjust callers to match the new behaviour.

This fixes the load_buffer_amd intrinsic with index source.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21358>
2023-02-16 15:29:37 +00:00
Timur Kristóf
881c52ba19 ac: Port ACO's get_fetch_format to ac_get_safe_fetch_size.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21358>
2023-02-16 15:29:36 +00:00
Konstantin Seurer
cf9e1b953c radv: Hash VK_PIPELINE_CREATE_RAY_TRACING_NO_NULL_* flags
Found by inspection.

Fixes: 687a82d ("radv/rt: Handle no-null shader flags")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21320>
2023-02-16 11:42:25 +00:00
Tapani Pälli
0d1c759f4e radv: revert Metro Exodus workaround which was moved to common code
This reverts commit 4397c166c0.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21281>
2023-02-16 07:35:20 +00:00
Pierre-Eric Pelloux-Prayer
affa8a9fb2 amd/surface: fix base_mip_width of subsampled formats
base_mip_width is used in si_compute_copy_image when the
SI_IMAGE_ACCESS_BLOCK_FORMAT_AS_UINT flag is used.

   width = tex->surface.u.gfx9.base_mip_width;

This will be incorrect if we don't adjust it. For instance,
with a 260x256 image, surf_pitch and base_mip_width are
320 before surf_pitch is updated to be 192.

Both need to match, or computing the width from base_mip_width
leads to incorrect result.

Cc: mesa-stable

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21253>
2023-02-16 00:25:17 -05:00
Erik Faye-Lund
29ffc79410 meson: don't pass vk wsi args where they don't belong
Only code that cares about Vulkan WSI should get the corresponding
arguments passed. Otherwise, the Vulkan headers might end up including
other headers that we don't have the correct dependencies passed for.
So let's give those a dedicated variable, and only pass that where it's
actually needed.

Fixes: b39958a3a1 ("anv,nir: Move the ANV YCbCr lowering pass to common code")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8193
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21185>
2023-02-15 18:35:14 +00:00
Samuel Pitoiset
073cced868 radv/ci: add missing expected failures with RADV_PERFTEST=gpl on GFX1100
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21334>
2023-02-15 17:29:10 +00:00
Eric Engestrom
a654a303f8 vk/runtime: keep track of supported instance extensions
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21261>
2023-02-14 23:28:29 +00:00
Georg Lehmann
4fbcd046ce aco: Don't use vcmpx with DPP.
V_CMPX+DPP returns 0 with reads from disabled lanes, unlike V_CMP+DPP (RDNA3 ISA doc, 7.7)

Fixes: baab6f18c9 ("aco: Optimize branching sequence during SSA elimination.")
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20537>
2023-02-14 19:15:17 +00:00
Georg Lehmann
281a505ef0 aco: new 16bit VOP3 opcodes can use opsel
No Foz-DB changes on gfx11.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20705>
2023-02-14 16:14:55 +00:00
Samuel Pitoiset
fa66c8954e radv: ignore registering pipeline libaries with SQTT
They aren't executable pipelines and they might not contain all
shader stages.

This fixes a crash when generating RGP captures with GPL.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21235>
2023-02-14 15:03:45 +00:00
Samuel Pitoiset
5d41d8258a radv: disable DCC for mipmaps on GFX11
It seems broken but can't really figure out why and DCC levels aren't
interleaved on GFX11. Skipping DCC initialization for levels seems to
also fix it but seems safer to disable completely, as a hotfix.

Fixes DCC issues with Hi-Fi Rush, Sonic Frontiers, Hogwarts Legacy
and probably more.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8230
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21186>
2023-02-14 07:15:57 +00:00
Emma Anholt
29ffdd72a1 ci/radv: Add a skip for navi21-llvm for a test that consistently timeouts.
I had seen this in my last round of work, but forgot to git add the skips
file.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21294>
2023-02-14 06:33:11 +00:00
Konstantin Seurer
77f59950fd radv/rra: Hide deferred accel struct data destruction behind an env var
Keeping around copies of the BVHs in CPU memory can cause issues with
Applications creating a large amount of acceleration structures (Control).
This commit adds back the old path of copying acceleration structures
while still keeping the deferred, possibly more accurate path around.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20700>
2023-02-13 20:39:30 +00:00
Konstantin Seurer
5093ea6d41 radv/rra: Find copy memory index when initializing the trace state
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20700>
2023-02-13 20:39:30 +00:00
Chia-I Wu
4e03cfa3c6 radv: fix a hang with binning on CHIP_RENOIR
Using (6, 16) for (context_states_per_bin, persistent_states_per_bin)
causes gpu hang in RDR2 benchmark on CHIP_RENOIR.  Follow radeonsi and
use (3, 8) instead.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21254>
2023-02-13 17:23:53 +00:00
Samuel Pitoiset
214d6d05ed radv simplify compiling graphics shaders with a mask of active NIR stages
Instead of recomputing the same bitfield everywhere.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21237>
2023-02-13 09:15:37 +00:00
Samuel Pitoiset
8dd0b98786 radv: only initialize shader arguments for the active stages
Other stages don't need to be initialized.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21237>
2023-02-13 09:15:37 +00:00
Samuel Pitoiset
d05a02018b radv: use last_vgt_api_stage for determining the last stage with XFB
It's shorter and cleaner.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21237>
2023-02-13 09:15:37 +00:00
Samuel Pitoiset
9f8c563e10 radv: fix importing retained NIR shaders when a lib uses the RETAIN bit
Fixes couple of GPL regressions with VKCTS, I thought this was test
bugs but it's valid behaviour as long as the libary uses
VK_PIPELINE_CREATE_RETAIN_LINK_TIME_OPTIMIZATION_INFO_BIT_EXT.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21245>
2023-02-13 08:17:05 +00:00
Alyssa Rosenzweig
edf78810a7 radv: Use common Get*OpaqueCaptureDescriptorDataEXT
Now stubbed by the Vulkan runtime for you, deduplicate.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by; Samuel Pitoiset <samuel.pitoiset@gmail.com>

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21230>
2023-02-13 07:43:11 +00:00
Hans-Kristian Arntzen
e40ed0f88b radv: Implement VK_ACCESS_2_DESCRIPTOR_BUFFER_READ_BIT_EXT.
Signed-off-by: Hans-Kristian Arntzen <post@arntzen-software.no>
Fixes: 46e0c77 ("radv: implement VK_EXT_descriptor_buffer")
Co-authored-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21271>
2023-02-12 15:45:52 +00:00
Hans-Kristian Arntzen
7efabfbbe4 radv: Fix missing VK_ACCESS_2_SHADER_SAMPLED_READ_BIT.
Cannot be used for SSBO, so ignore SCACHE invalidation.

Signed-off-by: Hans-Kristian Arntzen <post@arntzen-software.no>
Fixes: 8df17163c7 ("radv: implement vkCmdWaitEvents2KHR")
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21271>
2023-02-12 15:45:52 +00:00