Commit graph

194841 commits

Author SHA1 Message Date
Rohan Garg
a03b307eea tu: enable VK_KHR_shader_relaxed_extended_instruction
The extension only affects non semantic instructions that need no
handling in the backend compiler.

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Acked-by: Connor Abbott <cwabbott0@gmail.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30791>
2024-09-16 15:58:35 +00:00
Rohan Garg
85b8579bf6 radv: enable VK_KHR_shader_relaxed_extended_instruction
The extension only affects non semantic instructions that need no
handling in the backend compiler.

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30791>
2024-09-16 15:58:35 +00:00
Rohan Garg
630656827f nvk: enable VK_KHR_shader_relaxed_extended_instruction
The extension only affects non semantic instructions that need no
handling in the backend compiler.

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30791>
2024-09-16 15:58:35 +00:00
Rohan Garg
2a4fbce27a hk: enable VK_KHR_shader_relaxed_extended_instruction
The extension only affects non semantic instructions that need no
handling in the backend compiler.

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30791>
2024-09-16 15:58:35 +00:00
Rohan Garg
daea7e1651 intel/compiler: use the correct cache enum for loads and stores
Fixes: 74efde7 ('intel/brw/xehp+: Drop redundant arguments of lsc_msg_desc*()')

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30742>
2024-09-16 15:18:31 +00:00
Rohan Garg
b99fd944e8 intel/compiler: version can never be above 11 due to the previous check
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30742>
2024-09-16 15:18:31 +00:00
Samuel Pitoiset
32567f6a2a radv: compute shader hash for shaders created without internal cache
VK_KHR_pipeline_binary allows the application to take full control on
the cache and internal caches (disk or in-memory) can be disabled.

Though the shader hash should still be computed, otherwise all pipeline
binaries have a key with all zeroes.

Fixes: 8802612458 ("radv: advertise VK_KHR_pipeline_binary"
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31163>
2024-09-16 13:53:36 +00:00
Samuel Pitoiset
ad68c83f92 radv: fix copying the pipeline binary key
Only the first byte was copied.

Fixes: be06bfcbed ("radv: add initial support for pipeline binaries")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31163>
2024-09-16 13:53:36 +00:00
Samuel Pitoiset
f88cf56087 radv/rt: skip shaders cache for pipelines created with the capture/replay flag
Otherwise, if a pipeline is found in cache it will use a different
shader arena for allocation and the capture replay shader group handles
won't match.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31188>
2024-09-16 12:29:49 +00:00
Hans-Kristian Arntzen
f3c6bbdd8d radv: Always make sure to write the pipeline binary key.
There's nothing in spec that says that key is only written
if data is also written.

Signed-off-by: Hans-Kristian Arntzen <post@arntzen-software.no>
Fixes: be06bfcbed ("radv: add initial support for pipeline binaries")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31162>
2024-09-16 11:38:07 +00:00
Ganesh Belgur Ramachandra
9aa9f7211e radeonsi: fix decode corruptions with AMD_IMAGE_OPCODES=false on non-CDNA based chips
This fix doesn't concern CDNA based chips such as gfx940 directly
but it fixes corruptions on other chips that use emulated image
loads and stores using buffers when AMD_IMAGE_OPCODES=false.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30891>
2024-09-16 09:31:49 +00:00
Ganesh Belgur Ramachandra
62592674e0 amd: fix incorrect PIPE_INTERLEAVE_BYTES size for CDNA chips
The expected PIPE_INTERLEAVE_BYTES size is ADDR_PIPEINTERLEAVE_256B on
gfx940 (or other CDNA based chips). Since CDNA based chips like gfx940
doesn't support image opcodes, it gets gibberish value from the kernel.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30891>
2024-09-16 09:31:49 +00:00
Pierre-Eric Pelloux-Prayer
ca41fa6a09 radeonsi/sqtt: don't store pipeline->bo in shader->bo
This is useless and causes hangs (eg: with Firefox).

Instead use a custom emit function for the sqtt state, and
add the pipeline BO to the buffer list at this point.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31171>
2024-09-16 08:50:33 +00:00
Pierre-Eric Pelloux-Prayer
a1408e9ad3 radeonsi/sqtt: don't leak the pipeline bo
Pass the reference from si_aligned_buffer_create directly
to pipeline instead.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31171>
2024-09-16 08:50:33 +00:00
Pierre-Eric Pelloux-Prayer
721ccd0292 radeonsi/sqtt: use XXH64_update
No functional change, but this is the way the XXH API is meant
to be used.

Also avoid setting the pipeline_code_hash twice.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31171>
2024-09-16 08:50:32 +00:00
Pierre-Eric Pelloux-Prayer
3ab34c5ffe radeonsi/sqtt: don't store the offsets in the pipeline
We can pass them to si_sqtt_register_pipeline / si_sqtt_add_code_object
directly instead.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31171>
2024-09-16 08:50:32 +00:00
Pierre-Eric Pelloux-Prayer
bda9e1f856 radeonsi: check the pointer before adding an offset
Otherwise callers can't detect errors.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31171>
2024-09-16 08:50:32 +00:00
Pierre-Eric Pelloux-Prayer
7fedd7e178 radeonsi: include the shader key in the sqtt pipeline hash
Otherwise we'll use the wrong shader if the key changed.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31171>
2024-09-16 08:50:32 +00:00
Pierre-Eric Pelloux-Prayer
27644af79b radeonsi/tests: allow to run radeonsi-run-tests.py with AMD_DEBUG
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31171>
2024-09-16 08:50:32 +00:00
Samuel Pitoiset
ba7e3be425 radv: merge radv_emit_epilog() with radv_emit_ps_epilog_state()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31150>
2024-09-16 07:53:00 +00:00
Samuel Pitoiset
26d8f1a306 aco,radv,radeonsi: move has_epilog to the fragment shader info
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31150>
2024-09-16 07:53:00 +00:00
Samuel Pitoiset
b377ddfd04 radv,radeonsi: remove remaining occurrences of TCS epilog
TCS epilog has been removed few months ago.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31150>
2024-09-16 07:53:00 +00:00
David Rosca
e27eeff95d frontends/va: Remove fallback if app doesn't enable packed headers
This was only a best effort fallback and it doesn't always work.
Driver is expected to only output slice NALs when packed headers
are not enabled, so let's just do that.

Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31126>
2024-09-16 06:35:32 +00:00
David Rosca
d3429a7e00 radeonsi: Fix reported minimum width for video encode
Some apps would default to smaller resolutions than we currently report
as supported, despite the hardware being able to encode it. It's also
common for test apps/suites to use small resolutions.
libva-utils/h264encode uses 176x144 resolution by default and vulkan cts
have video clips with the same resolution too which would previously
fail to encode.

Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31109>
2024-09-15 14:22:29 +00:00
Marek Olšák
46b561b7cd radeonsi: fold si_improve_sync_flags logic into si_barrier_before_internal_op
This also implements the same thing for images.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
961c72d666 radeonsi: remove SI_OP_SYNC_CPDMA_BEFORE, always sync CP DMA
We want barriers to only consider compute as a possibility, and CP DMA
has to work with that, which means barriers can't have code specific
to CP DMA.

Always executing the CP DMA sync seems acceptable because CP DMA operations
are usually small, and CP DMA is almost never used on GFX10+.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
88eb1be2fc radeonsi: use si_barrier_{before,after}_internal_op for CP DMA
This makes the CP DMA code simpler and reuses the logic we use for internal
compute shaders.

The only thing that can't be handled in the barrier functions is
"!cp_dma_has_L2 -> SI_CONTEXT_INV_L2" because the barrier functions should
assume that only compute shader coherency is required to make them usable
everywhere, and the CP DMA code has to deal with it.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
0526ea067d radeonsi: remove CP DMA code for GDS & L2 prefetch in the clear/copy_buffer path
We don't need to access GDS with CP DMA, and L2 prefetches don't use this
codepath.

Some local variables are also moved closer to their use.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
0124527569 radeonsi: don't flag both VS and PS partial flush in si_barrier_before_internal_op
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
b73ba7b710 radeonsi: do init/deinit for multi-dispatch compute blits only once
Do it outside the loop, which is more optimal.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
4d6a90ddac radeonsi: split image saving/binding/restoring from si_launch_grid_internal_images
Since the compute blit can execute multiple dispatches in a loop,
we shouldn't save/bind/restore images for every single dispatch.
This will help move that out of the loop.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
69ad0fc61e radeonsi: only set need_check_render_feedback if binding textures for PS
Other shader stages can't realistically read and render to the same
pixel of the same texture simultaneously.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
d4178e58f7 radeonsi: skip no-op si_compute_blit()
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
ef136ab94e radeonsi: move barrier code out of si_launch_grid_internal_ssbos/images
and into the new si_barrier functions.

The new barrier function parameters might be excessive for now, but they
will be used later, hopefully.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
fd04919f4e radeonsi: move compute barrier calls up one level
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
797112d1e2 radeonsi: split si_launch_grid_internal to start preparing for barrier rework
This moves code from si_launch_grid_internal into 4 new functions:
- si_barrier_before_internal_op
- si_barrier_after_internal_op
- si_compute_begin_internal
- si_compute_end_internal

The goal is to move setting the flush flags into callers, which will
eventually become proper memory barriers.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
465f42e907 radeonsi: inline si_get_flush_flags
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
bf77786b79 radeonsi: replace enum si_cache_policy with cp_dma_has_L2()
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
1ce9729cce radeonsi: remove L2_STREAM enum
never used

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
9ed0530413 radeonsi: move enum si_cache_policy and si_get_flush_flags into si_cp_dma.c
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
b0ee7b0b8f radeonsi: don't use si_get_flush_flags in si_launch_grid_internal_ssbos
It didn't do anything. It only set INV_SCACHE and INV_VCACHE, which are
already set optimally by si_launch_grid_internal.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
cfe1f2b841 radeonsi: remove enum si_coherency
SI_COHERENCY_SHADER is the only one that's used.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:45 -04:00
Marek Olšák
a009aa7365 radeonsi: remove all SI_COHERENCY_* flags except SI_COHERENCY_SHADER
it will be simplified in the next commit

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:44 -04:00
Marek Olšák
eb52b774e1 radeonsi: don't use SI_COHERENCY_CP
It only skips flushing SMEM and VMEM caches.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:44 -04:00
Marek Olšák
d9bfe075a5 radeonsi: don't use SI_COHERENCY_CB_META
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:44 -04:00
Marek Olšák
413bdcd7c3 radeonsi: don't use SI_COHERENCY_NONE in si_compute_shorten_ubyte_buffer
This removes the last non-AMD_TEST use of SI_COHERENCY_NONE.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:44 -04:00
Marek Olšák
68adac4d28 radeonsi: move get_cache_policy into si_cp_dma.c
it's the only user

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:44 -04:00
Marek Olšák
8c872d46a8 radeonsi: don't use get_cache_policy in si_launch_grid_internal_ssbos
This is more correct.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:44 -04:00
Marek Olšák
d04150965b radeonsi: remove unused size parameter from get_cache_policy
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:44 -04:00
Marek Olšák
fe18c09703 radeonsi: add si_cp_pfp_sync_me
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31168>
2024-09-14 11:03:44 -04:00