Commit graph

7983 commits

Author SHA1 Message Date
Samuel Pitoiset
7d1cc5ec3d radv: rename ps_epilog_inputs to colors for PS epilogs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26231>
2023-11-21 08:47:50 +00:00
Chia-I Wu
16a270f646 radv: disable TC-compat htile on GFX9 in some cases
Similar to commit a38de4c011 ("radv: disable tc_compatible_cmask on
GFX9 in some cases"), GFX9 seems to have issues with TC-compat htile as
well.

I've only seen this with D16 and sample count 4.  But I am not sure
about the exact condition.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10161
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26263>
2023-11-20 20:09:44 +00:00
Samuel Pitoiset
fc044928b2 radv: re-enable sparseResidencyImage3D on POLARIS10+
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7214
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26260>
2023-11-20 08:30:05 +01:00
Qiang Yu
5932990e08 aco,radv: add aco_is_nir_op_support_packed_math_16bit
To be shared by radeonsi and radv.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25990>
2023-11-20 02:20:17 +00:00
Timur Kristóf
d487d9d05b radv: Use SDMA version instead of gfx_level where possible.
The SDMA IP is independent from the GFX IP, so it is technically
wrong to program it based on the GFX level.

This patch changes the RADV SDMA code to use SDMA IP versions
where possible.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26110>
2023-11-18 17:11:18 +01:00
Timur Kristóf
24022d5680 radv: Use GPU info for determining SDMA metadata support.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26110>
2023-11-18 17:11:15 +01:00
Timur Kristóf
d09ad16fd4 ac: Remove CIK prefix from SDMA opcodes.
The vast majority of AMD GPUs (except the very first GCN) have
the same SDMA packet format, so let's just call it SDMA instead
of CIK_SDMA.

(And leave the oldest GPUs with SI_SDMA as they are now.)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26110>
2023-11-18 17:11:00 +01:00
Timur Kristóf
6413b860c4 ac: Rename SDMA max copy size macros to reflect SDMA version.
This naming is more accurate and closer to the HW.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26110>
2023-11-18 17:10:56 +01:00
Rhys Perry
4d2a3b9573 radv: add radv_disable_trunc_coord option
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25911>
2023-11-17 15:15:27 +00:00
Connor Abbott
0d186d356c amd: Enable boolean subgroup lowering
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/218>
2023-11-17 09:45:40 +00:00
Samuel Pitoiset
c5e6edcddf radv: check earlier if a graphics pipeline can force VRS per vertex
When a graphics pipeline already enables VRS, forcing VRS isn't
possible, check this earlier.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26248>
2023-11-17 08:39:48 +00:00
Samuel Pitoiset
61c3796a95 radv: remove redundant check when forcing VRS rates
If force_vrs_per_vertex is TRUE, the primitive shading rate varying is
always written.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26248>
2023-11-17 08:39:48 +00:00
Samuel Pitoiset
ec82b42944 radv: add a missing async compute workaround for Tonga/Iceland
After digging into PAL code again, I figured that Tonga/Iceland are
both affected by a hw bug related to async compute dispatches.

The solution is to change the "threadgroup" dimension mode to the
"thread" dimension mode unconditionally.

This should fix a bunch of issues related to RADV_DEBUG=nocompute on
these GPUs.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7551
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6334
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4679
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26207>
2023-11-16 11:37:43 +00:00
Rhys Perry
a31c928c4f radv: use nir_lower_fp16_casts
This correctly implements RTNE f64->f16.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25566>
2023-11-16 11:02:30 +00:00
Samuel Pitoiset
4239e13ff6 radv: fix a descriptor leak with debug names and host base descriptor set
vk_object_base_finish() needs to be called on the descriptor set in
order to free the debug names.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10098
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26088>
2023-11-16 09:11:06 +00:00
Chia-I Wu
7718bad474 radv: minor clean up to image view extent override
Use the correct image plane consistently.  Compressed formats are
single-planar unless emulated.  This makes no difference in either case
but makes the code more concise.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26106>
2023-11-15 20:29:03 +00:00
Chia-I Wu
2533d0a0e2 radv: fix image view extent override for astc
When doing ASTC decoding, the image has format VK_FORMAT_ASTC_*, the
internal plane 1 has format VK_FORMAT_R8G8B8A8_UNORM, and the view has
format VK_FORMAT_R8G8B8A8_UINT.  It does not need the override for
compressed formats.

Fixes: f97b449e9e ("radv: integrate meta astc compute decoder to radv")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26106>
2023-11-15 20:29:03 +00:00
Konstantin Seurer
90f8cf0f36 radv/sqtt: Handle monolithic RT pipelines
This marks them as Unified in the RGP UI and shows the compute shader
view.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26158>
2023-11-15 13:46:50 +00:00
Konstantin Seurer
b6e09dd521 radv/sqtt: Fix tracing acceleration structure commands
ApiRayTracingSeparateCompiled can only used by trace commands.

Fixes: bfb55d0 ("ac/sqtt,radv/sqtt: Add and use marker for separate RT compilation")
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26158>
2023-11-15 13:46:50 +00:00
Samuel Pitoiset
627d593443 radv: fix registering queues for RGP with compute only
This crashes if the graphics queue isn't created.

Fixes: 930e77e903 ("radv/sqtt: add support for queue info")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10136
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26183>
2023-11-15 10:41:50 +00:00
Tatsuyuki Ishi
538ca7801a radv: Use shader part caching helpers for VS prolog and PS/TCS epilog.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26028>
2023-11-14 13:45:22 +00:00
Tatsuyuki Ishi
611545fbfe radv: Implement helpers for shader part caching.
Currently, shader part caching logic is duplicated between VS prolog and
PS/TCS epilogs. This commit introduces a common abstraction to
deduplicate the code.

Additionally, there are a few design decisions that diverts from the
current implementation:
1. A simple mutex is used instead of reader-writer lock. Prolog/epilog
   constructions are serialized, removing the need to free duplicate
   objects in case of a race.
2. A CS-local cache is used to quickly lookup an entry without holding a
   lock. This eliminates locking in over 99% of cases.
3. A set is used to reduce number of allocations.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26028>
2023-11-14 13:45:22 +00:00
Samuel Pitoiset
dfc125d423 radv: fix VRS subpass attachment when HTILE can't be enabled on GFX10.3
On GFX10.3, VRS rates need to be copied to the HTILE buffer but in some
situations, like for mips, it's not always possible to enable HTILE.
In this case, we can fallback to our internal HTILE buffer and tweak
the depth/stencil registers to use this HTILE buffer.

This fixes a bunch of VRS crashes on GFX10.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26025>
2023-11-14 09:02:46 +00:00
Hans-Kristian Arntzen
f69863fd5a radv/radeonsi: Forward correct GPU instance to umr.
Signed-off-by: Hans-Kristian Arntzen <post@arntzen-software.no>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24578>
2023-11-13 13:37:27 +00:00
Rhys Perry
fd80140723 radv: set prolog as_ls if has_ls_vgpr_init_bug=true
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26111>
2023-11-13 12:09:55 +00:00
Tatsuyuki Ishi
ad4b82e192 radv: Pre-mask misaligned_mask for VS prolog.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26023>
2023-11-13 11:47:42 +00:00
Tatsuyuki Ishi
55d21f2f12 radv, aco: Inline struct aco_vs_input_state.
Now that we no longer use the radv_vs_input_state pointer, we can simply
inline all the state-related fields.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26023>
2023-11-13 11:47:42 +00:00
Tatsuyuki Ishi
3fc3a94bce radv, aco: Rework VS prolog key handling.
The main change is to use struct radv_vs_prolog_key directly instead of
the compressed representation to simplify an upcoming rework in prolog /
epilog caching. In doing so the state struct pointer was replaced with
an inline struct.

Care was also taken to pre-mask all the states with the active attribute
mask and other masks when it makes sense; this ensures that we don't
accidentally use information not hashed into the key during compilation.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26023>
2023-11-13 11:47:42 +00:00
Tatsuyuki Ishi
5cc7f54f15 radv: Remove last VS prolog reuse logic.
This was broken as the field was never assigned to. This will also be
dropped from the upcoming prolog/epilog lookup rework, as it adds to
code complexity while the benefit of saving one hash table memory access
seems questionable.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26023>
2023-11-13 11:47:41 +00:00
Tatsuyuki Ishi
d8a5b76307 aco: Replace aco_vs_input_state.divisors with bitfields.
Instead of concrete divisor value, we only pass down the information
whether the divisor is zero or nontrivial (>1).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26023>
2023-11-13 11:47:41 +00:00
Samuel Pitoiset
9b840df9f6 radv: add drirc options to force re-compilation of shaders when needed
On Steam Deck, shaders are pre-compiled for better performance (less
stuttering, less CPU usage, etc). But when a compiler fix needs to be
backported, there is currently no way to handle this properly.

This introduces 3 drirc options
radv_override_{graphics,compute,ray_tracing}_shader_version in order to
force the driver to re-compile pipelines when needed. By default, the
shader version is 0 for all pipelines.

When one drirc is set for a specific game, RADV will re-compile all
pipelines only once with the compiler fix included (because the
pipeline key would be different).

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26094>
2023-11-13 10:47:26 +00:00
Samuel Pitoiset
88cbe32048 radv: add support for RGP queue events
This can be used for analysing queue submissions.

This is still experimental and it can be disabled with
RADV_THREAD_TRACE_QUEUE_EVENT=false if you have issues with it.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22779>
2023-11-13 08:53:10 +00:00
Samuel Pitoiset
33f4307796 radv: add radv_write_timestamp() helper
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22779>
2023-11-13 08:53:10 +00:00
Samuel Pitoiset
661d28cfec radv: restore sampling CPU/GPU clocks before starting SQTT trace
This was removed by mistake in "radv/rgp: Use common trace trigger"
and it introduced a regression with RGP queue events.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22779>
2023-11-13 08:53:09 +00:00
Samuel Pitoiset
55e48d7e0f radv: enable DGC preprocessing when all push constants are inlined
It's not possible when they aren't all inlined because they need to be
copied to the upload BO and the DGC shader also copies the ones that
come from the indirect layout.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25935>
2023-11-13 08:28:53 +00:00
Samuel Pitoiset
d067413a72 radv: optimize emitting inlined push constants with DGC
With DGC, push constants can be set from the cmdbuf (CmdPushConstants())
or from the indirect layout. Instead of always emitting inlined push
constants from the DGC shader, just update the ones that come from the
indirect layout and rely on cmdbuf updates for the other ones.

With that, it should be possible to preprocess push constants with
graphics when all can be inlined in shaders.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25935>
2023-11-13 08:28:53 +00:00
Samuel Pitoiset
c8140e4c0e radv: emit individual SET_SH_REG for inlined push constants with DGC
This should allow to preprocess if everything is inlined.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25935>
2023-11-13 08:28:53 +00:00
Samuel Pitoiset
0112a245f5 radv: add a helper to determine if it's possible to preprocess DGC
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25935>
2023-11-13 08:28:53 +00:00
Tatsuyuki Ishi
325d650c6e radv: Use sizeof(flags) instead of hardcoded size in radv_hash_shaders.
For consistency.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26145>
2023-11-11 05:26:27 +00:00
Tatsuyuki Ishi
ef414a8231 radv: Fix mis-sizing of pipeline_flags in radv_hash_rt_shaders.
pipeline_flags was 64-bit yet only the first 4 bytes were hashed.
Luckily, the mask included no flag above the 32nd bit, so this was
technically working fine. Still, it's better to use explicit sizeof
constructs to be more resilient to accidental type changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26145>
2023-11-11 05:26:26 +00:00
Rhys Perry
2011b678c5 radv: disable gs_fast_launch=2 by default
This currently causes severe performance problems in Remnant 2 and
Alan Wake 2.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: 28ebe62af2 ("radv: enable mesh shader gs_fast_launch=2 and multi-row export")
Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10071
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26090>
2023-11-07 19:54:47 +00:00
Georg Lehmann
ab87831ae8 aco, radv: vectorize f2f16 if rounding mode is rtz
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25952>
2023-11-06 21:05:34 +00:00
Connor Abbott
55f3f952aa vk/graphics_state, tu: Rewrite renderpass flags handling
Before this, the render pass code or the driver combined the pipeline
create flags and the implicit flags from the render pass, but the
pipeline create flags will need to be sanitized when they are dynamic
state, so we need to do it in vk_graphics_state where we know that
information.

We also weren't combining pipeline flags correctly when linking, which
on turnip was being hidden by the lack of sanitizing for driver-provided
flags. We can't combine them correctly if they're part of the render
pass state, so they need to be pulled out into the overall pipeline
state.

For drivers using emulated renderpasses or tracking feedback loop
information themselves, this won't make a difference, but we have to
adapt turnip to not pass pipeline flags. This also means that we can
drop all handling of feedback_loop_input_only in turnip and just set it
in the runtime.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25436>
2023-11-06 14:33:51 +00:00
Connor Abbott
e6f5d7222c vk,lvp,tu,radv,anv: Add common vk_*_pipeline_create_flags() helper
And replace the various homegrown or copy-pasted helpers in drivers.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25436>
2023-11-06 14:33:51 +00:00
Samuel Pitoiset
790fabd38e radv: advertise VK_EXT_device_fault
This is only exposed if the kernel supports GPUVM fault query.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25875>
2023-11-06 10:35:05 +00:00
Samuel Pitoiset
8097becc7f radv: add initial VK_EXT_device_fault support
This implementation only returns VM faults information for now, but
vendor crash dumps will be adder later.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25875>
2023-11-06 10:35:05 +00:00
Jesse Natalie
228329f4da vulkan: Consolidate common ICD methods
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25998>
2023-11-03 20:01:14 +00:00
Chia-I Wu
227300345e radv: stop using vk_render_pass_state::render_pass
vk_render_pass_state::pipeline_flags is derived from
vk_get_pipeline_rendering_flags and has the info we need.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10074
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26000>
2023-11-03 17:23:30 +00:00
Konstantin Seurer
11282598e6 radv: Add radv_nir_lower_hit_attrib_derefs_tests
Tests hit attrib lowering for various variable/type configurations.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24271>
2023-11-02 15:48:36 +00:00
Konstantin Seurer
f51227d253 radv/clang-format: Do not indent C++ modifiers
Turns

class asd {
 private:
};

into

class asd {
private:
};

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24271>
2023-11-02 15:48:36 +00:00