Commit graph

7957 commits

Author SHA1 Message Date
Bas Nieuwenhuizen
748b7f80ef radv: Move sparse binding into a dedicated queue.
1) This better reflects the reality that we only have one timeline
   of sparse binding changes.

2) Allows making it a threaded queue from the start in prep of
   explicit sync stuff.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16935>
2023-11-29 17:37:37 +00:00
Bas Nieuwenhuizen
00faefa08e radv: Remove the sparse binding queue from coherent images.
Never access the image on the queue family, so no need.

(Technically not sure if this is needed for Vulkan, somewhat of
 a backstop in case apps do it)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16935>
2023-11-29 17:37:37 +00:00
Bas Nieuwenhuizen
6ff98f9313 radv: Add implementation of cmd buffers for a sparse binding queue.
None of the commands are allowed on these ...

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16935>
2023-11-29 17:37:37 +00:00
Alyssa Rosenzweig
d50d9eccad ac,radv,radeonsi: use common 1D texture lowering
It was pulled from ac.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26397>
2023-11-29 14:04:15 +00:00
Konstantin Seurer
11897376c7 radv/rt: Skip null checks for small case counts
The individual cases make sure the sbt_idx is not null implicitly
because the handles are always != 0.

Totals from 60 (22.56% of 266) affected shaders:
Instrs: 47841 -> 47655 (-0.39%)
CodeSize: 255028 -> 253460 (-0.61%)
Latency: 1179658 -> 1225311 (+3.87%); split: -0.02%, +3.89%
InvThroughput: 224122 -> 232851 (+3.89%); split: -0.02%, +3.92%
Copies: 12049 -> 12043 (-0.05%); split: -0.37%, +0.32%
Branches: 3312 -> 3290 (-0.66%)
PreSGPRs: 3494 -> 3472 (-0.63%)

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25089>
2023-11-28 22:58:35 +00:00
Konstantin Seurer
fe674f67b1 radv/rt: Use a helper for inlining non-recursive stages
So we don't have to write the same logic multiple times.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25089>
2023-11-28 22:58:35 +00:00
Samuel Pitoiset
02ef01fa95 radv: enable DGC preprocessing for IBO
This seems to improve performance for Starfield by +1% and Halo Infinite
by +15%!

Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10025
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26172>
2023-11-28 14:07:37 +00:00
Samuel Pitoiset
e59a16bbb8 radv: use an indirect draw when IBO isn't updated as part of DGC
To remove the dependency on the cmd buffer state.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26172>
2023-11-28 14:07:37 +00:00
Samuel Pitoiset
2807e27231 radv: set the stream VA for DGC graphics
This will be used to emit indirect draws when needed.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26172>
2023-11-28 14:07:37 +00:00
Samuel Pitoiset
88bbdfd23e radv: remove useless NIR instructions when emitting IBO with DGC
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26172>
2023-11-28 14:07:37 +00:00
Marek Olšák
e0a24c7fe0 ac/nir: add kill_layer flag to VS/GS/NGG lowering
When the framebuffer state has only 1 layer, the output has no effect.

Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26274>
2023-11-24 15:37:24 +00:00
Samuel Pitoiset
ad7efdea6e radv: do not set OREO_MODE to fix rare corruption on GFX11
Ported from RadeonSI 3f108e7615.
Seems to be a recommendation from the AMD hw team.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26333>
2023-11-23 17:22:04 +00:00
Samuel Pitoiset
46cc7ffb79 radv: add missing FDCC_CONTROL bits for GFX1103 R2
Ported from RadeonSI.
Found by inspection, untested.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26318>
2023-11-23 13:56:53 +00:00
Samuel Pitoiset
ab34603115 radv: disable TC-compatible HTILE on Tonga and Iceland
According to RadeonSI, TC-compat HTILE have issues on Tonga/Iceland
(first GFX8 chips) and a bunch of games seem to have issues.

Let's disable it instead of using a feature that is known broken.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7101
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3894
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26204>
2023-11-23 12:55:47 +00:00
Samuel Pitoiset
b975d4e800 radv: enable meshShaderQueries on GFX10.3
GFX11 support will come later.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25950>
2023-11-21 15:24:49 +00:00
Samuel Pitoiset
6c7265338d radv: add support for task shader invocations queries on GFX10.3
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25950>
2023-11-21 15:24:49 +00:00
Samuel Pitoiset
623b7033c5 radv: make some gang functions non-static
They will be used to create a gang CS when beginning a query if not
already present.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25950>
2023-11-21 15:24:49 +00:00
Samuel Pitoiset
1b3efecd56 radv: rework gfx10_copy_gds_query() slightly
To prepare for the same function with ACE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25950>
2023-11-21 15:24:49 +00:00
Samuel Pitoiset
b7d668a819 radv: add support for mesh shader invocations queries on GFX10.3
Also emulated with a GDS atomic counter in shaders.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25950>
2023-11-21 15:24:49 +00:00
Samuel Pitoiset
48aabaf225 radv: do not harcode the pipeline stats mask for query resolves
Otherwise, mesh/task shader invocations would be ignored.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25950>
2023-11-21 15:24:49 +00:00
Samuel Pitoiset
272ad65397 radv: bump the pipeline state query size to 14 on GFX10.3
GFX10.3 doesn't natively support mesh/task shader invocations query
and they need to be emulated in shaders. In order to share more code
between GFX10.3 and GFX11, it's easier to use the same size as GFX11.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25950>
2023-11-21 15:24:49 +00:00
Samuel Pitoiset
7b13500a99 radv: define new pipeline statistics indices for mesh/task on GFX11
GFX11 uses pipeline statistics for mesh/task queries but on GFX10.3
they need to be emulated. Though the number of mesh/task shader
invocations would be copied to the pipeline statistics range to
simplify the implementation.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25950>
2023-11-21 15:24:49 +00:00
Samuel Pitoiset
2b93e9a02b radv: add support for mesh primitives queries on GFX10.3
This query is emulated using a GDS atomic counter in shaders.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25950>
2023-11-21 15:24:48 +00:00
Samuel Pitoiset
bc6d29b0ca radv: add radv_physical_device::emulate_mesh_shader_queries for GFX10.3
GFX11 supports them natively but not GFX10.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25950>
2023-11-21 15:24:48 +00:00
Samuel Pitoiset
7d1cc5ec3d radv: rename ps_epilog_inputs to colors for PS epilogs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26231>
2023-11-21 08:47:50 +00:00
Chia-I Wu
16a270f646 radv: disable TC-compat htile on GFX9 in some cases
Similar to commit a38de4c011 ("radv: disable tc_compatible_cmask on
GFX9 in some cases"), GFX9 seems to have issues with TC-compat htile as
well.

I've only seen this with D16 and sample count 4.  But I am not sure
about the exact condition.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10161
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26263>
2023-11-20 20:09:44 +00:00
Samuel Pitoiset
fc044928b2 radv: re-enable sparseResidencyImage3D on POLARIS10+
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7214
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26260>
2023-11-20 08:30:05 +01:00
Qiang Yu
5932990e08 aco,radv: add aco_is_nir_op_support_packed_math_16bit
To be shared by radeonsi and radv.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25990>
2023-11-20 02:20:17 +00:00
Timur Kristóf
d487d9d05b radv: Use SDMA version instead of gfx_level where possible.
The SDMA IP is independent from the GFX IP, so it is technically
wrong to program it based on the GFX level.

This patch changes the RADV SDMA code to use SDMA IP versions
where possible.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26110>
2023-11-18 17:11:18 +01:00
Timur Kristóf
24022d5680 radv: Use GPU info for determining SDMA metadata support.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26110>
2023-11-18 17:11:15 +01:00
Timur Kristóf
d09ad16fd4 ac: Remove CIK prefix from SDMA opcodes.
The vast majority of AMD GPUs (except the very first GCN) have
the same SDMA packet format, so let's just call it SDMA instead
of CIK_SDMA.

(And leave the oldest GPUs with SI_SDMA as they are now.)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26110>
2023-11-18 17:11:00 +01:00
Timur Kristóf
6413b860c4 ac: Rename SDMA max copy size macros to reflect SDMA version.
This naming is more accurate and closer to the HW.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26110>
2023-11-18 17:10:56 +01:00
Rhys Perry
4d2a3b9573 radv: add radv_disable_trunc_coord option
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25911>
2023-11-17 15:15:27 +00:00
Connor Abbott
0d186d356c amd: Enable boolean subgroup lowering
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/218>
2023-11-17 09:45:40 +00:00
Samuel Pitoiset
c5e6edcddf radv: check earlier if a graphics pipeline can force VRS per vertex
When a graphics pipeline already enables VRS, forcing VRS isn't
possible, check this earlier.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26248>
2023-11-17 08:39:48 +00:00
Samuel Pitoiset
61c3796a95 radv: remove redundant check when forcing VRS rates
If force_vrs_per_vertex is TRUE, the primitive shading rate varying is
always written.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26248>
2023-11-17 08:39:48 +00:00
Samuel Pitoiset
ec82b42944 radv: add a missing async compute workaround for Tonga/Iceland
After digging into PAL code again, I figured that Tonga/Iceland are
both affected by a hw bug related to async compute dispatches.

The solution is to change the "threadgroup" dimension mode to the
"thread" dimension mode unconditionally.

This should fix a bunch of issues related to RADV_DEBUG=nocompute on
these GPUs.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7551
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6334
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4679
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26207>
2023-11-16 11:37:43 +00:00
Rhys Perry
a31c928c4f radv: use nir_lower_fp16_casts
This correctly implements RTNE f64->f16.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25566>
2023-11-16 11:02:30 +00:00
Samuel Pitoiset
4239e13ff6 radv: fix a descriptor leak with debug names and host base descriptor set
vk_object_base_finish() needs to be called on the descriptor set in
order to free the debug names.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10098
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26088>
2023-11-16 09:11:06 +00:00
Chia-I Wu
7718bad474 radv: minor clean up to image view extent override
Use the correct image plane consistently.  Compressed formats are
single-planar unless emulated.  This makes no difference in either case
but makes the code more concise.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26106>
2023-11-15 20:29:03 +00:00
Chia-I Wu
2533d0a0e2 radv: fix image view extent override for astc
When doing ASTC decoding, the image has format VK_FORMAT_ASTC_*, the
internal plane 1 has format VK_FORMAT_R8G8B8A8_UNORM, and the view has
format VK_FORMAT_R8G8B8A8_UINT.  It does not need the override for
compressed formats.

Fixes: f97b449e9e ("radv: integrate meta astc compute decoder to radv")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26106>
2023-11-15 20:29:03 +00:00
Konstantin Seurer
90f8cf0f36 radv/sqtt: Handle monolithic RT pipelines
This marks them as Unified in the RGP UI and shows the compute shader
view.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26158>
2023-11-15 13:46:50 +00:00
Konstantin Seurer
b6e09dd521 radv/sqtt: Fix tracing acceleration structure commands
ApiRayTracingSeparateCompiled can only used by trace commands.

Fixes: bfb55d0 ("ac/sqtt,radv/sqtt: Add and use marker for separate RT compilation")
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26158>
2023-11-15 13:46:50 +00:00
Samuel Pitoiset
627d593443 radv: fix registering queues for RGP with compute only
This crashes if the graphics queue isn't created.

Fixes: 930e77e903 ("radv/sqtt: add support for queue info")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10136
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26183>
2023-11-15 10:41:50 +00:00
Tatsuyuki Ishi
538ca7801a radv: Use shader part caching helpers for VS prolog and PS/TCS epilog.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26028>
2023-11-14 13:45:22 +00:00
Tatsuyuki Ishi
611545fbfe radv: Implement helpers for shader part caching.
Currently, shader part caching logic is duplicated between VS prolog and
PS/TCS epilogs. This commit introduces a common abstraction to
deduplicate the code.

Additionally, there are a few design decisions that diverts from the
current implementation:
1. A simple mutex is used instead of reader-writer lock. Prolog/epilog
   constructions are serialized, removing the need to free duplicate
   objects in case of a race.
2. A CS-local cache is used to quickly lookup an entry without holding a
   lock. This eliminates locking in over 99% of cases.
3. A set is used to reduce number of allocations.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26028>
2023-11-14 13:45:22 +00:00
Samuel Pitoiset
dfc125d423 radv: fix VRS subpass attachment when HTILE can't be enabled on GFX10.3
On GFX10.3, VRS rates need to be copied to the HTILE buffer but in some
situations, like for mips, it's not always possible to enable HTILE.
In this case, we can fallback to our internal HTILE buffer and tweak
the depth/stencil registers to use this HTILE buffer.

This fixes a bunch of VRS crashes on GFX10.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26025>
2023-11-14 09:02:46 +00:00
Hans-Kristian Arntzen
f69863fd5a radv/radeonsi: Forward correct GPU instance to umr.
Signed-off-by: Hans-Kristian Arntzen <post@arntzen-software.no>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24578>
2023-11-13 13:37:27 +00:00
Rhys Perry
fd80140723 radv: set prolog as_ls if has_ls_vgpr_init_bug=true
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26111>
2023-11-13 12:09:55 +00:00
Tatsuyuki Ishi
ad4b82e192 radv: Pre-mask misaligned_mask for VS prolog.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26023>
2023-11-13 11:47:42 +00:00