Commit graph

9469 commits

Author SHA1 Message Date
Mike Blumenkrantz
a524f6b5d4 radv: fix external handle type queries for dmabuf/fd
the intent of the VkExternalMemoryProperties API is that all compatible
handle types are returned, not just the type being queried. these two
types are compatible, so return both when both are supported

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25474>
2023-09-29 18:57:13 +00:00
Samuel Pitoiset
83cad95358 radv: emit missing PA_{SC,SU}_LINE_STIPPLE_xxx regs in gfx preamble
Ported from RadeonSI and PAL.
This might fix the line stipple flakes in CI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25428>
2023-09-29 07:50:46 +00:00
Samuel Pitoiset
a9945216ba radv: fix COMPUTE_SHADER_INVOCATIONS query on compute queue
The VA needs to be adjusted, otherwise the hw always writes at offset 0.

This fixes dEQP-VK.query_pool.statistics_query.*_cq.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25406>
2023-09-29 07:16:44 +00:00
Samuel Pitoiset
c1b916ccb9 radv: re-enable DCC with mipmaps on GFX11
This is now fixed.

Hi-Fi Rush, Sonic Frontiers and Hogwarts Legacy were known broken games.

I personally reproduced the issue with Hi-Fi Rush which has been fixed
since e6735409ee ("radv: disable DCC with signedness reinterpretation
on GFX11"). I also tested Sonic Frontiers which has been fixed since
52b6886992 ("amd: update addrlib"). I didn't check Hogwarts Legacy but
I think it was also fixed by e6735409ee.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25435>
2023-09-29 06:54:42 +00:00
Samuel Pitoiset
af2a96bb37 radv: stop skip emitting CB states when there is no color attachment
This is actually wrong. For example, if there is a DCC decompress
draw followed by a draw without any color attachments,
CB_COLOR_CONTROL.MODE is still CB_DCC_DECOMPRESS but it should be
CB_DISABLED. For some reasons, this hangs on RDNA3 (VM faults are also
reported through dmesg).

This fixes GPU hangs with Resident Evil 6, Star Wars The Old Republic
and probably more games on RDNA3.

Strictly speaking, I don't think this dynamic state optimization is
worth a try, even for other states, and I think it would be safer to
remove it completely.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9335
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8327
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9878
Fixes: c08082e861 ("radv: ignore all CB dynamic states when there is no color attachments")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25402>
2023-09-28 13:32:35 +00:00
Samuel Pitoiset
7e00ec1c6f radv: declare shader_query_state for mesh/task shaders
This will control whether mesh/task queries is enabled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25331>
2023-09-26 07:50:16 +00:00
Samuel Pitoiset
fae4360a08 radv: enable lowering of mesh/task shader queries when enabled
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25331>
2023-09-26 07:50:16 +00:00
Samuel Pitoiset
918a57579f radv: adjust lowering of intrinsic queries for mesh/task shaders
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25331>
2023-09-26 07:50:16 +00:00
Samuel Pitoiset
8ea9680890 radv: add GDS counters offset for mesh/task queries
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25331>
2023-09-26 07:50:16 +00:00
Samuel Pitoiset
9a3b902cac ac/nir: add lowering for task shader queries
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25331>
2023-09-26 07:50:15 +00:00
Samuel Pitoiset
57dec0678e ac/nir: add lowering for mesh shader queries
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25331>
2023-09-26 07:50:15 +00:00
Samuel Pitoiset
1ce80653b2 nir: rename atomic_add_gs_invocation_count_amd to make it more generic
It will be re-used to implement mesh/tash shader invocations queries.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25331>
2023-09-26 07:50:15 +00:00
Samuel Pitoiset
219a14d093 radv: enable cache counters for RGP on GFX11
This was the last missing RGP feature on GFX11 compared to older chips.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25271>
2023-09-25 09:05:58 +02:00
Samuel Pitoiset
70a4290e28 radv: add SPM support for GFX11
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25271>
2023-09-25 09:05:58 +02:00
Caio Oliveira
3988d901ac meson: Remove unnecessary inc_compiler mentions
The inc_compiler should come as part of idep_compiler, idep_nir or
idep_nir_headers dependency.

Acked-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> (v3dv)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25314>
2023-09-22 14:52:50 +00:00
Connor Abbott
c93bcb32fe amd: Use inverse ballot intrinsic if available
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25123>
2023-09-20 14:41:18 +00:00
Connor Abbott
4282386311 nir/spirv: Add inverse_ballot intrinsic
This is actually a no-op on AMD, so we really don't want to lower it to
something more complicated.  There may be a more efficient way to do
this on Intel too. In addition, in the future we'll want to use this for
lowering boolean reduce operations, where the inverse ballot will
operate on the backend's "natural" ballot type as indicated by
options->ballot_bit_size, instead of uvec4 as produced by SPIR-V. In
total, there are now three possible lowerings we may have to perform:

- inverse_ballot with source type of uvec4 from SPIR-V to inverse_ballot
with natural source type, when the backend supports inverse_ballot
natively.
- inverse_ballot with source type of uvec4 from SPIR-V to arithmetic,
when the backend doesn't support inverse_ballot.
- inverse_ballot with natural source type from reduce operation, when
the backend doesn't support inverse_ballot.

Previously we just did the second lowering unconditionally in vtn, but
it's just a combination of the first and third. We add support here for
the first and third lowerings in nir_lower_subgroups, instead of simply
moving the second lowering, to avoid unnecessary churn.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25123>
2023-09-20 14:41:18 +00:00
Konstantin Seurer
2993853f49 radv/rt: Skip cull_mask handling if it is FF
Totals from 9 (1.32% of 680) affected shaders:
Instrs: 609329 -> 609057 (-0.04%)
CodeSize: 3267328 -> 3265664 (-0.05%)
Latency: 8289582 -> 8275874 (-0.17%)
InvThroughput: 2166498 -> 2163147 (-0.15%)
VClause: 23581 -> 23583 (+0.01%)
Copies: 51076 -> 51028 (-0.09%)
Branches: 24637 -> 24603 (-0.14%)
PreVGPRs: 996 -> 986 (-1.00%)

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25268>
2023-09-20 13:00:03 +00:00
Konstantin Seurer
e0cf4fbf38 radv/ray_queries: Skip cull_mask handling if it is FF
Stats for Metro Exodus:

Totals from 26 (0.99% of 2627) affected shaders:
Instrs: 14586 -> 14232 (-2.43%)
CodeSize: 77024 -> 75192 (-2.38%)
VGPRs: 1408 -> 1208 (-14.20%)
Latency: 315076 -> 309898 (-1.64%)
InvThroughput: 42345 -> 41677 (-1.58%)
VClause: 366 -> 374 (+2.19%)
Copies: 2840 -> 2800 (-1.41%); split: -1.48%, +0.07%
Branches: 587 -> 561 (-4.43%)
PreSGPRs: 897 -> 853 (-4.91%)
PreVGPRs: 1290 -> 1122 (-13.02%)

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25268>
2023-09-20 13:00:03 +00:00
Konstantin Seurer
3e7850f97b radv/bvh: Treat instances with mask == 0 as inactive
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25268>
2023-09-20 13:00:03 +00:00
Samuel Pitoiset
67ed899cd6 radv: remove absolute_depth_bias workaround
This was only used with Path of Exile and the game bug seems fixed.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25198>
2023-09-19 07:53:51 +00:00
Samuel Pitoiset
041d1150c1 radv: fix missing ISA with RGP and GPL
The pipeline hash is required for RGP to correctly report the ISA, so
it should be computed for fast-linked pipelines with GPL (libraries
aren't captured).

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9169
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25275>
2023-09-19 06:50:59 +00:00
Samuel Pitoiset
c314bc2ab9 radv: fix checking if RGP is enabled with others tracing tools
This is a bitmask.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25275>
2023-09-19 06:50:59 +00:00
Alyssa Rosenzweig
d1eb17e92e treewide: Drop nir_ssa_for_src users
Via Coccinelle patch:

    @@
    expression b, s, n;
    @@

    -nir_ssa_for_src(b, *s, n)
    +s->ssa

    @@
    expression b, s, n;
    @@

    -nir_ssa_for_src(b, s, n)
    +s.ssa

Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25247>
2023-09-18 10:25:17 -04:00
Rhys Perry
f6f9c07efe radv: disable mesh dispatch XYZ_DIM when possible
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25222>
2023-09-18 12:04:06 +00:00
Samuel Pitoiset
d5a5473185 ac,radv,radeonsi: prepare support for multi-instance SPM generic counters
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25240>
2023-09-18 07:07:31 +00:00
Samuel Pitoiset
ed0d3d8cbd ac,radv,radeonsi: prepare support for multi-instance SPM SQ counters
Each SQG modules can configure up to 16 counters.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25240>
2023-09-18 07:07:31 +00:00
Samuel Pitoiset
0e4d5b171a radv,radeonsi: make sure to emit GRBM_GFX_INDEX before SQ select registers
This was missing, but not sure if it was required given that only the
first SE is currently sampled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25240>
2023-09-18 07:07:31 +00:00
Samuel Pitoiset
037d7d0f5b radv: reserve more CS space in SQTT/SPM paths
This will prevent an assertion when SPM will emit more counters.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25240>
2023-09-18 07:07:31 +00:00
Konstantin Seurer
28dcc5959d radv/rt: Handle stages without nir properly
Fixes: e039e3cd76 ('radv/rt: Store NIR shaders separately')
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25142>
2023-09-14 16:07:46 +00:00
Konstantin Seurer
77bf1408f3 radv: Don't advertise features requiring PS epilogs with LLVM
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25010>
2023-09-14 15:39:39 +00:00
Konstantin Seurer
97b1caf9f6 radv: Perform multiple sorts in parallel
This was the last part that didn't scale with multiple infos. Reducing
the amount of barriers in this case improves DOOM Eternal performance by
50%. (Running with low resolution)

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24720>
2023-09-14 15:12:44 +00:00
Konstantin Seurer
44c47054bc radv/radix_sort: Vendor the radix sort dispatch code
This needs to be done so we can optimize it for occpuancy when building
multiple acceleration structures in parallel. Changes to the original
code:

- Change // to /* */
- clang-format
- Replace vkCmd calls with calls to the driver entrypoints
- Add a light weight info struct
- Use radv_fill_buffer directly

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24720>
2023-09-14 15:12:44 +00:00
Konstantin Seurer
1cacc64ea7 radv: Remove dead radix_sort_vk_get_memory_requirements call
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24720>
2023-09-14 15:12:43 +00:00
Samuel Pitoiset
303184e4e5 radv,radeonsi: use AC_SPM_SEGMENT_TYPE_xxx instead of magic values
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25211>
2023-09-14 14:17:18 +00:00
Samuel Pitoiset
db6e16a515 radv: enable the PKT3 CAM bit for some SPM register writes
PAL does that.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25211>
2023-09-14 14:17:18 +00:00
Samuel Pitoiset
31e6c05527 ac,radv,radeonsi: rework SPM counters configuration and share it
This should be easier to add GFX11 support.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25175>
2023-09-14 12:30:52 +00:00
Samuel Pitoiset
f8a7c8edd1 radv: emit relocation for mesh/task shaders
RGP requires shaders to be contiguous in memory.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25144>
2023-09-14 07:19:26 +00:00
Samuel Pitoiset
312103e0ff radv: set THREAD_TRACE_MARKER_ENABLE for mesh/task draws
PAL does that.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25144>
2023-09-14 07:19:25 +00:00
Samuel Pitoiset
12f1c5d23b radv: fix instruction timing on GFX11
GFX11 seems to operate on the last active CU.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25171>
2023-09-14 06:29:10 +00:00
Samuel Pitoiset
c04b10155e radv: disable unsupported hw shader stages for RGP on GFX11+
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25171>
2023-09-14 06:29:10 +00:00
Samuel Pitoiset
4f5dd54713 radv: set THREAD_TRACE_TOKEN_MASK.BOP_EVENTS_TOKEN_INCLUDE on GFX10.3+
PAL does that.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25171>
2023-09-14 06:29:10 +00:00
Dave Airlie
3740b6f599 radv/video: take db alignment into account when allocating images.
Make sure to take the db alignment into account when sizing the underlying
images.

Fixes a 360p sample from Lynne.

Cc: mesa-stable
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Lynne <dev@lynne.ee>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25168>
2023-09-14 01:48:49 +00:00
Dave Airlie
147c17c28c radv: don't emit event code on video queues.
I don't know if these can be done properly, but for now just don't
emit the standard cp stuff since it hangs the GPU.

"Fixes" dEQP-VK.video.synchronizat*

Cc: mesa-stable
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25170>
2023-09-12 21:42:28 +00:00
Rhys Perry
f9b341cbf4 radv: use shortcut_1d_workgroup_id
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24882>
2023-09-12 14:31:07 +00:00
Rhys Perry
2965a9d510 radv: optimize mesh workgroup ID using ts_mesh_dispatch_dimensions
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24882>
2023-09-12 14:31:07 +00:00
Samuel Pitoiset
2cc981a0cd radv: fix capturing RGP on RDNA3 with more than one Shader Engine
PKT3_RESET_FILTER_CAM_S seems required on GFX11. Otherwise, capturing
with more than on SE can hang.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25156>
2023-09-12 08:38:39 +00:00
Samuel Pitoiset
6caae898dd radv: fix emitting SQTT userdata when CAM is needed
The third parameter of PKT3 is the predicate bit and this was wrong.
PAL sets the RESET_FILTER_CAM bit when emitting SQTT userdata.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25158>
2023-09-12 07:07:10 +00:00
Alba Mendez
e3aab80d4f meson: support installation tags
the only thing missing is to tag custom install targets, namely:
 - ICDs (vulkan, glvnd, OpenCL, VA-API, VDPAU)
 - Vulkan layers
 - custom install script for megadrivers
 - drirc file

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24911>
2023-09-11 13:00:45 +00:00
Samuel Pitoiset
729cb4004a radv: fix enabling DGCC
This was broken if only DGC (graphics) is enabled.

Fixes: 559da06755 ("radv: implement NV_device_generated_commands_compute")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25126>
2023-09-11 07:13:10 +00:00