Commit graph

13866 commits

Author SHA1 Message Date
Karol Herbst
f2b7c4ce29 nir: rework and fix rotate lowering
No driver supports urol/uror on all bit sizes. Intel gen11+ only for 16
and 32 bit, Nvidia GV100+ only for 32 bit. Etnaviv can support it on 8,
16 and 32 bit.

Also turn the `lower` into a `has` option as only two drivers actually
support `uror` and `urol` at this momemt.

Fixes crashes with CL integer_rotate on iris and nouveau since we emit
urol for `rotate`.

v2: always lower 64 bit

Fixes: fe0965afa6 ("spirv: Don't use libclc for rotate")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by (Intel and nir): Ian Romanick <ian.d.romanick@intel.com>

Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27090>
2024-01-22 10:27:44 +00:00
Samuel Pitoiset
783e3c096f radv: fix indirect draws with NULL index buffer on GFX10
GFX10 has a hw bug and it can't handle 0-sized index buffer. The
non-indirect draw path was fine but not the indirect path where RADV
emits the index buffer.

This fixes flakes with dEQP-VK.*maintenance6* on NAVI14, and possibly
GPU hangs if there is an indirect draw with a valid index buffer right
before because it would re-use the same index buffer.

Fixes: db9816fd66 ("radv: add support for NULL index buffer")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27142>
2024-01-22 08:14:23 +00:00
Samuel Pitoiset
5c03cdbd02 radv: fix indirect dispatches on the compute queue on GFX7
GFX7 CP requires the indirect dispatch VA to be aligned to 32-bytes.

This fixes dEQP-VK.api.command_buffers.many_indirect_disps_on_secondary,
but it's unexpected that it uncovered this bug.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27148>
2024-01-22 07:41:05 +00:00
Konstantin Seurer
a9c057d5a3 radv: Implement NIR debug printf
Adds radv_build_printf which can print nir_defs. The arguments are written
to a buffer and printed after the submit.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25753>
2024-01-20 16:11:55 +01:00
Georg Lehmann
0a03cf5b3c aco: remove boolean shuffle isel
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27116>
2024-01-19 20:13:34 +00:00
Georg Lehmann
2571580b2b radv: lower boolean shuffle
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27116>
2024-01-19 20:13:34 +00:00
Georg Lehmann
37a15ba53a ac/llvm: implement as_uniform and ballot_relaxed
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27116>
2024-01-19 20:13:34 +00:00
Georg Lehmann
6b031daf16 aco: implement as_uniform and ballot_relaxed
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27116>
2024-01-19 20:13:34 +00:00
Georg Lehmann
74fc2e287f aco: stop scheduling at p_logical_end
No Foz-DB changes, but this fixes some issues when the spiller inserts
scratch loads after p_logical_end for p_return.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27119>
2024-01-19 17:04:28 +00:00
Daniel Schürmann
e3098bb232 aco: give spiller more room to assign spilled SGPRs to VGPRs
On chordal graphs, a greedy coloring can be done in a way that never uses
more colors than are required for the largest clique. However, since we
have vector values and force phi resources into the same spill slots, the
interference graphs are not chordal, and thus, this assumption doesn't hold.

Use twice as many spill slots as upper bound.

Totals from 10 (0.01% of 79242) affected shaders: (GFX11)
MaxWaves: 52 -> 54 (+3.85%)
Instrs: 271386 -> 271779 (+0.14%)
CodeSize: 1362544 -> 1365432 (+0.21%)
VGPRs: 2536 -> 2532 (-0.16%)
SpillVGPRs: 778 -> 818 (+5.14%)
Scratch: 73472 -> 76800 (+4.53%)
Latency: 3331718 -> 3328798 (-0.09%); split: -0.14%, +0.05%
InvThroughput: 1665860 -> 1643350 (-1.35%); split: -1.40%, +0.05%
VClause: 3292 -> 3329 (+1.12%); split: -0.06%, +1.18%
Copies: 46082 -> 46257 (+0.38%)

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27011>
2024-01-19 14:15:27 +00:00
Eric Engestrom
7a7aa01a49 amd/ci: consider much more of dEQP-VK.query_pool.statistics_query.host_query_reset.* to be flaky
Instead of continuously adding more and more lines to this list.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27170>
2024-01-19 11:12:46 +00:00
Eric Engestrom
7c4a797bc9 amd/ci: add flakes seen today
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27149>
2024-01-19 10:08:27 +00:00
Eric Engestrom
dcc8c8aa4a radv/ci: sort navi21 flakes
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27149>
2024-01-19 10:08:27 +00:00
Friedrich Vock
43bdfebbff radv: Fix shader replay allocation condition
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26891>
2024-01-19 08:15:12 +00:00
Joshua Ashton
86954c766f radv: Ensure vkGetQueryPoolResults returns in finite-time
When a device lost occured, we were not ensuring that
vkGetQueryPoolResults with VK_QUERY_RESULT_WAIT_BIT was returning.

We could have the caller stuck in our while(...) loop polling the query
atomics forever which violates the finite-time guarantees of this function.

This solves that by adding a timeout to this call to be double the
default TDR timeout for the potential queues relating to this query.

Signed-off-by: Joshua Ashton <joshua@froggi.es>

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27091>
2024-01-19 07:52:01 +00:00
Joshua Ashton
907fe6ea62 radv: Add radv_get_tdr_timeout_for_ip helper
Signed-off-by: Joshua Ashton <joshua@froggi.es>

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27091>
2024-01-19 07:52:01 +00:00
Joshua Ashton
ed4ef4dc65 radv/amdgpu: Remove ctx_query_reset_status
We don't use this anymore, it is all dead code.

Signed-off-by: Joshua Ashton <joshua@froggi.es>

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27091>
2024-01-19 07:52:01 +00:00
Joshua Ashton
f62bdde703 radv: Remove check_status
Following discussion on kernel mailing list[1], we are not gaining
anything from this right now, and it does not handle soft recovery.

We will hear about the context loss and rationale when we vkQueueSubmit
next.

We can come back to this if there is ever a Vulkan extension for
figuring out innocent vs guilty like GL_EXT_robustness.

This does mean however that we return VK_SUCCESS for cancelled semaphore
and fence waits, but this is legal per the Vulkan spec:

"Commands that wait indefinitely for device execution (namely
vkDeviceWaitIdle, vkQueueWaitIdle, vkWaitForFences with a maximum
timeout, and vkGetQueryPoolResults with the VK_QUERY_RESULT_WAIT_BIT
bit set in flags) must return in finite time even in the case of a lost
device, and return either VK_SUCCESS or VK_ERROR_DEVICE_LOST."

"If device loss occurs (see Lost Device) before the timeout has expired,
vkWaitSemaphores must return in finite time with either VK_SUCCESS or
VK_ERROR_DEVICE_LOST."

[1]: https://lists.freedesktop.org/archives/amd-gfx/2024-January/103337.html

Signed-off-by: Joshua Ashton <joshua@froggi.es>

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27091>
2024-01-19 07:52:01 +00:00
Joshua Ashton
b24a4b8949 radv: Mark device loss if QueueSubmit failed immediately
Otherwise we rely on check_status alone, and that's going away.

Signed-off-by: Joshua Ashton <joshua@froggi.es>

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27091>
2024-01-19 07:52:01 +00:00
Joshua Ashton
c58cff3ce4 radv/amdgpu: Handle -ENODATA and -ETIME from cs_submit
This can be sent in the event of a soft/hard recovery.

Signed-off-by: Joshua Ashton <joshua@froggi.es>

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27091>
2024-01-19 07:52:01 +00:00
Samuel Pitoiset
7a0b343495 aco: silent checking if clrxdisasm is available
Otherwise, this is reported a ton of times and the CI output is
unusable.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27136>
2024-01-19 07:33:56 +00:00
Konstantin Seurer
e83ef27ba6 radv: Vectorize load_global_constant
Totals from 325 (0.38% of 84845) affected shaders:
Instrs: 2883767 -> 2880939 (-0.10%); split: -0.10%, +0.00%
CodeSize: 15388396 -> 15364976 (-0.15%); split: -0.16%, +0.00%
Latency: 20212583 -> 20197719 (-0.07%); split: -0.20%, +0.13%
InvThroughput: 5489738 -> 5489551 (-0.00%); split: -0.14%, +0.14%
VClause: 104225 -> 103944 (-0.27%); split: -0.30%, +0.03%
SClause: 71616 -> 71612 (-0.01%); split: -0.01%, +0.01%
Copies: 252143 -> 252284 (+0.06%); split: -0.10%, +0.15%
Branches: 117826 -> 117821 (-0.00%); split: -0.01%, +0.00%
PreVGPRs: 23502 -> 23506 (+0.02%)

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27084>
2024-01-18 21:30:39 +00:00
Konstantin Seurer
220c912080 ac/llvm: Enable helper invocations for quad OPs
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9239
cc: mesa-stable

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27110>
2024-01-18 21:00:06 +00:00
Hans-Kristian Arntzen
4ab6d7e753 wsi/x11: Add drirc option to ignore SUBOPTIMAL.
Talos Principle and related games on that engine fail to handle
SUBOPTIMAL properly. Adds an option to ignore SUBOPTIMAL and pretend
it's SUCCESS.

Signed-off-by: Hans-Kristian Arntzen <post@arntzen-software.no>
Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27037>
2024-01-18 17:23:32 +00:00
Dave Airlie
f33683e4da radv: don't submit empty command buffers on encoder ring.
the vcn enc/unified rings don't do nop packets, and hang with 0 sized
cmd buffers. This just stops submitting 0 sized cmd buffers to the hw.

Fixes hangs with dEQP-VK.video.decode.h264_i on navi3x

Cc: mesa-stable
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25932>
2024-01-18 13:33:25 +10:00
Dave Airlie
d32f2ee7b6 radv/video: refactor sq start/end code to avoid decode hangs.
The extra cmd buffer layer was done wrong, need to emit the
sq start and ends around every reset/decode packet.

Fixes dEQP-VK.video.decode.h264_i on navi3x

Fixes: d8f3060bd9 ("radv/video: start adding gfx11 vcn decoder")
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25932>
2024-01-18 13:33:10 +10:00
Samuel Pitoiset
10e2dbb63b radv: delay emitting streamout enable at draw time
Since Vulkan 1.3.271, the spec allowed vkCmdBeginTransformFeedbackEXT
to be called without an active graphics pipeline bound when using
shader objects.

That means that the last VGT shader would be NULL once VKCTS is
updated accordingly. This change delays emitting streamout enable at
draw time to make sure the last VGT shader is present, regarldess if
ESO is enabled or not.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27075>
2024-01-17 17:33:17 +00:00
Eric Engestrom
bdfce158bd amd/ci: add flakes seen today
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27117>
2024-01-17 17:01:59 +00:00
Friedrich Vock
a9831caa14 radv/rt: Add workaround to make leaves always active
DOOM Eternal builds acceleration structures with inactive primitives and
tries to make them active in later AS updates. This is disallowed by the
spec and triggers a GPU hang. Fix the hang by working around the bug.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27034>
2024-01-17 15:14:48 +00:00
Samuel Pitoiset
ce200669b1 radv: rename radv_pipeline_key to radv_graphics_pipeline_key
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27073>
2024-01-17 12:08:24 +00:00
Samuel Pitoiset
c5a5ab46c9 radv: remove radv_generate_pipeline_key()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27073>
2024-01-17 12:08:24 +00:00
Samuel Pitoiset
72d2eb39fb radv: pass radv_graphics_state_key to radv_hash_shaders()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27073>
2024-01-17 12:08:24 +00:00
Marek Olšák
b9b00a0e7a ac,radeonsi: emulate GS primitive pipeline stat on gfx11 because of culling
GS culls too, so the pipeline stat is incorrect. This can be exposed by
forcing monolithic shader use, which makes culling shaders immediately
available for tests to use.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26917>
2024-01-17 09:05:55 +00:00
Marek Olšák
1d3f937142 ac/nir: simplify code at the beginning of ac_nir_gs_shader_query
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26917>
2024-01-17 09:05:54 +00:00
Marek Olšák
4747bd6ba8 ac/nir: optimize out multiplications in small line culling
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26917>
2024-01-17 09:05:54 +00:00
Marek Olšák
0d74123633 ac/nir: rename clipdist_enable_mask -> clip_cull_dist_mask
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26917>
2024-01-17 09:05:54 +00:00
Marek Olšák
af4e91f6b8 ac/nir: don't write TCS outputs to memory if no_varying is set
nir_opt_varyings() will set this, making it unnecessary to look
at the TES input mask.

Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26917>
2024-01-17 09:05:54 +00:00
Marek Olšák
4880c1e12e amd: unify NIR options between RADV and radeonsi
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26917>
2024-01-17 09:05:54 +00:00
Samuel Pitoiset
e19ecf49ea radv: remove an extra new line in radv_shader.h
To make clang-format CI job happy again.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00
Samuel Pitoiset
ee96de53e4 radv: remove the pipeline key for ray tracing pipelines
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00
Samuel Pitoiset
e799907071 radv: remove the pipeline key for compute pipelines
Eveyerthing needed is already in radv_shader_stage_key.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00
Samuel Pitoiset
e017eb3b58 radv: stop hashing radv_pipeline_key for compute/rt pipelines
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00
Samuel Pitoiset
739c8eb681 radv: hash radv_shader_stage_key
This will allow us to remove the pipeline key completely for compute/rt
pipelines.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00
Samuel Pitoiset
853298edb1 radv/rt: cleanup radv_parse_rt_stage()
To avoid calling radv_pipeline_stage_init() twice.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00
Samuel Pitoiset
ffe997d71d radv: add radv_pipeline_get_shader_key()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00
Samuel Pitoiset
844004c335 radv: cleanup radv_generate_pipeline_key()
Move everything in the same loop to prepare for a new helper.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00
Samuel Pitoiset
3615bcdde3 radv: make sure to retain shaders key for imported shaders with GPL
Now that everything is moved to the per-stage key, this information
needs to be retained/loaded with GPL, otherwise some flags might be
missing.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00
Samuel Pitoiset
082c524f86 radv: pass radv_shader_stage_key to radv_pipeline_stage_init()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00
Samuel Pitoiset
08b8356111 radv: add shader_version to radv_shader_stage_key
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00
Samuel Pitoiset
20fdbc7a44 radv: add keep_statistic_info to radv_shader_stage_key
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27036>
2024-01-17 07:25:49 +00:00