Commit graph

16844 commits

Author SHA1 Message Date
Marek Olšák
de3dd2afdf Revert "ac/nir: clamp vertex color outputs in the right place"
This reverts commit b3fc49686e.

It was a rebase failure.

Fixes: b3fc49686e

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33482>
(cherry picked from commit 177c9b173e)
2025-03-15 09:49:01 +01:00
Rhys Perry
6999073da9 aco: insert dependency waits in certain situations
This seems to fix some artifacts, but we're not sure why, so it might not
be a correct or optimal solution.

fossil-db (navi31):
Totals from 28424 (35.81% of 79377) affected shaders:
Instrs: 30112910 -> 30348977 (+0.78%); split: -0.00%, +0.78%
CodeSize: 159542980 -> 160485336 (+0.59%); split: -0.00%, +0.59%
Latency: 221438396 -> 221500856 (+0.03%); split: -0.00%, +0.03%
InvThroughput: 38154231 -> 38159984 (+0.02%); split: -0.00%, +0.02%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Backport-to: 25.0
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33853>
(cherry picked from commit 0ec174afd5)
2025-03-15 09:49:01 +01:00
Samuel Pitoiset
5200d13a0f radv: fix re-emitting fragment output state when resetting gfx pipeline state
When switching from pipeline to shader objects.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33840>
(cherry picked from commit 7f6e28db26)
2025-03-04 20:24:03 +01:00
Daniel Schürmann
553ab18656 aco/assembler: Fix short jumps over chained branches
If we insert

   <code>
   s_branch 1
   s_branch Target

at the end of some block, and later hide an additional chained branch
after the existing one, then we have to update the 's_branch 1' to
also jump over the newly added branch.

Fixes: cab5639a09 ('aco/assembler: chain branches instead of emitting long jumps')
Closes: #12673
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33762>
(cherry picked from commit 6659db285a)
2025-02-28 22:17:35 +01:00
Hans-Kristian Arntzen
1b6da4ed52 radv: Always set 0 dispatch offset for indirect CS.
Fixes severe glitching in Avowed.

Signed-off-by: Hans-Kristian Arntzen <post@arntzen-software.no>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33732>
(cherry picked from commit 13a3f9a972)
2025-02-28 22:17:34 +01:00
Samuel Pitoiset
20bb982788 radv: fix missing SQTT barriers for fbfetch color/depth decompressions
SQTT layout transitions need to be inside SQTT barrier. Otherwise, this
throws an assertion in RADV and might also crash when the capture is
opened with RGP.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12664
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33719>
(cherry picked from commit 67c150bf9e)
2025-02-28 22:17:34 +01:00
Natalie Vock
ea47f98811 radv/rt: Don't allocate the traversal shader in a capture/replay range
We never write the traversal shader address out to shader group handles,
so this is not necessary. On the flipside, it can cause conflicts if the
traversal shader is allocated in a range occupied by a replayed shader.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33711>
(cherry picked from commit 14b902c825)
2025-02-27 18:37:32 +01:00
Georg Lehmann
cb09b3f624 aco/insert_exec: fix continue_or_break on gfx6-7
s_cmp_lg_u64 is gfx8+

Fixes: 115ff5f95b ("aco/insert_exec_mask: don't restore exec in continue_or_break blocks")

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33715>
(cherry picked from commit c249556bf4)
2025-02-27 18:37:31 +01:00
Rhys Perry
36e1923284 ac/nir: fix tess factor optimization when workgroup barriers are reduced
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: b49eab68a8 ("ac/nir: use s_sendmsg(HS_TESSFACTOR) to optimize writing tess factors for gfx11")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12632
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33645>
(cherry picked from commit 2a3dce1b59)
2025-02-27 18:37:30 +01:00
Daniel Schürmann
f9c3499918 aco/ssa_elimination: insert parallelcopies for p_phi immediately before branch
Totals from 2499 (3.15% of 79377) affected shaders: (Navi31)
Instrs: 6011729 -> 6011761 (+0.00%); split: -0.00%, +0.00%
CodeSize: 31573216 -> 31574236 (+0.00%); split: -0.00%, +0.00%
Latency: 83364734 -> 83365781 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 13545643 -> 13545783 (+0.00%); split: -0.00%, +0.00%

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33527>
(cherry picked from commit 302678df91)
2025-02-27 18:37:30 +01:00
Daniel Schürmann
4118fef567 aco/insert_exec_mask: don't restore exec in continue_or_break blocks
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33527>
(cherry picked from commit 115ff5f95b)
2025-02-27 18:37:29 +01:00
Daniel Schürmann
1bb39be75e aco/insert_exec_mask: Don't immediately set exec to zero in break/continue blocks
Instead, only indicate that exec should be zero and do
so in the successive helper block. This allows to insert
the parallelcopies from logical phis directly before the
branch in break and continue blocks.

Totals from 56 (0.07% of 79377) affected shaders: (Navi31)
Latency: 2472367 -> 2472422 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 253053 -> 253055 (+0.00%); split: -0.00%, +0.00%

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33527>
(cherry picked from commit 7f7c1d463a)
2025-02-27 18:37:28 +01:00
Daniel Schürmann
c4d64f9f83 aco/scheduler: always respect min_waves on GFX10+
It could theoretically happen that for large workgroups,
the scheduler used more registers than allowed.

No fossil changes.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33644>
(cherry picked from commit 676b39d31f)
2025-02-21 17:07:28 +01:00
Samuel Pitoiset
5b8b81618e radv/video: fix adding the query pool BO to the cmdbuf list
Video queries work differently but the BO still need to be added to the
cmdbuf list.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33620>
(cherry picked from commit 5ba10cc57f)
2025-02-21 17:04:51 +01:00
Samuel Pitoiset
ef610a0d25 radv: fix adding the BO for unaligned SDMA copies to the cmdbuf list
It shouldn't be only added at creation time.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33600>
(cherry picked from commit efa23ef664)
2025-02-19 14:18:30 +01:00
Samuel Pitoiset
a026515817 radv: add initial DCC support on GFX12
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33255>
(cherry picked from commit 9af11bf306)
2025-02-19 13:16:03 +01:00
Samuel Pitoiset
ceaf6b2231 ac/gpu_info: add gfx12_supports_dcc_write_compress_disable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33255>
(cherry picked from commit 827cef7f7f)
2025-02-19 13:16:03 +01:00
Samuel Pitoiset
9b60c38646 ac,radv,radeonsi: add new GFX12_DCC_WRITE_COMPRESS_DISABLE tiling flag
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33301>
(cherry picked from commit 9993f3dd6a)
2025-02-19 13:16:03 +01:00
David Rosca
19e2eed688 radv/video: Move IB header from begin/end to encode_video
For decode this is also done in decode_video.

This breaks if app doesn't call vkCmdEncodeVideoKHR before end, eg:

  vkCmdBeginVideoCodingKHR
  vkCmdControlVideoCodingKHR
  vkCmdEndVideoCodingKHR

Cc: mesa-stable
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33582>
(cherry picked from commit ebd8893710)
2025-02-18 22:46:05 +01:00
David Rosca
70bb670e9f radv/video: Fix setting balanced preset for HEVC encode with SAO enabled
FW disables SAO in speed preset, so we need to switch to balanced.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12615
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33582>
(cherry picked from commit 77ff18aa3b)
2025-02-18 22:46:03 +01:00
Samuel Pitoiset
27b7056835 radv: fix adding the VRS image BO to the cmdbuf list on GFX11
This might cause random faults.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33584>
(cherry picked from commit e0070bc68b)
2025-02-18 22:46:02 +01:00
Samuel Pitoiset
7e54da043a radv/meta: disable conditional rendering for fill/update buffer operations
These commands shouldn't be affected by conditional rendering, similar
to the copy buffer operation.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33511>
(cherry picked from commit 5684c1687c)
2025-02-15 00:11:30 +01:00
David Rosca
19f1546fb3 ac/vcn_dec: Fix AV1 film grain on VCN5
Cc: mesa-stable
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33376>
(cherry picked from commit 62b0f84981)
2025-02-10 11:36:50 +01:00
Samuel Pitoiset
1cf778e011 radv: fix fetching draw vertex data from counter buffers with transform feedback
counterOffset was just ignored and nobody noticed (missing VKCTS
coverage).

VGT_STRMOUT_DRAW_OPAQUE_OFFSET will do the computation in hw for us.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33407>
(cherry picked from commit 8625decbcc)
2025-02-10 11:36:48 +01:00
Samuel Pitoiset
765cdedcd0 radv: fix adding the BO to cmdbuf list when starting conditional rendering
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33403>
(cherry picked from commit 9b827556f5)
2025-02-06 10:15:10 +01:00
Mike Blumenkrantz
78577b19bc radv: fix error reporting for VkExternalMemoryTypeFlagBitsKHR
wrong type name is confusing

cc: mesa-stable

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33323>
(cherry picked from commit ca8a740e3b)
2025-02-06 10:15:03 +01:00
Samuel Pitoiset
c00d4230ba radv: fix caching on-demand meta shaders
This switches to disk_cache instead of our own mechanism which only
stored meta shaders when the logical was destroyed.

Meta shaders are still stored separately from the application shaders
because they are common to all applications on a given GPU/Mesa version.
The default cache is 32MiB which should be large enough.

This fixes massive stuttering in FF7 Rebirth but all apps are
technically affected.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33405>
2025-02-06 07:33:37 +00:00
Samuel Pitoiset
03c3250e04 radv/meta: stop using string keys also for DGC and query objects
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33405>
2025-02-06 07:33:37 +00:00
Samuel Pitoiset
5443c23983 radv/meta: add missing pipeline lookups
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33405>
2025-02-06 07:33:36 +00:00
Konstantin Seurer
662bcc8717 radv/meta: Stop using strings for meta keys
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33405>
2025-02-06 07:33:36 +00:00
Valentine Burley
82b697ed69 amd/ci: Revert to 6.6 kernel on Raven
There's been a high number of GPU resets on Raven that amdgpu couldn't
recover from, leading to jobs timing out.

Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33317>
(cherry picked from commit 5c44d70684)
2025-02-04 21:10:10 +01:00
Timur Kristóf
150123349a ac/nir/ngg: Use SALU to calculate which threads store to attribute ring in GS.
This trades 1 VALU (v_cmpx) instruction in GS to 2 SALU,
and removes a VALU->SALU dependency for the branch that stores
attributes.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
a9e9ec30a5 radv, radeonsi: Disable early prim export on GFX11+.
We suspect that it has no perf benefits on GFX11+.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
5560763e99 ac/nir/ngg: Move GS lowering to separate file.
Both the VS/TES and GS lowering passes have grown a lot over time,
and therefore the C file has become unwieldy. Mitigate that by
moving the GS lowering out to a separate file.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
ab8ec78c93 ac/nir/ngg: Don't call has_input_primitive in GS lowering.
The entire GS lowering will be moved to another file, which
won't have this function.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
910416e5ba ac/nir/ngg: Remove now unused export_pos0_wait_attr_ring.
The issue is worked around now without needing to handle just
the pos0 separately.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
28fe7e2a0e ac/nir/ngg: Rework GS output code for better attribute ring handling.
Create separate branches for output processing and exports.

Normally, emit attribute ring stores at the end of the shader,
but with the attribute ring wait bug, insert them between the
primitive and position export branches.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
213759d2b6 ac/nir/ngg: Carve out ngg_gs_process_out_vertex.
This is	the part of the	code that processes things related to
a single vertex, mainly loading the outputs from LDS and
performing some adjustments on them.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
4f8ee9b045 ac/nir/ngg: Carve out ngg_gs_process_out_primitive.
This is the part of the code that processes things related to
a single primitive, mainly the primitive export argument.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
04b4c5d785 ac/nir/ngg: Rework attribute ring wait workaround in VS/TES.
Normally, emit attribute ring stores at the end of the shader.
When the attribute ring wait bug is present, insert them between
the primitive and position exports.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
dec5c957c3 ac/nir/ngg: Use has_attr_ring and has_attr_ring_wait_bug in mesh shader lowering too.
Instead of hardcoding according to GFX level.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
f7305f776e ac/nir/ngg: Pass radeon_info to mesh shader lowering.
Same idea as the VS/TES and GS lowering:
Make shader compilation decisions based on the features of the
current GPU instead of ad-hoc deciding according to GFX level.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
b8204c8df9 ac/nir/ngg: Remove gfx_level and family from NGG lowering options.
They can be read from radeon_info.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
dc6ae1855a ac/nir/ngg: Use gfx_level from radeon_info.
So we don't have to pass it separately.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:46 +00:00
Timur Kristóf
e1be943f10 ac/nir/ngg: Add and use a has_ngg_passthru_no_msg field to ac_gpu_info.
Instead of using the chip family field.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:45 +00:00
Timur Kristóf
a40000b85b ac/nir/ngg: Add and use a has_ngg_fully_culled_bug field to ac_gpu_info.
Better than applying the workaround ad-hoc based on GFX level.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:45 +00:00
Timur Kristóf
cad0d26dbf ac/nir/ngg: Add and use a has_attr_ring field to ac_gpu_info.
While theoretically all GFX11+ GPUs have an attribute ring, it is
nicer to have this property instead of deciding ad-hoc based on
the GFX level.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:45 +00:00
Timur Kristóf
b163ce51b1 ac/nir/ngg: Add and use a has_attr_ring_wait_bug field to ac_gpu_info.
And apply the attribute ring wait workaround based on the new field.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:45 +00:00
Timur Kristóf
e76361d626 ac/nir/ngg: Add radeon_info to NGG lowering options.
The intention is to have all the HW features affecting
shader compilation in one place, instead of ad-hoc decisions
in the code based on the GFX level and chip class.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:45 +00:00
Timur Kristóf
28c501f22c ac/nir/ngg: Slightly refactor emitting vertex parameters.
No functional changes, just some code cleanup.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
2025-01-30 15:26:45 +00:00