Commit graph

13981 commits

Author SHA1 Message Date
Friedrich Vock
d4c094dfb4 radv/rt: Optimize update shader VGPR usage
Brings VGPR allocation down from 72 (absolutely insane) to 32.

We can now reach the theoretical maximum occupancy of 16 waves per SIMD.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27472>
2024-02-05 21:14:45 +01:00
Georg Lehmann
2ad3b83d23 radv: report rotate subgroup feature bits
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27444>
2024-02-05 11:20:28 +00:00
Rhys Perry
48461c0d9e aco: enable VOPD scheduler
fossil-db (navi31, wave32):
Totals from 75237 (94.95% of 79242) affected shaders:
Instrs: 40383337 -> 36522907 (-9.56%); split: -9.56%, +0.00%
CodeSize: 209282312 -> 209883000 (+0.29%); split: -0.18%, +0.46%
Latency: 256898944 -> 257834204 (+0.36%); split: -0.23%, +0.60%
InvThroughput: 59105835 -> 52725948 (-10.79%); split: -10.90%, +0.11%
VClause: 759487 -> 758151 (-0.18%); split: -0.18%, +0.01%
SClause: 1264895 -> 1263230 (-0.13%); split: -0.14%, +0.01%
VALU: 22886416 -> 18837085 (-17.69%)
VOPD: 0 -> 4049331 (+inf%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367>
2024-02-05 10:51:15 +00:00
Rhys Perry
75a76ec3fd aco: implement VOPD scheduler
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367>
2024-02-05 10:51:15 +00:00
Rhys Perry
1fb79b4aa2 aco: refactor schedule_ilp main loop
Besides switching to "ctx.active_mask" as the condition, this is basically
the same.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367>
2024-02-05 10:51:14 +00:00
Rhys Perry
c66d42b9ed aco: add VOPD statistic
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367>
2024-02-05 10:51:14 +00:00
Rhys Perry
6547e17e60 aco: add VOPD format
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367>
2024-02-05 10:51:14 +00:00
Dave Airlie
47c725b53e radv: don't submit 0 length on UVD either.
The kernel checks for UVD msgs and if there aren't any gets upset,
so don't submit 0 length on UVD rings either to avoid that.

Cc: mesa-stable
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27186>
2024-02-05 08:14:26 +00:00
Dave Airlie
df9bc11589 radv/uvd: uvd kernel checks for full dpb allocation.
The CTS image allocation sometimes doesn't try to allocate a complete
DPB, but the amdgpu kernel module checks for this, so always make
the DPB max sized on uvd instances.

Fixes part of video decode on Fiji/Polaris

Cc: mesa-stable
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27186>
2024-02-05 08:14:26 +00:00
Dave Airlie
bba36df84d radv: init decoder ip block earlier.
This makes the queue decisions later correct.

Cc: mesa-stable
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27186>
2024-02-05 08:14:26 +00:00
Dave Airlie
6065671a7f radv: fix correct padding on uvd
Fixes: 8a29291dbe ("radv/video: add h264 support for uvd")

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27186>
2024-02-05 08:14:26 +00:00
Samuel Pitoiset
c6286e39ec radv/sqtt: fix describing queue submits for RGP
The submit_sub_index field is used by RGP to determine the number of
submits. Previously, it was incorrectly reporting the same number of
submits than command buffers.

Fixes: 88cbe32048 ("radv: add support for RGP queue events")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27439>
2024-02-05 07:32:48 +00:00
Konstantin Seurer
b10ee44308 radv/rra: Implement ahit/isec counters
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25548>
2024-02-03 12:59:47 +01:00
Konstantin Seurer
767f628079 radv/rra: Dump basic ray history tokens
This only dumps the begin tokens. Tokens are written to a buffer
containing a 12 byte header at the beginning.

We use an intermediate format for the ray history tokens because the RRA
format is very inefficient.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25548>
2024-02-03 12:59:47 +01:00
Konstantin Seurer
26939f016d radv/rra: Refactor error handling
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25548>
2024-02-03 11:29:58 +01:00
Konstantin Seurer
46dddb57f9 radv/rra: Remove useless variable
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25548>
2024-02-03 11:29:58 +01:00
Konstantin Seurer
dc813288c3 radv/rra: Use memcpy for chunk descriptions
The identifier is not null terminated. "HistoryTokensRaw" does not fit
otherwise.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25548>
2024-02-03 11:29:58 +01:00
Konstantin Seurer
71c363acc3 radv/rra: Rename rra_chunk_type to rra_chunk_version
Found by inspecting RRA source.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25548>
2024-02-03 11:29:58 +01:00
Daniel Schürmann
4fa27845e5 aco/insert_exec_mask: Reduce latency when switching to WQM.
Change pattern:
s_mov_b64 s[0:1], exec         s_mov_b64 s[0:1], exec
s_wqm_b64 exec, s[0:1]   ->    s_wqm_b64 exec, exec

Totals from 16667 (21.03% of 79242) affected shaders: (GFX11)

Instrs: 11317502 -> 11307484 (-0.09%); split: -0.09%, +0.00%
CodeSize: 60194272 -> 60155088 (-0.07%); split: -0.07%, +0.00%
Latency: 94345873 -> 94338374 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 13568872 -> 13568683 (-0.00%); split: -0.00%, +0.00%
Copies: 808334 -> 808332 (-0.00%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27112>
2024-02-02 18:55:15 +00:00
Daniel Schürmann
e89977ff71 aco: always terminate quads if they have been demoted entirely
Previously, quads got only terminated in top-level control flow.
This patch makes the behavior consistent.

Totals from 7811 (9.86% of 79242) affected shaders: (GFX11)

Instrs: 7859667 -> 7850757 (-0.11%); split: -0.18%, +0.07%
CodeSize: 41642280 -> 41611836 (-0.07%); split: -0.13%, +0.06%
Latency: 73692815 -> 73707588 (+0.02%); split: -0.02%, +0.04%
InvThroughput: 10672160 -> 10672323 (+0.00%); split: -0.01%, +0.01%
VClause: 137478 -> 137469 (-0.01%); split: -0.02%, +0.02%
SClause: 314905 -> 314924 (+0.01%); split: -0.19%, +0.20%
Copies: 587014 -> 576039 (-1.87%); split: -2.10%, +0.23%
Branches: 213101 -> 213123 (+0.01%); split: -0.01%, +0.02%
PreSGPRs: 313588 -> 313355 (-0.07%); split: -0.09%, +0.01%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27112>
2024-02-02 18:55:15 +00:00
Daniel Schürmann
a42b83e3fb aco/insert_exec_mask: tiny refactor
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27112>
2024-02-02 18:55:15 +00:00
Samuel Pitoiset
9698d5f0fd radv: add a workaround for mipmaps and minLOD on GFX6-8
This is spurious and it looks like we should be able to uses non-zero
base level everytime on GFX6-8 but it doesn't always work.

This fixes the remaining CTS failures on GFX6-8.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26290>
2024-02-02 18:06:25 +00:00
Samuel Pitoiset
f977501a7c radv: do not allow to enable VK_EXT_shader_object with LLVM
This isn't expected to work.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27415>
2024-02-02 17:14:56 +00:00
Konstantin Seurer
c925b6019d radv/rt: Lower ray payloads like hit attribs
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27051>
2024-02-02 16:36:15 +00:00
Konstantin Seurer
bb14ee53a5 radv/sqtt: Handle ray tracing pipelines with no traversal shader
Fixes: 0f87d40 ("radv/rt: Skip compiling a traversal shader")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27383>
2024-02-02 14:14:16 +01:00
Samuel Pitoiset
0aa9afa8e1 radv: add support for emitting VS+TCS compiled separately on GFX9+
With a VS prolog, we end up with 3 long jumps (VS prolog->VS->TCS->TCS
epilog), super annoying.

The shaders config must also be combined between VS and TCS.

This is for VK_EXT_shader_object.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27336>
2024-02-02 10:16:59 +01:00
Samuel Pitoiset
397a08b407 radv: always emit PGM_RSRC1_HS when emitting the TCS epilog state
This will simplify upcoming changes and it doesn't matter much because
this is for ESO only.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27336>
2024-02-02 10:16:59 +01:00
Samuel Pitoiset
542b9aaf18 radv: force TCS stage for VS as LS compiled separately on GFX9+
When VS as LS is compiled separately on GFX9+, the stage/previous_stage
must be VERTEX/TESS_CTRL.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27336>
2024-02-02 10:16:59 +01:00
Samuel Pitoiset
3d5d163693 radv: always mark drawid/base_instance used with ESO
The user SGPR is always declared for merged shaders compiled separately
because the args must match.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27336>
2024-02-02 10:16:59 +01:00
Samuel Pitoiset
3c23ae8547 radv: rework shader arguments for separate compilation of VS+TCS on GFX9+
When VS or TCS are compiled separately on GFX9+, the shader input args
must match. This is implemented using a complete separate path, it's
duplicated but it seems cleaner than adding a ton of checks here and
there.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27336>
2024-02-02 10:16:59 +01:00
Samuel Pitoiset
1e55d91c82 radv: only merge shader info stages if both stages exist on GFX9+
With shader objects, both stages might not exist and if the src stage
doesn't, this will copy garbage data because it's unitialized.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27336>
2024-02-02 10:16:59 +01:00
Samuel Pitoiset
0018faf384 radv: check active NIR stages before trying to merge shaders on GFX9+
For shader object.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27336>
2024-02-02 10:16:59 +01:00
Samuel Pitoiset
1fe8770bbe radv: constify radv_device in radv_emit_shader_pointer()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27336>
2024-02-02 10:16:59 +01:00
Samuel Pitoiset
3b2452da3c radv: set the default workgroup size for VS as LS
This will be optimized during shader info linking if TCS is present.
The main motivation for this change is ESO because the next stage
might not exist.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27396>
2024-02-02 08:53:20 +00:00
Samuel Pitoiset
2a58bbbed8 radv: determine the workgroup size for TCS earlier
This can be done before linking shader info pass.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27396>
2024-02-02 08:53:20 +00:00
Samuel Pitoiset
c6ca7fcc25 radv: remove radv_graphics_state_key::dynamic_patch_control_points
When the state isn't dynamic, the patch control points value must
greater than 0. Having a separate field isn't necessary.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27396>
2024-02-02 08:53:20 +00:00
Yiwei Zhang
5df083eff7 radv: refactor to use DETECT_OS_ANDROID instead of ANDROID
Include a tiny refactor in radv_physical_device_get_supported_extensions
to avoid a false formatting issue.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27374>
2024-02-01 19:29:49 +00:00
Friedrich Vock
f66055a6a6 radv/rt: Write inactive node data in ALWAYS_ACTIVE workaround
Fixes: a9831caa ("radv/rt: Add workaround to make leaves always active")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27340>
2024-02-01 05:42:59 +00:00
Friedrich Vock
e8b0e5cac9 radv: Use secure_getenv for RADV_THREAD_TRACE_TRIGGER
Reviewed-by: Eric Engestrom <eric@igalia.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27381>
2024-01-31 17:28:12 +00:00
Friedrich Vock
c01a07f2e4 radv: Use secure_getenv in radv_builtin_cache_path
Reviewed-by: Eric Engestrom <eric@igalia.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27381>
2024-01-31 17:28:12 +00:00
Eric Engestrom
0fa4ea98ca ci: always skip dEQP-VK.info.device_extensions
Exclude this test which might fail when a new extension is implemented.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27389>
2024-01-31 16:48:16 +00:00
Daniel Schürmann
c309d20172 aco/insert_exec_mask: Fix unconditional demote at top-level control flow.
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27362>
2024-01-31 13:50:46 +00:00
Konstantin Seurer
13ca3d0eeb radv/sqtt: Set SeparateCompiled for monolithic RT pipelines
RGP can not handle how we implement "Unified" ray tracing pipelines and
only shows the prolog.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26697>
2024-01-31 10:24:24 +01:00
Konstantin Seurer
6d2926c54e radv: Skip unused acceleration structure build paths
If the whole batch doesn't use a path, we can skip binding the pipeline
and writing an user event.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26697>
2024-01-31 10:24:20 +01:00
Konstantin Seurer
93b1eec09b radv: Emit user events during acceleration structure builds
This way, we can see exactly which dispatches belong to acceleration
structure builds and which dispatches were caused by the app. We loose
the red coloring in RGP for this, but we get way more information.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26697>
2024-01-31 10:24:15 +01:00
Konstantin Seurer
22b719b445 radv: Make radv_write_user_event_marker non-static
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26697>
2024-01-31 10:24:11 +01:00
Konstantin Seurer
f43099ecf5 radv: Reduce the amount of radv_device_to_handle calls
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26697>
2024-01-31 10:24:06 +01:00
Konstantin Seurer
41e78330e7 radv/radix_sort: clang-format
The formatting was changed and those files were missed.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26697>
2024-01-31 10:23:51 +01:00
Samuel Pitoiset
31e29f7304 radv: enable deviceFaultVendorBinary if RADV_DEBUG=hang is set
Ideally, if deviceFaultVendorBinary is enabled by the app,
RADV_DEBUG=hang should be enabled behind the scene, but this isn't
possible right now because CTS always enables the feature.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27293>
2024-01-31 08:45:08 +01:00
Samuel Pitoiset
a778d5fd0d radv: export GPU hang reports through VK_EXT_device_fault
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27293>
2024-01-31 08:45:08 +01:00