This will be reused by radv eventually, so let's move it all
over to common code. It might have other users eventually,
but we can worry about that later.
Reviewed-by: David Rosca <david.rosca@amd.com>
Signed-off-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35398>
This moves per-patch output VMEM stores to the end of the shader where they
execute only once. They are skipped if the whole workgroup discards
all patches.
If tcs_vertices_out == 1, per-patch output VMEM stores use the same lanes
as per-vertex output VMEM stores, which are aligned to 4 or 8 lanes to get
cached bandwidth for the stores.
Previously, per-patch outputs were stored to memory for every store_output
intrinsic in TCS.
Additionally, LDS is no longer allocated for per-patch outputs that are only
written and read by invocation 0, or they are written by all invocations
but not read, and don't have indirect indexing. This reduces LDS usage and
LDS traffic.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34780>
This unifies the duplicated LDS output patch size computation between
hs_output_lds_offset and ac_nir_compute_tess_wg_info.
"+ 4" to the output patch stride minimizes LDS bank conflicts by making
the beginning of each patch start on a different LDS bank.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34780>
Checking whether every compoment is valid in tess_level_has_effect() when
prim_mode is unknown generated too many SALU. Do this instead:
if (triangles) ...
subgroup vote for triangles
else if (quads) ..
subgroup vote for quads
else // isoline
subgroup vote for isolines
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34780>
This rewrites tess level value tracking to use the 2-bit masks, which
means LDS allocation is determined separately for outer and inner levels.
LDS is not allocated for tess levels that are only written by invocation 0
and never read or only read by invocation 0. If the number of output
patch vertices is 1, LDS is also not allocated for tess levels.
Tess level outputs for TES are always written as whole vec4 to get cached
bandwidth.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34780>
This improves write throughput for TCS outputs. It follows the same idea
as attribute stores in hw GS. The improvement is easily measurable with
a microbenchmark.
It also has the advantage that multiple output stores to the same address
don't result in multiple memory stores. Each output components gets only
one memory store at the end of the shader.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34780>
It's a stride of 1 output, which isn't 16. It's 16 * num_threads,
aligned to 256.
tcs_offchip_layout has 5 unused bits, so let's use them.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34780>
One is only used by TCS, the other is only used by TES.
Use the same field for both, call it PATCH_VERTICES_IN.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34780>
This just removes the if/endif wrapping for LLVM, and hopefully the ACO
change does the same thing.
ACO had redundant code in endif_merged_wave_info, which is removed here.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34780>
If we remove exec skips, then we can wait for an entry on all paths in the
linear cfg, but not the logical cfg.
fossil-db (gfx1201):
Totals from 0 (0.00% of 79653) affected shaders:
fossil-db (navi31):
Totals from 0 (0.00% of 79653) affected shaders:
fossil-db (navi21):
Totals from 1586 (1.99% of 79653) affected shaders:
Instrs: 5118897 -> 5113206 (-0.11%); split: -0.11%, +0.00%
CodeSize: 28365852 -> 28343696 (-0.08%); split: -0.08%, +0.00%
Latency: 47820341 -> 47799532 (-0.04%); split: -0.09%, +0.05%
InvThroughput: 9904391 -> 9908653 (+0.04%); split: -0.02%, +0.06%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34978>
Loosely based on RadeonSI.
This is supposed to be faster because parsing the packet header seems
to be the main bottleneck on GFX12.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35282>
Scalar block layout doesn't allow anything that our memory load/store vectorizer
couldn't create on its own. So I assume whatever reason there was to only
expose this feature on GFX7+ was incorrect or ended up being fixed.
Passes vkcts in CI on tahiti.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35279>
Looks like it's actually also affected by the memory explosion caused
by zerovram alloc by default in AMDGPU. Though it's very random,
sometimes the job will finish in 40 minutes, sometimes it needs more
than 1h15m. Let's bump the timeout because it's a post-merge job.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35157>
