This is a cleanup.
Old gs LDS layout: [es outputs][gs outputs][scratch]
Old nogs LDS layout: [xfb/cull][scratch]
New gs LDS layout: [es outputs][scratch|gs outputs]
New nogs LDS layout: [scratch|xfb/cull]
The LDS scratch is moved to the beginning of the preceding buffer in LDS,
while the addresses in that LDS buffer are offset by the scratch size.
It effectively merges the LDS scratch with the preceding buffer in LDS.
Thanks to that, we no longer need the ngg_scratch ABI and the offset
in a user SGPR.
The lowering passes now return the LDS scratch size, which is used
by the drivers to determine the final LDS size.
The ngg_lds_layout SGPR is now unused without GS in RADV.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35352>
This can't break until we have read all lanes, otherwise it might read
from an inactive lane.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35489>
These return floating point types for floating point ops.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Backport-to: 25.1
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35489>
The referenced commit was a step in the right direction, but not
complete.
ac_build_image_opcode returns a vec<4> or a struct<vec<4>, int>
so we can simplify visit_tex. We just need to map these 4/5 values
to the expected layout from NIR.
eg: depth + TFE would produces "<d, x, x, x>, t" so it has to be
transformed into <d, t>.
nir_texop_fragment_mask_fetch_amd + sparse doesn't exist, so it's
another opportunity for simplification.
This is required to get KHR-GL46.sparse_texture2_tests.SparseTexture2Lookup_texture_2d_depth_component16
working properly.
The same test fails with ACO so it probably needs a change in the
same area.
Fixes: c0ef2aa7f8 ("DEPENDENCY: ac/llvm: fix sparse code handling")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35206>
For coherent/volatile access, this would be too high for vector access.
Even when we didn't set the alignment, LLVM seemed to assume too high of
an alignment for 8/16-bit vector access.
Fixes generated_tests/cl/vload/vload-char-constant.cl
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Tested-by: Michel Dänzer <mdaenzer@redhat.com>
Backport-to: 25.0
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34903>
This assumes that the start of the load is 32-bit aligned.
For example, a vec3 16-bit store with align_offset=2 should split off the
first component, not the last.
This probably also fixed splitting with 8-bit stores.
Fixes arb_copy_buffer-overlap
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Tested-by: Michel Dänzer <mdaenzer@redhat.com>
Backport-to: 25.0
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34903>
With the current code in clpeak LLVM ended up generating v_mad_u64_u32
instructions, with this we get nice v_mad_u32_s24 ones instead and an 4x
performance increase in the int24 benchmark.
Suggested-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34630>
I didn't find dedicated 24bit intrinsics, but I also haven't looked that hard.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33871>
Without this, WQM is only used for the lds_param_load like this:
s_wqm_b64 exec, exec
lds_param_load v5, attr0.x wait_vdst:15
s_mov_b64 exec, s[0:1]
v_mov_b32_dpp v5, v5 quad_perm:[0,0,0,0] row_mask:0xf bank_mask:0xf
With this change we get:
s_wqm_b64 exec, exec
lds_param_load v5, attr0.x wait_vdst:15
s_mov_b64 exec, s[0:1]
...
s_wqm_b64 exec, exec
v_mov_b32_dpp v5, v5 quad_perm:[0,0,0,0] row_mask:0xf bank_mask:0xf
s_mov_b64 exec, s[0:1]
This fixes KHR-GL46.shaders.uniform_block.random.nested_structs_instance_arrays.0
and other similar tests with LLVM.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32959>
And to ac_build_dpp because it's used from quad_swizzle.
No functional changes but will be used in the next commit.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32959>
Not needed with ACO. It was used for big shaders on old APUs that took
forever to compile.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33046>
This should be faster than 32_ABGR.
Also, stencil exports are changed from UINT16_ABGR to 32_GR,
which should have no effect on performance.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33046>
alpha-to-coverage must be applied before alpha-to-one. The only way to do
that is to export alpha for alpha-to-coverage via mrtz, and export 1 via
mrt0.a.
ACO and monolithic shader support is already in place thanks to RADV,
so we only need to change the LLVM PS epilog and the shader key.
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32713>
