Commit graph

1533 commits

Author SHA1 Message Date
Alyssa Rosenzweig
8a656b29f3 agx: handle quad reduce
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
1be0e8e348 agx: stash early_fragment_test info
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Mary Guillemard
1c8aab388c agx: speed-up dce
Follows aco changes, there shouldn't be any loop header phis that are
dead code (as nir_opt_dce ensure that)

Signed-off-by: Mary Guillemard <mary@mary.zone>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
5b03e7bab3 agx: fix bogus unit test
broken SSA.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
aad82f4265 agx: optimize txf with lod 0
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
5c898e56b2 agx: reserve scratch registers for mem<-->mem swaps
Trying to spill/fill a temp on the fly has pathologically bad performance in bad
cases, reserving a scratch reg is much more well-behaved. Fixes a fragment
shader timeout in dEQP-VK.graphicsfuzz.spv-stable-maze-flatten-copy-composite

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
f7ff0041c1 agx: document another sample_mask restriction
hit this with force early fragment tests.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
840394c169 agx: expose agx_link_libagx
for deduplication.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
5a3ac73747 agx: report uses_txf
so driver can optimize usc binding.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
54ec9512ef agx: flesh out subgroup lowering
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
659db5049c agx: lower 8-bit subgroups
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
00cfe84fd9 agx: forbid uniforms on ballots
needs more investigation..

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
c2c49b261c agx: add missing b2b16 implementation
will show up with a subgroup lowering.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
a3cb0cbfda agx: handle quad swaps
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
f4a5ac3554 agx: handle quad_broadcast
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
7b33c549b9 agx: handle non-immediate shuffles in divergent CF
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
589c69a646 agx: lower shuffle
per dougallj's notes in applegpu.py

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
fbcd9a83e4 agx: model more subgroup ops
additional shuffles, scans, reduces.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
e718a536ac agx: delete unreachable blocks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
3efa723b63 agx: lower nir_intrinsic_load_num_subgroups
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
765bf9a25b agx: switch to demote internally
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
35d6f4a394 agx: fix spilling inside sample loop
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
bdd200a202 agx: handle subgroup barriers
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
d183b76fd4 agx: fix frag sidefx with sample shading
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
6269a1474d agx: fix load_helper_invocation with sample shading
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
94f0209fb2 agx: fix phi translation corruption
we can't stomp over srcs[], where we allocated our space for sources. unclear
how this worked before but it definitely breaks once you have a phi with 7
sources.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
f21dbfe5ae agx: allow 8-bit bcsel
can be generated from our lowerings but it just works with the implicit
conversion semantics we have.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
a948244058 agx: handle cross-workgroup memory barriers
there's no prior art for this, but experimentally this seems to do the right
thing.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
c22ce3cab9 agx: fix some ms texture packing
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
ec47f325f8 agx: fix query LOD of array
need to ignore the layer

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
8df39ac49b agx: enable more lowering
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
69d7063ec0 agx: optimize and/or with booleans
Beneficial so we can fuse the comparison.

total instructions in shared programs: 2188179 -> 2185535 (-0.12%)
instructions in affected programs: 392512 -> 389868 (-0.67%)
helped: 894
HURT: 9
Instructions are helped.

total alu in shared programs: 1706063 -> 1703445 (-0.15%)
alu in affected programs: 275063 -> 272445 (-0.95%)
helped: 880
HURT: 9
Alu are helped.

total fscib in shared programs: 1702385 -> 1699743 (-0.16%)
fscib in affected programs: 276199 -> 273557 (-0.96%)
helped: 894
HURT: 9
Fscib are helped.

total ic in shared programs: 462494 -> 462490 (<.01%)
ic in affected programs: 124 -> 120 (-3.23%)
helped: 1
HURT: 0

total bytes in shared programs: 14476964 -> 14464512 (-0.09%)
bytes in affected programs: 2870824 -> 2858372 (-0.43%)
helped: 888
HURT: 155
Bytes are helped.

total regs in shared programs: 662444 -> 662461 (<.01%)
regs in affected programs: 1025 -> 1042 (1.66%)
helped: 14
HURT: 12
Inconclusive result (value mean confidence interval includes 0).

total uniforms in shared programs: 1638301 -> 1638374 (<.01%)
uniforms in affected programs: 17778 -> 17851 (0.41%)
helped: 22
HURT: 54
Uniforms are HURT.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
eb5f82d221 nir,agx: fix load_active_subgroup_index
It can't be reordered globally, since its value is control-flow dependent.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
7fb60c4c81 nir,agx: add depth=never workaround
There seems to be a hardware issue where fragment shaders with side effects get
skipped if depth testing with NEVER. Add a workaround for this case where we
discard programmatically instead.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Faith Ekstrand
6dca6809b2 asahi/clc: Use the new spirv_capabilities struct
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Iván Briano <ivan.briano@intel.com>
Acked-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28905>
2024-05-09 01:14:23 +00:00
Karol Herbst
3217838fef nir: remove global_invocation_id_zero_base
This removes the need for drivers to handle both versions. The base will
get added once in nir_lower_system_values when converting from deref to
intrinsic and will be replaced by a zero for users not supporting it.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26800>
2024-04-24 20:18:49 +00:00
Mike Blumenkrantz
7967be922d agx: set compact_arrays in compiler options
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28601>
2024-04-12 18:43:48 +00:00
Alyssa Rosenzweig
fcf1a8062b asahi: switch to VS/FS prolog/epilog system
With the exception of some variants for framebuffer fetch (to be addressed in a
follow up MR, this is big enough as it is) -- this switches us to a shader
precompile path for VS & FS. VS prologs let us implement vertex buffer fetch
with dynamic formats, FS prologs let us implement misc emulation like API sample
masking and cull distance, while FS epilogs handle blending and tilebuffer
stores. This should cut down shader recompile jank significantly in the GL
driver. It also prepares us with most of what we need for big ticket Vulkan
extensions like ESO, GPL, and EDS3.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:20 +00:00
Alyssa Rosenzweig
742a842811 asahi/clc: stop padding binaries
we're going to switch away from the dynarray soon.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig
3a2d13f59e asahi: add fast linker
to stitch together programs from separate parts

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig
66862fa160 agx: add main_size info
so we can fastlink when there are preambles

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig
0298c5d905 agx: add agx_shader_part data structure
for fastlinking.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig
b0698b796e agx: drop shader stage assertion
we'll reuse these for SW VS too which is HW compute

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig
99a4d0fcad asahi: don't allocate tib space for gaps
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig
eadf4cfe1c asahi: constify agx_build_tilebuffer_layout
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig
5d375e6143 asahi: add agx_usc_push_packed helper
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig
9974f68bb9 agx: document non-monolithic ABI
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig
a9ccd72265 agx: implement exports
translate export/load_exported instructions into moves to/from the requested
GPRs at shader part boundaries, with coalescing in RA for perf.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig
c6839cfd69 agx: generalize preloaded cache
we'll need it for more regs

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00
Alyssa Rosenzweig
61a4414bf6 agx: wire up samples_log2 sr
details determined by trial & error.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28483>
2024-03-30 00:26:19 +00:00