Commit graph

189545 commits

Author SHA1 Message Date
Alyssa Rosenzweig
6373de5195 asahi: move agx_link_varyings_vs_fs
nothing gl here except for point sprites

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:26 +00:00
Alyssa Rosenzweig
fadb990463 asahi: cleanup fs epilog link info
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:26 +00:00
Alyssa Rosenzweig
71692a5d0b asahi: resize key
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:26 +00:00
Alyssa Rosenzweig
a38f7c9151 asahi: rm unused #include
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:26 +00:00
Alyssa Rosenzweig
ad5f46ff8c asahi: implement PIPE_CAP_QUERY_MEMORY_INFO
Steam uses it.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
74963d5364 asahi: unwrap pointless null check
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
8494288451 asahi: assert bo size > 0
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
4e05f549ea asahi: agx_translate_sample_count
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
feabbddc2a asahi: calculate validity when unpacking
for smarter printing.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
ac726ae5a9 asahi: mark eMRT loads as in-bounds
for layer clamping purposes.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
ac114030a7 ail: constify everything
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
efa9f242a8 agx: fix UB in cursor comparison
padding here is implementation-defined, do the cleaner thing. fixes invalid IR
generated with gcc but not clang.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reported-by: Janne Grunau <janne-fdr@jannau.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
bcffc84306 agx: optimize elect()
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
bc26e11c2a agx: lower more quad ops
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
7aa17a122f agx: implement quad_ballot
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
8a656b29f3 agx: handle quad reduce
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
1be0e8e348 agx: stash early_fragment_test info
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Mary Guillemard
1c8aab388c agx: speed-up dce
Follows aco changes, there shouldn't be any loop header phis that are
dead code (as nir_opt_dce ensure that)

Signed-off-by: Mary Guillemard <mary@mary.zone>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
5b03e7bab3 agx: fix bogus unit test
broken SSA.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
aad82f4265 agx: optimize txf with lod 0
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
5c898e56b2 agx: reserve scratch registers for mem<-->mem swaps
Trying to spill/fill a temp on the fly has pathologically bad performance in bad
cases, reserving a scratch reg is much more well-behaved. Fixes a fragment
shader timeout in dEQP-VK.graphicsfuzz.spv-stable-maze-flatten-copy-composite

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
f7ff0041c1 agx: document another sample_mask restriction
hit this with force early fragment tests.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
840394c169 agx: expose agx_link_libagx
for deduplication.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
5a3ac73747 agx: report uses_txf
so driver can optimize usc binding.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
54ec9512ef agx: flesh out subgroup lowering
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
659db5049c agx: lower 8-bit subgroups
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
00cfe84fd9 agx: forbid uniforms on ballots
needs more investigation..

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
c2c49b261c agx: add missing b2b16 implementation
will show up with a subgroup lowering.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
a3cb0cbfda agx: handle quad swaps
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
f4a5ac3554 agx: handle quad_broadcast
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
7b33c549b9 agx: handle non-immediate shuffles in divergent CF
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
589c69a646 agx: lower shuffle
per dougallj's notes in applegpu.py

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
fbcd9a83e4 agx: model more subgroup ops
additional shuffles, scans, reduces.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
e718a536ac agx: delete unreachable blocks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
3efa723b63 agx: lower nir_intrinsic_load_num_subgroups
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
765bf9a25b agx: switch to demote internally
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
35d6f4a394 agx: fix spilling inside sample loop
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
bdd200a202 agx: handle subgroup barriers
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
d183b76fd4 agx: fix frag sidefx with sample shading
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
6269a1474d agx: fix load_helper_invocation with sample shading
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
94f0209fb2 agx: fix phi translation corruption
we can't stomp over srcs[], where we allocated our space for sources. unclear
how this worked before but it definitely breaks once you have a phi with 7
sources.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
f21dbfe5ae agx: allow 8-bit bcsel
can be generated from our lowerings but it just works with the implicit
conversion semantics we have.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
a948244058 agx: handle cross-workgroup memory barriers
there's no prior art for this, but experimentally this seems to do the right
thing.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
c22ce3cab9 agx: fix some ms texture packing
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
ec47f325f8 agx: fix query LOD of array
need to ignore the layer

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
8df39ac49b agx: enable more lowering
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
69d7063ec0 agx: optimize and/or with booleans
Beneficial so we can fuse the comparison.

total instructions in shared programs: 2188179 -> 2185535 (-0.12%)
instructions in affected programs: 392512 -> 389868 (-0.67%)
helped: 894
HURT: 9
Instructions are helped.

total alu in shared programs: 1706063 -> 1703445 (-0.15%)
alu in affected programs: 275063 -> 272445 (-0.95%)
helped: 880
HURT: 9
Alu are helped.

total fscib in shared programs: 1702385 -> 1699743 (-0.16%)
fscib in affected programs: 276199 -> 273557 (-0.96%)
helped: 894
HURT: 9
Fscib are helped.

total ic in shared programs: 462494 -> 462490 (<.01%)
ic in affected programs: 124 -> 120 (-3.23%)
helped: 1
HURT: 0

total bytes in shared programs: 14476964 -> 14464512 (-0.09%)
bytes in affected programs: 2870824 -> 2858372 (-0.43%)
helped: 888
HURT: 155
Bytes are helped.

total regs in shared programs: 662444 -> 662461 (<.01%)
regs in affected programs: 1025 -> 1042 (1.66%)
helped: 14
HURT: 12
Inconclusive result (value mean confidence interval includes 0).

total uniforms in shared programs: 1638301 -> 1638374 (<.01%)
uniforms in affected programs: 17778 -> 17851 (0.41%)
helped: 22
HURT: 54
Uniforms are HURT.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
c43413f729 compiler: add ACCESS_IN_BOUNDS_AGX
useful for internal shaders on agx.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
eb5f82d221 nir,agx: fix load_active_subgroup_index
It can't be reordered globally, since its value is control-flow dependent.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00
Alyssa Rosenzweig
7fb60c4c81 nir,agx: add depth=never workaround
There seems to be a hardware issue where fragment shaders with side effects get
skipped if depth testing with NEVER. Add a workaround for this case where we
discard programmatically instead.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:25 +00:00