Alyssa Rosenzweig
27d39bf544
agx: reassociate bcsel with ior/iand
...
fairly dramatic improvement to citra shaders, citra seems to be a % faster with
this.
total instructions in shared programs: 2910477 -> 2909192 (-0.04%)
instructions in affected programs: 983562 -> 982277 (-0.13%)
helped: 529
HURT: 1
Instructions are helped.
total bytes in shared programs: 19400010 -> 19392214 (-0.04%)
bytes in affected programs: 6785058 -> 6777262 (-0.11%)
helped: 529
HURT: 58
Bytes are helped.
total halfregs in shared programs: 519553 -> 519429 (-0.02%)
halfregs in affected programs: 1877 -> 1753 (-6.61%)
helped: 37
HURT: 12
Halfregs are helped.
total spills in shared programs: 135040 -> 135022 (-0.01%)
spills in affected programs: 135040 -> 135022 (-0.01%)
helped: 6
HURT: 0
total fills in shared programs: 342326 -> 342308 (<.01%)
fills in affected programs: 342326 -> 342308 (<.01%)
helped: 6
HURT: 0
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
b45d54a388
agx: optimize b2x(inot)
...
easier to do as algebraic rules.
total instructions in shared programs: 2915611 -> 2910477 (-0.18%)
instructions in affected programs: 285251 -> 280117 (-1.80%)
helped: 1245
HURT: 5
Instructions are helped.
total bytes in shared programs: 19439752 -> 19400010 (-0.20%)
bytes in affected programs: 1881556 -> 1841814 (-2.11%)
helped: 1244
HURT: 6
Bytes are helped.
total halfregs in shared programs: 519253 -> 519553 (0.06%)
halfregs in affected programs: 5509 -> 5809 (5.45%)
helped: 24
HURT: 107
Halfregs are HURT.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
34ca925064
agx: don't inline imms into stack_store
...
we need opt_large_consts..
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
f6c17cbe48
ail: use template for tiled memcpy
...
less macro hell
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
075e595058
ail: port tiling routines to c++
...
to fix macro hell
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
104ba72547
ail: add tests for linear<-->twiddled copies
...
based on the panfrost tests.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
f836d573e4
agx: only run early tests if needed
...
if it'd be early anyway don't bother, speeds up dolphin with spec shaders
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Asahi Lina
fa475c1b56
asahi: scratch: Add feature to debug core IDs
...
Define SCRATCH_DEBUG_CORES to a raw core count to bypass the mapping
machinery and allocate memory for many cores, so we can work out how the
core IDs are mapped.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Asahi Lina
494399c65c
asahi: Implement scratch allocation
...
Implement our helper program to map scratch blocks into stack memory,
and the driver side that allocates these blocks as necessary.
Alloction is grow-only right now. Drivers are expected to instantiate
scratch memory managers for each shader type (VS, FS, CS) and the same
buffers are reused across commands for each one, growing as necessary.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
da9677f14b
asahi: cmdbuf: Fix scratch bucket offset/size
...
This is 4 bits. It is an opaque value that gets passed to the
helper, except 0 means no scratch.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
d6cc820d1a
asahi: libagx: Move PACKED and GLOBAL macros to libagx.h
...
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
eec7c4beb8
agx: compiler: Enable stack_adjust
...
We'll need this once we start using scratch.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
1f1751d258
agx: compiler: Export scratch size to the driver
...
So it can configure the pipeline correctly, and allocate the required
scratch space.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
b89da92a5e
agx: compiler: Add fence_helper_exit_agx barrier
...
This is used by the helper program on exit.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
c406ce793b
agx: Add scaffolding to build the helper shader at device init
...
Add the scaffolding to compile our helper program, and load it at device
init time.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
05c8b59f5b
agx: Hook up helper intrinsics into CL
...
So we can use these intrinsicts from libagx.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
30862d7189
agx: Hook up AGX helper NIR intrinsics
...
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
5d0f1ca77f
agx: Rename some SRs
...
I think these better represent what these SRs actually do.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
28d34f6352
asahi: libagx: introduce AGX_STATIC_ASSERT
...
Using the array size trick, this works in both OpenCL and C.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
8661006ef0
agx: add some more bitop tests
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Mary Guillemard
923767a968
agx: Add a bitop optimizer pass
...
Signed-off-by: Mary Guillemard <mary@mary.zone>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Mary Guillemard
2b89eb979e
agx: Fuse not into and/or/xor
...
Signed-off-by: Mary Guillemard <mary@mary.zone>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Mary Guillemard
ba508fe854
agx: Remove and/or/xor pseudo ops
...
Switch back to bitop while keeping the aliases on agx_print_instr.
Also add all variants of 2 args of agx_bitop_table.
Signed-off-by: Mary Guillemard <mary@mary.zone>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Mary Guillemard
a5b6ff3ccc
agx: Add more bitops in agx_bitop_table
...
Signed-off-by: Mary Guillemard <mary@mary.zone>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
877c180677
asahi: plug glsl type leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
1e973dd5a9
asahi: rework meta shader infra
...
this fixes the leaks, and is also a lot more pleasant to work with.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
5261962186
asahi: fix prim restart unrolling with indirects
...
need to account for draw->start.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
c0d3bf3608
agx: optimize vote_eq
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
0de7018879
agx: optimize first_invocation
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
fb81201a34
agx: implement active_subgroup_invocation_agx
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
da924e2f03
agx: fuse ballot+cmp
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
def00f7bf0
agx: introduce ballot pseudo
...
for more opts
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
92dcf75fcd
agx: lower more subgroups
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
f52e0bd70b
agx: implement load_subgroup_invocation
...
There's an sr.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
6f7511b522
asahi: decode uniform_high records
...
print them like low uniforms.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
1ee9b8b668
asahi: optimize more when linking libagx
...
Otherwise we get spurious scratch from lower_vars_to_explicit_types on dead
vars.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
5dc0f5ccba
asahi: implement VBO robustness
...
GL semantics. GLES (weaker) and VK (stronger) semantics are left as a todo, with
explanations given. Enabled always to deal with null VBOs, this should be
optimized once we have soft fault.
This necessitates a rework of VBO keys, but hopefully for the best.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
4aadf67523
asahi: fix metadata for images with VS lowered to GS
...
KHR-GL46.shader_image_load_store.basic-allTargets-atomicVS
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:28 +00:00
Alyssa Rosenzweig
9753cd44f7
asahi: Implement skeleton for tessellation
...
This implements a rough skeleton of what's needed for tessellation. It contains
the relevant lowerings to merge the VS and TCS, running them as a compute
kernel, and to lower the TES to a new VS (possibly merged in with a subsequent
GS). This is sufficient for both standalone tessellation and tess + geom/xfb
together. It does not yet contain a GPU accellerated tessellator, simply falling
back to the CPU for that for now. Nevertheless the data structures are
engineered with that end goal in mind, in particular to be able to tessellate
all patches in parallel without needing any prefix sums etc (using simple
watermark allocation for the heap).
Work on fleshing out the skeleton continues in parallel. For now, this does pass
the tests and lets the harder stuff get regression tested more easily. And
merging early will ease rebase.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:28 +00:00
Alyssa Rosenzweig
5e8168f1b7
asahi: add more uapi stubs
...
for scratch
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:28 +00:00
Karol Herbst
f2b7c4ce29
nir: rework and fix rotate lowering
...
No driver supports urol/uror on all bit sizes. Intel gen11+ only for 16
and 32 bit, Nvidia GV100+ only for 32 bit. Etnaviv can support it on 8,
16 and 32 bit.
Also turn the `lower` into a `has` option as only two drivers actually
support `uror` and `urol` at this momemt.
Fixes crashes with CL integer_rotate on iris and nouveau since we emit
urol for `rotate`.
v2: always lower 64 bit
Fixes: fe0965afa6 ("spirv: Don't use libclc for rotate")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by (Intel and nir): Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27090 >
2024-01-22 10:27:44 +00:00
Alyssa Rosenzweig
89de5e22ae
asahi: don't use NIR_PASS_V
...
find . -type f -exec sed -ie 's/NIR_PASS_V(/NIR_PASS(_, /' \{} \;
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976 >
2024-01-12 01:13:03 +00:00
Alyssa Rosenzweig
375ad0ef26
asahi: don't use NIR_PASS for removing entrypoints
...
it's not a real nir pass, doesn't do metadata, etc.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976 >
2024-01-12 01:13:03 +00:00
Alyssa Rosenzweig
afefcc3b30
asahi: avoid silly internal NIR_PASS in gs lowering
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976 >
2024-01-12 01:13:03 +00:00
Alyssa Rosenzweig
c87d46b600
asahi: return progress from passes
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976 >
2024-01-12 01:13:03 +00:00
Alyssa Rosenzweig
8a6850d33c
agx: return progress from passes
...
so we can be smarter about validation
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976 >
2024-01-12 01:13:03 +00:00
Alyssa Rosenzweig
26cedede0d
agx: unset silly nir opts
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976 >
2024-01-12 01:13:03 +00:00
Alyssa Rosenzweig
01344819b0
agx: fix metadata in layer lowering
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976 >
2024-01-12 01:13:03 +00:00
Alyssa Rosenzweig
fe04e85be9
agx: only lower vars to ssa once
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976 >
2024-01-12 01:13:03 +00:00
Alyssa Rosenzweig
08fc7a4082
agx: stop lowering in opt loop
...
silly
total instructions in shared programs: 2278756 -> 2278751 (<.01%)
instructions in affected programs: 587 -> 582 (-0.85%)
helped: 2
HURT: 0
total bytes in shared programs: 15008934 -> 15008916 (<.01%)
bytes in affected programs: 3794 -> 3776 (-0.47%)
helped: 2
HURT: 0
total halfregs in shared programs: 519245 -> 519253 (<.01%)
halfregs in affected programs: 63 -> 71 (12.70%)
helped: 0
HURT: 1
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976 >
2024-01-12 01:13:03 +00:00