Commit graph

184686 commits

Author SHA1 Message Date
Alyssa Rosenzweig
1362ae38d4 asahi: optimize "no changes" case
drawoverhead case 1

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
8fc373370f asahi: collapse stage uniform upload
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
8ba151aa16 asahi: dirty track stage uniforms
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
6be7c5f889 asahi: split up stage uniform upload
so we can update incrementally

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
23b4503225 asahi: rewrite queries
1. always keep the query in gpu memory, so we can implement qbos properly.

2. use a lightweight data structure for tracking writers to reduce overhead

3. allow many writers per query to eliminate stalls

4. use context-wide occlusion heap, to satisfy #1 without introducing
   flushes or silly copies. this is what the pvr mesa driver does :-)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
ca58bc239a asahi: drop silly else
default

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
6ffbf77983 asahi: assert invariant
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
24b7fbca9d asahi: allow disk cache with compblit
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
8ab443d549 asahi: enable compblit behind dbg flag
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
07a4ff5810 asahi: track bit count, not word count
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
0aa3eaa944 asahi: rm dead
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
46c70b72ca asahi: skip set if tested
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
7eac18b1bc asahi: force inline ppp update logic
benefits a lot from constant folding.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
5982ae9025 asahi: fix vbo dirty track
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
668bf461b1 asahi: don't use util_resource_size
drawoverhead.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
d9c4fcce5f asahi: log geometry shaders separate from xfb
different perf issues.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
c6c8262ce1 asahi: implement pipeline stats as a checkbox
real impl is blocked on uapi to plumb thru hw perf counters.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
27d39bf544 agx: reassociate bcsel with ior/iand
fairly dramatic improvement to citra shaders, citra seems to be a % faster with
this.

total instructions in shared programs: 2910477 -> 2909192 (-0.04%)
instructions in affected programs: 983562 -> 982277 (-0.13%)
helped: 529
HURT: 1
Instructions are helped.

total bytes in shared programs: 19400010 -> 19392214 (-0.04%)
bytes in affected programs: 6785058 -> 6777262 (-0.11%)
helped: 529
HURT: 58
Bytes are helped.

total halfregs in shared programs: 519553 -> 519429 (-0.02%)
halfregs in affected programs: 1877 -> 1753 (-6.61%)
helped: 37
HURT: 12
Halfregs are helped.

total spills in shared programs: 135040 -> 135022 (-0.01%)
spills in affected programs: 135040 -> 135022 (-0.01%)
helped: 6
HURT: 0

total fills in shared programs: 342326 -> 342308 (<.01%)
fills in affected programs: 342326 -> 342308 (<.01%)
helped: 6
HURT: 0

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
b45d54a388 agx: optimize b2x(inot)
easier to do as algebraic rules.

total instructions in shared programs: 2915611 -> 2910477 (-0.18%)
instructions in affected programs: 285251 -> 280117 (-1.80%)
helped: 1245
HURT: 5
Instructions are helped.

total bytes in shared programs: 19439752 -> 19400010 (-0.20%)
bytes in affected programs: 1881556 -> 1841814 (-2.11%)
helped: 1244
HURT: 6
Bytes are helped.

total halfregs in shared programs: 519253 -> 519553 (0.06%)
halfregs in affected programs: 5509 -> 5809 (5.45%)
helped: 24
HURT: 107
Halfregs are HURT.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
34ca925064 agx: don't inline imms into stack_store
we need opt_large_consts..

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
f6c17cbe48 ail: use template for tiled memcpy
less macro hell

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
075e595058 ail: port tiling routines to c++
to fix macro hell

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
104ba72547 ail: add tests for linear<-->twiddled copies
based on the panfrost tests.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
2c9189434f docs/asahi: fix strided linear note
turns out linear 2d arrays work

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
f836d573e4 agx: only run early tests if needed
if it'd be early anyway don't bother, speeds up dolphin with spec shaders

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
7825044c39 asahi: honour discard_whole | persistent
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
793ac04077 asahi: handle read-only SSBOs
reduces silliness in Yuzu.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
97a945246c asahi: fix valid buffer tracking for SSBO/image/XFB
smaller ranges.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
6689b083d8 asahi: don't sync for uninitialized buffer
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
9e8ce4550c asahi: enable tcs caching
works fine

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
19da2f6d30 asahi: sync with query mismatches
otherwise the following assertion can fail. fixes crash in yuzu trace.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
3b927567ac Revert "asahi: don't canonicalize nans/flush denorms when copying"
This reverts commit ab7fb3d400.

This breaks ./arb_copy_image-formats -auto --samples=4 since we need the blit
path for compression transitions. The correct thing to do is cast the formats.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Asahi Lina
74110ef21d asahi: batch: Trace before waiting for syncobj
If the render hangs, at least this way we get the trace.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Asahi Lina
ff900ceffe asahi: Enable scratch debugging
Via ASAHI_MESA_DEBUG=scratch. This will assert if enabled and the
scratch workgroup allocation count turns out not to be sufficient (that
is, there were failed allocations), to help debug the max occupancy
calculation.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Asahi Lina
8845702f8c asahi: Allocate scratch for shaders
Finally, now that scratch is available, tell the hardware how much we
need and allocate the scratch buffers as necessary.

We only do the max subgroup calculation for compute right now, and it's
probably conservative (we could probably calculate a reduced occupancy
depending on register pressure). Getting this too low won't break, the
GPU will just be less efficient if there is not enough scratch
allocated. Vertex and fragment just get the max 96 for now.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Asahi Lina
bb5277aa3d asahi: Hook up scratch
Instantiate our 3 scratch managers for each shader type, and pass them
to the hardware if a batch uses scratch.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Asahi Lina
fa475c1b56 asahi: scratch: Add feature to debug core IDs
Define SCRATCH_DEBUG_CORES to a raw core count to bypass the mapping
machinery and allocate memory for many cores, so we can work out how the
core IDs are mapped.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Asahi Lina
494399c65c asahi: Implement scratch allocation
Implement our helper program to map scratch blocks into stack memory,
and the driver side that allocates these blocks as necessary.

Alloction is grow-only right now. Drivers are expected to instantiate
scratch memory managers for each shader type (VS, FS, CS) and the same
buffers are reused across commands for each one, growing as necessary.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
da9677f14b asahi: cmdbuf: Fix scratch bucket offset/size
This is 4 bits. It is an opaque value that gets passed to the
helper, except 0 means no scratch.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
d6cc820d1a asahi: libagx: Move PACKED and GLOBAL macros to libagx.h
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
eec7c4beb8 agx: compiler: Enable stack_adjust
We'll need this once we start using scratch.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
1f1751d258 agx: compiler: Export scratch size to the driver
So it can configure the pipeline correctly, and allocate the required
scratch space.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
b89da92a5e agx: compiler: Add fence_helper_exit_agx barrier
This is used by the helper program on exit.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
c406ce793b agx: Add scaffolding to build the helper shader at device init
Add the scaffolding to compile our helper program, and load it at device
init time.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
05c8b59f5b agx: Hook up helper intrinsics into CL
So we can use these intrinsicts from libagx.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
30862d7189 agx: Hook up AGX helper NIR intrinsics
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
b07dbf7b0f nir: Add AGX-specific helper opcodes
These opcodes are used by the helper program to fetch the current
operation info and core ID.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
5d0f1ca77f agx: Rename some SRs
I think these better represent what these SRs actually do.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
28d34f6352 asahi: libagx: introduce AGX_STATIC_ASSERT
Using the array size trick, this works in both OpenCL and C.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
8661006ef0 agx: add some more bitop tests
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00