Commit graph

718 commits

Author SHA1 Message Date
Alyssa Rosenzweig
8f9cf43828 asahi/virtio: fix allocate-with-alignment
fixes crashes when spilling inside the microVM:

../src/asahi/lib/agx_scratch.c:176: void agx_scratch_realloc(struct agx_scratch *): Assertion `!(blocks_gpu & (block_size_bytes - 1))' failed

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31532>
2024-10-05 18:30:11 +00:00
Alyssa Rosenzweig
e0cda48a90 asahi/virtio: fix ptr cats
../src/asahi/lib/decode.c:933:7: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  933 |       (void *)c->vertex_attachments;
      |       ^
../src/asahi/lib/decode.c:941:7: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  941 |       (void *)c->fragment_attachments;

etc

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31532>
2024-10-05 18:30:11 +00:00
Alyssa Rosenzweig
9bd3a854e6 asahi/decode: fix ptr casts
../src/asahi/lib/decode.c:933:7: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  933 |       (void *)c->vertex_attachments;
      |       ^
../src/asahi/lib/decode.c:941:7: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
  941 |       (void *)c->fragment_attachments;

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31532>
2024-10-05 18:30:11 +00:00
Alyssa Rosenzweig
88a0da6b7c asahi: identify factor buffer size
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31532>
2024-10-05 18:30:11 +00:00
Alyssa Rosenzweig
bbad4454bf asahi: fix UB affecting x86 builds
dEQP-VK.robustness.image_robustness.bind.notemplate.r32f.unroll.nonvolatile.sampled_image.no_fmt_qual.img.samples_1.2d_array.comp

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31532>
2024-10-05 18:30:11 +00:00
Alyssa Rosenzweig
673f56fcb1 libagx: drop unused arg
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31532>
2024-10-05 18:30:11 +00:00
Alyssa Rosenzweig
8d95f584da libagx: simplify tcs out calc
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31532>
2024-10-05 18:30:10 +00:00
Alyssa Rosenzweig
9cfffc5e86 libagx: cleanup loop vars
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31532>
2024-10-05 18:30:10 +00:00
Asahi Lina
01ef3152c9 agx: Fix implicit sync for virtgpu
The asahi kernel driver is a pure-explicit-sync driver and userspace is
required to handle implicit sync itself, by importing/exporting fences
in shared dma-bufs. Mesa handles this in its own native or guest
context, but dma-buf fences are not shared between the guest and the
host, so this breaks implicit sync across the VM boundary.

To make this work, explicitly pass a resource list to the host so it can
perform the implicit sync dance, like we do in agx_batch.c. This
essentially turns the virtgpu protocol into an implicit sync protocol
(like many other legacy GPU drivers), which makes sense here since we
don't particularly have the primitives to pass through and manage "host"
syncobjs that we'd need to do it at that level.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31532>
2024-10-05 18:30:10 +00:00
Alyssa Rosenzweig
2f0935459f asahi: add tib sample setter helper
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31532>
2024-10-05 18:30:10 +00:00
Iago Toral Quiroga
aac1c074cc nir: make fclamp_pos_mali and fsat_signed_mali opcodes generic
V3D can use these too.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31480>
2024-10-03 09:02:07 +00:00
Alyssa Rosenzweig
d19ac31fa7 asahi: add missing upcast for eMRT MSAA
fixes crash with eMRT in dEQP-VK.fragment_operations.early_fragment.samplemask_early_fragment_tests_depth_samples_4

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30981>
2024-09-02 23:27:15 +00:00
Alyssa Rosenzweig
591bf066ba asahi: add agx_fill_decompress_push helper
easy share between gl and vk

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30981>
2024-09-02 23:27:15 +00:00
Alyssa Rosenzweig
93695ceb48 asahi: factor out agx_usc_shared_non_fragment
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30981>
2024-09-02 23:27:15 +00:00
Alyssa Rosenzweig
93785ae2fb asahi: skip unmapped bo in fault dbg
segfault

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30981>
2024-09-02 23:27:15 +00:00
Alyssa Rosenzweig
1e2dcae4da asahi,hk: deduplicate txf sampler settings
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30981>
2024-09-02 23:27:15 +00:00
Alyssa Rosenzweig
ed08fe94d2 libagx: add in-place decompression kernel
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30981>
2024-09-02 23:27:15 +00:00
Alyssa Rosenzweig
f977c52b84 ail: swallow up formats
ail is a more sensible place for the format tables to live. this does create a
bit of dependency soup but hey.

nfc

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30981>
2024-09-02 23:27:14 +00:00
Alyssa Rosenzweig
05d535cb87 agx: support bindless block image store
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30981>
2024-09-02 23:27:14 +00:00
Alyssa Rosenzweig
afc7557cb6 nir,agx: make block image store an image() intrinsic
so we can do a bindless version

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30981>
2024-09-02 23:27:14 +00:00
Alyssa Rosenzweig
6405153a07 asahi: switch to block based compute blitter
This fixes the compute blitter with compression in the general case, and then
flips the switch since the compute blitter is faster / less buggy than the
traditional path.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 19:11:23 -04:00
Alyssa Rosenzweig
e754e54f88 nir: model AGX explicit coordinate intrinsics
I don't know what Apple calls these, so we're using the name "explicit
coordinates".

AGX has instructions for loading/stores register <---> tilebuffer ---> storage
images. Usually these are used in the fragment shader and end-of-tile shader to
implement colour attachments, with implicitly specified coordinates based on the
shader stage. However they can also be used in compute shaders with explicitly
specified coordinates ("imageblocks" in Apple parlance). Model this in NIR.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:46:31 -04:00
Alyssa Rosenzweig
907fe00da1 agx: handle min_lod sources
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:46:31 -04:00
Alyssa Rosenzweig
a6d7991475 libagx: add drawIndirectCount clone&patch kernel
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:46:31 -04:00
Alyssa Rosenzweig
765e6ad2fc asahi,hk: generalize bo_bind hook
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:45:58 -04:00
Alyssa Rosenzweig
f4cca9d600 asahi: introduce agx_va data structure
prep for sparse.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:45:57 -04:00
Alyssa Rosenzweig
5c7dd392d3 asahi: always pass align parameter for BOs
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:36:26 -04:00
Alyssa Rosenzweig
7ff31a4bf8 asahi: drop iThing logic
wontfix.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:36:26 -04:00
Alyssa Rosenzweig
8dae4aacb4 asahi: stop fixing up USC addresses
use the real VA space.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:36:26 -04:00
Alyssa Rosenzweig
77ea124d7e asahi: drop agx_bo::type
macOS silliness.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:36:26 -04:00
Alyssa Rosenzweig
1899ea319f asahi: drop mprotect in decoding
now that genxml does our decoding it's not buying us much.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:36:26 -04:00
Alyssa Rosenzweig
91257a9bb8 asahi: drop segment list parsing
we're not targeting macOS so this is dead.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:36:26 -04:00
Alyssa Rosenzweig
df725d4f64 asahi: remove agx_bo::dev
track at call sites.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:36:26 -04:00
Alyssa Rosenzweig
1fd66d3859 asahi: optimize out vbo clamp with soft fault
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:36:26 -04:00
Alyssa Rosenzweig
dc3288dcb1 agx: exploit soft fault info
for preambles and for peephole selection.

total instructions in shared programs: 2159359 -> 2114124 (-2.09%)
instructions in affected programs: 359763 -> 314528 (-12.57%)
helped: 814
HURT: 6
Instructions are helped.

total alu in shared programs: 1685059 -> 1670200 (-0.88%)
alu in affected programs: 217210 -> 202351 (-6.84%)
helped: 589
HURT: 45
Alu are helped.

total fscib in shared programs: 1681202 -> 1666324 (-0.88%)
fscib in affected programs: 217477 -> 202599 (-6.84%)
helped: 590
HURT: 45
Fscib are helped.

total ic in shared programs: 460856 -> 455502 (-1.16%)
ic in affected programs: 41350 -> 35996 (-12.95%)
helped: 174
HURT: 8
Ic are helped.

total bytes in shared programs: 14302484 -> 14053982 (-1.74%)
bytes in affected programs: 2380614 -> 2132112 (-10.44%)
helped: 814
HURT: 7
Bytes are helped.

total regs in shared programs: 662302 -> 656517 (-0.87%)
regs in affected programs: 26979 -> 21194 (-21.44%)
helped: 432
HURT: 9
Regs are helped.

total uniforms in shared programs: 1651909 -> 1687077 (2.13%)
uniforms in affected programs: 95383 -> 130551 (36.87%)
helped: 17
HURT: 783
Uniforms are HURT.

total threads in shared programs: 20324608 -> 20326592 (<.01%)
threads in affected programs: 16192 -> 18176 (12.25%)
helped: 17
HURT: 3
Threads are helped.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:36:26 -04:00
Alyssa Rosenzweig
f0cd599913 agx: factor out agx_device_key
we will have other device-dependent key members soon.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:36:26 -04:00
Asahi Lina
c7994a2955 asahi: Update to UAPI 10011
Incompatible changes:
- Make VM layout more flexible to allow for SVM with rusticl
  (eventually, hopefully)

Compatible changes:
- Expose soft fault state to userspace as a flag

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633>
2024-08-12 18:35:47 -04:00
Alyssa Rosenzweig
fbbdc965aa asahi: don't count helper invs in pipeline stats query
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30488>
2024-08-06 11:48:04 -04:00
Alyssa Rosenzweig
767280fd73 libagx: rework tess i/o
this simplifies state which hk wants.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00
Alyssa Rosenzweig
2bf01845d0 libagx: mask counters to 32-bit
possibly we should use real 64-bit counters, that's something for later though.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00
Alyssa Rosenzweig
16caad4038 libagx: add statistic increment kernel
for TCS

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00
Alyssa Rosenzweig
76e3bd56f6 asahi: offset buffer images in software
this is needed for honeykrisp to implement
uniformTexelBufferOffsetSingleTexelAlignment.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00
Alyssa Rosenzweig
735c63c75e libagx: hoist code out of loop
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00
Alyssa Rosenzweig
4e5ce7e759 asahi: split up agx_nir_lower_vs_before_gs
honeykrisp needs to preserve info for a little longer.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00
Alyssa Rosenzweig
d3e7a7b8c8 asahi: add TES->compute lowering
honeykrisp wants to do this explicitly so we don't need prologs for TES. the gl
driver uses TES prologs implicitly for the same effect, but that's ...
suboptimal.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00
Alyssa Rosenzweig
9595d79b89 asahi: make GS inputs explicit
we don't want to assume VS->GS, since we want to reuse the root uniforms across
the whole draw with honeykrisp tess+gs.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00
Alyssa Rosenzweig
f6e9e7d3ce libagx: fix prefix sum kernel
the calculation of workgroup reductions was wrong, giving nondeterministic
results when prefix summing >= 1024 items. fixes misrendering in
terraintessellation on honeykrisp.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00
Alyssa Rosenzweig
bdbd81ea40 asahi: drop #includes
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00
Alyssa Rosenzweig
21fa5faeec asahi: drop pointless conversion
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00
Alyssa Rosenzweig
96306a8b21 asahi: refactor prolog lowering
combine 2 passes and get something less silly as the result.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30382>
2024-07-26 18:40:47 +00:00