Commit graph

180031 commits

Author SHA1 Message Date
Alyssa Rosenzweig
fe7650bcf7 asahi: Add GS lowering pass
The big monster.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
43e134b106 asahi: Add helpers for lowering GS
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
6a67e220b8 asahi: Add data structures for geometry shaders
Shared between GPU and CPU.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
0b96b62348 asahi: Make encoder_allocate public
In order to enqueue CDM and VDM to the same queue.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
c9dd59fef3 asahi: Factor out agx_launch
For launching compute kernels on particular batches without dirtying state.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
212f71734e asahi: Refactor encoder data structure
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
82e3eb4b74 asahi: Don't hazard track fake resources
Convenient for meta, which we use for implementing GS.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
949741d4c3 asahi: Upload tex/samplers properly with merged shaders
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
21d3c1e0af asahi: Use the sampler heap
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
dc44619c62 asahi: Add sampler heap data structure
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
9c575eb18a asahi: Handle load_sampler_handle
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
df2c145c91 agx: Handle bindless samplers
Unified encoding.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
ca42562c7f agx: Lower LOD bias earlier
To make the extra descriptor accesses explicit for drivers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
972f289740 asahi: Lower samplers to bindless if needed
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
111e526f19 agx: Allow drivers to lower texture handles
Rather than hardcoding u0_u1, this lets drivers map texture handles in whatever
way is convenient. In particular, this makes textures work properly with merged
shader stages (provided one of the stages is forced to use bindless access), by
giving each stage an independent texture heap.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
b69ab37bdc asahi: Pack non-border colour sampler desc
Apparently custom border colours don't work properly with bindless. Needs
investigation. Patch this out so we don't fault.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
a74fbb3840 agx: Translate simple subgroup ops
We'll use these for optimizing parallel prefix sums.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
77bb446e90 agx: Add scaffolding for subgroup ops
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
5b754410da agx: Require 32-bit alignment for EOT offset
Fixes piles of brokenness.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
7d7f5013f8 agx: Cleanup 8-bit math before lowering
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
3b84f371a8 docs/features: Mark ARB_mdi done on asahi
Sync with reality.

I think all gallium drivers support, it's a cpu-only thing unlike gl4.6's
indirect multidraws or the similarly named vk feature.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
b18181d924 agx: Check for spilling in release builds
Don't smash stack -- explain to the user what happened.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Alyssa Rosenzweig
7b92c63105 agx: Fix fragment side effects scheduling
We can't move discards across side effects, or the side effect might not happen.

Fixes KHR-GLES31.core.shader_image_load_store.basic-allFormats-load-fs
regression. Sigh.

CI is up next.

Fixes: 119e5b9719 ("agx: Schedule for register pressure")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:55 +00:00
Christian Gmeiner
e928f45735 agx: Re-index nir defs to reduce memory usage
nir_index_ssa_defs(..) will re-index the function impl and will
update ssa_alloc. In almost all cases this will result in a lower
ctx->alloc number which reduces memory usage in compiler passes
that are using ctx-alloc to allocate memory.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
b6b01aa1f2 agx: Legalize image MS index
Fix 2D MSAA Array tests in arb_shader_image_load_store-max-size

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
019a52fff0 asahi: Clamp 8-bit integer RTs
Fixes gl-3.0-render-integer.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
8d9d9d0207 nir/print: handle adjacency
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
d0a4a8cda0 nir: Add intrinsics for lowering bindless textures/samplers
Needed for merged stages to work properly.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
33e80918de nir: Add intrinsics for lowering GS
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
cc3f20ca6c nir: Also gather decomposed primitive count
Simple extension.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
b65636ca40 nir/lower_gs_intrinsics: Count decomposed primitives too
We need both: decomposed primitives for transform feedback and regular
primitives for the sizing the index buffer.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
0a35aa3a2b nir/lower_gs_intrinsics: Append EndPrimitive
This is simpler for generic GS lowering.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
f157a3de4e nir/lower_gs_intrinsics: Include primitive counts
Generic GS lowering needs this, we already calculate it.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
a147801f9b compiler: Make u_decomposed_prims_for_vertices available to CL
For indirect geometry shader setup.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
64f7b70763 compiler: Inline mesa_vertices_per_prim
Makes it more easily consumable from the gpu.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
7cfe2ecb33 compiler: Make shader_enums.h CL-safe
macros.h is not safe for CL for a bunch of reasons but shader_enums.h barely
uses its functionality. Stub out the minimum for CL.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Karol Herbst
1519ff7ebd rusticl/memory: fix new clippy::needless-borrow warning
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Antonio Gomes <antoniospg100@gmail.com>
Reviewed-by: @LingMan <18294-LingMan@users.noreply.gitlab.freedesktop.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26051>
2023-11-06 23:47:18 +00:00
Karol Herbst
98df65a875 rusticl: only support the matching device for gl_sharing
Cross vendor is bogus because of modifier screw ups. We could allow it on
all devices from the same vendor, but for that we have to check if the
supported modifiers match.

Also verify the device in questions actually supports gl_sharing.

Fixes: 57dfc013a6 ("rusticl: Add functions to create CL ctxs from GL, and also to query them")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Antonio Gomes <antoniospg100@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26051>
2023-11-06 23:47:18 +00:00
Georg Lehmann
6cd78281f6 aco: deduplicate Format definition
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25943>
2023-11-06 23:16:38 +00:00
Georg Lehmann
6e0bf33a89 aco: deduplicate instr_class definition
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25943>
2023-11-06 23:16:38 +00:00
Georg Lehmann
bdd81c6be7 aco: namespace aco_opcode
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25943>
2023-11-06 23:16:38 +00:00
Georg Lehmann
1b9a3b7466 aco: stop using cstdint
We use stdint.h everywhere else.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25943>
2023-11-06 23:16:38 +00:00
Georg Lehmann
04956d54ce aco: force uniform result for LDS load with uniform address if it can be non uniform
Because a LDS load is 2 separate loads on gfx10+ with wave64, a different wave
can write LDS in between and cause a non uniform result. Use v_readfirst_lane
instead of p_as_uniform because it cannot be copy propagated.

Fixes a OpenCL CTS test with zink+rusticl.

Totals from 136 (0.17% of 78196) affected shaders:
MaxWaves: 3236 -> 3244 (+0.25%)
Instrs: 130069 -> 131221 (+0.89%)
CodeSize: 698048 -> 703436 (+0.77%)
VGPRs: 5464 -> 5440 (-0.44%)
SpillSGPRs: 94 -> 96 (+2.13%)
Latency: 5361017 -> 5363781 (+0.05%); split: -0.00%, +0.05%
InvThroughput: 883010 -> 884100 (+0.12%)
SClause: 3822 -> 3821 (-0.03%); split: -0.05%, +0.03%
Copies: 14220 -> 14314 (+0.66%); split: -0.01%, +0.68%
Branches: 4549 -> 4551 (+0.04%)
PreSGPRs: 4934 -> 4940 (+0.12%)
PreVGPRs: 4666 -> 4655 (-0.24%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25973>
2023-11-06 22:43:33 +00:00
Eric Engestrom
a4597777fe ci: restore sanity (aka. Revert "ci: don't run sanity in Marge pipelines")
This reverts commit fac60c140b.

Turns out there are pipelines that do not create any other jobs, and
Marge requires a pipeline to pass, which means a pipeline needs to
exist, which means a job needs to always exist.

There is no reason `sanity` would be the one, but it's there so let's
just use it instead of making another one.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26080>
2023-11-06 21:49:44 +00:00
Georg Lehmann
ab87831ae8 aco, radv: vectorize f2f16 if rounding mode is rtz
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25952>
2023-11-06 21:05:34 +00:00
Eric Engestrom
b781bd478c zink+radv/ci: ensure renderer is "zink on radv"
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26065>
2023-11-06 18:54:37 +00:00
Eric Engestrom
cd600868b4 zink+radv/ci: simplify deqp config
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26065>
2023-11-06 18:54:37 +00:00
Andrew Gazizov
d1009481c8 venus: Add use_guest_vram capset to enable guest-based blob alloc
For hypervisors that do not support host memory injection into the
guest address space, it's necessary to have guest-based blob alloc.
Therefore, use a new 'use_guest_vram' virgl capset to decide on
performing guest blob allocations from dedicated heap (Host visible
memory).

Signed-off-by: Andrew D. Gazizov <andrew.gazizov@opensynergy.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25889>
2023-11-06 18:33:15 +00:00
Mary Guillemard
f71f5cf856 venus: skip bind sparse info when checking for feedback query
Fix an assertion when using vkQueueBindSparse.

Fixes: 7fbf608f2d ("venus: append query feedback at submission time")
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26027>
2023-11-06 18:15:03 +00:00
Vitaliy Triang3l Kuzmin
bae7538a02 r600/sfn: Remove unused sampler reference in emit_tex_lod
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26068>
2023-11-06 16:47:29 +00:00