Commit graph

5132 commits

Author SHA1 Message Date
Alyssa Rosenzweig
cb0b027c59 asahi: make clip_halfz dynamic
we could move this to the linker but meh, this is good enough for now

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:32 +00:00
Alyssa Rosenzweig
6673924b7e asahi: make gs topology dynamic
even with shobjs, we know the class of topology statically, so we just need to
select between the (up to) 3 compatible topologies, and luckily there are common
subexpressions we can factor out when calculating all 3 at once.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:32 +00:00
Alyssa Rosenzweig
17896f1699 nir: rm load_vert_id_in_prim_agx
now unused since we separate vs/gs

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:31 +00:00
Alyssa Rosenzweig
c6c8262ce1 asahi: implement pipeline stats as a checkbox
real impl is blocked on uapi to plumb thru hw perf counters.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:30 +00:00
Asahi Lina
b89da92a5e agx: compiler: Add fence_helper_exit_agx barrier
This is used by the helper program on exit.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Asahi Lina
b07dbf7b0f nir: Add AGX-specific helper opcodes
These opcodes are used by the helper program to fetch the current
operation info and core ID.

Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
311070f7af nir: add active_subgroup_invocation_agx sysval
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
5dc0f5ccba asahi: implement VBO robustness
GL semantics. GLES (weaker) and VK (stronger) semantics are left as a todo, with
explanations given. Enabled always to deal with null VBOs, this should be
optimized once we have soft fault.

This necessitates a rework of VBO keys, but hopefully for the best.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
9753cd44f7 asahi: Implement skeleton for tessellation
This implements a rough skeleton of what's needed for tessellation. It contains
the relevant lowerings to merge the VS and TCS, running them as a compute
kernel, and to lower the TES to a new VS (possibly merged in with a subsequent
GS). This is sufficient for both standalone tessellation and tess + geom/xfb
together. It does not yet contain a GPU accellerated tessellator, simply falling
back to the CPU for that for now. Nevertheless the data structures are
engineered with that end goal in mind, in particular to be able to tessellate
all patches in parallel without needing any prefix sums etc (using simple
watermark allocation for the heap).

Work on fleshing out the skeleton continues in parallel. For now, this does pass
the tests and lets the harder stuff get regression tested more easily. And
merging early will ease rebase.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:28 +00:00
Alyssa Rosenzweig
2d37d1b704 asahi: lower poly stipple
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616>
2024-02-14 21:02:28 +00:00
Mike Blumenkrantz
9e2c7314f2 nir/lower_io: fix handling for compact arrays with indirect derefs
this logic relies on constant indexing for compact arrays, but this is
frequently not the case for compact array builtins (e.g., gl_TessLevelOuter).
the usual strategy of lowering to temps isn't viable in TCS, which means
io lowering has to be able to handle indirect access to these builtins
without crashing

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27534>
2024-02-13 16:13:13 +00:00
Karol Herbst
727cddd338 nir/lower_cl_images: record image_buffers and msaa_images
Cc: mesa-stable
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27385>
2024-02-13 10:12:13 +00:00
Connor Abbott
6a744ddebc ir3: Initial support for pushing globals with ldg.k
Add a separate pass which uses the analyze_ubo_ranges machinery to
construct ranges of readonly globals accessed in the shader and push
them to constants in the preamble, using ldg.k if possible. This is
enough to handle inline uniforms in turnip but also provides a base for
OpenCL, although the pass would need further work for that.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26934>
2024-02-12 22:05:13 +00:00
Connor Abbott
45c71803f9 tu: Add more info to ldg inline uniform path
This will let us push the ldg into the preamble.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26934>
2024-02-12 22:05:13 +00:00
Sagar Ghuge
c984d6e2fc nir: Drop intel specific lowering code
In previous patches, we have moved the Intel specific lowering code in
brw_nir_lower_texture file. We can go ahead and drop the Intel specific
texture source too.

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27458>
2024-02-12 21:25:48 +00:00
Alyssa Rosenzweig
9c006d5dce nir/passthrough_gs: flesh out gs_in_prim
geometry shaders don't specify the input topology, only the class of topology.
normalize when generating a passthrough gs.

asahi will be more picky about this in the future.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27457>
2024-02-09 11:53:31 +00:00
Marek Olšák
e98bbcad17 nir: add vertex divergence into nir_divergence_analysis
This is a prerequisite for the new nir_opt_varyings pass.
It reuses the same divergent field in nir_def and nir_loop.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26918>
2024-02-02 16:45:52 -05:00
Marek Olšák
5ffa4d879c nir: add a lower_mediump_io callback into options
This will be called by the GLSL linker before nir_opt_varyings.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26918>
2024-02-02 16:45:51 -05:00
Marek Olšák
ecf0fe09f0 nir: replace lower_io_variables with a GLSL NIR flag
This stops using it in nir_lower_io_passes because all callers call it
only when it's true.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26918>
2024-02-02 16:45:49 -05:00
Marek Olšák
c4acab77a8 nir: remove and replace underused option pack_varying_options
This will also be used by nir_opt_varyings.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26918>
2024-02-02 16:45:47 -05:00
Marek Olšák
c844b5dc85 nir: relax validation failure for generic TCS outputs with no_varying
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26918>
2024-02-02 16:45:45 -05:00
Job Noorman
60413e11c2 ir3: optimize subgroup operations using brcst.active
Follow the blob and optimize subgroup operation using brcst.active and
getlast when supported.

The transformation consists of two parts. First, a NIR transform
replaces subgroup operations with a sequence of new brcst_active_ir3
intrinsics followed by a new [type]_clusters_ir3 intrinsic (where type
can be reduce, inclusive_scan, or exclusive_scan).

The brcst_active_ir3 intrinsic is lowered directly to a brcst.active
instruction. The other intrinsics get lowered to a new macro
(OPC_SCAN_CLUSTERS_MACRO) which later gets emitted as a loop (using
getlast/getone) that iterates all clusters and produces the requested
scan result.

OPC_SCAN_CLUSTERS_MACRO has a number of optional arguments. First, since
the exclusive scan result is not a natural by-product of the loop but
has to be calculated explicitly, its destination is optional. This is
necessary since adding it unconditionally will produce unused
instructions that won't be DCE'd anymore at this point. Second, when
performing 32b MUL_U reductions (that expand to multiple instructions),
an extra scratch register is necessary.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6387
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26950>
2024-02-02 19:49:22 +00:00
Konstantin Seurer
c925b6019d radv/rt: Lower ray payloads like hit attribs
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27051>
2024-02-02 16:36:15 +00:00
Ian Romanick
c8ba2bc2f0 nir: Pack texture LOD and array index to a single 32-bit value
v2: Fix clamped_ai calculation in nir_lower_tex.c. Add
nir_tex_src_combined_lod_and_array_index_intel to
print_tex_instr. Suggested by Sagar.

Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27305>
2024-02-02 02:39:10 +00:00
Konstantin Seurer
e3c2dc2324 nir/print: Rename workgroup-size to workgroup_size
Every other field uses _ instead of -.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27318>
2024-01-30 21:19:40 +00:00
Konstantin Seurer
449e44d6d3 nir/print: Don't print shared_size twice
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27318>
2024-01-30 21:19:40 +00:00
Gert Wollny
0ab3b3c641 nir/builder: Fix compilation with gcc-13 when tsan is enabled
../src/compiler/nir/nir_builder.h: In function ‘nir_build_deref_follower’:
../src/compiler/nir/nir_builder.h:1607:1: error: control reaches end of non-void function [-Werror=return-type]
 1607 | }

Fixes: 4a4e175738
    nir: Support deref instructions in lower_var_copies

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27345>
2024-01-30 20:42:07 +00:00
Gert Wollny
80a1b91601 nir/lower_int64: Fix compilation with gcc-13 and tsan enabled
../src/compiler/nir/nir_lower_int64.c: In function ‘lower_int64_intrinsic’:
../src/compiler/nir/nir_lower_int64.c:1347:1: error: control reaches end of non-void function [-Werror=return-type]
1347 | }

Fixes: bf7a114246
   nir/lower_int64: Add lowering for some 64-bit subgroup ops

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27345>
2024-01-30 20:42:07 +00:00
Faith Ekstrand
48ebfeba34 nak: Add a source barrier intrinsic
This just inserts a GPU stall until the given source is available.  We
need this in order to properly implement shader clock.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27303>
2024-01-26 16:55:50 +00:00
Friedrich Vock
9f22b95956 nir: Handle casts in nir_opt_copy_prop_vars
Cc: mesa-stable

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27197>
2024-01-24 12:39:48 +00:00
Friedrich Vock
6c845ed548 nir: Make is_trivial_deref_cast public
Cc: mesa-stable

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27197>
2024-01-24 12:39:48 +00:00
Rhys Perry
e465ac2561 nir/lower_shader_calls: remove CF before nir_opt_if
Otherwise, opt_if_simplification() can attempt to insert an inot after a
jump.

Fixes RADV compilation of a Cyberpunk 2077 pipeline with
PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27193>
2024-01-23 19:02:03 +00:00
Rhys Perry
015b0d678f nir/lower_non_uniform: set non_uniform=false when lowering is not needed
Fixes RADV compilation of a Doom Eternal pipeline with
PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT, because
nir_opt_non_uniform_access was skipped and later passes don't expect
non-uniform access.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: b1619109ca ("nir/lower_non_uniform: remove non_uniform flags after lowering")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27192>
2024-01-23 18:09:39 +00:00
Karol Herbst
f2b7c4ce29 nir: rework and fix rotate lowering
No driver supports urol/uror on all bit sizes. Intel gen11+ only for 16
and 32 bit, Nvidia GV100+ only for 32 bit. Etnaviv can support it on 8,
16 and 32 bit.

Also turn the `lower` into a `has` option as only two drivers actually
support `uror` and `urol` at this momemt.

Fixes crashes with CL integer_rotate on iris and nouveau since we emit
urol for `rotate`.

v2: always lower 64 bit

Fixes: fe0965afa6 ("spirv: Don't use libclc for rotate")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by (Intel and nir): Ian Romanick <ian.d.romanick@intel.com>

Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27090>
2024-01-22 10:27:44 +00:00
Georg Lehmann
d641750573 nir: add lowering for boolean shuffle
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27116>
2024-01-19 20:13:34 +00:00
Georg Lehmann
1cb5bf7009 nir: add ballot_relaxed and as_uniform intrinsics
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27116>
2024-01-19 20:13:33 +00:00
Alyssa Rosenzweig
3a72fc1cb7 nir/passthrough_gs: plug leak
freeing the nir shader should free the xfb info too. found with valgrind
leakcheck.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27093>
2024-01-19 09:10:29 +00:00
Faith Ekstrand
82fe981e35 nir,spirv: Add support for SPV_NV_shader_sm_builtins
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27154>
2024-01-18 20:20:06 +00:00
Karol Herbst
36012af17f nir/printf: remove treat_doubles_as_floats
It is broken and clang uses fp32 for float constants if the fp64 extension
isn't enabled anyway. SPIRVs can't use fp64 constants with printf unless
they enable the Float64 cap, which also requires cl_khr_fp64 to be
supported.

So just remove it and rely on clang handling -cl-single-precision-constant
correctly, which at the moment doesn't seem to be the case, but we can
think about that once we plan to support cl_khr_fp64.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26541>
2024-01-18 13:16:13 +01:00
Lionel Landwerlin
a18ea091af nir/comparison_pre_tests: update expectations
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27087>
2024-01-17 16:01:12 +02:00
Lionel Landwerlin
873fe637e2 nir/alu_srcs_negative_equal: bail earlier if possible
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27087>
2024-01-17 16:00:30 +02:00
Ian Romanick
4740ee8d67 nir: Minor clean up in nir_alu_srcs_negative_equal
Eliminate some cruft left after a8013644a1.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27087>
2024-01-17 16:00:30 +02:00
Sviatoslav Peleshko
6b0bfdfa9e nir: Use alu source components count in nir_alu_srcs_negative_equal
When we use source from ALU instruction directly, the default swizzle array
should be populated with the same amount of components as the src has.

Otherwise, if we use nir_ssa_alu_instr_src_components, it can return
the destination components count that is lower than component index
actually used in that source. This can lead to false equality
between 0 (uninitialized) and 0 (.x) in swizzle comparison below.

Fixes: c6ee46a7 ("nir: Add nir_alu_srcs_negative_equal")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8704
Signed-off-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22655>
2024-01-17 08:05:30 +00:00
Alyssa Rosenzweig
8fd18c4f20 nir/lower_flatshade: fix metadata
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
fcae4b469f nir/lower_io_arrays_to_elements: return prog
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
70fd20d2bc nir/lower_passthrough_edgeflags: return progress
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
460d2ca4f3 nir/lower_point_size_mov: return prog
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
8b7d765e59 nir/lower_alpha_test: rewrite with intrinsics_pass
returns progress now

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
086cbe5da2 nir/lower_bitmap: return prog
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
4833e42721 nir: return prog from drawpixels
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00