Commit graph

1285 commits

Author SHA1 Message Date
Karol Herbst
f2b7c4ce29 nir: rework and fix rotate lowering
No driver supports urol/uror on all bit sizes. Intel gen11+ only for 16
and 32 bit, Nvidia GV100+ only for 32 bit. Etnaviv can support it on 8,
16 and 32 bit.

Also turn the `lower` into a `has` option as only two drivers actually
support `uror` and `urol` at this momemt.

Fixes crashes with CL integer_rotate on iris and nouveau since we emit
urol for `rotate`.

v2: always lower 64 bit

Fixes: fe0965afa6 ("spirv: Don't use libclc for rotate")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by (Intel and nir): Ian Romanick <ian.d.romanick@intel.com>

Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27090>
2024-01-22 10:27:44 +00:00
Georg Lehmann
d641750573 nir: add lowering for boolean shuffle
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27116>
2024-01-19 20:13:34 +00:00
Karol Herbst
36012af17f nir/printf: remove treat_doubles_as_floats
It is broken and clang uses fp32 for float constants if the fp64 extension
isn't enabled anyway. SPIRVs can't use fp64 constants with printf unless
they enable the Float64 cap, which also requires cl_khr_fp64 to be
supported.

So just remove it and rely on clang handling -cl-single-precision-constant
correctly, which at the moment doesn't seem to be the case, but we can
think about that once we plan to support cl_khr_fp64.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26541>
2024-01-18 13:16:13 +01:00
Alyssa Rosenzweig
fcae4b469f nir/lower_io_arrays_to_elements: return prog
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
70fd20d2bc nir/lower_passthrough_edgeflags: return progress
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
460d2ca4f3 nir/lower_point_size_mov: return prog
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
8b7d765e59 nir/lower_alpha_test: rewrite with intrinsics_pass
returns progress now

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
086cbe5da2 nir/lower_bitmap: return prog
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
4833e42721 nir: return prog from drawpixels
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
6fa32b5b83 nir/lower_clip_cull_distance_arrays: return prog
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Alyssa Rosenzweig
a36812d9b4 nir/lower_io_to_temporaries: return prog
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26976>
2024-01-12 01:13:02 +00:00
Rhys Perry
0477421f7d nir: add msad_4x8
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
2024-01-05 18:55:22 +00:00
Daniel Schürmann
bdbf873b0f nir: remove redundant passes from nir_opt_if()
These are now covered by nir_opt_loop():
- opt_if_loop_last_continue()
- opt_merge_breaks()
- opt_if_loop_terminator()

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24940>
2024-01-03 20:48:05 +00:00
Daniel Schürmann
5b1b5cd794 nir: remove nir_opt_trivial_continues()
This pass is superseded by nir_opt_loop()

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24940>
2024-01-03 20:48:04 +00:00
Daniel Schürmann
9808ef0349 nir/opt_loop: move loop control-flow optimizations into separate pass
This new pass aims to simplify loop control-flow by reducing the number
of break and continue statements. It also supersedes nir_opt_trivial_continues().

For this purpose, it implements 3 optimizations:
- opt_loop_terminator(), as previously
- opt_loop_merge_break_continue(), similar to opt_merge_breaks() incl. continues
- opt_loop_last_block(), a generalization of opt_if_loop_last_continue()

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24940>
2024-01-03 20:48:04 +00:00
Yonggang Luo
0210b554d6 treewide: Replace the include of nir_types.h with glsl_types.h
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26753>
2023-12-30 15:08:11 +00:00
Bas Nieuwenhuizen
da6a5e1f63 nir: Add pass for clearing memory at the end of a shader.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26679>
2023-12-20 09:15:45 +00:00
Bas Nieuwenhuizen
bc99b73d70 nir: Add nir_static_workgroup_size helper.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26679>
2023-12-20 09:15:45 +00:00
Faith Ekstrand
3e042173e4 nir/lower_doubles: Add lowering for fmin/fmax/fsat
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26587>
2023-12-20 02:40:25 +00:00
Job Noorman
6cad2fc230 nir: add helper to create cursor after all @decl_regs
@decl_reg intrinsics must be in the first block so it's convenient to be
able to create an insertion point after all @decl_regs when the first
block needs to be split.

Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26737>
2023-12-18 14:52:02 +00:00
Job Noorman
6e7a61df4c nir: add _safe variants of nir_foreach_reg_load/store
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26175>
2023-12-15 17:19:28 +00:00
Faith Ekstrand
aac1e3f595 nir: Add a new has_fmulz_no_denorms flag
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26569>
2023-12-11 15:29:17 +00:00
Karol Herbst
6979a1aa07 nir/opt_preamble: make load_workgroup_size handling optional
not all drivers support it being in the preamble, e.g. asahi.

Signed-off-by: Karol Herbst <git@karolherbst.de>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26614>
2023-12-09 10:56:37 -04:00
Timothy Arceri
c468aa6059 nir: add nir_fixup_deref_types()
This will be used to fix up types after arrays have been resized.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26534>
2023-12-07 03:05:09 +00:00
Faith Ekstrand
09fc5e1c4d nir: Split has_[su]dot_4x8 bits into regular and _sat versions
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26533>
2023-12-06 23:15:33 +00:00
Faith Ekstrand
629af540ca spirv: Plumb variable alignments through to NIR
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26522>
2023-12-06 18:37:36 +00:00
Alyssa Rosenzweig
e14633fa7d nir/lower_tex: Add 1D lowering
From amd/common.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26377>
2023-11-28 20:32:03 +00:00
Marek Olšák
27a9ddad28 nir: return progress from nir_remove_sysval_output
Changing IO semantics doesn't affect the SSA structure.

Same as nir_remove_varying.

Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26274>
2023-11-24 15:37:24 +00:00
Marek Olšák
32ee6376ad nir: add lowering from FS LAYER input to LAYER_ID sysval
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26274>
2023-11-24 15:37:24 +00:00
Timothy Arceri
110887de2b glsl/nir: implement a nir based lower distance pass
This is based off the original GLSL IR pass but it is much much
simpler as it doesn't need to do all of the hackery required in
GLSL IR to achieve the lowering.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25860>
2023-11-21 00:45:21 +00:00
Qiang Yu
7e4aac46ad nir: add force_f2f16_rtz option to lower f2f16 to f2f16_rtz
Used by OpenGL driver like radeonsi which has undefined rounding mode.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25990>
2023-11-20 02:20:17 +00:00
Jesse Natalie
df4fc66649 nir: Add a flag to opt_if to prevent fighting with splitting 64bit phis
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Bill Kristiansen <billkris@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26210>
2023-11-17 19:19:11 +00:00
Alyssa Rosenzweig
6d73f62e42 nir: Simplify nir_alu_instr_channel_used definition
Deduplicates the "get # of channels" logic which was the same between the
helpers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
2023-11-17 09:36:08 -04:00
Connor Abbott
1dab2c5bd2 nir/subgroups: Add option to lower Boolean subgroup reductions
This will be useful for AMD, and probably Intel as well.

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/218>
2023-11-17 09:45:40 +00:00
Rhys Perry
288e9db053 nir/lower_fp16_casts: add option to split fp64 casts
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25566>
2023-11-16 11:02:30 +00:00
Karol Herbst
41f814df6f nir: allow vec derefs on system values
There is no real reason to prevent this as far as I know. And some of the
SPIR-V generated by DPCPP is running into this.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25701>
2023-11-15 08:34:57 +00:00
Alyssa Rosenzweig
cc3f20ca6c nir: Also gather decomposed primitive count
Simple extension.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
b65636ca40 nir/lower_gs_intrinsics: Count decomposed primitives too
We need both: decomposed primitives for transform feedback and regular
primitives for the sizing the index buffer.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Alyssa Rosenzweig
0a35aa3a2b nir/lower_gs_intrinsics: Append EndPrimitive
This is simpler for generic GS lowering.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
2023-11-07 00:05:54 +00:00
Faith Ekstrand
1793adbd3a nir/validate: Allow array derefs on vectors on function/shader_temp
This is required by OpenCL.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580>
2023-11-02 20:28:46 +00:00
Alyssa Rosenzweig
23bea25207 nir: Add nir_remove_non_exported
For libraries.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25498>
2023-11-02 11:37:46 +00:00
Alyssa Rosenzweig
6014f745d5 nir,vtn: Add exported bool to nir_function
For optimizing libraries.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25498>
2023-11-02 11:37:46 +00:00
Rhys Perry
debddca134 nir: add helpers to skip idempotent passes
For example, in the loop:
while (more_late_algebraic) {
   more_late_algebraic = false;
   NIR_PASS(more_late_algebraic, nir, nir_opt_algebraic_late);
   NIR_PASS(_, nir, nir_opt_constant_folding);
   NIR_PASS(_, nir, nir_copy_prop);
   NIR_PASS(_, nir, nir_opt_dce);
   NIR_PASS(_, nir, nir_opt_cse);
}
if nir_opt_algebraic_late makes no progress, later passes might be
skippable depending on which ones made progress in the previous iteration.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24197>
2023-11-01 14:16:37 +00:00
Faith Ekstrand
d5c310899a nir: Split nir_lower_subgroup_options::lower_vote_eq into two bits
On NVIDIA, we can do a vote_ieq on bool in one hardware op so we don't
want that lowered.  We do want to lower vote_feq and other vote_ieq,
though.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894>
2023-10-26 23:05:44 +00:00
Faith Ekstrand
16664b74a2 nir: Add a lower_read_first_invocation option to lower_subgroups
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894>
2023-10-26 23:05:44 +00:00
Faith Ekstrand
3d027cca1e nir: Add a lower_first_invocation_to_ballot option to lower_subgroups
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894>
2023-10-26 23:05:44 +00:00
Jani Nikula
eabd93bba8 docs/nir: use hawkmoth instead of doxygen
Use the hawkmoth c:auto* directives to incorporate nir documentation.

Convert @param style parameter descriptions to rst info field lists.
Add static stubs for generated headers. Fix a lot of references, in
particular the symbols are now in the Sphinx C domain, not C++
domain. Tweak syntax here and there.

Based on the earlier work by Erik Faye-Lund <kusmabite@gmail.com>

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24507>
2023-10-26 16:13:26 +00:00
Jani Nikula
d407cd8216 nir: drop **< style documentation comments
Prepare for using Hawkmoth.

Hawkmoth does not support trailing comments using /**< ... */
syntax. Replace with regular documentation comments.

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24507>
2023-10-26 16:13:25 +00:00
Jani Nikula
4024d0c196 nir: add names to some typedef'd structs/enums
In order to document "typedef struct { ... } T;" as a struct in
hawkmoth, the structs need to have names. Similar for enums.

Note: This is no longer required with Hawkmoth 0.16.0+ and Clang 16 and
later. With the next Hawkmoth release, this should be fixed also for
Clang 15 and earlier.

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24507>
2023-10-26 16:13:25 +00:00
Faith Ekstrand
1fa7c37a36 nir: Add NVIDIA-specific I/O intrinsics
NVIDIA hardware doesn't take a vertex index for per-vertex I/O.
Instead, it takes an offset into the primitive.  This has to be fetched
using a combination of SR_INVOCATION_INFO and the ISBERD instruction.
To keep things simple and allow for maximum CSE, we do the lowering in
NIR and patch the load/store_per_vertex_input/output intrinsic.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>
2023-10-24 22:21:18 +00:00