Commit graph

1243 commits

Author SHA1 Message Date
Alyssa Rosenzweig
4c45503aae nir: Add nir_before/after_impl cursors
These are common enough to merit their own helpers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24910>
2023-08-30 19:30:58 +00:00
Piotr Kocia
27eafbcd4e nir: Remove dead nir_const_value variables
nir_const_value variables in nir_const_value_for_int and
nor_const_value_for_uint are unused resulting in unnecessary dead code.
The unused-variable warning has been suppressed by the memset following
their declarations.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24851>
2023-08-23 19:29:19 +00:00
Georg Lehmann
9cf6984200 nir: unify lower_find_msb with has_{find_msb_rev,uclz}
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24662>
2023-08-22 12:08:37 +00:00
Georg Lehmann
2ac7e6614a nir: unify lower_bitfield_extract with has_bfe
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24662>
2023-08-22 12:08:37 +00:00
Georg Lehmann
34c3f81614 nir: unify lower_bitfield_insert with has_{bfm,bfi,bitfield_select}
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24662>
2023-08-22 12:08:37 +00:00
Alyssa Rosenzweig
a257e2daad nir: Lower fquantize2f16
Passes dEQP-VK.spirv_assembly.*opquantize*.

Unlike the DXIL lowering, this should correctly handle NaNs. (I belive Dozen has
a bug here that is masked by running constant folding early and poor CTS
coverage.) It is also faster than the DXIL lowering for hardware that supports
f2f16 conversions natively. It is not as good as a backend implementation that
could flush-to-zero in hardware... but for a debug instruction it should be more
than good enough.

It might be slightly better to multiply with 0.0 to get the appropriate zero,
but NIR really likes optimizing that out ...

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24616>
2023-08-18 22:20:02 +00:00
Konstantin Seurer
ccc52ae887 nir: Add shader enqueue data structures and handling
There are two new variable modes:
- nir_var_mem_node_payload
- nir_var_mem_node_payload_in

Also add a few more intrinsics and some shader info.

Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24512>
2023-08-18 16:57:22 +00:00
Faith Ekstrand
96c0f8c580 nir: Drop nir_instr_rewrite_src()
Replace all its remaining users with nir_src_rewrite().

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24729>
2023-08-18 01:00:15 +00:00
Faith Ekstrand
71ddaca2e2 nir: Drop nir_instr_rewrite_src_ssa()
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24729>
2023-08-18 01:00:15 +00:00
Faith Ekstrand
964c73e13e nir: Drop nir_if_rewrite_condition()
Use nir_src_rewrite() instead.  In a couple of cases, we can even drop a
switch on whether or not it's an if source.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24729>
2023-08-18 01:00:15 +00:00
Faith Ekstrand
267b4fb1b9 nir: Add and use a nir_instr_init_src() helper
This helper exists for a very tiny set of use-cases but it's better to
have the helper live in nir.c than hand-roll it elsewhere.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24729>
2023-08-18 01:00:15 +00:00
Faith Ekstrand
9c8cb69c15 nir: Add a nir_instr_clear_src() helper and use it
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24729>
2023-08-18 01:00:15 +00:00
Faith Ekstrand
53294de682 nir: Take a nir_def * in nir_phi_instr_add_src()
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24729>
2023-08-18 01:00:14 +00:00
Faith Ekstrand
298a3eebd8 nir: Take a nir_def * in nir_tex_instr_add_src()
NIR bits were hand-typed.  Driver updates done through the following
semantic patch:

    @@
    expression T, ST, D;
    @@

    -nir_tex_instr_add_src(T, ST, nir_src_for_ssa(D));
    +nir_tex_instr_add_src(T, ST, D);

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24729>
2023-08-18 01:00:14 +00:00
Caio Oliveira
74746ac03a nir: Add nir_opt_reuse_constants()
Currently SPIR-V does pull all the NIR constants it creates into the
first block, but we plan to change that behavior to let those constants
be defined as they are used.

This pass was written to provide a fallback to the old behavior, it will
be used for radv to avoid regressions when performing the SPIR-V change.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5282>
2023-08-17 14:58:02 +00:00
Faith Ekstrand
4e2830c9ef nir: Clean up nir_op_is_vec() and its callers
The nir_op_is_vec() helper I added in 842338e2f0 ("nir: Add a
nir_op_is_vec helper") treats nir_op_mov as a vec even though the
semanitcs of the two are different.  In retrospect, this was a mistake
as the previous three fixup commits show.  This commit splits the helper
into two: nir_op_is_vec() and nir_op_is_vec_or_mov() and uses the
appropriate helper at each call site.  Hopefully, this rename will
encurage any future users of these helpers to think about nir_op_mov as
separate from nir_op_vecN and we can avoid these bugs.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24704>
2023-08-16 21:42:30 +00:00
Faith Ekstrand
b64da56b1a nir: s/nir_instr_ssa_def/nir_instr_def/
Generated by sed:

    sed -i -e 's/nir_instr_ssa_def/nir_instr_def/g' src/**/*.h src/**/*.c src/**/*.cpp

Suggested-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24703>
2023-08-15 17:44:27 +00:00
Faith Ekstrand
43be4129d2 nir: s/live_ssa_def/live_def/
Generated mostly with sed:

    sed -i -e 's/live_ssa_def/live_def/g' src/compiler/nir/nir.h src/compiler/nir/*.c

Plus three fixups in various Intel drivers.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24703>
2023-08-15 17:44:27 +00:00
Faith Ekstrand
b781dd6200 nir s/nir_get_ssa_scalar/nir_get_scalar/
Generated with sed:

    sed -i -e 's/nir_get_ssa_scalar/nir_get_scalar/g' src/**/*.h src/**/*.c src/**/*.cpp

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24703>
2023-08-15 17:44:27 +00:00
Faith Ekstrand
65b6ac8aa4 nir: Rename nir_instr_type_ssa_undef to nir_instr_type_undef
We already renamed the type, we just need to rename the enum and the
casting helper functions.

Generated with sed:

    sed -i -e 's/nir_instr_type_ssa_undef/nir_instr_type_undef/g' src/**/*.h src/**/*.c src/**/*.cpp
    sed -i -e 's/nir_instr_as_ssa_undef/nir_instr_as_undef/g' src/**/*.h src/**/*.c src/**/*.cpp

and two tiny whitespace fixups in lima.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24703>
2023-08-15 17:44:27 +00:00
Faith Ekstrand
4695bebc79 nir: Drop nir_dest
Instead, we replace every use of it with nir_def.  Most of this commit
was generated by sed:

   sed -i -e 's/dest.ssa/def/g' src/**/*.h src/**/*.c src/**/*.cpp

A few manual fixups were required in lima and the nir_legacy code.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 21:22:53 +00:00
Faith Ekstrand
6c1d32581a nir: Drop nir_alu_dest
Instead, we replace it directly with nir_def.  We could replace it with
nir_dest but the next commit gets rid of that so this avoids unnecessary
churn.  Most of this commit was generated by sed:

   sed -i -e 's/dest.dest.ssa/def/g' src/**/*.h src/**/*.c src/**/*.cpp

There were a few manual fixups required in the nir_legacy.c and
nir_from_ssa.c as nir_legacy_reg and nir_parallel_copy_entry both have a
similar pattern.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 21:22:53 +00:00
Faith Ekstrand
977999d836 nir: Get rid of nir_dest_is_divergent()
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 21:22:53 +00:00
Faith Ekstrand
9d81f13a75 nir: Get rid of nir_dest_num_components()
We could add a nir_def_num_components() helper but we use
ssa.num_components about 3x as often as nir_dest_num_components() today
so that's a major Coccinelle refactor anyway and this doesn't make it
much worse.  Most of this commit was generated byt the following
semantic patch:

    @@
    expression D;
    @@

    <...
    -nir_dest_num_components(D)
    +D.ssa.num_components
    ...

Some manual fixup was needed, especially in cpp files where Coccinelle
tends to give up the moment it sees any interesting C++.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 21:22:53 +00:00
Faith Ekstrand
80a1836d8b nir: Get rid of nir_dest_bit_size()
We could add a nir_def_bit_size() helper but we use ->bit_size about 3x
as often as nir_dest_bit_size() today so that's a major Coccinelle
refactor anyway and this doesn't make it much worse.  Most of this
commit was generated byt the following semantic patch:

    @@
    expression D;
    @@

    <...
    -nir_dest_bit_size(D)
    +D.ssa.bit_size
    ...

Some manual fixup was needed, especially in cpp files where Coccinelle
tends to give up the moment it sees any interesting C++.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 21:22:53 +00:00
Alyssa Rosenzweig
6b01d6a6b8 nir: Drop nir_dest_init
Unused.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 21:22:53 +00:00
Faith Ekstrand
65cda2c0e1 nir: Drop nir_foreach_dest()
This requires an annoying bit of shuffling into nir_inline_helpers.h but
it's not horrible.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24658>
2023-08-13 17:12:52 +00:00
Faith Ekstrand
6203750d78 nir: nir_foreach_ssa_def() -> nir_foreach_def()
s/nir_foreach_ssa_def/nir_foreach_def/g

followed by

    ninja -C _build clang-format

and a little hand clean-up in nir.c.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24658>
2023-08-13 17:12:52 +00:00
Faith Ekstrand
0ec7b8455e nir: Drop nir_ssa_dest_init_for_type()
Replace it with a new nir_def_init_for_type()

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24658>
2023-08-13 17:12:52 +00:00
Faith Ekstrand
6914272c30 nir: Drop nir_ssa_dest_init()
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24658>
2023-08-13 17:12:52 +00:00
Alyssa Rosenzweig
09d31922de nir: Drop "SSA" from NIR language
Everything is SSA now.

   sed -e 's/nir_ssa_def/nir_def/g' \
       -e 's/nir_ssa_undef/nir_undef/g' \
       -e 's/nir_ssa_scalar/nir_scalar/g' \
       -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \
       -e 's/nir_gather_ssa_types/nir_gather_types/g' \
       -i $(git grep -l nir | grep -v relnotes)

   git mv src/compiler/nir/nir_gather_ssa_types.c \
          src/compiler/nir/nir_gather_types.c

   ninja -C build/ clang-format
   cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \;

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Acked-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:44:41 -04:00
Faith Ekstrand
777d336b1f nir: clang-format src/compiler/nir/*.[ch]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-12 19:27:28 +00:00
Faith Ekstrand
0d9254204b nir: Add a do to the do/while in nir_const_value_t_array()
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-12 19:27:28 +00:00
Faith Ekstrand
bb8f143749 nir: Wrap pass macros in braces
This makes clang-format not mess them up so bad.  It's also probably a
good idea to make sure anything we declare in the macro is properly
scoped.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-12 19:27:28 +00:00
Faith Ekstrand
4d54b423e7 nir: Don't clang-format a couple typedefs
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-12 19:27:28 +00:00
Faith Ekstrand
8579224fe1 nir: Don't clang-format const_value helpers
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-12 19:27:28 +00:00
Faith Ekstrand
a89fb36f6b nir: Re-align a couple enums and add clang-format comments
I actually kinda care about the human-readable formatting of these
enums.  Keep clang-format from messing them up.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-12 19:27:28 +00:00
Mike Blumenkrantz
e9a5da2f4b nir: add a filter cb to lower_io_to_scalar
this is useful for drivers that want to do selective scalarization
of io

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24565>
2023-08-11 09:02:53 +00:00
Mike Blumenkrantz
550f3dc437 nir/lower_io: add a new doubles-only 64bit lowering option
this allows lowering only 64bit float operations for drivers that
support 64bit integers

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24565>
2023-08-11 09:02:53 +00:00
Iago Toral Quiroga
b2de3f71dc nir/lower_tex: use a callback to check sampler return size packing
The lower_tex_packing pass relies on the sampler index to access packing
information, but this is only valid for tex instructions that have sampler
state (so not txf, etc). Instead, let backends provide a callback to inform
the lowering about the packing used with a given texture instruction which
is more flexible.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24537>
2023-08-10 07:10:00 +00:00
Alyssa Rosenzweig
91ae717d0b nir: Remove nir_foreach_def
Leftover.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:30 +00:00
Alyssa Rosenzweig
42ee8a55dd nir: Remove nir_alu_dest::write_mask
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:30 +00:00
Alyssa Rosenzweig
b23d951e96 nir: Drop nir_register
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:29 +00:00
Alyssa Rosenzweig
1c49586a1f nir: Remove nir_{src,dest}::is_ssa
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:29 +00:00
Alyssa Rosenzweig
70c64b5d49 nir: Remove def_is_register
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:29 +00:00
Alyssa Rosenzweig
a8013644a1 nir: Drop nir_alu_src::{negate,abs}
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:28 +00:00
Alyssa Rosenzweig
c512636cf0 nir: Remove reg-only dest manipulation
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:28 +00:00
Alyssa Rosenzweig
ab0d878932 treewide: Remove more is_ssa asserts
Stuff Coccinelle missed.

   sed -i -e '/assert(.*\.is_ssa)/d' $(git grep -l is_ssa)
   sed -i -e '/ASSERT.*\.is_ssa)/d' $(git grep -l is_ssa)

+ a manual fixup to restore the assert for parallel copy lowering.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:28 +00:00
Alyssa Rosenzweig
042a3eca00 nir: Collapse some SSA checks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:28 +00:00
Alyssa Rosenzweig
d559764e7c nir: Remove nir_alu_dest::saturate
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:28 +00:00