Commit graph

396 commits

Author SHA1 Message Date
Konstantin Seurer
60a20bcf3d nir: Stop using instructions for debug info
Annotating ssa defs without affecting compilation is impossible with
debug info instructions since referencing a nir_def from the debug info
instr will add uses.

The old approach also stops worrking if passes reorder instructions.

This patch proposes a solution which should not regress performance just
like the old approach. The difference is that this one allocates a bit
more space for debug info instead of adding a new instruction for it.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33141>
2025-01-30 20:14:01 +00:00
Friedrich Vock
9b1f31a615 nir: Add nir_instr_is_before helper
is_instr_between works similarly but requires special-casing w.r.t
the first instruction in a block for the same effect.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29577>
2025-01-29 04:38:51 +00:00
Friedrich Vock
bb40284f76 nir: Add indirect calls
Used to jump to a function referred to by a runtime pointer.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29577>
2025-01-29 04:38:51 +00:00
Rhys Perry
0eb5f66660 nir/validate: validate ssa dominance by default
This no longer modifies dominance metadata, so enable it by default.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32005>
2025-01-23 23:35:44 +00:00
Marek Olšák
02516ff0f9 nir: remove dead code due to IO being always lowered in st/mesa
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33146>
2025-01-22 02:15:04 +00:00
Marek Olšák
ff6e3e9f76 nir: add next_stage param to nir_slot_is_varying & nir_remove_sysval_output
The result of nir_slot_is_varying depends on what the next shader stage is,
and nir_remove_sysval_output uses it.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32855>
2025-01-16 16:28:15 +00:00
Georg Lehmann
15d754fefa nir: add load_front_face_fsign
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32791>
2024-12-30 22:31:34 +00:00
Rhys Perry
5368569d06 nir: make load_helper_invocation non-reorderable
This can't be moved to after demote, so it's not reorderable.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32512>
2024-12-11 14:47:12 +00:00
Caterina Shablia
9d5ba87ca1 Revert "nir: lower INSTANCE_{ID,INDEX} to an offset load_instance_{index,id} respectively"
This reverts commit a5bcf566a9.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32332>
2024-11-28 07:53:01 +00:00
Alyssa Rosenzweig
12cc22af4c nir: add nir_remove_entrypoints helper
opposite of nir_remove_non_entrypoint. this operation comes up with
precompiling.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32382>
2024-11-27 20:02:05 +00:00
Alyssa Rosenzweig
5555769102 nir: add workgroup size to functions
for cl kernel libraries with many entrypoints. spirv can represent, nir should
be able to as well.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32382>
2024-11-27 20:02:05 +00:00
Rhys Perry
4c6fdb113f nir: fix return value of nir_instr_move for some cases
This fixes a potential issue where nir_opt_move_discards_to_top would
always return progress.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Fixes: f97fb1fa55 ("nir: Add a nir_instr_move helper")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32145>
2024-11-21 14:50:44 +00:00
Georg Lehmann
4299809321 nir: return def for debug info in nir_instr_def
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32235>
2024-11-20 23:54:04 +00:00
Caterina Shablia
a5bcf566a9 nir: lower INSTANCE_{ID,INDEX} to an offset load_instance_{index,id} respectively
If the hardware does not support INSTANCE_INDEX natively, it will be
lowered to load_instance_id + base_instance. Otherwise, INSTANCE_ID
will be lowered to load_instance_index - base_instance.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32158>
2024-11-19 09:18:47 +00:00
Georg Lehmann
8f094a7762 nir: handle fmul(a,a)/ffma(a,a,b) in nir_def_all_uses_ignore_sign_bit
Foz-DB Navi31:
Totals from 436 (0.55% of 79395) affected shaders:
Instrs: 808917 -> 805868 (-0.38%)
CodeSize: 4269056 -> 4246512 (-0.53%)
Latency: 5827077 -> 5819815 (-0.12%); split: -0.13%, +0.00%
InvThroughput: 625482 -> 622959 (-0.40%); split: -0.41%, +0.00%
SClause: 21797 -> 21756 (-0.19%); split: -0.23%, +0.04%
Copies: 48502 -> 48505 (+0.01%); split: -0.04%, +0.05%
VALU: 481686 -> 479074 (-0.54%); split: -0.54%, +0.00%
SALU: 76699 -> 76700 (+0.00%)

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31844>
2024-11-12 18:03:57 +00:00
Georg Lehmann
34f41abe24 nir: add nir_def_all_uses_ignore_sign_bit
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31844>
2024-11-12 18:03:57 +00:00
Georg Lehmann
2f830f9b94 nir: add SYSTEM_VALUE_PIXEL_COORD
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31864>
2024-11-04 12:34:30 +00:00
Daniel Schürmann
1a55d6c23b nir/divergence: Introduce and set nir_def::loop_invariant
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30787>
2024-10-24 10:06:17 +00:00
Daniel Schürmann
c25c63ebc0 nir/divergence: separately indicate whether loops have divergent continues or breaks
bool nir_loop_is_divergent(nir_loop *)
 replaces the previous loop->divergent indicator.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30787>
2024-10-24 10:06:17 +00:00
Mary Guillemard
84d57e1fb1 nir: Move atomic_op_to_alu to common code
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27776>
2024-10-21 21:47:44 +00:00
Timothy Arceri
7c5b21c032 glsl: add support for converting global instructions to NIR
NIR doesn't really support global instructions such as global val
initilisation. So here we add functionality to glsl_to_nir() to
put these instructions into a temporary function that will be
later inlined into main.

We give the function a name starting with gl_mesa_tmp_ as functions
starting with gl_ are reserved and will not have any clashes with
user functions, we finish the name with the blake3 of the shader
source to avoid conflicts with multiple shaders attached to a single
stage.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137>
2024-09-25 09:39:43 +00:00
Rhys Perry
97f4250a7c nir: skip opt_loop_peel_initial_break if continue block only has phis
Doing that optimization wouldn't do anything useful in this case.

nir_block_has_non_copy() is used by opt_loop_peel_initial_break().

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31002>
2024-09-12 23:36:58 +00:00
Rhys Perry
4f44a944bb nir/opt_if: fix fighting between split_alu_of_phi and peel_initial_break
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 6b4b044739 ("nir/opt_loop: add loop peeling optimization")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11822
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31002>
2024-09-12 23:36:57 +00:00
Caio Oliveira
1e7f1c2039 nir: Allow Mesh/Task to use implicit LOD when DERIVATIVE_GROUP is set
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30956>
2024-09-10 18:22:42 +00:00
Caio Oliveira
74be809237 compiler: Allow derivative_group to be used for all stages in shader_info
These will now also be used by stages that have workgroups.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30950>
2024-09-03 20:03:18 +00:00
Alyssa Rosenzweig
afc7557cb6 nir,agx: make block image store an image() intrinsic
so we can do a bindless version

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30981>
2024-09-02 23:27:14 +00:00
Daniel Schürmann
50d416fe89 nir: add nir_block *nir_src_get_block(src) helper
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7710>
2024-08-29 09:42:55 +00:00
Konstantin Seurer
ce24486ee4 nir: Introduce nir_debug_info_instr
Adds a new instruction type that stores metadata that might be useful
for debugging purposes. Passes must ignore these instructions when
making decisions.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18903>
2024-08-25 10:26:33 +00:00
Karol Herbst
14ea102175 nir: add load_global_size intrinsic
There is no need to compute it in the shader as the result is known at
runtime already.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Tested-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30467>
2024-08-01 17:43:42 +00:00
Timothy Arceri
017770ff14 nir: add nir_tex_src_{sampler,texture}_deref_intrinsic
To be used as a placeholder until after function inlining so we can
replace function params with bindless handles if needed.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30315>
2024-07-29 00:06:10 +00:00
Marek Olšák
b2d32ae246 nir: add nir_intrinsic_load_per_primitive_input, split from io_semantics flag
Instead of having 1 bit in nir_io_semantics indicating a per-primitive
FS input, add a dedicated intrinsic for it.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29895>
2024-07-23 16:13:16 +00:00
M Henning
e506955056 nir: Handle texop_*_nv in nir_tex_instr_is_query
Fixes: aa1f00cf ("nir/gather_info: handle uses_fbfetch_output for texture operations")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11505
Tested-by: Thomas H.P. Andersen <phomes@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30166>
2024-07-13 15:36:29 +00:00
Georg Lehmann
99372c1ed7 nir: add ford, funord, fneo, fequ, fltu, fgeu
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29467>
2024-06-27 08:12:29 +00:00
Alyssa Rosenzweig
15257b65c6 treewide: use nir_metadata_control_flow
Via Coccinelle patch:

    @@
    @@

    -nir_metadata_block_index | nir_metadata_dominance
    +nir_metadata_control_flow

...plus some manual fixups for call sites missed by coccinelle.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Juan A. Suarez Romero <jasuarez@igalia.com> [broadcom]
Acked-by: Vasily Khoruzhick <anarsoul@gmail.com> [lima]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29745>
2024-06-17 16:28:14 -04:00
Alyssa Rosenzweig
9a8cb81f61 nir/tex_instr_result_size: handle subpass_ms
I hit this and don't see any reason it shouldn't work

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29249>
2024-05-16 18:09:39 -04:00
Alyssa Rosenzweig
2912f531a7 nir: add texops for AGX border colour emulation
AGX has limited border colour hardware. To support full
customBorderColorWithoutFormat semantics, we're forced to emulate in shaders at
a substantial performance penalty. Actually, that's needed just to pass CTS
because of other hardware issues stacking on top of each others... Hooray!

Add the texops we need to facilitate efficient custom border colour lowering.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29179>
2024-05-14 04:57:24 +00:00
Karol Herbst
25d697ef25 nir: add SYSTEM_VALUE_BASE_WORKGROUP_ID
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26800>
2024-04-24 20:18:48 +00:00
Timothy Arceri
baf13bff29 nir: add subroutine fields to nir_function
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28538>
2024-04-06 05:56:04 +00:00
Sagar Ghuge
30ead72e80 nir: Allow nir_texop_tg4 in implicit derivative
This allow us to invoke the quad helper.

v2: (Georg)
- Add check for is_gather_implicit_lod

Fixes: 48158636bf ("nir: add is_gather_implicit_lod")
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27447>
2024-02-27 00:22:46 +00:00
Bas Nieuwenhuizen
c7b2ac3377 radv: Remove ray_launch_size_addr_amd system value.
Not used anymore, so clean it up.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27664>
2024-02-17 11:08:16 +00:00
Sagar Ghuge
c984d6e2fc nir: Drop intel specific lowering code
In previous patches, we have moved the Intel specific lowering code in
brw_nir_lower_texture file. We can go ahead and drop the Intel specific
texture source too.

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27458>
2024-02-12 21:25:48 +00:00
Ian Romanick
c8ba2bc2f0 nir: Pack texture LOD and array index to a single 32-bit value
v2: Fix clamped_ai calculation in nir_lower_tex.c. Add
nir_tex_src_combined_lod_and_array_index_intel to
print_tex_instr. Suggested by Sagar.

Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27305>
2024-02-02 02:39:10 +00:00
Faith Ekstrand
82fe981e35 nir,spirv: Add support for SPV_NV_shader_sm_builtins
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27154>
2024-01-18 20:20:06 +00:00
Bas Nieuwenhuizen
bc99b73d70 nir: Add nir_static_workgroup_size helper.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26679>
2023-12-20 09:15:45 +00:00
Marek Olšák
27a9ddad28 nir: return progress from nir_remove_sysval_output
Changing IO semantics doesn't affect the SSA structure.

Same as nir_remove_varying.

Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26274>
2023-11-24 15:37:24 +00:00
Marek Olšák
32ee6376ad nir: add lowering from FS LAYER input to LAYER_ID sysval
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26274>
2023-11-24 15:37:24 +00:00
Alyssa Rosenzweig
6d73f62e42 nir: Simplify nir_alu_instr_channel_used definition
Deduplicates the "get # of channels" logic which was the same between the
helpers.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
2023-11-17 09:36:08 -04:00
Karol Herbst
d17dc3e9cd nir: Stop assuming glsl_get_length() returns 0 for vectors
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580>
2023-11-02 20:28:46 +00:00
Alyssa Rosenzweig
23bea25207 nir: Add nir_remove_non_exported
For libraries.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25498>
2023-11-02 11:37:46 +00:00
Alyssa Rosenzweig
6014f745d5 nir,vtn: Add exported bool to nir_function
For optimizing libraries.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25498>
2023-11-02 11:37:46 +00:00