Commit graph

1588 commits

Author SHA1 Message Date
Marek Olšák
adb17a8609 nir: move nir_recompute_io_bases into its own file
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35760>
2025-06-26 18:20:53 +00:00
Marek Olšák
97743980ce nir: remove unused nir_force_mediump_io & nir_unpack_16bit_varying_slots
I think I added these.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35760>
2025-06-26 18:20:52 +00:00
Marek Olšák
5bd3e0c08c nir: move nir_assign_var_locations to freedreno (its only use)
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35760>
2025-06-26 18:20:52 +00:00
Marek Olšák
12df9b3def nir: rename nir_vectorize_tess_levels -> nir_lower_tess_level_array_vars_to_vec
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35760>
2025-06-26 18:20:50 +00:00
Marek Olšák
2aa94caf82 nir: rename nir_lower_io_to_vector -> nir_opt_vectorize_io_vars
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35760>
2025-06-26 18:20:50 +00:00
Marek Olšák
439d805291 nir: rename nir_lower_io_to_scalar_early -> nir_lower_io_vars_to_scalar
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35760>
2025-06-26 18:20:49 +00:00
Georg Lehmann
7de352e99e nir,radv: add an option to not move 8/16bit vecs
ACO will overestimate the register demand of the sources, so we don't
want to create the vector later.

Foz-DB Navi48:
Totals from 240 (0.30% of 80265) affected shaders:
MaxWaves: 6429 -> 6435 (+0.09%)
Instrs: 3406069 -> 3406646 (+0.02%); split: -0.01%, +0.03%
CodeSize: 18231596 -> 18233288 (+0.01%); split: -0.01%, +0.02%
VGPRs: 14768 -> 14732 (-0.24%)
Latency: 18981274 -> 18979170 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 4247331 -> 4246634 (-0.02%); split: -0.02%, +0.01%
VClause: 85453 -> 85458 (+0.01%); split: -0.01%, +0.01%
Copies: 262046 -> 261971 (-0.03%); split: -0.06%, +0.03%
PreVGPRs: 10899 -> 10775 (-1.14%)
VALU: 1923441 -> 1923485 (+0.00%); split: -0.01%, +0.01%
SALU: 457983 -> 457982 (-0.00%)
VOPD: 4980 -> 4861 (-2.39%); split: +0.48%, -2.87%

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35729>
2025-06-26 09:29:43 +00:00
Emma Anholt
bc8994cb48 nir: Add a pass to reassociate multiplication of mat*mat*vec.
The typical case of mat4*mat4*vec4 is 80 scalar multiplications, but
mat4*(mat4*vec4) is only 32.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35622>
2025-06-23 17:49:51 +00:00
Faith Ekstrand
bb4c5edda1 nir: Add more tex_src helpers
This adds a variant of nir_steal_tex_src() which is for derefs as well
as versions that just return the source without removing it.

Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35623>
2025-06-23 14:25:30 +00:00
Faith Ekstrand
2b40fa09f2 nir: Move nir_steal_tex_src() to nir.h
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35623>
2025-06-23 14:25:30 +00:00
Faith Ekstrand
9f9cde04ec nir: Add a new load_input_attachment_coord intrinsic
This hoists all the annoyance of figuring out the current pixel's input
attachment coordinates to the driver.  The pass still deals with all the
annoyance of turning an image instruciton into a texture instruction but
it gives the driver more control over the position.  For most drivers,
this will be something like ivec3(int(gl_FragCoord.xy), gl_Layer) or
similar, some drivers need something more nuanced.  Turnip, for
instance, needs unscaled coordinates for some attachments and NVK
doesn't really want gl_Layer or gl_ViewIndex for the layer.  It's better
to just have a new system value that drivers can make what they want.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35551>
2025-06-19 02:14:04 +00:00
Marek Olšák
fa2e7c3dfd nir: return progress from nir_group_loads, nir_inline_uniforms
so that NIR_PASS is usable

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35315>
2025-06-12 19:35:37 +00:00
Marek Olšák
b636e5ca66 nir: add nir_clear_mediump_io_flag
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35315>
2025-06-12 19:35:36 +00:00
Marek Olšák
bf2ed20eb9 nir: remove unused nir_io_semantics::invariant
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Acked-by: Alyssa on IRC
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35256>
2025-06-02 23:08:58 +00:00
Kenneth Graunke
deb1d47155 nir: Add a new optimization for acquire/release atomics & barriers
Some shaders contain back-to-back atomic accesses in SPIR-V with
AcquireRelease semantics.  In NIR, we translate these to a release
memory barrier, the atomic, then an acquire memory barrier.

This results in a lot of unnecessary memory barriers in the middle
of the sequence of atomics:

   0. Release barrier
   1. Atomic
   2. Acquire barrier
   3. Release barrier
   4. Atomic
   5. Acquire barrier
   6. Release barrier
   7. Atomic
   8. Acquire barrier

In the absence of loads/stores, and when the atomic destinations are
unused, these barriers in-between atomics shouldn't be required.

This optimization pass would drop them (lines 2-3 and 5-6 above) while
leaving the first and last barriers (0 and 8), so the sequence remains
synchronized against other access elsewhere in the program.

One common example where this occurs is a sequence of min and max
atomics to clamp a certain memory location's value within a range.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33504>
2025-05-16 00:29:13 +00:00
Marek Olšák
deda05e2b7 nir: move nir_lower_color_inputs into radeonsi
it's the only user

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34492>
2025-05-14 20:19:17 +00:00
Konstantin Seurer
5926b63f66 nir: Print struct type declarations
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26267>
2025-05-12 18:28:50 +00:00
Karol Herbst
f0fa2209a8 nir: add nir_opt_algebraic_integer_promotion
This handles basic operations where clang promotes integers to 32 bits
according to the C99 spec in OpenCL C source code.

This is its own opt_algerbraic pass, because we don't wanna fight with
nir_lower_bit_size.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34641>
2025-05-12 09:29:20 +00:00
Daniel Schürmann
c59356e6a5 nir: add option to move terminate{_if} out of loops
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33479>
2025-05-09 17:20:29 +00:00
Rhys Perry
ddef4bddf8 ac/nir: round components when lowering 8/16-bit loads to 32-bit
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34162>
2025-05-08 13:30:50 +00:00
Alyssa Rosenzweig
5788770d91 nir: add nir_lower_default_point_size pass
this is useful across drivers for maint5 semantics on mobile hw.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34762>
2025-05-06 17:07:00 +00:00
Marek Olšák
f15399af0f nir: add gathering passes that gather which inputs affect specific outputs
The first pass computes which shader instructions contribute to each
output. It can be used to query how data flows within shaders towards
outputs.

The second pass computes which shader input components and which types of
memory loads are used to compute shader outputs.

The third pass uses the second pass to gather which input components are
used to compute pos and clip dist outputs, which input components are used
to compute all other outputs, and which input components are used to
compute both. This will be used by compaction in nir_opt_varyings for
drivers that split TES into a separate position cull shader and varying
shader to make it less likely that the same vec4 inputs are needed in both.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32262>
2025-04-23 17:47:37 +00:00
Ella Stanforth
b38c4e8982 nir/alpha_to_coverage: Add an intrinsic for better dithering
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33942>
2025-04-23 09:03:41 +00:00
Ella Stanforth
d3aedbfe9d asahi/lib: Move alpha_to_one and alpha_to_coverage lowering to common code.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33942>
2025-04-23 09:03:41 +00:00
Alyssa Rosenzweig
f1aeb46a34 nir: factor out nir_verts_in_output_prim helper
very useful for geometry shader lowering code.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34638>
2025-04-22 12:47:54 +00:00
Caio Oliveira
33295b2249 spirv, nir: Allow non-Aliased workgroup memory blocks
Allocate space for the aliased region first, then allocate the
non-Aliased blocks in sequence after that.

SPV_KHR_workgroup_memory_explicit_layout extension added support for
having Blocks of workgroup (shared) memory, which include layout
decoration.  For that extension all such blocks must be decorated with
Aliased.

SPV_KHR_untyped_pointers extension lifts that requirement, allowing
blocks that don't alias in workgroup memory.  They are still explicitly
laid out.

The motivation is that untyped pointers provide a different
mechanism to obtain the same effect as the Aliased blocks.  Instead of
having two Aliased variables with different types, have a single
variable and use an untyped pointer with a different type to access it.

This patch is a preparation for supporting untyped pointers.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34139>
2025-04-17 19:13:18 +00:00
Alyssa Rosenzweig
63eb27d166 nir: add sampler LOD bias lowering
this is a cleaned up version of the lowering originally written for asahi, moved
to common code so it can be shared with an upcoming Vulkan implementation (not
honeykrisp).

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34507>
2025-04-15 14:10:50 +00:00
Alyssa Rosenzweig
2e15b42eec nir: unvendor lod_bias(_agx)
this will be useful for other backends.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34507>
2025-04-15 14:10:49 +00:00
Faith Ekstrand
a3935c7aa2 nak,nir: Generalize nak_nir_split_64bit_conversions and move it to NIR
This pass was originally based on a similar pass from Intel but it's
grown support for some fancy stuff like fp64 -> fp16 conversion
splitting with proper rounding.

Reviewed-by: Mel Henning <mhenning@darkrefraction.com>
Reviewed-by: Benjamin Lee <benjamin.lee@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34126>
2025-03-29 03:02:17 +00:00
Lionel Landwerlin
772beb0ebf nir: add support for lowering non uniform texture offsets
Intel HW only has support for non-uniform offsets for TG4 operations.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33138>
2025-03-29 02:15:18 +00:00
Daniel Schürmann
a4cffa91b8 nir: remove nir_lower_discard_if_to_cf option
Since removing nir_intrinsic_discard{_if} it has no purpose anymore.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33492>
2025-03-07 15:44:49 +00:00
Konstantin Seurer
a04b5ebd3c nir/sweep: Fix handling instructions with debug info
When debug information is present, the nir_instr pointer is not the
start of the allocation.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28613>
2025-03-04 18:42:48 +00:00
Faith Ekstrand
a65009e808 nir: Add a nir_opt_tex_skip_helpers optimization
Arm and NVIDIA hardware both have this as a bit you can set on the
texture instruction so we may as well have a shared pass for it.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33402>
2025-03-01 08:44:15 +00:00
Faith Ekstrand
7ac6ec2ceb nir: Add a get_io_index_src() helper
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33402>
2025-03-01 08:44:15 +00:00
Alyssa Rosenzweig
593308a685 nir: eliminate nir_metadata_preserve
Everybody uses the wrapper now.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33722>
2025-02-26 15:19:53 +00:00
Alyssa Rosenzweig
24d088104f nir,nak: update comments referencing nir_metadata_preserve
in prep for removing this method.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33722>
2025-02-26 15:19:53 +00:00
Alyssa Rosenzweig
469b8bbf3c nir: add nir_progress/nir_no_progress helpers
These will replace nir_metadata_preserve as more ergonomic replacements that
convey a notion of impl progress instead of simply updating metadata.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33722>
2025-02-26 15:19:52 +00:00
Georg Lehmann
dd1a7f0e8c nir: add a pass to optimize phis to 1bit
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33498>
2025-02-25 20:38:08 +00:00
Timur Kristóf
65139305e2 nir: Don't use deprecated NIR_PASS_V macro anymore.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33609>
2025-02-22 08:54:16 +01:00
Timur Kristóf
7222bb397d nir: Add bool return value to nir_fixup_deref_types.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33609>
2025-02-22 08:54:16 +01:00
Timur Kristóf
2fa7711a2b nir: Add bool return value to nir_fixup_deref_modes.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33609>
2025-02-22 08:54:16 +01:00
Timur Kristóf
4f744998ef nir: Add comment to indicate that NIR_PASS_V is deprecated.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33609>
2025-02-22 08:54:16 +01:00
Georg Lehmann
f26069fdd9 nir: replace nir_opt_conditional_discard with nir_opt_peephole_select
Foz-DB Navi21:
Totals from 118 (0.15% of 79377) affected shaders:
Instrs: 208001 -> 207355 (-0.31%); split: -0.33%, +0.01%
CodeSize: 1080428 -> 1078432 (-0.18%); split: -0.20%, +0.02%
SpillSGPRs: 202 -> 211 (+4.46%)
Latency: 1923508 -> 1919093 (-0.23%); split: -0.62%, +0.39%
InvThroughput: 407475 -> 407081 (-0.10%); split: -0.12%, +0.02%
SClause: 7050 -> 7033 (-0.24%); split: -0.31%, +0.07%
Copies: 12156 -> 11821 (-2.76%); split: -3.04%, +0.28%
PreSGPRs: 8198 -> 8331 (+1.62%); split: -0.02%, +1.65%
PreVGPRs: 7628 -> 7528 (-1.31%)
VALU: 155747 -> 155657 (-0.06%); split: -0.06%, +0.00%
SALU: 18295 -> 17782 (-2.80%); split: -2.98%, +0.18%
SMEM: 10521 -> 10519 (-0.02%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33590>
2025-02-20 21:59:17 +00:00
Georg Lehmann
cfee9e1d9f nir/peephole_select: add option to allow discard without ~0 limit
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33590>
2025-02-20 21:59:16 +00:00
Georg Lehmann
ca8147edbe nir/peephole_select: add options struct
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33590>
2025-02-20 21:59:16 +00:00
Christian Gmeiner
09fa418b7d nir: Add bool return value to nir_lower_texcoord_replace(..)
Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33621>
2025-02-19 21:16:05 +00:00
Christian Gmeiner
13e750aabe nir: Add bool return value to nir_lower_clip_halfz(..)
Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33621>
2025-02-19 21:16:05 +00:00
Lionel Landwerlin
e22ab01dc7 nir: add options to lower only some image atomics to global
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33616>
2025-02-19 18:41:05 +00:00
Daniel Schürmann
01f2d494d0 nir: make divergence information metadata
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30814>
2025-02-13 10:08:43 +00:00
Timur Kristóf
89cc4d9807 nir: Remove struct keyword from nir.h where possible.
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33439>
2025-02-12 23:20:15 +01:00