Commit graph

11540 commits

Author SHA1 Message Date
Faith Ekstrand
11b6cd2f2c nir,pan: Rework the pafrost tile load intrinsic
Instead of making it explicitly about outputs, this switchies it to
being a NIR version of LD_TILE.  It means we have to do a bit of work in
NIR and add a builder helper but the end result is something much more
versatile.

Reviewed-by: Christoph Pillmayer <christoph.pillmayer@arm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39367>
2026-01-19 21:33:13 +00:00
Faith Ekstrand
4189865347 nir: panfrost tile loads are always divergent
Each lane refers to a different pixel.

Cc: mesa-stable
Reviewed-by: Christoph Pillmayer <christoph.pillmayer@arm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39367>
2026-01-19 21:33:13 +00:00
Georg Lehmann
70a951c3f3 nir/search: allow inexact patterns if denorms have to be flushed
Patterns should ensure that they flush denorms with fcanonicalize.

Removing in between denorm flushing when fusing operations is explicitly
allowed unless those optimizations are generally disallowed by other
floating point math control flags.

Foz-DB Navi21:
Totals from 291 (0.35% of 82377) affected shaders:
Instrs: 138347 -> 137773 (-0.41%)
CodeSize: 751460 -> 748516 (-0.39%)
Latency: 1686466 -> 1686226 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 270847 -> 269963 (-0.33%)
VClause: 2023 -> 2022 (-0.05%)
SClause: 5271 -> 5260 (-0.21%); split: -0.25%, +0.04%
Copies: 8929 -> 8912 (-0.19%)
VALU: 87108 -> 86552 (-0.64%)
SALU: 23460 -> 23443 (-0.07%)

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39180>
2026-01-19 16:11:29 +00:00
Georg Lehmann
442daeb54a nir/opt_algebraic: use fcanonicalize
Mostly optimizations, some minor fixes but I don't
think they are worth backporting.

Foz-DB Navi21:
Totals from 7570 (9.21% of 82151) affected shaders:
MaxWaves: 204288 -> 204476 (+0.09%); split: +0.09%, -0.00%
Instrs: 4511439 -> 4500261 (-0.25%); split: -0.25%, +0.00%
CodeSize: 23727088 -> 23644388 (-0.35%); split: -0.35%, +0.00%
VGPRs: 290944 -> 290616 (-0.11%); split: -0.12%, +0.01%
SpillSGPRs: 1256 -> 1251 (-0.40%)
Latency: 16738072 -> 16726717 (-0.07%); split: -0.10%, +0.04%
InvThroughput: 3736856 -> 3716631 (-0.54%); split: -0.55%, +0.01%
VClause: 66150 -> 66156 (+0.01%); split: -0.05%, +0.06%
SClause: 93644 -> 93631 (-0.01%); split: -0.02%, +0.01%
Copies: 448816 -> 458584 (+2.18%); split: -0.05%, +2.22%
Branches: 139817 -> 139775 (-0.03%); split: -0.03%, +0.00%
PreSGPRs: 321922 -> 321900 (-0.01%); split: -0.01%, +0.00%
PreVGPRs: 239709 -> 238856 (-0.36%); split: -0.39%, +0.03%
VALU: 2595164 -> 2584250 (-0.42%); split: -0.43%, +0.01%
SALU: 839038 -> 838965 (-0.01%); split: -0.02%, +0.01%
VMEM: 137584 -> 137583 (-0.00%)

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39180>
2026-01-19 16:11:29 +00:00
Rhys Perry
625afb0d29 nir: add fcanonicalize
v2(Georg Lehmann):
Always remove fcanonicalize if denorms must be neither flushed nor preserved.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39180>
2026-01-19 16:11:29 +00:00
Georg Lehmann
43d998df84 nir: document that both input and output denorms have to be flushed
This allows us to remove a * 1.0 or a - 0.0 if is_only_used_as_float.
We already rely on that.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39180>
2026-01-19 16:11:28 +00:00
Georg Lehmann
d7e88c0ccd nir/constant_expressions: flush input denorms if denorms have to be flushed
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39180>
2026-01-19 16:11:28 +00:00
Georg Lehmann
7e93aebbec nir/constant_expressions: don't avoid unused source variable warnings
The only use case for this was fddx/fddy and they are no longer alu
for good reasons. For current and future alu, unused sources don't make sense.

And if you really want it, you can still explicitly cast the variable to void.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39180>
2026-01-19 16:11:28 +00:00
Eric Engestrom
30c2e6dbf2 nir/meson: drop redundant --build-tests in favour of just checking if --out-tests is set
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39350>
2026-01-16 16:55:21 +00:00
Eric Engestrom
246095da49 nir/meson: only try to generate the nir_opt_algebraic tests when requested
Anything listed in a meson target's `output` is expected to exist once
the command has run. If it's missing, meson/ninja will run the command
again to try to generate it, resulting in a ton of files getting
re-generated/re-compiled for no reason.

Fixes: 4c30c44b75 ("nir: Generate unit tests for nir_opt_algebraic")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/14667
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39350>
2026-01-16 16:55:21 +00:00
Michal Vanis
75d95cb355 glsl: replace gl ctx direct access
Replaces GL API context access with an abstraction as
to allow for https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21343

Signed-off-by: Michal Vanis <mik@vanis.sh>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38981>
2026-01-15 22:21:05 +00:00
Konstantin Seurer
4c30c44b75 nir: Generate unit tests for nir_opt_algebraic
This catches a number of bugs in the current NIR algebraic optimizations
or opcodes implementations (as fixed in this series, or documented in the
XFAIL tests), and should prevent many future bugs from landing.

This required bumping the test timeout, because s390x is very slow to
emulate in CI.

Closes: #3338
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:43 +00:00
Emma Anholt
df215cc3cd nir/opt_algebraic_tests: Mark patterns as unsupported or xfails.
This way as a pattern author/editor you can immediately see whether it's
getting test coverage and if there are known issues with the pattern.
This will also give us clear outcomes from testing as we fix failing
patterns.

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:43 +00:00
Konstantin Seurer
f5864ed408 nir/opt_algebraic_tests: Add an option for generating unit tests
It only emits tests for exact patterns which do not use instructions
that drop precision by design.

Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:40 +00:00
Emma Anholt
14fafebc1a nir/algebraic: Fix typo in error message print.
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:40 +00:00
Konstantin Seurer
363b2655b6 nir: Add a unit test base class for algebraic patterns
nir_algebraic_pattern_test can validate shaders with the following
structure:

%0 = @provide(base = 0)
...
%N = @provide(base = input_count)

// multiple equivalent expressions
a = ...
b = ...

valid = ieq(a, b)
@use(valid)

Expressions are evaluated by emulating the shader using
nir_eval_const_opcode.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:37 +00:00
Emma Anholt
8ebe630a13 nir/search_helpers: Avoid UB in is_2x_16_bits()/is_neg2x_16_bits().
Same trick we do for nir_imul evaluation -- do the multiply in unsigned to
get defined behavior from C.  Fixes UBSan failures with
nir_opt_algebraic_pattern_tests.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:37 +00:00
Emma Anholt
7dbd170a7f nir/opcodes: Cast isub/iadd3's args to uint to avoid UB integer underflow.
Same treatment as iadd itself got.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:37 +00:00
Emma Anholt
8529aaa399 nir/opcodes: Avoid technical UB left shifting ints.
We all know that (int)0xff << 24 is fine, but UBSan doesn't like it.
These were triggered by nir_opt_algebraic_pattern_tests.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:37 +00:00
Konstantin Seurer
079d416e99 nir: Fix the types of udot_.*_uadd_sat
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:37 +00:00
Konstantin Seurer
38d0bd7dd3 nir: Add an assert_eq intrinsic for testing nir_opt_algebraic
During the test this will compares both sources and fails the test if they
are not equal.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:37 +00:00
Emma Anholt
ed8676dc28 nir: Rename the unit_test_*_amd intrinics to be un-vendored.
We'll reuse these from the nir_opt_algebraic_pattern_test.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:37 +00:00
Emma Anholt
0dc3276a26 nir: Define udot_2x16_uadd_sat to have UB according to the SPIRV spec.
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:32 +00:00
Emma Anholt
f638eb1b85 nir: Define extract/insert_i8 and friends to be UB if the shift is too large.
These opcodes are generated inside NIR algebraic when the shift is
constant, but this will help us do automated algebraic pattern testing
with arbitrary inputs that are unaware of the opcode's restrictions.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:32 +00:00
Emma Anholt
045ae759a5 nir: Specify f2i/f2u as undefined if the float is out of range of the int.
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:32 +00:00
Emma Anholt
94f0e2dbaf nir/constant_expressions: Set the poison flag during i/ubitfield_extract.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:32 +00:00
Emma Anholt
b375da7f2a nir: Let nir_eval_const_opcode() return a poison mask in case of UB.
This is unused by any callers currently, but will be useful for nir
algebraic pattern testing, and as a way to turn our comments in
nir_opcodes.py into actual C code.  For now, always returns false.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:32 +00:00
Emma Anholt
f6008645f6 nir: Fix constant evaluation of non-32-bit bitfield_extract.
Caught by nir_opt_algebraic_pattern_tests.

Fixes: 226b0e28db ("nir: generalize bitfield insert/extract sizes")
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39076>
2026-01-15 19:09:29 +00:00
Emma Anholt
5bd669868f nir: Add a note on how load_sample_pos_from_id works.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38911>
2026-01-15 07:52:14 +00:00
Aitor Camacho
fcf53988c4 nir/opt_varyings: Support implementations that cannot compact 16-bits
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Add nir_io_compact_to_higher_16 flag so that the pass knows if it can
compact 16-bit varyings into the higher 16 bits of a 32-bit varying.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Aitor Camacho <aitor@lunarg.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38994>
2026-01-14 20:44:41 +00:00
Georg Lehmann
fdfe3acdf0 nir/constant_expression: remove fquantize2f16 denorm special case
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Unnessecary, as any fp32 denorm would be 0 here already.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39266>
2026-01-14 17:05:24 +00:00
Georg Lehmann
631a7ef92a nir: make fquantize2f16 32bit only
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39266>
2026-01-14 17:05:24 +00:00
Natalie Vock
cc81c7de23 nir,aco: Clean up useless lowering of sbt_base_amd
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580>
2026-01-14 14:19:07 +00:00
Natalie Vock
0a1911b220 radv,aco: Use function call structure for RT programs
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580>
2026-01-14 14:19:07 +00:00
Natalie Vock
c5d796c902 radv/rt: Use function call structure in NIR lowering
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580>
2026-01-14 14:19:06 +00:00
Natalie Vock
9d2c3c3db2 nir/intrinsics: Add incoming/outgoing payload load/store instructions
With RT function calls, these are going to get lowered to:
- load/store_param (incoming payload)
- load/store_var (outgoing payload)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580>
2026-01-14 14:19:05 +00:00
Alyssa Rosenzweig
41cdc548ee nir/builder: infer txf_ms/txl/txb opcodes
I'm not convinced these really should be separate opcodes at all in NIR, but
that's not what this patch is about. Here we just infer the opcodes in the
texture builder to allow simplified usage.

This lets us drop nir_txl() & nir_txb() helpers in favour of nir_tex(.lod/bias)
which is more normalized. We could also drop nir_txf_ms in favour of nir_txf but
that affects more callsites and is not obviously a win (unlike nir_txl which is
used once and nir_txb which is unused).

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39271>
2026-01-14 08:18:15 +00:00
Jesse Natalie
7b82b52fd7 nir: Suppress 'potentially uninitialized local pointer variable used' warning
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Mel Henning <mhenning@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39181>
2026-01-13 23:31:28 +00:00
John Anthony
50682ec22c pan: Use correct architecture name for v12+
The official name for the architecture after Valhall is 'Arm 5th
Gen'. In code we can use 'FIFTHGEN' or 'fifthgen', while in
documentation and printed output we should use 'Arm 5th Gen' or '5th
Gen'.

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39267>
2026-01-13 13:28:34 +01:00
Lars-Ivar Hesselberg Simonsen
ce3e13774a nir: Add channels to pan texel_buf intrinsics
Rather than loading a single 64bit channel with
load_texel_buf_index_address_pan, load three channels of 32bit each. The
last channel is required by the next commit.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38490>
2026-01-13 10:00:58 +01:00
Lars-Ivar Hesselberg Simonsen
46b44cf941 glsl/nir: Add texture_buffers to shader info
While analyzing glsl shaders, keep a bitmask of texture buffers. This
information is needed by panfrost.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38490>
2026-01-13 10:00:58 +01:00
Faith Ekstrand
6fc1030e4f nir: Add some new panfrost fragment shader intrinsics
Reviewed-by: Christoph Pillmayer <christoph.pillmayer@arm.com>
Reviewed-by: Lars-Ivar Hesselberg Simonsen <lars-ivar.simonsen@arm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39244>
2026-01-12 18:14:43 +00:00
Lionel Landwerlin
6d19b898e7 anv/brw: prep work for SIMD32 ray queries
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36181>
2026-01-12 12:19:21 +00:00
Konstantin Seurer
f156743b0f spirv: Add internal f2f16 opcodes
The OpFConvert+FPRoundingModeRTP/FPRoundingModeRTN cannot be used
because GL_EXT_spirv_intrinsics does not allow decorations. Instead,
we need opcodes that encode the rounding mode so that they can be used
in glsl code.

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37883>
2026-01-10 11:34:07 +01:00
Konstantin Seurer
6d9cd36db6 nir: Add f2f16_ru/rd opcodes
Those are variants of f2f16 that always round up/down. Constant folding
requires nextafter that supports half floats (util_nextafter).

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37883>
2026-01-10 11:33:23 +01:00
Karol Herbst
2e2b86c64f clc: handle all optional subgroup extensions
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38015>
2026-01-09 21:53:28 +00:00
Alyssa Rosenzweig
4e59199cbb nir: add nir_is_shared_access helper
This is helpful to identify shared mem access for writing more generic code
operating on nir intrinsics.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39219>
2026-01-09 20:51:12 +00:00
Lionel Landwerlin
26e4632f64 nir: add a new push_data_intel intrinsic
We're finally moving on from misusing various intrinsics :
  - load_uniform
  - load_push_constant
  - load_ubo*

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38975>
2026-01-09 14:19:46 +00:00
Lionel Landwerlin
799258fdde nir: use load() helper for inline_data_intel
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38975>
2026-01-09 14:19:45 +00:00
Lionel Landwerlin
c84760a185 nir: add missing divergence handling for ray_query_global_intel
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38975>
2026-01-09 14:19:45 +00:00