Commit graph

213 commits

Author SHA1 Message Date
Emma Anholt
8fa6d473e7 nir: Add SYSTEM_VALUE_FRAG_COORD_Z/W.
Intel's going to use these.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25190>
2025-06-18 23:11:36 +00:00
Job Noorman
78f62d6d6d nir: remove unused global_atomic(_swap)_ir3 intrinsics
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
ir3 switched to using the generic ones.

Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33797>
2025-06-18 19:06:33 +00:00
Marek Olšák
f2c48652da nir: add shader_info::tess::tcs_*outputs_read_by_tes*
Gather no_varying for AMD.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34780>
2025-06-07 16:29:39 +00:00
Marek Olšák
f58c0cbb6a nir: split *_accessed_indirectly* bitmasks into *_read/written_indirectly*
for AMD

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34863>
2025-05-08 02:54:12 +00:00
Marek Olšák
afd8fefb79 nir: add shader_info::tess::tcs_cross_invocation_outputs_written
for AMD

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34863>
2025-05-08 02:54:12 +00:00
Ella Stanforth
d3aedbfe9d asahi/lib: Move alpha_to_one and alpha_to_coverage lowering to common code.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33942>
2025-04-23 09:03:41 +00:00
Connor Abbott
1621080df7 compiler,nir: Gather needs_full_quad_helper_invocations info
This is needed on Qualcomm, where there are separate fields to enable
just 3 fragments and all 4 fragments.

Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Fixes: 264d8a6766 ("ir3: Set need_full_quad depending on info.fs.require_full_quads")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33862>
2025-03-14 21:55:58 +00:00
Connor Abbott
7a55e13939 nir, compiler: Rename needs_quad_helper_invocations
This currently treats coarse and fine derivatives the same, but Qualcomm
needs to know whether just coarse derivatives are used or fine
derivatives/quad ops are also used. Rename this to
needs_coarse_quad_helper_invocations make clear the difference from the
new field, needs_full_quad_helper_invocations.

Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Fixes: 264d8a6766 ("ir3: Set need_full_quad depending on info.fs.require_full_quads")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33862>
2025-03-14 21:55:57 +00:00
Daniel Schürmann
dbd41e3ddd nir: set SYSTEM_VALUE_HELPER_INVOCATION read for nir_intrinsic_is_helper_invocation
is_helper_invocation is the volatile access of load_helper_invocation.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33492>
2025-03-07 15:44:49 +00:00
Alyssa Rosenzweig
91872c9c51 nir: clang-format
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33722>
2025-02-26 15:19:53 +00:00
Alyssa Rosenzweig
edb0164623 nir/gather_info: use subgroup/quadgroup flags
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33410>
2025-02-11 07:30:07 +00:00
Friedrich Vock
a8ce60eec1 nir: Apply passes to all functions
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29577>
2025-01-29 04:38:51 +00:00
Friedrich Vock
bb40284f76 nir: Add indirect calls
Used to jump to a function referred to by a runtime pointer.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29577>
2025-01-29 04:38:51 +00:00
Georg Lehmann
15d754fefa nir: add load_front_face_fsign
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32791>
2024-12-30 22:31:34 +00:00
Caterina Shablia
f4fcfa8016 pan,nir: introduce load_attribute_pan
load_attribute_pan is a panfrost-specific intrinsic for loading
vertex attributes. Takes explicit vertex and instance IDs which
we need in order to implement vertex attribute divisor with
non-zero base instance on v9+.

Passes which are used by panvk are modified to be aware of
load_attribute_pan.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32039>
2024-12-18 08:33:16 +00:00
Benjamin Lee
becb014d27 nir: treat per-view outputs as arrayed IO
This is needed for implementing multiview in panvk, where the address
calculation for multiview outputs is not well-represented by lowering to
nir_intrinsic_store_output with a single offset.

The case where a variable is both per-view and per-{vertex,primitive} is
now unsupported. This would come up with drivers implementing
NV_mesh_shader or using nir_lower_multiview on geometry, tessellation,
or mesh shaders. No drivers currently do either of these. There was some
code that attempted to handle the nested per-view case by unwrapping
per-view/arrayed types twice, but it's unclear to what extent this
actually worked.

ANV and Turnip both rely on per-view outputs being assigned a unique
driver location for each view, so I've added on option to configure that
behavior rather than removing it.

Signed-off-by: Benjamin Lee <benjamin.lee@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31704>
2024-12-09 20:31:49 +00:00
Caterina Shablia
7ca8c19246 Revert "nir: introduce instance_index system value"
This reverts commit b9be1f1f20.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32332>
2024-11-28 07:53:01 +00:00
Caterina Shablia
b9be1f1f20 nir: introduce instance_index system value
The semantics of this newly introduced system value match
Vulkan's InstanceIndex exactly, and are equivalent to
instance_id + base_instance.

Some hardware, such as Mali Valhall or later, only provides
instance id offset by base_instance. Introducing a new system
value to represent this, rather than handling the mismatch
when lowering to BIR lets us use NIR to eliminate redundant
arithmetic that would follow from mismatched semantics, e.g.
instance_id could be lowered to instance_index - base_instance,
so expressions such as instance_id + base_instance would be
optimized to a simple instance_index.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32158>
2024-11-19 09:18:47 +00:00
Georg Lehmann
2f830f9b94 nir: add SYSTEM_VALUE_PIXEL_COORD
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31864>
2024-11-04 12:34:30 +00:00
Marek Olšák
fb6184f89c nir: add shader_info::tess::tcs_same_invocation_inputs_read(_indirect)
We need both the same-invocation usage mask and cross-invocation usage
mask. The AMD reason is below.

Cross-invocation TCS input access doesn't prevent the same-invocation
fast path in AMD hw because it's just a different way to load the same
data, and we want to use both paths for the same TCS input based on
the load instruction. The fast path can't be used for indirect access,
which is gathered separately for same-invocation access.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31645>
2024-10-21 18:53:51 +00:00
Georg Lehmann
dbf63a0788 nir: remove nir_op_is_derivative
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31014>
2024-10-17 09:50:19 +00:00
Georg Lehmann
bf0d1a42b4 nir: remove uses_fddx_fddy
Unused and the code didn't even do what the comment said.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31014>
2024-10-17 09:50:19 +00:00
Timothy Arceri
fe9b93fc1c nir: handle wildcard array deref
Here we add handling of wildcard array derefs when attempting to mark
an io as partially used rather than hitting an assert.

Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31137>
2024-09-25 09:39:44 +00:00
Karol Herbst
fc88f04ba1 vtn, nir: handle OpImageQueryLevels on images
This is needed for cl_khr_mipmap_image, specifically the OpenCL C
function get_image_num_mip_levels.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30834>
2024-08-27 15:06:17 +00:00
Alyssa Rosenzweig
9f9f96d2f9 nir/gather_info: handle derivative intrinsics
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30565>
2024-08-08 15:26:07 +00:00
Karol Herbst
14ea102175 nir: add load_global_size intrinsic
There is no need to compute it in the shader as the result is known at
runtime already.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Tested-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30467>
2024-08-01 17:43:42 +00:00
Marek Olšák
b2d32ae246 nir: add nir_intrinsic_load_per_primitive_input, split from io_semantics flag
Instead of having 1 bit in nir_io_semantics indicating a per-primitive
FS input, add a dedicated intrinsic for it.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29895>
2024-07-23 16:13:16 +00:00
Samuel Pitoiset
aa1f00cf5c nir/gather_info: handle uses_fbfetch_output for texture operations
Like nir_texop_txf_ms.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30109>
2024-07-12 09:33:51 +00:00
Samuel Pitoiset
0d0b949cd7 nir/gather_info: handle uses_fbfetch_output for sparse image loads
Looks like this was missing.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30109>
2024-07-12 09:33:51 +00:00
Daniel Schürmann
7af16e9f1e nir/shader_info: remove uses_demote
This flag is mostly redundant with uses_discard and was only
introduced to implement demote with LLVM when it didn't have
that intrinsic.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27617>
2024-06-17 19:37:16 +00:00
Daniel Schürmann
9b1a748b5e nir: remove nir_intrinsic_discard
The semantics of discard differ between GLSL and HLSL and
their various implementations. Subsequently, numerous application
bugs occurred and SPV_EXT_demote_to_helper_invocation was written
in order to clarify the behavior. In NIR, we now have 3 different
intrinsics for 2 things, and while demote and terminate have clear
semantics, discard still doesn't and can mean either of the two.

This patch entirely removes nir_intrinsic_discard and
nir_intrinsic_discard_if and replaces all occurences either with
nir_intrinsic_terminate{_if} or nir_intrinsic_demote{_if} in the
case that the NIR option 'discard_is_demote' is being set.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27617>
2024-06-17 19:37:16 +00:00
Italo Nicola
62c8e58f39 nir: add {load,store}_global_etna intrinsics
Acked-by: David Heidelberg <david@ixit.cz>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Signed-off-by: Italo Nicola <italonicola@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29402>
2024-05-27 17:58:51 +00:00
Karol Herbst
25d697ef25 nir: add SYSTEM_VALUE_BASE_WORKGROUP_ID
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26800>
2024-04-24 20:18:48 +00:00
Timur Kristóf
433fe2022c nir/gather_info: Clear per-primitive I/O masks at the beginning.
Fixes: b085248819
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28685>
2024-04-14 19:51:11 +00:00
Mike Blumenkrantz
a9d366085c nir/gather_info: fix gathering for compact arrayed builtins
these are treated as vecs, but depending on driver caps num_slots
may or may not equal the array size

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28554>
2024-04-05 17:56:28 +00:00
Timur Kristóf
ecbf3464f6 nir: Record per-primitive inputs without variables.
Previously, this information would have been lost when the
shader has no I/O variables.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28489>
2024-04-02 23:00:01 +00:00
Timur Kristóf
b085248819 nir/gather_info: Record per-primitive outputs without variables.
Previously, this information would have been lost when the
shader has no I/O variables.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28489>
2024-04-02 23:00:01 +00:00
Timothy Arceri
ec240e2cd8 nir: allow gather info to handle nir_deref_type_array_wildcard
Needed for some changes to the glsl nir linker in the following
patches.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27669>
2024-02-20 23:29:17 +00:00
Bas Nieuwenhuizen
c7b2ac3377 radv: Remove ray_launch_size_addr_amd system value.
Not used anymore, so clean it up.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27664>
2024-02-17 11:08:16 +00:00
Daniel Schürmann
c1ef6037fd nir/gather_info: fix enumeration of wide subgroup intrinsics
nir_intrinsic_ballot_* are no subgroup operations.
nir_intrinsic_rotate was missing.
nir_intrinsic_mbcnt_amd is not a subgroup operation.
nir_intrinsic_writelane_amd only affects a single invocation.

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18249>
2024-01-08 10:01:47 +00:00
Qiang Yu
b16017d080 nir: fix load layer id system_values_read info gather
Otherwise radeons+ACO compilation fail.

Fixes: 32ee6376ad ("nir: add lowering from FS LAYER input to LAYER_ID sysval")
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26422>
2023-12-01 03:17:57 +00:00
Marek Olšák
7a9b73fcb8 nir: fix gathering TESS_LEVEL_INNER/OUTER usage with lowered IO
Those varyings shouldn't flag patch_inputs_read/patch_outputs_written.

Fixes: 10be706778 - nir: gather indirect info from lowered IO intrinsics

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26275>
2023-11-25 09:50:19 -05:00
Daniel Schürmann
f1110576d9 nir: add info.fs.require_full_quads
This flag indicates the requirement of helper invocations
in fragment shaders, independent from any present instructions.
This fixes the lowering of OpGroupNonUniformQuad* instructions.

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26026>
2023-11-22 11:31:52 +01:00
Daniel Schürmann
2db0507b5d nir/gather_info: add missing wide subgroup operations
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26026>
2023-11-22 11:31:46 +01:00
Daniel Schürmann
1179d83a89 nir: remove info.fs.needs_all_helper_invocations
Use info.uses_wide_subgroup_intrinsics instead.

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26026>
2023-11-22 11:31:11 +01:00
Connor Abbott
1cfb0ae92c nir: Add quad vote intrinsics
Both Intel and AMD have special hardware support for these.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/218>
2023-11-17 09:45:40 +00:00
Yonggang Luo
ffa458ee8f nir: remove redundant include of gallium headers
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25880>
2023-10-26 09:35:04 +00:00
Mary Guillemard
5308378a35 nir: Add NVIDIA-specific geometry shader opcodes
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>
2023-10-24 22:21:18 +00:00
Rhys Perry
4bd4ff5d9b nir: improve ms_cross_invocation_output_access with local_invocation_id
Since GFX11, RADV doesn't need to lower local_invocation_id.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25040>
2023-10-24 21:36:06 +00:00
Marek Olšák
b1bbe4e190 nir: gather dual slot input information
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25394>
2023-10-07 11:18:40 +00:00