If the break in the original loop isn't in the first top-level if,
this would have re-inserted it in the wrong block.
Fixes this by re-inserting the break block to the corresponding break
block in the new loop by using the remap hashtable.
fossils-db (NAVI21):
Totals from 88 (0.11% of 79330) affected shaders:
Instrs: 109602 -> 109929 (+0.30%); split: -0.10%, +0.40%
CodeSize: 570968 -> 573332 (+0.41%); split: -0.08%, +0.49%
Latency: 1682510 -> 1682505 (-0.00%); split: -0.01%, +0.01%
Copies: 12832 -> 12746 (-0.67%); split: -1.54%, +0.87%
Branches: 2879 -> 2930 (+1.77%)
Deathloop and F1 2023 are affected but I'm not aware of any issues
for these two games.
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10001
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26009>
Previosuly when lowering to txd for sampler array the index would be
derived as well, therefore the resulting derivative would have been a
vec with one more component than what the txd instruction expects.
This patch truncates the coordinate vector in this case to make sure the
index is not derived.
Fixes: b154a4154b ("nir/lower_tex: rewrite tex/txb -> txd/txl before saturating srcs")
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26012>
The changes are pretty straightforward. For vector splitting, we just
ignore those vectors for now. We could potentially handle array derefs
with a constant index (and probably should) but that's left for later.
For now, I'm mostly concerned with correctness of the pass.
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580>
We assumed that calling it on vectors would return 0 and then did a
MAX2(length, 1) to get 1 for vectors. Instead, use a ternary so we
don't make assumptions about non-sensical values.
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580>
Also, leave a big comment about why we're inserting movs and not just
propagating SSA values directly. Hopefully this will prevent idiots
like me from getting clever and thinking they can delete that mov. 😅
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580>
If we're propagating a copy from a cast where the copy copies an entire
array, we end up with something like &((S *)ssa_N)->f[*] in the source
where a wildcard has a cast in its parent chain. If we then try to
propagate the read into a non-wildcard array load, we have to specialize
the wildcard. This breaks because nir_build_deref_follower() doesn't
handle casts. Since we know a priori that, because wildcards are only
generated by copy_deref on arrays, we cannot have a cast with a wildcard
parent so simply chasing the source deref to the first wildcard will
ensure that any casts in the deref are handled properly.
Fixes: ba2bd20f87 ("nir: Rework opt_copy_prop_vars to use deref instructions")
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580>
This adds an idiomatic way to insert NIR function calls with the builder. Since
functions have variable numbers of arguments, this is a variadic function.
v2: Define with a variadic macro instead, for safety with the argument count.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25498>
This fixes intel_clc builds with llvm 17 on gfx125_bvh_build_DFS_DFS
where it dies in the lower indirect derefs pass.
Co-authored-by: Dave Airlie <airlied@redhat.com>
Fixes: 4a4e175738 ("nir: Support deref instructions in lower_var_copies")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25536>
For example, in the loop:
while (more_late_algebraic) {
more_late_algebraic = false;
NIR_PASS(more_late_algebraic, nir, nir_opt_algebraic_late);
NIR_PASS(_, nir, nir_opt_constant_folding);
NIR_PASS(_, nir, nir_copy_prop);
NIR_PASS(_, nir, nir_opt_dce);
NIR_PASS(_, nir, nir_opt_cse);
}
if nir_opt_algebraic_late makes no progress, later passes might be
skippable depending on which ones made progress in the previous iteration.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24197>
Changes a variable type from `nir_component_mask_t` to `uint32_t`. The
variable's name suggests it may have been meant to be a 32-bit integer
anyway.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25691>
There is no neeed #include "main/menums.h" in nir.c,
as it's belongs to gallium code
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24462>
Without this, we replace vote_ieq(b) with vote_ieq(u2u32(b)) which is
wonky because we're doing a u2u on a 1-bit type. With this, we now
replace it with vote_ieq(b2b32(b)). For other subgroup ops, we replace
things like *scan[op](b) with *scan[op](b2b32(b)). For scan ops, this
assumes that b2b1(op(b1b32(x), b2b32(y))) = op(x, y) for all of the ops
iand, ior, and ixor. This is true on all the back-ends I'm aware of.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894>
Without this, nir_type_conversion_op(bool, bool32, RND) will return
u2u32 instead of b2b32 which is pretty unexpected behavior.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894>
On NVIDIA, we can do a vote_ieq on bool in one hardware op so we don't
want that lowered. We do want to lower vote_feq and other vote_ieq,
though.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894>
They're different enough from all the other subgroup ops so it's best to
handle them as their own case.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894>
Using u2u is always correct for integers, including signed integers,
because we're doing a down-cast. It's wrong for floats, though.
Fixes: f95665cfeb ("nir/lower_bit_size: Add support for lowering subgroup ops")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25894>
glsl_type_is_vector_or_scalar would more accruately be called "can be an
r-value that isn't an array, structure, or matrix. This optimization
pass really shouldn't do anything to cooperative matrices. These
matrices will eventually be lowered to something else (dependent on the
backend), and that thing may (or may not) be handled by this or another
pass.
Fixes: 2d0f4f2c17 ("compiler/types: Add support for Cooperative Matrix types")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25871>
Use the hawkmoth c:auto* directives to incorporate nir documentation.
Convert @param style parameter descriptions to rst info field lists.
Add static stubs for generated headers. Fix a lot of references, in
particular the symbols are now in the Sphinx C domain, not C++
domain. Tweak syntax here and there.
Based on the earlier work by Erik Faye-Lund <kusmabite@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24507>
Prepare for using Hawkmoth.
Hawkmoth does not support trailing comments using /**< ... */
syntax. Replace with regular documentation comments.
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24507>
In order to document "typedef struct { ... } T;" as a struct in
hawkmoth, the structs need to have names. Similar for enums.
Note: This is no longer required with Hawkmoth 0.16.0+ and Clang 16 and
later. With the next Hawkmoth release, this should be fixed also for
Clang 15 and earlier.
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24507>
Fixes UBSan:
src/compiler/nir/nir_lower_shader_calls.c:1681:7: runtime error: null pointer passed as argument 1, which is declared to never be null
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25853>
Fixes UBSan error:
src/compiler/nir/nir_serialize.c:1277:70: runtime error: left shift of 524287 by 13 places cannot be represented in type 'int'
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25853>
NVIDIA hardware doesn't take a vertex index for per-vertex I/O.
Instead, it takes an offset into the primitive. This has to be fetched
using a combination of SR_INVOCATION_INFO and the ISBERD instruction.
To keep things simple and allow for maximum CSE, we do the lowering in
NIR and patch the load/store_per_vertex_input/output intrinsic.
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>
Some of us want to lower all TXD with min_lod regardless of whether or
not it's shadow or cube or whatever.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>
Extracted from nir_lower_to_source_mods, this is useful for back-ends
which don't want to rely on NIR source and destination modifiers.
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25000>
Right now some drivers are doing dsign lowering in GLSL and haven't had to
have a NIR path due to there not being a corresponding vulkan driver. We
want this in NIR now so that we can retire that batch of GLSL lowering
code.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25777>
