Yet another bit of branchiness we should tame. 99% of the time, sources are not
for if's, so we shouldn't need to do the extra checking to handle that 1%.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
We don't check the sizes for ALU srcs, which is the hot path here, so split out
that simplified version for ALU instructions to use, while deriving a sized
version for other kinds of instructions.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
There's no more nir_register.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
We have dominance validation elsewhere in the file.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
Instead, check it at the call sites when actually required (basically just
intrinsics), reducing the branching required when not (ALU validation, the
hottest of hot paths for CI).
IMHO this is more obvious too.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
No apparent performance difference, but documents the intention.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
Nothing should ever be reading them, they logically do not exist. So there's no
point validating them, especially when the validation in question is so useless
(just checking the bit width, without any semantic awareness). Yet now that we
support vec16, this loop is quite hot even on scalar ISAs, and rather
pointlessly so. Just remove it and bring the ALU src validation complexity to
O(# of channels in source) instead of O(max # of channels in NIR).
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
It doesn't inline and so is about 1% of M1 CTS time. Expand out the definition
and simplify the logic. Honestly, I think this is clearer too.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
Profiling showed that maintaining this ssa_srcs set consumes ~3% of CTS time
with a debugoptimized build. Unfortunately, we really do benefit from getting
this coverage in CI. So rather than remove the validation, let's optimize the
data structure used so we can keep the coverage at a fraction of the cost.
The expensive piece is the pointer set, which is backed by a relatively
expensive hash table. It would be much cheaper to use an invasive set instead,
with a single "present" bit. We don't want to bloat nir_src for this, however
there's an easy solution: use a tagged pointer to steal a bit in the nir_src for
the job. We untag everything at the end of validation (and this meta-invariant
is asserted with an auxiliary counter), so while we mutate the IR while
validating, the mutations do not escape nir_validate.
We tag the parent pointer and not the def pointer, because it is dramatically
less used and therefore has far fewer disrupted call sites.
The M1 job is improved from 3:03 to 2:55 of deqp-runner reported time, which is
excellent.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
Deduplicates the "get # of channels" logic which was the same between the
helpers.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26084>
Based on brw_nir_lower_conversions.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25566>
There is no real reason to prevent this as far as I know. And some of the
SPIR-V generated by DPCPP is running into this.
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25701>
The code here handled stores of actual 3-byte values (8-bit, 3-component), but didn't
correctly handle stores of larger 8-bit vectors that were constrained by write mask to
just 3 bytes. In that case, the pad-to-vec4 step was unnecessary and problematic.
Seen in CL CTS test_basic vector_swizzle test group for char3 with CLOn12.
Fixes: c70d94a8 ("nir_lower_mem_access_bit_sizes: Support unaligned stores via a pair of atomics")
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26034>
This gives FS I/O the same treatment as we did for vertex attributes in
that we now have a NIR intrinsic which pretty closely matches the
hardware and we lower to that before going into NAK. This gives us a
bit more control in the NIR.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26181>
`lower_pv_mode_vertices_for_prim` and `decomposed_primitive_size` return
the same values as `mesa_vertices_per_prim` for the primitives that can
be used as output in geometry shaders.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26117>
In the tagged commit, we stopped actually inverting the condition, and
instead relied on the "invert_cond" flag. But we missed a few places
where this flag should've been handled too.
Also, add a few more tests to make sure this won't regress in the future.
Fixes: 99a7a664 ("nir/loop_analyze: Change invert_cond instead of changing the condition")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10012
Signed-off-by: Sviatoslav Peleshko <sviatoslav.peleshko@globallogic.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26024>
We need both: decomposed primitives for transform feedback and regular
primitives for the sizing the index buffer.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
macros.h is not safe for CL for a bunch of reasons but shader_enums.h barely
uses its functionality. Stub out the minimum for CL.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Antonino Maniscalco <antonino.maniscalco@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26056>
If the break in the original loop isn't in the first top-level if,
this would have re-inserted it in the wrong block.
Fixes this by re-inserting the break block to the corresponding break
block in the new loop by using the remap hashtable.
fossils-db (NAVI21):
Totals from 88 (0.11% of 79330) affected shaders:
Instrs: 109602 -> 109929 (+0.30%); split: -0.10%, +0.40%
CodeSize: 570968 -> 573332 (+0.41%); split: -0.08%, +0.49%
Latency: 1682510 -> 1682505 (-0.00%); split: -0.01%, +0.01%
Copies: 12832 -> 12746 (-0.67%); split: -1.54%, +0.87%
Branches: 2879 -> 2930 (+1.77%)
Deathloop and F1 2023 are affected but I'm not aware of any issues
for these two games.
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10001
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26009>
Previosuly when lowering to txd for sampler array the index would be
derived as well, therefore the resulting derivative would have been a
vec with one more component than what the txd instruction expects.
This patch truncates the coordinate vector in this case to make sure the
index is not derived.
Fixes: b154a4154b ("nir/lower_tex: rewrite tex/txb -> txd/txl before saturating srcs")
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26012>
The changes are pretty straightforward. For vector splitting, we just
ignore those vectors for now. We could potentially handle array derefs
with a constant index (and probably should) but that's left for later.
For now, I'm mostly concerned with correctness of the pass.
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580>
This makes it consistent with glsl_get_array_element() which returns the
scalar type for vectors, column type for matrices, and array element
type for arrays.
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580>
We assumed that calling it on vectors would return 0 and then did a
MAX2(length, 1) to get 1 for vectors. Instead, use a ternary so we
don't make assumptions about non-sensical values.
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580>
Also, leave a big comment about why we're inserting movs and not just
propagating SSA values directly. Hopefully this will prevent idiots
like me from getting clever and thinking they can delete that mov. 😅
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22580>
