There's no good reason for this to be header-only besides laziness on my
part when I first wrote a few "small" helpers. Some of those are pretty
good sized and don't need to be inlined.
Keeping the original copyright since this is just moving code.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28793>
As long as drivers implement an fmin/fmax that do the right thing with
NaN, there's no reason for the integer comparison.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28793>
I have no idea why I flipped the order of these to checks vs. the C
code when I wrote the NIR helper. We need to deal with NaN first or
else the fmin will smash NaN to MAX_RGB9E5 and it won't get handled as
NaN.
Fixes: 9981709d8f ("nir/format_convert: Add a function to pack RGB9_E5 formats")
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28793>
Since nir_opt_varyings requires scalar IO and thus all drivers have to
scalarize it, this gives the option to re-vectorize IO after that.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29406>
This came up while reviewing
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29398 ... Possibly
this intrinsic should be renamed to load_smem_constant_amd for consistency with
load_global_constant. But if we're not going to convey constantness in the
intrinsic name, let's at least document the restriction, because NIR's optimizer
relies on it.
(I didn't inspect every call site, but it looks like load_smem_amd is just used
for descriptor loads so there's no bug to fix.)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29743>
Most passes want to preserve this specific combination of metadata, so let's add
an alias for the combination. The alias communicates that the control flow graph
is preserved, rather than a particular statement about e.g. dominance
preservation.
You don't need to understand dominance to write a simple
nir_shader_instructions_pass. And since you were going to cargo cult the
metadata anyway, this way you'll cargo cult a version you're more likely to
understand.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29745>
This flag is mostly redundant with uses_discard and was only
introduced to implement demote with LLVM when it didn't have
that intrinsic.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27617>
The semantics of discard differ between GLSL and HLSL and
their various implementations. Subsequently, numerous application
bugs occurred and SPV_EXT_demote_to_helper_invocation was written
in order to clarify the behavior. In NIR, we now have 3 different
intrinsics for 2 things, and while demote and terminate have clear
semantics, discard still doesn't and can mean either of the two.
This patch entirely removes nir_intrinsic_discard and
nir_intrinsic_discard_if and replaces all occurences either with
nir_intrinsic_terminate{_if} or nir_intrinsic_demote{_if} in the
case that the NIR option 'discard_is_demote' is being set.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27617>
We need to check the sample too. fixes on Honeykrisp with MSAA storage images:
dEQP-VK.robustness.robustness2.bind.notemplate.r32i.dontunroll.nonvolatile.storage_image.fmt_qual.img.samples_4.2d_array.comp
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29741>
On some ISAs (e.g., ir3) the offset calculation wraps the same way as
normal unsigned addition so potentially wrapping operations do not have
to be ignored.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28664>
To support cases where different instructions may be used for the same
storage type. For example, to load from an SSBO on ir3, either ldib (max
offset 127) or isam.v (max offset 255) can be used.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28664>
Add the BASE index to the load/store_ssbo_ir3 intrinsic to store an
immediate offset. This offset is encoded in the corresponding fields of
isam.v/ldib.b/stib.b.
One extra optimization is implemented: whenever the regular offset is
also a constant, the total offset (regular plus immediate) is aligned
down to a multiple of the max immediate offset and this is used as the
regular offset while the immediate is set to the remainder. This ensures
that the register used for the regular offset can often be reused among
multiple contiguous accesses.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28664>
The ldc_nv and ldcx_nv intrinsics correspond to the index and bindless
forms of NVIDIA's LDC instruction, respectively. ldc_nv is pretty much
load_ubo without some of the unnecessary constant bits while ldcx_nv
takes a 64-bit bindless handle instead of an index. The other two give
us a little control over register allocation at the NIR level to ensure
that LDCX handles are placed in uniform registers.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
Here we test the rematerialization of the deref produces valid nir
when both the deref and array index value are moved to the else branch of
the first terminator during the merge.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29686>
We were not restoring an outer loop as the current loop after we had
finished processing a nested loop.
Fixes: 9995f336e6 ("nir: add merge loop terminators optimisation")
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29686>
This value is intended to be used to remove out of bounds array
access when unrolling loops so it should contain the comparison
that contains the the induction variable not the overall
condition of the loop terminator. So here we update the instruction
when dealing with iand/ior loop terminator conditions.
Acked-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28998>
nir_lower_frag_coord_to_pixel_coord was adding .5 to work around that the
drivers were mistakenly setting PIXEL_COORD_HALF_INTEGER. With the
setting corrected, the GL frontend handles it appropriately (instead of
subtracting half in the frontend for ARB_fragment_coord_conventions
integer setting and then adding the half back here), and makes the pass
reusable from Intel.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29585>
