These handles can be emitted in control flow, which means that the handle
might be in a block which does not dominate a block that's processed
later on, which results in incorrect DXIL if we try to reference it.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26803>
HLSL docs say WaveReadLaneAt is undefined if the target lane is inactive. This makes
sense since the target lane may need to *send* the data, rather than it being pulled
by the calling lane. So don't early-out on the loop, iterate through the whole wave
on all threads and read the cross-lane data before branching.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27624>
DXIL doesn't have instruction-level coherency. We have 3 options:
1. Promote the instruction to an atomic instruction. We can only do this
for 32-bit or 64-bit ops.
2. If using bindless, declare the local resource declaration as globally-coherent.
3. If not using bindless, add globally-coherent to the global resource declaration.
This pass does all 3 of these, stopping at the intrinsic level for supported types
of atomics, otherwise assigning to the global resource declaration, which will be
unused if we're doing bindless, where instead we'll get it from the instruction.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27348>
No driver supports urol/uror on all bit sizes. Intel gen11+ only for 16
and 32 bit, Nvidia GV100+ only for 32 bit. Etnaviv can support it on 8,
16 and 32 bit.
Also turn the `lower` into a `has` option as only two drivers actually
support `uror` and `urol` at this momemt.
Fixes crashes with CL integer_rotate on iris and nouveau since we emit
urol for `rotate`.
v2: always lower 64 bit
Fixes: fe0965afa6 ("spirv: Don't use libclc for rotate")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by (Intel and nir): Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27090>
These are now covered by nir_opt_loop():
- opt_if_loop_last_continue()
- opt_merge_breaks()
- opt_if_loop_terminator()
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24940>
When given (e.g.) 3x 16-bit components to store on a device that
isn't using native 16-bit loads and stores, we should be lowering
that into one 32-bit store and one masked store. Instead, the logic
here ends up returning that the best we can do is one 8-byte store,
which is clearly wrong. Stores should round down, loads should
round up.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26293>
First, we need to give the parent_instr field a unique name to be able to
replace with a helper. We have parent_instr fields for both nir_src and
nir_def, so let's rename nir_src::parent_instr in preparation for rework.
This was done with a combination of sed and manual fix-ups.
Then we use semantic patches plus manual fixups:
@@
expression s;
@@
-s->renamed_parent_instr
+nir_src_parent_instr(s)
@@
expression s;
@@
-s.renamed_parent_instr
+nir_src_parent_instr(&s)
@@
expression s;
@@
-s->parent_if
+nir_src_parent_if(s)
@@
expression s;
@@
-s.renamed_parent_if
+nir_src_parent_if(&s)
@@
expression s;
@@
-s->is_if
+nir_src_is_if(s)
@@
expression s;
@@
-s.is_if
+nir_src_is_if(&s)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671>
The inc_compiler should come as part of idep_compiler, idep_nir or
idep_nir_headers dependency.
Acked-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> (v3dv)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25314>
With the new nir_opt_barrier_modes() pass, we may encounter control
barriers with no memory modes set, such as:
@barrier () (execution_scope=WORKGROUP, memory_scope=WORKGROUP, mem_semantics=ACQ|REL, mem_modes=0)
The DXIL validator documentation [1] mentions an
INSTR.BARRIERMODENOMEMORY validation rule:
"sync must include some form of memory barrier - _u (UAV) and/or
_g (Thread Group Shared Memory). Only _t (thread group sync) is
optional."
We were generating a dx.op.barrier instruction with only one flag,
DXIL_BARRIER_MODE_SYNC_THREAD_GROUP. This seems to run afoul of the
above validator rule. So, this patch adjusts the code generator to
set DXIL_BARRIER_MODE_UAV_FENCE_THREAD_GROUP too, whenever
UAV_FENCE_GLOBAL isn't required.
[1] https://github.com/microsoft/DirectXShaderCompiler/blob/main/docs/DXIL.rst
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24842>
