ctz is a CL2.0 opcode but 3.0 requires it as well so just add support
for it.
Tested against CTS integer_ops integer_ctz test.
(long line broken up)
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7468>
We need to consider shader calls as potential writes to their payloads.
For other ray-tracing intrinsics, we may not have a shader payload
pointer and have to treat them more like a barrier. We also need to
ensure that global and SSBO reads/writes aren't propagated across shader
call intrinsics.
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>
The SPV_KHR_ray_tracing extension adds 6 new storage classes which is a
bit on the ridiculous side. In order to avoid adding that many variable
modes to NIR, we make a few simplifying assumptions:
1. CallableData and RayPayload data actually lives on the stack
somewhere, presumably in the caller's stack. We assume that these
are no different from global variables and use nir_var_shader_temp
for them. We still need a separate storage class for the incoming
variants but only so we can figure out which one the incoming one
is and lower it to something useful.
2. There's no difference between incoming CallableData and RayPaolad
data. We can use a single storage class for both.
3. ShaderRecordBuffer data is just a global memory access. This lets
us avoid NIR variables entirely and just fetch the pointer via the
shader_record_ptr system value and it's accessed using a 64-bit
global memory pointer.
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>
If we were desperate to reduce bits, we could probably also use
shader_in/out for hit attributes as they really are an output from
intersection shaders and read-only in any-hit and closest-hit shaders.
However, other passes such as nir_gether_info like to assume that
anything with nir_var_shader_in/out is indexed using vec4 locations for
interface matching. It's easier to just add a new variable mode.
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>
These are a bit more tricky than most because they're matrix system
values. We make the intentional choice here to not bother with allowing
indirect addressing of columns for these. Since they're system values,
they may be magically constructed somehow or come from weird hardware so
it's easier on back-ends to just handle any indirects with bcsel.
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>
This is an operation we have to do already for nir_vector_extract and
I'm about to do something very similar for matrix columns. Having a
more generic helper is useful.
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>
Missing in this commit are NIR intrinsics for the ObjectToWorld and
WorldToObject built-ins. Those are matrices and so they take a bit more
work and justify a separate commit. For now, we add the enums and leave
the SYSTEM_VALUE <-> nir_intrinsic conversion commented out.
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>
For now, we assume its a 64-bit global pointer.
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>
some drivers may have pre-lowered gl_ClipDistance to 2x vec4 to match hw
usage, so for those cases we'll be getting deref_var here and then components
will be stored to the deref at some point
fixesmesa/mesa#3480
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6563>
we can just check the bits using clip_distance_array_size here to simplify
everything and more easily determine if we need to be running this pass
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6563>
This also seems to be done by nir_opt_algebraic, but RADV will be moving
nir_lower_bit_size() to after that (so it doesn't create unsupported
8/16-bit instructions) and it doesn't seem worth creating a new pass just
for this simple optimization.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4791>
The shuffles provided by the SPV_INTEL_subgroups extension generate
bcsel(b, shuffle(x, i), shuffle(y, j))
In the case where x and y are the same, we can turn this into a shuffle
with the bcsel on the index which lets us drop a whole shuffle.
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7366>
Typically, if we have one alu instruction, we call it "alu" and if we
have one intrinsic we call it "intrin".
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7366>
Fixes validation error in a Dota 2 shader.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Fixes: b031c64349 ("nir: Convert a bcsel with only phi node sources to a phi node")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7426>
Unlike most address formats, this address format is capable of handling
all of the fancy generic pointers stuff like is_global and friends.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
If the pointer is generic and we haven't yet figured out what kind of
pointer it is yet, we emit an if-ladder based on a mode check.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
The guts are still missing so it will blow up if it sees any
deref_mode_is intrinsic that it can't constant-fold from the mode.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
If opt_restrict_deref_modes makes progress, we may be able to figure out
the mode well enough to turn a deref_mode_is intrinsic into a constant.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
Most of this is fairly straightforward; we just set all the modes on any
derefs which are generic. The one tricky bit is OpGenericCastToPtrExplicit.
Instead of adding NIR intrinsics to do the cast, we add NIR intrinsics
to do a storage class check and then bcsel based on that.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
The way they're handled is that deref->modes is treated as a bitfield of
possible modes. Variables are required to have a specific mode and
derefs with deref_type_var are as well.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
We rename it to "modes" to make it clear that it may contain more than
one mode and adjust all the uses of nir_deref_instr::modes to attempt to
handle multiple modes.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
All three passes check the variables for complex uses and don't split
them if they have any complex uses. Most of these checks are just early
returns to avoid chasing the deref to the variable and a hash table
lookup if we can quickly determine it has the wrong mode. In a couple
of cases, we need to re-arrange or add other checks to ensure that it's
safe for generic pointers.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
All the checks being replaced are fore potential aliasing so we want to
flush stores whenever the mode might be something that aliases.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
We use the same nir_deref_mode_is_in_set helper that we use in
nir_lower_vars_to_explicit_types for the same reason. If there are any
generic pointers in play, we have to lower all generic pointer modes at
the same time or else we risk types getting out-of-sync.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
We can only lower a deref to SSA in this pass if it's guaranteed to be
nir_var_function_temp. We already flag any variables with complex uses
(i.e. casts) as not being lowerable and refuse to lower any derefs to
them so we don't have to worry about false negatives.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
If we don't know the actual mode then we can't get to the variable so
it's going to be a scratch or other indirect load anyway and we aren't
saving ourselves anything by unrolling the loop.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
In both cases, we're trying to determine if a load is scalarizable. We
don't want to scalarize if it's a function_temp or shader_temp because
it might turn into something we can't scalarize.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
For non-explicit nir_lower_io, we use nir_deref_mode_is because there's
no way it works for generic pointers. For nir_lower_vars_to_explicit_types,
and nir_lower_explicit_io, we use nir_deref_mode_is_in_set to ensure we
never get type confusion. For generic pointers, this means that they
must be called with the full set of generic pointer modes.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
