Commit graph

8839 commits

Author SHA1 Message Date
Marek Olšák
99fe3ef8ba compiler: add INTERP_MODE_COLOR for radeonsi
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6340>
2020-09-02 23:03:00 -04:00
Marek Olšák
8c43edf9f9 nir: fix a bug in is_dual_slot in nir_io_add_const_offset_to_base
Fixes: 01ab308edc "nir: update IO semantics in nir_io_add_const_offset_to_base"

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6540>
2020-09-02 20:05:05 +00:00
Jason Ekstrand
a06955f4ef nir/builder: Add load/store_global helpers
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6244>
2020-09-02 19:48:44 +00:00
Jason Ekstrand
91becd84ae intel/fs: Add support for a new load_reloc_const intrinsic
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6244>
2020-09-02 19:48:44 +00:00
Bas Nieuwenhuizen
191f8a4b9f spirv: Deal with glslang not setting NonUniform on constructors.
Especially a problem for OpImage/OpSampledImage. Note that the problem
doesn't seem to be propagation through glslang, but only in emitting
the SPIR-V. So it is fine if we are somewhat lossy in handling this, as
long as direct Op(Sampled)Image -> texture op chains work.

Fixes: af81486a8c "spirv: Simplify our handling of NonUniform"
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3406
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6451>
2020-09-02 21:17:02 +02:00
Bas Nieuwenhuizen
965b8441fe spirv: Deal with glslang bug not setting the decoration for stores.
Fixes: af81486a8c "spirv: Simplify our handling of NonUniform"
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6451>
2020-09-02 21:16:57 +02:00
Jason Ekstrand
9121afe861 nir/clone: Add a helper for cloning most instruction types
@anholt needed it for nir_to_tgsi, and the desire comes up frequently.

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6567>
2020-09-02 09:59:17 -07:00
Eric Anholt
f25e169897 nir/opt_vectorize: Add a callback for filtering of vectorizing.
For NIR-to-TGSI, we don't want to revectorize 64-bit ops that we split to
scalar beyond vec2 width.  We even have some ops that we would rather
retain as scalar due to TGSI opcodes being scalar, or having more unusual
requirements.

This could be used to do the vectorize_vec2_16bit filtering, but that
shader compiler option is also used in algebraic so leave it in place for
now.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6567>
2020-09-02 09:59:17 -07:00
Eric Anholt
479d9c97eb nir: Add simplistic lowering for bany_equal/ball_inequal.
It would be nice if we could do swizzling of an expression on the
replacement side so that we could have a single ieq/ine of the vector
after CSE.  However, if you do want vector operations, nir_opt_vectorize()
does just fine.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6567>
2020-09-02 09:58:44 -07:00
Samuel Pitoiset
abbfc5b8db spirv: fix retrieving dest type for OpFragmentMaskFetchAMD
Since the dest type is retrieved from the SPIR-V return type now,
we have to set it manually for OpFragmentMaskFetchAMD. The result
type must be a 32-bit unsigned integer type scalar.

Fix dEQP-VK.pipeline.multisample.shader_fragment_mask.* with RADV.

Fixes: a196f05fc2 ("nir/vtn: Use return type rather than image type for tex ops")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6533>
2020-09-02 08:54:22 +02:00
Jason Ekstrand
c93ade93fb nir/lower_explicit_io: Assert that compute address sizes match derefs
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6379>
2020-09-01 20:50:04 +00:00
Jason Ekstrand
6678f1e81c spirv: Drop the constant_as_global as option
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6379>
2020-09-01 20:50:04 +00:00
Jason Ekstrand
1bdf850638 spirv: Use nir_var_mem_constant for UniformConstant data in CL
For now, we leave the constant_as_global option intact and get rid of
the UBO path which no one upstream is using today.

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6379>
2020-09-01 20:50:04 +00:00
Jason Ekstrand
bb8d8ba9c7 nir: Allow opt_large_constants to be run with constant_data_size > 0
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6379>
2020-09-01 20:50:04 +00:00
Jason Ekstrand
4360a8a2b3 nir/lower_io: Add support for nir_var_mem_constant
This commit adds support for nir_var_mem_constant various places.  It
also adds a pass similar to nir_lower_vars_to_explicit_types except it
also scrapes out the constants and stuffs them into constant_data.

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6379>
2020-09-01 20:50:03 +00:00
Jason Ekstrand
ef142c68e1 nir/lower_io: Add a build_addr_for_var helper
The new version is more verbose but also more extensible.

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6379>
2020-09-01 20:50:03 +00:00
Jason Ekstrand
965c268865 nir/lower_io: Use the variable mode for load_scratch_base_ptr checks
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6379>
2020-09-01 20:50:03 +00:00
Jason Ekstrand
ff124e3fe3 nir: Add a load_global_constant intrinsic
This has the same semantics as load_global except the memory it reads is
known to be constant so load_global_constant intrinsics can be CSEd
rather than relying on more complex copy-propagation.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6379>
2020-09-01 20:50:03 +00:00
Jason Ekstrand
e4f07f8bdc nir: Add a new nir_var_mem_constant variable mode
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6379>
2020-09-01 20:50:03 +00:00
Karol Herbst
b9927c8c8d nir/serialize: fix serialization of system values
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6545>
2020-09-01 14:55:05 -05:00
Karol Herbst
5eacaa95a7 spirv: fix 64 bit atomic inc and dec
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jason Ekstrand <jason@jlekstran.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6433>
2020-09-01 18:47:30 +00:00
Karol Herbst
70cbddc4a7 nir: use enum operator helper for nir_variable_mode and nir_metadata
those are used quite a bit

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6520>
2020-09-01 17:45:08 +00:00
Karol Herbst
24dfd798d6 nir: use nir_var_all to get rid of casting
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6520>
2020-09-01 17:45:08 +00:00
Jason Ekstrand
15da983653 nir: Improve the comment on num_inputs and friends
This doesn't fix the problem that no one knows what any of these mean
half the time but it at least makes them better documented to hopefully
make people's expectations more accurate.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6524>
2020-09-01 17:30:51 +00:00
Jason Ekstrand
4d18e71fea nir: Rename num_shared to shared_size
This one is always a size in bytes.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6524>
2020-09-01 17:30:51 +00:00
Jason Ekstrand
471f260ef3 spirv: Delete some dead workgroup variable handling code
This is dead since 5ed4e31c08.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6524>
2020-09-01 17:30:51 +00:00
Rhys Perry
1459cce7ec spirv: add some tests for volatile/available/visible
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6090>
2020-09-01 17:15:22 +00:00
Jason Ekstrand
fb6b243c11 spirv: Support big-endian strings
This should be all that's required for the SPIR-V parser to work in
big-endian systems.  SPIR-V requires that everything be in host
byte-order except for strings which are always little-endian.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6090>
2020-09-01 17:15:22 +00:00
Rhys Perry
a8d8fbb9ce spirv: implement SpvMemoryAccessVolatileMask
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6090>
2020-09-01 17:15:22 +00:00
Rhys Perry
f81e1d2824 spirv: implement Volatile image operand
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6090>
2020-09-01 17:15:22 +00:00
Rhys Perry
2ba3ffa76c spirv: implement Volatile memory semantic
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6090>
2020-09-01 17:15:22 +00:00
Rhys Perry
23bfba8663 spirv: implement MakePointerAvailable/MakePointerVisible for OpCopyMemory
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6090>
2020-09-01 17:15:22 +00:00
Rhys Perry
e01d1a9f16 spirv: add vtn_emit_make_{visible,available}_barrier helpers
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6090>
2020-09-01 17:15:22 +00:00
Rhys Perry
b85c38a86f spirv: make OpLoad/OpStore visibility/availablity barriers acquire/release
I think these are needed to order the visibility/availability operation
with the access.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6090>
2020-09-01 17:15:22 +00:00
Rhys Perry
5b92392c48 spirv: fix Uniform and Output MemoryAccessMakePointer{Visible,Available}
The Uniform storage class can be used for SSBOs. This should also fix make
available/visible for the Output storage class.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6090>
2020-09-01 17:15:22 +00:00
Boris Brezillon
857b9c5027 spirv: Add a vtn_get_mem_operands() helper
Add a vtn_get_mem_operands() helper to extract memory operand attached
to load/store operations.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6090>
2020-09-01 17:15:22 +00:00
Samuel Pitoiset
bc123c396a nir/algebraic: mark some optimizations with fsat(NaN) as inexact
If a is Nan, fsat(NaN) is expected to be 0 and some optimizations
should be marked as inexact.

Fixes a GPU hang with Death Stranding and RADV/ACO (RADV/LLVM
isn't affected because it lowers fsat).

No fossils-db change.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3368
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6519>
2020-09-01 11:20:03 +02:00
Jesse Natalie
b778e7bd6c nir/vtn: Convert constant samplers to variables with data
Reviewd-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
397730edd5 nir/vtn: Add intrinsics for CL image format/order queries
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
5ce6821900 nir/vtn: ImageSizeLod op can be applied to images
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
701cb9d60c nir/vtn: Handle integer sampling coordinates
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
a196f05fc2 nir/vtn: Use return type rather than image type for tex ops
Since OpenCL images don't have types, we can't use the image type here.
Rather than special-casing and only using SPIR-V return type for CL images,
we can just always use the return type to fill out the tex info.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
de36b5b805 nir/vtn: Add support for kernel images to SPIRV-to-NIR.
There's a few quirks: kernel images are untyped, whether they're
sampled is unknown, and they're passed as inputs to the kernel even though
SPIR-V declares their address space as UniformConstant.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
ce6f66242a nir/vtn: Add type constant to image intrinsics
Since OpenCL supports untyped images, backends might need type info
to be able to support the load/store ops.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:38 +00:00
Jesse Natalie
865a2ad086 clover/nir/spirv: Use uniform rather than shader_in for kernel inputs
The semantics of inputs for CL are a closer match to the semantics of uniforms for graphics.
Rather than cross-stage data, it's data that every thread sees uniformly.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6497>
2020-08-31 19:58:14 +00:00
Eric Anholt
73d2c6cdce nir: Switch the indexing of block->live_in/out arrays.
In nir-to-tgsi, I want to free temps storing SSA values when they go dead,
and NIR liveness has most of the information I need.  Hoever, when I reach
the end of a block, I need to free whatever temps were in liveout which
are dead at that point.  If liveout is indexed by live_index, then I don't
know the maximum live_index for iterating the live_out bitset, and I also
don't have a way to map that index back to the def->index that my temps
are stored under.

We can use the more typical def->index for these bitsets, which resolves
both of those problems.  The only cost is that ssa_undefs don't get merged
into a single bit in the bitfield, but there are generally 1-4 of them in
a shader and we don't track liveness for those anyway so splitting them
apart is fine.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6408>
2020-08-31 18:28:36 +00:00
Eric Anholt
5d2b2b59c4 nir: Invalidate live SSA def information when making new SSA defs.
Noted by @jekstrand in review of the SSA live index changes.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6408>
2020-08-31 18:28:36 +00:00
Italo Nicola
ee288f293b nir: add shared/global atomics to nir_get_io_offset_src()
Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6521>
2020-08-31 17:36:12 +00:00
Eric Anholt
e281ee04df nir: Look up the shader when printing a single instruction.
If you've inserted the instruction into a block, then we can get to the
shader.  This improves our instruction output, giving you i/o semantics
and variable names in intrinsics.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6496>
2020-08-31 17:20:43 +00:00
Eric Anholt
a3a8322dcd nir: Fix printing of individual instructions with io semantics.
The state->shader is missing when used outside of nir_print_shader, just
drop these details in that case.  We can fix nir_print_instr() to look up
the shader, but let's also make sure that an instr detached from a shader
(such as one you're constructing but haven't yet inserted) still works.

Fixes: 2b1ef5df4e ("nir: print IO semantics (v2)")
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6496>
2020-08-31 17:20:43 +00:00