Commit graph

5419 commits

Author SHA1 Message Date
Connor Abbott
7f0cd6f153 nir/opt_if: Use early returns in opt_if_merge()
We would've had to add yet another level of indentation, or duplicated
finding the if conditions in the next commit. Refactor this function to
use early returns like our other optimizations, so that this isn't an
issue.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-30 15:47:51 +00:00
Connor Abbott
656e428ff4 nir/opt_if: Remove open-coded nir_ssa_def_rewrite_uses()
So that we don't have to change these two places later.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-30 15:47:51 +00:00
Connor Abbott
c6f871b62e nir/lower_returns: Use nir control flow insertion helpers
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-30 15:47:51 +00:00
Jason Ekstrand
92a594b154 spirv: Delete the legacy offset/index UBO/SSBO lowering
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5275>
2020-09-30 07:20:39 +00:00
Jason Ekstrand
657d49a9ba spirv: Use derefs for push constants
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5275>
2020-09-30 07:20:39 +00:00
Jason Ekstrand
ac7537f155 nir/lower_io: Add support for push constants
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5275>
2020-09-30 07:20:38 +00:00
Jason Ekstrand
7a2b4ce22e nir: Allow creating variables with nir_var_mem_push_const.
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5275>
2020-09-30 07:20:38 +00:00
Indrajit Kumar Das
40c1f9883e mesa,glsl: add support for GL_NV_shader_atomic_int64
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6708>
2020-09-29 14:24:44 +00:00
Connor Abbott
51e2b31039 nir: Handle per-view io in nir_io_add_const_offset_to_base()
This isn't strictly necessary for freedreno, since we aren't using it
yet, but I wanted to avoid any problems if we do. If we wanted to handle
this "properly", and handle matrix and array per-view variables, we'd
probably want to encode the "view stride" (number of views per user
location) and base view in the intrinsic, but for now we just don't do
any offsetting and assume the indirect offset is the view.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6514>
2020-09-29 10:32:00 +00:00
Connor Abbott
bc8a5c0752 nir: Add per_view to IO semantics
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6514>
2020-09-29 10:32:00 +00:00
Connor Abbott
5a88db682e nir/lower_io_arrays: Fix xfb_offset bug
I noticed this once I started gathering xfb_info after
nir_lower_io_arrays_to_elements_no_indirect.

Fixes: b2bbd978d0 ("nir: fix lowering arrays to elements for XFB outputs")
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6514>
2020-09-29 10:32:00 +00:00
Connor Abbott
df955ce6b6 nir: Count i/o slots correctly for per-view variables
This function wasn't counting driver slots correctly, resulting in
incorrect driver_location's and input_count. It seems intel doesn't use
this yet.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6514>
2020-09-29 10:32:00 +00:00
Connor Abbott
ffe946d7e8 nir: Add nir_lower_multiview pass
Taken mostly directly from the anv pass. A few anv-specific things that
I could leave in anv aren't included. Specifically on turnip we don't
need to set gl_Layer to 0, and we can handle the case where the FS reads
gl_ViewIndex, so that check is moved into anv.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6514>
2020-09-29 10:31:59 +00:00
Samuel Pitoiset
a0e35c7562 nir/lower_io: change nir_io_add_const_offset_to_base to use bitfield modes
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6890>
2020-09-29 09:40:21 +00:00
Eric Anholt
e8c5f8b9d3 nir/lower_clip: Add i/o semantics for load/store intrinsics.
ir3 looks at the .location on its inputs for handling
non-VARYING_SLOT_POS, but our uninitialized semantics matched that and
threw a compiler assertion failure.

Fixes: 502abfce7f ("nir: save IO semantics in lowered IO intrinsics")
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6716>
2020-09-28 17:35:30 +00:00
Samuel Pitoiset
39098a2053 nir/lower_memory_model: do not break with global atomic operations
Global atomics don't have an access flag.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6888>
2020-09-28 14:47:02 +00:00
Samuel Pitoiset
de1409089c nir/lower_memory_model: return progress when visiting instructions
It never returned progress=TRUE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6888>
2020-09-28 14:47:02 +00:00
Rhys Perry
ee073cb543 nir/load_store_vectorize: improve vectorization with identical operations
We might have several identical options to vectorize an entry with, but
only one might be vectorizable because of writes interfering.

An example of this is a pattern found in some CTS tests:
a = load(0)
b = load(4)
store(0, a)
store(4, b)
a = load(0)
b = load(4)
store(0, a)
store(4, b)
...

It might have attempted to vectorize the first load(0) with the second
load(4) without attempting the second load(4) when the first fails. This
changes vectorize_entries() to continue even if the first try_vectorize()
failed.

fossil-db (Navi):
Totals from 117 (0.09% of 137413) affected shaders:
SGPRs: 7040 -> 7088 (+0.68%)
CodeSize: 276504 -> 276308 (-0.07%); split: -0.08%, +0.01%
Instrs: 51152 -> 51111 (-0.08%); split: -0.09%, +0.01%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5415>
2020-09-28 12:06:30 +00:00
Louis-Francis Ratté-Boulianne
f2b94ae085 glsl/linker: Add support for XFB varying lowering in geometry shader
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6723>
2020-09-28 07:57:28 +00:00
Jason Ekstrand
8323c03bbf spirv: Add support for OpCopyMemorySized
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6713>
2020-09-25 23:48:03 +00:00
Jason Ekstrand
a3177cca99 nir: Add a lowering pass to lower memcpy
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6713>
2020-09-25 23:48:03 +00:00
Jason Ekstrand
b2899f7265 nir: Add a new memcpy intrinsic
This matches SPIR-V's OpCopyMemorySized

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6713>
2020-09-25 23:48:03 +00:00
Jason Ekstrand
8cbb6640f0 spirv: Use the new types helpers
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6713>
2020-09-25 23:48:03 +00:00
Jason Ekstrand
de295b61da compiler/types: Add glsl_baseN_t_type(bit_size) helpers
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6713>
2020-09-25 23:48:03 +00:00
Jesse Natalie
5f1855d896 vtn/opencl: Switch fma to conditionally use libclc for 32bit floats
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 13:20:39 -07:00
Jesse Natalie
8fb25c48a4 vtn/opencl: Only use libclc ldexp when lower_ldexp is set
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 13:14:46 -07:00
Jesse Natalie
09bca4cb95 vtn/opencl: Switch some nir-sequence ops to use libclc
All of these are pretty well-defined. Rather than implementing them
as a sequence of nir ops, we can just use the libclc implementation.

v2 (idr): Delete functions that are now unused.

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 13:14:45 -07:00
Jesse Natalie
03fd217d21 vtn/opencl: Switch division-related ops to use libclc
Specifically, fmod only uses libclc if it was going to be lowered.
Also, add missing half_divide and half_recip handling.

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
2624faf137 vtn/opencl: Switch exp/pow/log to use libclc
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
4cc9ad4eeb vtn/opencl: Switch non-native trig to use libclc
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
6aee49578f vtn/opencl: Hook up OpenCL async copy and group wait opcodes via libclc
Adds an additional hook for spirv_to_nir to handle a core opcode via
the OpenCL libclc infrastructure, and adds handling for SpvOpGroupAsyncCopy and
SpvOpGroupWaitEvents.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
00261d883d vtn/opencl: Rework handle_instr to be able to handle core SPIR-V opcodes via libclc
The OpenCL async copy/wait opcodes are core SPIR-V, rather than OpenCL extension opcodes.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
b08fd45be0 vtn/opencl: Implement a lot of opcodes via libclc
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
6436e3ac18 vtn/opencl: Add infrastructure for calling out to libclc
This patch adds a function remap table with name mangling, which
can convert a SPIR-V OpenCL extension opcode to a call to the external
libclc shader, which will be lowered/inlined after conversion.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
2013903bc2 vtn/opencl: Rework type handling for CL extension opcodes
- Change dest type information to be vtn_type rather than glsl_type
 - Add source type information, also in vtn_type form

Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
938be4d6fa nir/vtn: Add handling for SPIR-V event variables
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Dave Airlie
c9a6b94f9e vtn: add an option to create a nir library from spirv
This adds an options to turn a spir-v library into a set of nir
functions. There is no entry point and all function implentations
are emitted.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
2826a5a923 nir: Add glsl_base_type unsigned -> signed version helper
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
93db59e066 nir: Add an internal flag to shader_info
Don't print the shader if it's marked internal, unless NIR_PRINT
has been explicitly set to 2 (or higher).

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jason Ekstrand
0206fb3941 nir/liveness: Consider if uses in nir_ssa_defs_interfere
Fixes: f86902e75d "nir: Add an SSA-based liveness analysis pass"
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3428
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Yevhenii Kharchenko <yevhenii.kharchenko@globallogic.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6824>
2020-09-25 14:16:15 +00:00
Rhys Perry
a18c84ecce nir/instr_set: hash intrinsic sources
ministat (CSE only):
Difference at 95.0% confidence
	-9.80325 +/- 0.173089
	-41.4434% +/- 0.461972%
	(Student's t, pooled s = 0.0763653)

ministat (entire run):
Difference at 95.0% confidence
	-3.13667 +/- 0.61519
	-5.11107% +/- 0.990737%
	(Student's t, pooled s = 0.271416)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6860>
2020-09-25 10:18:36 +00:00
Marek Olšák
ea77958fea nir: gather information about fbfetch and dual source color
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Marek Olšák
a6abf175ef nir: fix input/output info gathering for lowered IO
Ooops.

Fixes: 17af07024d - nir: gather all IO info from IO intrinsics

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Marek Olšák
ef98c175c0 nir: gather fs.uses_sample_qualifier from lowered IO
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Marek Olšák
7b108e6ac4 nir: set system_values_read for all intrinsics
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Marek Olšák
abe9588ff0 nir: gather tess.tcs_cross_invocation info from lowered IO intrinsics
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Marek Olšák
10be706778 nir: gather indirect info from lowered IO intrinsics
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Jason Ekstrand
3173367a47 spirv: vtn_fail with a nice message on unsupported rounding modes
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6845>
2020-09-25 01:43:28 +00:00
Kenneth Graunke
140f53e646 Revert "nir: replace lower_ffma and fuse_ffma with has_ffma"
This reverts commit 939ddf3f67.

Intel has a separate pass for fusing FFMAs selectively.  We split
these flags in commit 1b72c31e1f and
the reasoning still stands.  The patch being reverted was just a
cleanup, so there should be no issue with reverting it.

Acked-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6849>
2020-09-24 13:11:50 -07:00
Marek Olšák
939ddf3f67 nir: replace lower_ffma and fuse_ffma with has_ffma
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6756>
2020-09-24 12:29:11 +00:00