Commit graph

4340 commits

Author SHA1 Message Date
Samuel Pitoiset
1ebd9459e7 radv: implement VK_AMD_device_coherent_memory
This extension adds the device coherent and device uncached memory
types. It's known to be slower than non-device coherent memory but
it might be useful for debugging.

This is only exposed for chips that support L2 uncached.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-18 08:20:19 +00:00
Samuel Pitoiset
2af7511ed2 ac: add radeon_info::has_l2_uncached
For chips that have uncached device memory (ie. MTYPE_UC).

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-18 08:20:19 +00:00
Rhys Perry
df645fa369 aco: implement VK_KHR_shader_float_controls
This actually supports more of the extension than the LLVM backend but we
can't enable it because ACO doesn't work with all stages yet.

With more of it enabled, some CTS tests fail because our 64-bit sqrt
is very imprecise. I can't find any precision requirements for it
anywhere, so I'm thinking it might be a CTS issue.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
2019-11-15 17:36:21 +00:00
Rhys Perry
be1d11249b aco: fix 64-bit fsign with 0
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: 93c8ebfa ('aco: Initial commit of independent AMD compiler')
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
2019-11-15 17:36:21 +00:00
Rhys Perry
b062b92ab1 aco: don't combine literals into v_cndmask_b32/v_subb/v_addc
No pipeline-db changes

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: 93c8ebfa ('aco: Initial commit of independent AMD compiler')
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
2019-11-15 17:36:21 +00:00
Rhys Perry
d7b0d9a8d8 radv: enable FP16/FP64 denormals earlier and only for LLVM
ACO sets this itself and will have to set it differently in the future to
support shaderDenormFlushToZeroFloat64.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
2019-11-15 17:36:21 +00:00
Samuel Pitoiset
41a1152cdc radv: make sure to not clear the ds attachment after resolves
To not overwrite the resolve if there is pending clear aspects,
same as color resolves.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-15 09:36:43 +01:00
Samuel Pitoiset
519d9b30de radv: remove useless RADV_DEBUG=unsafemath debug option
This option is useless and shouldn't be used at all.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-15 09:07:34 +01:00
Timur Kristóf
9b8dc6929e aco: Optimize out trivial code from uniform bools.
This should remove most of the excess code size that was
introduced by making all booleans per-lane.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
2019-11-14 17:27:11 +01:00
Timur Kristóf
8995c0b30a aco: Treat all booleans as per-lane.
Previously, instruction selection had two kinds of booleans:
1. divergent which was per-lane and stored in s2 (VCC size)
2. uniform which was stored in s1
Additionally, uniform booleans were made per-lane when they resulted
from operations which were supported only by the VALU.

To decide which type was used, we relied on the destination size,
which was not reliable due to the per-lane uniform bools, but it
mostly works on wave64.
However, in wave32 mode (where VCC is also s1) this approach
makes it impossible keep track of which boolean is uniform and
which is divergent.

This commit makes all booleans per-lane.
The resulting excess code size will be taken care of by the optimizer.

v2 (by Daniel Schürmann):
- Better names for some functions
- Use s_andn2_b64 with exec for nir_op_inot
- Simplify code due to using s_and_b64 in bool_to_scalar_condition

v3 (by Timur Kristóf):
- Fix several subgroups regressions

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
2019-11-14 17:27:11 +01:00
Daniel Schürmann
a1622c1a11 aco: use s_and_b64 exec to reduce uniform booleans to one bit
Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
2019-11-14 17:27:10 +01:00
Timur Kristóf
94e355148f aco: Make sure not to mistakenly propagate 64-bit constants.
ACO's optimizer would try to propagate 64-bit constants, but
does so in such a way that wouldn't work due to how the 64-bit
constants are handled in the IR.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
2019-11-14 17:27:10 +01:00
Daniel Schürmann
9d3e070524 aco: value number instructions using the execution mask
This patch tries to give instructions with the same execution
mask also the same pass_flags and enables VN for SALU instructions
using exec as Operand.
This patch also adds back VN for VOPC instructions and removes VN for phis.

v2 (by Timur Kristóf):
- Fix some regressions.
v3 (by Daniel Schürmann):
- Fix additional issues

Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
2019-11-14 17:27:10 +01:00
Daniel Schürmann
8657eede8a aco: check if SALU instructions are predeceeded by exec when calculating WQM needs
Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
2019-11-14 17:27:10 +01:00
Samuel Pitoiset
ee9811a0bb ac: fix build with recent LLVM
Build is broken since "Move CodeGenFileType enum to Support/CodeGen.h".

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
2019-11-14 14:41:55 +00:00
Rhys Perry
6914b0236f aco: combine read_invocation and shuffle implementations
They do mostly the same thing now.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
2019-11-12 17:21:38 +00:00
Rhys Perry
2c98d79d11 aco: don't propagate vgprs into v_readlane/v_writelane
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 93c8ebfa ('aco: Initial commit of independent AMD compiler')
2019-11-12 17:21:38 +00:00
Rhys Perry
5a1bacb6f9 aco: fix read_invocation with VGPR lane index
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 93c8ebfa ('aco: Initial commit of independent AMD compiler')
2019-11-12 17:21:38 +00:00
Rhys Perry
f97d933426 aco: fix shuffle with uniform operands
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 93c8ebfa ('aco: Initial commit of independent AMD compiler')
2019-11-12 17:21:38 +00:00
Rhys Perry
3204e83768 aco: use DPP instead of exec modification when lowering GFX10 shuffles
Seems we can use DPP's row_mask field to get an effect similar to
modifying exec.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
2019-11-12 17:21:38 +00:00
Daniel Schürmann
746b9380bd aco: rematerialize s_movk instructions
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
2019-11-12 15:59:48 +00:00
Daniel Schürmann
b6f5085dfe aco: preserve kill flag on moved operands during RA
Fixes: 93c8ebfa78 aco: Initial commit of independent AMD compiler

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
2019-11-12 15:59:48 +00:00
Daniel Schürmann
a2a6880743 aco: fix invalid access on Pseudo_instructions
Fixes: 93c8ebfa78 aco: Initial commit of independent AMD compiler

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
2019-11-12 15:59:48 +00:00
Samuel Pitoiset
bef7b2f805 ac: handle pointer types to LDS in ac_get_elem_bits()
This fixes crashes with some
dEQP-VK.spirv_assembly.instruction.spirv1p4.* tests.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
2019-11-12 08:32:15 +01:00
Luis Mendes
0cb5c96a83 radv: fix radv secure compile feature breaks compilation on armhf EABI and aarch64
__NR_select is not defined the same way across architectures, sometimes is
not even defined, like in armhf EABI and aarch64.

Signed-off-by: Luis Mendes <luis.p.mendes@gmail.com>

Acked-by: Timothy Arceri <tarceri@itsqueeze.com>

Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>

Closes: https://gitlab.freedesktop.org/mesa/mesa/issues/2042
2019-11-12 11:47:20 +11:00
Rhys Perry
de998d3eb5 radv: fix radv_nir_get_max_workgroup_size when nir=NULL
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: 84a1a2578 ('compiler: pack shader_info from 160 bytes to 96 bytes')
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
2019-11-11 20:44:12 +00:00
Timur Kristóf
911a826141 ac: Handle invalid GFX10 format correctly in ac_get_tbuffer_format.
It happens that some games try to access a vertex buffer without
a valid format. This case was incorrectly handled by
ac_get_tbuffer_format which made ACO emit an invalid instruction.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Cc: 19.3 <mesa-stable@lists.freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
2019-11-08 13:30:30 +01:00
Rhys Perry
78e3ea9a0f aco: add Instruction::usesModifiers() and add more checks in the optimizer
No pipeline-db changes.

v2: use early-exit for VOP3

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> (v1)
2019-11-08 00:14:06 +00:00
Rhys Perry
76544f632d radv: adjust loop unrolling heuristics for int64
In particular, increase the cost of 64-bit integer division.

Fixes huge shaders with dEQP-VK.spirv_assembly.type.scalar.i64.mod_geom
, with ACO used for GS this creates shaders requiring a branch with
>32767 dword offset.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-07 23:29:12 +00:00
Samuel Pitoiset
deafe4cc58 radv/gfx10: fix primitive indices orientation for NGG GS
The primitive indices have to be swapped to follow the drawing
order.

This fixes corruption with Overwatch when NGG GS is force enabled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-07 19:21:15 +00:00
Daniel Schürmann
a47e232ccd aco: workaround Tonga/Iceland hardware bug
The workaround got accidentally moved to the wrong place

Fixes: 08d510010b aco: increase accuracy of SGPR limits

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
2019-11-07 09:19:50 +01:00
Samuel Pitoiset
fb07fd4e6c radv: implement VK_EXT_subgroup_size_control
This extension allows to control the subgroup size by allowing a
varying subgroup size and also specifying a required subgroup size.

This implementation only allows to specify a required subgroup
size for compute shaders because there is some caveats with
other shader stages (eg. NGG with geometry shader). This
basically allows apps to use Wave32 for compute shaders.

This extension is enabled for all chips but only GFX10 supports
Wave32. ACO doesn't support it.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-06 09:20:39 +01:00
Samuel Pitoiset
da6c30f9f6 radv: rely on shader's wavesize when computing NGG info
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-06 09:20:36 +01:00
Samuel Pitoiset
d3f9957de4 radv: determine shaders wavesize at pipeline level
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-06 09:20:34 +01:00
Samuel Pitoiset
d1e1f7c4d5 radv: hardcode the number of waves for the GFX6 LS-HS bug
It's always 64.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-06 09:20:32 +01:00
Samuel Pitoiset
f010b90ac5 radv/gfx10: enable wave32 for compute based on shader's wavesize
This will allow to change wavesize on-demand.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-06 09:20:30 +01:00
Samuel Pitoiset
0b3bd1a7c2 radv: fix 32-bit compiler warnings
Closes: https://gitlab.freedesktop.org/mesa/mesa/issues/2031
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-06 08:00:33 +01:00
Samuel Pitoiset
50b3ec35d2 radv: add a note about perftest/debug options
Now that all environment variables are documented, it would be
appreciated if we can keep this up-to-date.

[skip ci]

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-06 07:58:33 +01:00
Eric Engestrom
2f652e0b36 meson: move the generic symbols check arguments to a common variable
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviwed-by: Dylan Baker <dylan@pnwbakers>
2019-11-05 20:30:47 +00:00
Eric Engestrom
2c4395e61c meson: add variable to control the symbols checks
Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Reviwed-by: Dylan Baker <dylan@pnwbakers>
2019-11-05 20:12:32 +00:00
Dylan Baker
ee4f1bc187 util: rename PIPE_ARCH_*_ENDIAN to UTIL_ARCH_*_ENDIAN
As requested by Tim.

This was generated with:
grep 'PIPE_ARCH_.*_ENDIAN' -rIl | xargs sed -ie 's@PIPE_ARCH_\(.*\)_ENDIAN@UTIL_ARCH_\1_ENDIAN@'g

v2: - add this patch

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-11-05 16:39:55 +00:00
Dylan Baker
f9f60da813 util/u_endian: set PIPE_ARCH_*_ENDIAN to 1
This will allow it to be used as a drop in replacement for
_mesa_little_endian in a number of cases.

v2: - Always define PIPE_ARCH_LITTLE_ENDIAN and PIPE_ARCH_BIG_ENDIAN,
      define the one that reflects the host system to 1 and the other to 0
    - replace all uses of #ifdef, #ifndef, and #if defined() with #if
      and #if ! with PIPE_ARCH_*_ENDIAN

Reviewed-by: Eric Engestrom <eric@engestrom.ch>
2019-11-05 16:39:55 +00:00
Leo Liu
352b57d709 ac: add missing Arcturus to the info of pc lines
Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Cc: Marek Olšák <marek.olsak@amd.com>
2019-11-04 16:27:35 -05:00
Daniel Schürmann
efe737fc4f aco: fix accidential reordering of instructions when scheduling
Fixes: 8678699918 "aco: implement VGPR spilling"

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
2019-11-04 20:14:14 +01:00
Daniel Schürmann
5c7dcb15e0 aco: only use single-dword loads/stores for spilling
Fixes: 8678699918 "aco: implement VGPR spilling"

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
2019-11-04 20:14:14 +01:00
Daniel Schürmann
d97c0bdd55 aco: fix immediate offset for spills if scratch is used
Fixes: 8678699918 "aco: implement VGPR spilling"

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
2019-11-04 20:14:14 +01:00
Samuel Pitoiset
9ab27647ff radv: fix compute pipeline keys when optimizations are disabled
If an app first creates a compute pipeline with
VK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT set, then re-compile it
without that flag, the driver should re-compile the compute shader.
Otherwise, it will return the unoptimized one.

Fixes: ce188813bf ("radv: add initial support for VK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
2019-11-04 08:50:00 +01:00
Bas Nieuwenhuizen
8efb8f55a6 radv: Close all unnecessary fds in secure compile.
The seccomp filter allows read/write, let us make sure nobody can
do anything with this.

Fixes: cff53da374 "radv: enable secure compile support"
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
2019-11-01 17:15:34 +01:00
Eric Engestrom
a2eba4b17d radv: drop unnecessary xmlpool_options_h
idep_xmlconfig already covers that

Signed-off-by: Eric Engestrom <eric.engestrom@intel.com>
Acked-by: Dylan Baker <dylan@pnwbakers.com>
2019-10-31 15:29:06 +00:00
Bas Nieuwenhuizen
6ced684e27 radv: Fix disk_cache_get size argument.
Got some int->pointer warnings and 20 is not a valid pointer ....

Fixes: 2e3a635ee6 "radv: Add an early exit in the secure compile if we already have the cache entries."
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
2019-10-31 11:40:43 +01:00