Commit graph

5423 commits

Author SHA1 Message Date
Rhys Perry
4fc0499049 aco: remove outdated assert in handle_operands()
"target" is no longer expected to be completely inside "swap".

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5626>
2020-06-24 20:38:35 +00:00
Rhys Perry
7cad27831d aco: ignore blocked registers when checking edges in get_reg_impl()
If the only two registers available are consecutive and used by killed
operands, both of them will be blocked and fail the edge check.

Totals from 903 (0.66% of 135946) affected shaders:
VGPRs: 30892 -> 30884 (-0.03%)
CodeSize: 1584468 -> 1584044 (-0.03%); split: -0.05%, +0.02%
MaxWaves: 14374 -> 14378 (+0.03%)
Instrs: 306482 -> 306399 (-0.03%); split: -0.06%, +0.03%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5626>
2020-06-24 20:38:35 +00:00
Samuel Pitoiset
91a82d0069 radv: fix checking the return value of cs_finalize()
cs_finalize() now returns a Vulkan error code and VK_SUCCESS is 0.

Fixes: 64a92ef7a2 ("radv/winsys: Distinguish device/host memory errors.")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5629>
2020-06-24 15:50:06 +02:00
Bas Nieuwenhuizen
aa35670fd0 radv: Make radv_alloc_shader_memory static.
Just a cleanup.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5578>
2020-06-24 13:00:02 +00:00
Bas Nieuwenhuizen
64a92ef7a2 radv/winsys: Distinguish device/host memory errors.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5578>
2020-06-24 13:00:02 +00:00
Bas Nieuwenhuizen
a5cb88eea4 radv: Handle mmap failures.
Which can happen if we have to many mmaps active in the process.

CC: <mesa-stable@lists.freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5578>
2020-06-24 13:00:02 +00:00
Bas Nieuwenhuizen
04765e6a9a radv/winsys: Deal with realloc failures in BO lists.
Otherwise if realloc fails we silently try to use it. Make recording fail instead.

CC: <mesa-stable@lists.freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5578>
2020-06-24 13:00:02 +00:00
Rhys Perry
519ddfd312 aco: improve vectorization of 8/16-bit loads/stores
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:28 +00:00
Rhys Perry
ddffcf3627 aco: fix when sub-dword create_vector operand cannot be placed perfectly
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:28 +00:00
Daniel Schürmann
91fd53884d aco: don't allow partial copies on GFX6/7
These are not supported due to missing SDWA instructions

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:28 +00:00
Daniel Schürmann
76b5d72921 aco: align swap operations to 4 bytes on GFX6/7
GFX6/7 can only swap full registers

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:28 +00:00
Rhys Perry
91d7e40176 aco: don't create byte-aligned short loads
The ISA docs don't seem to say if this is allowed, so just assume short
loads require short alignment.

In practice, the only situation this should affect are byte-aligned u8vec2
loads.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:28 +00:00
Rhys Perry
c3259b6e6a aco: add missing bld.scc() in byte_align_scalar()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:28 +00:00
Rhys Perry
a0f6ca4393 aco: don't store byte-aligned short stores
The ISA docs don't seem to say if this is allowed, so just assume short
stores require short alignment.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:28 +00:00
Rhys Perry
a18da83d18 aco: fix copy+paste error in split_buffer_store
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:28 +00:00
Rhys Perry
841fdfcd45 radv/aco,aco: allow SMEM SSBO loads on GFX6/7
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:28 +00:00
Rhys Perry
35b5e1fc7c aco: allow SMEM for some sub-dword accesses
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:27 +00:00
Rhys Perry
c702f8ed15 aco: only use SMEM if we can prove it's safe
Totals from 26 (0.02% of 127638) affected shaders:
SGPRs: 1680 -> 1664 (-0.95%)
VGPRs: 1492 -> 1504 (+0.80%)
CodeSize: 233140 -> 233016 (-0.05%); split: -0.09%, +0.04%
Instrs: 47121 -> 47114 (-0.01%); split: -0.08%, +0.06%
VMEM: 4930 -> 4655 (-5.58%); split: +0.12%, -5.70%
SMEM: 2030 -> 2001 (-1.43%); split: +3.79%, -5.22%
VClause: 891 -> 947 (+6.29%)
SClause: 876 -> 816 (-6.85%)
Copies: 4734 -> 4716 (-0.38%); split: -0.40%, +0.02%
Branches: 2048 -> 2047 (-0.05%)
PreSGPRs: 1400 -> 1396 (-0.29%)
PreVGPRs: 1440 -> 1443 (+0.21%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:27 +00:00
Rhys Perry
0cfee26bee radv: fix image variable types in meta shaders
We write to these variables using image intrinsics.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5207>
2020-06-24 10:52:27 +00:00
Samuel Pitoiset
0aca04afa5 aco: fix printing ASM on GFX6-7 again
Checking errno is actually wrong because it's only updated if
popen() fails (ie. NULL). One solution is to check if the first
line is empty.

Fixes: c95d258d1b ("aco: fix printing ASM on GFX6-7 if clrxdisasm is not found")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5591>
2020-06-23 07:45:03 +00:00
Marek Olšák
2b8b62c55b ac/nir: fix 64-bit division for GL CTS
This fixes: KHR-GL45.gpu_shader_fp64.builtin.mod_*

Fixes: ba2ec1f3 "ac/nir: use llvm.amdgcn.rcp in ac_build_fdiv()"

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5531>
2020-06-23 04:46:55 +00:00
Eric Engestrom
04e8eaf4e8 util: rename xmlpool.h to driconf.h
To make it clearer what it is and does.

Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5440>
2020-06-22 21:50:12 +00:00
Samuel Pitoiset
83d2a73b73 aco: improve validation checks for readlane/writelane
This allows literals for the lane select on GFX10+. The doc says
that is should be a SGPR or a constant but VOP3 on GFX10+ allows
literals.

Some later validation code checks if literals are allowed anyways.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5010>
2020-06-22 11:24:27 +00:00
Daniel Schürmann
f03a5f6cac radv/aco: implement logic64 instead of lowering
to make use of the scalar ALU

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5527>
2020-06-22 10:59:45 +00:00
Samuel Pitoiset
c95d258d1b aco: fix printing ASM on GFX6-7 if clrxdisasm is not found
Fixes some dEQP-VK.pipeline.executable_properties.* which expect
a valid string to be returned.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5560>
2020-06-22 10:38:33 +02:00
Rhys Perry
f4a643f65e radv: add new drirc option radv_no_dynamic_bounds
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Cc: <mesa-stable@lists.freedesktop.org>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5569>
2020-06-19 23:53:47 +00:00
Pierre-Eric Pelloux-Prayer
993c64e6fe ac/llvm: load 1 byte at a time if unaligned on gfx10
If buffer or stride is unaligned we use the same trick as on gfx6:
load 1 byte at a time and recompose the output if needed.
This change fixes lots of deqp/glcts tests:
  - dEQP-GLES2.functional.draw.random.1, 10, ...
  - dEQP-GLES2.functional.vertex_arrays.multiple_attributes.stride.3_float2_0_float2_0_float2_17, ...
  - dEQP-GLES2.functional.vertex_arrays.single_attribute.first.byte_first24_offset1_stride2_quads256, ...
  - dEQP-GLES2.functional.vertex_arrays.single_attribute.strides.buffer_0_17_byte2_vec4_dynamic_draw_quads_1, ...
  - dEQP-GLES31.functional.draw_indirect.random.14, ...

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5502>
2020-06-19 09:20:16 +02:00
Samuel Pitoiset
2ac5cce1a1 radv: require LLVM 11+ for GFX 10.3 if not using ACO
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5389>
2020-06-19 08:18:43 +02:00
Samuel Pitoiset
dc698fb5dc radv: add support for Sienna Cichlid
Bits copied from RadeonSI. Totally untested.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5389>
2020-06-19 08:18:41 +02:00
Samuel Pitoiset
8c144482ea aco: replace == GFX10 with >= GFX10 where it's needed
Assume the GFX10.3 ISA is similar to GFX10 which is likely (except
possible minor changes and new instructions for raytracing).

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5389>
2020-06-19 08:18:39 +02:00
Samuel Pitoiset
3c28438094 radv: replace == GFX10 with >= GFX10 where it's needed
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5389>
2020-06-19 08:18:37 +02:00
Rhys Perry
f7cc7079b0 aco: use the same regclass as the definition for undef phi operands
Subdword phis can't have SGPR operands on GFX6-8.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5544>
2020-06-18 17:29:33 +00:00
Rhys Perry
897a47d847 aco: fix edge check with sub-dword temporaries
Fixes RA failure for a parallel-rdp shader on pitcairn.

fossil-db (Navi):
Totals from 2 (0.00% of 128733) affected shaders:
CodeSize: 203656 -> 205724 (+1.02%)
Instrs: 32267 -> 32529 (+0.81%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5544>
2020-06-18 17:29:33 +00:00
Rhys Perry
365d0aa6c5 aco: shrink mad_info
From 24 bytes to 16 bytes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5281>
2020-06-18 14:26:01 +00:00
Rhys Perry
917260710f aco: make ssa_info::label 64-bit
We'll probably need these extra bits in the future.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5281>
2020-06-18 14:26:01 +00:00
Rhys Perry
47ca84a96d aco: shrink ssa_info
Reorder members so that it's 16 bytes instead of 24.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5281>
2020-06-18 14:26:01 +00:00
Daniel Schürmann
3817fa7a4d aco: fix WQM handling in nested loops
If on a nested loop
- the outer loop needs WQM but
- the inner loop doesn't need WQM and
- the break condition of the inner loop is computed in the outer loop
then it could happen that we transitioned to Exact before entering the inner loop
which could create an empty exec mask and lead to an infinite loop.

Fixes a GPU hang with RDR2

Cc: 20.1 <mesa-stable@lists.freedesktop.org>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5518>
2020-06-18 13:40:15 +00:00
Samuel Pitoiset
fa149b996d radv: only requires LLVM 9 for GFX10 if not using ACO
In case someone links RADV with LLVM 8 and wants to use ACO.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5454>
2020-06-18 09:45:20 +00:00
Samuel Pitoiset
70cc80805c radv: compute CB_SHADER_MASK from the fragment shader outputs
The fragment shader doesn't necessarily output the number of components
expected by the target format.

Fixes new dEQP-VK.draw.output_location.*.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5427>
2020-06-18 09:16:04 +00:00
Samuel Pitoiset
b848d88059 radv: make sure to set CB_SHADER_MASK correctly for internal CB operations
It should be always set to 0xf.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5427>
2020-06-18 09:16:04 +00:00
Samuel Pitoiset
c4aa64b4c3 radv: lower discards to demote to workaround a RDR2 game bug
This fixes some sort of LOD issue.

Cc: 20.1 <mesa-stable@lists.freedesktop.org>
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5525>
2020-06-18 08:39:28 +02:00
Eric Engestrom
405bffefe1 radv: replace all dup() with os_dupfd_cloexec()
Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5369>
2020-06-18 02:09:56 +00:00
Samuel Pitoiset
51fb3b09dc radv/aco: enable FP16 features/extensions on GFX9+
This enables shaderFloat16, VK_AMD_gpu_shader_half_float and
VK_AMD_gpu_shader_int16.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5347>
2020-06-17 18:12:51 +02:00
Rhys Perry
abfe28a6bb aco: fix validation of opsel when set for the definition
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5521>
2020-06-17 15:48:35 +00:00
Rhys Perry
de7c6950b3 aco: fix sub-dword opsel/sdwa checks
These should all check if the operand has a regclass. The opsel check
should also be skipped post-RA.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5504>
2020-06-17 10:57:17 +00:00
Rhys Perry
1e791e51a6 aco: fix validation error from vgpr spill/restore code
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5504>
2020-06-17 10:57:17 +00:00
Daniel Schürmann
8006feda09 aco: don't allow SGPRs on logical phis
aco_validate() is called after phi lowering, now.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5496>
2020-06-16 14:46:19 +01:00
Daniel Schürmann
0e47fe3fa2 aco: reorder calls to aco_validate() and cleanup aco_compile_shader()
The first call of aco_validate should happen after phi lowering.
Otherwise, subdword restrictions might be violated

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5496>
2020-06-16 14:46:19 +01:00
Rob Clark
603d3c2991 radv: don't set num_components for non-vectorized intrinsics
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5371>
2020-06-16 02:48:18 +00:00
Rhys Perry
a02e7f6799 aco: fix encoding of certain s_setreg_imm32_b32 instructions
If the mode is too small, the operand will be an inline constant and the
literal dword won't be written.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5245>
2020-06-15 18:24:22 +00:00