Commit graph

6939 commits

Author SHA1 Message Date
Timur Kristóf
d5a79dbcf8 ac/llvm: Add constant offset to load/store_shared.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
00c014aab2 ac/llvm: Make sure to always emit integer comparison for nir_op_ieq.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
252b5d5ecd ac/llvm: Make shared loads/stores work correctly for non-CS stages.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
6e7b1cd251 ac/llvm: Implement new Geometry Shader intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
20a2801011 ac/llvm: Implement the new tessellation intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
60114f3865 ac/llvm: Implement AMD-specific buffer load/store intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
582229585b aco: Implement new Geometry Shader intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
5c95b32c6e aco: Implement the new tessellation I/O related NIR intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
e10e74a7af aco: Implement new buffer load/store intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Rhys Perry
c580c3f9c7 aco/tests: add test for waNsaCannotFollowWritelane
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>
2021-03-17 12:31:05 +00:00
Rhys Perry
502a073552 aco: fix NSA following writelane
No fossil-db changes on GFX10.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: c353895c92 ("aco: use non-sequential addressing")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>
2021-03-17 12:31:05 +00:00
Rhys Perry
298d400e5c aco/tests: add test for NSAToVMEMBug
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>
2021-03-17 12:31:05 +00:00
Rhys Perry
194f3e4c69 aco: fix NSA MIMG followed by MUBUF/MTBUF
No fossil-db changes on GFX10.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: c353895c92 ("aco: use non-sequential addressing")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>
2021-03-17 12:31:05 +00:00
Samuel Pitoiset
7bdd569d7e radv: extend the dirty bits to 64-bit
For future work.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9603>
2021-03-17 09:21:38 +00:00
Samuel Pitoiset
9d42e71505 radv: add notccompatcmask debug option
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>
2021-03-17 08:55:16 +00:00
Samuel Pitoiset
f810d8cd50 radv: enable TC-compat CMASK on GFX10+
Untested on older chips. Should help MSAA games by 1-2%.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>
2021-03-17 08:55:16 +00:00
Samuel Pitoiset
72b58c4a5e radv: initialize TC-compat CMASK images with the DCC clear code
0xff is for fast-clears without MSAA.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>
2021-03-17 08:55:16 +00:00
Samuel Pitoiset
a3835964ce radv: only configure the CMASK tiling for TC-compat on GFX8
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>
2021-03-17 08:55:15 +00:00
Samuel Pitoiset
b9222cc906 radv: make sure FMASK is enabled for TC-compat CMASK
Otherwise it makes no sense at all.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>
2021-03-17 08:55:15 +00:00
Samuel Pitoiset
0c281283d8 radv: remove redundant check when enabling TC-compat CMASK
We already check that the image has CMASK.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>
2021-03-17 08:55:15 +00:00
Samuel Pitoiset
d173ed2e9c radv: do not enable TC-compat CMASK if the image isn't readable by a shader
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9415>
2021-03-17 08:55:15 +00:00
Samuel Pitoiset
fb648634b0 radv: use common entrypoints for VK_KHR_copy_commands2
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9600>
2021-03-17 07:43:36 +00:00
Iago Toral Quiroga
1e4abf1fe3 vulkan/util: call glsl_type_singleton_init_or_ref from vk_instance_init
v2: link libvulkan_util with libglsl so it can find the glsl singleton symbols.
v3: link with libcompiler instead of libglsl (Jason)

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Dylan Baker <dylan.c.baker@intel.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> for the v3dv bits.
Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com> for the turnip bits.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> for the radv bits.
Acked-by: Dave Airlie <airlied@redhat.com> for the lvp bits.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9457>
2021-03-17 08:15:36 +01:00
Marek Olšák
fdbcb58c06 ac/llvm: handle demote in LLVM 13 that just added support for it
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9362>
2021-03-17 00:42:27 +00:00
Timur Kristóf
8205cce007 aco: Use ASSERTED to avoid unused variable warning.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9632>
2021-03-16 21:46:52 +00:00
Michel Dänzer
d411691965 aco/tests: Use _exit in child process
Since the child process doesn't call exec(), exit() attempted to run
atexit handlers registered by the parent process. This could result in
the child process hanging in exit() if there were still disk cache
threads alive when the parent process called fork(). (The CI runners
hit this multiple times when running tests in strace)

Fixes: 6a246f5c6d "aco/tests: Fix deadlock for too large test lists"
Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de>
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9472>
2021-03-16 15:32:33 +00:00
Bas Nieuwenhuizen
013aa05edb radv: Use correct DCC compressed block size for sampling.
Don't need to change the pre-GFX9 samplers because this option doesn't
vary there.

Fixes: f848f2adfa ("radv: Use ac_surface DCC settings for shareable images.")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4455
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4425
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9611>
2021-03-16 00:50:11 +00:00
Mike Blumenkrantz
1d1c7a965c radv: use common interfaces for shader modules
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9508>
2021-03-15 21:47:44 +00:00
Samuel Pitoiset
321dadf229 radv: rework radv_use_dcc_for_image() a bit
To make it clear that only GFX8-9 have missing DCC features.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9526>
2021-03-15 13:22:44 +01:00
Samuel Pitoiset
9704ed3f57 radv: remove useless DCC disable check for 3D images on GFX10+
addrlib uses the S swizzle mode which disables DCC completely.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9526>
2021-03-15 13:22:44 +01:00
Samuel Pitoiset
2bc51226e1 radv: add missing SQTT events for copy_commands2/create_renderpass2
A bunch of entrypoints were missing.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9560>
2021-03-15 12:34:39 +01:00
Samuel Pitoiset
1aabfc77d3 radv: fix initialization of disable_compression when clearing color image
Just make sure it's initialized properly.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9473>
2021-03-12 16:39:21 +00:00
Samuel Pitoiset
15fc0c351a radv: fix potential clears with non renderable images on GFX9+
Found by inspection.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9474>
2021-03-12 16:09:42 +00:00
Samuel Pitoiset
9863ed9bf3 radv: fix meta save/restore state with non renderable images
For non renderable images, the driver performs some transfer operations
with compute shaders on the gfx queue, but it was saving the gfx state
instead of the compute state.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9474>
2021-03-12 16:09:42 +00:00
Samuel Pitoiset
b42204e4d0 radv: disable sampling with VK_FORMAT_R64_SFLOAT
It's not supported.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4433
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9519>
2021-03-12 16:07:54 +01:00
Rhys Perry
38b2e13766 aco: remove vmem/smem score statistics
Replaced by the Latency statistic.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8994>
2021-03-11 16:31:19 +00:00
Rhys Perry
a0243f5c47 aco: add ACO_DEBUG=perfinfo
This prints the program with each instruction's contribution to it's
latency and various factors for the calculation of the Inverse Throughput
statistic.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8994>
2021-03-11 16:31:19 +00:00
Rhys Perry
5d6a1095bf aco: add print option to print program without temporary IDs
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8994>
2021-03-11 16:31:19 +00:00
Rhys Perry
23ecceb160 aco: add latency and inverse throughput statistics
Latency is estimanted duration of a single wave, ignoring others in the
CU. It is similar to the old cycles statistic except it it's more accurate
and considers memory operations.

The InvThroughput statistic is a combination of MaxWaves, Latency and the
portion of the wave's execution which does not use various resources.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8994>
2021-03-11 16:31:19 +00:00
Rhys Perry
83ce9407f2 aco: add instruction classes
These should mostly match LLVM.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8994>
2021-03-11 16:31:19 +00:00
Rhys Perry
0af7ff49fd aco: lower p_constaddr into separate instructions earlier
This allows them to be scheduled properly and simplifies the assembler a
little.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8994>
2021-03-11 16:31:19 +00:00
Rhys Perry
ab957bb899 aco: move wait_imm to aco_ir.h
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8994>
2021-03-11 15:35:34 +00:00
Rhys Perry
7d5643c0fe aco: track divergent and uniform branch depth
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8994>
2021-03-11 15:35:30 +00:00
Rhys Perry
8f71be0a7b aco: simplify loop_nest_depth tracking in isel
Keep track of the current loop depth in Program and set the depth inside
Program::insert_block() instead of repeating it every time we insert one.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8994>
2021-03-11 15:35:24 +00:00
Rhys Perry
35fe62dad1 radv/llvm: fix enabled_channels for compressed exports
The old values seemed to work fine, but the ISA docs recommend 0x0,0x3,0xc
and 0xf:

COMPR==1: export half-dword enable. Valid values are: 0x0,3,c,f
[0] enables VSRC0 : R,G from one VGPR (R in low bits, G high)
[2] enables VSRC1 : B,A from one VGPR (B in low bits, A high)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9459>
2021-03-11 13:54:18 +00:00
Rhys Perry
341dd9d834 aco: set compr for fp16 exports
Obviously this didn't affect correctness. Not sure about performance.

It also changes enabled_channels to match radeonsi.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: f29c81f863 ("aco: use VOP2 for v_cvt_pkrtz_f16_f32 if possible")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9459>
2021-03-11 13:54:18 +00:00
Jason Ekstrand
5d8fa880d6 radv: Drop CreateRenderPass
We can use the generic fall-back which calls CreateRenderPass2 instead.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8857>
2021-03-10 18:17:31 +00:00
Jason Ekstrand
8304b4eef7 radv/meta: Use CreateRenderPass2
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8857>
2021-03-10 18:17:31 +00:00
Jason Ekstrand
4fb6c051c9 anv: Move vk_format helpers to common code
The Android ones we put in anv_android.c.  Maybe one day we'll want a
vk_android.h to put some common Android stuff but, for now, let's keep
it contained to ANV's android code.

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8857>
2021-03-10 18:17:31 +00:00
Marek Olšák
3b7b2df509 ac: remove switch cases for pc_lines for compute-only chips
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9389>
2021-03-10 18:02:28 +00:00