Commit graph

12146 commits

Author SHA1 Message Date
Qiang Yu
f9d54b1d36 ac/llvm,radeonsi: lower idiv in nir
aco does not implement these idiv ops.

nir_lower_idiv is for idiv ops <= 32bit and ported from
llvm amdgpu, so llvm do the same.

nir_lower_divmod64 is for 64bit idiv ops.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
4b90347595 ac/binary: pack prefech align code to a function
To be used by radeonsi raw shader binary.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
5bc6c62486 meson: build radeonsi with aco
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Konstantin Seurer
1671b5fd1f radv: Stop running constant folding during ray query lowering
Now that committed is an intrinsic index, there is no need for constant
folding.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22963>
2023-05-14 17:28:40 +00:00
Konstantin Seurer
0cf22f9af3 nir: Make rq_load committed src an index
committed has to be a constant so there is no need to have a src and
depend on constant folding to remove the i2b.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22963>
2023-05-14 17:28:40 +00:00
Marek Olšák
f98871608c ac/llvm: rewrite and unify how GLC, DLC, SLC are set
Use ACCESS_* flags in call sites instead of GLC/DLC/SLC.

ACCESS_* flags are extended to describe other aspects of memory instructions
like load/store/atomic/smem.

Then add a function that converts the access flags to GLC, DLC, SLC.

The new functions are also usable by ACO.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22770>
2023-05-12 21:45:44 +00:00
Marek Olšák
968db0208d ac/llvm: don't treat ACCESS_NON_READABLE as ACCESS_COHERENT
... and expect it to behave like ACCESS_NON_TEMPORAL.
Handling ACCESS_NON_TEMPORAL is sufficient.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22770>
2023-05-12 21:45:44 +00:00
Marek Olšák
3a043b5739 aco: don't treat ACCESS_NON_READABLE as ACCESS_COHERENT
... and expect it to behave like ACCESS_NON_TEMPORAL.
Handling ACCESS_NON_TEMPORAL is sufficient.

This was copied from ac_nir_to_llvm.c.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22770>
2023-05-12 21:45:44 +00:00
Alyssa Rosenzweig
ce638eafe2 aco,radv: Use unified atomics
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>
2023-05-12 20:39:46 +00:00
Alyssa Rosenzweig
0241d8894e ac/llvm: Use unified atomics
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>
2023-05-12 20:39:46 +00:00
Alyssa Rosenzweig
1e9c01523a ac/llvm: Don't handle atomic derefs
Should not be seen, already would be stubbed out.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Suggested-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>
2023-05-12 20:39:46 +00:00
Rhys Perry
88f6d7f4bd aco/gfx11: fix VMEM/DS->VALU WaW/RaW hazard
Previously, we could safely read/write unused lanes of VMEM/DS
destination VGPRs without waiting for the load to finish. That doesn't
seem to be the case on GFX11.

fossil-db (gfx1100):
Totals from 6698 (4.94% of 135636) affected shaders:
Instrs: 11184274 -> 11199420 (+0.14%); split: -0.00%, +0.14%
CodeSize: 57578344 -> 57638928 (+0.11%); split: -0.00%, +0.11%
Latency: 198348808 -> 198382472 (+0.02%); split: -0.00%, +0.02%
InvThroughput: 24376324 -> 24378439 (+0.01%); split: -0.00%, +0.01%
VClause: 192420 -> 192559 (+0.07%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8722
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8239
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22965>
2023-05-12 20:19:21 +00:00
Alyssa Rosenzweig
82430b91bb ac/llvm: Use nir_foreach_phi
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22967>
2023-05-12 14:02:23 +00:00
Georg Lehmann
04976beac7 aco: don't apply dpp if the alu instr uses the operand twice
CP77 has a ton of fma(dpp(a), dpp(a), b).

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>
2023-05-12 13:31:16 +00:00
Georg Lehmann
151bcc1e8b aco: use VOP3+DPP
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>
2023-05-12 13:31:16 +00:00
Georg Lehmann
41b0eafc4b aco/ra: convert VOPC_DPP instructions without vcc to VOP3
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>
2023-05-12 13:31:16 +00:00
Georg Lehmann
d32526caf2 aco: add assembler tests for VOP3(P) with DPP
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>
2023-05-12 13:31:15 +00:00
Georg Lehmann
984bdc0fb1 aco/builder: support VOP3(P) with dpp
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>
2023-05-12 13:31:15 +00:00
Georg Lehmann
2548f28ab3 aco/assembler: support VOP3P with DPP
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>
2023-05-12 13:31:15 +00:00
Georg Lehmann
d0e73cb313 aco/optimizer: copy pass flags for newly created valu instructions
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>
2023-05-12 13:31:15 +00:00
Georg Lehmann
d27e03d719 aco/optimizer: don't use pass_flags for mad idx
fma can use DPP on GFX11+, so we want to keep the exec id in pass_flags

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>
2023-05-12 13:31:15 +00:00
Georg Lehmann
7c5e8dedab aco/vn: compare all valu modifers
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: 9b4ea9ff90 ("aco/vn: hash opsel for VOP12C")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>
2023-05-12 13:31:15 +00:00
Bas Nieuwenhuizen
0a87f1b7ed amd/drm-shim: Update docs for more devices.
We don't have to update the docs for every new entry.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22975>
2023-05-12 07:44:32 +00:00
Bas Nieuwenhuizen
b96aa6b9e2 amd/drm-shim: Add raphael&mendocino, polaris12 and gfx1100.
Decided to follow the chip names pretty much.This set happens to be
what is in my workstation currently.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22975>
2023-05-12 07:44:32 +00:00
Bas Nieuwenhuizen
d16cd4c758 amd/drm-shim: Add vangogh entry.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22975>
2023-05-12 07:44:32 +00:00
Samuel Pitoiset
088e25216f radv: configure PA_CL_VRS_CNTL entirely from the cmd buffer
We already have all the information needed to configure it.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22942>
2023-05-12 06:39:11 +00:00
Samuel Pitoiset
be0ba9a1c0 radv: re-emit fragment shading rate state when PA_CL_VRS_CNTL changes
Found by inspection.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22942>
2023-05-12 06:39:11 +00:00
Samuel Pitoiset
041cf2d48e radv: dirty the dynamic vertex input state only when needed
This shouldn't be necessary when the VS doesn't have a prolog.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22944>
2023-05-12 06:20:02 +00:00
Samuel Pitoiset
8688e11013 radv: reset the emitted VS prolog when a new vertex shader is bound
When a new vertex shader is bound, the VS prolog needs to be
re-emitted, and this allows us to avoid tracking if the pipeline is
dirty.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22944>
2023-05-12 06:20:02 +00:00
Chia-I Wu
6aee7848bb radv: improve externalMemoryFeatures for android ahb
VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT should always be set, as
required by the spec.

VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT should be set when
radv_ahb_format_for_vk_format knowns the format.  That is,
radv_create_ahb_memory should at least know how to call
AHardwareBuffer_allocate.

VK_EXTERNAL_MEMORY_FEATURE_IMPORTABLE_BIT is always set.  We can't know
if gralloc can allocate the format/flags/usage combo or not (gralloc
might use a private format for the combo).

Fixed
dEQP-VK.api.external.memory.android_hardware_buffer.image_formats.*.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22619>
2023-05-11 22:18:03 +00:00
Chia-I Wu
47b37651f8 vulkan: add vk_image_format_to_ahb_format
There should be no functional change.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22619>
2023-05-11 22:18:03 +00:00
Chia-I Wu
380180516c anv,hasvk,radv: do not fall back to AHARDWAREBUFFER_FORMAT_BLOB
When allocating a VkDeviceMemory exportable as AHB, it seems incorrect
to fall back to AHARDWAREBUFFER_FORMAT_BLOB when the image has no known
AHB format.  We should fail the allocation instead.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22619>
2023-05-11 22:18:03 +00:00
Chia-I Wu
50e703f347 vulkan: add vk_ahb_format_to_image_format
There should be no functional change.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22619>
2023-05-11 22:18:02 +00:00
Chia-I Wu
2bbe0462e8 vulkan: define inline stubs when android api level < 26
This allows us to reduce ANDROID #ifdef's.

v2: always include vk_android.h in radv_formats.c

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22619>
2023-05-11 22:18:02 +00:00
Chia-I Wu
f81dce9bcc vulkan: rename vk_image::ahardware_buffer_format
Rename it to ahb_format.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22619>
2023-05-11 22:18:02 +00:00
Leo Liu
ffbbf23ef8 radeonsi: Use vcn version instead of CHIP family for VCNs
Decouple it from CHIP family, based on HW query infomation.

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22904>
2023-05-11 18:01:10 +00:00
Leo Liu
09e59553ec amd: Add vcn ip version info
And make it support for kernel w/wo ip_discovery.

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22904>
2023-05-11 18:01:10 +00:00
Leo Liu
82a064020c radeonsi: Remove redundant vcn_decode from info
Use the number of queue instead.

Signed-off-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22904>
2023-05-11 18:01:10 +00:00
Georg Lehmann
2a1e6a140d aco: also reassign p_extract_vector post ra
Foz-DB Navi21:
Totals from 1223 (0.91% of 134864) affected shaders:
CodeSize: 6923888 -> 6913516 (-0.15%)
Instrs: 1293744 -> 1291151 (-0.20%)
Latency: 16928653 -> 16925035 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 2985304 -> 2984775 (-0.02%); split: -0.02%, +0.00%
VClause: 32260 -> 32319 (+0.18%)
SClause: 54952 -> 54949 (-0.01%)
Copies: 83968 -> 81377 (-3.09%)

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22821>
2023-05-11 10:26:24 +00:00
Georg Lehmann
c1cf40da8a aco: Assert that operands have the same byte offset when reassigning split vectors
This can not happen because the post-RA optimizer doesn't support sub dword
writes at the moment, but everytime I look at this I wonder if there might
be a bug here.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22821>
2023-05-11 10:26:24 +00:00
Daniel Schürmann
d3f06cf5ce vulkan/pipeline_cache: don't log warnings for internal caches
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22850>
2023-05-11 08:36:12 +00:00
Chia-I Wu
df387306d6 amd/drm-shim: add amdgpu drm-shim
This is enough to run offscreen apps such as vulkaninfo or deqp-vk.

v2: remove unnecessary idep_amdgfxregs_h dependency

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21892>
2023-05-11 00:58:02 +00:00
Jiadong Zhu
3cfdcabc78 ac: enable SHADOW_GLOBAL_CONFIG for preemptible ib
SHADOW_GLOBAL_CONFIG is mandatory for mid command buffer preemmption.

Fixes: 69014d8c94 (radeonsi: implement CP register shadowing)
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22916>
2023-05-10 17:11:19 +00:00
Timur Kristóf
dc676ed711 aco: Initialize vcmpx field in get_cmp_info.
Fixes: 578d0a1934
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22885>
2023-05-10 12:41:25 +00:00
Timur Kristóf
16a05f1903 aco: Don't allow any VALU instruction to write m0.
Fixes: d5398b62da
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22885>
2023-05-10 12:41:25 +00:00
Samuel Pitoiset
8d5003d91a radv/ci: stop setting MESA_SPIRV_LOG_LEVEL
Use the default VTN logging level which is warning.

Suggested-by: Emma Anholt <emma@anholt.net>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6263
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Martin Roukala <martin.roukala@mupuf.org>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22917>
2023-05-10 10:32:47 +00:00
Simon Ser
31e6d15801 radv: advertise LINEAR filter support for multiplanar/subsampled
It seems like radv supports this but doesn't advertise it.

Signed-off-by: Simon Ser <contact@emersion.fr>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22875>
2023-05-10 10:02:27 +00:00
Daniel Schürmann
1e334e9818 radv/rt: store stack_sizes per stage instead of per group
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22503>
2023-05-10 07:02:13 +00:00
Daniel Schürmann
f50a5a62bf radv/rt: use vk_multialloc for radv_ray_tracing_pipeline
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22503>
2023-05-10 07:02:13 +00:00
Daniel Schürmann
dfa5fd480c radv/rt: refactor radv_rt_pipeline_compile()
This patch moves the NIR shader creation into radv_rt_pipeline_compile()
and simplifies radv_rt_pipeline_create().

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22503>
2023-05-10 07:02:13 +00:00