Commit graph

51 commits

Author SHA1 Message Date
Rhys Perry
dfa8ac6b91 aco: remove buffer_load_lds instructions
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
They don't exist

See https://github.com/llvm/llvm-project/pull/132916

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/14041
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37716>
2025-10-07 09:50:26 +00:00
Georg Lehmann
404e1f13e8 aco/print_asm: use real true16 instr on gfx11+
Fake16 doesn't print opsel on v_cndmask_b16, so it looks really broken.
Restrict to LLVM20+ because older versions have incomplete tru16 support.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35919>
2025-07-31 12:07:07 +00:00
Daniel Schürmann
3c27a9f0e2 aco/tests: add more tests for chained branches
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33762>
2025-02-27 10:40:01 +00:00
Daniel Schürmann
7e4687fd04 aco: remove definition from SOPP branch instructions
Totals from 17942 (22.60% of 79395) affected shaders: (Navi31)

Instrs: 20334063 -> 20312676 (-0.11%); split: -0.11%, +0.00%
CodeSize: 108458732 -> 108377540 (-0.07%); split: -0.08%, +0.00%
Latency: 180510540 -> 180479666 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 28079325 -> 28077938 (-0.00%); split: -0.01%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32037>
2024-12-06 14:34:03 +00:00
Daniel Schürmann
cab5639a09 aco/assembler: chain branches instead of emitting long jumps
As regular branch instructions cannot jump further than
32768 dwords, previously we used long jumps as fallback
solution. The disadvantage of that is that an extra SGPR
pair must be provided in order to temporarily store the PC.

This patch changes that to chained branch instructions by
inserting an artificial extra block into the code to be
targeted by the original branch. This block contains a
single branch instruction jumping to the original target.
Before the block, if necessary, we insert a <branch 1>
instruction for the existing code in order to jump over
the newly inserted block.

Only a few RT shaders are affected.

Totals from 29 (0.04% of 79395) affected shaders: (Navi31)

CodeSize: 17281176 -> 17276332 (-0.03%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32037>
2024-12-06 14:34:03 +00:00
Daniel Schürmann
bc2d166b50 aco/lower_to_hw: don't allocate new temporaries
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31362>
2024-10-07 07:00:19 +00:00
Rhys Perry
7f092cbd91 aco: workaround hazards in emit_long_jump
fossil-db (navi31):
Totals from 29 (0.04% of 79395) affected shaders:
CodeSize: 17612888 -> 17615096 (+0.01%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31316>
2024-09-30 09:04:35 +00:00
Rhys Perry
9fb97085d1 aco/tests: update assembler tests for llvm
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31316>
2024-09-30 09:04:35 +00:00
Rhys Perry
00eccf524f aco: use GFX12 scope/temporal-hint
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243>
2024-06-07 13:22:42 +00:00
Rhys Perry
b41f0f6cc1 aco: use ac_hw_cache_flags
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243>
2024-06-07 13:22:42 +00:00
Rhys Perry
e79a8219d2 aco/gfx12: sign-extend s_getpc_b64
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29330>
2024-05-28 10:52:11 +00:00
Rhys Perry
5e58e32832 aco/tests: add GFX12 assembler tests
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29162>
2024-05-14 20:50:27 +00:00
Georg Lehmann
18706947e8 aco/tests: add assembler tests for interp high_16bits
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28435>
2024-04-10 07:49:27 +00:00
Georg Lehmann
af199c6949 aco: swap opsel and wait_exp for vinterp
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28435>
2024-04-10 07:49:26 +00:00
Samuel Pitoiset
7a69d78ba2 aco: use SPDX-License-Identifier
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622>
2024-04-08 15:49:25 +00:00
Daniel Schürmann
a863c7951e aco: remove create_instruction() template parameter
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
2024-03-28 11:25:43 +00:00
Daniel Schürmann
9b0ebcc39b aco: change return type of create_instruction() to Instruction*
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
2024-03-28 11:25:43 +00:00
Daniel Schürmann
5d265257a0 aco: remove SOPP_instruction::block member
Re-use SOPP_instruction::imm instead.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
2024-03-28 11:25:43 +00:00
Rhys Perry
5651aa7644 aco/gfx11: fix scratch ST mode assembly
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27913>
2024-03-22 10:29:43 +00:00
Rhys Perry
6547e17e60 aco: add VOPD format
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367>
2024-02-05 10:51:14 +00:00
Rhys Perry
09eb6e3106 aco/tests: fix tests with LLVM 18
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26092>
2023-11-08 10:40:17 +00:00
Rhys Perry
e4d9f6fb50 aco/tests: fix tests with LLVM 17
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10106
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26092>
2023-11-08 10:40:17 +00:00
Daniel Schürmann
7e4870e8e5 amd: Do shader binary alignment for prefetch at memory allocation time.
This makes it consistent between drivers and compilers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23799>
2023-07-11 12:01:45 +00:00
Daniel Schürmann
437bf4fccb amd: move end-of-code marker padding to ACO.
This makes it consistent between drivers and compilers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23799>
2023-07-11 12:01:45 +00:00
Eric Engestrom
6b21653ab4 aco: reformat according to its .clang-format
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23253>
2023-06-16 19:59:52 +00:00
Marek Olšák
6dc1ae1759 amd: drop support for LLVM 14
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23471>
2023-06-07 19:56:55 +00:00
Georg Lehmann
d32526caf2 aco: add assembler tests for VOP3(P) with DPP
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>
2023-05-12 13:31:15 +00:00
Georg Lehmann
097a97cc42 aco: remove VOP[123C]P? structs
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21023>
2023-03-07 11:53:23 +00:00
Rhys Perry
26e4621fa2 aco/tests: update assembler tests for latest LLVM 16
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20747>
2023-01-23 12:30:28 +00:00
Rhys Perry
b0fa106dc6 aco/tests: fix assembler.gfx11.vop12c_v128 with LLVM 15
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8089
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20747>
2023-01-23 12:30:28 +00:00
Rhys Perry
6872f8d861 aco/gfx11: allow true 16-bit instructions to access v128+
It looks like the LLVM assembler promotes true 16-bit instructions to VOP3
in this case.

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20251>
2023-01-10 16:01:38 +00:00
Rhys Perry
254b178d5b aco: disallow SGPRS/constants with interpolation instructions
https://reviews.llvm.org/D137575

The VINTRP format cannot encode anything except VGPRs.

Reading VINTERPInstructions.td, looks like it's the same for GFX11.

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20251>
2023-01-10 16:01:38 +00:00
Rhys Perry
c3dd1931d9 aco: allow Builder::Result to be dereferenced
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20251>
2023-01-10 16:01:38 +00:00
Rhys Perry
296b4d95a3 aco/gfx11: workaround LdsDirectVALUHazard
fossil-db (gfx1100):
Totals from 57858 (42.85% of 135032) affected shaders:

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18273>
2022-10-19 02:46:03 +00:00
Rhys Perry
7e54fea373 aco: fix assembler.gfx11.vinterp test
This was missed. I guess CI doesn't have a recent enough LLVM for these
tests.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17710>
2022-09-30 20:57:02 +00:00
Rhys Perry
826ed52174 aco/tests: add GFX11 assembly tests
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:57 +00:00
Rhys Perry
fb13ed6ff0 aco: fix long-jump version of discard early exit
It isn't safe to modify the exec mask before the discard block, and the
definition interferes with GFX11 NOP insertion.

Just use s[0:1] instead, since we won't be using it.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18125>
2022-08-25 16:10:53 +00:00
Rhys Perry
dd105f7c1e aco: fix assembly of vopc_sdwa writing exec
We would assemble an instruction writing vcc instead.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: 5ffc73896f ("aco/assembler: Fix v_cmpx with SDWA.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18077>
2022-08-16 17:31:33 +00:00
Rhys Perry
33e7ba2e3e aco: update SMEM offset workaround for LLVM 15
This isn't needed since LLVM 15's b0ccf38b018.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-By: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6663
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17036>
2022-06-16 00:47:51 +00:00
Rhys Perry
4513cb8d41 aco: only add/subtract low bits of program addresses
fossil-db (Sienna Cichlid):
Totals from 4007 (2.47% of 162293) affected shaders:
Instrs: 3733239 -> 3728018 (-0.14%)
CodeSize: 20770340 -> 20749456 (-0.10%)
Latency: 46883958 -> 46872764 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 10550392 -> 10548698 (-0.02%); split: -0.02%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16460>
2022-05-23 11:52:54 +00:00
Rhys Perry
69d1f4186a aco/tests: add test for p_constaddr with a non-zero offset
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16460>
2022-05-23 11:52:54 +00:00
Rhys Perry
bd8f8dda8c aco: fix p_constaddr with a non-zero offset
Seems this broke a while ago and we never noticed.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 0af7ff49fd ("aco: lower p_constaddr into separate instructions earlier")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16460>
2022-05-23 11:52:54 +00:00
Marek Olšák
39800f0fa3 amd: change chip_class naming to "enum amd_gfx_level gfx_level"
This aligns the naming with PAL.

Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pellou-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16469>
2022-05-13 14:56:22 -04:00
Rhys Perry
63e40adf8c aco: fix disassembly of SMEM with both SGPR and constant offset
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15890>
2022-04-14 20:58:36 +00:00
Tony Wasserka
66e51dc474 aco: Remove use of deprecated Operand constructors
This migration was done with libclang-based automatic tooling, which
performed these replacements:
* Operand(uint8_t) -> Operand::c8
* Operand(uint16_t) -> Operand::c16
* Operand(uint32_t, false) -> Operand::c32
* Operand(uint32_t, bool) -> Operand::c32_or_c64
* Operand(uint64_t) -> Operand::c64
* Operand(0) -> Operand::zero(num_bytes)

Casts that were previously used for constructor selection have automatically
been removed (e.g. Operand((uint16_t)1) -> Operand::c16(1)).

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11653>
2021-07-13 17:43:26 +00:00
Rhys Perry
0af7ff49fd aco: lower p_constaddr into separate instructions earlier
This allows them to be scheduled properly and simplifies the assembler a
little.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8994>
2021-03-11 16:31:19 +00:00
Rhys Perry
441ead5fb3 aco: remove Format::{VOP3A,VOP3B}
These are really the same as Format::VOP3.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8595>
2021-01-22 14:12:32 +00:00
Samuel Pitoiset
18fd6274b2 aco/tests: add disassembler tests to reproduce the add3+clamp crash
Like some other v_add instructions, LLVM fails to disassemble
v_add3_u32 + clamp.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6961>
2020-10-02 14:21:33 +02:00
Rhys Perry
eb3c16e1f8 aco/tests: add tests for long jumps
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>
2020-08-26 13:26:58 +00:00
Rhys Perry
ae6330d955 aco/tests: add test for GFX10 0x3f bug
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6212>
2020-08-26 13:26:58 +00:00