Georg Lehmann
c279e63a79
aco: rename p_early_exit_if to if_not
...
It exits the shaders if the condition is false, not true.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32731 >
2024-12-30 13:05:23 +00:00
Samuel Pitoiset
132b7a85c7
aco: drop the second M0 operand for s_set_gpr_idx_on
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32090 >
2024-11-13 15:27:54 +00:00
Konstantin Seurer
f8ef1afec8
aco: Handle nir_debug_info_instr
...
Propagated debug info using p_debug_info and Program::debug_info.
Offsets into the shader binary are gathered during assembly.
This will be usefull for mapping back the disassembled shader to
nir, glsl or spirv.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29298 >
2024-11-11 08:39:13 +00:00
Georg Lehmann
9bb10b58f3
aco: use v_cvt_pk_u8_f32 for f2u8
...
Foz-DB Navi31:
Totals from 42 (0.05% of 79395) affected shaders:
Instrs: 3253747 -> 3248867 (-0.15%); split: -0.15%, +0.00%
CodeSize: 16690136 -> 16661772 (-0.17%); split: -0.17%, +0.00%
VGPRs: 4176 -> 4128 (-1.15%)
Latency: 18485157 -> 18479752 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 3659404 -> 3658222 (-0.03%); split: -0.03%, +0.00%
Copies: 231891 -> 228145 (-1.62%)
VALU: 1785800 -> 1782054 (-0.21%)
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29532 >
2024-09-02 19:49:20 +00:00
Georg Lehmann
62fa5b9d6f
aco/gfx11+: apply neg to vinterp
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30350 >
2024-07-30 15:25:19 +00:00
Georg Lehmann
4399c7bac3
aco: add aco_opcode::p_s_cvt_f16_f32_rtne
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245 >
2024-07-18 08:36:14 +00:00
Rhys Perry
b41f0f6cc1
aco: use ac_hw_cache_flags
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243 >
2024-06-07 13:22:42 +00:00
Rhys Perry
fae2a85d57
aco/gfx12: implement subgroup shader clock
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29330 >
2024-05-28 10:52:11 +00:00
Rhys Perry
e1e5bc0dd0
aco: support GFX12 in assembler
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29162 >
2024-05-14 20:50:27 +00:00
Rhys Perry
74aa6437d6
aco: add GFX11.5+ opcodes
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29162 >
2024-05-14 20:50:27 +00:00
Rhys Perry
97698e564a
aco: add SFPU/ValuPseudoScalarTrans instr class
...
The latency is from LLVM's SISchedule.td
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29162 >
2024-05-14 20:50:27 +00:00
Georg Lehmann
80b8bbf0c5
aco/gfx11: use v_swap_b16
...
I tested that v_swap_b16 can be encoded as VOP3, because the ISA doc doesn't list
it as a possible VOP3 opcode. VOP3 is nessecary to access v128+.
Foz-DB Navi31:
Totals from 32 (0.04% of 79395) affected shaders:
Instrs: 201865 -> 195168 (-3.32%)
CodeSize: 1082220 -> 1031228 (-4.71%); split: -4.71%, +0.00%
Latency: 2258198 -> 2238586 (-0.87%)
InvThroughput: 796731 -> 788934 (-0.98%)
Copies: 34514 -> 29220 (-15.34%)
VALU: 122457 -> 117163 (-4.32%)
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29143 >
2024-05-13 18:42:19 +00:00
Daniel Schürmann
6e3446422f
aco: introduce aco_opcode::p_boolean_phi
...
This opcode is only used during instruction selection and
immediately lowered to linear phis afterwards.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28661 >
2024-04-26 08:39:01 +00:00
Pierre-Eric Pelloux-Prayer
27a3880ada
aco: don't use python 3.7+ feature in aco_opcodes.py
...
Use the suggestion from https://stackoverflow.com/questions/11351032/named-tuple-and-default-values-for-optional-keyword-arguments
so the script works on older Python.
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28831 >
2024-04-23 22:45:51 +00:00
Georg Lehmann
e2cb9c57a2
aco: use v_interp_p2_f16 opsel
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28435 >
2024-04-10 07:49:27 +00:00
Georg Lehmann
4b5016a537
aco: support high_16bits FS IO
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28435 >
2024-04-10 07:49:27 +00:00
Georg Lehmann
af199c6949
aco: swap opsel and wait_exp for vinterp
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28435 >
2024-04-10 07:49:26 +00:00
Samuel Pitoiset
7a69d78ba2
aco: use SPDX-License-Identifier
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622 >
2024-04-08 15:49:25 +00:00
Rhys Perry
6b301eae36
aco: implement mqsad_4x8 and shfr
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26251 >
2024-04-05 11:01:39 +00:00
Daniel Schürmann
cd62f97719
aco/builder: use accessor functions instead of casting to subtypes
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
Daniel Schürmann
1187189235
aco: unify different SALU types into single struct SALU_instruction
...
This removes
- SOP1_instruction
- SOP2_instruction
- SOPC_instruction
- SOPK_instruction
- SOPP_instruction
and their corresponding methods.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
Daniel Schürmann
5d265257a0
aco: remove SOPP_instruction::block member
...
Re-use SOPP_instruction::imm instead.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
Daniel Schürmann
cef01e817d
aco: use instr_class::branch to identify SOPP branches
...
Also changes the instr_class of s_trap to instr_class::other.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
Rhys Perry
cdcc856cbf
aco: unify MIMG opcode lists
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27913 >
2024-03-22 10:29:43 +00:00
Rhys Perry
20b14381c4
aco: move dot/wmma instructions into VOP3P list
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27913 >
2024-03-22 10:29:43 +00:00
Rhys Perry
6bfd3dc39f
aco: use op()
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27913 >
2024-03-22 10:29:43 +00:00
Rhys Perry
a806e48982
aco: use single tuple for all opcode numbers
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27913 >
2024-03-22 10:29:43 +00:00
Rhys Perry
5864c22fb7
aco: refactor VOPC opcode list
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27913 >
2024-03-22 10:29:43 +00:00
Rhys Perry
d23f8b0dcf
aco: rename opcode->instruction
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27913 >
2024-03-22 10:29:43 +00:00
Rhys Perry
b723a15bd0
aco: fix GFX6 buffer_load_dwordx4 opcode number
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27913 >
2024-03-22 10:29:43 +00:00
Rhys Perry
6547e17e60
aco: add VOPD format
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367 >
2024-02-05 10:51:14 +00:00
Georg Lehmann
91539713bb
aco: add src/def count and size for all ALU opcodes
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26163 >
2023-11-15 12:35:32 +00:00
Georg Lehmann
6cd78281f6
aco: deduplicate Format definition
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25943 >
2023-11-06 23:16:38 +00:00
Georg Lehmann
6e0bf33a89
aco: deduplicate instr_class definition
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25943 >
2023-11-06 23:16:38 +00:00
Bas Nieuwenhuizen
5e7c828c0e
aco: Add WMMA instructions.
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683 >
2023-10-24 13:24:18 +00:00
Georg Lehmann
5e9fad48bf
aco/gfx11: apply clamp/omod to vinterp
...
Totals from 2504 (3.27% of 76572) affected shaders:
MaxWaves: 74098 -> 74106 (+0.01%)
Instrs: 1829278 -> 1823427 (-0.32%); split: -0.32%, +0.00%
CodeSize: 9775908 -> 9759308 (-0.17%); split: -0.18%, +0.01%
Latency: 13494107 -> 13485390 (-0.06%); split: -0.10%, +0.04%
InvThroughput: 2052428 -> 2048724 (-0.18%); split: -0.18%, +0.00%
VClause: 26637 -> 26640 (+0.01%); split: -0.04%, +0.05%
SClause: 62027 -> 61988 (-0.06%); split: -0.14%, +0.08%
Copies: 73776 -> 73815 (+0.05%); split: -0.07%, +0.12%
PreVGPRs: 84403 -> 84397 (-0.01%)
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25220 >
2023-10-05 20:02:53 +00:00
Rhys Perry
0e79f76aa5
aco: add fetch_inactive field to DPP instructions
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25525 >
2023-10-04 18:53:43 +00:00
Rhys Perry
26fce534b5
aco: shrink DPP8_instruction
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25525 >
2023-10-04 18:53:43 +00:00
Daniel Schürmann
45f6d38a76
aco: insert a single p_end_wqm after the last derivative calculation
...
This new instruction replaces p_wqm.
Totals from 28065 (36.65% of 76572) affected shaders: (GFX11)
MaxWaves: 823922 -> 823952 (+0.00%); split: +0.01%, -0.01%
Instrs: 22221375 -> 22180465 (-0.18%); split: -0.26%, +0.08%
CodeSize: 117310676 -> 117040684 (-0.23%); split: -0.30%, +0.07%
VGPRs: 1183476 -> 1186656 (+0.27%); split: -0.19%, +0.46%
SpillSGPRs: 2305 -> 2302 (-0.13%)
Latency: 176559310 -> 176427793 (-0.07%); split: -0.21%, +0.14%
InvThroughput: 26245204 -> 26195550 (-0.19%); split: -0.26%, +0.07%
VClause: 368058 -> 369460 (+0.38%); split: -0.21%, +0.59%
SClause: 857077 -> 842588 (-1.69%); split: -2.06%, +0.37%
Copies: 1245650 -> 1249434 (+0.30%); split: -0.33%, +0.63%
Branches: 394837 -> 396070 (+0.31%); split: -0.01%, +0.32%
PreSGPRs: 1019139 -> 1019567 (+0.04%); split: -0.02%, +0.06%
PreVGPRs: 925739 -> 931860 (+0.66%); split: -0.00%, +0.66%
Changes are due to scheduling and re-enabling cross-lane optimizations.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25038 >
2023-09-14 09:25:23 +00:00
Rhys Perry
9169fbf83c
aco: clarify bpermute pseudo opcode names
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693 >
2023-08-23 12:36:46 +01:00
Qiang Yu
85d9646288
aco: add p_end_with_regs pseudo instruction
...
Used by radeonsi shader parts to pass args from one part to another.
It has variable number of operands to reserve fixed registers with
wanted value.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24442 >
2023-08-16 02:27:45 +00:00
Vitaliy Triang3l Kuzmin
f8e744f07f
aco: Add Primitive Ordered Pixel Shading pseudo-instructions
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:04 +00:00
Georg Lehmann
99e44cc527
aco/opcodes: delete wrong comment copy pasted from NIR
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21882 >
2023-06-16 13:45:38 +00:00
Georg Lehmann
fe3993447d
aco: remove v_cvt_pkrtz_f16_f32_e64 when it's actually VOP2
...
More consistent with how we handle all the opcodes which were moved to VOP3.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21882 >
2023-06-16 13:45:38 +00:00
Georg Lehmann
49bdc3a1c2
aco/opcodes: move v_cndmask_b32 back to the VOP2 list
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21882 >
2023-06-16 13:45:38 +00:00
Daniel Schürmann
393f3426b6
aco: implement select_program_rt()
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22096 >
2023-06-08 00:37:03 +00:00
Daniel Schürmann
f66f274304
aco: implement nir_intrinsic_load_resume_shader_address_amd
...
Similar to p_constaddr but targeting BBs.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22096 >
2023-06-08 00:37:03 +00:00
Rhys Perry
1a6a57ac96
aco: let p_start_linear_vgpr take an operand
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22636 >
2023-05-25 16:29:16 +00:00
Georg Lehmann
2548f28ab3
aco/assembler: support VOP3P with DPP
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698 >
2023-05-12 13:31:15 +00:00
Qiang Yu
360176b671
aco,radv: support symbol relocation in aco
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22727 >
2023-04-28 11:33:28 +08:00