Commit graph

371 commits

Author SHA1 Message Date
Rhys Perry
5de990f5a9 aco: add SSA repair pass
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31143>
2024-11-25 10:32:58 +00:00
Daniel Schürmann
1ff9a0fe80 aco: remove Pseudo_instruction::tmp_in_scc
This information is redundant, now.

No fossil-changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32217>
2024-11-20 11:04:32 +00:00
Daniel Schürmann
a1a4a6061c aco/ra: explicitly assign scratch SGPR for linear phis
We are about to remove the branch definitions which previously
served this purpose. Also remove Block::scc_live_out.
Some changes due to round-robin RA.

Totals from 939 (1.18% of 79395) affected shaders: (Navi31)

Instrs: 5038786 -> 5038611 (-0.00%); split: -0.01%, +0.00%
CodeSize: 26153412 -> 26152904 (-0.00%); split: -0.00%, +0.00%
Latency: 41649989 -> 41650120 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 6447508 -> 6447536 (+0.00%); split: -0.00%, +0.00%
SClause: 131319 -> 131276 (-0.03%); split: -0.03%, +0.00%
Copies: 359362 -> 359256 (-0.03%); split: -0.05%, +0.02%
SALU: 639275 -> 639169 (-0.02%); split: -0.03%, +0.01%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32217>
2024-11-20 11:04:32 +00:00
Samuel Pitoiset
44dfeb4479 radv,aco: add a separate function to compile the trap handler shader
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32056>
2024-11-12 11:16:13 +00:00
Konstantin Seurer
f8ef1afec8 aco: Handle nir_debug_info_instr
Propagated debug info using p_debug_info and Program::debug_info.
Offsets into the shader binary are gathered during assembly.
This will be usefull for mapping back the disassembled shader to
nir, glsl or spirv.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29298>
2024-11-11 08:39:13 +00:00
Samuel Pitoiset
d1d41be43f aco: declare phys regs for tba_hi/tma_hi
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32026>
2024-11-08 14:00:15 +00:00
Rhys Perry
0ad713ca9f aco: add waitcnt build helper
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24884>
2024-11-06 09:58:04 +00:00
Daniel Schürmann
21ceeb22ed aco: move jump threading optimization into separate pass
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31888>
2024-10-30 09:23:54 +00:00
Georg Lehmann
8d148401cb aco/ir: rework Operand equality to return true for equal fixed non-temp ops
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31560>
2024-10-22 17:03:26 +00:00
Daniel Schürmann
e2705a9d85 aco: set Precolored flag before register allocation
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31387>
2024-10-22 12:29:18 +00:00
Daniel Schürmann
c2ed4b474a aco: introduce 'isPrecolored' flag for Operand and Definition
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31387>
2024-10-22 12:29:18 +00:00
Georg Lehmann
977f435f4c aco/ir: add function to parse depctr waits
No Foz-DB changes on Navi31.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31132>
2024-10-17 11:16:16 +00:00
Daniel Schürmann
1d3e01cd62 aco: remove Program::allocationId
It is a duplicate of temp_rc.size().

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31362>
2024-10-07 07:00:19 +00:00
Daniel Schürmann
39fc327b8f aco/reindex_ssa: remove update_live_out parameter
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31362>
2024-10-07 07:00:19 +00:00
Daniel Schürmann
30e7644e5f aco: simplify Definition constructors
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31362>
2024-10-07 07:00:19 +00:00
Georg Lehmann
27cf11dc8a aco: remove per block inf/nan/sz control
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31172>
2024-09-18 20:46:17 +00:00
Georg Lehmann
8912f16464 aco/ir: add float control definition flags
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31172>
2024-09-18 20:46:17 +00:00
Daniel Schürmann
dfc13fcf9f aco: introduce Operand flag 'CopyKill'
This flag indicates that the Operand must be copied in order to satisfy register
constraints. The copy is immediately killed by the instruction.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30299>
2024-09-10 09:44:53 +00:00
Daniel Schürmann
5a6fa8a8eb aco: introduce new Operand flag 'Clobbered'
This flag indicates that the Operand's register gets clobbered by the instruction.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30299>
2024-09-10 09:44:53 +00:00
Rhys Perry
7b92e11e16 aco: forget valu delays after certain s_waitcnt_depctr/LDSDIR
fossil-db (navi31):
Totals from 55242 (69.58% of 79395) affected shaders:
Instrs: 40507666 -> 40138006 (-0.91%); split: -0.91%, +0.00%
CodeSize: 212516104 -> 211025880 (-0.70%); split: -0.70%, +0.00%
Latency: 281643258 -> 281628053 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 46370668 -> 46369637 (-0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23337>
2024-08-22 13:57:01 +00:00
Rhys Perry
807651561e aco: split insert_wait_states into two
No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23337>
2024-08-22 13:57:00 +00:00
Rhys Perry
d91e634c13 aco: calculate indices from dominance tree
This lets us do fast dominance checks.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30440>
2024-08-21 10:47:20 +00:00
Rhys Perry
b934255510 aco: split selection_control_remove into rarely_taken and never_taken
No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Backport-to: 24.1
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30321>
2024-08-15 16:00:18 +00:00
Daniel Schürmann
5a39cbdef6 aco: change signature of get_live_changes() and get_temp_registers()
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30182>
2024-08-14 08:11:48 +00:00
Daniel Schürmann
b0c8c5e42e aco: implement aco::validate_live_vars()
This is intended for passes which manually update live variables
and RegisterDemand, like e.g. the scheduler, and can be enabled
with ACO_DEBUG=validate-livevars.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30182>
2024-08-14 08:11:48 +00:00
Georg Lehmann
53155ba12d aco: add CompilationProgress::after_lower_to_hw
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30399>
2024-07-29 18:35:33 +00:00
Rhys Perry
0919ce1ac4 aco/gfx11.5: workaround export priority issue
https://github.com/llvm/llvm-project/pull/99273

fossil-db (gfx1150):
Totals from 73996 (93.20% of 79395) affected shaders:
Instrs: 36015357 -> 36807177 (+2.20%)
CodeSize: 189072544 -> 192238748 (+1.67%)
Latency: 245845181 -> 246790550 (+0.38%); split: -0.00%, +0.38%
InvThroughput: 45068018 -> 45116177 (+0.11%); split: -0.00%, +0.11%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30241>
2024-07-23 13:14:51 +00:00
Daniel Schürmann
bb5af6bede aco: remove live-out variables from IR
Since we changed all passes to use the live-in variables,
these are not needed anymore.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
2024-07-16 14:00:49 +00:00
Daniel Schürmann
29262f8cf3 aco: compute live-in variables in addition to live-out variables
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
2024-07-16 14:00:49 +00:00
Daniel Schürmann
6c6f382d68 aco: add RegisterDemand member to Instruction
Since we never need both at the same time, we can use
a union with pass_flags.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29962>
2024-07-10 12:31:02 +00:00
Georg Lehmann
7fc8ad2ddd aco/ir: remove unused vopc helpers
And rename get_swapped and get_inverse to show that they should only be used for VOPC.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29467>
2024-06-27 08:12:30 +00:00
Daniel Schürmann
5cfa5b784b aco: remove get_demand_before()
The register demand before executing an instruction is now included
in the instruction's register demand and this function is unused.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
2024-06-26 09:42:34 +00:00
Daniel Schürmann
09f1c40f2e aco: track and use the live-in register demand per basic block
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
2024-06-26 09:42:34 +00:00
Daniel Schürmann
b7982152ff aco: use aco::monotonic_allocator for IDSet
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29713>
2024-06-14 14:32:35 +00:00
Daniel Schürmann
95967c2ca0 aco/reindex_ssa: replace live_var parameter with boolean
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29713>
2024-06-14 14:32:35 +00:00
Daniel Schürmann
a497d105e3 aco: move live var information into struct Program
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29713>
2024-06-14 14:32:35 +00:00
Rhys Perry
b41f0f6cc1 aco: use ac_hw_cache_flags
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243>
2024-06-07 13:22:42 +00:00
Rhys Perry
2a4424425a aco/gfx12: fix s_wait_event immediate
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29466>
2024-06-06 14:26:52 +00:00
Rhys Perry
c8123b67e0 aco/gfx12: don't create v_fmac_legacy_f32
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29330>
2024-05-28 10:52:11 +00:00
Georg Lehmann
cc404d45ff aco: remove perfwarn
This didn't do anything useful.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29270>
2024-05-21 13:31:23 +00:00
Rhys Perry
f01cac835f aco/stats: support GFX12 in collect_preasm_stats()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29225>
2024-05-20 10:45:39 +00:00
Rhys Perry
cadce0f3b7 aco: add GFX12 wait counters
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29225>
2024-05-20 10:45:38 +00:00
Rhys Perry
e1e5bc0dd0 aco: support GFX12 in assembler
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29162>
2024-05-14 20:50:27 +00:00
Rhys Perry
75532d8687 aco: add wait_imm::unpack and wait_imm::max
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28981>
2024-05-10 11:53:08 +00:00
Rhys Perry
20b4e30e25 aco: make wait_imm indexable
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28981>
2024-05-10 11:53:08 +00:00
Georg Lehmann
47566d0df3 aco: add a subdword lowering pass
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28836>
2024-05-02 11:09:35 +00:00
Georg Lehmann
81a334a594 aco/assembler: add vintrp high_16bit support
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28435>
2024-04-10 07:49:26 +00:00
Samuel Pitoiset
7a69d78ba2 aco: use SPDX-License-Identifier
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622>
2024-04-08 15:49:25 +00:00
Georg Lehmann
be93e38152 aco/vn: remove instruction hash templates
This reuses the instruction data size helper used for creating instruction.
It also removes the hashing of the aco::spans used for operands and definitions,
which was redundant with hash finalization.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28527>
2024-04-05 10:23:16 +00:00
Timur Kristóf
8883b88dd4 aco: Delete all TCS epilog code.
Now that neither RADV nor RadeonSI uses TCS epilogs, we don't
need to keep the code to compile them in ACO either.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28425>
2024-03-30 21:56:39 +01:00