fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2026-05-17 20:28:05 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
207574 commits 66 branches 935 tags 932 MiB
Commit graph

169 commits

Author SHA1 Message Date
Natalie Vock
4a62b342f3 aco: Add common utility to load scratch descriptor 
Also modifies the scratch descriptor to take the stack pointer into
account.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35031>
 2025-06-26 11:02:52 +00:00
Natalie Vock
cd2caa5e2b aco/spill: Use scratch stack pointer 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35031>
 2025-06-26 11:02:52 +00:00
Rhys Perry
6338ed44c5 aco/gfx12: increase maximum vbuffer offset 
fossil-db (gfx1201):
Totals from 301 (0.38% of 79377) affected shaders:
Instrs: 2734478 -> 2728816 (-0.21%); split: -0.21%, +0.00%
CodeSize: 14347476 -> 14306568 (-0.29%)
Latency: 15508055 -> 15502202 (-0.04%); split: -0.04%, +0.00%
InvThroughput: 2846419 -> 2842387 (-0.14%); split: -0.14%, +0.00%
VClause: 68286 -> 68101 (-0.27%); split: -0.30%, +0.03%
SClause: 49487 -> 49500 (+0.03%)
Copies: 207179 -> 206093 (-0.52%); split: -0.57%, +0.04%
Branches: 72941 -> 72942 (+0.00%); split: -0.00%, +0.00%
VALU: 1549156 -> 1544727 (-0.29%); split: -0.29%, +0.00%
SALU: 339620 -> 338989 (-0.19%); split: -0.19%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34730>
 2025-04-29 17:44:41 +00:00
Rhys Perry
de896234d8 aco: improve spilling of clobbered operands
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
Details
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Details 
We can ignore live_changes for these.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34699>
 2025-04-28 10:04:43 +00:00
Natalie Vock
3d8db3cbbb aco: Make private_segment_buffer/scratch_offset per-resume 
We need different Temps for each resume shader, because registers aren't
preserved across resume boundaries.

This was likely fine in practice because arg registers are the same for
each shader, but resulted in invalid IR and asserts.

Fixes crashes in Indiana Jones RT with assertions enabled on GFX8.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34114>
 2025-04-09 14:21:37 +00:00
Natalie Vock
d6cb45dbb0 aco/spill: Allow spilling live-through operands
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
Details
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Details 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29730>
 2025-03-26 19:18:30 +00:00
Natalie Vock
ca7ce1fb33 aco/spill: Invert reloads map 
So we can quickly look up if an operand was reloaded without having to
check renames.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29730>
 2025-03-26 19:18:30 +00:00
Daniel Schürmann
52253da783 aco: unify get_addr_sgpr_from_waves() and get_addr_vgpr_from_waves() into one function 
which returns the limit as RegisterDemand.

Also remove the unused get_extra_sgprs() from aco_ir.h.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33644>
 2025-02-21 13:49:41 +00:00
Daniel Schürmann
e8472d484f aco/spill: use float division for score() calculation rather than integers 
This was the original intention and should result in more fine-grained
and thus better decisions.

Totals from 63 (0.08% of 79395) affected shaders: (Navi31)
Instrs: 3173500 -> 3174012 (+0.02%); split: -0.01%, +0.02%
CodeSize: 16345348 -> 16349288 (+0.02%); split: -0.01%, +0.03%
Latency: 18528036 -> 18526082 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 3619125 -> 3618709 (-0.01%); split: -0.02%, +0.01%
VClause: 82654 -> 82648 (-0.01%)
SClause: 61256 -> 61257 (+0.00%); split: -0.00%, +0.01%
Copies: 250037 -> 250158 (+0.05%); split: -0.06%, +0.11%
Branches: 101302 -> 101303 (+0.00%)
VALU: 1791447 -> 1791435 (-0.00%); split: -0.00%, +0.00%
SALU: 401898 -> 402007 (+0.03%); split: -0.03%, +0.06%
VOPD: 730 -> 741 (+1.51%)

Totals from 40 (0.06% of 63053) affected shaders: (Vega10)

Instrs: 161584 -> 161567 (-0.01%); split: -0.04%, +0.03%
CodeSize: 891168 -> 891004 (-0.02%); split: -0.04%, +0.03%
Latency: 3550766 -> 3549770 (-0.03%); split: -0.05%, +0.03%
InvThroughput: 2627028 -> 2626484 (-0.02%); split: -0.03%, +0.01%
VClause: 2970 -> 2971 (+0.03%)
SClause: 4203 -> 4205 (+0.05%); split: -0.26%, +0.31%
Copies: 19923 -> 19893 (-0.15%); split: -0.44%, +0.29%
VALU: 116045 -> 116054 (+0.01%); split: -0.01%, +0.02%
SALU: 22100 -> 22066 (-0.15%); split: -0.39%, +0.24%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31769>
 2024-10-23 14:35:29 +00:00
Daniel Schürmann
30d85b23ef aco/spill: fix faulty assertions 
By unintentionally using integer division for score(), these
assertions were likely to be raised by accident.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31769>
 2024-10-23 14:35:29 +00:00
Daniel Schürmann
541cfb21ba aco: don't attempt to spill dead phis 
These don't affect register pressure.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30182>
 2024-08-14 08:11:48 +00:00
Georg Lehmann
dedfff9dbf aco: only set latekill in live_var_analysis 
Cleaner to have this all in one place, in my opinion.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30368>
 2024-08-12 10:31:09 +00:00
Daniel Schürmann
6723128e94 aco/spill: Don't add phi definitions to live-in variables 
Changes are because we don't add artificial uses to the phi definitions anymore.

Totals from 13 (0.02% of 79395) affected shaders: (GFX10.3)

Instrs: 230510 -> 230285 (-0.10%); split: -0.10%, +0.00%
CodeSize: 1269916 -> 1268760 (-0.09%); split: -0.10%, +0.01%
SpillSGPRs: 2057 -> 2058 (+0.05%)
Latency: 2729731 -> 2723103 (-0.24%)
InvThroughput: 696888 -> 695286 (-0.23%)
VClause: 5795 -> 5768 (-0.47%)
SClause: 6855 -> 6858 (+0.04%)
Copies: 32336 -> 32275 (-0.19%); split: -0.22%, +0.03%
VALU: 151782 -> 151731 (-0.03%); split: -0.04%, +0.01%
SALU: 30766 -> 30758 (-0.03%); split: -0.03%, +0.01%
VMEM: 12157 -> 12078 (-0.65%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
 2024-07-16 14:00:49 +00:00
Daniel Schürmann
c146d4b6b6 aco/spill: use live-in variables directly rather than computing them 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
 2024-07-16 14:00:49 +00:00
Daniel Schürmann
6c6f382d68 aco: add RegisterDemand member to Instruction 
Since we never need both at the same time, we can use
a union with pass_flags.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29962>
 2024-07-10 12:31:02 +00:00
Daniel Schürmann
c4a38c6583 aco/spill: don't remove spilled phis 
They will be removed during register allocation.

Few changes due to different phi order.
Totals from 14 (0.02% of 79395) affected shaders: (GFX11)

Instrs: 315724 -> 315675 (-0.02%); split: -0.02%, +0.01%
CodeSize: 1673608 -> 1673268 (-0.02%); split: -0.03%, +0.00%
Latency: 3194243 -> 3189025 (-0.16%); split: -0.19%, +0.03%
InvThroughput: 638369 -> 637323 (-0.16%); split: -0.19%, +0.03%
VClause: 5716 -> 5714 (-0.03%)
Copies: 37786 -> 37748 (-0.10%); split: -0.13%, +0.03%
Branches: 10469 -> 10454 (-0.14%); split: -0.16%, +0.02%
VALU: 182498 -> 182454 (-0.02%); split: -0.03%, +0.00%
SALU: 36038 -> 36046 (+0.02%); split: -0.01%, +0.04%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:34 +00:00
Daniel Schürmann
5cfa5b784b aco: remove get_demand_before() 
The register demand before executing an instruction is now included
in the instruction's register demand and this function is unused.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:34 +00:00
Daniel Schürmann
09f1c40f2e aco: track and use the live-in register demand per basic block 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:34 +00:00
Daniel Schürmann
001c8caae0 aco: calculate register demand per instruction as maximum necessary to execute the instruction 
Previously, the register demand per instruction was calculated as the number of
live variables in the register file after executing an instruction plus additional
temporary registers, necessary during the execution of the instruction.
With this change, now it also includes all variables which are live right before
executing an instruction, i.e. killed Operands.

Care has been taken so that the invariant

register_demand[idx] = register_demand[idx - 1] - get_temp_registers(prev_instr)
                        + get_live_changes(instr) + get_temp_registers(instr)

still holds.

Slight changes in scheduling:

Totals from 316 (0.40% of 79395) affected shaders: (GFX11)

Instrs: 301329 -> 300777 (-0.18%); split: -0.31%, +0.12%
CodeSize: 1577976 -> 1576204 (-0.11%); split: -0.21%, +0.10%
SpillSGPRs: 448 -> 447 (-0.22%)
Latency: 1736349 -> 1726182 (-0.59%); split: -2.01%, +1.42%
InvThroughput: 243894 -> 243883 (-0.00%); split: -0.03%, +0.03%
VClause: 6134 -> 6280 (+2.38%); split: -1.04%, +3.42%
SClause: 6142 -> 6137 (-0.08%); split: -0.13%, +0.05%
Copies: 14037 -> 14032 (-0.04%); split: -0.56%, +0.52%
Branches: 3284 -> 3283 (-0.03%)
VALU: 182750 -> 182718 (-0.02%); split: -0.04%, +0.03%
SALU: 18522 -> 18538 (+0.09%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:33 +00:00
Daniel Schürmann
4c2f231cc0 aco/spill: Unconditionally add 2 SGPRs to live-in demand 
Due to undefined Operands, it might not be enough to check the
predecessors' register demand.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:33 +00:00
Daniel Schürmann
a497d105e3 aco: move live var information into struct Program 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29713>
 2024-06-14 14:32:35 +00:00
Friedrich Vock
ec8512ce85 aco/spill: Don't spill phis with all-undef operands 
Fixes some crashes when limiting RT stages to 128 VGPRs.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29593>
 2024-06-10 19:39:52 +00:00
Rhys Perry
b41f0f6cc1 aco: use ac_hw_cache_flags 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243>
 2024-06-07 13:22:42 +00:00
Samuel Pitoiset
3d6957268b aco: use new common helpers for building buffer descriptors 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29268>
 2024-05-22 08:31:39 +00:00
Friedrich Vock
590ea76104 aco/spill: Insert p_start_linear_vgpr right after p_logical_end 
If p_start_linear_vgpr allocates a VGPR that is already blocked, RA
will try moving the blocking VGPR somewhere else. If
p_start_linear_vgpr is inserted right before the branch, that move will
be inserted after exec has been overwritten, which might cause the move
to be skipped for some threads.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28041>
 2024-05-14 13:54:34 +00:00
Marek Olšák
58a5de5c34 amd: add gfx12 register definitions into the register header generator 
The generator renamed some definitions to resolve conflicts.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29007>
 2024-05-11 22:14:05 -04:00
Samuel Pitoiset
7a69d78ba2 aco: use SPDX-License-Identifier 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622>
 2024-04-08 15:49:25 +00:00
Daniel Schürmann
a4231d4f56 aco/spill: use average use distances to spill loop variables 
Totals from 83 (0.10% of 79395) affected shaders: (GFX11)

Instrs: 3429510 -> 3427917 (-0.05%); split: -0.05%, +0.01%
CodeSize: 17587884 -> 17570224 (-0.10%); split: -0.11%, +0.01%
SpillSGPRs: 4660 -> 4630 (-0.64%); split: -1.07%, +0.43%
Latency: 20054145 -> 20054454 (+0.00%); split: -0.00%, +0.01%
InvThroughput: 4989606 -> 4989740 (+0.00%); split: -0.00%, +0.01%
VClause: 90844 -> 90843 (-0.00%)
SClause: 69534 -> 69535 (+0.00%); split: -0.04%, +0.04%
Copies: 283288 -> 283415 (+0.04%); split: -0.11%, +0.15%
Branches: 113543 -> 113409 (-0.12%); split: -0.12%, +0.01%
VALU: 1888933 -> 1887489 (-0.08%); split: -0.08%, +0.00%
SALU: 423548 -> 423609 (+0.01%); split: -0.07%, +0.09%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
c371882060 aco/spill: use average use distances in init_live_in_vars() for merge blocks 
Totals from 18 (0.02% of 79395) affected shaders: (GFX11)

Instrs: 726400 -> 726409 (+0.00%); split: -0.00%, +0.00%
CodeSize: 3750396 -> 3750420 (+0.00%); split: -0.00%, +0.00%
Latency: 4363014 -> 4363039 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 1483486 -> 1483485 (-0.00%)
SClause: 13393 -> 13392 (-0.01%); split: -0.01%, +0.01%
Copies: 70236 -> 70254 (+0.03%); split: -0.01%, +0.03%
Branches: 23811 -> 23810 (-0.00%)
VALU: 404250 -> 404247 (-0.00%); split: -0.00%, +0.00%
SALU: 92208 -> 92226 (+0.02%); split: -0.01%, +0.02%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
c2ac4f1917 aco/spill: use average use distances in process_block() 
Totals from 128 (0.16% of 79395) affected shaders: (GFX11)

Instrs: 672936 -> 672086 (-0.13%); split: -0.40%, +0.28%
CodeSize: 3574396 -> 3565540 (-0.25%); split: -0.55%, +0.30%
SpillSGPRs: 6254 -> 6636 (+6.11%); split: -0.32%, +6.43%
SpillVGPRs: 967 -> 839 (-13.24%)
Latency: 6489362 -> 6469427 (-0.31%); split: -0.36%, +0.05%
InvThroughput: 2216723 -> 2212417 (-0.19%); split: -0.24%, +0.05%
VClause: 11670 -> 11517 (-1.31%); split: -1.50%, +0.19%
SClause: 15711 -> 15754 (+0.27%); split: -0.29%, +0.57%
Copies: 85185 -> 84155 (-1.21%); split: -2.14%, +0.93%
Branches: 18706 -> 18708 (+0.01%); split: -0.05%, +0.06%
VALU: 386892 -> 386204 (-0.18%); split: -0.40%, +0.23%
SALU: 85754 -> 86211 (+0.53%); split: -0.46%, +1.00%
VMEM: 18777 -> 18524 (-1.35%); split: -1.37%, +0.02%
VOPD: 670 -> 628 (-6.27%); split: +0.75%, -7.01%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
9833120e91 aco/spill: gather information about average use distances 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
b6e56ab356 aco/spill: use live variables instead of next_use_distances in add_coupling_code() 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
7b96482ec9 aco/spill: maintain valid live vars at any point 
In order to do so, we need to process blocks unconditionally.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
fab8757831 aco/spill: compute live-in variables from live-out 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
bfb62fa70d aco/spill: pass live_vars to spill_ctx 
We will use them in future commits.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
66616f0862 aco/spill: Fix assertion for nested loops 
Fixes: 898fd9227a ('aco/spill: keep loop variables spilled during nested loops')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
a863c7951e aco: remove create_instruction() template parameter 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
9b0ebcc39b aco: change return type of create_instruction() to Instruction* 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
1187189235 aco: unify different SALU types into single struct SALU_instruction 
This removes
- SOP1_instruction
- SOP2_instruction
- SOPC_instruction
- SOPK_instruction
- SOPP_instruction

and their corresponding methods.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
898fd9227a aco/spill: keep loop variables spilled during nested loops 
Totals from 36 (0.05% of 79395) affected shaders: (GFX11)

Instrs: 2775993 -> 2781480 (+0.20%); split: -0.01%, +0.20%
CodeSize: 14184524 -> 14231956 (+0.33%); split: -0.01%, +0.34%
SpillSGPRs: 1441 -> 1207 (-16.24%)
SpillVGPRs: 477 -> 421 (-11.74%)
Scratch: 35840 -> 35072 (-2.14%)
Latency: 15559241 -> 15606047 (+0.30%); split: -0.01%, +0.31%
InvThroughput: 3787994 -> 3803749 (+0.42%); split: -0.02%, +0.44%
VClause: 76422 -> 76411 (-0.01%)
SClause: 55717 -> 55671 (-0.08%); split: -0.10%, +0.01%
Copies: 214877 -> 215185 (+0.14%); split: -0.04%, +0.18%
Branches: 93158 -> 93129 (-0.03%); split: -0.03%, +0.00%
VALU: 1520556 -> 1525425 (+0.32%); split: -0.00%, +0.32%
SALU: 341104 -> 341408 (+0.09%); split: -0.02%, +0.11%
VMEM: 119673 -> 119617 (-0.05%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
80bc658efc aco/spill: select more loop-carried variables to be spilled 
This changes the heuristic to also spill live-through variables
if they are used within the loop if the register pressure is too high.

Totals from 68 (0.09% of 79395) affected shaders: (GFX11)

Instrs: 3385477 -> 3386659 (+0.03%); split: -0.11%, +0.14%
CodeSize: 17329668 -> 17353796 (+0.14%); split: -0.12%, +0.26%
SpillSGPRs: 4246 -> 3659 (-13.82%); split: -17.12%, +3.30%
SpillVGPRs: 978 -> 884 (-9.61%); split: -10.02%, +0.41%
Scratch: 58880 -> 59648 (+1.30%)
Latency: 20567445 -> 20753412 (+0.90%); split: -0.08%, +0.98%
InvThroughput: 5091128 -> 5188098 (+1.90%); split: -0.07%, +1.97%
VClause: 90446 -> 90499 (+0.06%); split: -0.05%, +0.11%
SClause: 68270 -> 68072 (-0.29%); split: -0.35%, +0.06%
Copies: 280689 -> 278003 (-0.96%); split: -1.08%, +0.12%
Branches: 112577 -> 112608 (+0.03%); split: -0.10%, +0.13%
VALU: 1863415 -> 1866878 (+0.19%); split: -0.07%, +0.26%
SALU: 415572 -> 413062 (-0.60%); split: -0.63%, +0.03%
VMEM: 140890 -> 141045 (+0.11%); split: -0.04%, +0.15%
VOPD: 650 -> 670 (+3.08%); split: +3.54%, -0.46%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
31f6aebb2c aco/spill: keep loop-carried variables spilled at merge blocks 
If loop-carried variables are partially spilled at some predecessor,
keep them spilled without extra instruction.

Also slightly refactors the code for better readability.

Totals from 13 (0.02% of 79395) affected shaders: (GFX11)

Instrs: 259129 -> 258194 (-0.36%)
CodeSize: 1337604 -> 1331768 (-0.44%)
SpillSGPRs: 561 -> 560 (-0.18%)
SpillVGPRs: 146 -> 140 (-4.11%)
Latency: 2410886 -> 2408560 (-0.10%)
InvThroughput: 759229 -> 758416 (-0.11%)
VClause: 4178 -> 4154 (-0.57%)
SClause: 7217 -> 7219 (+0.03%)
Copies: 32411 -> 32174 (-0.73%)
Branches: 9426 -> 9296 (-1.38%)
VALU: 144602 -> 144098 (-0.35%)
SALU: 30582 -> 30405 (-0.58%)
VMEM: 6961 -> 6908 (-0.76%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
971407bd2f aco/spill: keep loop-carried variables spilled at loop headers 
Now, that we avoid re-spilling loop-carried variables, we can
keep them spilled at loop-headers.

Totals from 31 (0.04% of 79395) affected shaders: (GFX11)

Instrs: 1539175 -> 1538109 (-0.07%); split: -0.14%, +0.07%
CodeSize: 7877948 -> 7871916 (-0.08%); split: -0.16%, +0.08%
SpillSGPRs: 1936 -> 1523 (-21.33%)
SpillVGPRs: 160 -> 146 (-8.75%)
Scratch: 18176 -> 16896 (-7.04%)
Latency: 9439964 -> 9441197 (+0.01%); split: -0.01%, +0.02%
InvThroughput: 2406253 -> 2406700 (+0.02%); split: -0.01%, +0.03%
VClause: 38903 -> 38922 (+0.05%)
SClause: 33347 -> 33349 (+0.01%); split: -0.01%, +0.02%
Copies: 131420 -> 130974 (-0.34%); split: -0.57%, +0.23%
Branches: 53091 -> 53212 (+0.23%); split: -0.03%, +0.26%
VALU: 844372 -> 843876 (-0.06%); split: -0.12%, +0.06%
SALU: 190329 -> 189891 (-0.23%); split: -0.40%, +0.17%
VMEM: 60491 -> 60510 (+0.03%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
e968a18652 aco/spill: avoid re-spilling loop-carried variables in add_coupling_code() 
These variables only need to be spilled once at the loop-preheader.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
7726b92707 aco/spill: avoid re-spilling loop-carried variables in process_block() 
If a variable is spilled at the loop-header of the current loop,
there is no need to spill it redundantly. Instead, re-use the
spill-slot from the loop-header.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
bb3b6a5ff3 aco/spill: add interferences with variables spilled at loop headers 
We will re-use the spill slots rather than re-spilling the variables.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
09c5414760 aco/spill: don't prefer to spill phis at merge blocks 
Totals from 44 (0.06% of 79242) affected shaders: (GFX11)

Instrs: 165578 -> 165545 (-0.02%); split: -0.08%, +0.06%
CodeSize: 833528 -> 832988 (-0.06%); split: -0.11%, +0.04%
SpillSGPRs: 346 -> 329 (-4.91%)
Latency: 1176341 -> 1176231 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 235771 -> 235761 (-0.00%); split: -0.01%, +0.01%
SClause: 3287 -> 3289 (+0.06%); split: -0.21%, +0.27%
Copies: 16290 -> 16299 (+0.06%); split: -0.62%, +0.68%
Branches: 3633 -> 3641 (+0.22%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
d234f789b5 aco/spill: keep live-out variables spilled at branch blocks 
There is no reason to reload early. Leave scheduling to the scheduler.

Totals from 98 (0.12% of 79242) affected shaders: (GFX11)

Instrs: 841944 -> 841995 (+0.01%); split: -0.02%, +0.03%
CodeSize: 4354332 -> 4354544 (+0.00%); split: -0.02%, +0.02%
SpillSGPRs: 2557 -> 2554 (-0.12%)
Latency: 5772080 -> 5772209 (+0.00%); split: -0.00%, +0.01%
InvThroughput: 1694834 -> 1694873 (+0.00%); split: -0.00%, +0.00%
SClause: 16036 -> 16042 (+0.04%); split: -0.07%, +0.11%
Copies: 85117 -> 85125 (+0.01%); split: -0.18%, +0.19%
Branches: 25222 -> 25219 (-0.01%); split: -0.02%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
32882d647e aco/spill: refactor adding spilled vars into separate function add_to_spills() 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
a5e8f9f879 aco/spill: add spills_entry interferences only when necessary 
For variables which are already spilled in previous blocks,
the interferences already exist.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00