fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2026-01-13 16:20:18 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
197548 commits 64 branches 915 tags 995 MiB
Commit graph

161 commits

Author SHA1 Message Date
Daniel Schürmann
e8472d484f aco/spill: use float division for score() calculation rather than integers 
This was the original intention and should result in more fine-grained
and thus better decisions.

Totals from 63 (0.08% of 79395) affected shaders: (Navi31)
Instrs: 3173500 -> 3174012 (+0.02%); split: -0.01%, +0.02%
CodeSize: 16345348 -> 16349288 (+0.02%); split: -0.01%, +0.03%
Latency: 18528036 -> 18526082 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 3619125 -> 3618709 (-0.01%); split: -0.02%, +0.01%
VClause: 82654 -> 82648 (-0.01%)
SClause: 61256 -> 61257 (+0.00%); split: -0.00%, +0.01%
Copies: 250037 -> 250158 (+0.05%); split: -0.06%, +0.11%
Branches: 101302 -> 101303 (+0.00%)
VALU: 1791447 -> 1791435 (-0.00%); split: -0.00%, +0.00%
SALU: 401898 -> 402007 (+0.03%); split: -0.03%, +0.06%
VOPD: 730 -> 741 (+1.51%)

Totals from 40 (0.06% of 63053) affected shaders: (Vega10)

Instrs: 161584 -> 161567 (-0.01%); split: -0.04%, +0.03%
CodeSize: 891168 -> 891004 (-0.02%); split: -0.04%, +0.03%
Latency: 3550766 -> 3549770 (-0.03%); split: -0.05%, +0.03%
InvThroughput: 2627028 -> 2626484 (-0.02%); split: -0.03%, +0.01%
VClause: 2970 -> 2971 (+0.03%)
SClause: 4203 -> 4205 (+0.05%); split: -0.26%, +0.31%
Copies: 19923 -> 19893 (-0.15%); split: -0.44%, +0.29%
VALU: 116045 -> 116054 (+0.01%); split: -0.01%, +0.02%
SALU: 22100 -> 22066 (-0.15%); split: -0.39%, +0.24%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31769>
 2024-10-23 14:35:29 +00:00
Daniel Schürmann
30d85b23ef aco/spill: fix faulty assertions 
By unintentionally using integer division for score(), these
assertions were likely to be raised by accident.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31769>
 2024-10-23 14:35:29 +00:00
Daniel Schürmann
541cfb21ba aco: don't attempt to spill dead phis 
These don't affect register pressure.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30182>
 2024-08-14 08:11:48 +00:00
Georg Lehmann
dedfff9dbf aco: only set latekill in live_var_analysis 
Cleaner to have this all in one place, in my opinion.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30368>
 2024-08-12 10:31:09 +00:00
Daniel Schürmann
6723128e94 aco/spill: Don't add phi definitions to live-in variables 
Changes are because we don't add artificial uses to the phi definitions anymore.

Totals from 13 (0.02% of 79395) affected shaders: (GFX10.3)

Instrs: 230510 -> 230285 (-0.10%); split: -0.10%, +0.00%
CodeSize: 1269916 -> 1268760 (-0.09%); split: -0.10%, +0.01%
SpillSGPRs: 2057 -> 2058 (+0.05%)
Latency: 2729731 -> 2723103 (-0.24%)
InvThroughput: 696888 -> 695286 (-0.23%)
VClause: 5795 -> 5768 (-0.47%)
SClause: 6855 -> 6858 (+0.04%)
Copies: 32336 -> 32275 (-0.19%); split: -0.22%, +0.03%
VALU: 151782 -> 151731 (-0.03%); split: -0.04%, +0.01%
SALU: 30766 -> 30758 (-0.03%); split: -0.03%, +0.01%
VMEM: 12157 -> 12078 (-0.65%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
 2024-07-16 14:00:49 +00:00
Daniel Schürmann
c146d4b6b6 aco/spill: use live-in variables directly rather than computing them 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
 2024-07-16 14:00:49 +00:00
Daniel Schürmann
6c6f382d68 aco: add RegisterDemand member to Instruction 
Since we never need both at the same time, we can use
a union with pass_flags.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29962>
 2024-07-10 12:31:02 +00:00
Daniel Schürmann
c4a38c6583 aco/spill: don't remove spilled phis 
They will be removed during register allocation.

Few changes due to different phi order.
Totals from 14 (0.02% of 79395) affected shaders: (GFX11)

Instrs: 315724 -> 315675 (-0.02%); split: -0.02%, +0.01%
CodeSize: 1673608 -> 1673268 (-0.02%); split: -0.03%, +0.00%
Latency: 3194243 -> 3189025 (-0.16%); split: -0.19%, +0.03%
InvThroughput: 638369 -> 637323 (-0.16%); split: -0.19%, +0.03%
VClause: 5716 -> 5714 (-0.03%)
Copies: 37786 -> 37748 (-0.10%); split: -0.13%, +0.03%
Branches: 10469 -> 10454 (-0.14%); split: -0.16%, +0.02%
VALU: 182498 -> 182454 (-0.02%); split: -0.03%, +0.00%
SALU: 36038 -> 36046 (+0.02%); split: -0.01%, +0.04%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:34 +00:00
Daniel Schürmann
5cfa5b784b aco: remove get_demand_before() 
The register demand before executing an instruction is now included
in the instruction's register demand and this function is unused.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:34 +00:00
Daniel Schürmann
09f1c40f2e aco: track and use the live-in register demand per basic block 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:34 +00:00
Daniel Schürmann
001c8caae0 aco: calculate register demand per instruction as maximum necessary to execute the instruction 
Previously, the register demand per instruction was calculated as the number of
live variables in the register file after executing an instruction plus additional
temporary registers, necessary during the execution of the instruction.
With this change, now it also includes all variables which are live right before
executing an instruction, i.e. killed Operands.

Care has been taken so that the invariant

register_demand[idx] = register_demand[idx - 1] - get_temp_registers(prev_instr)
                        + get_live_changes(instr) + get_temp_registers(instr)

still holds.

Slight changes in scheduling:

Totals from 316 (0.40% of 79395) affected shaders: (GFX11)

Instrs: 301329 -> 300777 (-0.18%); split: -0.31%, +0.12%
CodeSize: 1577976 -> 1576204 (-0.11%); split: -0.21%, +0.10%
SpillSGPRs: 448 -> 447 (-0.22%)
Latency: 1736349 -> 1726182 (-0.59%); split: -2.01%, +1.42%
InvThroughput: 243894 -> 243883 (-0.00%); split: -0.03%, +0.03%
VClause: 6134 -> 6280 (+2.38%); split: -1.04%, +3.42%
SClause: 6142 -> 6137 (-0.08%); split: -0.13%, +0.05%
Copies: 14037 -> 14032 (-0.04%); split: -0.56%, +0.52%
Branches: 3284 -> 3283 (-0.03%)
VALU: 182750 -> 182718 (-0.02%); split: -0.04%, +0.03%
SALU: 18522 -> 18538 (+0.09%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:33 +00:00
Daniel Schürmann
4c2f231cc0 aco/spill: Unconditionally add 2 SGPRs to live-in demand 
Due to undefined Operands, it might not be enough to check the
predecessors' register demand.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:33 +00:00
Daniel Schürmann
a497d105e3 aco: move live var information into struct Program 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29713>
 2024-06-14 14:32:35 +00:00
Friedrich Vock
ec8512ce85 aco/spill: Don't spill phis with all-undef operands 
Fixes some crashes when limiting RT stages to 128 VGPRs.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29593>
 2024-06-10 19:39:52 +00:00
Rhys Perry
b41f0f6cc1 aco: use ac_hw_cache_flags 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243>
 2024-06-07 13:22:42 +00:00
Samuel Pitoiset
3d6957268b aco: use new common helpers for building buffer descriptors 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29268>
 2024-05-22 08:31:39 +00:00
Friedrich Vock
590ea76104 aco/spill: Insert p_start_linear_vgpr right after p_logical_end 
If p_start_linear_vgpr allocates a VGPR that is already blocked, RA
will try moving the blocking VGPR somewhere else. If
p_start_linear_vgpr is inserted right before the branch, that move will
be inserted after exec has been overwritten, which might cause the move
to be skipped for some threads.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28041>
 2024-05-14 13:54:34 +00:00
Marek Olšák
58a5de5c34 amd: add gfx12 register definitions into the register header generator 
The generator renamed some definitions to resolve conflicts.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29007>
 2024-05-11 22:14:05 -04:00
Samuel Pitoiset
7a69d78ba2 aco: use SPDX-License-Identifier 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622>
 2024-04-08 15:49:25 +00:00
Daniel Schürmann
a4231d4f56 aco/spill: use average use distances to spill loop variables 
Totals from 83 (0.10% of 79395) affected shaders: (GFX11)

Instrs: 3429510 -> 3427917 (-0.05%); split: -0.05%, +0.01%
CodeSize: 17587884 -> 17570224 (-0.10%); split: -0.11%, +0.01%
SpillSGPRs: 4660 -> 4630 (-0.64%); split: -1.07%, +0.43%
Latency: 20054145 -> 20054454 (+0.00%); split: -0.00%, +0.01%
InvThroughput: 4989606 -> 4989740 (+0.00%); split: -0.00%, +0.01%
VClause: 90844 -> 90843 (-0.00%)
SClause: 69534 -> 69535 (+0.00%); split: -0.04%, +0.04%
Copies: 283288 -> 283415 (+0.04%); split: -0.11%, +0.15%
Branches: 113543 -> 113409 (-0.12%); split: -0.12%, +0.01%
VALU: 1888933 -> 1887489 (-0.08%); split: -0.08%, +0.00%
SALU: 423548 -> 423609 (+0.01%); split: -0.07%, +0.09%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
c371882060 aco/spill: use average use distances in init_live_in_vars() for merge blocks 
Totals from 18 (0.02% of 79395) affected shaders: (GFX11)

Instrs: 726400 -> 726409 (+0.00%); split: -0.00%, +0.00%
CodeSize: 3750396 -> 3750420 (+0.00%); split: -0.00%, +0.00%
Latency: 4363014 -> 4363039 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 1483486 -> 1483485 (-0.00%)
SClause: 13393 -> 13392 (-0.01%); split: -0.01%, +0.01%
Copies: 70236 -> 70254 (+0.03%); split: -0.01%, +0.03%
Branches: 23811 -> 23810 (-0.00%)
VALU: 404250 -> 404247 (-0.00%); split: -0.00%, +0.00%
SALU: 92208 -> 92226 (+0.02%); split: -0.01%, +0.02%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
c2ac4f1917 aco/spill: use average use distances in process_block() 
Totals from 128 (0.16% of 79395) affected shaders: (GFX11)

Instrs: 672936 -> 672086 (-0.13%); split: -0.40%, +0.28%
CodeSize: 3574396 -> 3565540 (-0.25%); split: -0.55%, +0.30%
SpillSGPRs: 6254 -> 6636 (+6.11%); split: -0.32%, +6.43%
SpillVGPRs: 967 -> 839 (-13.24%)
Latency: 6489362 -> 6469427 (-0.31%); split: -0.36%, +0.05%
InvThroughput: 2216723 -> 2212417 (-0.19%); split: -0.24%, +0.05%
VClause: 11670 -> 11517 (-1.31%); split: -1.50%, +0.19%
SClause: 15711 -> 15754 (+0.27%); split: -0.29%, +0.57%
Copies: 85185 -> 84155 (-1.21%); split: -2.14%, +0.93%
Branches: 18706 -> 18708 (+0.01%); split: -0.05%, +0.06%
VALU: 386892 -> 386204 (-0.18%); split: -0.40%, +0.23%
SALU: 85754 -> 86211 (+0.53%); split: -0.46%, +1.00%
VMEM: 18777 -> 18524 (-1.35%); split: -1.37%, +0.02%
VOPD: 670 -> 628 (-6.27%); split: +0.75%, -7.01%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
9833120e91 aco/spill: gather information about average use distances 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
b6e56ab356 aco/spill: use live variables instead of next_use_distances in add_coupling_code() 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
7b96482ec9 aco/spill: maintain valid live vars at any point 
In order to do so, we need to process blocks unconditionally.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
fab8757831 aco/spill: compute live-in variables from live-out 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
bfb62fa70d aco/spill: pass live_vars to spill_ctx 
We will use them in future commits.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
66616f0862 aco/spill: Fix assertion for nested loops 
Fixes: 898fd9227a ('aco/spill: keep loop variables spilled during nested loops')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27799>
 2024-03-28 13:57:36 +00:00
Daniel Schürmann
a863c7951e aco: remove create_instruction() template parameter 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
9b0ebcc39b aco: change return type of create_instruction() to Instruction* 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
1187189235 aco: unify different SALU types into single struct SALU_instruction 
This removes
- SOP1_instruction
- SOP2_instruction
- SOPC_instruction
- SOPK_instruction
- SOPP_instruction

and their corresponding methods.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
898fd9227a aco/spill: keep loop variables spilled during nested loops 
Totals from 36 (0.05% of 79395) affected shaders: (GFX11)

Instrs: 2775993 -> 2781480 (+0.20%); split: -0.01%, +0.20%
CodeSize: 14184524 -> 14231956 (+0.33%); split: -0.01%, +0.34%
SpillSGPRs: 1441 -> 1207 (-16.24%)
SpillVGPRs: 477 -> 421 (-11.74%)
Scratch: 35840 -> 35072 (-2.14%)
Latency: 15559241 -> 15606047 (+0.30%); split: -0.01%, +0.31%
InvThroughput: 3787994 -> 3803749 (+0.42%); split: -0.02%, +0.44%
VClause: 76422 -> 76411 (-0.01%)
SClause: 55717 -> 55671 (-0.08%); split: -0.10%, +0.01%
Copies: 214877 -> 215185 (+0.14%); split: -0.04%, +0.18%
Branches: 93158 -> 93129 (-0.03%); split: -0.03%, +0.00%
VALU: 1520556 -> 1525425 (+0.32%); split: -0.00%, +0.32%
SALU: 341104 -> 341408 (+0.09%); split: -0.02%, +0.11%
VMEM: 119673 -> 119617 (-0.05%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
80bc658efc aco/spill: select more loop-carried variables to be spilled 
This changes the heuristic to also spill live-through variables
if they are used within the loop if the register pressure is too high.

Totals from 68 (0.09% of 79395) affected shaders: (GFX11)

Instrs: 3385477 -> 3386659 (+0.03%); split: -0.11%, +0.14%
CodeSize: 17329668 -> 17353796 (+0.14%); split: -0.12%, +0.26%
SpillSGPRs: 4246 -> 3659 (-13.82%); split: -17.12%, +3.30%
SpillVGPRs: 978 -> 884 (-9.61%); split: -10.02%, +0.41%
Scratch: 58880 -> 59648 (+1.30%)
Latency: 20567445 -> 20753412 (+0.90%); split: -0.08%, +0.98%
InvThroughput: 5091128 -> 5188098 (+1.90%); split: -0.07%, +1.97%
VClause: 90446 -> 90499 (+0.06%); split: -0.05%, +0.11%
SClause: 68270 -> 68072 (-0.29%); split: -0.35%, +0.06%
Copies: 280689 -> 278003 (-0.96%); split: -1.08%, +0.12%
Branches: 112577 -> 112608 (+0.03%); split: -0.10%, +0.13%
VALU: 1863415 -> 1866878 (+0.19%); split: -0.07%, +0.26%
SALU: 415572 -> 413062 (-0.60%); split: -0.63%, +0.03%
VMEM: 140890 -> 141045 (+0.11%); split: -0.04%, +0.15%
VOPD: 650 -> 670 (+3.08%); split: +3.54%, -0.46%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
31f6aebb2c aco/spill: keep loop-carried variables spilled at merge blocks 
If loop-carried variables are partially spilled at some predecessor,
keep them spilled without extra instruction.

Also slightly refactors the code for better readability.

Totals from 13 (0.02% of 79395) affected shaders: (GFX11)

Instrs: 259129 -> 258194 (-0.36%)
CodeSize: 1337604 -> 1331768 (-0.44%)
SpillSGPRs: 561 -> 560 (-0.18%)
SpillVGPRs: 146 -> 140 (-4.11%)
Latency: 2410886 -> 2408560 (-0.10%)
InvThroughput: 759229 -> 758416 (-0.11%)
VClause: 4178 -> 4154 (-0.57%)
SClause: 7217 -> 7219 (+0.03%)
Copies: 32411 -> 32174 (-0.73%)
Branches: 9426 -> 9296 (-1.38%)
VALU: 144602 -> 144098 (-0.35%)
SALU: 30582 -> 30405 (-0.58%)
VMEM: 6961 -> 6908 (-0.76%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
971407bd2f aco/spill: keep loop-carried variables spilled at loop headers 
Now, that we avoid re-spilling loop-carried variables, we can
keep them spilled at loop-headers.

Totals from 31 (0.04% of 79395) affected shaders: (GFX11)

Instrs: 1539175 -> 1538109 (-0.07%); split: -0.14%, +0.07%
CodeSize: 7877948 -> 7871916 (-0.08%); split: -0.16%, +0.08%
SpillSGPRs: 1936 -> 1523 (-21.33%)
SpillVGPRs: 160 -> 146 (-8.75%)
Scratch: 18176 -> 16896 (-7.04%)
Latency: 9439964 -> 9441197 (+0.01%); split: -0.01%, +0.02%
InvThroughput: 2406253 -> 2406700 (+0.02%); split: -0.01%, +0.03%
VClause: 38903 -> 38922 (+0.05%)
SClause: 33347 -> 33349 (+0.01%); split: -0.01%, +0.02%
Copies: 131420 -> 130974 (-0.34%); split: -0.57%, +0.23%
Branches: 53091 -> 53212 (+0.23%); split: -0.03%, +0.26%
VALU: 844372 -> 843876 (-0.06%); split: -0.12%, +0.06%
SALU: 190329 -> 189891 (-0.23%); split: -0.40%, +0.17%
VMEM: 60491 -> 60510 (+0.03%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
e968a18652 aco/spill: avoid re-spilling loop-carried variables in add_coupling_code() 
These variables only need to be spilled once at the loop-preheader.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
7726b92707 aco/spill: avoid re-spilling loop-carried variables in process_block() 
If a variable is spilled at the loop-header of the current loop,
there is no need to spill it redundantly. Instead, re-use the
spill-slot from the loop-header.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
bb3b6a5ff3 aco/spill: add interferences with variables spilled at loop headers 
We will re-use the spill slots rather than re-spilling the variables.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
09c5414760 aco/spill: don't prefer to spill phis at merge blocks 
Totals from 44 (0.06% of 79242) affected shaders: (GFX11)

Instrs: 165578 -> 165545 (-0.02%); split: -0.08%, +0.06%
CodeSize: 833528 -> 832988 (-0.06%); split: -0.11%, +0.04%
SpillSGPRs: 346 -> 329 (-4.91%)
Latency: 1176341 -> 1176231 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 235771 -> 235761 (-0.00%); split: -0.01%, +0.01%
SClause: 3287 -> 3289 (+0.06%); split: -0.21%, +0.27%
Copies: 16290 -> 16299 (+0.06%); split: -0.62%, +0.68%
Branches: 3633 -> 3641 (+0.22%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
d234f789b5 aco/spill: keep live-out variables spilled at branch blocks 
There is no reason to reload early. Leave scheduling to the scheduler.

Totals from 98 (0.12% of 79242) affected shaders: (GFX11)

Instrs: 841944 -> 841995 (+0.01%); split: -0.02%, +0.03%
CodeSize: 4354332 -> 4354544 (+0.00%); split: -0.02%, +0.02%
SpillSGPRs: 2557 -> 2554 (-0.12%)
Latency: 5772080 -> 5772209 (+0.00%); split: -0.00%, +0.01%
InvThroughput: 1694834 -> 1694873 (+0.00%); split: -0.00%, +0.00%
SClause: 16036 -> 16042 (+0.04%); split: -0.07%, +0.11%
Copies: 85117 -> 85125 (+0.01%); split: -0.18%, +0.19%
Branches: 25222 -> 25219 (-0.01%); split: -0.02%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
32882d647e aco/spill: refactor adding spilled vars into separate function add_to_spills() 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
a5e8f9f879 aco/spill: add spills_entry interferences only when necessary 
For variables which are already spilled in previous blocks,
the interferences already exist.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
1a1d6aef56 aco/spill: don't allocate extra spill_id for phi operands in add_coupling_code() 
We can re-use the phi definition's spill_id.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:20 +00:00
Daniel Schürmann
b1468fc952 aco/spill: refactor SSA repairing 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27774>
 2024-03-25 09:21:19 +00:00
Daniel Schürmann
9bbb9f1104 aco: use small_vec as Block::edge_vec for predecessors and successors 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27984>
 2024-03-19 13:06:58 +00:00
Rhys Perry
07eab4ab40 aco: require linear vgpr uses to be late kill 
This also removes some linear VGPR tests which will be replaced soon
anyway.

fossil-db (navi31):
Totals from 107 (0.14% of 79242) affected shaders:
Instrs: 66203 -> 66211 (+0.01%); split: -0.09%, +0.10%
CodeSize: 354644 -> 354588 (-0.02%); split: -0.08%, +0.07%
VGPRs: 4476 -> 4452 (-0.54%); split: -0.80%, +0.27%
Latency: 513863 -> 513877 (+0.00%); split: -0.08%, +0.08%
InvThroughput: 68871 -> 68870 (-0.00%); split: -0.02%, +0.02%
SClause: 1589 -> 1590 (+0.06%)
PreVGPRs: 3404 -> 3415 (+0.32%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27697>
 2024-03-06 12:55:45 +00:00
Daniel Schürmann
e3098bb232 aco: give spiller more room to assign spilled SGPRs to VGPRs 
On chordal graphs, a greedy coloring can be done in a way that never uses
more colors than are required for the largest clique. However, since we
have vector values and force phi resources into the same spill slots, the
interference graphs are not chordal, and thus, this assumption doesn't hold.

Use twice as many spill slots as upper bound.

Totals from 10 (0.01% of 79242) affected shaders: (GFX11)
MaxWaves: 52 -> 54 (+3.85%)
Instrs: 271386 -> 271779 (+0.14%)
CodeSize: 1362544 -> 1365432 (+0.21%)
VGPRs: 2536 -> 2532 (-0.16%)
SpillVGPRs: 778 -> 818 (+5.14%)
Scratch: 73472 -> 76800 (+4.53%)
Latency: 3331718 -> 3328798 (-0.09%); split: -0.14%, +0.05%
InvThroughput: 1665860 -> 1643350 (-1.35%); split: -1.40%, +0.05%
VClause: 3292 -> 3329 (+1.12%); split: -0.06%, +1.18%
Copies: 46082 -> 46257 (+0.38%)

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27011>
 2024-01-19 14:15:27 +00:00
Konstantin Seurer
24555cdbbd aco/spill: Make sure that offset stays in bounds 
If a shader spills a lot, the offset can be above the HW limit.

cc: mesa-stable

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24652>
 2023-09-22 06:30:01 +00:00
Rhys Perry
c7bb2f7bb9 aco/spill: add all live-in to merge block spill candidates 
Previously, only already spilled live-in or phis were added to the spill
candidates. Because of branch definitions, this might not be enough.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9722
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24949>
 2023-08-30 15:20:02 +00:00
Rhys Perry
cb096b85ff aco/spill: skip p_branch in process_block 
Fixes compilation of a Dead by Daylight shader.

fossil-db (gfx1100):
Totals from 58 (0.04% of 133461) affected shaders:
Instrs: 319824 -> 319421 (-0.13%); split: -0.13%, +0.00%
CodeSize: 1711260 -> 1708744 (-0.15%); split: -0.15%, +0.00%
SpillSGPRs: 2567 -> 2459 (-4.21%)
Latency: 3274930 -> 3274921 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 924106 -> 924105 (-0.00%); split: -0.00%, +0.00%
Copies: 41883 -> 41757 (-0.30%); split: -0.31%, +0.00%
Branches: 9144 -> 9146 (+0.02%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9599
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24896>
 2023-08-29 11:53:57 +00:00