Commit graph

15 commits

Author SHA1 Message Date
Georg Lehmann
df1de388a3 aco/sched_ilp: reorder VINTRP
VINTRP(gfx6-gfx10.3) is mostly just VALU, but we treated it like memory
instructions as an afterthought. This had issues as VINTRP was never reordered
with itself, or other memory instructions. Reordering VINTRP in clauses
increases ILP. We don't really need collect_clause_dependencies for VINTRP
either, because they ususally have the same dependencies already. That means
we can still form VINTRP clauses by selecting preferably VINTRP after a
previous one.

Foz-DB Navi21:
Totals from 34184 (43.16% of 79206) affected shaders:
Instrs: 18811270 -> 18812046 (+0.00%); split: -0.01%, +0.02%
CodeSize: 103627276 -> 103630056 (+0.00%); split: -0.01%, +0.01%
Latency: 188379364 -> 187936731 (-0.23%); split: -0.27%, +0.03%
InvThroughput: 42600163 -> 42590608 (-0.02%); split: -0.03%, +0.00%
VClause: 378960 -> 378912 (-0.01%); split: -0.02%, +0.00%
SClause: 727560 -> 720573 (-0.96%); split: -1.08%, +0.12%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111>
2025-01-27 11:59:45 +00:00
Georg Lehmann
45ccd6487f aco/sched_ilp: only remove WaW/WaR for inter clause dependencies
Direct RaW shouldn't be removed, because the clause is split by a wait anyway.

Foz-DB Navi21:
Totals from 52 (0.07% of 79206) affected shaders:
Instrs: 1603523 -> 1603485 (-0.00%); split: -0.00%, +0.00%
CodeSize: 8223048 -> 8222788 (-0.00%)
Latency: 9741674 -> 9738884 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 2322621 -> 2322010 (-0.03%); split: -0.03%, +0.00%
SClause: 31325 -> 31320 (-0.02%); split: -0.02%, +0.01%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111>
2025-01-27 11:59:45 +00:00
Georg Lehmann
4b332afb32 aco/sched_ilp: add dependencies of later clause instrs more aggressively
Foz-DB GFX1150:
Totals from 22246 (28.03% of 79377) affected shaders:
Instrs: 22689053 -> 22684012 (-0.02%); split: -0.06%, +0.03%
CodeSize: 117622416 -> 117603292 (-0.02%); split: -0.04%, +0.03%
Latency: 182725630 -> 182702465 (-0.01%); split: -0.06%, +0.05%
InvThroughput: 37963256 -> 37956961 (-0.02%); split: -0.03%, +0.02%
VClause: 471019 -> 467248 (-0.80%)
SClause: 592620 -> 590034 (-0.44%); split: -0.44%, +0.01%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111>
2025-01-27 11:59:45 +00:00
Georg Lehmann
ea514e9385 aco/sched_ilp: continue open clauses
Foz-DB GFX1150:
Totals from 13789 (17.37% of 79395) affected shaders:
Instrs: 16567395 -> 16570832 (+0.02%); split: -0.03%, +0.05%
CodeSize: 85737492 -> 85751072 (+0.02%); split: -0.02%, +0.04%
Latency: 140988872 -> 140831767 (-0.11%); split: -0.12%, +0.01%
InvThroughput: 29639206 -> 29614890 (-0.08%); split: -0.09%, +0.00%
VClause: 347065 -> 343779 (-0.95%); split: -0.96%, +0.01%
SClause: 424881 -> 418657 (-1.46%); split: -1.48%, +0.02%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111>
2025-01-27 11:59:45 +00:00
Rhys Perry
4574cd9fae aco: form export clauses
This is useful for the export priority workaround. It also moves
copy/conversion instructions up from in-between exports.

fossil-db (gfx1150):
Totals from 9974 (12.56% of 79395) affected shaders:
Instrs: 7038168 -> 6949186 (-1.26%)
CodeSize: 37988148 -> 37632220 (-0.94%)
Latency: 44759046 -> 44324922 (-0.97%); split: -0.97%, +0.00%
InvThroughput: 8213681 -> 8201222 (-0.15%); split: -0.15%, +0.00%

fossil-db (navi21):
Totals from 16785 (21.14% of 79395) affected shaders:
Instrs: 9282414 -> 9282302 (-0.00%)
CodeSize: 50252480 -> 50252032 (-0.00%)
Latency: 59716988 -> 59610334 (-0.18%); split: -0.18%, +0.00%
InvThroughput: 12681965 -> 12680380 (-0.01%); split: -0.01%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30241>
2024-07-23 13:14:52 +00:00
Rhys Perry
52e9370c13 aco: replace constant v_bfrev_b32 with v_mov_b32 to create vopd
fossil-db (navi31, wave32):
Totals from 1523 (1.92% of 79395) affected shaders:
Instrs: 1502625 -> 1501998 (-0.04%); split: -0.05%, +0.01%
CodeSize: 8980508 -> 8983032 (+0.03%); split: -0.00%, +0.03%
Latency: 8405687 -> 8405375 (-0.00%); split: -0.01%, +0.01%
InvThroughput: 1567484 -> 1566728 (-0.05%); split: -0.05%, +0.00%
VALU: 732709 -> 732058 (-0.09%); split: -0.09%, +0.00%
VOPD: 158191 -> 158842 (+0.41%); split: +0.41%, -0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29912>
2024-07-01 17:34:22 +00:00
Rhys Perry
ae18c88409 aco/gfx12: implement workgroup barrier
Same sequence LLVM uses for llvm.amdgcn.s.barrier.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29330>
2024-05-28 10:52:11 +00:00
Daniel Schürmann
a863c7951e aco: remove create_instruction() template parameter
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
2024-03-28 11:25:43 +00:00
Daniel Schürmann
9b0ebcc39b aco: change return type of create_instruction() to Instruction*
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
2024-03-28 11:25:43 +00:00
Rhys Perry
ea92aea9f2 aco: turn v_mov_b32 into addition to create VOPD instructions
fossil-db (navi31, wave32):
Totals from 15655 (19.76% of 79242) affected shaders:
Instrs: 10699119 -> 10688239 (-0.10%); split: -0.11%, +0.00%
CodeSize: 61290308 -> 61288596 (-0.00%); split: -0.01%, +0.00%
Latency: 89159743 -> 89150355 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 16966295 -> 16955427 (-0.06%); split: -0.07%, +0.00%
VALU: 5484626 -> 5473993 (-0.19%); split: -0.20%, +0.00%
VOPD: 1446725 -> 1457358 (+0.73%); split: +0.74%, -0.01%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27485>
2024-02-08 12:15:23 +00:00
Rhys Perry
65dfb27f8f aco: swap operands to create VOPD instructions
fossil-db (navi31, wave32):
Totals from 61565 (77.69% of 79242) affected shaders:
Instrs: 34874995 -> 34439344 (-1.25%); split: -1.27%, +0.02%
CodeSize: 200611536 -> 200564028 (-0.02%); split: -0.12%, +0.10%
Latency: 242520024 -> 242120510 (-0.16%); split: -0.28%, +0.11%
InvThroughput: 50236383 -> 49588742 (-1.29%); split: -1.31%, +0.02%
VClause: 713308 -> 712902 (-0.06%); split: -0.07%, +0.01%
SClause: 1184865 -> 1184620 (-0.02%); split: -0.03%, +0.01%
VALU: 18235068 -> 17803847 (-2.36%); split: -2.37%, +0.00%
VOPD: 3930904 -> 4362125 (+10.97%); split: +10.99%, -0.02%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27485>
2024-02-08 12:15:23 +00:00
Rhys Perry
96d8b7c59c aco: refactor create_vopd_instruction
Prepare for operand swapping.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27485>
2024-02-08 12:15:23 +00:00
Rhys Perry
75a76ec3fd aco: implement VOPD scheduler
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367>
2024-02-05 10:51:15 +00:00
Rhys Perry
1fb79b4aa2 aco: refactor schedule_ilp main loop
Besides switching to "ctx.active_mask" as the condition, this is basically
the same.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367>
2024-02-05 10:51:14 +00:00
Daniel Schürmann
023e78b4d7 aco: add new post-RA scheduler for ILP
Totals from 77247 (97.37% of 79330) affected shaders: (GFX11)

Instrs: 44371374 -> 43215723 (-2.60%); split: -2.64%, +0.03%
CodeSize: 227819532 -> 223188224 (-2.03%); split: -2.06%, +0.03%
Latency: 301016823 -> 290147626 (-3.61%); split: -3.70%, +0.09%
InvThroughput: 48551749 -> 47646212 (-1.87%); split: -1.88%, +0.01%
VClause: 870581 -> 834655 (-4.13%); split: -4.13%, +0.00%
SClause: 1487061 -> 1340851 (-9.83%); split: -9.83%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25676>
2024-01-06 11:30:42 +00:00