Georg Lehmann
df1de388a3
aco/sched_ilp: reorder VINTRP
...
VINTRP(gfx6-gfx10.3) is mostly just VALU, but we treated it like memory
instructions as an afterthought. This had issues as VINTRP was never reordered
with itself, or other memory instructions. Reordering VINTRP in clauses
increases ILP. We don't really need collect_clause_dependencies for VINTRP
either, because they ususally have the same dependencies already. That means
we can still form VINTRP clauses by selecting preferably VINTRP after a
previous one.
Foz-DB Navi21:
Totals from 34184 (43.16% of 79206) affected shaders:
Instrs: 18811270 -> 18812046 (+0.00%); split: -0.01%, +0.02%
CodeSize: 103627276 -> 103630056 (+0.00%); split: -0.01%, +0.01%
Latency: 188379364 -> 187936731 (-0.23%); split: -0.27%, +0.03%
InvThroughput: 42600163 -> 42590608 (-0.02%); split: -0.03%, +0.00%
VClause: 378960 -> 378912 (-0.01%); split: -0.02%, +0.00%
SClause: 727560 -> 720573 (-0.96%); split: -1.08%, +0.12%
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111 >
2025-01-27 11:59:45 +00:00
Georg Lehmann
45ccd6487f
aco/sched_ilp: only remove WaW/WaR for inter clause dependencies
...
Direct RaW shouldn't be removed, because the clause is split by a wait anyway.
Foz-DB Navi21:
Totals from 52 (0.07% of 79206) affected shaders:
Instrs: 1603523 -> 1603485 (-0.00%); split: -0.00%, +0.00%
CodeSize: 8223048 -> 8222788 (-0.00%)
Latency: 9741674 -> 9738884 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 2322621 -> 2322010 (-0.03%); split: -0.03%, +0.00%
SClause: 31325 -> 31320 (-0.02%); split: -0.02%, +0.01%
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111 >
2025-01-27 11:59:45 +00:00
Georg Lehmann
4b332afb32
aco/sched_ilp: add dependencies of later clause instrs more aggressively
...
Foz-DB GFX1150:
Totals from 22246 (28.03% of 79377) affected shaders:
Instrs: 22689053 -> 22684012 (-0.02%); split: -0.06%, +0.03%
CodeSize: 117622416 -> 117603292 (-0.02%); split: -0.04%, +0.03%
Latency: 182725630 -> 182702465 (-0.01%); split: -0.06%, +0.05%
InvThroughput: 37963256 -> 37956961 (-0.02%); split: -0.03%, +0.02%
VClause: 471019 -> 467248 (-0.80%)
SClause: 592620 -> 590034 (-0.44%); split: -0.44%, +0.01%
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111 >
2025-01-27 11:59:45 +00:00
Georg Lehmann
ea514e9385
aco/sched_ilp: continue open clauses
...
Foz-DB GFX1150:
Totals from 13789 (17.37% of 79395) affected shaders:
Instrs: 16567395 -> 16570832 (+0.02%); split: -0.03%, +0.05%
CodeSize: 85737492 -> 85751072 (+0.02%); split: -0.02%, +0.04%
Latency: 140988872 -> 140831767 (-0.11%); split: -0.12%, +0.01%
InvThroughput: 29639206 -> 29614890 (-0.08%); split: -0.09%, +0.00%
VClause: 347065 -> 343779 (-0.95%); split: -0.96%, +0.01%
SClause: 424881 -> 418657 (-1.46%); split: -1.48%, +0.02%
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111 >
2025-01-27 11:59:45 +00:00
Rhys Perry
4574cd9fae
aco: form export clauses
...
This is useful for the export priority workaround. It also moves
copy/conversion instructions up from in-between exports.
fossil-db (gfx1150):
Totals from 9974 (12.56% of 79395) affected shaders:
Instrs: 7038168 -> 6949186 (-1.26%)
CodeSize: 37988148 -> 37632220 (-0.94%)
Latency: 44759046 -> 44324922 (-0.97%); split: -0.97%, +0.00%
InvThroughput: 8213681 -> 8201222 (-0.15%); split: -0.15%, +0.00%
fossil-db (navi21):
Totals from 16785 (21.14% of 79395) affected shaders:
Instrs: 9282414 -> 9282302 (-0.00%)
CodeSize: 50252480 -> 50252032 (-0.00%)
Latency: 59716988 -> 59610334 (-0.18%); split: -0.18%, +0.00%
InvThroughput: 12681965 -> 12680380 (-0.01%); split: -0.01%, +0.00%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30241 >
2024-07-23 13:14:52 +00:00
Rhys Perry
52e9370c13
aco: replace constant v_bfrev_b32 with v_mov_b32 to create vopd
...
fossil-db (navi31, wave32):
Totals from 1523 (1.92% of 79395) affected shaders:
Instrs: 1502625 -> 1501998 (-0.04%); split: -0.05%, +0.01%
CodeSize: 8980508 -> 8983032 (+0.03%); split: -0.00%, +0.03%
Latency: 8405687 -> 8405375 (-0.00%); split: -0.01%, +0.01%
InvThroughput: 1567484 -> 1566728 (-0.05%); split: -0.05%, +0.00%
VALU: 732709 -> 732058 (-0.09%); split: -0.09%, +0.00%
VOPD: 158191 -> 158842 (+0.41%); split: +0.41%, -0.00%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29912 >
2024-07-01 17:34:22 +00:00
Rhys Perry
ae18c88409
aco/gfx12: implement workgroup barrier
...
Same sequence LLVM uses for llvm.amdgcn.s.barrier.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29330 >
2024-05-28 10:52:11 +00:00
Daniel Schürmann
a863c7951e
aco: remove create_instruction() template parameter
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
Daniel Schürmann
9b0ebcc39b
aco: change return type of create_instruction() to Instruction*
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370 >
2024-03-28 11:25:43 +00:00
Rhys Perry
ea92aea9f2
aco: turn v_mov_b32 into addition to create VOPD instructions
...
fossil-db (navi31, wave32):
Totals from 15655 (19.76% of 79242) affected shaders:
Instrs: 10699119 -> 10688239 (-0.10%); split: -0.11%, +0.00%
CodeSize: 61290308 -> 61288596 (-0.00%); split: -0.01%, +0.00%
Latency: 89159743 -> 89150355 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 16966295 -> 16955427 (-0.06%); split: -0.07%, +0.00%
VALU: 5484626 -> 5473993 (-0.19%); split: -0.20%, +0.00%
VOPD: 1446725 -> 1457358 (+0.73%); split: +0.74%, -0.01%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27485 >
2024-02-08 12:15:23 +00:00
Rhys Perry
65dfb27f8f
aco: swap operands to create VOPD instructions
...
fossil-db (navi31, wave32):
Totals from 61565 (77.69% of 79242) affected shaders:
Instrs: 34874995 -> 34439344 (-1.25%); split: -1.27%, +0.02%
CodeSize: 200611536 -> 200564028 (-0.02%); split: -0.12%, +0.10%
Latency: 242520024 -> 242120510 (-0.16%); split: -0.28%, +0.11%
InvThroughput: 50236383 -> 49588742 (-1.29%); split: -1.31%, +0.02%
VClause: 713308 -> 712902 (-0.06%); split: -0.07%, +0.01%
SClause: 1184865 -> 1184620 (-0.02%); split: -0.03%, +0.01%
VALU: 18235068 -> 17803847 (-2.36%); split: -2.37%, +0.00%
VOPD: 3930904 -> 4362125 (+10.97%); split: +10.99%, -0.02%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27485 >
2024-02-08 12:15:23 +00:00
Rhys Perry
96d8b7c59c
aco: refactor create_vopd_instruction
...
Prepare for operand swapping.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27485 >
2024-02-08 12:15:23 +00:00
Rhys Perry
75a76ec3fd
aco: implement VOPD scheduler
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367 >
2024-02-05 10:51:15 +00:00
Rhys Perry
1fb79b4aa2
aco: refactor schedule_ilp main loop
...
Besides switching to "ctx.active_mask" as the condition, this is basically
the same.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23367 >
2024-02-05 10:51:14 +00:00
Daniel Schürmann
023e78b4d7
aco: add new post-RA scheduler for ILP
...
Totals from 77247 (97.37% of 79330) affected shaders: (GFX11)
Instrs: 44371374 -> 43215723 (-2.60%); split: -2.64%, +0.03%
CodeSize: 227819532 -> 223188224 (-2.03%); split: -2.06%, +0.03%
Latency: 301016823 -> 290147626 (-3.61%); split: -3.70%, +0.09%
InvThroughput: 48551749 -> 47646212 (-1.87%); split: -1.88%, +0.01%
VClause: 870581 -> 834655 (-4.13%); split: -4.13%, +0.00%
SClause: 1487061 -> 1340851 (-9.83%); split: -9.83%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25676 >
2024-01-06 11:30:42 +00:00