fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2026-05-20 15:38:19 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 4
201490 commits 66 branches 936 tags 985 MiB
Commit graph

3560 commits

Author SHA1 Message Date
Rhys Perry
539f9b4ba6 nir,aco,radv: add align_mul/offset to buffer_amd intrinsics 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29242>
 2025-02-07 13:52:57 +00:00
Daniel Schürmann
1a8a643bbd aco/isel: track control flow divergence in loops more accurately 
We introduce two new variables, cf_context::in_divergent_cf and
cf_context::parent_loop.has_divergent_break, in order to determine
whether there is any other invocations on a different CF path.

Totals from 1305 (1.64% of 79395) affected shaders: (Navi31)

Instrs: 659211 -> 657815 (-0.21%); split: -0.22%, +0.01%
CodeSize: 3483228 -> 3477960 (-0.15%); split: -0.16%, +0.01%
VGPRs: 68820 -> 48048 (-30.18%)
Latency: 14197750 -> 14170767 (-0.19%); split: -0.26%, +0.07%
InvThroughput: 1619103 -> 1619826 (+0.04%); split: -0.02%, +0.07%
VClause: 12384 -> 12350 (-0.27%)
SClause: 26693 -> 26844 (+0.57%); split: -0.01%, +0.57%
Copies: 44994 -> 43535 (-3.24%); split: -3.26%, +0.02%
PreSGPRs: 49007 -> 48907 (-0.20%)
PreVGPRs: 32171 -> 32121 (-0.16%)
VALU: 349984 -> 349857 (-0.04%); split: -0.04%, +0.00%
SALU: 84252 -> 83988 (-0.31%); split: -0.32%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33206>
 2025-02-05 10:54:21 +00:00
Daniel Schürmann
583c3586fe aco/isel: remove loop nest information from exec_info 
Since we never enter loops with an empty exec mask, and the
control flow is structured, we don't need to consider the
loop nest depth.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33206>
 2025-02-05 10:54:21 +00:00
Daniel Schürmann
a77258346c aco/isel: fix assumptions about potential empty exec mask in nested control flow 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33206>
 2025-02-05 10:54:21 +00:00
Daniel Schürmann
44216e035f aco/isel: add and use exec_info::empty() helper 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33206>
 2025-02-05 10:54:21 +00:00
Daniel Schürmann
8e8398832c aco/isel: use cf_context in loop_context to restore cf information 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33206>
 2025-02-05 10:54:21 +00:00
Daniel Schürmann
8b9c9fb904 aco/isel: use cf_context in if_context to restore cf information 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33206>
 2025-02-05 10:54:21 +00:00
Daniel Schürmann
c2bfc05d71 aco/isel: rename cf_context::has_divergent_branch 
Make it more consistent with cf_context::has_branch.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33206>
 2025-02-05 10:54:21 +00:00
Daniel Schürmann
0c5a91b9f2 aco/isel: move cf_info into separate struct cf_context 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33206>
 2025-02-05 10:54:21 +00:00
Daniel Schürmann
61fa007e48 aco/isel: fix empty exec tracking for uniform branches 
Totals from 5 (0.01% of 79395) affected shaders: (Navi31)

Instrs: 54730 -> 54715 (-0.03%)
CodeSize: 276928 -> 276852 (-0.03%)
Latency: 215212 -> 214874 (-0.16%)
InvThroughput: 40154 -> 40150 (-0.01%)
Copies: 6824 -> 6821 (-0.04%); split: -0.06%, +0.01%
Branches: 1625 -> 1615 (-0.62%)
SALU: 5682 -> 5678 (-0.07%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33206>
 2025-02-05 10:54:21 +00:00
Konstantin Seurer
60a20bcf3d nir: Stop using instructions for debug info 
Annotating ssa defs without affecting compilation is impossible with
debug info instructions since referencing a nir_def from the debug info
instr will add uses.

The old approach also stops worrking if passes reorder instructions.

This patch proposes a solution which should not regress performance just
like the old approach. The difference is that this one allocates a bit
more space for debug info instead of adding a new instruction for it.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33141>
 2025-01-30 20:14:01 +00:00
Timur Kristóf
e9069eec8a aco: Move NGG pos export scheduling determination to drivers. 
And don't schedule them on GFX11+ at all.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33218>
 2025-01-30 15:26:45 +00:00
Daniel Schürmann
65f95ae74e aco/insert_NOPs: implement VALU -> VALU case for VALUReadSGPRHazard on GFX12 
Totals from 36918 (46.50% of 79395) affected shaders: (GFX1200)

Instrs: 34997889 -> 35296429 (+0.85%); split: -0.00%, +0.85%
CodeSize: 186161112 -> 187334364 (+0.63%); split: -0.00%, +0.63%
Latency: 250265551 -> 250330784 (+0.03%); split: -0.00%, +0.03%
InvThroughput: 41185298 -> 41192503 (+0.02%); split: -0.00%, +0.02%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32682>
 2025-01-30 03:13:16 +00:00
Daniel Schürmann
6c7355f0e6 aco/insert_NOPs: refactor VALUReadSGPRHazard detection 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32682>
 2025-01-30 03:13:16 +00:00
Georg Lehmann
b23ff87db4 aco/sched_ilp: base latency and issue cycles on aco_statistics 
This matters for trans and scalar fpu instructions.

Foz-DB GFX1150:
Totals from 53894 (67.90% of 79377) affected shaders:
Instrs: 38528421 -> 38481337 (-0.12%); split: -0.16%, +0.04%
CodeSize: 200206016 -> 200023916 (-0.09%); split: -0.12%, +0.03%
Latency: 265011734 -> 264303762 (-0.27%); split: -0.28%, +0.02%
InvThroughput: 53804490 -> 53696097 (-0.20%); split: -0.21%, +0.01%
VClause: 736996 -> 736988 (-0.00%); split: -0.00%, +0.00%
SClause: 1118494 -> 1118474 (-0.00%); split: -0.01%, +0.01%
VALU: 21982349 -> 21982358 (+0.00%); split: -0.00%, +0.00%

Foz-DB Navi31:
Totals from 50791 (63.99% of 79377) affected shaders:
Instrs: 37511862 -> 37495712 (-0.04%); split: -0.11%, +0.07%
CodeSize: 197990892 -> 197925104 (-0.03%); split: -0.09%, +0.06%
Latency: 261929261 -> 261273534 (-0.25%); split: -0.27%, +0.01%
InvThroughput: 43978329 -> 43921618 (-0.13%); split: -0.14%, +0.01%
VClause: 727683 -> 727695 (+0.00%); split: -0.00%, +0.00%
SClause: 1092527 -> 1092544 (+0.00%); split: -0.01%, +0.01%
VALU: 22646553 -> 22646566 (+0.00%)

Foz-DB Navi21:
Totals from 43899 (55.30% of 79377) affected shaders:
Instrs: 35649081 -> 35649110 (+0.00%); split: -0.00%, +0.00%
CodeSize: 192336212 -> 192337276 (+0.00%); split: -0.00%, +0.00%
Latency: 270621538 -> 270221431 (-0.15%); split: -0.16%, +0.02%
InvThroughput: 66757841 -> 66715918 (-0.06%); split: -0.07%, +0.01%
VClause: 734884 -> 734867 (-0.00%); split: -0.01%, +0.01%
SClause: 1072956 -> 1072951 (-0.00%); split: -0.01%, +0.01%

Foz-DB Vega10:
Totals from 52687 (83.60% of 63026) affected shaders:
Instrs: 24595280 -> 24595693 (+0.00%); split: -0.01%, +0.01%
CodeSize: 127199836 -> 127200164 (+0.00%); split: -0.01%, +0.01%
Latency: 252281578 -> 252497934 (+0.09%); split: -0.03%, +0.12%
InvThroughput: 136551527 -> 136577609 (+0.02%); split: -0.01%, +0.03%
VClause: 536798 -> 536718 (-0.01%); split: -0.04%, +0.03%
SClause: 819978 -> 819693 (-0.03%); split: -0.04%, +0.01%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33222>
 2025-01-28 17:00:45 +00:00
Georg Lehmann
068f9b51de aco/sched_ilp: use more realistic memory latencies 
The last commit changes order of instructions more aggressively,
and because the memory load latencies here are wastly underestimated,
it ruins some of the work of pre-RA memory scheduling.
With the new heuristic large latency values work fine, so use them.

Foz-DB GFX1150:
Totals from 71343 (89.88% of 79377) affected shaders:
Instrs: 41627671 -> 41915029 (+0.69%); split: -0.01%, +0.70%
CodeSize: 215901308 -> 217051132 (+0.53%); split: -0.01%, +0.54%
Latency: 288714439 -> 286556159 (-0.75%); split: -0.76%, +0.02%
InvThroughput: 55834139 -> 55645301 (-0.34%); split: -0.35%, +0.01%
VClause: 829066 -> 828984 (-0.01%); split: -0.04%, +0.03%
SClause: 1237366 -> 1237448 (+0.01%); split: -0.02%, +0.02%
VALU: 23643291 -> 23643292 (+0.00%); split: -0.00%, +0.00%

Foz-DB Navi31:
Totals from 70576 (88.91% of 79377) affected shaders:
Instrs: 40928125 -> 41211820 (+0.69%); split: -0.01%, +0.70%
CodeSize: 215770956 -> 216897948 (+0.52%); split: -0.00%, +0.53%
Latency: 288139802 -> 286038405 (-0.73%); split: -0.75%, +0.02%
InvThroughput: 46391629 -> 46300275 (-0.20%); split: -0.20%, +0.01%
VClause: 829987 -> 829997 (+0.00%); split: -0.02%, +0.02%
SClause: 1229345 -> 1229425 (+0.01%); split: -0.02%, +0.02%
VALU: 24515334 -> 24515335 (+0.00%)

Foz-DB Navi21:
Instrs: 45512672 -> 45527322 (+0.03%); split: -0.01%, +0.04%
CodeSize: 244254716 -> 244311472 (+0.02%); split: -0.01%, +0.03%
Latency: 314034443 -> 311473726 (-0.82%); split: -0.83%, +0.01%
InvThroughput: 73373201 -> 73220438 (-0.21%); split: -0.21%, +0.00%
VClause: 914819 -> 914853 (+0.00%); split: -0.02%, +0.02%
SClause: 1283331 -> 1283302 (-0.00%); split: -0.01%, +0.01%

Foz-DB Vega10:
Totals from 41908 (66.49% of 63026) affected shaders:
Instrs: 22770415 -> 22779136 (+0.04%); split: -0.01%, +0.04%
CodeSize: 118195752 -> 118230540 (+0.03%); split: -0.00%, +0.03%
Latency: 242119940 -> 239665380 (-1.01%); split: -1.02%, +0.01%
InvThroughput: 131459884 -> 131182979 (-0.21%); split: -0.21%, +0.00%
VClause: 493311 -> 493215 (-0.02%); split: -0.05%, +0.03%
SClause: 758814 -> 758761 (-0.01%); split: -0.02%, +0.01%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33222>
 2025-01-28 17:00:45 +00:00
Georg Lehmann
ce897b336e aco/sched_ilp: rename priority to wait_cycles 
And invert.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33222>
 2025-01-28 17:00:44 +00:00
Georg Lehmann
819938d2fa aco/sched_ilp: new latency heuristic 
The main train of thought is that we should consider latency after
the write was scheduled. This means we rely a lot less on the input
order of instructions for good results.

Foz-DB GFX1150:
Totals from 75606 (95.25% of 79377) affected shaders:
Instrs: 43274326 -> 42129011 (-2.65%); split: -2.65%, +0.01%
CodeSize: 223049932 -> 218465796 (-2.06%); split: -2.06%, +0.00%
Latency: 297614199 -> 292317054 (-1.78%); split: -1.84%, +0.06%
InvThroughput: 57020160 -> 56336213 (-1.20%); split: -1.21%, +0.02%
VClause: 841775 -> 841861 (+0.01%); split: -0.06%, +0.07%
SClause: 1253516 -> 1253798 (+0.02%); split: -0.03%, +0.05%
VALU: 23893837 -> 23893828 (-0.00%); split: -0.00%, +0.00%

Foz-DB Navi31:
Totals from 75606 (95.25% of 79377) affected shaders:
Instrs: 42717592 -> 41531696 (-2.78%); split: -2.78%, +0.00%
CodeSize: 223582476 -> 218866196 (-2.11%); split: -2.11%, +0.00%
Latency: 297736383 -> 292450493 (-1.78%); split: -1.83%, +0.05%
InvThroughput: 47298730 -> 46934084 (-0.77%); split: -0.78%, +0.01%
VClause: 844982 -> 844892 (-0.01%); split: -0.07%, +0.06%
SClause: 1248433 -> 1248693 (+0.02%); split: -0.03%, +0.05%
VALU: 24819703 -> 24819704 (+0.00%); split: -0.00%, +0.00%

Foz-DB Navi21:
Totals from 76224 (96.03% of 79377) affected shaders:
Instrs: 46019515 -> 46015691 (-0.01%); split: -0.03%, +0.03%
CodeSize: 246992544 -> 246977404 (-0.01%); split: -0.03%, +0.02%
Latency: 324647457 -> 318661132 (-1.84%); split: -1.90%, +0.05%
InvThroughput: 74834800 -> 74269723 (-0.76%); split: -0.76%, +0.01%
VClause: 927601 -> 927579 (-0.00%); split: -0.04%, +0.04%
SClause: 1302666 -> 1303178 (+0.04%); split: -0.02%, +0.06%

Foz-DB Vega10:
Totals from 60142 (95.42% of 63026) affected shaders:
Instrs: 25117688 -> 25098175 (-0.08%); split: -0.10%, +0.02%
CodeSize: 129847464 -> 129769456 (-0.06%); split: -0.08%, +0.02%
Latency: 261606546 -> 262407481 (+0.31%); split: -0.12%, +0.43%
InvThroughput: 138422594 -> 138500401 (+0.06%); split: -0.03%, +0.09%
VClause: 555424 -> 555321 (-0.02%); split: -0.11%, +0.09%
SClause: 851219 -> 851620 (+0.05%); split: -0.03%, +0.08%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33222>
 2025-01-28 17:00:44 +00:00
Mike Blumenkrantz
4e1b18178b aco: exclude novalidateir from codegen flags 
this otherwise bricks caching

cc: mesa-stable

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33239>
 2025-01-27 19:15:47 +00:00
Georg Lehmann
df1de388a3 aco/sched_ilp: reorder VINTRP 
VINTRP(gfx6-gfx10.3) is mostly just VALU, but we treated it like memory
instructions as an afterthought. This had issues as VINTRP was never reordered
with itself, or other memory instructions. Reordering VINTRP in clauses
increases ILP. We don't really need collect_clause_dependencies for VINTRP
either, because they ususally have the same dependencies already. That means
we can still form VINTRP clauses by selecting preferably VINTRP after a
previous one.

Foz-DB Navi21:
Totals from 34184 (43.16% of 79206) affected shaders:
Instrs: 18811270 -> 18812046 (+0.00%); split: -0.01%, +0.02%
CodeSize: 103627276 -> 103630056 (+0.00%); split: -0.01%, +0.01%
Latency: 188379364 -> 187936731 (-0.23%); split: -0.27%, +0.03%
InvThroughput: 42600163 -> 42590608 (-0.02%); split: -0.03%, +0.00%
VClause: 378960 -> 378912 (-0.01%); split: -0.02%, +0.00%
SClause: 727560 -> 720573 (-0.96%); split: -1.08%, +0.12%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111>
 2025-01-27 11:59:45 +00:00
Georg Lehmann
45ccd6487f aco/sched_ilp: only remove WaW/WaR for inter clause dependencies 
Direct RaW shouldn't be removed, because the clause is split by a wait anyway.

Foz-DB Navi21:
Totals from 52 (0.07% of 79206) affected shaders:
Instrs: 1603523 -> 1603485 (-0.00%); split: -0.00%, +0.00%
CodeSize: 8223048 -> 8222788 (-0.00%)
Latency: 9741674 -> 9738884 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 2322621 -> 2322010 (-0.03%); split: -0.03%, +0.00%
SClause: 31325 -> 31320 (-0.02%); split: -0.02%, +0.01%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111>
 2025-01-27 11:59:45 +00:00
Georg Lehmann
4b332afb32 aco/sched_ilp: add dependencies of later clause instrs more aggressively 
Foz-DB GFX1150:
Totals from 22246 (28.03% of 79377) affected shaders:
Instrs: 22689053 -> 22684012 (-0.02%); split: -0.06%, +0.03%
CodeSize: 117622416 -> 117603292 (-0.02%); split: -0.04%, +0.03%
Latency: 182725630 -> 182702465 (-0.01%); split: -0.06%, +0.05%
InvThroughput: 37963256 -> 37956961 (-0.02%); split: -0.03%, +0.02%
VClause: 471019 -> 467248 (-0.80%)
SClause: 592620 -> 590034 (-0.44%); split: -0.44%, +0.01%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111>
 2025-01-27 11:59:45 +00:00
Georg Lehmann
ea514e9385 aco/sched_ilp: continue open clauses 
Foz-DB GFX1150:
Totals from 13789 (17.37% of 79395) affected shaders:
Instrs: 16567395 -> 16570832 (+0.02%); split: -0.03%, +0.05%
CodeSize: 85737492 -> 85751072 (+0.02%); split: -0.02%, +0.04%
Latency: 140988872 -> 140831767 (-0.11%); split: -0.12%, +0.01%
InvThroughput: 29639206 -> 29614890 (-0.08%); split: -0.09%, +0.00%
VClause: 347065 -> 343779 (-0.95%); split: -0.96%, +0.01%
SClause: 424881 -> 418657 (-1.46%); split: -1.48%, +0.02%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111>
 2025-01-27 11:59:45 +00:00
Georg Lehmann
997ea2e273 aco: update is_dual_issue_capable for gfx11.5+ 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33111>
 2025-01-27 11:59:45 +00:00
Rhys Perry
e18e293e6c aco: don't use divergence information for most ALU defs 
If one of the sources are divergent but all are SGPRs, then this can
cause issues (for example, imul with two SGPR sources and a VGPR definition).

No fossil-db changes (navi21)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12454
Backport-to: 24.3
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32997>
 2025-01-27 10:48:07 +00:00
Marek Olšák
f98613d47c aco: implement replacement of sample_mask_in with helper_invocation in PS prolog 
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33024>
 2025-01-25 12:20:26 -05:00
Marek Olšák
a842f198d7 aco: simplify how broadcast_last_cbuf is implemented in PS epilog 
So PS epilogs only need a single bool flag that determines whether all
enabled color buffers should be written.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33024>
 2025-01-25 12:20:26 -05:00
Marek Olšák
5c4f737b84 aco: implement replacing frag_coord with pixel_coord in PS prolog 
This adds an option to replace frag_coord.xy with pixel_coord when sample
shading is disabled, which is most of the time. This reduces the number of
input VGPRs.

It's already implement in ac_nir_lower_ps_early for monolithic shaders.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33024>
 2025-01-25 12:20:26 -05:00
Marek Olšák
d7d4d56f5b ac,aco,radeonsi: replace SampleMaskIn with 1 << SampleID if full sample shading 
Since the sample mask is always 1 << sample_id with full sample shading,
just use that instead of loading sample_mask_in. Set it to 0 if it's
a helper invocation. This removes the sample mask input VGPR.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33024>
 2025-01-25 12:20:25 -05:00
Rhys Perry
0eb5f66660 nir/validate: validate ssa dominance by default 
This no longer modifies dominance metadata, so enable it by default.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32005>
 2025-01-23 23:35:44 +00:00
Daniel Schürmann
08560b8ff8 aco/lower_branches: stitch linear blocks if there is exactly one successor with one predecessor 
Totals from 12906 (16.26% of 79395) affected shaders: (Navi31)

Instrs: 22051521 -> 22049488 (-0.01%); split: -0.01%, +0.00%
CodeSize: 116591240 -> 116583920 (-0.01%)
Latency: 196625178 -> 196538410 (-0.04%); split: -0.04%, +0.00%
InvThroughput: 33943045 -> 33930615 (-0.04%); split: -0.04%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32477>
 2025-01-23 00:11:06 +00:00
Daniel Schürmann
c90ae5f773 aco: delete aco_jump_threading.cpp 
This is now handled by lower_branches().

Totals from 47236 (59.49% of 79395) affected shaders: (Navi31)
Instrs: 29490400 -> 29490507 (+0.00%)
CodeSize: 152316812 -> 152317248 (+0.00%); split: -0.00%, +0.00%
Latency: 229665459 -> 229665106 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 36870605 -> 36870504 (-0.00%); split: -0.00%, +0.00%
Copies: 1966751 -> 2233467 (+13.56%)
SALU: 3122941 -> 3123048 (+0.00%)

Note, that only about 20 shaders are actually affected.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32477>
 2025-01-23 00:11:06 +00:00
Daniel Schürmann
c677809f25 aco/lower_branches: allow for non-fallthrough loop exits in try_merge_break_with_continue() 
Totals from 211 (0.27% of 79395) affected shaders: (Navi31)

Instrs: 276961 -> 276545 (-0.15%)
CodeSize: 1404356 -> 1402248 (-0.15%)
Latency: 1344722 -> 1344887 (+0.01%); split: -0.00%, +0.01%
InvThroughput: 165624 -> 165622 (-0.00%); split: -0.00%, +0.00%
Branches: 6149 -> 5987 (-2.63%)
SALU: 25722 -> 25468 (-0.99%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32477>
 2025-01-23 00:11:06 +00:00
Daniel Schürmann
12656ea5f5 aco: move try_merge_break_with_continue() to lower_branches() 
Totals from 3 (0.00% of 79395) affected shaders: (Navi31)

Instrs: 12888 -> 12882 (-0.05%)
Latency: 83253 -> 83246 (-0.01%)
InvThroughput: 9251 -> 9249 (-0.02%)
Branches: 483 -> 480 (-0.62%)
SALU: 1329 -> 1326 (-0.23%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32477>
 2025-01-23 00:11:06 +00:00
Daniel Schürmann
13ad3db43f aco/lower_branches: implement try_remove_simple_block() in lower_branches() 
This is mostly the same as in jump_threading, but can handle
multiple predecessors.

Totals from 3523 (4.44% of 79395) affected shaders: (Navi31)

Instrs: 10244892 -> 10244753 (-0.00%); split: -0.00%, +0.00%
CodeSize: 54171500 -> 54168540 (-0.01%); split: -0.01%, +0.00%
Latency: 75070425 -> 75059570 (-0.01%); split: -0.02%, +0.00%
InvThroughput: 11606911 -> 11605767 (-0.01%); split: -0.01%, +0.00%
Branches: 331778 -> 331675 (-0.03%); split: -0.05%, +0.02%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32477>
 2025-01-23 00:11:06 +00:00
Daniel Schürmann
2b5a893e29 aco/lower_branches: do eliminate_useless_exec_writes_in_block() during branch lowering. 
Totals from 728 (0.92% of 79395) affected shaders: (Navi31)

Instrs: 452926 -> 452161 (-0.17%)
CodeSize: 2255536 -> 2252504 (-0.13%)
Latency: 1683404 -> 1683470 (+0.00%); split: -0.01%, +0.01%
InvThroughput: 210887 -> 210888 (+0.00%); split: -0.00%, +0.00%
SALU: 77865 -> 77106 (-0.97%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32477>
 2025-01-23 00:11:06 +00:00
Daniel Schürmann
eecdb45d61 aco: consider s_cbranch_exec* instructions in needs_exec_mask() 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32477>
 2025-01-23 00:11:06 +00:00
Daniel Schürmann
de1e38e214 aco/assembler: Find loop exits using the successor's loop nest depth 
Previously, we just used the next block after a loop that
has a back-edge. This assumes that loop-exit blocks can
only be removed when falling through to the next block,
when in fact it can also be a jump to somewhere else,
in future even to some block before the actual loop.

12 (0.02% of 79395) affected shaders.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32477>
 2025-01-23 00:11:06 +00:00
Daniel Schürmann
29c63de062 aco/jump_threading: don't remove loop preheaders 
They might be needed as convergence point in order to
insert code (e.g. for loop alignment, wait states, etc.).

Totals from 1 (0.00% of 79395) affected shaders:

CodeSize: 12672 -> 12716 (+0.35%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32477>
 2025-01-23 00:11:06 +00:00
Georg Lehmann
71cb394b02 aco: implement some more std::vector functions for small_vec 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33043>
 2025-01-17 09:25:48 +00:00
Georg Lehmann
31de188bc2 aco: support less trivial component types in small_vec 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33043>
 2025-01-17 09:25:48 +00:00
Georg Lehmann
15cba08db0 aco: guard small_vector move/copy operator against self assignment 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33043>
 2025-01-17 09:25:48 +00:00
Marek Olšák
d160252270 ac: use Z_EXPORT_FORMAT=32_AR for Z + Alpha mrtz exports 
This should be faster than 32_ABGR.

Also, stencil exports are changed from UINT16_ABGR to 32_GR,
which should have no effect on performance.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33046>
 2025-01-16 02:58:03 +00:00
Timur Kristóf
50035f0316 ac/nir: Move all ac_nir_* files to a new folder. 
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966>
 2025-01-14 13:46:30 +01:00
Timur Kristóf
305fdfddb5 ac/nir: Move ac_set_nir_options to ac_nir.c 
And rename it to ac_nir_set_options to match other functions.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966>
 2025-01-14 13:45:34 +01:00
Samuel Pitoiset
10e424f586 aco: always use ds_bpermute for shuffle/rotate on GFX12 
ds_bpermute supports both 32 and 64 lanes now.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32974>
 2025-01-13 08:33:38 +00:00
Rhys Perry
2b10930b48 aco: use VOP3 v_mov_b16 if necessary 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Backport-to: 24.3
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32944>
 2025-01-10 15:05:00 +00:00
Rhys Perry
46787fc2d0 aco/util: fix bit_reference::operator&= 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Backport-to: 24.3
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32944>
 2025-01-10 15:05:00 +00:00
Rhys Perry
8ac4744706 aco/tests: fix skip_lines=True with remaining characters in matches 
If the remaining character check fails, we should try a later line if
skip_lines=True. So the check has to be done earlier.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32902>
 2025-01-08 15:28:37 +00:00
Friedrich Vock
71392fff25 aco: Fix dead instruction/index handling for try_insert_saveexec_out_of_loop 
The loop checking if exec is overwritten didn't check for NULL
instructions, and didn't fix up reg write indices after inserting
instructions.

Fixes: fcd94a8c ("aco: move try_optimize_branching_sequence() to postRA optimizations")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32746>
 2025-01-08 10:48:01 +00:00