Commit graph

421 commits

Author SHA1 Message Date
Rhys Perry
5ebefceb42 aco/ra: move split_blocking_vectors higher
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39390>
2026-01-20 13:33:16 +00:00
Rhys Perry
24fe4a5b58 aco/ra: copy precolor affinities to p_create_vector/p_split_vector
fossil-db (navi31):
Totals from 7 (0.01% of 84369) affected shaders:
Instrs: 2742 -> 2704 (-1.39%); split: -1.82%, +0.44%
CodeSize: 15300 -> 15052 (-1.62%); split: -1.93%, +0.31%
VGPRs: 516 -> 504 (-2.33%)
Latency: 12478 -> 12504 (+0.21%); split: -0.24%, +0.45%
InvThroughput: 2350 -> 2300 (-2.13%)
Copies: 350 -> 272 (-22.29%)
VALU: 1626 -> 1592 (-2.09%)
VOPD: 280 -> 236 (-15.71%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39387>
2026-01-20 10:53:18 +00:00
Rhys Perry
da1dca28a7 aco/ra: consider already-used preserved registers to be free
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679>
2026-01-19 09:17:53 +00:00
Rhys Perry
103160ff2f aco/ra: prefer clobbered registers in get_reg_specified()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679>
2026-01-19 09:17:53 +00:00
Rhys Perry
2b20d568e0 aco/ra: prefer clobbered registers in callees
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679>
2026-01-19 09:17:53 +00:00
Rhys Perry
21b1118a08 aco/ra: refactor get_reg_impl and get_regs_for_copies using tuples
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679>
2026-01-19 09:17:52 +00:00
Rhys Perry
61919b93fa aco/ra: always abort loop in get_regs_for_copies() if candidate is worse
fossil-db (navi31):
Totals from 2 (0.00% of 79825) affected shaders:
Instrs: 58162 -> 58178 (+0.03%)
CodeSize: 299132 -> 299212 (+0.03%)
Latency: 72864 -> 72861 (-0.00%)
VClause: 977 -> 984 (+0.72%)
Copies: 7433 -> 7428 (-0.07%)
Branches: 1602 -> 1603 (+0.06%)
VALU: 37100 -> 37095 (-0.01%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679>
2026-01-19 09:17:51 +00:00
Rhys Perry
137f71895d aco/ra: always prefer earlier regs in get_reg_impl() if costs are the same
Before, this would have updated best_win if it encounters an assignment
with the same num_moves and num_vars but is aligned, even if the previous
best assignment was also aligned.

fossil-db (navi31):
Totals from 7693 (9.64% of 79825) affected shaders:
Instrs: 9767559 -> 9766370 (-0.01%); split: -0.11%, +0.09%
CodeSize: 51008244 -> 51002396 (-0.01%); split: -0.09%, +0.08%
VGPRs: 497224 -> 497212 (-0.00%); split: -0.00%, +0.00%
Latency: 80435654 -> 80373798 (-0.08%); split: -0.20%, +0.12%
InvThroughput: 12426170 -> 12414490 (-0.09%); split: -0.15%, +0.06%
VClause: 217725 -> 217438 (-0.13%); split: -0.59%, +0.46%
SClause: 230555 -> 230551 (-0.00%); split: -0.00%, +0.00%
Copies: 758396 -> 757988 (-0.05%); split: -0.81%, +0.75%
Branches: 200669 -> 200662 (-0.00%); split: -0.01%, +0.01%
VALU: 6040415 -> 6039979 (-0.01%); split: -0.10%, +0.09%
SALU: 960327 -> 960671 (+0.04%); split: -0.03%, +0.07%
VOPD: 426 -> 458 (+7.51%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679>
2026-01-19 09:17:50 +00:00
Rhys Perry
a932bc40d3 aco/ra: don't use update_vgpr_sgpr_demand in increase_register_file
In a future commit, update_vgpr_sgpr_demand() should only be given the
register demand calculated from the number of live temporaries, not the
maximum register used.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38679>
2026-01-19 09:17:49 +00:00
Rhys Perry
aa09335485 aco/ra: emit p_split_vector after p_parallelcopy
This fixes situations where the variable being split is a parallelcopy
definition.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811>
2026-01-16 10:05:43 +00:00
Rhys Perry
7caa0ab897 aco/ra: fix split_blocking_vectors with some subdword vectors
For example, v6b at byte=2 should be split into v2b,v1 instead of v1,v2b.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811>
2026-01-16 10:05:43 +00:00
Rhys Perry
667771950d aco/ra: update register file when recreating blocking vectors
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811>
2026-01-16 10:05:43 +00:00
Rhys Perry
5cf991ecfb aco/ra: use original name for blocking vectors rename
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811>
2026-01-16 10:05:43 +00:00
Rhys Perry
829e3a6a7d aco/ra: fix operands when recreating blocking vectors
It is likely that these have been moved.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811>
2026-01-16 10:05:43 +00:00
Rhys Perry
6e6e40ab5c aco/ra: discard tmp_file after get_regs_for_copies fails
get_regs_for_copies() might change tmp_file, so we need to create a new
one.

Recursively handle_call() with an modified register_file instead of
repeating get_regs_for_copies().

This also updates register_file for the caller.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811>
2026-01-16 10:05:43 +00:00
Rhys Perry
159154a741 aco/ra: remove dead code in split_blocking_vectors
collect_vars_from_bitset() from the caller should have already cleared
these.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811>
2026-01-16 10:05:43 +00:00
Rhys Perry
bf30a57440 aco/ra: omit renaming when necessary when moving copy definitions
This should resolve the FIXME.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811>
2026-01-16 10:05:41 +00:00
Rhys Perry
8dcfce3587 aco/ra: refactor update_renames slightly
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38811>
2026-01-16 10:05:39 +00:00
Natalie Vock
8525993d02 aco/ra: Move register preservation logic in last block to p_return
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29580>
2026-01-14 14:19:05 +00:00
Rhys Perry
21414e0898 aco/ra: add first loop header phi operand to temp_to_phi_resources
If the first operand is a CSSA copy, we might want to add this to
temp_to_phi_resources, so that we later mark it as the last-seen phi
operand.

fossil-db (navi31):
Totals from 284 (0.36% of 79825) affected shaders:
Instrs: 4160233 -> 4157517 (-0.07%); split: -0.09%, +0.03%
CodeSize: 21546420 -> 21532884 (-0.06%); split: -0.09%, +0.02%
VGPRs: 31404 -> 31416 (+0.04%)
Latency: 40266308 -> 40253731 (-0.03%); split: -0.06%, +0.02%
InvThroughput: 8140751 -> 8139724 (-0.01%); split: -0.05%, +0.04%
VClause: 99849 -> 99835 (-0.01%); split: -0.02%, +0.01%
Copies: 344512 -> 341732 (-0.81%); split: -1.08%, +0.28%
Branches: 113620 -> 113629 (+0.01%); split: -0.02%, +0.03%
VALU: 2502619 -> 2499836 (-0.11%); split: -0.15%, +0.04%
SALU: 499245 -> 499341 (+0.02%); split: -0.02%, +0.04%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Emre Cecanpunar <emreleno@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38695>
2025-12-11 16:51:21 +00:00
Rhys Perry
43b3901362 aco/ra: copy vector_info to affinities
This eliminates some copies in BVH traversal loops.

fossil-db (navi31):
Totals from 200 (0.25% of 79825) affected shaders:
Instrs: 734931 -> 732521 (-0.33%); split: -0.34%, +0.01%
CodeSize: 3801080 -> 3791692 (-0.25%); split: -0.26%, +0.01%
VGPRs: 13704 -> 13728 (+0.18%); split: -0.44%, +0.61%
Latency: 6094605 -> 6082060 (-0.21%); split: -0.24%, +0.03%
InvThroughput: 1081982 -> 1080121 (-0.17%); split: -0.19%, +0.02%
VClause: 18835 -> 18837 (+0.01%); split: -0.01%, +0.02%
Copies: 64602 -> 62239 (-3.66%); split: -3.75%, +0.09%
Branches: 20111 -> 20112 (+0.00%); split: -0.01%, +0.02%
VALU: 438618 -> 436257 (-0.54%); split: -0.55%, +0.01%
SALU: 85092 -> 85085 (-0.01%); split: -0.01%, +0.00%
VOPD: 76 -> 74 (-2.63%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Emre Cecanpunar <emreleno@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38695>
2025-12-11 16:51:21 +00:00
Natalie Vock
6b2e766617 aco/ra: Handle linear VGPRs allocated by p_startpgm
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38281>
2025-12-08 19:12:54 +00:00
Natalie Vock
e3404af0ed aco/ra: Handle call ABI constraints
Make sure no caller temporaries stay in call-clobbered registers during
execution of a function call.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37381>
2025-11-06 12:09:43 +00:00
Natalie Vock
86a8b1753a aco/ra: Handle callee ABI preserved register constraints
Block preserved register ranges while they can't yet be overwritten,
i.e. before p_spill_preserved and after the last p_reload_preserved.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37381>
2025-11-06 12:09:42 +00:00
Natalie Vock
75b89f233f aco/ra: Don't clear fixed operand sources if they were blocked
That would undo the blocking and allow for conflicting assignments.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37381>
2025-11-06 12:09:42 +00:00
Natalie Vock
d4013e7432 aco/ra: Skip blocked regs in get_reg_impl
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37381>
2025-11-06 12:09:41 +00:00
Natalie Vock
7ed1aea9c5 aco/ra: Also consider blocked registers as not containing temps
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37381>
2025-11-06 12:09:41 +00:00
Natalie Vock
642b1f0dce aco/ra: Add utility to clear PhysRegInterval
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37381>
2025-11-06 12:09:40 +00:00
Daniel Schürmann
441d5aab08 aco/ra: coalesce vector affinities with tied definitions
Totals from 19310 (24.19% of 79839) affected shaders: (Navi48)

MaxWaves: 564238 -> 564542 (+0.05%); split: +0.06%, -0.01%
Instrs: 10856428 -> 10803360 (-0.49%); split: -0.53%, +0.04%
CodeSize: 56405088 -> 56189384 (-0.38%); split: -0.41%, +0.02%
VGPRs: 986120 -> 985952 (-0.02%); split: -0.50%, +0.48%
Latency: 53956142 -> 53940850 (-0.03%); split: -0.11%, +0.09%
InvThroughput: 8769260 -> 8735595 (-0.38%); split: -0.49%, +0.11%
VClause: 237471 -> 237452 (-0.01%); split: -0.05%, +0.04%
SClause: 225385 -> 225389 (+0.00%)
Copies: 799792 -> 744150 (-6.96%); split: -7.25%, +0.30%
Branches: 208574 -> 208572 (-0.00%); split: -0.00%, +0.00%
VALU: 6116920 -> 6061448 (-0.91%); split: -0.95%, +0.04%
SALU: 1442068 -> 1441990 (-0.01%); split: -0.01%, +0.00%
VOPD: 1914 -> 1744 (-8.88%); split: +0.10%, -8.99%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36851>
2025-09-02 10:24:27 +00:00
Daniel Schürmann
2f303636f3 aco/ra: consider precolor affinities in get_reg_vector()
No fossil changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36851>
2025-09-02 10:24:27 +00:00
Daniel Schürmann
6dbf8f7b90 aco/ra: don't set precolor affinities for already assigned temporaries
Also don't overwrite existing precolor affinities.

Totals from 248 (0.31% of 79839) affected shaders: (Navi48)

Instrs: 154427 -> 154401 (-0.02%); split: -0.12%, +0.10%
CodeSize: 812880 -> 812568 (-0.04%); split: -0.12%, +0.08%
VGPRs: 12432 -> 12408 (-0.19%)
Latency: 851623 -> 851801 (+0.02%); split: -0.03%, +0.05%
InvThroughput: 156569 -> 156581 (+0.01%); split: -0.04%, +0.05%
VClause: 2672 -> 2681 (+0.34%); split: -0.34%, +0.67%
Copies: 12645 -> 12660 (+0.12%); split: -0.53%, +0.65%
VALU: 82894 -> 82909 (+0.02%); split: -0.08%, +0.10%
SALU: 25406 -> 25424 (+0.07%); split: -0.07%, +0.14%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36851>
2025-09-02 10:24:26 +00:00
Daniel Schürmann
eb557fd090 aco/ra: add vector_info::index to indicate the Operand's index into the vector
This simplifies the code and will allow for a mismatch between the index and
the Operand's temporary.

Totals from 28 (0.04% of 79839) affected shaders: (Navi48)

Instrs: 18453 -> 18440 (-0.07%); split: -0.08%, +0.01%
CodeSize: 98588 -> 98532 (-0.06%); split: -0.06%, +0.00%
Copies: 1347 -> 1333 (-1.04%); split: -1.11%, +0.07%
VALU: 10431 -> 10417 (-0.13%); split: -0.14%, +0.01%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36851>
2025-09-02 10:24:26 +00:00
Daniel Schürmann
219c53e6fc aco/ra: don't clear lateKill operands in get_reg_create_vector()
Fixes: 08f088479a ('aco/ra: set late-kill for operands of temporary p_create_vector')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36871>
2025-08-22 15:25:04 +00:00
Georg Lehmann
0e66f2b2cc aco: use new disable_wqm for mimg
Foz-DB GFX1201:
Totals from 88 (0.11% of 80251) affected shaders:
Instrs: 81954 -> 82218 (+0.32%); split: -0.02%, +0.34%
CodeSize: 451824 -> 452880 (+0.23%); split: -0.02%, +0.25%
Latency: 308818 -> 308746 (-0.02%); split: -0.05%, +0.02%
VClause: 1324 -> 1318 (-0.45%)
Copies: 2795 -> 2784 (-0.39%)
PreSGPRs: 4029 -> 4035 (+0.15%)
SALU: 6563 -> 6809 (+3.75%); split: -0.15%, +3.90%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35970>
2025-08-15 07:03:46 +00:00
Rhys Perry
08f088479a aco/ra: set late-kill for operands of temporary p_create_vector
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13543
Fixes: c279dd6e61 ("aco: Support vector-aligned ops fixed to defs")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36469>
2025-08-06 09:44:01 +00:00
Daniel Schürmann
4ca3cc5a1a aco/ra: propagate precolor affinities through parallelcopies and tied definitions
Totals from 214 (0.27% of 79839) affected shaders: (Navi48)

Instrs: 65339 -> 65311 (-0.04%); split: -0.05%, +0.00%
CodeSize: 352616 -> 350952 (-0.47%); split: -0.55%, +0.07%
VGPRs: 9984 -> 9960 (-0.24%)
Latency: 207556 -> 207508 (-0.02%); split: -0.03%, +0.01%
InvThroughput: 40422 -> 40397 (-0.06%)
Copies: 3180 -> 3155 (-0.79%)
VALU: 38347 -> 38322 (-0.07%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36345>
2025-08-01 17:15:54 +00:00
Daniel Schürmann
a667d9a68d aco/ra: propagate precolor affinities through phis
Totals from 917 (1.15% of 79839) affected shaders: (Navi48)

Instrs: 3217861 -> 3216947 (-0.03%); split: -0.04%, +0.01%
CodeSize: 17427204 -> 17432264 (+0.03%); split: -0.06%, +0.09%
VGPRs: 65328 -> 65316 (-0.02%)
Latency: 35336268 -> 35335528 (-0.00%); split: -0.01%, +0.01%
InvThroughput: 7305032 -> 7302187 (-0.04%); split: -0.04%, +0.00%
SClause: 120537 -> 120553 (+0.01%); split: -0.01%, +0.02%
Copies: 307257 -> 306852 (-0.13%); split: -0.21%, +0.08%
Branches: 115744 -> 115743 (-0.00%)
VALU: 1572522 -> 1572183 (-0.02%); split: -0.02%, +0.00%
SALU: 574229 -> 574155 (-0.01%); split: -0.05%, +0.04%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36345>
2025-08-01 17:15:54 +00:00
Daniel Schürmann
2ddd8ef0a3 aco/ra: don't optimize encodings on precolor affinity mismatch
Totals from 238 (0.30% of 79839) affected shaders: (Navi48)
Instrs: 137836 -> 137176 (-0.48%); split: -0.50%, +0.02%
CodeSize: 728616 -> 728668 (+0.01%); split: -0.06%, +0.07%
Latency: 1503248 -> 1500202 (-0.20%); split: -0.56%, +0.36%
InvThroughput: 297725 -> 296715 (-0.34%); split: -0.70%, +0.36%
Copies: 9390 -> 8825 (-6.02%); split: -6.33%, +0.31%
VALU: 89861 -> 89296 (-0.63%); split: -0.66%, +0.03%
SALU: 13166 -> 13167 (+0.01%); split: -0.05%, +0.06%

Suggested-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36345>
2025-08-01 17:15:54 +00:00
Daniel Schürmann
93606a19c6 aco/ra: collect register affinities for all precolored operands.
Totals from 1280 (1.60% of 79839) affected shaders: (Navi48)

Instrs: 817363 -> 812639 (-0.58%); split: -0.58%, +0.00%
CodeSize: 4262644 -> 4243540 (-0.45%); split: -0.45%, +0.00%
VGPRs: 61692 -> 61668 (-0.04%)
Latency: 4354318 -> 4347818 (-0.15%); split: -0.15%, +0.00%
InvThroughput: 711914 -> 707698 (-0.59%); split: -0.59%, +0.00%
VClause: 14685 -> 14677 (-0.05%); split: -0.09%, +0.03%
SClause: 25623 -> 25621 (-0.01%)
Copies: 50663 -> 46242 (-8.73%); split: -8.73%, +0.00%
VALU: 427744 -> 423323 (-1.03%); split: -1.03%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36345>
2025-08-01 17:15:54 +00:00
Daniel Schürmann
e32eec52f0 aco/ra: generalize register affinities
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36345>
2025-08-01 17:15:54 +00:00
Antonio Ospite
ddf2aa3a4d build: avoid redefining unreachable() which is standard in C23
In the C23 standard unreachable() is now a predefined function-like
macro in <stddef.h>

See https://android.googlesource.com/platform/bionic/+/HEAD/docs/c23.md#is-now-a-predefined-function_like-macro-in

And this causes build errors when building for C23:

-----------------------------------------------------------------------
In file included from ../src/util/log.h:30,
                 from ../src/util/log.c:30:
../src/util/macros.h:123:9: warning: "unreachable" redefined
  123 | #define unreachable(str)    \
      |         ^~~~~~~~~~~
In file included from ../src/util/macros.h:31:
/usr/lib/gcc/x86_64-linux-gnu/14/include/stddef.h:456:9: note: this is the location of the previous definition
  456 | #define unreachable() (__builtin_unreachable ())
      |         ^~~~~~~~~~~
-----------------------------------------------------------------------

So don't redefine it with the same name, but use the name UNREACHABLE()
to also signify it's a macro.

Using a different name also makes sense because the behavior of the
macro was extending the one of __builtin_unreachable() anyway, and it
also had a different signature, accepting one argument, compared to the
standard unreachable() with no arguments.

This change improves the chances of building mesa with the C23 standard,
which for instance is the default in recent AOSP versions.

All the instances of the macro, including the definition, were updated
with the following command line:

  git grep -l '[^_]unreachable(' -- "src/**" | sort | uniq | \
  while read file; \
  do \
    sed -e 's/\([^_]\)unreachable(/\1UNREACHABLE(/g' -i "$file"; \
  done && \
  sed -e 's/#undef unreachable/#undef UNREACHABLE/g' -i src/intel/isl/isl_aux_info.c

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36437>
2025-07-31 17:49:42 +00:00
Georg Lehmann
a6a6c2f691 aco/ra: convert bitwise instruction to gfx11+ 16bit on demand
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
The 32bit versions are smaller, allow more optimizations and VOPD,
so only use the 16bit opcodes if nessecary.

Foz-DB Navi31:
Totals from 84 (0.10% of 80237) affected shaders:
Instrs: 176673 -> 176347 (-0.18%); split: -0.20%, +0.01%
CodeSize: 970148 -> 969716 (-0.04%); split: -0.08%, +0.03%
VGPRs: 5876 -> 5864 (-0.20%)
Latency: 2805974 -> 2805674 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 769007 -> 768738 (-0.03%); split: -0.04%, +0.01%
VClause: 2593 -> 2597 (+0.15%)
Copies: 23749 -> 23487 (-1.10%); split: -1.11%, +0.00%
VALU: 107124 -> 106862 (-0.24%); split: -0.25%, +0.00%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35919>
2025-07-31 12:07:07 +00:00
Georg Lehmann
b12db991eb aco/gfx10: optimize subgroupRotate(x, 32) and subgroupShuffleXor(x, 32)
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
We don't have v_permlane64_b32 yet, but we can still optimize it using
shared vgprs. Using the DPP16 row mask, we can even avoid writing exec.

With v0 input/output and v24/v25 as shared vgprs, this results in:
v_mov_b32_dpp v24, v0 quad_perm:[0,1,2,3] row_mask:0x3 bank_mask:0xf
v_mov_b32_dpp v25, v0 quad_perm:[0,1,2,3] row_mask:0xc bank_mask:0xf
v_mov_b32_dpp v0, v24 quad_perm:[0,1,2,3] row_mask:0xc bank_mask:0xf
v_mov_b32_dpp v0, v25 quad_perm:[0,1,2,3] row_mask:0x3 bank_mask:0xf

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36390>
2025-07-29 06:33:20 +00:00
Natalie Vock
b2a95d2133 aco/ra: Add affinities for DS vector-aligned operands
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35269>
2025-07-15 21:34:40 +00:00
Natalie Vock
c279dd6e61 aco: Support vector-aligned ops fixed to defs
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35269>
2025-07-15 21:34:38 +00:00
Daniel Schürmann
47ef60cbf1 aco/ra: always use bytes for register stride requirements
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
instead of a mixture between dwords and bytes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36053>
2025-07-14 08:45:29 +00:00
Daniel Schürmann
3f35b1329e aco: allow subdword vector-definitions on some VALU instructions
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35784>
2025-07-09 14:10:36 +00:00
Rhys Perry
dce1d4ad4c aco/ra: fix repeated compact_linear_vgprs() in get_reg()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: b7738de4f9 ("aco/ra: rework linear VGPR allocation")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13431
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35838>
2025-07-02 09:26:04 +00:00
Daniel Schürmann
7620957193 aco/ra: always set fill_operands=true when handling operands
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
This makes the behavior consistent and less prone to error.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35735>
2025-06-26 10:05:07 +00:00
Daniel Schürmann
ee8424d839 aco/ra: always fill moved operands when handling vector-operands
update_renames() assumes that killed operands are already removed from
the register file, except for precolored and copy-kill operands.
When dealing with vector-operands, however, unrelated operands might
also be moved, in order to make space.

Fixes: fb689f133e ('aco/ra: handle register assignment of vector-aligned operands')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35735>
2025-06-26 10:05:07 +00:00