fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2026-01-28 18:20:23 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 5
216488 commits 66 branches 919 tags 949 MiB
Commit graph

150 commits

Author SHA1 Message Date
Georg Lehmann
6ba462bf26 aco/disable_wqm: optimize local mask creation 
Foz-DB Navi48:
Totals from 7861 (9.79% of 80287) affected shaders:
Instrs: 13276809 -> 13183483 (-0.70%)
CodeSize: 71221260 -> 70852500 (-0.52%); split: -0.52%, +0.00%
Latency: 124001421 -> 123976480 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 17820119 -> 17817551 (-0.01%); split: -0.01%, +0.00%
SALU: 1736356 -> 1666673 (-4.01%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35970>
 2025-08-15 07:03:47 +00:00
Georg Lehmann
de4b345949 aco/insert_exec: remove per instruction wqm/exact exec handling 
No Foz-DB changes on GFX1201.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35970>
 2025-08-15 07:03:46 +00:00
Georg Lehmann
11cee3d634 aco: use new disable_wqm for p_dual_src_export_gfx11 
No Foz-DB changes on GFX1201.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35970>
 2025-08-15 07:03:46 +00:00
Georg Lehmann
8e53ba9a0a aco: use new disable_wqm for exp 
No Foz-DB changes on GFX1201.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35970>
 2025-08-15 07:03:46 +00:00
Georg Lehmann
0e66f2b2cc aco: use new disable_wqm for mimg 
Foz-DB GFX1201:
Totals from 88 (0.11% of 80251) affected shaders:
Instrs: 81954 -> 82218 (+0.32%); split: -0.02%, +0.34%
CodeSize: 451824 -> 452880 (+0.23%); split: -0.02%, +0.25%
Latency: 308818 -> 308746 (-0.02%); split: -0.05%, +0.02%
VClause: 1324 -> 1318 (-0.45%)
Copies: 2795 -> 2784 (-0.39%)
PreSGPRs: 4029 -> 4035 (+0.15%)
SALU: 6563 -> 6809 (+3.75%); split: -0.15%, +3.90%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35970>
 2025-08-15 07:03:46 +00:00
Georg Lehmann
922f559c3c aco: use new disable_wqm for flatlike 
No Foz-DB changes on GFX1201.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35970>
 2025-08-15 07:03:46 +00:00
Georg Lehmann
a4c537c5b3 aco: use new disable_wqm for mubuf/mtbuf 
Foz-DB GFX1201:
Totals from 66 (0.08% of 80251) affected shaders:
Instrs: 45373 -> 45663 (+0.64%); split: -0.01%, +0.65%
CodeSize: 251708 -> 252900 (+0.47%); split: -0.00%, +0.48%
Latency: 278977 -> 278652 (-0.12%); split: -0.14%, +0.02%
InvThroughput: 38259 -> 38245 (-0.04%); split: -0.05%, +0.02%
VClause: 982 -> 962 (-2.04%)
Copies: 2882 -> 2808 (-2.57%)
PreSGPRs: 2564 -> 2599 (+1.37%)
SALU: 4748 -> 5010 (+5.52%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35970>
 2025-08-15 07:03:46 +00:00
Georg Lehmann
63af48ae2e aco/insert_exec: new way to handle instructions that need wqm disabled 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35970>
 2025-08-15 07:03:46 +00:00
Georg Lehmann
ca25553b92 aco: add a post-RA pass to disable wqm 
By disabling WQM post-RA, we don't have RA/spilling mov issues with image_sample
operands that need to be computed in WQM.

We also don't restrict scheduling by inserting exec writes.
The only downside is more scalar ALU usage, but the SALU is almost always underutilized.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35970>
 2025-08-15 07:03:46 +00:00
Georg Lehmann
34b154866f aco/insert_exec: remove p_jump_to_epilog from needs exact 
p_end_wqm will always be emitted before it by isel.

No Foz-DB changes on GFX1201.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35970>
 2025-08-15 07:03:46 +00:00
Antonio Ospite
ddf2aa3a4d build: avoid redefining unreachable() which is standard in C23 
In the C23 standard unreachable() is now a predefined function-like
macro in <stddef.h>

See https://android.googlesource.com/platform/bionic/+/HEAD/docs/c23.md#is-now-a-predefined-function_like-macro-in

And this causes build errors when building for C23:

-----------------------------------------------------------------------
In file included from ../src/util/log.h:30,
                 from ../src/util/log.c:30:
../src/util/macros.h:123:9: warning: "unreachable" redefined
  123 | #define unreachable(str)    \
      |         ^~~~~~~~~~~
In file included from ../src/util/macros.h:31:
/usr/lib/gcc/x86_64-linux-gnu/14/include/stddef.h:456:9: note: this is the location of the previous definition
  456 | #define unreachable() (__builtin_unreachable ())
      |         ^~~~~~~~~~~
-----------------------------------------------------------------------

So don't redefine it with the same name, but use the name UNREACHABLE()
to also signify it's a macro.

Using a different name also makes sense because the behavior of the
macro was extending the one of __builtin_unreachable() anyway, and it
also had a different signature, accepting one argument, compared to the
standard unreachable() with no arguments.

This change improves the chances of building mesa with the C23 standard,
which for instance is the default in recent AOSP versions.

All the instances of the macro, including the definition, were updated
with the following command line:

  git grep -l '[^_]unreachable(' -- "src/**" | sort | uniq | \
  while read file; \
  do \
    sed -e 's/\([^_]\)unreachable(/\1UNREACHABLE(/g' -i "$file"; \
  done && \
  sed -e 's/#undef unreachable/#undef UNREACHABLE/g' -i src/intel/isl/isl_aux_info.c

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36437>
 2025-07-31 17:49:42 +00:00
Daniel Schürmann
83fcd0e908 aco/insert_exec_mask: don't create unnecessary loop-header and -exit phis after demote
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
Details
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Details 
No fossil changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33479>
 2025-05-09 17:20:29 +00:00
Daniel Schürmann
2b0536e921 aco: remove block_kind_continue_or_break workaround and tests 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33479>
 2025-05-09 17:20:29 +00:00
Georg Lehmann
65411350ac aco/insert_exec: disable empty quads when leaving divergent control, even if not top level 
We don't restore exec after uniform top level branches, so nothing disabled
empty quads after a demote in divergent control flow in a uniform branch.

Foz-DB Navi31:
Totals from 17 (0.02% of 79789) affected shaders:
Instrs: 34573 -> 34572 (-0.00%)
CodeSize: 186876 -> 186872 (-0.00%)
Latency: 324145 -> 324141 (-0.00%)
Copies: 1467 -> 1458 (-0.61%)
PreSGPRs: 802 -> 800 (-0.25%)
SALU: 2531 -> 2530 (-0.04%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34670>
 2025-04-24 15:43:09 +00:00
Georg Lehmann
dd3e1190a2 aco/insert_exec: reset temporary when recreating wqm mask from exact mask 
The old, now incorrect temporary was still used for invert blocks and loop masks.

Foz-DB Navi31:
Totals from 379 (0.48% of 79789) affected shaders:
Instrs: 399471 -> 399897 (+0.11%); split: -0.00%, +0.11%
CodeSize: 2197292 -> 2198908 (+0.07%); split: -0.00%, +0.08%
Latency: 2500636 -> 2500895 (+0.01%); split: -0.00%, +0.01%
SClause: 7912 -> 7918 (+0.08%); split: -0.04%, +0.11%
Copies: 25687 -> 26068 (+1.48%); split: -0.04%, +1.53%
PreSGPRs: 15648 -> 15562 (-0.55%)
SALU: 35125 -> 35517 (+1.12%)

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12901
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13019
Fixes: b872ff6ef2 ("aco/insert_exec_mask: if applicable, use s_wqm to restore exec after divergent CF")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34659>
 2025-04-23 09:37:50 +00:00
Georg Lehmann
13f6be262a aco/insert_exec: only restore wqm mask after control flow if necessary 
The next commit will make this not free, so we should avoid it if possible.

Foz-DB Navi31:
Totals from 3933 (4.93% of 79789) affected shaders:
Instrs: 5726914 -> 5727295 (+0.01%); split: -0.00%, +0.01%
CodeSize: 31307100 -> 31308884 (+0.01%); split: -0.00%, +0.01%
SpillSGPRs: 1797 -> 1793 (-0.22%); split: -0.33%, +0.11%
Latency: 58973929 -> 58974343 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 8591893 -> 8591911 (+0.00%); split: -0.00%, +0.00%
SClause: 209074 -> 209115 (+0.02%); split: -0.00%, +0.02%
Copies: 423965 -> 432420 (+1.99%)
Branches: 149976 -> 149979 (+0.00%); split: -0.00%, +0.00%
PreSGPRs: 200175 -> 200663 (+0.24%)
VALU: 3440165 -> 3440156 (-0.00%); split: -0.00%, +0.00%
SALU: 555727 -> 556143 (+0.07%); split: -0.00%, +0.08%

Fixes: b872ff6ef2 ("aco/insert_exec_mask: if applicable, use s_wqm to restore exec after divergent CF")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34659>
 2025-04-23 09:37:50 +00:00
Georg Lehmann
de45676efd aco/insert_exec: reset exec temporary after combined p_demote + p_end_wqm
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
Details
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Details 
Otherwise the next divergent merge block might re-enable demoted invocations.

Fixes: 90faadae72 ("aco/insert_exec_mask: don't disable dead quads on demote in divergent CF")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12898
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12912
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34278>
 2025-03-31 06:43:22 +00:00
Daniel Schürmann
90faadae72 aco/insert_exec_mask: don't disable dead quads on demote in divergent CF 
Also force-enalbe helpers in case of demote in divergent CF.

Totals from 1305 (1.64% of 79377) affected shaders: (Navi31)

Instrs: 926923 -> 922516 (-0.48%); split: -0.48%, +0.00%
CodeSize: 5045292 -> 5027408 (-0.35%); split: -0.36%, +0.00%
Latency: 6176577 -> 6174708 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 931603 -> 931583 (-0.00%); split: -0.00%, +0.00%
SClause: 22816 -> 22855 (+0.17%); split: -0.17%, +0.34%
Copies: 57347 -> 55170 (-3.80%); split: -3.81%, +0.01%
Branches: 18990 -> 18974 (-0.08%)
PreSGPRs: 42734 -> 43248 (+1.20%)
SALU: 90511 -> 86153 (-4.81%); split: -4.85%, +0.04%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33619>
 2025-03-26 08:45:12 +00:00
Daniel Schürmann
b872ff6ef2 aco/insert_exec_mask: if applicable, use s_wqm to restore exec after divergent CF 
Totals from 4740 (5.97% of 79377) affected shaders: (Navi31)

Instrs: 6273963 -> 6273410 (-0.01%); split: -0.01%, +0.00%
CodeSize: 34306560 -> 34304284 (-0.01%); split: -0.01%, +0.00%
SpillSGPRs: 1793 -> 1797 (+0.22%); split: -0.11%, +0.33%
Latency: 62599300 -> 62598714 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 9117199 -> 9117189 (-0.00%); split: -0.00%, +0.00%
SClause: 223548 -> 223529 (-0.01%); split: -0.02%, +0.01%
Copies: 464248 -> 454711 (-2.05%); split: -2.06%, +0.00%
Branches: 161446 -> 161443 (-0.00%); split: -0.00%, +0.00%
PreSGPRs: 226278 -> 225608 (-0.30%)
VALU: 3793235 -> 3793244 (+0.00%); split: -0.00%, +0.00%
SALU: 606184 -> 605759 (-0.07%); split: -0.08%, +0.01%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33619>
 2025-03-26 08:45:12 +00:00
Georg Lehmann
c249556bf4 aco/insert_exec: fix continue_or_break on gfx6-7 
s_cmp_lg_u64 is gfx8+

Fixes: 115ff5f95b ("aco/insert_exec_mask: don't restore exec in continue_or_break blocks")

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33715>
 2025-02-24 20:41:17 +00:00
Daniel Schürmann
115ff5f95b aco/insert_exec_mask: don't restore exec in continue_or_break blocks 
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33527>
 2025-02-24 13:11:20 +00:00
Daniel Schürmann
7f7c1d463a aco/insert_exec_mask: Don't immediately set exec to zero in break/continue blocks 
Instead, only indicate that exec should be zero and do
so in the successive helper block. This allows to insert
the parallelcopies from logical phis directly before the
branch in break and continue blocks.

Totals from 56 (0.07% of 79377) affected shaders: (Navi31)
Latency: 2472367 -> 2472422 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 253053 -> 253055 (+0.00%); split: -0.00%, +0.00%

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33527>
 2025-02-24 13:11:20 +00:00
Georg Lehmann
272ff275fa aco/insert_exec: reset top exec for p_discard_if 
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12363
Fixes: 31f62a6123 ("aco/insert_exec: don't always reset top exec")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32830>
 2025-01-02 15:18:48 +00:00
Georg Lehmann
42512208d8 aco/insert_exec: exit shader using exec for top level discard 
Totals from 14538 (18.31% of 79395) affected shaders:
no changes

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32731>
 2024-12-30 13:05:23 +00:00
Georg Lehmann
c279e63a79 aco: rename p_early_exit_if to if_not 
It exits the shaders if the condition is false, not true.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32731>
 2024-12-30 13:05:23 +00:00
Daniel Schürmann
b64fff7731 aco: remove definition from Pseudo branch instructions 
They are not needed anymore.

Totals from 7019 (8.84% of 79395) affected shaders: (Navi31)

Instrs: 14805400 -> 14824196 (+0.13%); split: -0.00%, +0.13%
CodeSize: 78079972 -> 78132932 (+0.07%); split: -0.01%, +0.08%
SpillSGPRs: 4485 -> 4515 (+0.67%); split: -0.76%, +1.43%
Latency: 165862000 -> 165836134 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 30061764 -> 30057781 (-0.01%); split: -0.01%, +0.00%
SClause: 392323 -> 392286 (-0.01%); split: -0.01%, +0.00%
Copies: 1012262 -> 1012234 (-0.00%); split: -0.04%, +0.04%
Branches: 365910 -> 365909 (-0.00%); split: -0.00%, +0.00%
PreSGPRs: 360167 -> 355363 (-1.33%)
VALU: 8837197 -> 8837276 (+0.00%); split: -0.00%, +0.00%
SALU: 1402593 -> 1402621 (+0.00%); split: -0.03%, +0.03%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32037>
 2024-12-06 14:34:03 +00:00
Georg Lehmann
d2dcaf1f5e aco/insert_exec: reuse old exec temp instead using s_and_saveexec 
This means the v_cmpx optimization in ssa_elimination no longer
needs to insert a copy to save exec.

Foz-DB Navi31:
Totals from 13816 (17.40% of 79395) affected shaders:
Instrs: 23694267 -> 23670199 (-0.10%); split: -0.11%, +0.01%
CodeSize: 124559288 -> 124457508 (-0.08%); split: -0.09%, +0.01%
SpillSGPRs: 5324 -> 5354 (+0.56%); split: -1.00%, +1.56%
Latency: 207245846 -> 207213681 (-0.02%); split: -0.03%, +0.01%
InvThroughput: 35442657 -> 35437220 (-0.02%); split: -0.02%, +0.01%
VClause: 444672 -> 444670 (-0.00%); split: -0.00%, +0.00%
SClause: 639419 -> 639373 (-0.01%); split: -0.04%, +0.03%
Copies: 1529008 -> 1515871 (-0.86%); split: -1.02%, +0.16%
Branches: 557201 -> 557701 (+0.09%); split: -0.00%, +0.09%
PreSGPRs: 682840 -> 686048 (+0.47%)
VALU: 13978010 -> 13978032 (+0.00%); split: -0.00%, +0.00%
SALU: 2214600 -> 2197061 (-0.79%); split: -0.81%, +0.02%
VOPD: 5561 -> 5560 (-0.02%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31567>
 2024-10-23 19:34:53 +00:00
Georg Lehmann
0471522377 aco/insert_exec: reuse old exec temp in loop pre-header 
Avoid an exec copy.

Foz-DB Navi31:
Totals from 2315 (2.92% of 79395) affected shaders:
Instrs: 9082831 -> 9058990 (-0.26%); split: -0.27%, +0.00%
CodeSize: 48017244 -> 47858064 (-0.33%); split: -0.34%, +0.01%
SpillSGPRs: 1680 -> 1684 (+0.24%); split: -0.48%, +0.71%
Latency: 109511718 -> 109525041 (+0.01%); split: -0.01%, +0.02%
InvThroughput: 20287085 -> 20289370 (+0.01%); split: -0.00%, +0.02%
VClause: 192259 -> 192260 (+0.00%)
SClause: 234082 -> 234124 (+0.02%); split: -0.01%, +0.03%
Copies: 667271 -> 645577 (-3.25%); split: -3.27%, +0.02%
Branches: 264086 -> 264088 (+0.00%)
PreSGPRs: 136831 -> 136966 (+0.10%)
VALU: 5234735 -> 5234740 (+0.00%); split: -0.00%, +0.00%
SALU: 949283 -> 927327 (-2.31%); split: -2.32%, +0.01%
VOPD: 1529 -> 1535 (+0.39%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31567>
 2024-10-23 19:34:53 +00:00
Georg Lehmann
31f62a6123 aco/insert_exec: don't always reset top exec 
This allows to re-use previous temporaries in case exec was restored
from a Temp, rather than having to create a new copy of exec.

Foz-DB Navi31:
Totals from 545 (0.69% of 79395) affected shaders:
Instrs: 216563 -> 215698 (-0.40%)
CodeSize: 1183536 -> 1180076 (-0.29%)
Latency: 1135269 -> 1135294 (+0.00%); split: -0.00%, +0.00%
Copies: 11933 -> 11072 (-7.22%)
SALU: 18990 -> 18129 (-4.53%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31567>
 2024-10-23 19:34:53 +00:00
Georg Lehmann
4f04e6f0c4 aco/insert_exec: avoid phis for masks in exec 
Exec always contains the same value as the top of stack, even if the
top of stack is a temporary/constant. So if the predecessors have different
top of stack operands, don't insert a phi and use exec as the new top of stack.

No Foz-DB changes.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31567>
 2024-10-23 19:34:53 +00:00
Georg Lehmann
5da34ebee4 aco/insert_exec: remove get_exec_op 
We used to only store Temps in the stack, so undef meant exec.
Then the stack was changed to operands, and some places started storing exec
directly, drop the undef handling by replacing everything with Operand(exec, lm)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31560>
 2024-10-22 17:03:26 +00:00
Georg Lehmann
6716fb08d8 aco/insert_exec: remove unused includes 
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31560>
 2024-10-22 17:03:26 +00:00
Georg Lehmann
23fb0883eb aco/insert_exec: untangle add_branch_code control flow 
All of the single ifs with return hide that this is effectively almost an
if-else chain, so convert it to one for real.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31560>
 2024-10-22 17:03:26 +00:00
Georg Lehmann
de7d931962 aco/insert_exec: remove stray break_cond variable 
This was always trivial since some discard rework.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31560>
 2024-10-22 17:03:26 +00:00
Georg Lehmann
ade7f1a203 aco/insert_exec: replace pair with a named struct 
.first and .second everywhere was hard to read.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31560>
 2024-10-22 17:03:26 +00:00
Georg Lehmann
a3054499ba aco/insert_exec: don't pretend WQMState is a bit mask 
It's a simple enum.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31560>
 2024-10-22 17:03:26 +00:00
Rhys Perry
b934255510 aco: split selection_control_remove into rarely_taken and never_taken 
No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Backport-to: 24.1
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30321>
 2024-08-15 16:00:18 +00:00
Rhys Perry
cccfbe6141 aco: move s_setprio to before NGG exec initialization 
fossil-db (gfx1150):
Totals from 32 (0.04% of 79395) affected shaders:
Instrs: 17397 -> 17365 (-0.18%)
CodeSize: 83700 -> 83580 (-0.14%)
Latency: 59006 -> 58974 (-0.05%)

fossil-db (navi21):
Totals from 4 (0.01% of 79395) affected shaders:

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30241>
 2024-07-23 13:14:52 +00:00
Rhys Perry
71afacff39 aco/insert_exec_mask: ensure top mask is not a temporary at loop exits 
This is problematic when the successor of the loop exit is an invert
block. It assumes that the top mask is Operand(bld.lm) and doesn't change
it when entering the else branch.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11348
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29767>
 2024-06-20 12:47:05 +00:00
Samuel Pitoiset
7a69d78ba2 aco: use SPDX-License-Identifier 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622>
 2024-04-08 15:49:25 +00:00
Daniel Schürmann
a863c7951e aco: remove create_instruction() template parameter 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
9b0ebcc39b aco: change return type of create_instruction() to Instruction* 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
1187189235 aco: unify different SALU types into single struct SALU_instruction 
This removes
- SOP1_instruction
- SOP2_instruction
- SOPC_instruction
- SOPK_instruction
- SOPP_instruction

and their corresponding methods.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
9bbb9f1104 aco: use small_vec as Block::edge_vec for predecessors and successors 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27984>
 2024-03-19 13:06:58 +00:00
Daniel Schürmann
4fa27845e5 aco/insert_exec_mask: Reduce latency when switching to WQM. 
Change pattern:
s_mov_b64 s[0:1], exec         s_mov_b64 s[0:1], exec
s_wqm_b64 exec, s[0:1]   ->    s_wqm_b64 exec, exec

Totals from 16667 (21.03% of 79242) affected shaders: (GFX11)

Instrs: 11317502 -> 11307484 (-0.09%); split: -0.09%, +0.00%
CodeSize: 60194272 -> 60155088 (-0.07%); split: -0.07%, +0.00%
Latency: 94345873 -> 94338374 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 13568872 -> 13568683 (-0.00%); split: -0.00%, +0.00%
Copies: 808334 -> 808332 (-0.00%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27112>
 2024-02-02 18:55:15 +00:00
Daniel Schürmann
e89977ff71 aco: always terminate quads if they have been demoted entirely 
Previously, quads got only terminated in top-level control flow.
This patch makes the behavior consistent.

Totals from 7811 (9.86% of 79242) affected shaders: (GFX11)

Instrs: 7859667 -> 7850757 (-0.11%); split: -0.18%, +0.07%
CodeSize: 41642280 -> 41611836 (-0.07%); split: -0.13%, +0.06%
Latency: 73692815 -> 73707588 (+0.02%); split: -0.02%, +0.04%
InvThroughput: 10672160 -> 10672323 (+0.00%); split: -0.01%, +0.01%
VClause: 137478 -> 137469 (-0.01%); split: -0.02%, +0.02%
SClause: 314905 -> 314924 (+0.01%); split: -0.19%, +0.20%
Copies: 587014 -> 576039 (-1.87%); split: -2.10%, +0.23%
Branches: 213101 -> 213123 (+0.01%); split: -0.01%, +0.02%
PreSGPRs: 313588 -> 313355 (-0.07%); split: -0.09%, +0.01%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27112>
 2024-02-02 18:55:15 +00:00
Daniel Schürmann
a42b83e3fb aco/insert_exec_mask: tiny refactor 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27112>
 2024-02-02 18:55:15 +00:00
Daniel Schürmann
c309d20172 aco/insert_exec_mask: Fix unconditional demote at top-level control flow. 
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27362>
 2024-01-31 13:50:46 +00:00
Daniel Schürmann
09413ff745 aco/insert_exec_mask: only create loop phis for exec mask if necessary 
Totals from 195 (0.25% of 79242) affected shaders: (GFX11)

Instrs: 476457 -> 476031 (-0.09%); split: -0.23%, +0.14%
CodeSize: 2453964 -> 2452108 (-0.08%); split: -0.23%, +0.16%
SpillSGPRs: 944 -> 913 (-3.28%); split: -3.39%, +0.11%
SpillVGPRs: 838 -> 835 (-0.36%); split: -0.95%, +0.60%
Latency: 10811026 -> 10810125 (-0.01%); split: -0.08%, +0.07%
InvThroughput: 2276677 -> 2276698 (+0.00%); split: -0.12%, +0.12%
VClause: 9223 -> 9233 (+0.11%); split: -0.10%, +0.21%
SClause: 9025 -> 9005 (-0.22%); split: -0.38%, +0.16%
Copies: 67419 -> 67382 (-0.05%); split: -0.97%, +0.92%
PreSGPRs: 10830 -> 10668 (-1.50%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26937>
 2024-01-12 09:05:15 +00:00
Daniel Schürmann
e83d8e1366 aco/insert_exec_mask: replace phi for loop restore mask with explicit copies 
Totals from 1785 (2.25% of 79242) affected shaders: (GFX11)

Instrs: 6787574 -> 6787041 (-0.01%); split: -0.01%, +0.00%
CodeSize: 34906500 -> 34904704 (-0.01%); split: -0.01%, +0.01%
SpillSGPRs: 5848 -> 5816 (-0.55%)
Latency: 88616877 -> 88617209 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 16644948 -> 16644717 (-0.00%); split: -0.00%, +0.00%
VClause: 141122 -> 141121 (-0.00%)
SClause: 178929 -> 178906 (-0.01%); split: -0.03%, +0.02%
Copies: 569444 -> 569081 (-0.06%); split: -0.09%, +0.03%
Branches: 186980 -> 186961 (-0.01%); split: -0.01%, +0.00%
PreSGPRs: 133648 -> 133369 (-0.21%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26937>
 2024-01-12 09:05:15 +00:00