fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2025-12-23 19:50:11 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
186929 commits 64 branches 914 tags 1 GiB
Commit graph

109 commits

Author SHA1 Message Date
Daniel Schürmann
9b0ebcc39b aco: change return type of create_instruction() to Instruction* 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
1187189235 aco: unify different SALU types into single struct SALU_instruction 
This removes
- SOP1_instruction
- SOP2_instruction
- SOPC_instruction
- SOPK_instruction
- SOPP_instruction

and their corresponding methods.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
9bbb9f1104 aco: use small_vec as Block::edge_vec for predecessors and successors 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27984>
 2024-03-19 13:06:58 +00:00
Daniel Schürmann
4fa27845e5 aco/insert_exec_mask: Reduce latency when switching to WQM. 
Change pattern:
s_mov_b64 s[0:1], exec         s_mov_b64 s[0:1], exec
s_wqm_b64 exec, s[0:1]   ->    s_wqm_b64 exec, exec

Totals from 16667 (21.03% of 79242) affected shaders: (GFX11)

Instrs: 11317502 -> 11307484 (-0.09%); split: -0.09%, +0.00%
CodeSize: 60194272 -> 60155088 (-0.07%); split: -0.07%, +0.00%
Latency: 94345873 -> 94338374 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 13568872 -> 13568683 (-0.00%); split: -0.00%, +0.00%
Copies: 808334 -> 808332 (-0.00%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27112>
 2024-02-02 18:55:15 +00:00
Daniel Schürmann
e89977ff71 aco: always terminate quads if they have been demoted entirely 
Previously, quads got only terminated in top-level control flow.
This patch makes the behavior consistent.

Totals from 7811 (9.86% of 79242) affected shaders: (GFX11)

Instrs: 7859667 -> 7850757 (-0.11%); split: -0.18%, +0.07%
CodeSize: 41642280 -> 41611836 (-0.07%); split: -0.13%, +0.06%
Latency: 73692815 -> 73707588 (+0.02%); split: -0.02%, +0.04%
InvThroughput: 10672160 -> 10672323 (+0.00%); split: -0.01%, +0.01%
VClause: 137478 -> 137469 (-0.01%); split: -0.02%, +0.02%
SClause: 314905 -> 314924 (+0.01%); split: -0.19%, +0.20%
Copies: 587014 -> 576039 (-1.87%); split: -2.10%, +0.23%
Branches: 213101 -> 213123 (+0.01%); split: -0.01%, +0.02%
PreSGPRs: 313588 -> 313355 (-0.07%); split: -0.09%, +0.01%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27112>
 2024-02-02 18:55:15 +00:00
Daniel Schürmann
a42b83e3fb aco/insert_exec_mask: tiny refactor 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27112>
 2024-02-02 18:55:15 +00:00
Daniel Schürmann
c309d20172 aco/insert_exec_mask: Fix unconditional demote at top-level control flow. 
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27362>
 2024-01-31 13:50:46 +00:00
Daniel Schürmann
09413ff745 aco/insert_exec_mask: only create loop phis for exec mask if necessary 
Totals from 195 (0.25% of 79242) affected shaders: (GFX11)

Instrs: 476457 -> 476031 (-0.09%); split: -0.23%, +0.14%
CodeSize: 2453964 -> 2452108 (-0.08%); split: -0.23%, +0.16%
SpillSGPRs: 944 -> 913 (-3.28%); split: -3.39%, +0.11%
SpillVGPRs: 838 -> 835 (-0.36%); split: -0.95%, +0.60%
Latency: 10811026 -> 10810125 (-0.01%); split: -0.08%, +0.07%
InvThroughput: 2276677 -> 2276698 (+0.00%); split: -0.12%, +0.12%
VClause: 9223 -> 9233 (+0.11%); split: -0.10%, +0.21%
SClause: 9025 -> 9005 (-0.22%); split: -0.38%, +0.16%
Copies: 67419 -> 67382 (-0.05%); split: -0.97%, +0.92%
PreSGPRs: 10830 -> 10668 (-1.50%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26937>
 2024-01-12 09:05:15 +00:00
Daniel Schürmann
e83d8e1366 aco/insert_exec_mask: replace phi for loop restore mask with explicit copies 
Totals from 1785 (2.25% of 79242) affected shaders: (GFX11)

Instrs: 6787574 -> 6787041 (-0.01%); split: -0.01%, +0.00%
CodeSize: 34906500 -> 34904704 (-0.01%); split: -0.01%, +0.01%
SpillSGPRs: 5848 -> 5816 (-0.55%)
Latency: 88616877 -> 88617209 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 16644948 -> 16644717 (-0.00%); split: -0.00%, +0.00%
VClause: 141122 -> 141121 (-0.00%)
SClause: 178929 -> 178906 (-0.01%); split: -0.03%, +0.02%
Copies: 569444 -> 569081 (-0.06%); split: -0.09%, +0.03%
Branches: 186980 -> 186961 (-0.01%); split: -0.01%, +0.00%
PreSGPRs: 133648 -> 133369 (-0.21%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26937>
 2024-01-12 09:05:15 +00:00
Daniel Schürmann
d375d297cf aco/insert_exec_mask: unify exec restore code after divergent control flow 
No fossil-db changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26937>
 2024-01-12 09:05:15 +00:00
Daniel Schürmann
dce695b24f aco: refactor and speed-up dead code analysis 
Assuming that no loop header phis are dead code,
we can perform the dead code analysis in a single iteration.

Totals from 25 (0.03% of 79330) affected shaders: (GFX11)

MaxWaves: 664 -> 662 (-0.30%)
Instrs: 487618 -> 488822 (+0.25%)
CodeSize: 2451548 -> 2459756 (+0.33%)
VGPRs: 1296 -> 1332 (+2.78%)
Latency: 2337256 -> 2338098 (+0.04%); split: -0.00%, +0.04%
InvThroughput: 560682 -> 576158 (+2.76%)
VClause: 15782 -> 15790 (+0.05%)
Copies: 37905 -> 38731 (+2.18%)
PreVGPRs: 1124 -> 1156 (+2.85%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26901>
 2024-01-08 09:43:53 +00:00
Qiang Yu
67244fc88a aco: remove p_end_with_regs from needs_exact() 
ps needs to handle wqm:
1. main part may compute with args from prolog in wqm mode, so
   prolog need to compute these args in wqm mode too.
2. prolog and main part need to end with exact exec, so next
   shader part which inherit previous shader part's exec won't
   do valid job for helper threads

1 need p_end_with_regs to operate in wqm mode and itself can't
be exact, otherwise some move instruction added by it won't be
in wqm mode so helper threads' compute result is not passed to
next shader part as args.

2 is done by p_end_wqm added by finish_program automatically
after p_end_with_regs.

Piglit tests can trigger the problem:

1. gl-2.1-polygon-stipple-fs
  a. ps prolog call discard_if
  b. ps main pass wqm exec to epilog
  c. ps epilog export color for discarded pixel

2. fs-fwidth-color.shader_test
  a. ps prolog need to pass args computed in wqm mode
  b. set p_end_with_regs to exact will end wqm mode before
     the move instructions, so helper threads's result is not
     passed to next shader part

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24973>
 2023-10-10 02:36:33 +00:00
Daniel Schürmann
6eaf416f35 aco/insert_exec_mask: Simplify WQM handling (2/2) 
by calculating WQM requirements on demand.

No fossil-db changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25038>
 2023-09-14 09:25:23 +00:00
Daniel Schürmann
5f66723188 aco/insert_exec_mask: Simplify WQM handling (1/2) 
by using p_end_wqm as indicator for when to end WQM mode.

Totals from 10049 (13.12% of 76572) affected shaders: (GFX11)

MaxWaves: 301126 -> 301136 (+0.00%)
Instrs: 7061909 -> 7049272 (-0.18%); split: -0.21%, +0.03%
CodeSize: 37720684 -> 37664244 (-0.15%); split: -0.18%, +0.03%
VGPRs: 357204 -> 357180 (-0.01%); split: -0.13%, +0.12%
Latency: 62757830 -> 62827080 (+0.11%); split: -0.06%, +0.17%
InvThroughput: 8589248 -> 8589963 (+0.01%); split: -0.02%, +0.02%
VClause: 132541 -> 132547 (+0.00%); split: -0.03%, +0.03%
SClause: 322916 -> 322964 (+0.01%); split: -0.04%, +0.05%
Copies: 546446 -> 547657 (+0.22%); split: -0.13%, +0.35%
Branches: 189527 -> 188293 (-0.65%)
PreSGPRs: 332792 -> 332529 (-0.08%); split: -0.08%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25038>
 2023-09-14 09:25:23 +00:00
Daniel Schürmann
45f6d38a76 aco: insert a single p_end_wqm after the last derivative calculation 
This new instruction replaces p_wqm.

Totals from 28065 (36.65% of 76572) affected shaders: (GFX11)
MaxWaves: 823922 -> 823952 (+0.00%); split: +0.01%, -0.01%
Instrs: 22221375 -> 22180465 (-0.18%); split: -0.26%, +0.08%
CodeSize: 117310676 -> 117040684 (-0.23%); split: -0.30%, +0.07%
VGPRs: 1183476 -> 1186656 (+0.27%); split: -0.19%, +0.46%
SpillSGPRs: 2305 -> 2302 (-0.13%)
Latency: 176559310 -> 176427793 (-0.07%); split: -0.21%, +0.14%
InvThroughput: 26245204 -> 26195550 (-0.19%); split: -0.26%, +0.07%
VClause: 368058 -> 369460 (+0.38%); split: -0.21%, +0.59%
SClause: 857077 -> 842588 (-1.69%); split: -2.06%, +0.37%
Copies: 1245650 -> 1249434 (+0.30%); split: -0.33%, +0.63%
Branches: 394837 -> 396070 (+0.31%); split: -0.01%, +0.32%
PreSGPRs: 1019139 -> 1019567 (+0.04%); split: -0.02%, +0.06%
PreVGPRs: 925739 -> 931860 (+0.66%); split: -0.00%, +0.66%

Changes are due to scheduling and re-enabling cross-lane optimizations.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25038>
 2023-09-14 09:25:23 +00:00
Daniel Schürmann
0907b53740 aco/insert_exec_mask: set Exact mode after p_discard_if when necessary 
Fixes: 5e9df85b1a ('aco: optimize discard_if when WQM is not needed afterwards')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25038>
 2023-09-14 09:25:22 +00:00
Rhys Perry
41b6020ff3 aco: remove fast path in insert_exec_mask's process_instructions 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25038>
 2023-09-14 09:25:22 +00:00
Samuel Pitoiset
37aa6d25e1 aco: ensure to initialize exec manually for non-monolithic {VS,TES}/GS on GFX9+ 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24862>
 2023-08-25 10:22:41 +00:00
Samuel Pitoiset
196b355db6 aco: ensure to initialize exec manually for VS as LS on GFX9+ 
When VS and TCS are compiled separately with shader object on GFX9+.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
 2023-08-25 07:22:04 +00:00
Qiang Yu
85d9646288 aco: add p_end_with_regs pseudo instruction 
Used by radeonsi shader parts to pass args from one part to another.
It has variable number of operands to reserve fixed registers with
wanted value.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24442>
 2023-08-16 02:27:45 +00:00
Timur Kristóf
05928f4200 aco: Use ac_hw_stage instead of aco-specific HWStage. 
The new ac_hw_stage is going to be used by drivers as well.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23597>
 2023-06-23 12:49:04 +00:00
Eric Engestrom
6b21653ab4 aco: reformat according to its .clang-format 
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23253>
 2023-06-16 19:59:52 +00:00
Friedrich Vock
9de8134410 aco: Fix assert in insert_exec_mask 
This assert would trigger on unconditional demotes, because the demotes
don't remove the mask_type_global flag from the exec mask.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23594>
 2023-06-12 14:20:28 +00:00
Timur Kristóf
8e9d269da6 aco: Don't use nir_selection_control in aco_ir. 
We don't want to rely on any NIR structures in ACO, because
we would like to avoid the need to include nir.h in aco_ir.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22241>
 2023-04-10 20:01:28 +00:00
Daniel Schürmann
caec48529b aco/insert_exec_mask: allow for disconnected CFG 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20853>
 2023-03-12 18:07:18 +00:00
Timur Kristóf
81620fc7b0 aco: Enable constant exec mask based optimization on compute shaders. 
We know for sure exec is initially -1 when the shader always has full subgroups.

Fossil DB stats on GFX11:
Totals from 3884 (2.88% of 134913) affected shaders:
SpillSGPRs: 1673 -> 1697 (+1.43%); split: -1.67%, +3.11%
SpillVGPRs: 2316 -> 2310 (-0.26%); split: -0.65%, +0.39%
CodeSize: 19584436 -> 19567156 (-0.09%); split: -0.13%, +0.04%
Scratch: 217088 -> 216832 (-0.12%)
Instrs: 3784596 -> 3780303 (-0.11%); split: -0.15%, +0.03%
Latency: 39971204 -> 39794967 (-0.44%); split: -0.47%, +0.03%
InvThroughput: 7885552 -> 7801247 (-1.07%); split: -1.14%, +0.07%
VClause: 74654 -> 74611 (-0.06%); split: -0.07%, +0.01%
SClause: 103139 -> 103043 (-0.09%); split: -0.13%, +0.04%
Copies: 279864 -> 281995 (+0.76%); split: -0.72%, +1.48%
Branches: 92082 -> 92084 (+0.00%); split: -0.03%, +0.03%
PreSGPRs: 155637 -> 149491 (-3.95%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20670>
 2023-01-26 01:59:26 +00:00
Rhys Perry
c3dd1931d9 aco: allow Builder::Result to be dereferenced 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20251>
 2023-01-10 16:01:38 +00:00
Samuel Pitoiset
bb90d29660 aco: add p_dual_src_export_gfx11 for dual source blending on GFX11 
Dual source blending must be in strict WQM mode.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19643>
 2022-11-16 18:35:10 +00:00
Timur Kristóf
d8639b7a80 aco: Allow explicitly removing jumps on GFX10+ when beneficial. 
"Removing jumps" in ACO means skipping the jump instruction
at the beginning of a divergent branch (but still modify exec).

ACO already supports implicitly removing jumps when it decides
that executing a branch with empty exec mask is more beneficial
than a jump.

This commit adds the possibility to use this explicitly
through nir_selection_control. ACO will respect this
setting and remove the branch instructions when this is specified,
unless it decides that this would cause bugs (eg. exp instruction).

There are two cases that benefit from the new change:

1. When the application requests to "flatten" a branch (ie.
remove control flow), we now respect that.
2. When the compiler stack determines that a divergent branch
is always taken.

v2 by Georg Lehmann: fixed applying sel_ctrl to else blocks

Fossil DB stats on Navi 21:

Totals from 13 (0.01% of 134906) affected shaders:
CodeSize: 136616 -> 136496 (-0.09%)
Instrs: 26196 -> 26166 (-0.11%)
Latency: 417928 -> 417889 (-0.01%)
Branches: 1241 -> 1211 (-2.42%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-By: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17921>
 2022-10-11 15:42:54 +00:00
Samuel Pitoiset
e840ba9ed8 aco: requires Exact for p_jump_to_epilog 
Otherwise, in presence of p_exit_early_if the main FS will always
jump to the PS epilog regardless the exact mask.

This fixes dEQP-VK.draw.renderpass.shader_invocation.helper_invocation
and few vkd3d-proton regressions when PS epilogs are forced.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17617>
 2022-07-19 17:52:36 +00:00
Daniel Schürmann
ac39e7bf23 aco: fix assertion in insert_exec_mask 
The exec mask might also be of type mask_type_loop.

Fixes: d068eb53e8 ('aco/insert_exec_mask: optimize top-level transition to exact before demote')
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17402>
 2022-07-19 16:56:37 +00:00
Daniel Schürmann
6de68c5dca aco: Avoid live-range splits in Exact mode 
Because the data register of atomic VMEM instructions
is shared between src and dst, it might be necessary
to create live-range splits during RA.
Make the live-range splits explicit in WQM mode.

Totals from 7 (0.01% of 134913) affected shaders: (GFX10.3)
Latency: 17209 -> 17210 (+0.01%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15347>
 2022-07-19 16:30:49 +00:00
Rhys Perry
d2d94b62f2 aco: initialize scratch base registers on GFX9-GFX10.3 
fossil-db (navi21):
Totals from 1142 (0.70% of 162293) affected shaders:
Instrs: 271636 -> 271974 (+0.12%)
CodeSize: 1532020 -> 1533792 (+0.12%)
Latency: 7484066 -> 7485698 (+0.02%)
InvThroughput: 4048824 -> 4049579 (+0.02%)
SClause: 4171 -> 4212 (+0.98%)
PreSGPRs: 11203 -> 12276 (+9.58%)

fossil-db (vega10):
Totals from 3327 (2.06% of 161355) affected shaders:
Instrs: 257413 -> 257601 (+0.07%)
CodeSize: 1424244 -> 1425372 (+0.08%)
Latency: 8598402 -> 8600466 (+0.02%)
InvThroughput: 7906335 -> 7908234 (+0.02%)
SClause: 4932 -> 4973 (+0.83%)
PreSGPRs: 22010 -> 25405 (+15.42%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17079>
 2022-07-08 14:49:03 +00:00
Rhys Perry
d068eb53e8 aco/insert_exec_mask: optimize top-level transition to exact before demote 
fossil-db (Sienna Cichlid):
Totals from 5767 (3.55% of 162293) affected shaders:
Instrs: 3264949 -> 3257527 (-0.23%); split: -0.23%, +0.00%
CodeSize: 17835692 -> 17806004 (-0.17%); split: -0.17%, +0.00%
Latency: 45990060 -> 45987924 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 7643850 -> 7643835 (-0.00%); split: -0.00%, +0.00%
Copies: 193641 -> 186219 (-3.83%); split: -3.84%, +0.01%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15244>
 2022-03-08 12:49:59 +00:00
Rhys Perry
42a5be975a aco/insert_exec_mask: use get_exec_op 
No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15244>
 2022-03-08 12:49:59 +00:00
Rhys Perry
aa55ecc296 aco/insert_exec_mask: fix top-level to-exact with non-global exact mask 
After transitioning to exact after a discard, the exec stack might be:
[exact|global, wqm, exact]

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15244>
 2022-03-08 12:49:59 +00:00
Rhys Perry
ceca5e68c4 aco: remove vcc hint from branch definitions 
This doesn't seem to have much benefit anymore.

fossil-db (Sienna Cichlid):
Totals from 198 (0.15% of 134913) affected shaders:
CodeSize: 2610536 -> 2610872 (+0.01%); split: -0.01%, +0.02%
Instrs: 479001 -> 479085 (+0.02%); split: -0.01%, +0.03%
Latency: 7310684 -> 7300735 (-0.14%); split: -0.16%, +0.02%
InvThroughput: 2439084 -> 2437446 (-0.07%); split: -0.07%, +0.00%
SClause: 14760 -> 14722 (-0.26%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13432>
 2022-03-03 20:21:08 +00:00
Daniel Schürmann
1bbbabedb7 aco/insert_exec_mask: refactor and remove some unnecessary WQM handling code 
Some cases cannot happen and don't need to be handled anymore.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>
 2022-02-11 19:05:30 +00:00
Daniel Schürmann
d7d7b9974a aco/insert_exec_mask: refactor and simplify get_block_needs() 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>
 2022-02-11 19:05:30 +00:00
Daniel Schürmann
fcc5dec8d6 aco/insert_exec_mask: remove ever_again_needs and Exact_Branch 
This information is not required anymore.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>
 2022-02-11 19:05:30 +00:00
Daniel Schürmann
cbb1b095ca aco/insert_exec_mask: remove some unnecessary WQM loop handling code 
These workarounds are were necessary to prevent infinite loops
with helper lane registers containing wrong data.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>
 2022-02-11 19:05:30 +00:00
Daniel Schürmann
580a63b4ac aco/insert_exec_mask: remove Preserve_WQM flag 
If WQM is needed anywhere after discard_if(), it will also
be flagged as WQM. We can rely on that to preserve the WQM mask.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>
 2022-02-11 19:05:30 +00:00
Daniel Schürmann
f816dd1be7 aco: don't propagate WQM for p_as_uniform 
This was needed, so that in case of active helper lanes,
these contain the correct value. It is now handled implicitly.

Totals from 1004 (0.74% of 134913) affected shaders: (GFX10.3)
CodeSize: 7581020 -> 7580892 (-0.00%); split: -0.00%, +0.00%
Instrs: 1454940 -> 1454908 (-0.00%); split: -0.00%, +0.00%
Latency: 12984953 -> 12984894 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 3173037 -> 3173049 (+0.00%); split: -0.00%, +0.00%
PreSGPRs: 47498 -> 47273 (-0.47%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>
 2022-02-11 19:05:30 +00:00
Daniel Schürmann
825cd696dc aco/insert_exec_mask: stay in WQM while helper lanes are still needed 
This patch flags all instructions WQM which don't require
Exact mode, but depend on the exec mask as long as WQM
is needed on any control flow path afterwards.
This will mostly prevent accidental copies of WQM values
within Exact mode, and also makes a lot of other workarounds
unnecessary.

Totals from 17374 (12.88% of 134913) affected shaders: (GFX10.3)
VGPRs: 526952 -> 527384 (+0.08%); split: -0.01%, +0.09%
CodeSize: 33740512 -> 33766636 (+0.08%); split: -0.06%, +0.14%
MaxWaves: 488166 -> 488108 (-0.01%); split: +0.00%, -0.02%
Instrs: 6254240 -> 6260557 (+0.10%); split: -0.08%, +0.18%
Latency: 66497580 -> 66463472 (-0.05%); split: -0.15%, +0.10%
InvThroughput: 13265741 -> 13264036 (-0.01%); split: -0.03%, +0.01%
VClause: 122962 -> 122975 (+0.01%); split: -0.01%, +0.02%
SClause: 334805 -> 334405 (-0.12%); split: -0.51%, +0.39%
Copies: 275728 -> 282341 (+2.40%); split: -0.91%, +3.31%
Branches: 92546 -> 90990 (-1.68%); split: -1.68%, +0.00%
PreSGPRs: 504119 -> 504352 (+0.05%); split: -0.00%, +0.05%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14951>
 2022-02-11 19:05:30 +00:00
Daniel Schürmann
5e9df85b1a aco: optimize discard_if when WQM is not needed afterwards 
Totals from 11560 (8.57% of 134913) affected shaders: (GFX10.3)
CodeSize: 12092560 -> 11997652 (-0.78%)
Instrs: 2205325 -> 2181598 (-1.08%)
Latency: 15376048 -> 15356958 (-0.12%); split: -0.12%, +0.00%
InvThroughput: 3526105 -> 3525120 (-0.03%); split: -0.03%, +0.00%
Copies: 98543 -> 87601 (-11.10%)
Branches: 16919 -> 16873 (-0.27%)
PreSGPRs: 291584 -> 291532 (-0.02%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14805>
 2022-02-08 16:16:07 +00:00
Daniel Schürmann
13c3137960 aco: merge block_kind_uses_[demote|discard_if] 
These serve the same purpose. The new name is
block_kind_uses_discard.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14805>
 2022-02-08 16:16:07 +00:00
Daniel Schürmann
e7d1c8cc5e aco: make Preserve_WQM independent from block_kind_uses_discard_if 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14805>
 2022-02-08 16:16:07 +00:00
Daniel Schürmann
08b8500dfb aco: remove block_kind_discard 
This case doesn't seem to happen in practice.
No need to micro-optimize it.

This patch merges instruction selection for discard/discard_if.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14805>
 2022-02-08 16:16:07 +00:00
Daniel Schürmann
b67092e685 aco: emit nir_intrinsic_discard() as p_discard_if() 
This simplifies the code and emits a slightly better
sequence in some cases.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14805>
 2022-02-08 16:16:07 +00:00
Timur Kristóf
e66f54e5c8 aco: Allow elect to take advantage of knowing when all lanes are active. 
Implement elect using a pseudo-op which is lowered during the
insert_exec_mask pass. This makes it possible to emit a more
optimal sequence when the exec mask is constant.

Fossil DB results on Sienna Cichlid:
Totals from 211 (0.16% of 128647) affected shaders:
CodeSize: 2254356 -> 2240468 (-0.62%); split: -0.62%, +0.00%
Instrs: 438471 -> 434996 (-0.79%); split: -0.80%, +0.01%
Latency: 2717082 -> 2709400 (-0.28%); split: -0.28%, +0.00%
InvThroughput: 566987 -> 566342 (-0.11%); split: -0.11%, +0.00%
Copies: 40058 -> 40162 (+0.26%)
Branches: 31209 -> 31211 (+0.01%)
PreSGPRs: 9927 -> 10125 (+1.99%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11458>
 2021-07-16 14:31:54 +00:00