fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2026-05-18 18:08:15 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
198369 commits 66 branches 935 tags 949 MiB
Commit graph

98 commits

Author SHA1 Message Date
Daniel Schürmann
bb87832ce0 aco/insert_NOPs: add early exit to handle_valu_partial_forwarding_hazard_instr 
No need to continue if there was already a hazard found in
a different control flow path.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32191>
 2024-11-22 08:46:32 +00:00
Daniel Schürmann
07df37ba01 aco/insert_NOPs: use RegCounterMap as replacement for the CounterMap implementation 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32191>
 2024-11-22 08:46:32 +00:00
Daniel Schürmann
fb5e5adfb3 aco/insert_NOPs: implement vector-based RegCounterMap as replacement for VGPRCounterMap 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32191>
 2024-11-22 08:46:32 +00:00
Rhys Perry
295b7d606f aco: insert NOP before dealloc_vgpr in the insert_NOPs pass 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24884>
 2024-11-06 09:58:05 +00:00
Rhys Perry
47e0f468cf aco: workaround VALUReadSGPRHazard 
fossil-db (gfx1200):
Totals from 65112 (82.01% of 79395) affected shaders:
Instrs: 41732906 -> 42987198 (+3.01%); split: -0.00%, +3.01%
CodeSize: 222451964 -> 226942644 (+2.02%); split: -0.01%, +2.03%
Latency: 290411063 -> 290944688 (+0.18%); split: -0.00%, +0.18%
InvThroughput: 45854913 -> 45910275 (+0.12%); split: -0.00%, +0.12%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30478>
 2024-10-24 16:08:07 +00:00
Rhys Perry
9ab0c4b047 aco: minor CounterMap::operator== fix 
I don't think this matters for how we use CounterMap::operator==.

The BITSET_TEST() was unnecessary because of the BITSET_EQUAL above.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30478>
 2024-10-24 16:08:07 +00:00
Rhys Perry
f5b871f825 aco: split CounterMap off from VGPRCounterMap 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30478>
 2024-10-24 16:08:07 +00:00
Georg Lehmann
977f435f4c aco/ir: add function to parse depctr waits 
No Foz-DB changes on Navi31.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31132>
 2024-10-17 11:16:16 +00:00
Rhys Perry
11262a01ce aco: preserve bitsets after a lane mask is written 
fossil-db (navi31):
Totals from 4840 (6.10% of 79395) affected shaders:
Instrs: 13733449 -> 13761177 (+0.20%); split: -0.00%, +0.21%
CodeSize: 71997868 -> 72102520 (+0.15%); split: -0.00%, +0.15%
Latency: 128385177 -> 128408780 (+0.02%); split: -0.00%, +0.02%
InvThroughput: 21105847 -> 21109475 (+0.02%); split: -0.00%, +0.02%
VALU: 7741209 -> 7741210 (+0.00%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Backport-to: 24.1
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30818>
 2024-08-26 19:16:34 +00:00
Rhys Perry
61e73c2323 aco: check SALU writing lanemask later for VALUMaskWriteHazard 
This should be done after reads are checked and
sgpr_read_by_valu_as_lanemask_then_wr_by_salu is reset. The old version
also skipped checking the reads if the write check passed.

fossil-db (navi31):
Totals from 193 (0.24% of 79395) affected shaders:
Instrs: 3212435 -> 3212735 (+0.01%)
CodeSize: 16462868 -> 16463848 (+0.01%); split: -0.00%, +0.01%
Latency: 19492377 -> 19492462 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 4419705 -> 4419718 (+0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Backport-to: 24.1
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30818>
 2024-08-26 19:16:34 +00:00
Rhys Perry
b1ba7d1b99 aco: don't consider sa_sdst=0 before SALU write to fix VALUMaskWriteHazard 
LLVM does but that's probably a bug.

fossil-db (navi31):
Totals from 311 (0.39% of 79395) affected shaders:
Instrs: 380453 -> 381075 (+0.16%)
CodeSize: 1961012 -> 1964744 (+0.19%)
Latency: 4799095 -> 4800313 (+0.03%)
InvThroughput: 958358 -> 958904 (+0.06%)
VALU: 242322 -> 242633 (+0.13%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Backport-to: 24.1
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30818>
 2024-08-26 19:16:34 +00:00
Rhys Perry
8f5ee70d85 aco: also consider VALU reads for VALUMaskWriteHazard 
fossil-db (navi31):
Totals from 9776 (12.31% of 79395) affected shaders:
Instrs: 19348258 -> 19383680 (+0.18%); split: -0.00%, +0.19%
CodeSize: 101223460 -> 101366964 (+0.14%); split: -0.01%, +0.15%
Latency: 172853115 -> 172866070 (+0.01%); split: -0.01%, +0.01%
InvThroughput: 27590468 -> 27592390 (+0.01%); split: -0.00%, +0.01%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11550
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11436
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11337
Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11738
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11741
Backport-to: 24.1
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30818>
 2024-08-26 19:16:34 +00:00
Rhys Perry
ee648326d9 aco: ignore exec and literals when mitigating VALUMaskWriteHazard 
LLVM ignores exec and literals don't seem to work in some cases.

fossil-db (navi31):
Totals from 2676 (3.37% of 79395) affected shaders:
Instrs: 10638979 -> 10646019 (+0.07%); split: -0.00%, +0.07%
CodeSize: 55929640 -> 55959416 (+0.05%); split: -0.00%, +0.06%
Latency: 107707408 -> 107712893 (+0.01%); split: -0.00%, +0.01%
InvThroughput: 18119843 -> 18120442 (+0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Backport-to: 24.1
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30818>
 2024-08-26 19:16:34 +00:00
Rhys Perry
7b92e11e16 aco: forget valu delays after certain s_waitcnt_depctr/LDSDIR 
fossil-db (navi31):
Totals from 55242 (69.58% of 79395) affected shaders:
Instrs: 40507666 -> 40138006 (-0.91%); split: -0.91%, +0.00%
CodeSize: 212516104 -> 211025880 (-0.70%); split: -0.70%, +0.00%
Latency: 281643258 -> 281628053 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 46370668 -> 46369637 (-0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23337>
 2024-08-22 13:57:01 +00:00
Rhys Perry
0919ce1ac4 aco/gfx11.5: workaround export priority issue 
https://github.com/llvm/llvm-project/pull/99273

fossil-db (gfx1150):
Totals from 73996 (93.20% of 79395) affected shaders:
Instrs: 36015357 -> 36807177 (+2.20%)
CodeSize: 189072544 -> 192238748 (+1.67%)
Latency: 245845181 -> 246790550 (+0.38%); split: -0.00%, +0.38%
InvThroughput: 45068018 -> 45116177 (+0.11%); split: -0.00%, +0.11%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30241>
 2024-07-23 13:14:51 +00:00
Rhys Perry
17758f0a02 aco: fix wmma raw hazard 
No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29912>
 2024-07-01 17:34:22 +00:00
Rhys Perry
872dda2bc5 aco: support GFX12 in insert_NOPs 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29330>
 2024-05-28 10:52:11 +00:00
Rhys Perry
75532d8687 aco: add wait_imm::unpack and wait_imm::max 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28981>
 2024-05-10 11:53:08 +00:00
Samuel Pitoiset
7a69d78ba2 aco: use SPDX-License-Identifier 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622>
 2024-04-08 15:49:25 +00:00
Daniel Schürmann
a863c7951e aco: remove create_instruction() template parameter 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
9b0ebcc39b aco: change return type of create_instruction() to Instruction* 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
1187189235 aco: unify different SALU types into single struct SALU_instruction 
This removes
- SOP1_instruction
- SOP2_instruction
- SOPC_instruction
- SOPK_instruction
- SOPP_instruction

and their corresponding methods.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Daniel Schürmann
5d265257a0 aco: remove SOPP_instruction::block member 
Re-use SOPP_instruction::imm instead.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28370>
 2024-03-28 11:25:43 +00:00
Georg Lehmann
b90ec971d7 aco/gfx11: resolve VcmpxPermlaneHazard for v_permlane64 
The GFX11 ISA docs description of this hazard says it's about v_permlane in
general, not just v_permlane(x)16.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27118>
 2024-01-24 16:38:40 +00:00
Georg Lehmann
19876386e2 aco/gfx11: use v_nop to resolve VcmpxPermlaneHazard 
The GFX11 ISA doc explicitly recommends using v_nop in
7.2.8. PERMLANE Specific Rules.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27118>
 2024-01-24 16:38:40 +00:00
Daniel Schürmann
42e9ba1c70 aco: remove VCCZ and EXECZ register handling 
We don't use these registers and since RDNA3 removed the explicit usage,
it is unlikely that we will properly support them in the future.
Removing the registers from the ACO IR prevents accidentally using them
without proper support.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26664>
 2023-12-14 20:08:28 +00:00
Georg Lehmann
2f4e53b22a aco: fix detecting sgprs read by SMEM hazard 
s_waitcnt_lgkmcnt is SOPK, not SOPP and there are other SOPK instructions
that don't mitigate the hazard.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26163>
 2023-11-15 12:35:32 +00:00
Georg Lehmann
e49c413a86 aco: use null operand for SOPK s_waitcnt 
Both null def and op result in the same correct encoding, but these
instructions optionally read a sgpr, so it makes more sense to use an operand.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26163>
 2023-11-15 12:35:32 +00:00
Rhys Perry
690cb0b211 aco: resolve all possible hazards at the end of shader parts 
fossil-db (vega10):
Totals from 1266 (2.01% of 63055) affected shaders:
Instrs: 707116 -> 708382 (+0.18%)
CodeSize: 3512452 -> 3517516 (+0.14%)
Latency: 6661724 -> 6666788 (+0.08%)
InvThroughput: 4393626 -> 4393904 (+0.01%); split: -0.00%, +0.01%

fossil-db (navi10):
Totals from 1305 (2.07% of 63015) affected shaders:
Instrs: 719699 -> 722009 (+0.32%)
CodeSize: 3650836 -> 3660076 (+0.25%)
Latency: 5691633 -> 5693933 (+0.04%)
InvThroughput: 1532010 -> 1532024 (+0.00%); split: -0.00%, +0.00%

fossil-db (navi31):
Totals from 1580 (1.99% of 79332) affected shaders:
Instrs: 1678242 -> 1679879 (+0.10%)
CodeSize: 8463464 -> 8470168 (+0.08%)
Latency: 14273661 -> 14275298 (+0.01%)
InvThroughput: 3668049 -> 3668080 (+0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25374>
 2023-10-11 15:14:04 +00:00
Rhys Perry
ed3ca5b781 aco: fix s_setreg hazards 
s_setreg doesn't have any definitions.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25374>
 2023-10-11 15:14:04 +00:00
Rhys Perry
ce27875f09 aco: only mitigate VcmpxExecWARHazard when necessary 
fossil-db (navi10):
Totals from 5059 (8.03% of 63015) affected shaders:
Instrs: 7384947 -> 7351196 (-0.46%)
CodeSize: 39393180 -> 39299196 (-0.24%); split: -0.28%, +0.04%
Latency: 119683018 -> 119585224 (-0.08%); split: -0.08%, +0.00%
InvThroughput: 29647188 -> 29623895 (-0.08%); split: -0.08%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25374>
 2023-10-11 15:14:04 +00:00
Rhys Perry
a73f76750b aco: fix LdsDirectVMEMHazard WaW with the wrong waitcnt 
Seems we missed this case.

fossil-db (navi31):
Totals from 24 (0.03% of 79332) affected shaders:
Instrs: 3562 -> 3538 (-0.67%)
CodeSize: 18740 -> 18644 (-0.51%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 2cdb3e4b6b ("aco: add VMEMtoScalarWriteHazard tests")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25374>
 2023-10-11 15:14:04 +00:00
Rhys Perry
6518d09601 aco: don't combine DPP into v_cmpx 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25471>
 2023-09-29 18:23:21 +00:00
Eric Engestrom
6b21653ab4 aco: reformat according to its .clang-format 
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23253>
 2023-06-16 19:59:52 +00:00
Timur Kristóf
2ff1267959 aco: Only include nir.h in instruction selection. 
Don't recompile entire ACO when something changes in NIR.
Instead, only use some headers which are actually needed,
include these in ACO files instead of relying on nir.h to
include them.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22241>
 2023-04-10 20:01:28 +00:00
Georg Lehmann
77afe7d960 aco: treat VINTERP_INREG as VALU 
It's just v_fma with fixed DPP8 and builtin s_waitcnt_expcnt, so it can mostly
be handled as a pure VALU instruction.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21023>
 2023-03-07 11:53:23 +00:00
Rhys Perry
94abccf3ce aco: fix pathological case in LdsDirectVALUHazard 
Similar to bfd4ac4581.

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: 296b4d95a3 ("aco/gfx11: workaround LdsDirectVALUHazard")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21423>
 2023-02-22 20:46:12 +00:00
Rhys Perry
bfd4ac4581 aco: limit VALUPartialForwardingHazard search 
Complicated CFG and lots of SALU can cause this to take an extremely long
time to finish.

Fixes
dEQP-VK.graphicsfuzz.cov-value-tracking-selection-dag-negation-clamp-loop
and Monster Hunter Rise demo compile times.

fossil-db (gfx1100):
Totals from 57 (0.04% of 134574) affected shaders:
Instrs: 170919 -> 171165 (+0.14%)
CodeSize: 860144 -> 861128 (+0.11%)
Latency: 961466 -> 961505 (+0.00%)
InvThroughput: 127598 -> 127608 (+0.01%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8153
Fixes: 5806f0246f ("aco/gfx11: workaround VALUPartialForwardingHazard")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20941>
 2023-02-01 18:52:40 +00:00
Bas Nieuwenhuizen
352e492c7b aco: Add isTrans helper. 
For the s_delay_alu tracking.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19743>
 2022-12-07 22:05:25 +00:00
Rhys Perry
36703a60bf aco: add ACO_DEBUG=force-waitdeps 
GFX11 has a lot of complicated data dependency hazards.

For debugging GFX10+ data dependency hazards. This creates an excessive
amount of s_waitcnt_depctr.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18273>
 2022-10-19 02:46:03 +00:00
Rhys Perry
6f368431fd aco/gfx11: workaround VALUMaskWriteHazard 
fossil-db (gfx1100):
Totals from 62812 (46.52% of 135032) affected shaders:
Instrs: 43971580 -> 44069887 (+0.22%)
CodeSize: 233473420 -> 233866648 (+0.17%)
Latency: 463487489 -> 463520688 (+0.01%); split: -0.00%, +0.01%
InvThroughput: 86505748 -> 86509679 (+0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18273>
 2022-10-19 02:46:03 +00:00
Rhys Perry
5806f0246f aco/gfx11: workaround VALUPartialForwardingHazard 
fossil-db (gfx1100):
Totals from 18121 (13.42% of 135032) affected shaders:
Instrs: 28272367 -> 28323468 (+0.18%)
CodeSize: 152581916 -> 152786320 (+0.13%)
Latency: 366477785 -> 366482318 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 74475188 -> 74475758 (+0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18273>
 2022-10-19 02:46:03 +00:00
Rhys Perry
b0b48b7687 aco/gfx11: workaround VALUTransUseHazard 
fossil-db (gfx1100):
Totals from 116990 (86.64% of 135032) affected shaders:
Instrs: 67942325 -> 69493991 (+2.28%)
CodeSize: 366448984 -> 372655648 (+1.69%)
Latency: 673236871 -> 673269808 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 128266905 -> 128270175 (+0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18273>
 2022-10-19 02:46:03 +00:00
Rhys Perry
98ee3e1468 aco/gfx11: workaround LdsDirectVMEMHazard 
fossil-db (gfx1100):
Totals from 27217 (20.16% of 135032) affected shaders:
Instrs: 18010853 -> 18047277 (+0.20%)
CodeSize: 99369568 -> 99515264 (+0.15%)
Latency: 207454040 -> 207464932 (+0.01%); split: -0.00%, +0.01%
InvThroughput: 39810158 -> 39810628 (+0.00%); split: -0.00%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18273>
 2022-10-19 02:46:03 +00:00
Rhys Perry
296b4d95a3 aco/gfx11: workaround LdsDirectVALUHazard 
fossil-db (gfx1100):
Totals from 57858 (42.85% of 135032) affected shaders:

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18273>
 2022-10-19 02:46:03 +00:00
Rhys Perry
1a9840b341 aco/gfx11: workaround VcmpxPermlaneHazard 
Same as GFX10, but in a separate pass because it's the only hazard that's
shared.

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18273>
 2022-10-19 02:46:03 +00:00
Rhys Perry
484df5b30e aco: add search_backwards helper 
This will be useful for VALUPartialForwardingHazard.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18273>
 2022-10-19 02:46:03 +00:00
Rhys Perry
a1022dbf07 aco: use some helpers in GFX10 hazard workarounds 
No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18270>
 2022-09-30 11:44:38 +00:00
Rhys Perry
a4f6acece1 aco: improve VMEMtoScalarWriteHazard s_waitcnt mitigation 
fossil-db (navi10):
Totals from 171 (0.13% of 134563) affected shaders:
Instrs: 69348 -> 69144 (-0.29%)
CodeSize: 381112 -> 380296 (-0.21%)
Latency: 451534 -> 451387 (-0.03%)
InvThroughput: 88692 -> 88686 (-0.01%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18270>
 2022-09-30 11:44:38 +00:00
Rhys Perry
2bd16256a6 aco: fix VMEMtoScalarWriteHazard s_waitcnt mitigation 
It doesn't make sense for a "s_waitcnt vmcnt(0)" to affect a store or DS
instruction.

LLVM checks for "s_waitcnt vmcnt(0) lgkmcnt(0) expcnt(0)" but ignores
s_waitcnt_vscnt (which I assume is a bug).

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: bcf94bb933 ("aco: properly recognize that s_waitcnt mitigates VMEMtoScalarWriteHazard")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18270>
 2022-09-30 11:44:38 +00:00