Commit graph

90 commits

Author SHA1 Message Date
Georg Lehmann
22dc06798b aco/optimizer: never unfuse fma
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
This shouldn't change anything in practice, and reducing precision
if precise isn't set is weird.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38183>
2025-11-04 07:54:02 +00:00
Georg Lehmann
a17afd5edd aco/tests: add some simple fp64 modifier tests
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38011>
2025-10-29 17:57:53 +00:00
Georg Lehmann
859505d95a aco/optimizer: use new helpers to apply literals
Foz-DB Navi21:
Totals from 21009 (26.33% of 79789) affected shaders:
MaxWaves: 495342 -> 495414 (+0.01%)
Instrs: 22345587 -> 22335371 (-0.05%); split: -0.05%, +0.00%
CodeSize: 122095820 -> 121795112 (-0.25%); split: -0.25%, +0.00%
VGPRs: 1025800 -> 1025480 (-0.03%)
Latency: 202876235 -> 203076272 (+0.10%); split: -0.04%, +0.14%
InvThroughput: 47599930 -> 47596113 (-0.01%); split: -0.03%, +0.02%
VClause: 475271 -> 475439 (+0.04%); split: -0.02%, +0.05%
SClause: 700679 -> 700629 (-0.01%); split: -0.01%, +0.01%
Copies: 1628498 -> 1618165 (-0.63%); split: -0.64%, +0.01%
Branches: 567199 -> 567216 (+0.00%); split: -0.00%, +0.00%
PreSGPRs: 952134 -> 952043 (-0.01%); split: -0.01%, +0.00%
PreVGPRs: 846614 -> 846272 (-0.04%)
VALU: 15572374 -> 15564050 (-0.05%); split: -0.05%, +0.00%
SALU: 2423329 -> 2421319 (-0.08%); split: -0.08%, +0.00%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>

Foz-DB Navi31:
Totals from 13049 (16.44% of 79395) affected shaders:

MaxWaves: 357242 -> 357268 (+0.01%)
Instrs: 19955572 -> 19944106 (-0.06%); split: -0.06%, +0.00%
CodeSize: 105689464 -> 105454348 (-0.22%); split: -0.23%, +0.00%
VGPRs: 765744 -> 764952 (-0.10%); split: -0.11%, +0.00%
Latency: 179063640 -> 179141591 (+0.04%); split: -0.02%, +0.07%
InvThroughput: 27978134 -> 27971318 (-0.02%); split: -0.03%, +0.01%
VClause: 386791 -> 386826 (+0.01%); split: -0.02%, +0.03%
SClause: 598113 -> 598106 (-0.00%); split: -0.01%, +0.01%
Copies: 1393111 -> 1383102 (-0.72%); split: -0.73%, +0.01%
Branches: 498533 -> 498535 (+0.00%); split: -0.00%, +0.00%
PreSGPRs: 573310 -> 573236 (-0.01%); split: -0.01%, +0.00%
PreVGPRs: 591459 -> 591043 (-0.07%)
VALU: 11623734 -> 11615755 (-0.07%); split: -0.07%, +0.00%
SALU: 1962055 -> 1960005 (-0.10%); split: -0.11%, +0.00%
VOPD: 3544 -> 3566 (+0.62%); split: +0.73%, -0.11%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35272>
2025-10-14 08:33:39 +00:00
Georg Lehmann
0d8219f367 aco/tests: allow even more literals
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35272>
2025-10-14 08:33:37 +00:00
Georg Lehmann
8903bb4618 aco/optimizer: don't apply packed clamp to v_fma_mix
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13758
Fixes: 345bf8a2f2 ("aco/optimizer: remove label_vop3p")
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36963>
2025-08-25 16:47:38 +00:00
Daniel Schürmann
caa2c22d8b aco/tests: Fix p_startpgm definitions to registers
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36345>
2025-08-01 17:15:54 +00:00
Samuel Pitoiset
f46830912e aco: do not apply OMOD/CLAMP for pseudo scalar trans instrs
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
This optimization seems broken because eg. v_s_log_f32 uses SGPRs
for both the source and destination but applying OMOD seems to require
VGPRs.

This fixes a GPU hang when launching Enshrouded on GFX1201.

No fossils db changes on GFX1201.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34027>
2025-03-13 11:22:10 +00:00
Georg Lehmann
7eb43c3b1c aco/optimizer: delete combine_and_subbrev
This is now done in NIR. No Foz-DB changes on Navi21.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33761>
2025-03-01 07:49:28 +00:00
Rhys Perry
93372ea9af aco: do not use inline constants for 16-bit pseudo scalar trancendentals
Like https://github.com/llvm/llvm-project/pull/104395

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30729>
2024-09-27 11:11:42 +00:00
Georg Lehmann
b20d793774 aco/tests: fix omod signed zero tests
If we keep denorms, omod won't be used no matter what.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31172>
2024-09-18 20:46:17 +00:00
Georg Lehmann
27cf11dc8a aco: remove per block inf/nan/sz control
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31172>
2024-09-18 20:46:17 +00:00
Georg Lehmann
5c3ea47793 aco/tests: use new float control flags
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31172>
2024-09-18 20:46:17 +00:00
Georg Lehmann
246e22ff4f aco/tests: do not use mul with constant to tests neg modifier
The neg can be moved to the constant operand, which defeats the point
of the test.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30781>
2024-08-27 20:41:10 +00:00
Georg Lehmann
bf67ac30fe aco/tests: allow literals with resolved swizzles in vop3p test
My new optimizer code will resolve swizzles for constants.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30781>
2024-08-27 20:41:09 +00:00
Georg Lehmann
6a18eb6afc aco/tests: parse neg(constant) in vop3p test
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30781>
2024-08-27 20:41:09 +00:00
Georg Lehmann
52465956ca aco/print_ir: use neg() for constants
Otherwise, it's not clear if -1 is 0xffffffff or 0x80000001.
LLVM uses a similar logic.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30781>
2024-08-27 20:41:09 +00:00
Georg Lehmann
fb8e730d9b aco/tests: do not use add to tests neg modifer
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30781>
2024-08-27 20:41:09 +00:00
Georg Lehmann
f71522e5cf aco/tests: don't test dpp constant propagation with row shift
With bc=1, removing DPP for shifts is invalid.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30781>
2024-08-27 20:41:09 +00:00
Rhys Perry
db39685e31 aco: validate temporary reachability
These errors are nicer to read than the ones from live variable analysis.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30440>
2024-08-21 10:47:20 +00:00
Rhys Perry
ca161a96d1 aco: combine extracts into s_pack_ll_b32_b16
fossil-db (navi21):
Totals from 3 (0.00% of 79395) affected shaders:
Instrs: 45941 -> 45924 (-0.04%)
CodeSize: 241768 -> 241756 (-0.00%)
Latency: 176501 -> 176491 (-0.01%)
Copies: 6884 -> 6882 (-0.03%)
SALU: 6101 -> 6088 (-0.21%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29912>
2024-07-01 17:34:21 +00:00
Georg Lehmann
2225a32bb0 aco: remove ordered/unordered optimizations
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29467>
2024-06-27 08:12:30 +00:00
Georg Lehmann
be7c137229 aco/gfx11+: optimize v_fma_mix throughput
Foz-DB Navi31:
Totals from 18677 (23.58% of 79206) affected shaders:
Latency: 83613889 -> 83558801 (-0.07%)
InvThroughput: 12696661 -> 12635199 (-0.48%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29047>
2024-05-08 19:36:07 +00:00
Samuel Pitoiset
7a69d78ba2 aco: use SPDX-License-Identifier
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622>
2024-04-08 15:49:25 +00:00
Georg Lehmann
9acd9c0100 aco/tests: use correct operand size for some 64bit ops
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26163>
2023-11-15 12:35:32 +00:00
Georg Lehmann
6a136b4e05 aco/tests: add some missing scc defs
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26163>
2023-11-15 12:35:32 +00:00
Georg Lehmann
7b4f0e714c aco/gfx11: support vinterp as fma_mix
Totals from 718 (0.94% of 76572) affected shaders:
Instrs: 657897 -> 654219 (-0.56%)
CodeSize: 3471668 -> 3457352 (-0.41%); split: -0.41%, +0.00%
VGPRs: 34200 -> 34164 (-0.11%)
Latency: 11687698 -> 11677030 (-0.09%); split: -0.10%, +0.00%
InvThroughput: 1455371 -> 1451537 (-0.26%); split: -0.26%, +0.00%
VClause: 7598 -> 7600 (+0.03%)
SClause: 18293 -> 18241 (-0.28%); split: -0.44%, +0.15%
Copies: 34641 -> 34644 (+0.01%); split: -0.05%, +0.06%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25220>
2023-10-05 20:02:53 +00:00
Georg Lehmann
5e9fad48bf aco/gfx11: apply clamp/omod to vinterp
Totals from 2504 (3.27% of 76572) affected shaders:
MaxWaves: 74098 -> 74106 (+0.01%)
Instrs: 1829278 -> 1823427 (-0.32%); split: -0.32%, +0.00%
CodeSize: 9775908 -> 9759308 (-0.17%); split: -0.18%, +0.01%
Latency: 13494107 -> 13485390 (-0.06%); split: -0.10%, +0.04%
InvThroughput: 2052428 -> 2048724 (-0.18%); split: -0.18%, +0.00%
VClause: 26637 -> 26640 (+0.01%); split: -0.04%, +0.05%
SClause: 62027 -> 61988 (-0.06%); split: -0.14%, +0.08%
Copies: 73776 -> 73815 (+0.05%); split: -0.07%, +0.12%
PreVGPRs: 84403 -> 84397 (-0.01%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25220>
2023-10-05 20:02:53 +00:00
Rhys Perry
0e79f76aa5 aco: add fetch_inactive field to DPP instructions
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25525>
2023-10-04 18:53:43 +00:00
Eric Engestrom
6b21653ab4 aco: reformat according to its .clang-format
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23253>
2023-06-16 19:59:52 +00:00
Georg Lehmann
dfb6d3e443 aco: use v_fma_mix for f2f32 and f2f16 on gfx11 if wave64
v_fma_mix can be dual issued, trade some code size for throughput.

Foz-DB GFX1100:
Totals from 8204 (6.08% of 134864) affected shaders:
CodeSize: 89608584 -> 89693968 (+0.10%)
Latency: 160744811 -> 160699309 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 19737977 -> 19678308 (-0.30%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21402>
2023-06-07 12:30:11 +00:00
Georg Lehmann
177dba62a1 aco: use v_add_f{16,32} with clamp for fsat
v_add can be dual issued on gfx11, v_med3 cannot.
Don't use v_add directly to still optimize omod(fsat(x)).

Foz-DB GFX1100:
Totals from 32702 (24.24% of 134913) affected shaders:
Latency: 475008203 -> 474928037 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 59226198 -> 59140787 (-0.14%); split: -0.14%, +0.00%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21402>
2023-06-07 12:30:11 +00:00
Rhys Perry
0f60c18f29 aco: don't optimize s_or_b64(v_cmp_u_f32(a, b), cmp(a, a))
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22214>
2023-03-31 19:41:54 +00:00
Georg Lehmann
dae13f3dc1 aco: add tests for neg(mul) with opsel
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>
2023-03-30 03:34:35 +00:00
Georg Lehmann
02b94037f6 aco/tests: run optimize.mad_mix.input_conv.modifiers on gfx11
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>
2023-03-30 03:34:35 +00:00
Georg Lehmann
728146b2fc aco: add test for min/max combining with opsel
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>
2023-03-30 03:34:35 +00:00
Georg Lehmann
9499f202e8 aco: add tests for cmp ordering with opsel
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>
2023-03-30 03:34:35 +00:00
Georg Lehmann
22903bcded aco: add tests for swap operand with opsel
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>
2023-03-30 03:34:35 +00:00
Georg Lehmann
0b29dc5c06 aco: add tests for dpp with opsel
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>
2023-03-30 03:34:34 +00:00
Georg Lehmann
f7bb794dda aco: add tests for fma with opsel
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>
2023-03-30 03:34:34 +00:00
Georg Lehmann
097a97cc42 aco: remove VOP[123C]P? structs
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21023>
2023-03-07 11:53:23 +00:00
Rhys Perry
736d6643bb aco/tests: add tests for v_fma_f32 with 2 fp16 literals
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21633>
2023-03-03 14:20:55 +00:00
Georg Lehmann
ed349951cb aco: mark mad definition as precise if the mul/add were precise
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20587>
2023-03-02 10:59:05 +00:00
Rhys Perry
ab3184c0a2 aco: don't apply modifiers through DPP to unsupported instructions
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21201>
2023-02-21 14:59:38 +00:00
Rhys Perry
c3dd1931d9 aco: allow Builder::Result to be dereferenced
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20251>
2023-01-10 16:01:38 +00:00
Rhys Perry
917cfd587c aco: use v_minmax/v_maxmin opcodes
fossil-db (gfx1100):
Totals from 29868 (22.12% of 135032) affected shaders:
MaxWaves: 741336 -> 741344 (+0.00%)
Instrs: 34624902 -> 34539766 (-0.25%); split: -0.25%, +0.00%
CodeSize: 187196804 -> 187192100 (-0.00%); split: -0.01%, +0.01%
VGPRs: 1816860 -> 1816788 (-0.00%); split: -0.01%, +0.01%
Latency: 502597202 -> 502245627 (-0.07%); split: -0.08%, +0.01%
InvThroughput: 84813176 -> 84586122 (-0.27%); split: -0.28%, +0.01%
VClause: 633826 -> 633749 (-0.01%); split: -0.02%, +0.01%
SClause: 1317738 -> 1317047 (-0.05%); split: -0.06%, +0.01%
Copies: 2130610 -> 2130954 (+0.02%); split: -0.03%, +0.05%
Branches: 766093 -> 765969 (-0.02%); split: -0.02%, +0.00%
PreSGPRs: 1630250 -> 1630034 (-0.01%); split: -0.02%, +0.00%
PreVGPRs: 1590777 -> 1590664 (-0.01%); split: -0.01%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19933>
2022-12-01 21:43:28 +00:00
Rhys Perry
dfbc8e0192 aco: change order in combine_minmax()
Prepare for future optimizations.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19933>
2022-12-01 21:43:28 +00:00
Rhys Perry
d55c4180d5 aco/tests: add vop3p constant combine tests
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16296>
2022-07-05 16:39:56 +00:00
Rhys Perry
5d8f5615d0 aco: ignore precise flag when optimizing integer clamps
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16296>
2022-07-05 16:39:56 +00:00
Marek Olšák
39800f0fa3 amd: change chip_class naming to "enum amd_gfx_level gfx_level"
This aligns the naming with PAL.

Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pellou-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16469>
2022-05-13 14:56:22 -04:00
Daniel Schürmann
2fe005a3fe aco: remove occurences of VCC hint
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15408>
2022-04-13 21:52:43 +00:00