Daniel Schürmann
c452a4d1cc
aco/ra: use round robin register allocation
...
Totals from 74681 (94.06% of 79395) affected shaders: (GFX11)
MaxWaves: 2265668 -> 2263546 (-0.09%); split: +0.01%, -0.10%
Instrs: 44941647 -> 44412809 (-1.18%); split: -1.23%, +0.05%
CodeSize: 234173852 -> 232009132 (-0.92%); split: -0.97%, +0.05%
VGPRs: 3033208 -> 3403000 (+12.19%); split: -0.02%, +12.22%
Latency: 305575738 -> 301100302 (-1.46%); split: -1.70%, +0.23%
InvThroughput: 49366070 -> 49020000 (-0.70%); split: -0.91%, +0.21%
VClause: 875748 -> 854930 (-2.38%); split: -2.65%, +0.27%
SClause: 1369614 -> 1327212 (-3.10%); split: -3.43%, +0.33%
Copies: 2887932 -> 2883061 (-0.17%); split: -1.93%, +1.76%
Branches: 885041 -> 885101 (+0.01%); split: -0.01%, +0.02%
VALU: 25218078 -> 25215170 (-0.01%); split: -0.20%, +0.19%
SALU: 4328640 -> 4326052 (-0.06%); split: -0.20%, +0.14%
VOPD: 9129 -> 9611 (+5.28%); split: +7.48%, -2.20%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29235 >
2024-06-06 21:02:15 +00:00
Georg Lehmann
6ecbda83f8
aco/ra: remove gfx6/7 subdword paths
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28836 >
2024-05-02 11:09:35 +00:00
Georg Lehmann
702f40f415
aco: add ra test for hi v_interp_p2_f16
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28435 >
2024-04-10 07:49:27 +00:00
Samuel Pitoiset
7a69d78ba2
aco: use SPDX-License-Identifier
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622 >
2024-04-08 15:49:25 +00:00
Rhys Perry
8cd3a3a520
aco/tests: add tests for linear VGPR register allocation
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27697 >
2024-03-06 12:55:46 +00:00
Rhys Perry
b7738de4f9
aco/ra: rework linear VGPR allocation
...
We allocate them at the end of the register file and keep them separate
from normal VGPRs. This is for two reasons:
- Because we only ever move linear VGPRs into an empty space or a space
previously occupied by a linear one, we never have to swap a normal VGPR
and a linear one. This simplifies copy lowering.
- As linear VGPR's live ranges only start and end on top-level blocks, we
never have to move a linear VGPR in control flow.
fossil-db (navi31):
Totals from 5493 (6.93% of 79242) affected shaders:
MaxWaves: 150365 -> 150343 (-0.01%)
Instrs: 7974740 -> 7976073 (+0.02%); split: -0.06%, +0.08%
CodeSize: 41296024 -> 41299024 (+0.01%); split: -0.06%, +0.06%
VGPRs: 283192 -> 329560 (+16.37%)
Latency: 64267936 -> 64268414 (+0.00%); split: -0.17%, +0.17%
InvThroughput: 10954037 -> 10951735 (-0.02%); split: -0.09%, +0.07%
VClause: 132792 -> 132956 (+0.12%); split: -0.06%, +0.18%
SClause: 223854 -> 223841 (-0.01%); split: -0.01%, +0.01%
Copies: 559574 -> 561395 (+0.33%); split: -0.24%, +0.56%
Branches: 179630 -> 179636 (+0.00%); split: -0.02%, +0.02%
VALU: 4572683 -> 4574487 (+0.04%); split: -0.03%, +0.07%
SALU: 772076 -> 772111 (+0.00%); split: -0.01%, +0.01%
VOPD: 1095 -> 1099 (+0.37%); split: +0.73%, -0.37%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27697 >
2024-03-06 12:55:46 +00:00
Rhys Perry
07eab4ab40
aco: require linear vgpr uses to be late kill
...
This also removes some linear VGPR tests which will be replaced soon
anyway.
fossil-db (navi31):
Totals from 107 (0.14% of 79242) affected shaders:
Instrs: 66203 -> 66211 (+0.01%); split: -0.09%, +0.10%
CodeSize: 354644 -> 354588 (-0.02%); split: -0.08%, +0.07%
VGPRs: 4476 -> 4452 (-0.54%); split: -0.80%, +0.27%
Latency: 513863 -> 513877 (+0.00%); split: -0.08%, +0.08%
InvThroughput: 68871 -> 68870 (-0.00%); split: -0.02%, +0.02%
SClause: 1589 -> 1590 (+0.06%)
PreVGPRs: 3404 -> 3415 (+0.32%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27697 >
2024-03-06 12:55:45 +00:00
Rhys Perry
d0595e4805
aco/ra: fix GFX9- writelane
...
061b8bfd29 moved handling of fixed operands earlier, but it should have
moved the fixing of writelane operands earlier too.
This fixes Crucible's func.uniform-subgroup.exclusive.imin64 on GFX8.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 061b8bfd29 ("aco/ra: rework fixed operands")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27583 >
2024-02-21 12:40:14 +00:00
Eric Engestrom
6b21653ab4
aco: reformat according to its .clang-format
...
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23253 >
2023-06-16 19:59:52 +00:00
Eric Engestrom
1725d9aa67
radv,aco: tweaks to get clang-format to print nicer code
...
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23253 >
2023-06-16 19:59:52 +00:00
Daniel Schürmann
6fcabb3b02
aco/ra: only rename fixed Operands if the copy-location matches
...
With epilogs, the same Operand can be fixed to multiple registers.
Fixes: 061b8bfd29 ('aco/ra: rework fixed operands')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18485 >
2022-10-24 13:12:34 +00:00
Rhys Perry
7cecc81683
aco/gfx11: fix s_waitcnt printing
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17710 >
2022-09-30 20:57:02 +00:00
Rhys Perry
aadb7aef01
aco: add VINTERP instruction format
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333 >
2022-09-26 14:49:56 +00:00
Rhys Perry
061b8bfd29
aco/ra: rework fixed operands
...
This moves all fixed operands at once, so they don't interfere with one
another.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17493 >
2022-09-01 11:22:46 +00:00
Rhys Perry
efcbccaf0e
aco/ra: handle empty def_reg interval in get_regs_for_copies
...
If def_reg is empty, then def_reg.lo() may be lower than bounds.lo() if
we're moving VGPRs and info.bounds will be invalid.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17493 >
2022-09-01 11:22:46 +00:00
Eric Engestrom
013b022924
aco: drop unused variable
...
Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18193 >
2022-08-22 23:05:20 +00:00
Rhys Perry
c8bde76a42
aco/tests: disable regalloc.subdword_alloc.reuse_16bit_operands on GFX11
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16595 >
2022-05-31 18:07:34 +00:00
Marek Olšák
39800f0fa3
amd: change chip_class naming to "enum amd_gfx_level gfx_level"
...
This aligns the naming with PAL.
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pellou-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16469 >
2022-05-13 14:56:22 -04:00
Daniel Schürmann
70aea6b41a
aco/ra: refactor collect_vars() to return a sorted vector
...
The vector of IDs is sorted with decreasing sizes,
and by increasing assigned registers.
This decouples register assingment from ssa IDs.
Totals from 12694 (9.41% of 134913) affected shaders: (GFX10.3)
VGPRs: 757864 -> 757848 (-0.00%); split: -0.00%, +0.00%
CodeSize: 72350540 -> 72348688 (-0.00%); split: -0.02%, +0.02%
MaxWaves: 237018 -> 237020 (+0.00%); split: +0.00%, -0.00%
Instrs: 13545494 -> 13544699 (-0.01%); split: -0.03%, +0.02%
Latency: 148539203 -> 148533292 (-0.00%); split: -0.01%, +0.00%
InvThroughput: 30319086 -> 30320382 (+0.00%); split: -0.01%, +0.01%
VClause: 326875 -> 327028 (+0.05%); split: -0.05%, +0.09%
SClause: 479833 -> 479837 (+0.00%); split: -0.00%, +0.00%
Copies: 862152 -> 860914 (-0.14%); split: -0.43%, +0.28%
Branches: 317775 -> 317777 (+0.00%)
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11526 >
2022-03-14 08:32:10 +00:00
Rhys Perry
c3070773f8
aco/tests: add test for branch definition RA
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13432 >
2022-03-03 20:21:08 +00:00
Rhys Perry
6ed18749de
aco: allow live-range splits of linear vgprs in top-level blocks
...
Fixes dEQP-VK.ssbo.phys.layout.random.8bit.all_per_block_buffers.46 on
GFX8.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172 >
2021-09-17 14:36:03 +00:00
Rhys Perry
b1e4794f0f
aco/tests: add regalloc.scratch_sgpr.create_vector
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172 >
2021-09-17 14:36:03 +00:00
Rhys Perry
f41200d289
aco/tests: fix finish_ra_test()
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12172 >
2021-09-17 14:36:03 +00:00
Tony Wasserka
66e51dc474
aco: Remove use of deprecated Operand constructors
...
This migration was done with libclang-based automatic tooling, which
performed these replacements:
* Operand(uint8_t) -> Operand::c8
* Operand(uint16_t) -> Operand::c16
* Operand(uint32_t, false) -> Operand::c32
* Operand(uint32_t, bool) -> Operand::c32_or_c64
* Operand(uint64_t) -> Operand::c64
* Operand(0) -> Operand::zero(num_bytes)
Casts that were previously used for constructor selection have automatically
been removed (e.g. Operand((uint16_t)1) -> Operand::c16(1)).
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11653 >
2021-07-13 17:43:26 +00:00
Rhys Perry
6fd6374e27
aco/ra: fix get_reg_for_operand() with vector operands
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10459 >
2021-05-17 13:31:07 +00:00
Rhys Perry
c08bfa110c
aco/ra: fix get_reg_for_operand() when the blocking var is a vector
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10459 >
2021-05-17 13:31:07 +00:00
Rhys Perry
bc95d55e1f
aco/ra: fix get_reg_for_operand() with no free registers
...
fossil-db (Sienna Cichlid):
Totals from 195 (0.13% of 149839) affected shaders:
CodeSize: 2352160 -> 2356720 (+0.19%); split: -0.00%, +0.20%
Instrs: 431976 -> 433124 (+0.27%); split: -0.00%, +0.27%
Latency: 10174434 -> 10174897 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 4044388 -> 4044425 (+0.00%); split: -0.00%, +0.00%
Copies: 67634 -> 68762 (+1.67%); split: -0.00%, +1.67%
fossil-db (Polaris):
Totals from 186 (0.12% of 151365) affected shaders:
CodeSize: 2272356 -> 2276848 (+0.20%); split: -0.00%, +0.20%
Instrs: 432390 -> 433513 (+0.26%); split: -0.00%, +0.26%
Latency: 13153394 -> 13160194 (+0.05%); split: -0.00%, +0.05%
InvThroughput: 10889509 -> 10889967 (+0.00%); split: -0.00%, +0.00%
SClause: 12745 -> 12747 (+0.02%)
Copies: 74832 -> 75945 (+1.49%); split: -0.01%, +1.50%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10459 >
2021-05-17 13:31:07 +00:00
Daniel Schürmann
8b793f9567
aco: remove dead code for the handling of exec temporaries
...
Totals from 26026 (18.67% of 139391) affected shaders (Navi10):
PreSGPRs: 370993 -> 326177 (-12.08%)
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8870 >
2021-02-12 22:41:31 +00:00
Rhys Perry
4c1953a9b8
aco: add test for incorrect convert_to_SDWA() check
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8577 >
2021-01-19 15:38:56 +00:00
Tony Wasserka
b841b4fde8
aco: Add tests for subdword register allocation
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7461 >
2020-12-29 18:57:10 +00:00