fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2026-05-17 20:28:05 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
208692 commits 66 branches 935 tags 932 MiB
Commit graph

100 commits

Author SHA1 Message Date
Natalie Vock
c279dd6e61 aco: Support vector-aligned ops fixed to defs 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35269>
 2025-07-15 21:34:38 +00:00
Daniel Schürmann
a9645fdd89 aco: introduce concept of vector-aligned Operands 
Operand::isVectorAligned indicates that the Operand is part of a vector
consisting of multiple operands. Therefore, it must reside in a register
aligned with the next Operand.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
 2025-05-28 09:24:16 +00:00
Daniel Schürmann
a4fa3935fd aco/live_var_analysis: set same lateKill flags for same operands 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
 2025-05-28 09:24:16 +00:00
Rhys Perry
cdf3767001 aco: set copy-kill for tied operands with the same temporary 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34700>
 2025-05-20 15:40:47 +00:00
Rhys Perry
c04ef28c56 aco: rename ops_fixed_to_def to tied_defs 
This is less words.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34700>
 2025-05-20 15:40:46 +00:00
Rhys Perry
7fe84024cb aco: fix get_temp_reg_changes with clobbered operands 
The spiller might have tried to spill a live-through first or second
s_fmac_f32 operand, but this wouldn't have reduced the SGPRs if the third
operand wasn't killed

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13038
Fixes: d6cb45dbb0 ("aco/spill: Allow spilling live-through operands")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34699>
 2025-04-28 10:04:43 +00:00
Natalie Vock
f309d76aab aco: Add support for multiple ops fixed to defs 
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
 2025-04-17 20:20:40 +00:00
Natalie Vock
39413ef78f aco: Add get_temp_reg_changes helper 
Similar to get_live_changes, but considers live temporary registers
as well.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29730>
 2025-03-26 19:18:30 +00:00
Daniel Schürmann
52253da783 aco: unify get_addr_sgpr_from_waves() and get_addr_vgpr_from_waves() into one function 
which returns the limit as RegisterDemand.

Also remove the unused get_extra_sgprs() from aco_ir.h.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33644>
 2025-02-21 13:49:41 +00:00
Daniel Schürmann
b64fff7731 aco: remove definition from Pseudo branch instructions 
They are not needed anymore.

Totals from 7019 (8.84% of 79395) affected shaders: (Navi31)

Instrs: 14805400 -> 14824196 (+0.13%); split: -0.00%, +0.13%
CodeSize: 78079972 -> 78132932 (+0.07%); split: -0.01%, +0.08%
SpillSGPRs: 4485 -> 4515 (+0.67%); split: -0.76%, +1.43%
Latency: 165862000 -> 165836134 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 30061764 -> 30057781 (-0.01%); split: -0.01%, +0.00%
SClause: 392323 -> 392286 (-0.01%); split: -0.01%, +0.00%
Copies: 1012262 -> 1012234 (-0.00%); split: -0.04%, +0.04%
Branches: 365910 -> 365909 (-0.00%); split: -0.00%, +0.00%
PreSGPRs: 360167 -> 355363 (-1.33%)
VALU: 8837197 -> 8837276 (+0.00%); split: -0.00%, +0.00%
SALU: 1402593 -> 1402621 (+0.00%); split: -0.03%, +0.03%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32037>
 2024-12-06 14:34:03 +00:00
Timur Kristóf
e2b8c4a9ac radv, aco: Consolidate num_interp + num_prim_interp into num_inputs. 
num_inputs contains the total number of FS inputs.

Note that this also fixes a bug where some calculations in RADV
and ACO were missing the per-primitive attributes from the LDS
usage of PS.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32220>
 2024-11-28 18:14:57 +00:00
Daniel Schürmann
d5581b1124 aco/live_var_analysis: check isFixed() for definitions in order to set needs_vcc 
In rare cases, it could happen that during post-RA validation,
live-var-analysis sets needs_vcc = false after if was true
before register allocation.

Fixes: bb5eace0dc ('aco/live_var_analysis: check for isPrecolored flag rather than isFixed')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31791>
 2024-10-23 09:42:26 +00:00
Daniel Schürmann
bb5eace0dc aco/live_var_analysis: check for isPrecolored flag rather than isFixed 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31387>
 2024-10-22 12:29:18 +00:00
Daniel Schürmann
1499848487 aco/live_var_analysis: don't test whether phis are assigned to VCC 
This check is redundant.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31362>
 2024-10-07 07:00:19 +00:00
Daniel Schürmann
07b1fd64b7 aco/live_var_analysis: don't set lateKill for p_interp_gfx11 m0 operand 
There is no danger of overwriting m0 anymore.

Fixes: dfc13fcf9f ('aco: introduce Operand flag 'CopyKill'')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31127>
 2024-09-11 10:36:34 +00:00
Daniel Schürmann
dfc13fcf9f aco: introduce Operand flag 'CopyKill' 
This flag indicates that the Operand must be copied in order to satisfy register
constraints. The copy is immediately killed by the instruction.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30299>
 2024-09-10 09:44:53 +00:00
Daniel Schürmann
91f65d5935 aco/live_var_analysis: use Clobbered flag to calculate additional operand demand 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30299>
 2024-09-10 09:44:53 +00:00
Daniel Schürmann
5a6fa8a8eb aco: introduce new Operand flag 'Clobbered' 
This flag indicates that the Operand's register gets clobbered by the instruction.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30299>
 2024-09-10 09:44:53 +00:00
Daniel Schürmann
1c14013b9e aco/live_var_analysis: Don't attempt to re-insert the same temporary twice into live set 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30299>
 2024-09-10 09:44:53 +00:00
Rhys Perry
db39685e31 aco: validate temporary reachability 
These errors are nicer to read than the ones from live variable analysis.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30440>
 2024-08-21 10:47:20 +00:00
Daniel Schürmann
5a39cbdef6 aco: change signature of get_live_changes() and get_temp_registers() 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30182>
 2024-08-14 08:11:48 +00:00
Georg Lehmann
dedfff9dbf aco: only set latekill in live_var_analysis 
Cleaner to have this all in one place, in my opinion.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30368>
 2024-08-12 10:31:09 +00:00
Georg Lehmann
510f5e55be aco/gfx10+: set lateKill for sgprs used by wave64 VALU writing a mask 
RDNA2 ISA doc, 6.2.4. Wave64 Destination Restrictions:
The first pass of a wave64 VALU instruction may not overwrite a scalar value
used by the second half.

Foz-DB Navi31:
Totals from 5221 (6.58% of 79395) affected shaders:
Instrs: 9751484 -> 9752179 (+0.01%); split: -0.01%, +0.01%
CodeSize: 50624072 -> 50626088 (+0.00%); split: -0.00%, +0.01%
Latency: 85646450 -> 85647419 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 15039160 -> 15039277 (+0.00%); split: -0.00%, +0.00%
VClause: 200275 -> 200204 (-0.04%)
SClause: 248645 -> 248607 (-0.02%); split: -0.03%, +0.01%
Copies: 640802 -> 641413 (+0.10%); split: -0.01%, +0.11%
PreSGPRs: 236297 -> 236735 (+0.19%)
VALU: 5666449 -> 5666440 (-0.00%)
SALU: 967482 -> 968111 (+0.07%); split: -0.01%, +0.07%

Cc: mesa-stable

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30368>
 2024-08-12 10:31:09 +00:00
Daniel Schürmann
bb5af6bede aco: remove live-out variables from IR 
Since we changed all passes to use the live-in variables,
these are not needed anymore.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
 2024-07-16 14:00:49 +00:00
Daniel Schürmann
29262f8cf3 aco: compute live-in variables in addition to live-out variables 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
 2024-07-16 14:00:49 +00:00
Daniel Schürmann
ad01e473f5 aco/live_var_analysis: use separate allocator for temporary live sets 
This drastically reduces the memory footprint of the live sets.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29962>
 2024-07-10 12:31:02 +00:00
Daniel Schürmann
7c466157d0 aco/live_var_analysis: remove unused includes 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29962>
 2024-07-10 12:31:02 +00:00
Daniel Schürmann
6729e81d15 aco/live_var_analysis: inline block->register_demand updates 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29962>
 2024-07-10 12:31:02 +00:00
Daniel Schürmann
2f4fb9eecf aco/live_var_analysis: ignore phi definition and operand demand at predecessors 
The linear_phi changes are already reflected in the live-in demand
of the successor. The logical_phi_sgpr_ops can only reduce the
register demand at the predecessor. Although this might slightly
overestimate the register-demand, no differences in code quality
were found.

Totals from 1610 (2.03% of 79395) affected shaders: (GFX11)

PreSGPRs: 54002 -> 56954 (+5.47%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29962>
 2024-07-10 12:31:02 +00:00
Daniel Schürmann
68c1e7237c aco/live_var_analysis: refactor using ctx struct 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29962>
 2024-07-10 12:31:02 +00:00
Daniel Schürmann
6c6f382d68 aco: add RegisterDemand member to Instruction 
Since we never need both at the same time, we can use
a union with pass_flags.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29962>
 2024-07-10 12:31:02 +00:00
Daniel Schürmann
634051f913 aco/live_var_analysis: ignore dead phis 
Since we don't emit code for dead phis, we also don't
have to keep their operands around.

Totals from 44 (0.06% of 79395) affected shaders: (GFX11)

MaxWaves: 648 -> 650 (+0.31%)
Instrs: 449898 -> 449120 (-0.17%); split: -0.18%, +0.00%
CodeSize: 2395000 -> 2389300 (-0.24%); split: -0.24%, +0.00%
VGPRs: 5504 -> 5468 (-0.65%)
Latency: 9005058 -> 9000966 (-0.05%); split: -0.07%, +0.03%
InvThroughput: 2154567 -> 2139095 (-0.72%); split: -0.77%, +0.06%
VClause: 8362 -> 8354 (-0.10%)
SClause: 9135 -> 9134 (-0.01%)
Copies: 60678 -> 60118 (-0.92%); split: -0.93%, +0.01%
Branches: 14379 -> 14385 (+0.04%)
PreSGPRs: 3877 -> 3863 (-0.36%)
PreVGPRs: 6318 -> 6286 (-0.51%)
VALU: 266975 -> 266301 (-0.25%); split: -0.25%, +0.00%
SALU: 52741 -> 52667 (-0.14%); split: -0.15%, +0.01%
VMEM: 16140 -> 16132 (-0.05%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:34 +00:00
Daniel Schürmann
708e1a73f5 aco/live_var_analysis: slightly refactor handling of additional register demand for Operand copies 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:34 +00:00
Daniel Schürmann
5cfa5b784b aco: remove get_demand_before() 
The register demand before executing an instruction is now included
in the instruction's register demand and this function is unused.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:34 +00:00
Daniel Schürmann
09f1c40f2e aco: track and use the live-in register demand per basic block 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:34 +00:00
Daniel Schürmann
001c8caae0 aco: calculate register demand per instruction as maximum necessary to execute the instruction 
Previously, the register demand per instruction was calculated as the number of
live variables in the register file after executing an instruction plus additional
temporary registers, necessary during the execution of the instruction.
With this change, now it also includes all variables which are live right before
executing an instruction, i.e. killed Operands.

Care has been taken so that the invariant

register_demand[idx] = register_demand[idx - 1] - get_temp_registers(prev_instr)
                        + get_live_changes(instr) + get_temp_registers(instr)

still holds.

Slight changes in scheduling:

Totals from 316 (0.40% of 79395) affected shaders: (GFX11)

Instrs: 301329 -> 300777 (-0.18%); split: -0.31%, +0.12%
CodeSize: 1577976 -> 1576204 (-0.11%); split: -0.21%, +0.10%
SpillSGPRs: 448 -> 447 (-0.22%)
Latency: 1736349 -> 1726182 (-0.59%); split: -2.01%, +1.42%
InvThroughput: 243894 -> 243883 (-0.00%); split: -0.03%, +0.03%
VClause: 6134 -> 6280 (+2.38%); split: -1.04%, +3.42%
SClause: 6142 -> 6137 (-0.08%); split: -0.13%, +0.05%
Copies: 14037 -> 14032 (-0.04%); split: -0.56%, +0.52%
Branches: 3284 -> 3283 (-0.03%)
VALU: 182750 -> 182718 (-0.02%); split: -0.04%, +0.03%
SALU: 18522 -> 18538 (+0.09%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29804>
 2024-06-26 09:42:33 +00:00
Daniel Schürmann
b7982152ff aco: use aco::monotonic_allocator for IDSet 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29713>
 2024-06-14 14:32:35 +00:00
Daniel Schürmann
a497d105e3 aco: move live var information into struct Program 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29713>
 2024-06-14 14:32:35 +00:00
Samuel Pitoiset
7a69d78ba2 aco: use SPDX-License-Identifier 
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28622>
 2024-04-08 15:49:25 +00:00
Daniel Schürmann
9bbb9f1104 aco: use small_vec as Block::edge_vec for predecessors and successors 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27984>
 2024-03-19 13:06:58 +00:00
Daniel Schürmann
5ebba87772 aco: rename max_wave64_per_simd -> max_waves_per_simd 
and update usage. Changes are because the scheduler targets
a different number of waves.

Totals from 195 (0.25% of 79330) affected shaders: (GFX11)

MaxWaves: 3120 -> 3108 (-0.38%)
Instrs: 71202 -> 71070 (-0.19%); split: -0.27%, +0.09%
CodeSize: 383272 -> 382828 (-0.12%); split: -0.21%, +0.10%
VGPRs: 7392 -> 7752 (+4.87%)
Latency: 2280141 -> 2262487 (-0.77%); split: -0.79%, +0.02%
InvThroughput: 4759022 -> 5725442 (+20.31%); split: -0.01%, +20.32%
VClause: 1737 -> 1741 (+0.23%); split: -3.11%, +3.34%
SClause: 2385 -> 2376 (-0.38%); split: -0.80%, +0.42%
Copies: 5257 -> 5274 (+0.32%); split: -0.25%, +0.57%
Branches: 1213 -> 1212 (-0.08%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26521>
 2023-12-11 10:39:50 +00:00
Rhys Perry
c9b177db0e aco: don't create sendmsg(dealloc_vgprs) if scratch is used 
LLVM does something similar: https://reviews.llvm.org/D153295

fossil-db (gfx1100):
Totals from 21 (0.02% of 133461) affected shaders:
Instrs: 147428 -> 147396 (-0.02%)
CodeSize: 797188 -> 797060 (-0.02%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 2930317cea ("aco/gfx11: deallocate VGPRs at the end of the shader")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24669>
 2023-08-15 11:00:31 +00:00
Rhys Perry
efb80c1715 aco: summarize register demand after handling branches 
Fixes various dEQP-VK.ray_query.builtin.rayqueryterminate.* crashes.

fossil-db (gfx1100):
Totals from 196 (0.15% of 133461) affected shaders:
PreSGPRs: 8342 -> 8558 (+2.59%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 5a536eca9c ("aco: calculate correct register demand for branch instructions")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24579>
 2023-08-14 15:08:37 +00:00
Friedrich Vock
3ea01b86f0 aco: Fix live_var_analysis assert 
Fixes: 3d4f6a00b ('aco/spill: allow for disconnected CFG')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23586>
 2023-06-12 13:52:03 +00:00
Rhys Perry
fab911858c aco: consider how definitions fixed to operands can change register demand 
Usually the register demand before an instruction would be considered part
of the previous instruction, since it's not greater than the register
demand for that previous instruction. Except, it can be greater in the
case of an definition fixed to a non-killed operand: the RA needs to
reserve space between the two instructions for the definition (containing
a copy of the operand).

fossil-db (navi21):
Totals from 5 (0.00% of 135636) affected shaders:
PreVGPRs: 35 -> 40 (+14.29%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8807
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22446>
 2023-05-24 18:58:15 +00:00
Rhys Perry
596f2ef361 aco: set needs_flat_scr=true for RT 
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Fixes: 39c828cb9f ("aco: remove aco::rt_stack variable")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21961>
 2023-03-17 16:55:57 +00:00
Daniel Schürmann
39c828cb9f aco: remove aco::rt_stack variable 
Since we initialize scratch in the RT proglog,
there is no need for this variable anymore.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
 2023-03-16 01:40:30 +00:00
Daniel Schürmann
b338d59047 radv: unconditionally enable scratch for RT shaders 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>
 2023-02-16 19:37:25 +00:00
Rhys Perry
50073d6135 aco/gfx11: increase gfx1100/gfx1101 physical vgprs 
https://reviews.llvm.org/D134522

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18825>
 2022-11-02 17:09:32 +00:00
Daniel Schürmann
f676326a1a aco/live_var_analysis: implement faster merging of live_out sets for some cases 
If we know that logical and linear predecessors are the same,
we don't need to check for the register type of each variable.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18105>
 2022-08-30 16:03:26 +00:00