Previously, every wave had multiple active lanes read the LDS, and
the data was processed by VALU DPP instructions.
Now, only the first lane reads the LDS in order to avoid bank
conflicts, and the results are processed by SALU.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10155>
The previous pass was based on misconceptions and
rounded up with bug fixes. The new pass is entirely
rewritten and basically just one-to-one from the paper:
"Revisiting Out-of-SSA Translation for Correctness, CodeQuality, and Efficiency"
by B. Boissinot et al.
It also incorporates the value-equality testing.
The regressions are mainly due to creating parallelcopies for
exec phis at loop headers (mitigated in the next commit).
Totals from 4933 (3.61% of 136546) affected shaders (Raven):
SpillSGPRs: 16249 -> 16527 (+1.71%); split: -0.28%, +1.99%
SpillVGPRs: 1771 -> 1595 (-9.94%)
CodeSize: 57544436 -> 58280304 (+1.28%); split: -0.00%, +1.28%
Scratch: 176128 -> 179200 (+1.74%)
Instrs: 11265783 -> 11445884 (+1.60%); split: -0.00%, +1.60%
Latency: 552596156 -> 555880540 (+0.59%); split: -0.53%, +1.13%
InvThroughput: 271431862 -> 273097423 (+0.61%); split: -0.18%, +0.79%
VClause: 160240 -> 160241 (+0.00%); split: -0.02%, +0.02%
SClause: 386863 -> 386685 (-0.05%); split: -0.07%, +0.02%
Copies: 1180801 -> 1345633 (+13.96%); split: -0.02%, +13.98%
Branches: 379129 -> 393052 (+3.67%); split: -0.01%, +3.69%
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9196>
To decide which variables to spill, we must use the distances at the
beginning of the loop-header, and not the distances at the end of the
loop-preheader. The difference are that the former includes phis which are
viable to be spilled as opposed to the phi operands which would be reloaded
by add_coupling_code(), ending up in potentially too high register pressure
before the loop.
Totals from 206 (0.15% of 136546) affected shaders (Raven):
SpillSGPRs: 5154 -> 5000 (-2.99%)
CodeSize: 3654072 -> 3647184 (-0.19%); split: -0.19%, +0.00%
Instrs: 701482 -> 700526 (-0.14%); split: -0.14%, +0.00%
Latency: 40988780 -> 40872506 (-0.28%); split: -0.29%, +0.00%
InvThroughput: 20364560 -> 20306006 (-0.29%)
SClause: 20192 -> 20198 (+0.03%)
Copies: 77732 -> 77688 (-0.06%); split: -0.08%, +0.03%
Branches: 24204 -> 24050 (-0.64%)
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9196>
This also fixes some hypothetical issue for loops without phis
and for loops with higher register pressure at the end of the
loop preheader.
No fossil-db changes.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9196>
This handles the case of
exec = p_linear_phi %a, %b
where %a or %b might have been spilled.
By directly reloading these variables into the exec mask register,
we can avoid additional CSSA parallelcopies.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9196>
This shouldn't sign-extend.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10081>
This allows to force the VRS rates via RADV_FORCE_VRS, the supported
values are 2x2, 1x2 and 2x1. This supports the primitive shading rate
mode for non GUI elements.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7794>
The upper 32 bits are truncated before converting, which still produces
correct results since they never meaningfully contribute to the result.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9597>
The previous code produced incorrect results for inputs outside the
range [INT16_MIN, INT16_MAX].
A problematic case is e.g. i2f16 32768, which previously would be
converted to -32768.0 instead of returning the exactly representable
floating point result.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9597>
This function has evolved to be a generic helper function used throughout
the file, so having those assumptions written down explicitly and document
unsupported edge cases should help prevent incorrect use.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9597>
This doesn't change semantics but allows us to reject this potentially
ambiguous configuration in convert_int in a later change.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9597>
Previously, inputs such as 0x100000000 would have their upper 32-bits
ignored despite being representable by 32-bit floats.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9597>
The previous approach attempted to construct phi nodes
on-demand and on-the-fly. Due to several bugs, it became
necessary to always create incomplete phis for all live-in
variables on loop headers, which is highly inefficient.
The new approach assumes that live-in variables on loop-
headers don't get renamed, and afterwards does one renaming
pass per loop nest. This greatly simplifies the code and
reduces the memory footprint.
Totals from 37 (0.03% of 136546) affected shaders (Navi10):
CodeSize: 588148 -> 588020 (-0.02%); split: -0.03%, +0.01%
Instrs: 111793 -> 111761 (-0.03%); split: -0.04%, +0.01%
Latency: 4546013 -> 4545611 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 2806217 -> 2805730 (-0.02%); split: -0.03%, +0.01%
VClause: 2044 -> 2046 (+0.10%)
SClause: 3889 -> 3884 (-0.13%)
Copies: 17730 -> 17700 (-0.17%); split: -0.23%, +0.06%
Branches: 3282 -> 3280 (-0.06%)
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Tony Wasserka <tony.wasserka@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9763>
fossil-db (GFX8):
Totals from 2 (0.00% of 147787) affected shaders:
VMEM: 662 -> 642 (-3.02%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9079>
It used to be that this intrinsic was never created and texture
instructions were always used.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: 50881d59e6 ("compiler/spirv: fix image sample queries")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9686>
