Sadly amd only made this behavior controlable with global state.
We add a new pseudo opcode for this purpose and change FP16_OVFL
for each instruction. Ideally we would only do it once for clauses
and after ilp scheduling, but this can be improved in the future.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35434>
This isn't done in validate_cfg() because that's called less frequently.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35202>
We should update linear_preds so that the predecessors we can remove are
actually removed.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35202>
It might be the case that both the branch and exec mask write in a
divergent branch block are removed. try_remove_simple_block() might then
try to remove it, but fail because it has multiple logical successors.
Instead, just skip these blocks.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Backport-to: 25.1
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35202>
For cases when less than 4 components are read, the original code
would compute an incorrect dmask. eg: with a single component + is_sparse,
the dmask was 0x13:
- 0x 3 = coming from nir_def_components_read
- 0x10 = the sparse bit
While it should have at 2 bits set (1 for the color/depth, 1 for tfe).
This caused problem when expand_vector() used the dmask to generate
the final results, because the value for the sparse component was
read from the wrong index.
So after the call to emit_mimg() dmask needs to be adjusted
because the components will be stored in order, so if mask is 0x11
the tfe value would be stored at invalid index=5 (while it should
be at index=1).
This fixes KHR-GL46.sparse_texture_clamp_tests.SparseTextureClampLookupResidency_texture_2d_depth_component16
and KHR-GL46.sparse_texture2_tests.SparseTexture2Lookup_texture_2d_depth_component16
with ACO.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35206>
The ISA docs don't mention this, but instead of always truncating
like other integer conversions, this opcode actually uses the single
precision rounding mode.
We could continue to use the opcode and set the rounding mode to rtz
in lower_to_hw_instrs, but I think I should just concede that f2u8
isn't worth the effort.
Fixes: 9bb10b58 ("aco: use v_cvt_pk_u8_f32 for f2u8")
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35391>
This just removes the if/endif wrapping for LLVM, and hopefully the ACO
change does the same thing.
ACO had redundant code in endif_merged_wave_info, which is removed here.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34780>
If we remove exec skips, then we can wait for an entry on all paths in the
linear cfg, but not the logical cfg.
fossil-db (gfx1201):
Totals from 0 (0.00% of 79653) affected shaders:
fossil-db (navi31):
Totals from 0 (0.00% of 79653) affected shaders:
fossil-db (navi21):
Totals from 1586 (1.99% of 79653) affected shaders:
Instrs: 5118897 -> 5113206 (-0.11%); split: -0.11%, +0.00%
CodeSize: 28365852 -> 28343696 (-0.08%); split: -0.08%, +0.00%
Latency: 47820341 -> 47799532 (-0.04%); split: -0.09%, +0.05%
InvThroughput: 9904391 -> 9908653 (+0.04%); split: -0.02%, +0.06%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34978>
This now includes image_msaa_load and the new atomic instructions in
GFX12.
It also treats point sample accelerated MIMG as either sample or load,
like the waitcnt insertion pass. I'm not sure if that's necessary or not,
though.
No fossil-db changes (gfx1201, gfx1150 and navi31).
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35235>
LLVM commit 62dea99a7d7df9daedbb86133f3d46699cd2728d made this instruction
a sample for all GFX levels, then with f898161bfa95723954a273a519180e070a5ccd2e
it was changed to be GFX12+. Now 34b6285735c999d2fab77b0ff8e5b497d86df3af
changed it to be all GFX levels again.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35235>
Vector-aligned operands are handled by temporarily allocating
a vector-SSA value for the duration of the instruction.
On completion of the register assignment, the individual
operands are assigned to the reserved register space and,
if necessary, parallelcopies are emitted.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
The old version worked by precoloring both the operand and definition to
whatever register the operand was at the time. This didn't allow moving
the operand/definition after precoloring them, or two tied operands with
the same temporary.
The new version works by temporarily making the operands late kill and
creating a copy if the temporary is live-through or used as another tied
operand. Then we can simply later make the operands early kill again and
assign the definitions to the operand's register.
This way, we can move the operand to make space and the new location will
not intersect with any other definition and won't cause the operand and
definition registers to mismatch.
fossil-db (gfx1201):
Totals from 2253 (2.84% of 79377) affected shaders:
Instrs: 1634747 -> 1630799 (-0.24%); split: -0.27%, +0.03%
CodeSize: 8688148 -> 8672348 (-0.18%); split: -0.20%, +0.02%
VGPRs: 106500 -> 106512 (+0.01%)
Latency: 11385480 -> 11382965 (-0.02%); split: -0.04%, +0.01%
InvThroughput: 1754430 -> 1754326 (-0.01%); split: -0.01%, +0.00%
SClause: 38954 -> 38964 (+0.03%); split: -0.01%, +0.04%
Copies: 110772 -> 110800 (+0.03%); split: -0.02%, +0.04%
Branches: 29093 -> 29092 (-0.00%)
VALU: 902011 -> 902008 (-0.00%)
SALU: 260175 -> 260203 (+0.01%); split: -0.01%, +0.02%
fossil-db (navi31):
Totals from 2 (0.00% of 79377) affected shaders:
Latency: 1766 -> 1765 (-0.06%)
InvThroughput: 3219 -> 3215 (-0.12%)
fossil-db (navi21):
Totals from 14 (0.02% of 79377) affected shaders:
(no affected stats)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34700>
