uint_max instance divisors are used by Deathloop to get stride==0 behavior.
Totals from 36 (0.02% of 202440) affected shaders:
Instrs: 116064 -> 116028 (-0.03%)
CodeSize: 579120 -> 578880 (-0.04%)
Latency: 142081 -> 142054 (-0.02%)
InvThroughput: 34268 -> 34178 (-0.26%)
PreSGPRs: 2264 -> 2272 (+0.35%)
VALU: 66228 -> 66192 (-0.05%)
We could replace (InstanceIndex / uint_max) with 0 if we assumed that
InstanceIndex == uint_max can't practically occur, and promote the VBO
loads to SMEM.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42229>
nir_scalar contains a pointer followed by an unsigned, leaving 4 bytes
of compiler-inserted trailing padding. Copying a nir_scalar via struct
assignment propagates whatever garbage bytes were in the source
temporary's padding, so both XXH32(entry, sizeof(dedup_entry)) and
memcmp(a, b, sizeof(dedup_entry)) could produce wrong results for
semantically identical entries.
Entries are allocated with rzalloc, which zeros all bytes. Preserve that
invariant by assigning nir_scalar fields member-by-member instead of via
struct assignment, keeping the padding bytes zero throughout the entry's
lifetime. XXH32 and memcmp over the full struct are then correct.
Add a static_assert on sizeof(dedup_entry) to catch future layout
changes that would require auditing the assignments.
Fixes: ca137e545c ("nir/opt_varyings: rewrite elimination of duplicated outputs")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42235>
If the loop latch is not the header, then the preheader ends in
a branch in order to jump over the loop latch on the first iteration.
Emit the s_inst_prefetch before the branch.
2415 (1.16% of 208626) affected shaders (Navi31)
Fixes: 102aca9843 ('aco/assembler: emit block_kind_loop_latch before the loop header')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41990>
After doing a bit of hardware testing on v13, it seems that all
shift+lop are unsigned when it comes to widening. By switching the
opcode itself, we'll enable future optimizations where we fold widens
into sources.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42274>
This makes it a bit more useful as it now returns None instead of just
asserting on you if you attempt an invalid conversion. This also
switches it to be basically exactly the same big switch statement as
from_swizzle(), just in the other direction.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42274>
On Xe2+, the SWSB encoding supports combined RegDist + SBID annotations
in more cases than the scoreboard lowering pass was utilizing:
- SBID DST wait can combine with RegDist when ordered_pipe is ALL
(encoded as mode 0b11)
- SBID SRC wait can combine with RegDist when the ordered pipe matches
the instruction's inferred execution pipe (encoded as mode 0b10)
Previously, DST could only be baked when ordered_pipe matched the
inferred pipe exactly, and SRC could never be baked when an ordered
dependency was present. This caused unnecessary sync nop insertions
for the multi-dependency case.
Totals:
Instrs: 635590947 -> 635755058 (+0.03%)
CodeSize: 8898399200 -> 8749325392 (-1.68%)
Cycle count: 74079919465 -> 73911266034 (-0.23%); split: -0.23%, +0.00%
Max dispatch width: 28235024 -> 28233232 (-0.01%); split: +0.01%, -0.02%
Totals from 842018 (69.11% of 1218461) affected shaders:
Instrs: 578157685 -> 578321796 (+0.03%)
CodeSize: 8100966624 -> 7951892816 (-1.84%)
Cycle count: 73446641513 -> 73277988082 (-0.23%); split: -0.23%, +0.00%
Max dispatch width: 20028944 -> 20027152 (-0.01%); split: +0.02%, -0.03%
Signed-off-by: Michael Cheng <michael.cheng@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42205>
These are each very simple and are now only used once.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42175>
It doesn't make sense to use a key from a random intersection shader.
fossil-db (gfx1201):
Totals from 3 (0.00% of 210263) affected shaders:
Instrs: 9728 -> 10024 (+3.04%)
CodeSize: 60140 -> 60012 (-0.21%)
Latency: 95724 -> 95905 (+0.19%)
InvThroughput: 15015 -> 15044 (+0.19%)
VALU: 2985 -> 2997 (+0.40%)
VMEM: 345 -> 429 (+24.35%)
VOPD: 307 -> 323 (+5.21%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42175>
This should be done if the any-hit enables robustness but the intersection
does not.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42175>
The traversal stage key is still a bit nonsense.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42175>
It doesn't really make sense to make this per-mesa_shader_stage. Each
VkPipelineShaderStageCreateInfo can have different flags.
This is just a refactor at the moment. Actually letting them differ within
a mesa_shader_stage is for a later commit.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42175>
This is more appropriate, and can be done now that the function is called
in radv_shader_deserialize().
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42175>
This improves RADV_DEBUG=hang's pipeline.log when shader caching is not
disabled.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42175>
Instead of passing various fields from stage, just pass the entire object.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42175>
NIR printing is done earlier without nir_string, so I don't know why this
was done.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42175>
No need to create these again and pass them around as parameters. These
functions already have plenty of those.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42175>
It points to the heap variable.
This fixes
dEQP-VK.binding_model.descriptor_heap.basic.raygen.acceleration_structure_untyped.
Fixes: 20d11c59a4 ("vulkan: Add a lowering pass for descriptor heap mappings")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42252>
This isn't intended to be used for sparse BOs and it was incorrect
anyways because flags isn't initialized, so it was only clearing the
original VA range, not including the padding. Since sparse is still
experimental on GFX6-7, let's just apply the workaround to non-sparse
BOs.
This fixes sparse support on VEGA10, since addc719ec2
("radv: workaround has_smem_partial_oob_access_bug").
Fixes: 10a5e5e4f3 ("radv/amdgpu: Add ability to pad BOs with a read-only VM page")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42245>
Because free_list is always NULL for REPLAYED arenas, freed blocks were
never passed to add_hole() and freelist.prev was still NULL. So,
adjacent blocks were never merged together and that caused a memleak
with unreachable blocks.
This fixes a memleak detected by ASAN in
dEQP-VK.ray_tracing_pipeline.pipeline_library.configurations.singlethreaded_compilation.s0_l11_check_capture_replay_handles
and similar tests.
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42012>
Add semantic location check, because multiple variables can share same
component location.
Fixes: ea863c0c1c ("nir/print: Do not access invalid indices of load_uniform")
Signed-off-by: Caius Moldovan <caius.moldovan@imgtec.com>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42114>
UNORM16/SNORM16 render targets are backed by 16-bit-integer TLB
formats, which V3D HW cannot blend. The compiler already supports
software blend lowering in NIR, but V3DV only enabled it for dual-src
blending. As a result format_supports_blending refused the BLEND_BIT
for these formats and Dawn could not advertise the WebGPU
Unorm16TextureFormats feature.
Set pipeline->blend.use_software when any color attachment uses a
software-normalised format so the existing NIR blend lowering kicks
in, and expose VK_FORMAT_FEATURE_COLOR_ATTACHMENT_BLEND_BIT for
those formats.
Assisted-by: Claude Opus 4.7
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42176>
Add a Foldable trait similar to what is already used in NAK for software
emulation of opcodes, since Mali has many variations like V4I8 that run
the same exact operation independently on each component of the vector,
this commit also adds a FoldableComp trait that lets the implementor
only focus on a single component and automatically implements Foldable.
We also add tests on OpShiftLop as an initial subject, we'll add most of
the arithmetic opcodes as time goes on to have a tight description of
the hardware.
Signed-off-by: Lorenzo Rossi <lorenzo.rossi@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42189>
Add the generic infrastructure to load/store the test data and compile
the shader, along simple tests that use the hw_runner.
Signed-off-by: Lorenzo Rossi <lorenzo.rossi@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42189>
This is a very small driver that just sends compute jobs to the graphics
card without any of the Vulkan or OpenGL indirections. For now it only
supports v10-v13 since it's what Kraid is targeting. Lots of the
low-level code that handles CSF encoding and descriptor handling is in C
foir semplicity (and because there is no genxml equivalent for rust yet).
device.rs also implements a barebone memory-safe Rust abstraction for
mali GPUs, as a treat.
Signed-off-by: Lorenzo Rossi <lorenzo.rossi@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42189>
We'll need the extra ensurance if we want to share the model across
threads.
Signed-off-by: Lorenzo Rossi <lorenzo.rossi@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/42189>
