We have an SSA register allocator now that should be a lot faster on
most (if not all) of these tests now. Let's re-enable them to gain more
CI coverage.
Acked-by: Valentine Burley <valentine.burley@collabora.com>
Reviewed-by: Eric R. Smith <eric.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36310>
We were not supporting non replicate swizzle and this trigger an
assertion on fossils/parallel-rdp/small_subgroup.foz.
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Fixes: 1481b14fcb ("pan/bi: Lower SWZ.v4i8 to multiple MKVEC.v2i8 on v11+")
Reviewed-by: Olivia Lee <olivia.lee@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36349>
We have a lot of pattern like this on Valhall:
FCMP_OR.f32.lt.m1 r28, ^r28, r27, 0x0
FCMP_OR.f32.lt.m1 r29, r27, r25, 0x0
LSHIFT_AND.i32 r28, ^r28, 0x0.b00, ^r29
That can be simplified into:
FCMP_OR.f32.lt.m1 r29, r27, r25, 0x0
FCMP_AND.f32.lt.m1 r28, ^r28, r27, ^r29
This pass merge those specific cases while setting the appropriate
logical variant of the CMP instruction.
We do not try to merge the srcs that do not originate from a matching CMP
instruction with matching result type as the logical operation is
performed before the result type transformation.
Now this is enough to optimise a lot of common cases anyway so it is
still a win.
Results on fossils/sascha-willems:
Totals:
Instrs: 42157 -> 42059 (-0.23%)
CodeSize: 582784 -> 581760 (-0.18%)
Estimated normalized SFU cycles: 159.9375 -> 153.75 (-3.87%)
Totals from 13 (2.07% of 627) affected shaders:
Instrs: 3490 -> 3392 (-2.81%)
CodeSize: 29696 -> 28672 (-3.45%)
Estimated normalized SFU cycles: 15.8125 -> 9.625 (-39.13%)
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Eric R. Smith <eric.smith@collabora.com>
Reviewed-by: Olivia Lee <olivia.lee@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36327>
Most of it is tracking stuff that is ending synchronously anyways.
For example, in emit_barrier_insert_waits, cs_sync64_wait is sync and
therefore there is no need to defer the timestamp write on any SBs.
Reviewed-by: Lars-Ivar Hesselberg Simonsen <lars-ivar.simonsen@arm.com>
Reviewed-by: Olivia Lee <olivia.lee@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36161>
This is so that in subsequent commits the wait mask can be:
- set by the caller
- not provided at all -> synchronous ts write
Reviewed-by: Lars-Ivar Hesselberg Simonsen <lars-ivar.simonsen@arm.com>
Reviewed-by: Olivia Lee <olivia.lee@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36161>
We are going to need proper variants for the vertex stage to emulate
geometry/tesselation stages.
This switch our codebase to use something similar as Honeykrisp.
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Olivia Lee <olivia.lee@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36131>
We were missing parenthesis on pan_cast_and_pack macros causing various
issues.
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Fixes: 410e5a36ec ("pan/genxml: Rework gen_pack.py to support OpenCL")
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36261>
We were allocating a fixed number of temporary registers; this isn't
always enough, and in fact we should have calculated the number of
temporaries required.
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Fixes: 6c64ad934f ("panfrost: spill registers in SSA form")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36135>
Copy between memory and a depth/stencil image requires copying the depth
and stencil aspects in separate calls. For D32S8, this needs to be
special cased in order to handle (de)interleaving.
For image->image copies, deinterleaving is not supported. Aspects must
match between src and dest for non-planar images.
Signed-off-by: Olivia Lee <olivia.lee@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35910>
This is needed for VK_EXT_host_image_copy which, like the buffer<->image
copy commands, treats depth/stencil like separate image planes and
requires copying each separately.
Signed-off-by: Olivia Lee <olivia.lee@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35910>
We don't need to use fixed-size pixel_t types and put the tiling loop in
a macro in order to get good codegen for this. Replacing the fixed-size
types with memcpy/__builtin_assume_aligned, the compiler is still able
to generate multi-word load/store instructions. Without the fixed-size
types, the only advantage of putting this in a macro is to ensure the
code is specialized on size/is_store/shift, but we can get the same
specialization by making the functions ALWAYS_INLINE.
Measured performance in VK_EXT_host_image_copy benchmraks is unchanged,
and generated assembly looks effectively identical to the previous version.
Signed-off-by: Olivia Lee <olivia.lee@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35910>
Since we don't have a CPU implementation of AFBC compression, host copy
is only implemented for u-interleaved tiling.
Signed-off-by: Olivia Lee <olivia.lee@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35910>
This is needed for VK_EXT_host_image_copy.
Most other mesa drivers use a similar approach to implement tiled->tiled
copy, with a few differences. They use a temp buffer sized for only one
tile, don't attempt to tile-align the copies in either the src or dest,
and they don't have the memcpy fast path. I measured performance of a
variety of implementations on a rock5b, and found:
- The fast path for when the copy region is tile-aligned is a 167%
improvement.
- Aligning the temp buffer chunks to src tiles is a 20% improvement.
- Using a 64k buffer instead of a tile-sized buffer is a 14%
improvement. This buffer size appears optimal in my benchmark,
smaller and larger buffers are both slower. Skipping the chunk
approach and just (de)tiling to a temp buffer that fits the whole
image (what NVK does) is also slower.
- I had no luck with attempts at a direct tiled->tiled copy algorithm
that didn't need a temp buffer. The fastest I got was ~1/4 the speed
of the temp buffer implementation.
Signed-off-by: Olivia Lee <olivia.lee@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35910>
For supporting VK_EXT_host_image_copy for tiled images, we need to be
to determine whether AFBC may be supported in
vkGetPhysicalDeviceImageFormatProperties2.
Signed-off-by: Olivia Lee <olivia.lee@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35910>
Depth/stencil and tiled images require some additional complexity, so
will be implemented in later commits.
Signed-off-by: Olivia Lee <olivia.lee@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35910>
For VK_EXT_host_image_copy, we need to access image memory from the CPU
after mapping the BO. The existing base field in pan_image_plane doesn't
work for this because it's a GPU address and we don't have a mechanism
to recover the GPU base address of an image's BO to calculate the offset.
Signed-off-by: Olivia Lee <olivia.lee@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35910>
Advertising SAMPLED_IMAGE_DEPTH_COMPARISON is a no-op for images that
don't have SAMPLED_IMAGE_BIT, but it's confusing and results in us
advertising a lot of formats that with only the
SAMPLE_IMAGE_DEPTH_COMPARISON feature that aren't usable for anything.
For R32_UINT and R32_SINT, the change is just a cleanup, because we
always support these for storage images.
Whe we implement VK_EXT_host_image_copy, advertising unusable formats
triggers failures in dEQP-VK.api.image_clearing.*, so it's convenient to
have features==0 for all unusable formats.
Fixes: 70b8056df1 ("panvk: Enable KHR_format_feature_flags2 and use them")
Signed-off-by: Olivia Lee <olivia.lee@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35910>
All current callers check for MAP_FAILED, not NULL, and we are returning
MAP_FAILED already on the other error paths.
Fixes: d5f4f918f3 ("panfrost: clean up mmap-diagnostics")
Signed-off-by: Olivia Lee <olivia.lee@collabora.com>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36122>
Fixes: c257bf5142 ("panvk: Conditionally register an host address when tracking user memory")
Signed-off-by: Olivia Lee <olivia.lee@collabora.com>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36122>
Signed-off-by: Loïc Molinari <loic.molinari@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Acked-by: Eric R. Smith <eric.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35001>
This commit proposes an optimized version using Arm A32 NEON
intrinsics.
Signed-off-by: Loïc Molinari <loic.molinari@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Acked-by: Eric R. Smith <eric.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35001>
Payload size retrieval can greatly benefit from using SIMD to sum up
the 16 6-bit packed sizes. This commit proposes an optimized version
using Arm A64 NEON intrinsics. This was measured on a Rock 5B to be ~2
times faster than the original.
Signed-off-by: Loïc Molinari <loic.molinari@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Acked-by: Eric R. Smith <eric.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35001>
The AFBC-P payload layout is currently retrieved in 2 steps starting
with the payload sizes retrieval using a CS job on the GPU followed by
a CPU pass to set the payload offsets. This commit proposes to do both
steps on the CPU at once using a new utility function
pan_afbc_payload_layout_packed().
A new utility function pan_afbc_payload_uncompressed_size() is added
to help retrieve the uncompressed size from a pipe_format and
modifier. Both the CPU and GPU versions use it now.
A new AFBC-P driconf option "pan_afbcp_gpu_payload_sizes" is added to
fallback to the original payload sizes retrieval on the GPU.
Signed-off-by: Loïc Molinari <loic.molinari@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Acked-by: Eric R. Smith <eric.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35001>
Add an AFBC header block structure pan_afbc_headerblock to improve
readability when accessing header blocks. get_superblock_size(), which
will be used for AFBC packing in the next commits, has been moved to
pan_afbc.h and renamed to pan_afbc_payload_size() so that it can be
tested. Other utility functions pan_afbc_header_subblock_size() and
pan_afbc_header_subblock_uncompressed_size() hasve been added to help
retrieve the compressed or uncompressed size of a subblock from a
header. This commit also fixes a few issues like arch handling.
Signed-off-by: Loïc Molinari <loic.molinari@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Acked-by: Eric R. Smith <eric.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35001>
Will be used by etnaviv too.
Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35753>
Bifrost will require more work
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Eric R. Smith <eric.smith@collabora.com>
Signed-off-by: Ashley Smith <ashley.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35789>
Adds support for 64-bit atomic operations for KHR_shader_atomic_int64
using 64-bit atomic instructions. Valhall is working but Bifrost will
require some more work to implement as it requires two instructions to
execute a 64-bit atomic.
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Eric R. Smith <eric.smith@collabora.com>
Signed-off-by: Ashley Smith <ashley.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35789>
Add 64-bit atomic instructions for bifrost/valhall
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Eric R. Smith <eric.smith@collabora.com>
Signed-off-by: Ashley Smith <ashley.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35789>
Fixes: 563823c9ca ("panvk: Implement vk_shader")
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Caterina Shablia <caterina.shablia@collabora.com>
Reviewed-by: Eric R. Smith <eric.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36006>
Previously this was allowing invalid forms like
"CLPER.i32.subgroup8.zero lane-id, src1" to reach bi_pack.
This fixes the assert that can be seen with
"dEQP-VK.glsl.derivate.dfdxsubgroup.*" but doesn't fix failures.
Fixes: 0acc6b564e ("pan/bi: Rework FAU lowering")
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Eric R. Smith <eric.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36006>
Previously we were allowing passthrough to temps without using
bi_reads_temps.
This was causing instructions like CLPER to create undefined encodings.
We now check if the instruction support temps.
Fixes: 4252fb84f4 ("pan/bi: Add passthrough register rewriting helper")
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Eric R. Smith <eric.smith@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36006>
Since commit 8bb46de0, the correct way to check for a compute shader is
with `gl_shader_stage_is_compute()`.
Fixes: d2838f3c ("pan/bi: handle barriers with SUBGROUP scope")
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: John Anthony <john.anthony@arm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35957>
We don't need to depend on the generated wl_drm files, as wl_drm support
was removed from Vulkan quite some time ago.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Fixes: e090316570 ("vulkan/wsi/wayland: drop support for wl_drm")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35885>
