The Gallium API that Teflon uses now supports a variable number of
inputs per operation.
Adapt to this change without any change in functionality.
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32105>
Operations other than tensor addition will also need to be able to
handle multiple inputs, and a variable number of them.
And for testing individual operations, we also need to support models
with multiple inputs.
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32105>
A few bugs due to uninitialized buffers have cropped up. For now let's
zero them all and see if we want to do something else when we get
concerned about compilation times.
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32104>
The BROADCOM_SAND128 modifier is usually used with an extra parameter
to pass in the stride via a side channel. Quoting from drm_fourcc.h:
> The pitch between the start of each column is set to optimally
> switch between SDRAM banks. This is passed as the number of lines
> of column width in the modifier (we can't use the stride value due
> to various core checks that look at it , so you should set the
> stride to width*cpp).
So apparently this is just a workaround for limitations in some kernel
APIs. DRM modifiers, however, are arguably a bad fit for extra
parameters that aren't known in advance. In the Wayland/KMS ecosystem
many components depend on being able to treat modifiers as opaque, e.g.
for negotiations etc. In practice the current approach requires various
software components to manually use the
`DRM_FORMAT_MOD_BROADCOM_SAND128_COL_HEIGHT()` macro - using the
`DRM_FORMAT_MOD_BROADCOM_SAND128` modifier directly with formats like
`NV12` results in a rejection in the KMS driver and corrupted output
in Mesa (because we'd bail out early in `v3d_sand8_blit()`).
Fortunately the stride check limitations mentioned above don't seem to
apply to Mesa though. Thus we can just add support for the base modifier
and stride (coming from V4L2), allowing various toolkits, Wayland
compositors and V4L2 decoder implementations to support e.g.
`NV12` + `DRM_FORMAT_MOD_BROADCOM_SAND128` (`NC12` in V4L2) in a generic
way.
Notes:
1. Wayland compositors trying to offload composition to KMS will still
fail when doing a test commit.
2. There is another limitation - in the V4L2 MPLANE API - that
requires userspace to know the correct offset of the second plane. That's
a known API limitation though and only affects V4L2 decoder implementations.
Cc: mesa-stable
Signed-off-by: Robert Mader <robert.mader@collabora.com>
Reviewed-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32033>
Currently pointless, Pentium II or Celeron and later has SSE.
Cc: mesa-stable # 24.3
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Signed-off-by: David Heidelberg <david@ixit.cz>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31998>
It's generally useful to use mesa_log for error messages etc. This makes
it easier to forward diagnostics into the right logs etc.
So let's be more consistent about where we're logging things.
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32094>
The V8 hardware supports a faster way of executing depthwise
convolutions, instead of having to fully lower them to regular
convolutions.
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31842>
Because of how depthwise convolutions are implemented on V8, we
sometimes don't need reshuffling the input with strided convolutions.
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31842>
Have had to tweak the code to stay safe on the i.MX8MP.
Also, we are for now being very conservative with tiling to prevent
underruns.
In the future, we may want to consider testing different possibilities
during compilation and choosing the optimal one. Also maybe detecting
underruns by checking whether the NPU hung with a given combination.
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31842>
In V8 the weights and biases of convolution operations are encoded with
a totally different scheme.
The initial reverse engineering and implementation was done by:
Philipp Zabel <p.zabel@pengutronix.de>
Support for zero run length encoding and average bias is not implemented yet.
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31842>
etnaviv_ml.h uses dynarray, but the u_inlines.h header is needed by
some of the files that include it.
Fixes: d6473ce28e ("etnaviv: Use NN cores to accelerate convolutions")
Reviewed-by: Philipp Zabel <p.zabel@pengutronix.de
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31842>
The Early-Z optimization is disabled when there is a discard
instruction in the shader used in the draw call.
But if discard is the only reason to disable Early-Z, and at
draw call time the updates in the draw call are disabled we
can enable Early-Z using a shader variant.
If there are occlussion queries active we also need to disable
Early-z optimization.
So this patch enables Early-Z in this scenario.
The performance improvement is significant when running gfxbench
benchmark showing an average improvement of 11.15%
fps_avg helped: gl_gfxbench_aztec_high.trace: 3.13 -> 3.73 (19.13%)
fps_avg helped: gl_gfxbench_aztec.trace: 4.82 -> 5.68 (17.88%)
fps_avg helped: gl_gfxbench_manhattan31.trace: 5.10 -> 6.00 (17.59%)
fps_avg helped: gl_gfxbench_manhattan.trace: 7.24 -> 8.36 (15.52%)
fps_avg helped: gl_gfxbench_trex.trace: 19.25 -> 20.17 ( 4.81%)
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32028>
Only radeon_surface.h/c is used from libdrm and radeon_drm.h is imported
too. This code doesn't change anymore. We don't need the dependency.
Acked-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Acked-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31827>
AGX load/stores supports a single family of addressing modes:
64-bit base + sign/zero-extend(32-bit) << (format shift + optional shift)
This is a base-index addressing mode, where the index is minimally in elements
(never bytes, unless we're doing 8-bit load/stores). Both base and the resulting
address must be aligned to the format size; the mandatory shift means that
alignment of base is equivalent to alignment of the final address, which is
taken care of by lower_mem_access_bit_size anyhow.
The other key thing to note is that this is a 64-bit shift, after the sign- or
zero-extension of the 32-bit index. That means that AGX does NOT implement
64-bit base + sign/zero-extend(32-bit << shift)
This has sweeping implications.
For addressing math from C-based languages (including OpenCL C), the AGX mode is
more helpful, since we tend to get 64-bit shifts instead of 32-bit shifts.
However, for addressing math coming from GLSL, the AGX mode is rather annoying
since we know UBOs/SSBOs are at most 4GB so nir_lower_io & friends are all
32-bit byte indexing. It's tricky to teach them to do otherwise, and would not
be optimal either since 64-bit adds&shifts are *usually* much more expensive
than 32-bit on AGX *except* for when fused into the load/store.
So we don't want 32-bit NIR, since then we can't use the hardware addressing
mode at all. We also don't want 64-bit NIR, since then we have excessive 64-bit
math resulting from deep deref chains from complex struct/array cases. Instead,
we want a middle ground: 32-bit operations that are guaranteed not to overflow
32-bit and can therefore be losslessly promoted to 64-bit.
We can make that no-overflow guarantee as a consequence of the maximum UBO/SSBO
size, and indeed Mesa relies on this already all over the place. So, in this
series, we use relaxed amul opcodes for addressing
math. Then, we rewrite our address mode pattern matching to fuse AGX address
modes.
The actual pattern matching is rewritten. The old code was brittle handwritten
nir_scalar chasing, based on a faulty model of the hardware (with the 32-bit
shift). We delete it all, it's broken. In the new approach, we add some NIR
pseudo-opcodes for address math (ulea_agx/ilea_agx) which we pattern match with
NIR algebraic rules. Then the chasing required to fuse LEA's into load/stores is
trivial because we never go deeper than 1 level. After fusing, we then lower the
leftover lea/amul opcodes and let regular nir_opt_algebraic take it from
here.
We do need to be very careful around pass order to make sure things like
load/store vectorization still happen. Some passes are shuffled in this commit
to make this work. We also need to cleanup amul before fusing since we
specifically do not have nir_opt_algebraic do so - the entire point of the
pseudo-opcodes is to make nir_opt_algebraic ignore the opcodes until we've had a
chance to fuse. If we simply used the .nuw bit on iadd/imul, nir_opt_algebraic
would "optimize" things and lose the bit and then we would fail to fuse
addressing modes, which is a much more expensive failure case than anything
nir_opt_algebraic can do for us. I don't know what the "optimal" pass order for
AGX would look like at this point, but what we have here is good enough for now
and is a net positive for shader-db.
That all ends up being much less code and much simpler code, while fixing the
soundness holes in the old code, and also optimizing a significantly richer set
of addressing calculations. Now we don't juts optimize GL/VK modes, but also CL.
This is crucial even for GL/VK performance, since we rely on CL via libagx even
in graphics shaders.
Terraintessellation is up 10% to ~310fps, which is quite nice.
The following stats are for the end of the series together, including this
change + libagx change + the NIR changes building up to this... but not
including the SSBO vectorizer stats or the IC modelling fix. In other words,
these are the stats for "rewriting address mode handling". This is on OpenGL,
and since the old code was targeted at GL, anything that's not a loss is good
enough - we need this for the soundness fix regardless.
total instructions in shared programs: 2751356 -> 2750518 (-0.03%)
instructions in affected programs: 372143 -> 371305 (-0.23%)
helped: 715
HURT: 75
Instructions are helped.
total alu in shared programs: 2279559 -> 2278721 (-0.04%)
alu in affected programs: 304170 -> 303332 (-0.28%)
helped: 715
HURT: 75
Alu are helped.
total fscib in shared programs: 2277843 -> 2277008 (-0.04%)
fscib in affected programs: 304167 -> 303332 (-0.27%)
helped: 715
HURT: 75
Fscib are helped.
total ic in shared programs: 632686 -> 621886 (-1.71%)
ic in affected programs: 113078 -> 102278 (-9.55%)
helped: 1159
HURT: 82
Ic are helped.
total bytes in shared programs: 21489034 -> 21477530 (-0.05%)
bytes in affected programs: 3018456 -> 3006952 (-0.38%)
helped: 751
HURT: 107
Bytes are helped.
total regs in shared programs: 865148 -> 865114 (<.01%)
regs in affected programs: 1603 -> 1569 (-2.12%)
helped: 10
HURT: 9
Inconclusive result (value mean confidence interval includes 0).
total uniforms in shared programs: 2120735 -> 2120792 (<.01%)
uniforms in affected programs: 22752 -> 22809 (0.25%)
helped: 76
HURT: 49
Inconclusive result (value mean confidence interval includes 0).
total threads in shared programs: 27613312 -> 27613504 (<.01%)
threads in affected programs: 1536 -> 1728 (12.50%)
helped: 3
HURT: 0
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31964>
