Commit graph

188610 commits

Author SHA1 Message Date
Kenneth Graunke
f5473e6edd intel/brw: Don't use inst return value when it isn't needed
We just want to emit an instruction, but we don't need to do anything
further with it, so we don't need to store the resulting inst pointer
anywhere.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28957>
2024-04-29 07:51:45 +00:00
Samuel Pitoiset
dfe5e56671 radv/ci: add more flakes
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28963>
2024-04-29 08:34:45 +02:00
David Heidelberg
42b992cfab turnip: rename tu_queue_submit struct to follow ODR
Signed-off-by: David Heidelberg <david@ixit.cz>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28728>
2024-04-28 20:06:33 -07:00
Konstantin Seurer
ea863c0c1c nir/print: Do not access invalid indices of load_uniform
load_uniform does not have io_semantics and component.

Fixes: a83fd26 ("nir/print: stop trying to match i/o vars using base/driver_location")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28962>
2024-04-28 16:06:46 +02:00
Karol Herbst
cc9141f044 rust/program: remove Program::kernels
This was a terrible method as it cloned the entire list on each call.
Instead consumers should just take the lock and operate on a slice instead
to lower CPU overhead.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28872>
2024-04-27 18:19:53 +00:00
Karol Herbst
d8ed73b5f6 rusticl/program: Arc the stored KernelInfo
This way we don't have to constantly copy the full thing at kernel
creation time lowering CPU overhead significantly.

With the previous changes clCreateKernel is basically for free.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28872>
2024-04-27 18:19:53 +00:00
Karol Herbst
672de78d66 core/kernel: skip validating unique kernel signatures
We do not support it at runtime anyway and assert on them to be unique
across devices at build time. This significantly reduces overhead of
clCreateKernel as this is something applications actually rely on being
fast.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28872>
2024-04-27 18:19:52 +00:00
Georg Lehmann
6ab4b2d7a0 spirv: preserve signed zero in modf
fsign's result can be +0.0 or -0.0 for -0.0. We already calculate
the signed zero, it's even faster to replace the fmul(fsign(x), ...) with ior.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28938>
2024-04-26 21:31:53 +00:00
Philipp Zabel
c2053c5363 etnaviv: Allow collecing both GPU and NPU specs
If the primary core is a GPU, but a separate NPU exists, collect
NPU specs in addition to GPU specs.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28921>
2024-04-26 19:30:08 +00:00
Philipp Zabel
a4653587cc etnaviv: Add a separate NPU pipe
Add a separate pipe for the NPU device when the primary device is a GPU.
In case of compute-only contexts, prefer to use the separate NPU pipe.

This allows to create a compute-only context that uses the NPU pipe on
a screen that has a 3D GPU as primary device.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28921>
2024-04-26 19:30:08 +00:00
Philipp Zabel
108d2103ea etnaviv: Pass npu to etna_screen_create in a separate parameter
Allow to pass both gpu and npu to etna_screen_create() separately,
in preparetion for devices with both 3D GPU and NPU.
Iterate over all cores or until both GPU and NPU are found.

If no 3D GPU was found, screen->gpu will be set to the npu as well,
so nothing changes for NPU-only devices.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28921>
2024-04-26 19:30:08 +00:00
Philipp Zabel
06683288e0 etnaviv: drm: Stop after model query failure
Calling etna_gpu_new() with a nonexisting core can happen when iterating
all cores. Bail immediately if querying the model failed, there is no
use in also failing to query the revision.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28921>
2024-04-26 19:30:08 +00:00
Philipp Zabel
ba59882212 etnaviv: drm: Suppress get-param error message for non-existent core
The -ENXIO return value isn't necessarily an error condition.
When iterating over cores, this signals that there are no more
cores to be found.

Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28921>
2024-04-26 19:30:08 +00:00
Yiwei Zhang
4ec84adbed venus: fix to destroy all pipeline handles on early error paths
For early error returns, all pipeline handles have to be destroyed.
Otherwise the caller will treat those valid handles as successfully
created pipeline objects.

Cc: mesa-stable
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28944>
2024-04-26 17:35:29 +00:00
Michel Dänzer
c3be21f177 wsi/wayland: Dispatch event queue in wsi_wl_swapchain_queue_present
With explicit sync, only if it wasn't done earlier for FIFO.

Prevents potentially unbounded memory usage for (wl_buffer.release
events in) the queue, since we don't dispatch the queue anywhere else
with explicit sync.

v2:
* Use wl_display_dispatch_queue_pending instead of
  wl_display_dispatch_queue_timeout. (Sebastian Wick)
* Call it from wsi_wl_swapchain_queue_present instead of
  wsi_wl_swapchain_acquire_next_image_explicit. (Joshua Ashton)

Fixes: 5f7a5a27ef ("wsi: Implement linux-drm-syncobj-v1")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28874>
2024-04-26 15:11:00 +00:00
Matt Turner
2a417e3fc1 intel: Build float64 shader only for Vulkan
It's only used by anv and it requires glslang, which isn't otherwise
required for building iris.

Fixes: b52e25d3a8 ("anv: rewrite internal shaders using OpenCL")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28943>
2024-04-26 14:08:32 +00:00
Eric Engestrom
bdbcba5269 v3dv/ci: add rpi5 failure
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28950>
2024-04-26 13:47:37 +00:00
Connor Abbott
b4874aa5cf ir3: Use scalar ALU instructions when possible
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:14 +00:00
Connor Abbott
32308fe9f1 ir3/nir: Fix imadsh_mix16 definition
The constant-folding definition and comments say that it takes the high
16 bits of the first source and low 16 bits of the second source, but
actually it's the opposite. The algebraic optimization, which actually
happens and needs to be correct, was correct but the comment above it
was wrong.

Note that in the way we use it when lowering multiplications, the
ordering doesn't matter.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:14 +00:00
Connor Abbott
17cb1c78bd ir3: Directly use shared registers when possible
Avoid unnecessary copies.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:14 +00:00
Connor Abbott
3bec9e684d ir3: Rewrite shared reg handling when translating from NIR
In the future we will have many ALU instructions passing shared
registers to each other, and surrounding them each with moves to/from
shared registers will severely bloat the IR size coming out of NIR and
make more pointless work for copy propagation. Instead, do something
more like the ACO approach and allow values stored in the hash table to
be shared, and move the burden of emitting a mov to non-shared to
ir3_get_src(). We will then use ir3_get_src_shared() or
ir3_get_src_maybe_shared() as appropriate in cases where we can handle
shared registers or where we can handle both.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:14 +00:00
Connor Abbott
4828942d0c ir3: Get sources before emitting scan_clusters.macro
We will emit conversion move when getting sources and shared-ness
doesn't match, so it needs to be before emitting the instruction.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:14 +00:00
Connor Abbott
ce6c4f0320 ir3: Add scalar ALU-specific passes
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
4c4234501f ir3: Support scalar ALU in the builder
Propagate shared-ness to the the source, and when creating an ALU
instruction with all scalar sources and the instruction is supported on
the scalar ALU, automatically make the destination scalar. This includes
MOV/COV.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
823e034db2 ir3: Make type_flags() return a bitmask enum
So that it can further be operated on.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
ac132b3f62 ir3: Create reduce identity directly
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
497fcd26b5 ir3: Add builder support for shared immediates
In addition to replacing existing no-longer-needed usage of the
readfirst macro, we will use this for other NIR ALU instructions that
need to materialize constants when they use the shared ALU.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
736570b74d ir3: Add support for ldc.u
This will be important for using shared registers as much as possible.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
94c1ff415b ir3: Distinguish lowered shared->normal moves
When we use the scalar ALU we will start inserting moves with different
API-level semantics from readInvocation() or readFirstInvocation(). We
need to distinguish between these moves and lowered readInvocation()
moves, to avoid unnecessarily keeping helper invocations alive when
inserting (eq).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
a64dd98e55 ir3/cf: Don't fold shared conversions
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
0f62203edf ir3/cp: Support swapping mad srcs for shared regs
This was missed when adding shared reg constraints, and helps with
eliminating extra moves caused by uniform (but not const) mad sources.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
ef75ea18cd ir3: Don't emit single-source collects
This will help us propagate shared-ness through to stc when it has a
single component.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
4ffef73bf5 ir3: Immediate source for stc is invalid
For some reason some CTS tests are hitting this after removing the
single-source collect.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
37748bbe78 ir3: Validate scalar ALU sources
This is a HW restriction that leads to a hang if violated.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
06cf178ede ir3: Implement source restrictions for shared ALU
cat1-cat4 instructions executed on the shared ALU can use shared
registers in an unlimited capacity, as opposed to the vector ALU which
apparently treats shared registers and consts the same. However they
cannot use "normal" sources (which must be "uniformized" via a mov).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
876c5396a7 ir3: Add support for "scalar ALU"
On a650 and later, there is a "scalar ALU" capable of executing cat2
instructions, a subset of cat3 instructions (csel but *not* mad), and
cat4 instructions. There is also another copy of the scalar ALU embedded
in HLSQ, which is responsible for executing preambles with the "early
preamble" bit set. The two new features are closely intertwined, because
the scalar ALU makes it possible to make most preambles only use shared
registers, letting us optimistically use shared registers and only fall
back to normal preambles if we ran out of shared registers. But the
scalar ALU is also generally useful for moving calculations of uniform
values like loop indices to the scalar ALU to reduce normal register
pressure and increase parallelism, because like SFU/EFU and texture
units different waves can execute ALU and scalar ALU instructions in
parallel.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
f8ac16b4b9 ir3: Use correct category for OPC_PUSH_CONSTS_LOAD_MACRO
This prevents is_scalar_alu() from crashing, and seems like the right
thing to do to prevent other "surprises." Make it have the category of
the instructions it expands to.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
507b51e7ae ir3/legalize: any/all/getone are non-prefetch helper users
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
4ac1b13f1a ir3/legalize: Remove bad (eq) micro-optimization
At some point I added extra handling for shpe to the initial loop
calculating which blocks need helpers, but forgot to remove the break
above.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
a56de0774b ir3/legalize: Take (ss) into account in WaR hazards
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
ae2db62aab ir3: Moves with shared destination are always legal
I got this wrong before because I missed the need for (ss), once that
was fixed then a move from anything to a shared register is legal,
include non-shared registers, as long as all active channels have the
same value.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
4ee0f6d1fb ir3: Allow propagation of normal->shared copies
Copies from normal to shared registers are only allowed architecturally
if all of the active threads have the same value for the normal
register, which means that they can normally be propagated into e.g. ALU
instructions or other copies. However, there are a few instruction types
where this is not (currently) allowed, namely the scan macro where the
source is tied to a shared destination and the collect/split macros
where the lowering doesn't currently allow differently-typed sources and
destinations (although we may want to allow that in the future), so we
need to clean up ir3_valid_flags() to catch that.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
b309418380 ir3: Validate that shared registers are in-bound
This would've caught some bugs with copy lowering.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
468f070a91 ir3: Reset num when creating parallel copies
It may have been overwritten when folding in constants.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
c00e06bc62 ir3: Use INVALID_REG in array store
We now use INVALID_REG to mean that a source or destination does not
have a preassigned register. We ignore this for anything but inputs and
outputs for now, but don't make it look like we're preassigning the
array to r0.x. This also will allow us to assert that preassigned
registers are in the correct range.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
b2cf2dfd78 ir3/ra: Use ra_reg_get_num() for validating num
This is what the rest of ra validation uses, because it returns the
correct thing for arrays (i.e. the base of the array, instead of the
actual register accessed). num is sometimes not set, so it was causing
spurious assertion failures.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
81015b2620 ir3/lower_copies: Fix "inaccessible" half reg lowering with shared regs
With shared phi nodes we may start needing this lowering for the same
reason normal parallel copies need it.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
fec5b9397f ir3/lower_copies: Handle HW bug with shared half-floats
In the past we avoided emitting pure 16-bit subgroup macros because of
this bug, but because we're going to start emitting the special moves
they expand to directly, we also have to handle the bug directly.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
100096394f ir3: Don't use swz with shared registers
It seems the two moves it decomposes into aren't always atomically
executed on the scalar ALU, which means that it randomly doesn't work.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
ec036fe51e ir3: Fix shared parallel copy validation
It's legal to have copies from immediate/const to shared, which was
asserting.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00