Commit graph

188529 commits

Author SHA1 Message Date
Connor Abbott
ef75ea18cd ir3: Don't emit single-source collects
This will help us propagate shared-ness through to stc when it has a
single component.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
4ffef73bf5 ir3: Immediate source for stc is invalid
For some reason some CTS tests are hitting this after removing the
single-source collect.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
37748bbe78 ir3: Validate scalar ALU sources
This is a HW restriction that leads to a hang if violated.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
06cf178ede ir3: Implement source restrictions for shared ALU
cat1-cat4 instructions executed on the shared ALU can use shared
registers in an unlimited capacity, as opposed to the vector ALU which
apparently treats shared registers and consts the same. However they
cannot use "normal" sources (which must be "uniformized" via a mov).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
876c5396a7 ir3: Add support for "scalar ALU"
On a650 and later, there is a "scalar ALU" capable of executing cat2
instructions, a subset of cat3 instructions (csel but *not* mad), and
cat4 instructions. There is also another copy of the scalar ALU embedded
in HLSQ, which is responsible for executing preambles with the "early
preamble" bit set. The two new features are closely intertwined, because
the scalar ALU makes it possible to make most preambles only use shared
registers, letting us optimistically use shared registers and only fall
back to normal preambles if we ran out of shared registers. But the
scalar ALU is also generally useful for moving calculations of uniform
values like loop indices to the scalar ALU to reduce normal register
pressure and increase parallelism, because like SFU/EFU and texture
units different waves can execute ALU and scalar ALU instructions in
parallel.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
f8ac16b4b9 ir3: Use correct category for OPC_PUSH_CONSTS_LOAD_MACRO
This prevents is_scalar_alu() from crashing, and seems like the right
thing to do to prevent other "surprises." Make it have the category of
the instructions it expands to.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
507b51e7ae ir3/legalize: any/all/getone are non-prefetch helper users
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
4ac1b13f1a ir3/legalize: Remove bad (eq) micro-optimization
At some point I added extra handling for shpe to the initial loop
calculating which blocks need helpers, but forgot to remove the break
above.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
a56de0774b ir3/legalize: Take (ss) into account in WaR hazards
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
ae2db62aab ir3: Moves with shared destination are always legal
I got this wrong before because I missed the need for (ss), once that
was fixed then a move from anything to a shared register is legal,
include non-shared registers, as long as all active channels have the
same value.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
4ee0f6d1fb ir3: Allow propagation of normal->shared copies
Copies from normal to shared registers are only allowed architecturally
if all of the active threads have the same value for the normal
register, which means that they can normally be propagated into e.g. ALU
instructions or other copies. However, there are a few instruction types
where this is not (currently) allowed, namely the scan macro where the
source is tied to a shared destination and the collect/split macros
where the lowering doesn't currently allow differently-typed sources and
destinations (although we may want to allow that in the future), so we
need to clean up ir3_valid_flags() to catch that.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
b309418380 ir3: Validate that shared registers are in-bound
This would've caught some bugs with copy lowering.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
468f070a91 ir3: Reset num when creating parallel copies
It may have been overwritten when folding in constants.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
c00e06bc62 ir3: Use INVALID_REG in array store
We now use INVALID_REG to mean that a source or destination does not
have a preassigned register. We ignore this for anything but inputs and
outputs for now, but don't make it look like we're preassigning the
array to r0.x. This also will allow us to assert that preassigned
registers are in the correct range.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
b2cf2dfd78 ir3/ra: Use ra_reg_get_num() for validating num
This is what the rest of ra validation uses, because it returns the
correct thing for arrays (i.e. the base of the array, instead of the
actual register accessed). num is sometimes not set, so it was causing
spurious assertion failures.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
81015b2620 ir3/lower_copies: Fix "inaccessible" half reg lowering with shared regs
With shared phi nodes we may start needing this lowering for the same
reason normal parallel copies need it.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
fec5b9397f ir3/lower_copies: Handle HW bug with shared half-floats
In the past we avoided emitting pure 16-bit subgroup macros because of
this bug, but because we're going to start emitting the special moves
they expand to directly, we also have to handle the bug directly.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
100096394f ir3: Don't use swz with shared registers
It seems the two moves it decomposes into aren't always atomically
executed on the scalar ALU, which means that it randomly doesn't work.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
ec036fe51e ir3: Fix shared parallel copy validation
It's legal to have copies from immediate/const to shared, which was
asserting.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
ca91b58457 ir3/lower_pcopy: Fix immed/const flags for copy from shared
We were accidentally setting IR3_REG_SHARED on the source for
immediate/const sources.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
90dabe5a18 ir3: Fix lowering shared parallel copies with immed src
We need to look at the destination to determine whether the copy should
be classed as shared, because the source may be an immediate.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
4937172534 ir3/ra: Prepare for shared phis
Correctly copy the shared-ness to the parallel copy when the destination
is shared.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
a7cae84078 ir3/ra: Fix printing shared reg file
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
478cd71308 ir3/ra: Prepare for shared half-regs
Keep track of shared half-reg pressure and make RA aware of the
limitations on shared half-regs, similar to normal half-regs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
f8632862d0 ir3: Rewrite regmask implementation
Use ir3_reg_file_offset() in order to properly handle shared half-regs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
90067425a6 ir3/legalize: Use define for register size
This was introduced in a previous commit

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
750e6843c0 ir3: Rewrite postsched dependency handling
Split up the dependencies into multiple files, similar to RA, and
calculate the file + index. This lets us remove the previous hack we had
and lets us handle half shared registers correctly.

The actual calculation of the file is moved into a shared
ir3_reg_file_offset() function so that it can be reused in other places
which have to check for overlapping registers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
dbeeec2570 ir3/ra: Don't demote movmsk instructions to non-shared
It only supports shared register destinations.

Fixes: fa22b0901a ("ir3/ra: Add specialized shared register RA/spilling")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
79c89a3670 ir3: Validate tied sources better
Catch when we try to propagate an immediate or const, which can happen
if we forget to specify the valid flags for the instruction, and make
sure that it's the same size and type.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
3c71667dda ir3: Add scan_clusters.macro to is_subgroup_cond_mov_macro()
Fixes: 60413e1 ("ir3: optimize subgroup operations using brcst.active")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Connor Abbott
031b612449 ir3: Add scan_clusters_macro to ir3_valid_flags()
Fixes: 60413e1 ("ir3: optimize subgroup operations using brcst.active")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22075>
2024-04-26 12:55:13 +00:00
Alejandro Piñeiro
063ef2254b v3dv: enable VK_EXT_extended_dynamic_state
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27609>
2024-04-26 12:34:44 +00:00
Alejandro Piñeiro
499d5cb18d v3dv/ci: update expected list due VK_EXT_extended_dynamic_state
Those crashes are gone if VK_EXT_extended_dynamic_state is supported.

It is worth to note that the previous entry mentioned Ricardo's patch
as tentative. That is already on vk-gl-cts main (not still on the tag
used by the CI), and the code at this point has been tested with and
without that patch.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27609>
2024-04-26 12:34:44 +00:00
Alejandro Piñeiro
60e9237e81 v3dv: StencilOp and StencilTestEnable are now dynamic
This commit introduces a significant change when we emit STENCIL_CFG,
with any dynamic state: we stop to use cl_emit_with_prepacked, and use
directly cl_emit. The reason is that now most of the STENCIL_CFG
parameters are dynamic, any improvement of using
cl_emit_with_prepacked is minimized. Also gets the code simpler, and
avoid the need to be extra careful with the fact that
cl_emit_with_prepaked doesn't override values.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27609>
2024-04-26 12:34:44 +00:00
Alejandro Piñeiro
2526f74ade v3dv: PrimitiveTopology is now dynamic
Note that although the topology affects the final shader, and it is
part of the v3d_fs_key (through is_points and is_lines), changing
dynamically the topology would not trigger a shader recompilation as
that would only needed if there was a topology class change. From
spec:

  "VUID-vkCmdDraw-dynamicPrimitiveTopologyUnrestricted-07500

   If the bound graphics pipeline state was created with the
   VK_DYNAMIC_STATE_PRIMITIVE_TOPOLOGY dynamic state enabled and the
   dynamicPrimitiveTopologyUnrestricted is VK_FALSE, then the
   primitiveTopology parameter of vkCmdSetPrimitiveTopology must be of
   the same topology class as the pipeline
   VkPipelineInputAssemblyStateCreateInfo::topology state"

dynamicPrimitiveTopologyUnrestricted is defined at
VK_EXT_extended_dynamic_state3, so for now it is false. And even if in
the future we support that extension, it is really likely that we
would return False there.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27609>
2024-04-26 12:34:44 +00:00
Alejandro Piñeiro
fbfb99cbc3 v3dv: ez_state/incompatible_ez_test could be recomputed at cmd_buffer
As the values depends on several values that can be dynamic too.

Note that the current approach of this commit is keeping this info
duplicated on the pipeline and the cmd_buffer. The alternative would
be to just track it at the cmd_buffer, like we did recently with
z_updates_enable, but getting the values for ez_state/incompatible_ez
were more complex, so this commit still computes it when the pipeline
is created, and uses it as default value.

This is debatable though, and the alternative would be to just keep
ez_state/incompatible_ez_state at the command buffer.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27609>
2024-04-26 12:34:44 +00:00
Alejandro Piñeiro
b6e473cd58 v3dv: move depth CFG bits setting to cmd buffer emission
As it depends on values that could be dynamic now. Technically we
could try to keep pre-emitting, just in case that info is provided
statically.

But for the dynamic case, we would still need to compute that bits,
and we would need to discard all the pre-emitted CFG set, and
recompute it completely (as right now cl_emit_with_prepacked doesn't
allow to override values).

It is also gets a simpler code by setting those flags in only one
codepath.

As we are here, we also move z_updates_enable from the pipeline to the
cmd_buffer. This values doesn't require a complex compute, so it is
easier to just keep it on one place.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27609>
2024-04-26 12:34:44 +00:00
Alejandro Piñeiro
9fa023f111 v3dv: DepthBoundsTestEnable is dynamic now
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27609>
2024-04-26 12:34:44 +00:00
Alejandro Piñeiro
29c8aca881 v3dv: CullMode and FrontFace are dynamic now
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27609>
2024-04-26 12:34:44 +00:00
Alejandro Piñeiro
e3061e6281 v3dv: provide implementation for CmdSetViewportWithCount
As with CmdSetViewport, we need to provide a custom implementation
because we want to call and save the outcome of viewport_compute_xform
when the viewport is set, not during emission.

We can just call v3dv_CmdSetViewport, as that one is already calling
vk_common_SetViewportWithCount.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27609>
2024-04-26 12:34:44 +00:00
Alejandro Piñeiro
f4d426fae6 v3dv: provide implementation for vkCmdBindVertexBuffers2
Mostly equal to vkCmdBindVertexBuffers, but adding strides, that with
VK_EXT_extended_dynamic_state become dynamic, and setting pSizes.

It is worth to note that at this moment we don't use
CmdBindVertexBuffers2 pSizes.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27609>
2024-04-26 12:34:44 +00:00
Alejandro Piñeiro
f2236065b7 v3dv: port dynamic state tracking to use Mesa Vulkan
Specifically to use the common vk_dynamic_graphics_state.

The advantage of using the common struct is not only reducing the size
of our custom one, but also using common helpers (like all those cmd
buffer setters), and a lot of the logic that in the future will be
used for other extensions.

Some notes:

 * We still keep dirty flags, for things like PIPELINE,
   DESCRIPTOR_SETS, etc. Other driver do the same. FWIW, this is also
   an improvement, as before we were mixing those with the per-spec
   Vulkan dynamic info.

 * For the port viewport/scissor we still keep some data on a custom
   structure, as we cache the translate/scale info that is derived
   from scissor/viewport, but used in three different places.

   For that we also maintain a custom implementation of
   CmdSetViewport, that computes translate/scale, and call the common
   implementation.

 * We make the same for color_write_enables. The vulkan runtime saves
   it as a 8-bit bitset, with a bit per attachment. But when combining
   with color_write_mask you need a 32bit with 4 bits set per
   attachment. To avoid recompute it during emission, we also cache
   the color_write_enables, using the runtime just to track the dirty
   status.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27609>
2024-04-26 12:34:44 +00:00
Karmjit Mahil
858154b84e ir3: Don't set saturation on flat.b
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10999

Signed-off-by: Karmjit Mahil <karmjit.mahil@igalia.com>
Reviewed-by: Job Noorman <jnoorman@igalia.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28935>
2024-04-26 11:39:26 +00:00
Erik Faye-Lund
3b3df7b8a9 panvk: avoid dereferencing a null-pointer
If we're passed a memory-info, but no memory-prop, we'd end up
dereferencing a null-pointer here. Let's use a fallback struct instead,
similar to what RADV does.

Fixes: d970fe2e9d ("panfrost: Add a Vulkan driver for Midgard/Bifrost GPUs")
CID: 1496060
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28856>
2024-04-26 11:18:18 +00:00
Erik Faye-Lund
8456588b1f panvk: drop needless null-checks
iview can't be null here, so let's drop these checks.

CID: 1596487
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28856>
2024-04-26 11:18:18 +00:00
Erik Faye-Lund
5df20cac22 panvk: do not leak bindings
There were nothing here cleaning this up.

CID: 1596490
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28856>
2024-04-26 11:18:17 +00:00
Erik Faye-Lund
8fd171b02d panvk: drop needless null-check
This argument is never null, and we already dereference it earlier.
Let's remove the needless check here.

CID: 1503115
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28856>
2024-04-26 11:18:17 +00:00
Erik Faye-Lund
9058d5ff62 panfrost: correct first-tracking for signature
If we unconditionally assign false to first *before* we use it, it's
never true when used. Instead, let's assign it *both* at the end *and*
when continuing.

Fixes: 4da88060d0 ("panfrost: Skip blit shader labelling if the buffer has no space")
CID: 1476270
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28856>
2024-04-26 11:18:17 +00:00
Erik Faye-Lund
f852f86a31 panfrost: check return-code of drmSyncobjWait
Realistically, this isn't going to fail. But let's return an error here
in case it does, just for good measure.

CID: 1558596
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28856>
2024-04-26 11:18:17 +00:00
Erik Faye-Lund
70dcdb3130 panfrost: assert that drmSyncobjWait returns 0
This is really just a small band-aid, and instead we should start
reporting errors from this function. But for now, let's just assert that
no error occurrecd, as that's slightly better than ignoring it.

CID: 1592892
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28856>
2024-04-26 11:18:17 +00:00