This handles the new inlines queries for VK_KHR_video_maintenance1
(Decode doesn't do queries on AMD at the moment).
Reviewed-by: Lynne <dev@lynne.ee>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30671>
The static assert used in encode deref modes used the fact there was
less than 16 modes that we wanted to compress as an opportunity to reuse
MODE_ENC_GENERIC_BIT as it just happened to represent 16. However if we
add more than 16 modes i.e need to compress to 6 bits not 5 bits then
MODE_ENC_GENERIC_BIT becomes 32 and the logic in the assert breaks.
Instead we more precisely make sure MODE_ENC_GENERIC_BIT is large
enough to fit all but the last 4 generic modes and that the last 4 modes
defined in the enum are in fact the 4 generic modes.
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30654>
Similar to commit 6cef804067 ("nir/opt_if: fix opt_if_merge when
destination branch has a jump"), we shouldn't combine if statements when
the second if-then-else has a block that ends in a jump.
This fixes a case where opt_if_merge combines
if (cond) {
[then-block-1]
} else {
[else-block-1]
}
if (cond) {
[then-block-2]
} else {
[else-block-2]
}
where `then-block-2` or `else-block-2` ends in a jump. The phi nodes
following the control flow will be incorrectly updated to have an input
from a block that is not a predecessor.
Fixes: 4d3f6cb973 ("nir: merge some basic consecutive ifs")
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30629>
This is to support QP map enabled and disabled mixed case.
When qp map disabled, it still needs the IB package to tell
VCN engine qp map is not needed.
Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Reviewed-By: Sil Vilerino <sivileri@microsoft.com>
Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30665>
reason:
roi number is an indication to do qp_map in vcn encoder.
if not resetting this number, the previous roi style
will be used if not changed, or not used. In the case
non roi case mixed with roi, the behavior will not be
expected.
reset roi_num at the beginning of each frame, if application
doesn't send roi map, then roi will be stopped.
Reviewed-by: Boyuan Zhang <Boyuan.Zhang@amd.com>
Reviewed-By: Sil Vilerino <sivileri@microsoft.com>
Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30665>
Right now mesa always returns GL_FULL_SUPPORT, so use the
is_format_supported hook to get the actual info from drivers instead.
Signed-off-by: Pavel Ondračka <pavel.ondracka@gmail.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30612>
This is a hint that the branch is worth removing. Assume that's the case,
regardless of the gfx level.
fossil-db (vega10):
Totals from 22 (0.03% of 63053) affected shaders:
Instrs: 23927 -> 23856 (-0.30%)
CodeSize: 125096 -> 124812 (-0.23%)
Latency: 138258 -> 137765 (-0.36%)
InvThroughput: 55900 -> 55884 (-0.03%)
Branches: 391 -> 320 (-18.16%)
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30321>
No fossil-db changes.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Backport-to: 24.1
Backport-to: 24.2
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30321>
Before we were using direct CP_LOAD_STATE, which is broken with multiple
back-to-back draws. This caused regressions in some DX11 traces when
enabling early preamble. We still need to use indirect CP_LOAD_STATE for
VS params, which are sometimes written by the CP, however for everything
else we should use the new UBO path instead.
Fixes: 76e417ca59 ("turnip,ir3/a750: Implement consts loading via preamble")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30675>
It's one header dword and 5 payload dwords. This was papered over by us
not actually using the UBO path for one of the loads, but that's changed
in the next commit.
Fixes: 76e417ca59 ("turnip,ir3/a750: Implement consts loading via preamble")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30675>
It turns out that mova executes on the normal pipeline, which means that
users of a0.x on the scalar pipeline might cause a WAR hazard with mova.
Fixes: 876c5396a7 ("ir3: Add support for "scalar ALU"")
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
fullsync would only sync after cat4/5/6 instructions. However, since the
introduction of scalar ALU, we also need to sync after writes to shared
registers. This commit fixes this by using the is_ss/sy_producer
helpers. This should also catch all cases where (ss) is need for WAR
hazards.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
These can be repeated like other instructions with one interesting
wrinkle: their immediate input location can also be repeated and its
value gets incremented by one for every repeat. They seem to be the only
instructions to support this.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
Some store intrinsics (e.g., store_const_ir3) don't have a wrmask so
don't assume it always exists.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
Some load/store intrinsics (e.g., load/store_const_ir3) use offsets in
units other than bytes. Currently, byte offsets were assumed in multiple
places.
This patch adds a new offset_scale field to intrinsic_info and uses it
were needed.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
Phi nodes are mostly handled the same way as ALU instructions: if all
sources point to the same def (which happens if they are scalar or have
been previously vectorized), combine them into a single vectorized phi
node.
There is one case where this doesn't work, however: sources that come
from a loop back-edge. Since their defs haven't been processed yet, they
are generally not the same. We could simply refuse to vectorize such
phi nodes but this could leave many values used in loops unnecessarily
scalarized.
Instead, this patch implements a simple heuristic: if all defs coming
from a back-edge have the same instructions type and, in case of ALU,
the same operation, assume they will be vectorized later. Since we
require that normal edges are vectorized already, chances are that the
back-edge can also be vectorized.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
To handle phi nodes, it's important that all sources have been processed
before processing the phi node itself. The current traversal order
(depth-first on dom_children) does not guarantee this. This patch
rewrites the pass to visit blocks in source-code order.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
Dispatch to different functions inside instr_try_combine. To prepare for
upcoming support for phi nodes.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
Use the new repeat group builders to lower vectorized NIR instructions.
Add NIR pass to vectorize NIR before lowering.
Support for repeated instruction is added over a number of different
commits. Here's how they all tie together:
ir3 is a scalar architecture and as such most instructions cannot be
vectorized. However, many instructions support the (rptN) modifier that
allows us to mimic vector instructions. Whenever an instruction has the
(rptN) modifier set it will execute N more times, incrementing its
destination register for each repetition. Additionally, source registers
with the (r) flag set will also be incremented.
For example:
(rpt1)add.f r0.x, (r)r1.x, r2.x
is the same as:
add.f r0.x, r1.x, r2.x
add.f r0.y, r1.y, r2.x
The main benefit of using repeated instructions is a reduction in code
size. Since every iteration is still executed as a scalar instruction,
there's no direct benefit in terms of runtime. The only exception seems
to be for 3-source instructions pre-a7xx: if one of the sources is
constant (i.e., without the (r) flag), a repeated instruction executes
faster than the equivalent expanded sequence. Presumably, this is
because the ALU only has 2 register read ports. I have not been able to
measure this difference on a7xx though.
Support for repeated instructions consists of two parts. First, we need
to make sure NIR is (mostly) vectorized when translating to ir3. I have
not been able to find a way to keep NIR vectorized all the way and still
generate decent code. Therefore, I have taken the approach of
vectorizing the (scalarized) NIR right before translating it to ir3.
Secondly, ir3 needs to be adapted to ingest vectorized NIR and translate
it to repeated instructions. To this end, I have introduced the concept
of "repeat groups" to ir3. A repeat group is a group of instructions
that were produced from a vectorized NIR operation and linked together.
They are, however, still separate scalar instructions until quite late.
More concretely:
1. Instruction emission: for every vectorized NIR operation, emit
separate scalar instructions for its components and link them
together in a repeat group. For every instruction builder ir3_X, a
new repeat builder ir3_X_rpt has been added to facilitate this.
2. Optimization passes: for now, repeat groups are completely ignored by
optimizations.
3. Pre-RA: clean up repeat groups that can never be merged into an
actual rptN instruction (e.g., because their instructions are not
consecutive anymore). This ensures no useless merge sets will be
created in the next step.
4. RA: create merge sets for the sources and defs of the instructions in
repeat groups. This way, RA will try to allocate consecutive
registers for them. This will not be forced though because we prefer
to split-up repeat groups over creating movs to reorder registers.
5. Post-RA: create actual rptN instructions for repeat groups where the
allocated registers allow it.
The idea for step 2 is that we prefer that any potential optimizations
take precedence over creating rptN instructions as the latter will only
yield a code size benefit. However, it might be interesting to
investigate if we could make some optimizations repeat aware. For
example, the scheduler could try to schedule instructions of a repeat
group together.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
Clean up repeat groups that can never be merged into an actual rptN
instruction (e.g., because their instructions are not consecutive
anymore). This ensures no useless merge sets will be created for RA.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
Create merge sets for the sources and defs of the instructions in repeat
groups. This way, RA will try to allocate consecutive registers for
them. This will not be forced though because we prefer to split-up
repeat groups over creating movs to reorder registers.
When choosing a register for a repeat group's merge set, if its merge
set is unique (i.e., only used for these repeated instructions), try to
first allocate one of their sources (for the same reason as for ALU/SFU
instructions). This also prevents us from allocating a new register
range for this merge set when the one from a source could be reused.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
For every instruction builder ir3_X, this patch adds new repeat builder
ir3_X_rpt to create a repeated version of an instruction.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
In order to represent repeated instructions (rptN) in ir3, this patch
introduces the concept of "repeat groups". A repeat group is a group of
instructions that were produced from a vectorized NIR operation and
linked together. They are, however, still separate scalar instructions.
Repeat groups are created by linking together multiple instructions
using a new rpt_node list. This patch adds this list as well as a number
of helper functions the can be used to create and manipulate repeat
groups.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
The tex prefetch heuristic simply counts the number of NIR instructions.
Since a vectorized NIR instruction expands to an ir3 instruction per
component, we have to take this into account while counting them.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341>
