Fixes: 4aa3b2d ('anv: LNL+ doesn't need the special flush for sparse')
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31737>
Emit dummy VF_STATISTICS state before each VF state.
Cc: mesa-stable
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31759>
This recommended values should improve the performance of async
compute in gfx20, we may want to tweek this for Linux but at least
this values should give us a better baseline than default values.
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30796>
Setting the missing registers to specification recommended values that
is also the default value, so it is not expected any changes in
behavior or performance here.
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30796>
DG2 has the 'Force Non-Coherent' fields but MTL and ARL has
'Z Async Throttle settings', so here adding the missing one.
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30796>
At the point we were calling this, we hadn't necessarily cleaned up
derefs via nir_lower_vars_to_ssa, nor movs/vecs via copy propagation,
so it wasn't necessarily easy for this pass to see the actual usage of
the destination.
Moving this later allows us to detect f2f32(txf(...)) and avoid
converting it to a 16-bit txf (why convert with ALU instructions
when the sampler could do it for us?).
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Sushma Venkatesh Reddy <sushma.venkatesh.reddy@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31750>
This also means the infrastructure added by @gallo in 1dc64d0613
("ci: Use merge-skips files during merge pipelines") can be used and all
the manual adding of these files can be dropped, reducing the likeliness
of bugs.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31739>
Add opcodes for VOTE_ALL, VOTE_ANY and VOTE_EQUAL. The first two
are also used for the quad variants. Move their lowering from
NIR conversion to brw_lower_subgroup_ops.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31029>
If we have a non-register-aligned source, MOV it to a new register
so that the invariant expected when generating SHADER_OPCODE_BROADCAST
is respected.
Added to ensure a later patch won't hit the `src.subnr == 0` assertion
in brw_broadcast() generation code.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31029>
Include in the helper which already take care of using exec_all() and
taking the first component of the result. Both are expected by
SHADER_OPCODE_BROADCAST.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31029>
ANGLE has a waiver for certain XFB tests, but this wasn't properly applied
on Alder Lake and these tests weren't skipped there.
Add a global angle-skips.txt file so that we don't have to keep copy-pasting
these skips.
Signed-off-by: Valentine Burley <valentine.burley@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31721>
There is a fresher device type with a CML GPU, with also a bigger number of
boards. Those are more reliable, so also we can remove the manual rules.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26830>
ANGLE currently pulls absolutely loads of stuff that we don't need. Fix
it up so we don't need to do that anymore, so it's much faster to build.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31716>
Quoting the documentation :
"The returned range is a half-open interval where all of the
addresses within the subimage are < end_tile_B."
This is obviously not true with images smaller than a logical tile.
Currently the code return 1.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24276>
We jump out of the loops whenever result is not VK_SUCCESS, there is
no need to check for it there. I guess I missed this detail in the
most recent rework for this function.
Reviewed-by: Iván Briano <ivan.briano@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31698>
When the VkBuffer is of size 2^32 (which matches maxBufferSize), we
have vm_bind->size set to 2^32, which is fine because it fits in an
uint64_t. What is not fine is the 'i' variable being size_t, because
on 32bit systems it will loop forever since it will always be smaller
than 2^32.
Credits to Iván for not only reporting it, but also coming up with the
solution at the same time as I did, then testing it.
Cc: mesa-stable
Reported-by: Iván Briano <ivan.briano@intel.com>
Reviewed-by: Iván Briano <ivan.briano@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31698>
This will make things easier in situations where we don't want to use
the binding table at all (indirect draws/dispatches).
The mechanism is simple, upload a vec3 either through push constants
(<= Gfx12.0) or through the inline parameter register (>= Gfx12.5).
In the shader, do this :
if vec.x == 0xffffffff:
addr = pack64_2x32 vec.y, vec.z
vec = load_global addr
This works because we limit the maximum number of workgroup size to
0xffff in all dimension :
maxComputeWorkGroupCount = { 65535, 65535, 65535 },
So we can use the large values to signal the need for indirect
loading.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31508>
This intrinsic was initially dedicated to mesh/task shaders, but the
mechanism it exposes also exists in the compute shaders on Gfx12.5+.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31508>
In SIMD32, the fence instruction is currently going to read grf0-3
leading to such assertions in the backend :
../src/intel/compiler/brw_fs_reg_allocate.cpp:206:
void fs_visitor::calculate_payload_ranges(bool, unsigned int, int*) const:
Assertion `j < payload_node_count' failed.
The reason we haven't seen the problem yet is that there always enough
payload register to accomodate this. But the following change is going
to make the inline parameter register optional.
Since SHADER_OPCODE_MEMORY_FENCE is emitted in the generator as SIMD1
NoMask (see brw_memory_fence), we can limit ourselves to SIMD1
exec_all() in the IR as well so that the IR accounts for grf0 as a
source.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31508>
Useful to insert debug traces a bit later in the lowering process (in
particular after load/store vectorization).
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31508>
When flushing the render target cache for future operations, we need a
stall at pixel scoreboard. We likely didn't see any issue until now
because a change in render target added the pb-stall.
When using a 2 compute shaders with the following pattern :
vkCmdDispatch()
vkCmdPipelineBarrier() ImageBarrier with (src|dst)AccessMask=0 & identical layout
vkCmdDispatch()
we should ensure that the first dispatch is completed before executing
the second one, otherwise they can race to on resource accesses. This
fixes failures in some new CTS tests.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31676>
We have no cases where we intentionally pass a NULL layout when dynamic
offsets, and doing so would cause a null dereference. Le't asd an assert
for that.
CID: 1620447
Fixes: f39cd30f4f ("anv: Track all the descriptor sets")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31638>
Coverity notices that we've insured that index index is < MAX_RTS in one
case, but that it must be greater in one case. Since `color_att_count`
is a uint32_t, it can easily exceed MAX_RTS (8), and would thus create
an out-of-bounds read situation. While the type system would allow this,
the actually implementation shouldn't, so an assert should make Coverity
happy and help us check our assumption.
CID: 1620440
Fixes: d2f7b6d5a7 ("anv: implement VK_KHR_dynamic_rendering_local_read")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31640>
The code that initializes each queue got big enough that the
repetitive error handling is getting ugly and it could benefit from
being on its own function.
v2: Rebase, try to improve the comments.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30953>
Since the L2 bug fix we've been overestimating l3l2_binds by a lot in
most of the cases: almost every single call to anv_sparse_bind_trtt
ends up using either 0 or 1 elements for l3l2_binds, with occasionally
something using 512 or more. By switching to util_dynarray we can
guarantee the best of every case:
- l1_binds will remain a stack array for the vast majority of the
calls
- even more than before, since STACK_ARRAY was limited to 8
elements and now we do 32
- l1 will be properly dimensioned without the need for reallocs
- l3l2_binds will be completely empty most of the times and only
trigger allocations when necessary
Here's the top 10 most common results of anv_sparse_bind_trtt() for a
trace of Assassin's Creed: Valhalla. The first column is how many
times we had that case while running the trace. After this patch, all
these cases will proceed without any memory allocations.
168 trtt_binds: num_vm_binds:04 l3l2:0000 l1:0004
344 trtt_binds: num_vm_binds:01 l3l2:0000 l1:0004
420 trtt_binds: num_vm_binds:01 l3l2:0000 l1:0012
422 trtt_binds: num_vm_binds:04 l3l2:0000 l1:0008
479 trtt_binds: num_vm_binds:01 l3l2:0000 l1:0024
560 trtt_binds: num_vm_binds:03 l3l2:0000 l1:0003
1005 trtt_binds: num_vm_binds:01 l3l2:0000 l1:0002
1024 trtt_binds: num_vm_binds:02 l3l2:0000 l1:0004
2145 trtt_binds: num_vm_binds:02 l3l2:0000 l1:0002
3735 trtt_binds: num_vm_binds:01 l3l2:0000 l1:0001
Only 70 out of total 11340 calls to anv_sparse_bind_trtt() contained
l3l2 elements.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30953>
We use 2MB page table BOs, as defined by ANV_TRTT_PAGE_TABLE_BO_SIZE.
Each BO is enough to hold 512 pages, since each one has 4096 bytes.
Each L1 page can fit 1024 entries of 64kb size, which means our 512
pages should be able to fit a little less than 32gb of sparse resource
memory, since we also need some L2 pages and an L3 page. I don't see
any real world application using more than a single BO.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30953>
Move it past the (n_l3l2_binds == 0 && n_l1_binds == 0) check so we
don't end up trying to do garbage collection more often than we submit
batches.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30953>
Otherwise code such as anv_sparse_trtt_garbage_collect_batches() may
end up stuck waiting forever on a timeline of a submission that
failed.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30953>
The application can submit bind operations where it simply resets
state that is already in our page tables, so there's nothing to do.
Before commit 7da5b1caef ("anv: move trtt submissions over to the
anv_async_submit") we would simply return and not submit any batches
when this happened, but the commit reorganized things in a way where
we started submitting empty batches instead.
Fix this by simply jumping out when we detect this case. Because of
this, rename the "error" labels to "out" as they can now happen on a
happy case.
It should be noted that an alternative to this implementation would be
to move all the handling of 'submit' to after the n_lX_binds check,
but this would put all the initialization inside the trtt->mutex,
creating extra contention even when we have stuff to bind. Since the
"there's nothing to bind" check is now rare (after we stopped doing
NULL binds during resource creation), it is probably better to reduce
lock contention in the common case at the expense of a little more CPU
in the rare case.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30953>
We're missing a check for 'result' in the middle of a loop and we have
an unnecessary check for 'result' after the loop.
Fixes: 7da5b1caef ("anv: move trtt submissions over to the anv_async_submit")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30953>
There is a lot that can go wrong during initialization after we assign
trtt->l3_addr, and we use its value to check if trtt is initialized.
If an initialization fails after l3_addr is already assigned, the next
bind will attempt to use the leftover values from the failed
initialization attempt and will likely cause all sorts of random
errors. So when we fail, just set l3_addr back to 0, causing the next
bind to attempt to initialize everything again.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30953>
