The hardware's clear color conversion feature requires invalidating the
texture cache for every fast clear. We're no longer using the hardware
feature, so we longer need the invalidation.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30646>
This improves drivers in the following ways:
* iris_hiz_exec() and crocus_hiz_exec() gets rid of the narrowly-used
update_clear_depth parameters.
* iris avoids fast-clearing if the aux state is CLEAR. crocus avoids
this too, but didn't actually need it in the first place.
* iris updates the value once per fast_clear_depth() call instead of
doing an update for each layer being cleared.
* anv now updates the clear value when transitioning from an undefined
layout instead of doing so on every fast-clear. This should be safer
because we don't perform state cache invalidates when changing the
clear value. So, existing surface states won't have any stale values.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30520>
The PRMs suggest that certain classes of auxiliary surface operations
will automatically synchronize when performed back-to-back:
Any transition from any value in {Clear, Render, Resolve} to a
different value in {Clear, Render, Resolve} requires end of pipe
synchronization.
Make use of this functionality by batching CCS and MCS flushes when
compatible auxiliary surface operations are performed within a command
buffer.
Ref: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11325
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29922>
According to the HSD, this is an alternative option for Wa_14016712196.
Taking this option allows us to combine this workaround with a couple
other depth workarounds. Make sure to execute these workarounds before
the workaround for the depth register mode, so that the stalling flush
is not impacted.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29922>
This flush was introduced with the following commits:
8949d27bb8 ("anv: implement gen9 post sync pipe control workaround")
bcb611361b ("anv: implement gen12 post sync pipe control workaround")
The flush was unsued with the following commit:
e79e1ca304 ("intel: Drop Tigerlake revision 0 workarounds")
This prevents some extra pipecontrols caused by a following patch.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29922>
We want to reduce the buffer allocations for other type of data than
timestamps.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29944>
Pretty big change... Sorry for that.
I can't exactly remember why I created 2 heaps. I think it's because I
mistakenly thought the samplers in the binding sampler pointers needed
to be indexed from the binding table. But that's not the case, they
just need to be in the dynamic state heap.
In the future, this change will allow to also allocate buffers for
push constant data in the newly created dynamic_visible_pool which
will be useful on < Gfx12.0 where this is the only place push constant
data can live for compute shaders.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30047>
Newer hardware is smart enough to know that if something writes to a
NULL tile and immediately reads back the value (from the cache), the
value should read back as zero, not whatever was written to the cache
but not the memory. Due to that, we don't need to flush the tile
cache, which is quite expensive.
Link: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11029
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29953>
These were only applied in emit_apply_pipe_flushes but in theory could
be required for some other individually shot pipe controls.
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29897>
Completely skip the stall & programming if the bindless address has
not changed. Only on Gfx12.5+ since previous generations also program
the binding table pool base address through STATE_BASE_ADDRESS.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29595>
The mi-builder already takes care of mi write/read fences, but we have
a few cases in Anv where we also need to fence mi-write ->
shader-read.
We also have one case where a command buffer jump address is modified
by a previous mi write command.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29595>
Since we don't need to share that data with other fixed functions.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29571>
We can remove a bunch of TRTT specific code from the backends as well
as manual submission tracking.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28975>
This AUX-TT is only updated on the CPU since ee6e2bc4a3 ("anv: Place
images into the aux-map when safe to do so"). So the only really
important invalidation that needs to happens is on the beginning of a
primary command buffer.
We are required to idle the pipes prior invalidation the AUX-TT. This
might not be happening when the invalidation is put at the beginning
of the secondary command buffers.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29671>
We increased the size of the timestamps but only copied 64bit values
from the secondaries.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 521c216efc ("anv: use COMPUTE_WALKER post sync field to track compute work")
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29438>
Most of what we do in this function is conditional to not have
VK_RENDERING_SUSPENDING_BIT, so check for it once.
Suggested-by: Iván Briano <ivan.briano@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27306>
Consider the following program:
- Uses a multi-sampled image as the color attachment.
- Draws simple geometry at the color attachment.
- Uses the (non-multi-sampled) swapchain image as the resolve image.
- Presents the result.
If the color attachment image (the multi-sampled one) is a sparse
image and it's fully bound, everything works and this patch is not
required.
If the image is partially bound (or just completely unbound), without
this patch the unbound area of the image that ends up being displayed
on the screen is not completely black, and it should be completely
black due to the fact that we claim to support
residencyNonResidentStrict (which is required by vkd3d for DX12).
On DG2, what ends up being displayed in the swapchain image is
actually the whole image as if it was completely bound. On TGL the
unbound area partially displays the geometry that was supposed to be
drawn, but the background is a different color: it's a weird corrupted
image. On both platforms the unbound areas should all be fully black.
This patch applies the proper flushing so that we get the results we
should have.
The bug fixed by this patch is not caught by dEQP or anything our CI
runs (dEQP does have some checks for residencynonResidentStrict
correctness, but none that catch this issue in particular). I was able
to catch this with my own sample program. Using INTEL_DEBUG=stall also
makes the problem go away.
If we had a way to track which images are fully bound we would be able
to avoid this flush. I had code for that in the earliest versions of
sparse before xe.ko had support for gpuva, but it requires maintaining
a bunch of lists, so I'm not sure that's actually worth it.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27306>
Initial work by Rafael Antognolli <rafael.antognolli@intel.com>
Reworks
- Rebase to main
- Emit the right hiz op for higher mip levels when transitioning the
depth buffer
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28629>
The device->using_sparse variable is only used at cmd_buffer_barrier()
to decide if we need to apply the heavier-weight flushes that are only
applicable to sparse resources. The big problem here is that we need
to apply the flushes to the non-image and non-buffer memory barriers,
so we were trying to limit those only to applications that ever submit
a sparse resource to the sparse queue.
The reason why we were applying this only to devices that ever
submitted sparse resources is that dxvk games have this thing where
during startup they create and then delete tiny sparse resources, so
switching device->using_sparse to true at resource creation would make
basically every dxvk game start applying the heavier-weight
workaround.
The problem with all that is that even if an application creates a
sparse resource but doesn't ever bind them, the resource should still
behave as an unbound resource (because they are bound with a NULL
bind), so the flushes affecting them should happen. This case is
exercised by vkd3d-proton/test_buffer_feedback_instructions_sm51.
In order to satisfy all the above cases and only really apply the
heavier-weight flushes to applications actually using sparse
resources, let's just count the number of sparse resources that
currently exist and then apply the workaround only if it's not zero.
That covers the dxvk case since dxvk deletes the resources as soon as
they create, so num_sparse_resources goes back to 0.
Testcase: vkd3d-proton/test_buffer_feedback_instructions_sm51
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10960
Fixes: 6368c1445f ("anv/sparse: add the initial code for Sparse Resources")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28724>
The current helper anv_physical_device_bindless_heap_size()
artificially limited the surface heap size on DG2+ to 128MB. The HW is
actually 4GB capable, but we have workaround requiring to overlap the
dynamic state heap with the bindless surface state heap.
The actual limit comes from our virtual address space setup. It is
different between descriptor buffers and regular descriptors.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: fe037dec6e ("anv: expose VK_EXT_descriptor_buffer")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27873>
The BSpec page "Flush Types" (46213) says the following about the Tex
Invalidate bit:
"Requires stall bit ([20] of DW) set for all GPGPU Workloads."
For newer platforms, this is documented in the description of the
texture invalidation bit in the PIPE_CONTROL page (56551):
"CS Stall bit in PIPE_CONTROL command must be always set for GPGPU
workloads when Texture Cache Invalidation Enable bit is set"
Iris had it only for GFX_VER 9 and 11, while Anv had it missing for
everything.
Please notice that this patch includes a revert of 397e728ef4.
Fixes: 397e728ef4 ("iris: Drop GPGPU Tex Invalidate restriction for TGL+")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28608>
We're changing the memory address translation tables, we should
invalidate their cache.
It seems i915.ko is already doing this for us in between batches. The
xe.ko driver only adds invalidates to the ring before submissions if
scratch page is enabled in the VM (which it is today, but may change
in the future), and after some vm_bind and all vm_unbind ioctls, but
we don't use vm_bind for TR-TT. Still, it won't hurt to have it here
righ tnow.
v2: Use PIPE_CONTROL_length (José).
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v1)
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27928>
This should drop it from MTL as there it should apply only for a0
stepping.
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28047>
