I've found myself adding this piece of code to our codebase when
debugging some Zink sparse failures recently, so let's upstream it.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29337>
This calculation was wrong for both compressed formats and
multi-sampled images. As a result, we misreported the image as having
a single miptail.
No Vulkan or GL CTS tests were tripping on this bug. I found this
while looking for tile size calculations after fixing a similar bug
elsewhere in the code.
The calculation should now match what we have in
anv_sparse_bind_image_memory(), which is widely tested.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29337>
We have to take the number of samples into account when calculating
the tile size. If we don't do this, multi-sampled images may end up
falling in the "goto out_everything_is_miptail" case, while in reality
multi-sampled images don't even have miptails.
Also assert that the value is one of the only two values we expect
this to be. This assert would have been useful to catch this issue,
since with multi-sampled images we were getting values like 16k or 32k
depending on the number of samples.
This helps move forward progress in some Zink tests, but does not
make them fully pass yet, as those tests are full of sub-cases and
this only helps some of them:
KHR-GL46.sparse_texture2_tests.UncommittedRegionsAccess
KHR-GL46.sparse_texture2_tests.SparseTexture2Commitment
KHR-GL46.sparse_texture2_tests.SparseTexture2Lookup
Fixes: 7ef3d652b2 ("anv/sparse: enable MSAA for Sparse when applicable")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29337>
The Vulkan spec splits sparse resources in two different features:
sparse binding and sparse residency.
Sparse binding is much simpler. It requires the resources to be fully
bound before being used and it treats them as a black box. We're
required to support sparse binding for all the formats that are
supported by non-sparse, but that's easy beacause this feature is
simpler.
Now sparse residency is the one where we're allowed to partially bind
resources, and the one that comes with more complicated features such
as block shapes and non-opaque binding of images. This feature is
subdivided into:
- sparseResidencyBuffer
- sparseResidencyImage2D
- sparseResidencyImage3D
- sparseResidency{2,4,8,16}Samples (which refers to 2D images)
Notice that there's no sparseResidencyImage1D. And if you read the
specs it's clear that sparse residency is meant for non-1D images.
Still, supporting it didn't require any extra effort in Anv so we just
did it.
That's until we started running GL CTS tests on Zink. There's a CTS
test that checks for the standard block shapes. It creates 1D images
and expects the block shapes for them to be the standard 2D block
shapes. While we could very well just patch
anv_sparse_calc_image_format_properties() to return the standard 2D
block shapes for 1D images, that's just wrong (block shapes for 1D
images are just line segments, not rectangles!) so let's just reject
this all until maybe one day Vulkan defines sparseResidencyImage1D and
we get GL_ARB_sparse_texture3 to match it, or somebody decides to
change the GL CTS test.
Testcase: KHR-GL46.sparse_texture2_tests.StandardPageSizesTestCase
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29337>
We can remove a bunch of TRTT specific code from the backends as well
as manual submission tracking.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28975>
Drop usage of pthread mutex so initialization never fails.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28975>
A format can't be standard and non-standard at the same time. If we
ever hit this assertion, it's because something behind the scenes has
evolved (such as the tiling formats) so something that was marked as
non-standard became standard. Add an assertion so we can quickly catch
these issues in the future and adjust the code.
I don't want to mix this assertion with the one in the line above
since that one is the most useful assertion we have in all the sparse
code, so it's good to know which one we're hitting.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27306>
The Tile64 format from Xe2 is weird and some of its MSAA shapes are
non-standard. Reject them. Otherwise, we'll get dEQP failures such as:
deqp-vk: ../../src/intel/vulkan/anv_sparse.c:829: anv_sparse_calc_image_format_properties: Assertion `is_standard || is_known_nonstandard_format' failed.
Many tests can reproduce this issue, including:
dEQP-VK.memory.requirements.extended.image.sparse_tiling_optimal
Testcase: dEQP-VK.memory.requirements.extended.image.sparse_tiling_optimal
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27306>
This is all that's needed to make anv_sparse_bind_image_memory() work
with multi-sampled images.
The assert() we just added would have been really helpful when
debugging this.
All the dEQP tests with "sparse" in their names are passing *even*
without this patch. Real-world applications show very clear visual
corruption for sparse MSAA images bound through non-opaque binds since
only a fraction of the the actual image ends up being bound.
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27306>
Yes, I understand that this looks like the kind of check that the
applications should be doing instead of us, but if we don't that, dEQP
will have failures. If we claim support for any multi-sampled sparse
feature, dEQP will try to create multi-sampled sparse images with all
possible sample counts, including the ones supported by non-sparse but
not supported by sparse (x8 and x16 on Tile64 platforms) and also the
ones not supported at all, like x32 and x64.
This change affects a number of dEQP tests, including:
- dEQP-VK.api.info.sparse_image_format_properties2.2d.optimal.r32g32_sfloat
Without this patch, and with sparse multi-sampling enabled, this would
hit the following assertion:
anv_sparse.c:866: anv_sparse_calc_image_format_properties: Assertion `is_standard || is_known_nonstandard_format' failed.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27306>
Our hardware has more than one layout for multi-sampled images that
use the tiling formats that give us the sparse standard block shapes:
see enum isl_msaa_layout. Only the layout we use for colored images is
compatible with the standard block shapes, so it's the only one we can
expose for multi-sampled sparse.
This change affects a number of dEQP tests, including:
- dEQP-VK.memory.requirements.create_info.image.sparse_residency_aliased_tiling_optimal
Without this patch, and with sparse multi-sampling enabled, this test
would hit the following assertion:
anv_sparse.c:866: anv_sparse_calc_image_format_properties: Assertion `is_standard || is_known_nonstandard_format' failed.
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27306>
We're not enabling sparse on multi-sampled images yet, but having the
table here is a first step. The current approach should make the code
a little more compact.
These tables are in section 33.4.3: Standard Sparse Image Block Shapes
of the Vulkan 1.3 spec.
PS: I know we've questioned the need for us to have these tables here
as they are something dEQP should check, but I've hit the "this shape
is not standard" assertion multiple times during development of the
various sparse features, and that really helps narrowing down the
problems. For example, see the next 2 patches in this MR.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27306>
Some places doing driver internal allocations was not setting
ANV_BO_ALLOC_INTERNAL, so adding the flag in those places here.
This will increase the accuracy of the RMV report.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28677>
None of the callers of anv_free_sparse_bindings() check for its return
result, and they also don't have a way to propagate it up the stack.
So just don't return error codes that won't be checked. Instead,
add an assertion so at least we can detect failures in our CI or
development runs.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28724>
The device->using_sparse variable is only used at cmd_buffer_barrier()
to decide if we need to apply the heavier-weight flushes that are only
applicable to sparse resources. The big problem here is that we need
to apply the flushes to the non-image and non-buffer memory barriers,
so we were trying to limit those only to applications that ever submit
a sparse resource to the sparse queue.
The reason why we were applying this only to devices that ever
submitted sparse resources is that dxvk games have this thing where
during startup they create and then delete tiny sparse resources, so
switching device->using_sparse to true at resource creation would make
basically every dxvk game start applying the heavier-weight
workaround.
The problem with all that is that even if an application creates a
sparse resource but doesn't ever bind them, the resource should still
behave as an unbound resource (because they are bound with a NULL
bind), so the flushes affecting them should happen. This case is
exercised by vkd3d-proton/test_buffer_feedback_instructions_sm51.
In order to satisfy all the above cases and only really apply the
heavier-weight flushes to applications actually using sparse
resources, let's just count the number of sparse resources that
currently exist and then apply the workaround only if it's not zero.
That covers the dxvk case since dxvk deletes the resources as soon as
they create, so num_sparse_resources goes back to 0.
Testcase: vkd3d-proton/test_buffer_feedback_instructions_sm51
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10960
Fixes: 6368c1445f ("anv/sparse: add the initial code for Sparse Resources")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28724>
This went unused a while ago. If we decide we want it again we can
just add it back.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28724>
Since we moved the dump_anv_vm_bind() call to anv_sparse_bind(), that
BEGIN/END block stopped making sense, so just keep the first set of
messages.
Also wrap everything around a single INTEL_DEBUG() check so we'll only
run this check once when debug is disabled (we don't care about
running the check multiple times if it's enabled).
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28724>
In both cases we end up calling anv_image_aspect_to_plane(), which
already includes the same assertion.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28724>
If isl_surf_get_tile_info() returned the struct instead of having it
passed as a pointer, gcc would have detected this. I can write patches
for that if we want it.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28724>
Having just one place to check the Sparse type is less error prone.
For example in i915 it was always setting sparse_uses_trtt to true
even if running in gfx 9 that don't support sparse.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28161>
All these vfuncs funnel down to either stubs or the xe_vm_bind_op()
function. By returning int we're shifting VkResult generation to the
callers, which are simply not doing the correct job. If they get
VkResult they can simply throw the errors up the stack without having
to erroneously try to figure out what really happened.
Today the callers are returning either VK_ERROR_UNKNOWN or
VK_ERROR_OUT_OF_DEVICE_MEMORY, but after the patch we're returning
either VK_ERROR_OUT_OF_HOST_MEMORY or VK_ERROR_DEVICE_LOST.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27926>
We can now finally leave the semaphore waits and signals to the
vm_bind ioctl, making vm_bind operations truly asynchronous.
This was previously done for TR-TT in 18bd00c024 ("anv/trtt: don't
wait/signal syncobjs using the CPU anymore").
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27926>
The xe.ko driver finally fixed bug 746, which means we can finally
pass multiple bind operations in a single ioctl. There's a dEQP test
that issues 960 bind operations in a single call, so our gains here
have potential, although most real-world apps are not even remotely
close to this.
Link: https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/746
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27926>
The resource size doesn't need to match the binding granularity. For
example, if the user wants to create a 32kb buffer, Anv will require
its memory to have 64kb, but the buffer size will still be the
original 32kb. And the spec says:
VUID-VkSparseMemoryBind-size-01100:
"size must be less than or equal to the size of the resource minus
resourceOffset"
VUID-VkSparseMemoryBind-size-01102:
"size must be less than or equal to the size of memory minus
memoryOffset"
So when binding such buffer, size should actually be the lesser of the
two values: 32kb, and we have to accept that. Since our binding
granularity is 64kb, we're safe to simply extend the requested size to
match our binding granularity, since we already require the memory to
be appropriately sized.
None of this is exercised by dEQP. This was caught by
piglit/arb_sparse_buffer-basic using Zink.
Testcase: piglit/arb_sparse_buffer-basic
Issue: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10220
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26410>
I need to add some sparse-related checks that require having the
anv_buffer and anv_image, and putting them directly inside
anv_queue_submit_sparse_bind_locked() doesn't feel like the right
thing to do. Here we change the interface so now we have
anv_sparse_bind_buffer() and anv_sparse_bind_image_opaque() as the
main interface into anv_sparse.c, so they both can call the lower
level anv_sparse_bind_resource_memory() function.
In the next patch we'll be adding changing the code of the functions
we just created, justifying their addition.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26410>
Xe2+ changed the msaa mapping for 2D/3D Tile64 surfaces, introduce a
Xe2+ specific enum to handle this change.
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27113>
Launch with :
$ MESA_VK_TRACE=rmv MESA_VK_TRACE_TRIGGER=/tmp/trig ./my_app
In another terminal, trigger a capture :
$ touch /tmp/trig
The application with create a snapshot and print out :
RMV capture saved to '/tmp/my_app_2024.01.19_10.56.33.rmv'
Then just open it with RMV :
./RadeonMemoryVisualizer /tmp/my_app_2024.01.19_10.56.33.rmv
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26843>
The standard block shapes (and by extension, the tiling formats they
require) are simply incompatible with getting a 2D view of a 3D image.
I couldn't find in the Vulkan spec anything related to what are the
expectations when trying to use both at the same time.
So here we "document" that this case is known non-standard. Please
notice that since we report residencyStandard3DBlockShape as true we
were actually supposed to support this case, but I can't see how this
would be possible, so set is_known_nonstandard_format to true so we
can avoid the assert() that comes right after.
Fixes the following when using Zink:
KHR-GL46.sparse_texture_tests.SparseTextureAllocation
Also "moves forward" the following test on Zink, so it now hits a
different assertion:
KHR-GL46.sparse_texture_tests.SparseTextureCommitment
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26454>
If the size here is not a multiple of the granularity (64kb) then
we'll miss our "pages" estimation by 1. We could fix this with
DIV_ROUND_UP() or by simply putting a "+1" there, but the upper layers
should now be preventing this case so let's just put the assertion
here.
Previously it was possible to hit this case with Zink by running
under certain conditions piglit/arb_sparse_buffer-basic.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26454>
From the spec:
"Resources can be bound at some defined (sparse block) granularity."
"The sparse block size in bytes for sparse buffers and
fully-resident images is reported as
VkMemoryRequirements::alignment. alignment represents both the
memory alignment requirement and the binding granularity (in bytes)
for sparse resources."
Not only the upper layer (the Spec) doesn't allow this, the lower
layers (both the vm_bind ioctl and TR-TT) also work on a granularity.
Just check for this case and return an error.
Before this check, what would happen was:
- for the vm_bind backend, the vm_bind ioctl would fail
- for the TR-TT backend, we'd understimate l1_binds_capacity and
fail an assertion, or we'd just silently bind 64kb instead of the
original size
Currently, some Zink tests such as piglit/arb_sparse_buffer-basic can
trigger this behavior, but we're working to fix Zink for this case
(and that commit may be merged before this one).
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26454>
Regarding supporting these formats, the spec says:
"A sparse image created using VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT
supports all non-compressed color formats with power-of-two element
size that non-sparse usage supports. Additional formats may also be
supported and can be queried via
vkGetPhysicalDeviceSparseImageFormatProperties.
VK_IMAGE_TILING_LINEAR tiling is not supported."
Regarding the formats themselves, the spec says:
"VK_FORMAT_B8G8R8G8_422_UNORM specifies a four-component, 32-bit
format containing a pair of G components, an R component, and a B
component, collectively encoding a 2×1 rectangle of unsigned
normalized RGB texel data. One G value is present at each i
coordinate, with the B and R values shared across both G values and
thus recorded at half the horizontal resolution of the image. This
format has an 8-bit B component in byte 0, an 8-bit G component for
the even i coordinate in byte 1, an 8-bit R component in byte 2,
and an 8-bit G component for the odd i coordinate in byte 3. This
format only supports images with a width that is a multiple of two.
For the purposes of the constraints on copy extents, this format is
treated as a compressed format with a 2×1 compressed texel block."
Since these formats are to be considered compressed 2x1 blocks and we
don't necessarily have to support non-compressed formats that
non-sparse support, we can claim them as not supported with sparse.
In addition to all of that, if you look at isl_gfx125_filter_tiling()
you'll see that we don't even support Tile64 for these formats, so
sparse residency (i.e., non-opaque image binds) doesn't really make
sense for them yet.
The Vulkan spec defines 4 other YCBCR "2x1 compressed" formats like
the ones we have in this commit, but we don't support them even
without sparse, so there's no reason to check them here.
A recent change in VK-GL-CTS made tests that use these formats go from
unsupported to failures:
7ecc7716a983 ("Do not use and check for STORAGE image support, when
it is not used in the test")
This commit "fixes" the following VK-GL-CTS failures (by making them
return NotSupported):
dEQP-VK.sparse_resources.image_block_shapes.2d.b8g8r8g8_422_unorm.samples_1
dEQP-VK.sparse_resources.image_block_shapes.2d.g8b8g8r8_422_unorm.samples_1
dEQP-VK.sparse_resources.image_block_shapes.2d_array.b8g8r8g8_422_unorm.samples_1
dEQP-VK.sparse_resources.image_block_shapes.2d_array.g8b8g8r8_422_unorm.samples_1
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
We need to wait for the batches to complete before we return the BOs
to the pool. We were previously doing this completely synchronously,
which made the code unnecessarily wait. Now we have a timeline syncobj
that signals completion of the previous BOs, so sometimes we check
where we are in the timeline and then return the BOs that we know are
unused.
This, in addition to the previous patch that made us wait for the
other syncobjs through the execbuf ioctl instead of through the CPU,
makes TR-TT batches at least an order of magnitude faster. Still, I
don't think we'll notice any changes in games's FPS as they don't bind
sparse resources that often.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Pass them as part of the TR-TT batch. This is what a lot of the
previous commits were building up to.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Move waiting/signaling to the backends so we can fix each backend
separately.
As I write this patch the vm_bind backend is back to using synchronous
vm_binds so we can't pass syncobjs to the synchronous vm_bind ioctl
anymore. We'll need more discussions and possibly some rework before
we go back to asynchronous vm_binds. This commit should allow us to
fix the TR-TT backend in the next commit and leave vm_bind for later.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
If we're going to move syncobj waiting/signaling down to the backend
we're going to need a queue to signal as lost in case those operations
fail.
In some places of the stack we don't have a queue available, such as
when we're creating or destroying resources. For those, for vm_bind
cases we don't use the queue for anything so passing it as NULL is
fine. For TR-TT we are already using device->trtt.queue.
For TR-TT specifically this also means we're going to start using the
actual queues from the call stack instead of trtt->queue, but that
shouldn't make any difference since we only ever have one queue.
Still, this is more technigally correct.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Our ultimate goal is to have the backend functions deal with the wait
and signal syncobjs instead of waiting for them on the CPU inside
anv_queue_submit_sparse_bind_locked(). For that, we'll need waits and
signals parameters to be passed all the way to the backend functions
that actually make the submission, and this is what this patch does,
through struct anv_sparse_submission.
This patch just deals with passing the parameters to the functions,
nothing is using the new variables yet. There should be no functional
changes here. The goal here is to make code review easier.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Currently, a single vkQueueBindSparse() call may lead to multiple bind
calls in the backend (either a vm_bind ioctl or a command submission
that updates the TR-TT page tables). These operations can be quite
slow so it's better for us if we try to emit as few of them as
possible.
On top of that, this gives our "just extend the last operation's size
if possible" code a little more chance to act and save us real time.
Our ultimate goal here is to also pass submit->waits and
submit->signals to the backend so we can avoid doing CPU waits, so
having a single call to the backend helps simplify things a little
too, and we just created the structure to carry these extra pointers
forward.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Having it helped us printing the resource offset, which made debugging
some situations easier. The problem is that we want to rework the code
a little bit and we won't have a 'sparse' struct anymore to pass
around. Since it's all debug code drop it for now so it doesn't get in
the way of the rework. If we need it later we can find a way to add it
back, or we find another way to print the value.
Drive-by drop the DEBUG_SPARSE check that's already in the caller.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
TR-TT is a hardware feature supported by both i915.ko and xe.ko, which
means we can now finally have Sparse Resources on i915.ko and we also
have 2 options for xe.ko (and whatever is the best should be the
default).
In this patch we use batch commands to write the page tables and
forever keep them in device memory. We maintain a mirror of both the
L3 and and L2 tables because that helps us never having to read the
tables that are in device memory.
We still have some things to improve, but with this commit, workloads
that didn't work at all due to the lack of sparse resources should
at least run.
This is still all disabled by default in i915.ko, you can turn it on
by exporting ANV_SPARSE=1 before launching the applications. For
xe.ko, switch the default with ANV_SPARSE_USE_TRTT=1.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
This function will be able to transparently handle sparse binding
regardless of the backend: vm_bind ioctls or TR-TT. For now we only
support the vm_bind ioctls, but soon we'll have anv_sparse_bind_trtt()
as an option.
It is important to notice that even backends that support the vm_bind
ioctl may choose to do Sparse binding via TR-TT, that's why we're
adding the indirection at this specific point.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26036>
When it's a NULL bind we always set the bo_offset (aka memory offset)
to zero, so we have to avoid the "bind.offset == prev.offset + size"
check.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26036>
If the next bind is just an extension of the previous one, join both
in the same bind operation. Due to how mip levels are laid in memory,
this can only happen for mip level 0.
As of today xe.ko doesn't try to join contiguous operations for us.
Due to how rebinds happen each additional rebind operation may end up
resulting in many extra things done, so these simple checks end up
saving us a lot of cycles the Kernel would otherwise waste. This will
be true even after we issue all binds in a single ioctl.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26036>
This pollutes stderr a lot, but I've used it countless times while
developing this code.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23045>
Game testing shows it's common for this operation to result in
multiple bind regions, so try to use a single ioctl when we can.
Actual testing reveals 136 shader-related tests fail when we actually
do this, so for now keep doing a single bind per ioctl while leaving a
very easy way to the desired behavior when we figure this out.
It should also be possible to go even higher-level and do this at the
anv_queue_submit_sparse_bind_locked() layer, but that should happen in
future commits.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23045>
This giant patch implements a huge chunk of the Vulkan Sparse
Resources API. I previously had this as a nice series of many smaller
patches that evolved as the xe.ko added more features, but once I was
asked to squash some of the major reworks I realized I wouldn't be
able easily rewrite history, so I just squased basically the whole
series into a giant patch. I may end up splitting this again later if
I find a way to properly do it.
If we want to support the DX12 API through vkd3d we need to support
part of the the Sparse Resources API. If we don't, a bunch of Steam
games won't work.
For now we only support the xe.ko backend, but the vast majority of
the code is KMD-independent and so an i915.ko implementation would use
most of what's here, just extending the part that binds and unbinds
memory.
v2+: There's no way to sanely track the version history of this patch
in this commit message. Please refer to Gitlab.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23045>
