If we're going to move syncobj waiting/signaling down to the backend
we're going to need a queue to signal as lost in case those operations
fail.
In some places of the stack we don't have a queue available, such as
when we're creating or destroying resources. For those, for vm_bind
cases we don't use the queue for anything so passing it as NULL is
fine. For TR-TT we are already using device->trtt.queue.
For TR-TT specifically this also means we're going to start using the
actual queues from the call stack instead of trtt->queue, but that
shouldn't make any difference since we only ever have one queue.
Still, this is more technigally correct.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Our ultimate goal is to have the backend functions deal with the wait
and signal syncobjs instead of waiting for them on the CPU inside
anv_queue_submit_sparse_bind_locked(). For that, we'll need waits and
signals parameters to be passed all the way to the backend functions
that actually make the submission, and this is what this patch does,
through struct anv_sparse_submission.
This patch just deals with passing the parameters to the functions,
nothing is using the new variables yet. There should be no functional
changes here. The goal here is to make code review easier.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
TR-TT is a hardware feature supported by both i915.ko and xe.ko, which
means we can now finally have Sparse Resources on i915.ko and we also
have 2 options for xe.ko (and whatever is the best should be the
default).
In this patch we use batch commands to write the page tables and
forever keep them in device memory. We maintain a mirror of both the
L3 and and L2 tables because that helps us never having to read the
tables that are in device memory.
We still have some things to improve, but with this commit, workloads
that didn't work at all due to the lack of sparse resources should
at least run.
This is still all disabled by default in i915.ko, you can turn it on
by exporting ANV_SPARSE=1 before launching the applications. For
xe.ko, switch the default with ANV_SPARSE_USE_TRTT=1.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
v2: Add assert on VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT in vkMapMemory
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8064>
When anv_device_map_bo() is called from anv_device_alloc_bo() it gets
VkMemoryPropertyFlags set to 0 so it ends up with a write-combine
caching for integrated platforms with LLC, see 'if (!(property_flags &
VK_MEMORY_PROPERTY_HOST_CACHED_BIT)))'.
Current approach also has issues when mapping with anv_MapMemory2KHR()
as it would not have information to know that BO is a scanout.
It was also not properly calculating mmap mode for platforms with PAT
uAPI before "anv: Change default PAT entry to WC".
So here storing alloc_flags to anv_bo so there is no mismatches
between different code paths then using it to properly
calculate the mmap mode.
alloc_flags in anv_bo will also be used to calculate PAT index in
future patches.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
Estimating the batch space required can be tricky because of all the
workarounds. So implement chaining of batches like we do for command
buffers.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26087>
It was never actually async as it was doing a DRM_IOCTL_SYNCOBJ_WAIT
right after DRM_IOCTL_XE_VM_BIND but it was required to allow the
partial binds required by sparse.
But it is now fixed and we can switch back to sync vm bind.
In future we will switch back to async vm bind to improve performance
but this time it will be properly implemented.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25300>
Sync xe_drm.h with commit xxxxx ("drm/xe/uapi: Fix naming of XE_QUERY_CONFIG_MAX_EXEC_QUEUE_PRIORITY").
One not so straght forward change is that sync VM binds now don't
require a syncobj anymore, the uAPI will return as soon the VM bind
operations are done.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25300>
The next patch is going to introduce some locking that needs to happen
before the submission to the backend.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24744>
The bo_flags are i915 specific and should not be handled in common
code, so here adding it to backend as it is in the hot-path.
There still i915 bo_flags handling in anv_device_import_bo() that
will be handled in the next patch.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25044>
Don't hide failures, we have xe.ko bugs related to that, such as:
https://gitlab.freedesktop.org/drm/xe/kernel/-/issues/496
The bind ioctl may fail if the application does something wrong, but
the wait really should never fail.
v2: Don't print an error message (Lionel).
Reviewed-by: José Roberto de Souza <jose.souza@intel.com> (v1)
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24681>
Our sparse implementation will require us to issue partial unbinds,
but partial unbinds are not supported in synchronous vm_bind ioctls,
requiring us to to have our VM be marked with the ASYNC flag. This is
not properly documented and is subject to change in the next
iterations of the API.
Error handling with async binds is also not documented anywhere and is
being actively discussed in the mailing lists, so whatever we decide
to implement here is likely to end up changing in a few weeks. Also, I
haven't seen these errors happening in the real world, so for now
they're a very corner case. So for now just foward errors to
user-space and hope things work.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24681>
For Sparse Resources we need to be able to specify the address, size
and offsets and we also want to be able to issue multiple binds at the
same time. Extend xe_vm_bind_op() to handle those cases and add
the new vfunc.
v2:
- use STACK_ARRAY() (Lionel)
- no more need to work around xe.ko bug that was fixed (José)
Reviewed-by: José Roberto de Souza <jose.souza@intel.com> (v1)
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24681>
The only driver that has a vm_bind ioctl is xe.ko, and its vm_bind
ioctl is not called GEM vm_bind, it's just DRM_IOCTL_XE_VM_BIND
(without GEM anywhere). Back when i915.ko was going to have a vm_bind
ioctl it had GEM on its name, so I guess that's how "gem" appeared in
the naming here, but now nothing does, so let's get rid of it.
Also, these vfuncs we have are specifically made to bind and unbind
whole BOs, so rename them to vm_bind_bo() and vm_unbind_bo() in order
to try to clarify what they mean. The goal is to add a more generic
vm_bind() later that can do anything.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24681>
We need to synchronize main (CCS/BCS) and companion rcs batch, so let's
create an empty batch and make both the batches (CCS/BCS) and companion
RCS batch wait on empty sync batch and signal the fence.
Reason to execute the empty batch is we need to make sure the companion
RCS batch finish as soon as the CCS/BCS batch finish. Preemption could
prevent the companion RCS batch execution and we might end up destroying
the CCS/BCS batch before companion RCS finishes.
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23661>
Add all the wait fences from the main (CCS/BCS) command buffer to the
companion RCS command buffer so that the companion RCS batch starts at
the same time as the main (CCS/BCS) batch.
v2:
- Drop unncessary flush (Jose)
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23661>
We need to create companion RCS engine when there is CCS/BCS engine
creation requested.
v2:
- Factor out anv_xe_create_engine code in create_engine (Jose)
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23661>
Sync with commit f16c04291100 ("drm/xe: Rename engine to exec_queue").
With that Iris and ANV had some major renames that were done manually
as there is to many "engine" in non-related code.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24476>
This is required to support discrete GPUs placed in systems with large
PCI bar or resizeble PCI bar not available or disabled.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23781>
Xe KMD only requires userptr to be bound to VM, so here reusing
workaround_bo->gem_handle id to all userptr bos in Xe version of
gem_create_userptr(). The Xe version of gem_close() will make sure
that workaround_bo->gem_handle is not closed when userptr bos
are closed.
With the same gem_handle for all userptr bos, it was also necessary
skip the anv_device_lookup_bo() and manually allocate memory to store
anv_bo in host heap memory, what lead to some small changes in
anv_device_release_bo() as well.
The remaining changes are the support to VM bind userptr bos and the
gem_vm_bind() call in anv_device_import_bo_from_host_ptr().
Fixes: dEQP-VK.memory.external_memory_host*
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23787>
struct anv_bo will be needed in the next patch to properly handle
closure of userptr bos.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23787>
Xe support of userptr will be implemented in the next patch,
this is just moving the i915 and stub functions to KMD backend.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23787>
utrace submits can either have a batch or not.
When there is a batch, the utrace vk_sync is signaled by the utrace
batch (because utrace does a timestamp buffer copy using its own
batch). When there is no batch, the utrace vk_sync should be signaled
by the application batch (no timestamp copy required, utrace can read
the timestamps when the application batch has completed).
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: fdea48df5e ("anv: Implement Xe version of anv_queue_exec_locked() and queue_exec_trace()")
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24085>
Due the lack of APIs to set mmap modes, Xe KMD can't support the same
memory types as i915.
So here adding a i915 and Xe function to set memory types supported
by each KMD.
Iris function iris_xe_bo_flags_to_mmap_mode() has a table with all the
mmaps modes of each type of placement.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22906>
MTL and newer platforms on Xe kmd will have engines with gt_id != 0.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22477>
Imported buffers may be created in a device with different
memory alignment and this can cause vm bind to fail because bo
size can be smaller than the calculated vm bind range using the
importer device memory alignment.
So here adding actual_size to anv_bo, this will be set with the actual
size of the bo allocated by kmd for bos allocate in the current device.
For other bo the lseek or the Vulkan API size will be used.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22219>
A new property was added to drm_xe_query_config with the max engine
priority for running process, so we can use it directly on
anv_xe_physical_device_get_parameters() and nuke
anv_xe_physical_device_max_priority_update().
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22112>
To replace the hard-coded values.
Not fully importing gpu_scheduler.h because it includes several Linux
specific headers that will make the life of anyone porting Xe to
other Unix harder.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22112>
As Xe requires engine info, it is not possible to fetch max priority
from anv_physical_device_get_parameters() call chain, so here
anv_xe_physical_device_max_priority_update() was added.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22058>
Xe engine is not equal to hardware engine, it is just a submission
queue that will be scheduled in the hardware engine during process
time slice of the GPU.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22058>
Without alignment vm bind will fail and during gem buffer creation
size also need to be aligned otherwise the range in vm bind can be
bigger than allocated size for smem.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21972>
External(imported or exported) objects needs to have vm_id set to 0.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21885>
Xe KMD does a special caching handling for buffers that will be
scanout to display, so that is why it needs a flag set during
allocation.
Checking if VK_STRUCTURE_TYPE_WSI_MEMORY_ALLOCATE_INFO_MESA
is available in AllocateMemory() and marking the buffer as scanout.
All WSI code paths but one sets
VK_STRUCTURE_TYPE_WSI_MEMORY_ALLOCATE_INFO_MESA.
The only one that doesn't requires that WSI is initialize with
wsi_device_options.sw_device = true to be executed, what is not the
case for ANV.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21885>
Not using it yet, that will be done in the next patch.
Xe only supports submission using VM.
For i915 the backend functions are just a noop.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21698>
