When anv_device_map_bo() is called from anv_device_alloc_bo() it gets
VkMemoryPropertyFlags set to 0 so it ends up with a write-combine
caching for integrated platforms with LLC, see 'if (!(property_flags &
VK_MEMORY_PROPERTY_HOST_CACHED_BIT)))'.
Current approach also has issues when mapping with anv_MapMemory2KHR()
as it would not have information to know that BO is a scanout.
It was also not properly calculating mmap mode for platforms with PAT
uAPI before "anv: Change default PAT entry to WC".
So here storing alloc_flags to anv_bo so there is no mismatches
between different code paths then using it to properly
calculate the mmap mode.
alloc_flags in anv_bo will also be used to calculate PAT index in
future patches.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
PAT entry will be needed to calculate mmap mode and also will be
used during BO creating in Xe KMD when PAT uAPi lands.
So here moving the PAT entry selection to common code.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
Integrated GPUs almost always works with write-back caching(only
scanout and external bos works in write-combine) but in platforms
without LLC the coherency is broken if not explict asked to KMD.
vkFlushMappedMemoryRanges and vkInvalidateMappedMemoryRanges()
don't do any flushing or invalidate for memory allocated with
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT.
So if an application asked for a memory coherent, the
ANV_BO_ALLOC_SNOOPED flag needs to be set in alloc_flags and that
will be passed to KMD backends to properly ask to KMD for coherent
buffer.
The other chunk here removes the assert(alloc_flags & ANV_BO_ALLOC_MAPPED),
that is needed otherwise application can't ask for a coherent and
mapped memory.
Tried to find a reason for that assert in git history but did not
found what was the reasoning of this assert.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
mmap mode information will be used to properly calculate the mmap flags
in the i915 mmap uAPI and also will be used for BO creation when the
PAT uAPI lands in Xe KMD.
Xe KMD will also require the coherency mode during the BO creation.
So to avoid information duplication, adding this information to
intel_device_info platform entries.
No changes in behavior here.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
Estimating the batch space required can be tricky because of all the
workarounds. So implement chaining of batches like we do for command
buffers.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26087>
Some of the names are a bit confusing. The main change is to introduce
the "indirect_" prefix.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25955>
Stop allocating CCS at the end of some BOs. Anv no longer uses that
memory range.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jianxun Zhang <jianxun.zhang@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25003>
The next patch is going to introduce some locking that needs to happen
before the submission to the backend.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24744>
Imported bos already imported need special handling in i915.
That handling was moved to
anv_i915_gem_import_bo_alloc_flags_to_bo_flags() as the number of
imported bos is low.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25044>
The bo_flags are i915 specific and should not be handled in common
code, so here adding it to backend as it is in the hot-path.
There still i915 bo_flags handling in anv_device_import_bo() that
will be handled in the next patch.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25044>
We don't fill batch_len later when chaining batches. But that doesn't
seem to be a problem, I checked i915.ko and nothing Gen8+ seems to use
batch_len. The new xe.ko exec ioctl doesn't even ask for batch_len.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24681>
For Sparse Resources we need to be able to specify the address, size
and offsets and we also want to be able to issue multiple binds at the
same time. Extend xe_vm_bind_op() to handle those cases and add
the new vfunc.
v2:
- use STACK_ARRAY() (Lionel)
- no more need to work around xe.ko bug that was fixed (José)
Reviewed-by: José Roberto de Souza <jose.souza@intel.com> (v1)
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24681>
The only driver that has a vm_bind ioctl is xe.ko, and its vm_bind
ioctl is not called GEM vm_bind, it's just DRM_IOCTL_XE_VM_BIND
(without GEM anywhere). Back when i915.ko was going to have a vm_bind
ioctl it had GEM on its name, so I guess that's how "gem" appeared in
the naming here, but now nothing does, so let's get rid of it.
Also, these vfuncs we have are specifically made to bind and unbind
whole BOs, so rename them to vm_bind_bo() and vm_unbind_bo() in order
to try to clarify what they mean. The goal is to add a more generic
vm_bind() later that can do anything.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24681>
We need to synchronize main (CCS/BCS) and companion rcs batch, so let's
create an empty batch and make both the batches (CCS/BCS) and companion
RCS batch wait on empty sync batch and signal the fence.
Reason to execute the empty batch is we need to make sure the companion
RCS batch finish as soon as the CCS/BCS batch finish. Preemption could
prevent the companion RCS batch execution and we might end up destroying
the CCS/BCS batch before companion RCS finishes.
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23661>
Add all the wait fences from the main (CCS/BCS) command buffer to the
companion RCS command buffer so that the companion RCS batch starts at
the same time as the main (CCS/BCS) batch.
v2:
- Drop unncessary flush (Jose)
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23661>
We need to create companion RCS engine when there is CCS/BCS engine
creation requested.
v2:
- Factor out anv_xe_create_engine code in create_engine (Jose)
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23661>
This enables us to create more logical engines than HW engines are
available. This also brings the uAPI usage closer to what is happening
on Xe.
Rework: (Sagar)
- Correct exec_flag at the time of submission
- Handle device status check
- Set queue parameters
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23661>
In future patches, we will be creating a separate companion RCS engine
and each engine is created with it's own address space, and we really
don't want. CCS and RCS engine writes should be visible to each other in
order to get the wait/signal mechanism working.
v2:
- Move drm_i915_gem_context_create_ext_setparam out of if block (Lionel)
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23661>
This allow us to remove one more i915_drm.h include from code shared
by both backends.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23905>
struct anv_bo will be needed in the next patch to properly handle
closure of userptr bos.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23787>
Xe support of userptr will be implemented in the next patch,
this is just moving the i915 and stub functions to KMD backend.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23787>
The Vulkan specification indicates that if memory types have
properties which are a strict subset of another type's, then they
should appear before that memory type. Otherwise the specification
does not require a specific ordering of memory types.
But, it appears that Aztec Ruins and the Vulkan CTS make an assumption
that the first host-accessible memory type is host-coherent and select
it when they expect data written by the CPU to become visible without
calling vkFlushMappedMemoryRanges(), even though flushing is required
by the spec, which leads to misrendering and hangs on MTL platforms.
We found that other drivers also put a host-coherent, but not cached
memory type as the first host-accessible memory type, so let's do the
same in order to match the expectations of such broken applications.
Host-coherent uncached memory types are currently implemented with a
WC CPU map on non-LLC platforms, so there shouldn't be a huge
performance penalty from this: If an application intends to do heavy
R/W CPU access on a memory range it's expected to loop over the
available memory types and select one marked as host-cached -- If an
application fails to do that and simply selects the first available
type it seems more robust to stay on the safe side and give them a
host-coherent type rather than a cached one.
Rework:
* Jordan: Add initial explanation to body of commmit message.
* Curro: Add additional comments to commit message.
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22878>
We'll use the fact that the pool is aligned to 4Gb to limit the amount
of address computations to build the address in the shaders.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21645>
In i915 if the device has local memory it can only mmap bo with
I915_MMAP_OFFSET_FIXED, so all this set of ANV_BO_ALLOC_WRITE_COMBINE
were useless.
In Xe KMD there is no way to change mmap mode for all GPUs types.
So we can nuke bo->map_wc, ANV_BO_ALLOC_WRITE_COMBINE and related
dead code.
No changes in behavior expected here.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22483>
VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT is also set in all memory types of
integrated GPUs.
This flag means that memory will be allocated in the most efficient
place for the GPU to access, which is true in integrated GPUs.
However, this was causing ANV_BO_ALLOC_WRITE_COMBINE to be set in
integrated GPUs in the block right below when allocating in the non-cached memory type.
But the comment only talks about lmem, so to still keep the write
combine behavior for iGPUs it was used VkMemoryPropertyFlags in mmap_calc_flags().
Additionally, this was causing anv_bo.has_implicit_ccs to always be
set, which could change the expected behavior of
anv_BindImageMemory2() in MTL.
Fixes: fbd32a04da ("anv: add a third memory type for LLC configuration") added a new heap
Fixes: 582bf4d9f7 ("anv: flag BO for write combine when CPU visible and potentially in lmem")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22483>
Not doing so all the reads/writes go to the scratch page on i915.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: f9fa09ec92 ("anv/image: Add ANV_IMAGE_MEMORY_BINDING_PRIVATE")
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22957>
Due the lack of APIs to set mmap modes, Xe KMD can't support the same
memory types as i915.
So here adding a i915 and Xe function to set memory types supported
by each KMD.
Iris function iris_xe_bo_flags_to_mmap_mode() has a table with all the
mmaps modes of each type of placement.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22906>
Imported buffers may be created in a device with different
memory alignment and this can cause vm bind to fail because bo
size can be smaller than the calculated vm bind range using the
importer device memory alignment.
So here adding actual_size to anv_bo, this will be set with the actual
size of the bo allocated by kmd for bos allocate in the current device.
For other bo the lseek or the Vulkan API size will be used.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22219>
This also need to be executed in Xe kmd, so moving it to a function.
No changes in behavior intended here.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22171>
We want to use this for submission of traces outside command buffers,
so it won't just execute copies of timestamp buffers.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22006>
Not using it yet, that will be done in the next patch.
Xe only supports submission using VM.
For i915 the backend functions are just a noop.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21698>
Functions that are in hot paths will have a different treatment to
support i915 and Xe KMD.
Each KMD will have an anv_kmd_backend that will have the hot path
functions set, this way we can avoid branch prediction misses.
Other functions will gradually be moved to anv_kmd_backend.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20948>
The anv_execbuf_add_bo() function already sets the offsets for the
exec_objects. Since we're always using softpin and never using
relocations all these objects should have non-changing offsets, all
set during anv_bo creation and never changed. Not only we should not
change these offsets, we definitely don't change them between
anv_execbuf_add_bo() and this loop we're removing.
Previously, we'd have the offset set as -1 for BOs that had never been
submitted when we were not using softpin.
Notice that with games we can have several hundreds of BOs in this
array.
This loop was added by:
c5f7e1f5b4 ("anv: Delete relocation support from batch submission")
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20885>
Three reasons for that:
0. The operation we're doing here is actually a reallocation.
1. The newer code is, IMHO, easier to read.
2. Realloc has this property where sometimes, when possible, it will
expand your array without moving it somewhere else, so it doesn't
need to copy the memory contents, returning the original pointer
back to you. I did some analysis and while that case is not common,
it does happen sometimes in real world applications (I could see it
happening in Shootergame and Aztec Ruins, but not Dota 2), so we're
able to save a few CPU cycles.
v2: Rebase.
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20703>
