This reworks the intel_compute_pixel_hash_table_nway() pixel pipe
hashing table computation helper to handle cases where some pixel
pipes have processing power different from the others, this is helpful
for Gfx12.7+ platforms where there are pixel pipes with 1 DSS as well
as pixel pipes with 2 DSSes, which currently can lead to a serious
performance bottleneck in the pixel pipes with lower processing power.
In order to avoid such a load imbalance the
intel_compute_pixel_hash_table_nway() function will now take two pixel
pipe bitsets instead of one: Pixel pipes enabled on both bitsets will
appear with twice the frequency on the table as pixel pipes which only
appear on one bitset. See the comments below for more details on the
algorithm used to construct a pixel hashing table with the desired
properties.
With this change rendering performance improves by about 25% on a
fused MTL platform -- The list of specific configs this is expected to
show an improvement on is not included here since the list is rather
long and some of the configs may still be embargoed or may never be
productized, but in order to find out whether your Gfx12.7+ device
could be affected by this you can check the output of the
intel_dev_info tool from the Mesa tree and see if there are multiple
"pixel pipe" entries with different DSS count. That isn't expected to
occur on any DG2 configuration, only on MTL+ platforms, so this change
should have no effect at all on DG2 (it's easy to convince oneself
that it won't since for DG2 mask1 should equal mask2 so mask2 will be
set to zero at the beginning of intel_compute_pixel_hash_table_nway()
and the new swzx[] permutation will be set to the identity).
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26266>
Sync xe_drm.h with commit 3b8183b7efad ("drm/xe/uapi: Be more specific
about the vm_bind prefetch region").
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26238>
DRM_XE_EXEC_QUEUE_SET_PROPERTY is the offset,
while DRM_IOCTL_XE_EXEC_QUEUE_SET_PROPERTY is the real number.
Signed-off-by: Shuicheng Lin <shuicheng.lin@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26253>
Regarding supporting these formats, the spec says:
"A sparse image created using VK_IMAGE_CREATE_SPARSE_RESIDENCY_BIT
supports all non-compressed color formats with power-of-two element
size that non-sparse usage supports. Additional formats may also be
supported and can be queried via
vkGetPhysicalDeviceSparseImageFormatProperties.
VK_IMAGE_TILING_LINEAR tiling is not supported."
Regarding the formats themselves, the spec says:
"VK_FORMAT_B8G8R8G8_422_UNORM specifies a four-component, 32-bit
format containing a pair of G components, an R component, and a B
component, collectively encoding a 2×1 rectangle of unsigned
normalized RGB texel data. One G value is present at each i
coordinate, with the B and R values shared across both G values and
thus recorded at half the horizontal resolution of the image. This
format has an 8-bit B component in byte 0, an 8-bit G component for
the even i coordinate in byte 1, an 8-bit R component in byte 2,
and an 8-bit G component for the odd i coordinate in byte 3. This
format only supports images with a width that is a multiple of two.
For the purposes of the constraints on copy extents, this format is
treated as a compressed format with a 2×1 compressed texel block."
Since these formats are to be considered compressed 2x1 blocks and we
don't necessarily have to support non-compressed formats that
non-sparse support, we can claim them as not supported with sparse.
In addition to all of that, if you look at isl_gfx125_filter_tiling()
you'll see that we don't even support Tile64 for these formats, so
sparse residency (i.e., non-opaque image binds) doesn't really make
sense for them yet.
The Vulkan spec defines 4 other YCBCR "2x1 compressed" formats like
the ones we have in this commit, but we don't support them even
without sparse, so there's no reason to check them here.
A recent change in VK-GL-CTS made tests that use these formats go from
unsupported to failures:
7ecc7716a983 ("Do not use and check for STORAGE image support, when
it is not used in the test")
This commit "fixes" the following VK-GL-CTS failures (by making them
return NotSupported):
dEQP-VK.sparse_resources.image_block_shapes.2d.b8g8r8g8_422_unorm.samples_1
dEQP-VK.sparse_resources.image_block_shapes.2d.g8b8g8r8_422_unorm.samples_1
dEQP-VK.sparse_resources.image_block_shapes.2d_array.b8g8r8g8_422_unorm.samples_1
dEQP-VK.sparse_resources.image_block_shapes.2d_array.g8b8g8r8_422_unorm.samples_1
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
On i915.ko we don't have the vm_bind ioctl, so sparse requires TR-TT.
Unfortunately, on gfx < 20 TR-TT is not compatible with non-render
queues, so we have to disable those when sparse is enabled. Notice
that although we don't have TR-TT for non-render queues on gfx >= 20,
vm_bind is the default, and it doesn't have this restriction.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
We need to wait for the batches to complete before we return the BOs
to the pool. We were previously doing this completely synchronously,
which made the code unnecessarily wait. Now we have a timeline syncobj
that signals completion of the previous BOs, so sometimes we check
where we are in the timeline and then return the BOs that we know are
unused.
This, in addition to the previous patch that made us wait for the
other syncobjs through the execbuf ioctl instead of through the CPU,
makes TR-TT batches at least an order of magnitude faster. Still, I
don't think we'll notice any changes in games's FPS as they don't bind
sparse resources that often.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
At this moment this patch won't buy us anything since we're already
being completely synchronous, but the next patch is going to change
this and so queue->sync will start making sense.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
For now it just wraps the bo and size, so there's really no value to
having it. In the next commit we'll add more elements to the struct.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Pass them as part of the TR-TT batch. This is what a lot of the
previous commits were building up to.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
When sparse images are being used, applications normally use
non-opaque binds and leave opaque binds just for the miptail part.
Since miptails are always at the end of the array layers, processing
the opaque binds after processing the non-opaque binds increases the
chance that anv_sparse_submission_add() will join the miptail bind
operation with the last non-opaque opreration, especially if the user
is trying to bind the last few non-miptail levels and the miptail in
the same vkQueueBindSparse opration.
In the real world this case does happen, so we're able to save a bind
operation every once in a while in Steam games.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Move waiting/signaling to the backends so we can fix each backend
separately.
As I write this patch the vm_bind backend is back to using synchronous
vm_binds so we can't pass syncobjs to the synchronous vm_bind ioctl
anymore. We'll need more discussions and possibly some rework before
we go back to asynchronous vm_binds. This commit should allow us to
fix the TR-TT backend in the next commit and leave vm_bind for later.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Don't pass it as a parameter when it's also part of a struct. Have to
touch 9 files just for that...
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
If we're going to move syncobj waiting/signaling down to the backend
we're going to need a queue to signal as lost in case those operations
fail.
In some places of the stack we don't have a queue available, such as
when we're creating or destroying resources. For those, for vm_bind
cases we don't use the queue for anything so passing it as NULL is
fine. For TR-TT we are already using device->trtt.queue.
For TR-TT specifically this also means we're going to start using the
actual queues from the call stack instead of trtt->queue, but that
shouldn't make any difference since we only ever have one queue.
Still, this is more technigally correct.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Our ultimate goal is to have the backend functions deal with the wait
and signal syncobjs instead of waiting for them on the CPU inside
anv_queue_submit_sparse_bind_locked(). For that, we'll need waits and
signals parameters to be passed all the way to the backend functions
that actually make the submission, and this is what this patch does,
through struct anv_sparse_submission.
This patch just deals with passing the parameters to the functions,
nothing is using the new variables yet. There should be no functional
changes here. The goal here is to make code review easier.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Currently, a single vkQueueBindSparse() call may lead to multiple bind
calls in the backend (either a vm_bind ioctl or a command submission
that updates the TR-TT page tables). These operations can be quite
slow so it's better for us if we try to emit as few of them as
possible.
On top of that, this gives our "just extend the last operation's size
if possible" code a little more chance to act and save us real time.
Our ultimate goal here is to also pass submit->waits and
submit->signals to the backend so we can avoid doing CPU waits, so
having a single call to the backend helps simplify things a little
too, and we just created the structure to carry these extra pointers
forward.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Having it helped us printing the resource offset, which made debugging
some situations easier. The problem is that we want to rework the code
a little bit and we won't have a 'sparse' struct anymore to pass
around. Since it's all debug code drop it for now so it doesn't get in
the way of the rework. If we need it later we can find a way to add it
back, or we find another way to print the value.
Drive-by drop the DEBUG_SPARSE check that's already in the caller.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
Same as the L1 case, but this one deals with 64bit entry addresses and
pte addresses.
Consecutive L3/L2 writes are much rarer than L1 writes since they
require some pretty big buffers, but we can still those cases in the
wild. I just don't think any change will be noticeable though.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
If the addresses are sequential, we can emit only a single
MI_STORE_DATA_IMM instruction. This is a very common case, it should
save us some space: 4 bytes per extra_write.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
When using vm_bind (not TR-TT), in practice sparse addresses will be
allocated from the high_heap, so narrow down the available
sparseAddressSpaceSize from the whole address space to the part we can
actually allocate things from.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
TR-TT is a hardware feature supported by both i915.ko and xe.ko, which
means we can now finally have Sparse Resources on i915.ko and we also
have 2 options for xe.ko (and whatever is the best should be the
default).
In this patch we use batch commands to write the page tables and
forever keep them in device memory. We maintain a mirror of both the
L3 and and L2 tables because that helps us never having to read the
tables that are in device memory.
We still have some things to improve, but with this commit, workloads
that didn't work at all due to the lack of sparse resources should
at least run.
This is still all disabled by default in i915.ko, you can turn it on
by exporting ANV_SPARSE=1 before launching the applications. For
xe.ko, switch the default with ANV_SPARSE_USE_TRTT=1.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25512>
The driver already need to track this WA for blorp. We can completely
remove any blorp code dealing with this and instead have the flush
required by the workaround be combined with potential other flushes
the driver already has to insert before blorp operations.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26247>
ANV_CMD_DIRTY_PIPELINE also includes reprogramming of
3DSTATE_PUSH_CONSTANT_ALLOC_* instructions.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 50f6903bd9 ("anv: add new low level emission & dirty state tracking")
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26247>
There are rendering issues with FCV on DG2 and Unreal engine 5.1,
patch adds option to disable fcv in drirc.
Cc: mesa-stable
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26169>
Patch fxes ESO shadow pass ground corruption on Arc A750. In the colour
pass where the rendering corruption first appears, the depth resource
was used as a "PS - Texture". Immediately afterwards there's a Barrier
where it goes from
VK_IMAGE_LAYOUT_DEPTH_STENCIL_READ_ONLY_OPTIMAL =>
VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL
immediately following that there's a Clear from vkCmdBeginRendering
which appears to be a HiZ clear. Things work when using AUX_USAGE_HIZ
but AUX_USAGE_HIZ_CCS_WT (XXX: and AUX_USAGE_HIZ_CCS?) doesn't work.
current thinking is this is related to 14015264727 where we had to add
HDC and DC flushes to CCS and MCS fast clears. Maybe HiZ clears with
CCS also have similar problems? The docs don't appear to indicate that
but the docs were also wrong for color clears until recently...
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9277
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9444
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22717>
v2: Add the missing PIPE_CONTROL enable/disable bits
v3: Use new I915_PROTECTED_CONTENT_DEFAULT_SESSION
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8064>
v2: Add assert on VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT in vkMapMemory
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8064>
On Blitter engine, we don't support PIPE_CONTROL, we have to update
memory locations using the MI_FLUSH_DW command.
v2:
- Handle video queue (Lionel)
Fixes: 056b0cb87f ("anv: add video engine support in various places")
Fixes: 5112b42146 ("anv: Handle end of pipe with MI_FLUSH_DW on transfer queue")
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26121>
Commit 50c29e1ffa ("anv: simplify buffer address+size loads from descriptor buffer")
is making use of AuxiliarySurfaceBaseAddress field to store buffer
lenght as it was not used but a LNL workaround will make use of it
so we need to bring back this non optimized version of
build_load_render_surface_state_address().
There is some conflicts so a simple revert do not works.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26152>
Add anv_get_default_cpu_clock_id() to return the default cpu clock
id to be used in the begin and end time captures of
anv_GetCalibratedTimestampsEXT().
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24591>
Refactor some of the blorp code into something more generic that we can
reuse for functionality needed post 3DPRIMITIVE emission.
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25039>
Now that alloc_flags is stored in anv_bo we can get rid of is_external,
has_fixed_address and has_client_visible_address flags that can
be inferred from alloc_flags.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
When anv_device_map_bo() is called from anv_device_alloc_bo() it gets
VkMemoryPropertyFlags set to 0 so it ends up with a write-combine
caching for integrated platforms with LLC, see 'if (!(property_flags &
VK_MEMORY_PROPERTY_HOST_CACHED_BIT)))'.
Current approach also has issues when mapping with anv_MapMemory2KHR()
as it would not have information to know that BO is a scanout.
It was also not properly calculating mmap mode for platforms with PAT
uAPI before "anv: Change default PAT entry to WC".
So here storing alloc_flags to anv_bo so there is no mismatches
between different code paths then using it to properly
calculate the mmap mode.
alloc_flags in anv_bo will also be used to calculate PAT index in
future patches.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
i915 mmap_calc_flags() is calculating WC caching for all MTL memory
types.
It will be fixed in the next patch but doing so causes tests to
fail due to incoherency in BOs not allocated with
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT.
So here switching the default/non-coherent BO allocation to a WC
PAT entry.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
PAT entry will be needed to calculate mmap mode and also will be
used during BO creating in Xe KMD when PAT uAPi lands.
So here moving the PAT entry selection to common code.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
Integrated GPUs almost always works with write-back caching(only
scanout and external bos works in write-combine) but in platforms
without LLC the coherency is broken if not explict asked to KMD.
vkFlushMappedMemoryRanges and vkInvalidateMappedMemoryRanges()
don't do any flushing or invalidate for memory allocated with
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT.
So if an application asked for a memory coherent, the
ANV_BO_ALLOC_SNOOPED flag needs to be set in alloc_flags and that
will be passed to KMD backends to properly ask to KMD for coherent
buffer.
The other chunk here removes the assert(alloc_flags & ANV_BO_ALLOC_MAPPED),
that is needed otherwise application can't ask for a coherent and
mapped memory.
Tried to find a reason for that assert in git history but did not
found what was the reasoning of this assert.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
