With the removal of DRM_IOCTL_XE_MMIO xe_gem_read_render_timestamp()
was always returning false but with DRM_XE_DEVICE_QUERY_ENGINE_CYCLES
it can be re implemented making use of
xe_gem_read_correlate_cpu_gpu_timestamp().
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24591>
The Intel Xe driver added the ability to do cpu/gpu timestamp
correlation giving a much better alignment of timestamps (we use to
have ~20us delta between the 2 samples, just because of the ioctl
barrier potentially sneaking in some work).
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24591>
Add anv_get_default_cpu_clock_id() to return the default cpu clock
id to be used in the begin and end time captures of
anv_GetCalibratedTimestampsEXT().
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24591>
This function will make use of Xe DRM_XE_DEVICE_QUERY_ENGINE_CYCLES by
returning correlate CPU ang GPU timestamp to be used by Intel drives.
This correlate timestamps gives us more accuracy.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24591>
This is lowered to 32-bit integer execution type by the regioning
lowering pass now, so the existing special casing is redudant for
Gfx12 and buggy for Xe2+, since SEL_EXEC is now emitted without
lowering for 64-bit integers.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25514>
Extended math instructions are now synchronized as in-order
instructions like other ALU operations, which is more efficient than
the out-of-order tracking we had to do in previous generations, and
avoids false dependencies introduced due to SBID aliasing.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25514>
Xe2 hardware has a "long" EU pipeline specifically for FP64
instructions, so these are handled as in-order instructions which
require RegDist synchronization. 64-bit integer instructions are now
handled by the normal integer pipeline, so the existing special-casing
inherited from ATS needs to be disabled.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25514>
This implements the extended 10-bit encoding of the software
scoreboard information used by Xe2 platforms. The new encoding is
different enough that there are few opportunities for sharing code
during translation to machine code, but the high-level tgl_swsb
representation remains roughly the same.
Among other changes the 10-bit SWSB format provides 5 bits worth of
SBID tokens (though they're only usable in large GRF mode) instead of
4 bits, the extended math pipeline is handled as an in-order (RegDist)
pipeline instead of as an out-of-order one, and the dual-argument
encodings support additional combinations of RegDist and SBID
synchronization modes. A new encoding is introduced for preventing
the accumulator hardware scoreboard from being updated, but this is
currently not needed.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25514>
The workaround applies specifically to Cube and Cube Arrays, so we can
still apply the optimization for the others.
Ideally we would like to pull opt_zero_samples logic into the lowering
sends -- to avoid adding a bit to communicate between passes. However
the texture coordinates for the LOGICAL backend instructions, which
are a common target for the optimization, are combined into offsets over
a single VGRF, so we can't easily identify the constant cases. The
copy-prop pass make this more visible for opt_zero_samples.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25742>
Inadvertently, because of a sequence of changes elsewhere, this pass
ended up not having any effect:
- Before Gfx5 the optimization is not applicable.
- On Gfx5-6 it doesn't apply because it sampler operations don't
currently use LOAD_PAYLOAD, but write the MOVs directly. Not clear to
me whether they ever did.
- On Gfx7+ it doesn't apply anymore because now the logical sampler
operations are now lowered directly to SENDs, and the is_tex() check
would skip them.
Since the LOAD_PAYLOAD implementation applies for Gfx7+ only, rework the
pass to work again by handling SEND instructions. To make the pass
easier, the optimization will happen before opt_split_sends() so only
one LOAD_PAYLOAD needs to be cared for.
Update the code to accept BAD_FILE sources in addition to zeros, these
are added in some cases as padding and effectively are don't care
values, so we can assume them zeros.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25742>
This is a preparation to (re-)enable opt_zero_samples(), which will reduce
a SEND mlen before we split it. When that happen, opt_split_sends()
won't be able to rely on the fact that mlen covers the entire
LOAD_PAYLOAD.
Since we are changing that, take the opportunity to also not modify the
existing LOAD_PAYLOAD, just create two new ones with the exact set of
sources. This allows the pass to be further simplified by iterating
forward and not require live_variables analysis.
The helper function was added so can be used later for
opt_zero_samples().
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25742>
For Task/Mesh WorkgroupID is now lowered to WorkgroupIndex by the
generic NIR pass, so we shouldn't hit this. We can now simplify the
asserting code in emit_work_group_id_setup().
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25977>
Refactor some of the blorp code into something more generic that we can
reuse for functionality needed post 3DPRIMITIVE emission.
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25039>
Now that alloc_flags is stored in anv_bo we can get rid of is_external,
has_fixed_address and has_client_visible_address flags that can
be inferred from alloc_flags.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
When anv_device_map_bo() is called from anv_device_alloc_bo() it gets
VkMemoryPropertyFlags set to 0 so it ends up with a write-combine
caching for integrated platforms with LLC, see 'if (!(property_flags &
VK_MEMORY_PROPERTY_HOST_CACHED_BIT)))'.
Current approach also has issues when mapping with anv_MapMemory2KHR()
as it would not have information to know that BO is a scanout.
It was also not properly calculating mmap mode for platforms with PAT
uAPI before "anv: Change default PAT entry to WC".
So here storing alloc_flags to anv_bo so there is no mismatches
between different code paths then using it to properly
calculate the mmap mode.
alloc_flags in anv_bo will also be used to calculate PAT index in
future patches.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
i915 mmap_calc_flags() is calculating WC caching for all MTL memory
types.
It will be fixed in the next patch but doing so causes tests to
fail due to incoherency in BOs not allocated with
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT.
So here switching the default/non-coherent BO allocation to a WC
PAT entry.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
PAT entry will be needed to calculate mmap mode and also will be
used during BO creating in Xe KMD when PAT uAPi lands.
So here moving the PAT entry selection to common code.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
Integrated GPUs almost always works with write-back caching(only
scanout and external bos works in write-combine) but in platforms
without LLC the coherency is broken if not explict asked to KMD.
vkFlushMappedMemoryRanges and vkInvalidateMappedMemoryRanges()
don't do any flushing or invalidate for memory allocated with
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT.
So if an application asked for a memory coherent, the
ANV_BO_ALLOC_SNOOPED flag needs to be set in alloc_flags and that
will be passed to KMD backends to properly ask to KMD for coherent
buffer.
The other chunk here removes the assert(alloc_flags & ANV_BO_ALLOC_MAPPED),
that is needed otherwise application can't ask for a coherent and
mapped memory.
Tried to find a reason for that assert in git history but did not
found what was the reasoning of this assert.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
Iris and ANV will need to switch to this PAT entry for BOs without
special needs.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
Previous integrated platforms had GT and Display caches not coherent
and there is nothing proven that it changed in MTL, so here
changing the PAT entry for scanout bos.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
mmap mode information will be used to properly calculate the mmap flags
in the i915 mmap uAPI and also will be used for BO creation when the
PAT uAPI lands in Xe KMD.
Xe KMD will also require the coherency mode during the BO creation.
So to avoid information duplication, adding this information to
intel_device_info platform entries.
No changes in behavior here.
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
This flag is required to properly calculate the PAT index of the
imported BO.
Cc: mesa-stable
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26099>
Estimating the batch space required can be tricky because of all the
workarounds. So implement chaining of batches like we do for command
buffers.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26087>
Before this, the render pass code or the driver combined the pipeline
create flags and the implicit flags from the render pass, but the
pipeline create flags will need to be sanitized when they are dynamic
state, so we need to do it in vk_graphics_state where we know that
information.
We also weren't combining pipeline flags correctly when linking, which
on turnip was being hidden by the lack of sanitizing for driver-provided
flags. We can't combine them correctly if they're part of the render
pass state, so they need to be pulled out into the overall pipeline
state.
For drivers using emulated renderpasses or tracking feedback loop
information themselves, this won't make a difference, but we have to
adapt turnip to not pass pipeline flags. This also means that we can
drop all handling of feedback_loop_input_only in turnip and just set it
in the runtime.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25436>
Switch to using VkPipelineCreateFlags2KHR, and use the new common helper
to get the right flags.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25436>
And replace the various homegrown or copy-pasted helpers in drivers.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25436>
"16TB ought to be enough for anybody."
- Probably some Intel graphics hardware engineer
TR-TT addresses are fixed regardless of the platform's gtt_size.
Unconditionally reserve this space for it: our total 48bit address
space is 256tb and TR-TT takes 16tb out of it (1/16th).
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26036>
This function will be able to transparently handle sparse binding
regardless of the backend: vm_bind ioctls or TR-TT. For now we only
support the vm_bind ioctls, but soon we'll have anv_sparse_bind_trtt()
as an option.
It is important to notice that even backends that support the vm_bind
ioctl may choose to do Sparse binding via TR-TT, that's why we're
adding the indirection at this specific point.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26036>
When it's a NULL bind we always set the bo_offset (aka memory offset)
to zero, so we have to avoid the "bind.offset == prev.offset + size"
check.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26036>
If the next bind is just an extension of the previous one, join both
in the same bind operation. Due to how mip levels are laid in memory,
this can only happen for mip level 0.
As of today xe.ko doesn't try to join contiguous operations for us.
Due to how rebinds happen each additional rebind operation may end up
resulting in many extra things done, so these simple checks end up
saving us a lot of cycles the Kernel would otherwise waste. This will
be true even after we issue all binds in a single ioctl.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26036>
Kill vma_cva and just toggle heap->alloc_high instead. This way,
client visible addresses will remain isolated in their own little
corner, except we have one less vma to deal with.
For TR-TT we'll need a special vma, and if we don't use the trick
above we'll need yet another trtt_cva_vma, increasing complexity even
more.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26036>
This actually doesn't fix any bugs or leaks, because according to the
man page:
"In the LinuxThreads implementation, no resources are associated
with mutex objects, thus pthread_mutex_destroy actually does
nothing except checking that the mutex is unlocked.
still, it's better to have it than not to have it, especially since
other implementations may do something.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26036>
