Round up allocations to nearest 2MB interval if this increases
the allocation by no more than 1.33x. This reduces page count but
at the cost of extra memory consumption. Optimization only applied
to MTL(Xe KMD only)/LNL platforms, which are particularly impacted by
page misses.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
BOs larger than 1MB don't go memory pool due the size but applications
tend to use a lot of VkMemory with size larger than 1MB so to reduce
the number of pages and improve performance here I'm aligning the size
of BOs larger than 1MB to 64kb, this allows 64kb pages to be used at
least on Xe KMD.
This bring substantial perfomance benefit in exchange of a small
memory waste.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
MTL and newer integrated platforms has a performance gain when using
transparent huge pages, because of the fixed address requirement
we can't use slab for this case but we can change the initial pool
size to 2MB so all allocations get the transparent huge page
optimization.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
I can't think in any case where that would be false, so lets drop it.
While at it, also making some variables const.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
Because of the ANV_BO_ALLOC_CAPTURE flag, batch buffers were not
allowed to use memory pool.
So to workaround that here adding a new anv_bo_slab_heap heap for
cached+coherent+capture buffers with the main goal to get batch
buffer to memory pool but other buffers will as well.
For now that will only work in Xe KMD as i915 requires more changes
to support it.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
The whole purpose of anv_bo_pool is to reduce the number of
gem_create/destroy calls in command buffers that is something with
a short life span.
But slab_bo/memory pool does the same with even other benefits like
doing 2MB allocations to enable THP.
So here skipping the meat of anv_bo_pool_free() to directly return
the bo to slab_bo. This change is also necessary because the way
anv_bo_pool stores freed buffers it requires that all bos has a unique
gem handle, what not true of buffer allocated by anv_slab.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Suggested-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
This flag was not supported in anv_slab_bo because it is set together
with ANV_BO_ALLOC_CAPTURE and more important it has a specific VMA
range.
We can support it by adding a custom heap and allocating all bos in
the heap with all necessary flags, but because application can also
allocate those with vkAllocateMemory() here the ANV_BO_ALLOC_CAPTURE
is appended to the vkAllocateMemory() path for integrated gpu and
anv_slab_bo check if all the alloc_flags matches, because application
could choose to allocate it in a cached but not coherent memory type
for example.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
This flag was not supported in anv_slab_bo because it is set together
with ANV_BO_ALLOC_CAPTURE and more important it has a specific VMA
range.
But we can easily support it by adding a custom heap with it and
allocating all bos in the heap with all necessary flags.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
This changes allow us to support memory pool of bos with
ANV_BO_ALLOC_AUX_CCS set.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
This is implementing the functions in anv_slab_bo and actually
enabling memory pool.
This is heavily based on Iris memory pool implementation, the main
difference is that the concept of heaps only exist in anv_slab_bo, we
have function that takes the anv_bo_alloc_flags and decides what heap
to place that bo.
Some anv_bo_alloc_flags blocks memory pool, we can relax and remove
some flags from this denied list later.
This feature can be disabled in runtime by setting
ANV_DISABLE_SLAB=true, this can help us to easily check if bugs
are due to this feature or not.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
Allocating larger buffers allows KMD/HW to enable optimizations
that makes access to memory faster, also because of minimum alignment
required in some cases we allocate 4k or 64k long buffers for
usages that only needs a few bytes, wasting a lot of memory.
Memory pool takes care of both of those things and here I'm
adding the base infrastruture to implement this feature.
The next patch will implement the functions in anv_slab_bo.c, spliting
it in two to make review easier.
The idea here is take the same approach as Iris and use pb_slab.h.
In 99% of the places it will be transparent that anv_bo is actually
a slab of a larger and real anv_bo, the remaning 1% of the places are
handled here.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
That will make easy to implement memory pool in the next patches as we
need to calculate the VMA aligment without the KMD alignment requirement
for memory pool.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
This function will be needs in two places in the next patches.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
That will be used outside of gallium drivers in the next commit, so
moving it out to util folder.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
pb_slab.h don't make use of anything defined in pb_buffer.h.
But removing it breaks Iris build because pb_buffer.h then includes
util/format/u_formats.h that Iris was using but not including.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
Commit 3fc79582a1db ("drm/i915: Increase I915_PARAM_MMAP_GTT_VERSION version to indicate support for partial mmaps")
increased the I915_PARAM_MMAP_GTT_VERSION version, with that we can
detect what kernel version has the partial mmap fix or not and limit
the usage of this workaround.
This time o mmap will be used in memory pool, so here adding this
propertly to enable or not the feature.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
Secondary command buffers don't add much information to the traces, but
add more noise and overhead. In order to disable their tracepoints by
default, a separate "secondary_cmd_buffer" tracepoint is created.
Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34588>
This brings back c9e33e5cbf ("intel/fs/cse: Make HALT instruction act
as CSE barrier."), from the old CSE pass into the new one.
Fixes new CTS test: dEQP-VK.subgroups.shader_quad_control.terminated_invocation
Fixes: 9690bd369d ("intel/brw: Delete old local common subexpression elimination pass")
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34643>
and also stop legalizing away src 1 modifiers. Both of these are present,
they just move to a different place in the encoding.
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34750>
This fixes a compilation issue in Marvel Rivals where the legalization
logic and the encoding logic don't line up, which results in an
assertion failure on this instruction:
r17 = hfma2 r17.xx -r18.xx 0x3c003c00
The fix here is a little overly restrictive because it turns out we
actually do have modifiers for all 3 sources. Those modifiers will
be added in later commits.
Fixes: 567cae69c3 ("nak: Add 16-bits float operations")
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34750>
For Xe2+, from Bspec 64643, bit field "StackID": The maximum number of
StackIDs can be 2^12- 1.
Cc: mesa-stable
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34709>
Re-organize the configuration lists to make easier to include BFloat16
only for the Gfx125+ that support it, while keeping MTL supporting the
"lowered" configurations from pre-Gfx125.
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34105>
Handle bfloat16 by converting sources to float, performing the
operation, and converting result back to bfloat16 if needed. This is
done because not all ALU ops have a `bf` version in NIR.
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34105>
Take the opportunity to add a comment about why the bit_size
comes from the NIR def and not the original type.
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34105>
When converting BFloat16 from/to non-Float32 type, use
the Float32 conversion as an intermediate step. Take the
opportunity to separate the unary_op/convert code-paths.
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34105>
