BOs larger than 1MB don't go memory pool due the size but applications
tend to use a lot of VkMemory with size larger than 1MB so to reduce
the number of pages and improve performance here I'm aligning the size
of BOs larger than 1MB to 64kb, this allows 64kb pages to be used at
least on Xe KMD.
This bring substantial perfomance benefit in exchange of a small
memory waste.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
MTL and newer integrated platforms has a performance gain when using
transparent huge pages, because of the fixed address requirement
we can't use slab for this case but we can change the initial pool
size to 2MB so all allocations get the transparent huge page
optimization.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
I can't think in any case where that would be false, so lets drop it.
While at it, also making some variables const.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
Because of the ANV_BO_ALLOC_CAPTURE flag, batch buffers were not
allowed to use memory pool.
So to workaround that here adding a new anv_bo_slab_heap heap for
cached+coherent+capture buffers with the main goal to get batch
buffer to memory pool but other buffers will as well.
For now that will only work in Xe KMD as i915 requires more changes
to support it.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
The whole purpose of anv_bo_pool is to reduce the number of
gem_create/destroy calls in command buffers that is something with
a short life span.
But slab_bo/memory pool does the same with even other benefits like
doing 2MB allocations to enable THP.
So here skipping the meat of anv_bo_pool_free() to directly return
the bo to slab_bo. This change is also necessary because the way
anv_bo_pool stores freed buffers it requires that all bos has a unique
gem handle, what not true of buffer allocated by anv_slab.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Suggested-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
This flag was not supported in anv_slab_bo because it is set together
with ANV_BO_ALLOC_CAPTURE and more important it has a specific VMA
range.
We can support it by adding a custom heap and allocating all bos in
the heap with all necessary flags, but because application can also
allocate those with vkAllocateMemory() here the ANV_BO_ALLOC_CAPTURE
is appended to the vkAllocateMemory() path for integrated gpu and
anv_slab_bo check if all the alloc_flags matches, because application
could choose to allocate it in a cached but not coherent memory type
for example.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
This flag was not supported in anv_slab_bo because it is set together
with ANV_BO_ALLOC_CAPTURE and more important it has a specific VMA
range.
But we can easily support it by adding a custom heap with it and
allocating all bos in the heap with all necessary flags.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
This changes allow us to support memory pool of bos with
ANV_BO_ALLOC_AUX_CCS set.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
This is implementing the functions in anv_slab_bo and actually
enabling memory pool.
This is heavily based on Iris memory pool implementation, the main
difference is that the concept of heaps only exist in anv_slab_bo, we
have function that takes the anv_bo_alloc_flags and decides what heap
to place that bo.
Some anv_bo_alloc_flags blocks memory pool, we can relax and remove
some flags from this denied list later.
This feature can be disabled in runtime by setting
ANV_DISABLE_SLAB=true, this can help us to easily check if bugs
are due to this feature or not.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
Allocating larger buffers allows KMD/HW to enable optimizations
that makes access to memory faster, also because of minimum alignment
required in some cases we allocate 4k or 64k long buffers for
usages that only needs a few bytes, wasting a lot of memory.
Memory pool takes care of both of those things and here I'm
adding the base infrastruture to implement this feature.
The next patch will implement the functions in anv_slab_bo.c, spliting
it in two to make review easier.
The idea here is take the same approach as Iris and use pb_slab.h.
In 99% of the places it will be transparent that anv_bo is actually
a slab of a larger and real anv_bo, the remaning 1% of the places are
handled here.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
That will make easy to implement memory pool in the next patches as we
need to calculate the VMA aligment without the KMD alignment requirement
for memory pool.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
This function will be needs in two places in the next patches.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33558>
Re-organize the configuration lists to make easier to include BFloat16
only for the Gfx125+ that support it, while keeping MTL supporting the
"lowered" configurations from pre-Gfx125.
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34105>
Add debug option to show current shader type being
compiled within anv_shader_bin_create.
Signed-off-by: Michael Cheng <michael.cheng@intel.com>
Reviewed-by: Casey Bowman <casey.g.bowman@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34596>
Xe2 changed the MOCS field in few instructions, those now have a field
for the MOCS index and other the encryption enable bit but ISL returns
the combination of both aka MEMORY_OBJECT_CONTROL_STATE.
To minimize changes I have added 2 macros to extract the values
from the value returned by isl.
From all the instructions changed Mesa only make use of two, so the
other instruction will be handled in the next patch.
Cc: mesa-stable
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34592>
Not all images with DRM format modifiers use
ANV_IMAGE_MEMORY_BINDING_PRIVATE.
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Change-Id: Idc6bae70ec7080f96555a85dcdc0ead915b02935
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32295>
Adding a boolean "enable_64b_rt" in anv_accel_struct_header for the
interpret.py to properly decode anv_instance_leaf
Signed-off-by: Kevin Chuang <kaiwenjon23@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33047>
This change separate the encode, header, and copy shader into versions
for Xe1/2 and Xe3+, including adding compile options and handling 64bit
version of instance leaf for Xe3+.
Signed-off-by: Kevin Chuang <kaiwenjon23@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33047>
This commit moves the calculation of tri_bary out of
brw_nir_rt_load_mem_hit_from_addr(), and only do the calculation on
demand, since unorm_float_convert can be expensive. We do this for both
Xe1/2 and Xe3+ for consistency.
Signed-off-by: Kevin Chuang <kaiwenjon23@gmail.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33047>
Move the calculation to nir_lower_vars_to_explicit_types(). This
consolidates the check of shader_info::shared_memory_explicit_layout
in a single place instead of in all drivers.
This is motivated by SPV_KHR_untyped_pointers. Before that extension
we had essentially two modes for shared memory variables
- No layout decorations in the SPIR-V, and both internal layout and
driver location was _given by the driver_.
- Explicitly laid out, i.e. they are blocks, and decorated with Aliased.
Because they all alias, we could assign them driver location directly
to the start of the shared memory.
With the untyped pointers extension, there's a third option, to be added
by a later commit
- Explicitly laid out, i.e. they are blocks, and NOT decorated
with Aliased. Driver location is _given by the driver_. Blocks
with and without Aliased can be mixed.
The driver location of multiple blocks that don't alias depend on
alignment that is driver-specific, which we can more easily do from
the nir_lower_vars_to_explicit_types() that already has access to
a function to obtain such value.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> (hk)
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> (v3dv)
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (anv/hasvk)
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (panvk)
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> (radv)
Reviewed-by: Rob Clark <robdclark@gmail.com> (tu)
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34139>
I accidentally disabled compression on CPS surfaces marked as storage or
color attachment for all platforms, when this should only be limited to
Xe.
Fixes: 80f9b6 ('anv: CPB surfaces that are used as color attachments or for stores cannot be compressed')
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34297>
This patch does a couple of things to make CL integration with drivers
as seamless as possible:
- We pull in opencl-c.h and opencl-c-base.h to stop relying on system
headers.
- Parts of libcl.h are moved to new headers that are incomplete CL-safe
variants of libc headers.
- A couple of util headers are changed to remove now unnecessary
__OPENCL_VERSION__ guards and make more headers CL safe.
- Drivers now include src/compiler/libcl and use headers like
macros.h,u_math.h instead of libcl.h.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33576>
Some upcoming changes in the runtime will make it impossible to rely
on the pipeline or runtime information to know whether a fragment
shader has input attachments.
Instead we gather that information at compile time and store it in our
shader bind_map.
At runtime we check whether the fragment shader has input attachments
and whether those map to the runtime depth/stencil input attachments
to set the 3DSTATE_PS_EXTRA::PixelShaderKillsPixel.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: d2f7b6d5a7 ("anv: implement VK_KHR_dynamic_rendering_local_read")
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32540>
If you suspect a workload is failing because it needs more memory, you
can set ANV_SYS_MEM_LIMIT=100 to give it all the memory available.
This could make, for example, certain games start working (it really
depends on how much RAM you have and how much the game wants).
If you suspect a workload is too resource hungry, you can try to limit
it with ANV_SYS_MEM_LIMIT=30 (or some other value) to see if it can
deal with the more restricted environment and behave accordingly.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28513>
"We paid for sixteen gigs of RAM, so we gonna use the whole damn
sixteen gigs of RAM!"
- My Mom
First, some history:
The Anv 50%-or-75% rule was originally added in 2017 by 060a6434ec
("anv: Advertise larger heap sizes"). When i915.ko started reporting
memory sizes in its ioctls, it didn't impose any restrictions: 100% of
SRAM was reported as available, so the restriction was in Mesa. When
xe.ko was introduced, it only reported 50% of the SRAM as available
through its ioctls, so commit b571ae6e7a ("intel: Make memory heaps
consistent between KMDs") adapted the code to not take an extra 25% of
the 50% that was already cut, and restricted i915.ko to 50% instead of
the 50%-or-75%. In Kernel commit d2d5f6d57884 ("drm/xe: Increase the
XE_PL_TT watermark"), xe.ko changed to reporting 100% of SRAM through
its ioctls, so we adapted Mesa to do the right thing depending on
which Kernel version was running.
While this was all happening, we were discussing about which behavior
was actually the best: restrict everything to 50% in order to avoid
issues when many things are running in parallel, or keep the
restriction only at 75% in order to allow high demanding workloads to
make full use of the hardware.
The way I see, if parallel applications are causing the system to run
out of resources, the user always has the option to kill applications
and use one thing at a time. On the other hand, if a single
application needs more than 50% of the SRAM and we don't allow it in
our heaps, the application will never work (unless, of course, the
user patches Mesa). So in this commit we go back to allowing
high-demanding applications to work by restoring the 50%-or-75% rule.
This commit is especially useful in systems with integrated graphics,
like LNL, where the option to upgrade RAM is not present.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28513>
Kernel commit d2d5f6d57884 ("drm/xe: Increase the XE_PL_TT watermark")
changed how xe.ko reportes memory: its ioctls now report 100% of the
system RAM as available. Since our policy is to report 50% of the SRAM
as available for the heaps, add some code to check the amount reported
by xe.ko against the amount reported by the system, then act
accordingly.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28513>
Before commit b571ae6e7a ("intel: Make memory heaps consistent
between KMDs"), we had the following policy for reporting Sytem RAM
memory sizes:
- For OpenGL, we reported the total available RAM.
- For Vulkan, we reported the total available RAM as:
- 50% of the total RAM if the total RAM was <= 4GB,
- 75% otherwise
- In addition, the Memory Budget (for VK_EXT_memory_budget) is 90%
of the "free" memory, which can be an extra 10% off of the 50% or
75%.
When xe.ko was added, one key difference was noted: while i915.ko
reported the "real" RAM memory sizes in its ioctls, xe.ko reported
only 50% of the system RAM as available. Because of that (and other
reasons, see this discussion on MR 28513), commit b571ae6e7a decided
to unify the behavior by changing the Anv i915.ko rule to "always 50%"
instead of "50% or 75%". This also changed the Iris rule to 50%
instead of 100%.
In my research, I couldn't find any reason why this restriction should
also apply to Iris, so here we revert back to handling these size
restrictions on Anv only.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28513>
VUID-VkDescriptorSetAllocateInfo-pSetLayouts-09380 says that :
"If pSetLayouts[i] was created with an element of pBindingFlags
that includes VK_DESCRIPTOR_BINDING_VARIABLE_DESCRIPTOR_COUNT_BIT,
and VkDescriptorSetVariableDescriptorCountAllocateInfo is included
in the pNext chain, and
VkDescriptorSetVariableDescriptorCountAllocateInfo::descriptorSetCount
is not zero, then
VkDescriptorSetVariableDescriptorCountAllocateInfo::pDescriptorCounts[i]
must be less than or equal to
VkDescriptorSetLayoutBinding::descriptorCount for the corresponding
binding used to create pSetLayouts[i]"
But applications like are not following the spec. RADV doesn't apply
that limit and allocates if there is enough space in the pool. Let's
just do the same.
Note that this issue got resolved with a vkd3d-proton change :
a7ac1a7d2f
But since this change is deleting more code than it adds, might as
well go with it.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12185
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32305>
Add support for dumping shader asm containing instruction line numbers
matching offsets within instruction state pool buffer. Offsets
should match values collected from eu stall sampling. This is
required for match eu stall data with individual shader instructions.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30142>
