Commit graph

4786 commits

Author SHA1 Message Date
Emma Anholt
647ca81654 anv: Only enable GPL if ANV_GPL=true, or if zink or DXVK are the engine.
Since there are concerns that the VK_EXT_GPL implementation may have
issues with mesh shading, disable it by default but give users a knob to
turn it on to experiment.

This doesn't automatically enable GPL use in zink, because we lack
extendedDynamicState2PatchControlPoints, but it means that you only need
to set ZINK_DEBUG=gpl and not both env vars.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15637>
2023-04-17 22:43:37 +00:00
Lionel Landwerlin
3d49cdb71e anv: implement VK_EXT_graphics_pipeline_library
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15637>
2023-04-17 22:43:37 +00:00
Lionel Landwerlin
0b8a2de2a1 anv: add dynamic buffer offsets support with independent sets
With independent sets, we're not able to compute immediate values for
the index at which to read anv_push_constants::dynamic_offsets to get
the offset of a dynamic buffer. This is because the pipeline layout
may not have all the descriptor set layouts when we compile the
shader.

To solve that issue, we insert a layer of indirection.

This reworks the dynamic buffer offset storage with a 2D array in
anv_cmd_pipeline_state :

   dynamic_offsets[MAX_SETS][MAX_DYN_BUFFERS]

When the pipeline or the dynamic buffer offsets are updated, we
flatten that array into the
anv_push_constants::dynamic_offsets[MAX_DYN_BUFFERS] array.

For shaders compiled with independent sets, the bottom 6 bits of
element X in anv_push_constants::desc_sets[] is used to specify the
base offsets into the anv_push_constants::dynamic_offsets[] for the
set X.

The computation in the shader is now something like :

  base_dyn_buffer_set_idx = anv_push_constants::desc_sets[set_idx] & 0x3f
  dyn_buffer_offset = anv_push_constants::dynamic_offsets[base_dyn_buffer_set_idx + dynamic_buffer_idx]

It was suggested by Faith to use a different push constant buffer with
dynamic_offsets prepared for each stage when using independent sets
instead, but it feels easier to understand this way. And there is some
room for optimization if you are set X and that you know all the sets in
the range [0, X], then you can still avoid the indirection. Separate
push constant allocations per stage do have a CPU cost.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15637>
2023-04-17 22:43:37 +00:00
Lionel Landwerlin
16c7c37718 anv: move preprocessing of NIR right before compilation
For graphics pipelines, we'll need to load NIR for retained shaders.
We want to avoid as much processing as possible while doing that when
we're able to load ISA from cache.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15637>
2023-04-17 22:43:37 +00:00
Lionel Landwerlin
17e7fe9d97 anv: make input attachments available through bindless
With independent sets, we cannot bake into the shader the binding
table entry of input attachments anymore because that final location
is affected by multiple sets.

We can still access them by looking into the descriptor buffer. This
change enables the image handle to be stored in the descriptor buffer.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15637>
2023-04-17 22:43:37 +00:00
Lionel Landwerlin
e82b05dc64 anv: move force shading rate writes checks
With variable fragment shading rate, the last pre-rasterization stage
is responsible to write the shading rate value.

The current checks is as follow :

   If the fragment shader can be dispatched at variable shading rate,
   look for the last pre-raster stage to force the write.

We change this to :

   If we're the last pre-raster stage, force the write.

That way this works for pre-rasterization shaders compiled without a
fragment shader.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15637>
2023-04-17 22:43:37 +00:00
Lionel Landwerlin
b2d3d818d5 anv: introduce a base graphics pipeline object
Pipeline libraries and linked pipelines will inherit from this.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15637>
2023-04-17 22:43:37 +00:00
José Roberto de Souza
1563210a41 intel/common: Add gt_id to intel_engine_class
MTL and newer platforms on Xe kmd will have engines with gt_id != 0.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22477>
2023-04-17 14:43:06 +00:00
Lionel Landwerlin
a787728906 anv: enable blorp query reset for performance queries
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22480>
2023-04-15 12:25:57 +03:00
Felix DeGrood
0417cfd7a0 anv: Enable INTEL_MEASURE=cpu
Reviewed-by: Mark Janes <markjanes@swizzler.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21505>
2023-04-14 21:57:50 +00:00
Felix DeGrood
c45dee34aa anv: split INTEL_MEASURE multi events
Measure performance of each draw separately in multi_draw event.
Previously, we measured duration of the sum of all draws launched
per multi_draw. This should provide more detailed data for
multi_draws.

Reviewed-by: Mark Janes <markjanes@swizzler.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21505>
2023-04-14 21:57:49 +00:00
Felix DeGrood
50bda45d15 anv: Add flush reason to NEEDS_END_OF_PIPE_SYNC
cs_stall gets inserted if both flushes and invalidates are required.
This cs_stall reason was not called out explicitly, until now.

Reviewed-by: Mark Janes <markjanes@swizzler.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21505>
2023-04-14 21:57:49 +00:00
Felix DeGrood
bdeb849e25 anv: Add flush reasons to raytracing flushes
Reviewed-by: Mark Janes <markjanes@swizzler.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21505>
2023-04-14 21:57:49 +00:00
Felix DeGrood
9a30493ccb anv: Add END_OF_PIPE_SYNC reporting to INTEL_DEBUG=pc
Reviewed-by: Mark Janes <markjanes@swizzler.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21505>
2023-04-14 21:57:49 +00:00
Felix DeGrood
0a52002a1c anv: disable reset query pools using blorp opt on MTL
This optimization causes some MTL tests to run forever. Not
yet sure why. Disabling optimization until we have a fix.

Reviewed-by: Mark Janes <markjanes@swizzler.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22373>
2023-04-13 19:35:34 +00:00
Lionel Landwerlin
66edd030ab anv: add utrace tracking of frame boundaries
Based on vkQueuePresentKHR calls. It just helps spotting the beginning
end of a frame in perfetto when apps are using 3/4 command buffers per
frame.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22276>
2023-04-13 01:14:38 +00:00
Lionel Landwerlin
68bba1539f anv: exclude performance queries from blorp clears
The query buffer contains a batch to implement the multi pass
replay/accumulation of results. So we can't clear it with a memset.

An optimization for later would be to move the batches to the very end
of the query buffer so we can clear the query data without touching
the batches.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 4dc7256bf9 ("anv: reset query pools using blorp")
Reviewed-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22421>
2023-04-13 00:44:29 +00:00
José Roberto de Souza
b1299f42ff anv: Fix vm bind of imported buffers
Imported buffers may be created in a device with different
memory alignment and this can cause vm bind to fail because bo
size can be smaller than the calculated vm bind range using the
importer device memory alignment.

So here adding actual_size to anv_bo, this will be set with the actual
size of the bo allocated by kmd for bos allocate in the current device.
For other bo the lseek or the Vulkan API size will be used.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22219>
2023-04-12 10:05:32 +00:00
Lionel Landwerlin
cff71ae8ff anv: fixup streamout write barriers
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8796
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22336>
2023-04-11 09:53:10 +00:00
Daniel Schürmann
53eb3ad375 vulkan/pipeline_cache: add cache parameter to deserialize() function
This allows for secondary cache lookups during deserialization.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21967>
2023-04-10 09:14:30 +00:00
Daniel Schürmann
5daff41e27 vulkan/pipeline_cache: remove vk_device from vk_pipeline_cache_object
It is not necessary to store the extra pointer.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21967>
2023-04-10 09:14:30 +00:00
Alyssa Rosenzweig
4fa2924610 anv,hasvk: Use vk_features2_to_features
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22217>
2023-04-07 18:16:40 -04:00
Felix DeGrood
4dc7256bf9 anv: reset query pools using blorp
Previously we used PC to set query data to 0 during
CmdResetQueryPool. This was slow when clearing large query pools.
Switching to blorp to clear pools is faster for large query pools.

Red Dead Redemption 2: +1.5% speedup

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>
2023-04-07 15:51:20 +00:00
Lionel Landwerlin
bb49610973 anv: replace query flush before gpu copy by semaphore wait
All the flushes should already have happened, we just need CS to wait
for the operations to complete. Just use a MI_SEMAPHORE_WAIT to check
the availability bit is set.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>
2023-04-07 15:51:20 +00:00
Lionel Landwerlin
abc4111d19 anv: pass steam output as argument for anv_dump_pipe_bits
Just if you need to change it at some point ;)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>
2023-04-07 15:51:20 +00:00
Felix DeGrood
2415d57a99 anv/blorp: add flush reasons to RT flushes
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>
2023-04-07 15:51:20 +00:00
Felix DeGrood
43f93f5043 anv/blorp: implement anv_cmd_buffer_fill_area
Implemented function to fill an area at an address.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>
2023-04-07 15:51:20 +00:00
Felix DeGrood
0130a4f667 anv/blorp: support surf generation for addresses
Already have support for anv_buff. Extended to support addresses.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22178>
2023-04-07 15:51:20 +00:00
Rohan Garg
e21cca78ea anv,blorp,iris: Set PreferredSLMAllocationSize on gfx125+
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22307>
2023-04-06 10:54:47 +00:00
Emma Anholt
f1ea6c1b40 intel: Always call nir_lower_frexp.
We have NIR lowering for Vulkan, and rely on GLSL's lowering in the
frontend, but this will let us drop the GLSL lowering.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22083>
2023-04-06 02:32:01 +00:00
Marcin Ślusarz
554b28d0b1 anv: ignore structure types handled in vk_device_memory_create
Without that we get warning from anv_debug_ignored_stype, called in
the default case.

Fixes: 4b0b75c27a ("anv: Use the new vk_device_memory base struct")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22295>
2023-04-05 11:53:29 +00:00
Sagar Ghuge
10fc12fd17 anv: Drop dead code that sets the L3BypassDisable field
These helper functions will only get invoked for GFX < 11 and the
L3BypassDisable field is present starting from GFX12+.

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22275>
2023-04-04 17:39:20 +00:00
Sagar Ghuge
e3b172d75d anv: Drop unused param from add_surface_reloc
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22275>
2023-04-04 17:39:20 +00:00
Tapani Pälli
b0cdd5a1d2 anv: check for MESA_SHADER_TESS_CTRL with get_tcs_prog_data
Fixes: 86d931724d ("anv: Implement Wa_14015297576")
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22282>
2023-04-04 10:42:12 +03:00
Tapani Pälli
98c6db3fc8 anv: take primitive ID override to account Wa_14015297576
Commit ca4ec49b0e took primitive ID override fields in to use, this
has to be checked as part of Wa_14015297576.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22282>
2023-04-04 10:41:58 +03:00
Faith Ekstrand
8df8a3c19b anv/pipeline: Use feedback loop flags for self-dependencies
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22191>
2023-04-03 18:13:01 +00:00
Nataraj Deshpande
71fe9dfe07 anv: Bump VkDeviceMemory objects limit to 4GB
Android CTS 13_r4 tests dEQP-VK.memory.allocation.random* fail
with VK_ERROR_OUT_OF_DEVICE_MEMORY on ADL boards with 32GB memory
as memory allocation requests from DEQP are much larger(~2.9GB+)
based on device heap size/8.

Increase the limit to unsigned 32bit max(~4GB) which helps to
fix the dEQP-VK.memory.allocation.random* tests.

v1: Bound allocation by the largest memory heap size (Lionel Landwerlin)

v2: Clean up comments to reflect the code change (Ivan Briano)
    Update the value of MAX_MEMORY_ALLOCATION_SIZE (Lionel Landwerlin)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22066>
2023-04-03 06:18:52 +00:00
Lionel Landwerlin
00fc927e52 anv: take care of maxStorageBufferRange being uint32_t
Not fixing anything, but required for another fix.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22066>
2023-04-03 06:18:52 +00:00
Faith Ekstrand
4b0b75c27a anv: Use the new vk_device_memory base struct
Reviewed-by: Lina Versace <lina@kiwitree.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22038>
2023-03-31 14:57:03 +00:00
Faith Ekstrand
b16cfe23ef vulkan,anv,hasvk,radv: Unify Android hardware buffer creation
Reviewed-by: Lina Versace <lina@kiwitree.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22038>
2023-03-31 14:57:03 +00:00
Faith Ekstrand
0a86df21d0 anv,hasvk: Set vk_image.ahardware_buffer_format
This effectively just modifies it for YCbCr images where the common code
doesn't have a useable format mapping.

Reviewed-by: Lina Versace <lina@kiwitree.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22038>
2023-03-31 14:57:03 +00:00
Faith Ekstrand
41f88be282 vulkan,anv,hasvk,radv: Add a common vk_image_usage_to_ahb_usage helper
Reviewed-by: Lina Versace <lina@kiwitree.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22038>
2023-03-31 14:57:03 +00:00
Felix DeGrood
49f34675f3 anv: set CFE_STATE.OverDispatchControl to default
BSpec specifies default value for CFE_STATE.OverDispatchControl
is 2, or 50% overdispatch. No observed performance impact.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22179>
2023-03-31 14:18:59 +00:00
Felix DeGrood
ecb709c853 anv: only emit CFE_STATE when scratch space increases
On Gen12.5+, we only need to emit CFE_STATE when scratch space
has changed, not on every pipeline binding. Also, only grow the
scratch space, never shrink it. Need to reset after secondary buf.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22179>
2023-03-31 14:18:58 +00:00
Lionel Landwerlin
c88de6c18c anv: move queue check helpers to anv_private
Also fix missing trace point stuff in command buffer begin/end

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22179>
2023-03-31 14:18:58 +00:00
Lionel Landwerlin
546a6752ac anv: compute the largest GRL kernel scratch size
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22179>
2023-03-31 14:18:58 +00:00
Felix DeGrood
97e64aef60 anv: cs_stall during compute state flush on < gen12.5
The CS Stall in cmd_buffer_flush_compute_state is only required
on HW that uses MEDIA_VFE_STATE, gen12 and previous.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22179>
2023-03-31 14:18:58 +00:00
José Roberto de Souza
66a8f7dcdb anv: Add assert in functions not supported by Xe kmd
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22171>
2023-03-30 01:57:44 +00:00
José Roberto de Souza
74ff665bd0 anv: Disable anv_bo_sync_type for Xe kmd
anv_bo_sync_type is only supported with i915, if necessary it will be
implemented for Xe kmd.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22171>
2023-03-30 01:57:44 +00:00
José Roberto de Souza
fdea48df5e anv: Implement Xe version of anv_queue_exec_locked() and queue_exec_trace()
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22171>
2023-03-30 01:57:44 +00:00