Commit graph

152 commits

Author SHA1 Message Date
Tatsuyuki Ishi
325d650c6e radv: Use sizeof(flags) instead of hardcoded size in radv_hash_shaders.
For consistency.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26145>
2023-11-11 05:26:27 +00:00
Tatsuyuki Ishi
ef414a8231 radv: Fix mis-sizing of pipeline_flags in radv_hash_rt_shaders.
pipeline_flags was 64-bit yet only the first 4 bytes were hashed.
Luckily, the mask included no flag above the 32nd bit, so this was
technically working fine. Still, it's better to use explicit sizeof
constructs to be more resilient to accidental type changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26145>
2023-11-11 05:26:26 +00:00
Connor Abbott
e6f5d7222c vk,lvp,tu,radv,anv: Add common vk_*_pipeline_create_flags() helper
And replace the various homegrown or copy-pasted helpers in drivers.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25436>
2023-11-06 14:33:51 +00:00
Konstantin Seurer
e039e3cd76 radv/rt: Store NIR shaders separately
In order to compile monolithic shaders with pipeline libraries, we need
to keep the NIR around for inlining recursive stages.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21929>
2023-09-10 11:40:12 +00:00
Samuel Pitoiset
9089b091b2 radv: add support for VkPipelineCreateFlags2CreateInfoKHR
If the structure is present in pNext, it's used instead of flags.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24392>
2023-08-10 03:05:02 +00:00
Samuel Pitoiset
f4b9c5b1d0 radv: store pipeline create flags to radv_pipeline::create_flags
This is a common practice in vulkan/runtime and this will be easier
to use extended pipeline create flags.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24392>
2023-08-10 03:05:02 +00:00
Samuel Pitoiset
08bfcc12d4 radv: rename radv_pipeline_stage to radv_shader_stage
It's more generic and it will fit shader object just well.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>
2023-07-26 07:44:49 +00:00
Daniel Schürmann
437bf4fccb amd: move end-of-code marker padding to ACO.
This makes it consistent between drivers and compilers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23799>
2023-07-11 12:01:45 +00:00
Daniel Schürmann
e88c0bb945 radv: migrate radv_shader hash to BLAKE3
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23974>
2023-07-11 09:02:43 +00:00
Daniel Schürmann
8740fb0026 radv/meta: disable disk cache for meta shaders
Meta shaders are already stored in a separate cache file,
inserting them into the disk cache is unnecessary.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23909>
2023-07-11 07:36:53 +00:00
Friedrich Vock
e3bd54d2a8 radv: Add support for creating capture/replay shaders
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:25 +00:00
Friedrich Vock
51f2fa1a5e radv: Break up radv_shader_nir_to_asm
radv_shader_nir_to_asm actually had 3 functions: compiling the NIR to
asm, uploading the shaders and generating debug info for them.
This reduces the functionality of radv_shader_nir_to_asm to only compile
NIR to asm. Uploading the shader and generating debug info is split into
separate functions.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:25 +00:00
Daniel Schürmann
3bd72eec1e radv: reference pipeline cache object in radv_pipeline
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23878>
2023-07-03 17:59:07 +00:00
Eric Engestrom
8b319c6db8 radv: reformat according to its .clang-format
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23253>
2023-06-16 19:59:52 +00:00
Daniel Schürmann
09e4535f20 radv/rt: don't write cache hit feedback per stage.
Too much boilerplate for no benefit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22096>
2023-06-08 00:37:03 +00:00
Daniel Schürmann
772b5b3758 radv/rt: add and use specialized cache search/insert functions
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22096>
2023-06-08 00:37:03 +00:00
Daniel Schürmann
1e334e9818 radv/rt: store stack_sizes per stage instead of per group
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22503>
2023-05-10 07:02:13 +00:00
Daniel Schürmann
1100f7dcd5 radv/rt: unify radv_ray_tracing_lib_pipeline and radv_ray_tracing_pipeline
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22503>
2023-05-10 07:02:13 +00:00
Daniel Schürmann
8665506cbd radv/pipeline_cache: add NIR caching capabilities
This patch adds functions to radv_pipeline_cache in order to cache
serialized NIR shaders as opaque cache objects.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22100>
2023-05-02 19:15:10 +00:00
Friedrich Vock
c809c05f4f radv: Hash pipeline libraries separately
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22686>
2023-04-26 02:48:29 +00:00
Daniel Schürmann
1a0ae06091 radv/rt: rename radv_ray_tracing_module -> radv_ray_tracing_group
This name better reflects the purpose and content of this struct.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22686>
2023-04-26 02:48:29 +00:00
Daniel Schürmann
dcfca69e67 radv: clean up pipeline-cache interface
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
c2b9279227 radv: use vk_pipeline_cache
Replaces the RADV pipeline cache with an implementation
based on the common vk_pipeline_cache.

We use a dual-layer approach with two types of cache entries.

  1. radv_shader:
    - serialized as radv_shader_binary
    - uses SHA1 of the binary as key
  2. radv_pipeline_cache_object:
    - contains pointers to associated radv_shaders
    - serialized as list of SHA1
    - uses the pipeline hash as key

In combination with single-file disk-cache, this reduces the cache size by ~60%.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
9b3679aff2 radv: implement radv_shader_create_cached()
This function takes a radv_shader_binary and writes it to the
disk cache before creating and returning a radv_shader cache entry.
The key of the cache entry is the full SHA1 hash of the binary.
This way, we will be able to deduplicate identical shaders.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
55cc2fb088 radv: add struct radv_pipeline_cache_object
This patch implements a per pipeline cache-entry which contains
pointers to the associated shaders.
The serialized format contains the binary hashes of the shaders.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
577d76f60f radv: add radv_shader_serialize() and radv_shader_deserialize() functions
These functions convert between radv_shader and radv_shader_binary.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
5303a57964 radv: derive struct radv_shader from vk_pipeline_cache_object
We will use radv_shader directly as cache entries.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Samuel Pitoiset
a9529d9644 radv: allow to create/insert PS epilogs from/to the cache for libs
To cache PS epilogs compiled inside graphics pipeline libraries.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22264>
2023-04-04 14:47:39 +00:00
Samuel Pitoiset
ba967e1a28 radv: rely on non-NULL binaries when inserting shaders to the cache
With GPL, a stage can be imported from a library which means that the
binary is NULL (it's freed right after compilation) but the shader is
non-NULL. To avoid crashing, rely on non-NULL binaries because this
implies that the shader is non-NULL as well.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22264>
2023-04-04 14:47:39 +00:00
Samuel Pitoiset
1239fcab4d radv: only copy non-NULL shaders when loaded from the cache
With GPL, we don't necessarily load all stages from the same cache
entry and other stages might have been imported from libraries, so
don't overwrite them.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22264>
2023-04-04 14:47:39 +00:00
Daniel Schürmann
2c8d101c71 radv: separate radv_capture_shader_executable_info() from radv_shader_create()
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22029>
2023-03-24 19:03:29 +00:00
Daniel Schürmann
fe716c2428 radv: separate radv_postprocess_binary_config() from radv_shader_create()
The goal is to make radv_shader_create() a function that creates a shader
from a binary without any additional information.
Postprocessing the config is only needed after compilation.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22029>
2023-03-24 19:03:29 +00:00
Samuel Pitoiset
5082b6b034 radv: add support for caching PS epilogs
For PS epilogs created at link time because libraries are still not
cached.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:59 +00:00
Konstantin Seurer
0f18bb4076 radv: Fix inserting stack_size into the cache
Fixes: 3e03fe4 ("radv/rt: move stack_sizes into radv_ray_tracing_module")
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21969>
2023-03-18 14:57:51 +00:00
Konstantin Seurer
3887f64dc3 radv: Fix loading stack_size from the cache
Fixes: 3e03fe4 ("radv/rt: move stack_sizes into radv_ray_tracing_module")
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21969>
2023-03-18 14:57:51 +00:00
Konstantin Seurer
0fc8335ccb radv/rt: Use vk_pipeline_hash_shader_stage for RT stages
Fixes
dEQP-VK.pipeline.pipeline_library.graphics_library.misc.non_graphics.shader_module_info_rt.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21707>
2023-03-12 13:18:15 +00:00
Daniel Schürmann
3e03fe44e7 radv/rt: move stack_sizes into radv_ray_tracing_module
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21667>
2023-03-06 13:58:54 +00:00
Daniel Schürmann
48edcd03c5 radv/rt: introduce struct radv_ray_tracing_module
This is preliminary work for separate shader functions.
The ray_tracing_module is eventually intended as self-contained
pipeline struct per RT group.

For now, these modules only contain the group handles.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21667>
2023-03-06 13:58:54 +00:00
Bas Nieuwenhuizen
430170702e radv: Hash group handles as part of RT pipeline key.
So that we can start varying them  to avoid collisions while keeping
handles stable.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21406>
2023-02-23 22:17:30 +00:00
Bas Nieuwenhuizen
9eb76ab638 radv: Add helper to hash stages.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21406>
2023-02-23 22:17:30 +00:00
Daniel Schürmann
f6251b21f9 radv/rt: don't hash maxPipelineRayRecursionDepth
The stack size has no effect on the generated shader anymore.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21159>
2023-02-16 19:37:25 +00:00
Konstantin Seurer
cf9e1b953c radv: Hash VK_PIPELINE_CREATE_RAY_TRACING_NO_NULL_* flags
Found by inspection.

Fixes: 687a82d ("radv/rt: Handle no-null shader flags")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21320>
2023-02-16 11:42:25 +00:00
Samuel Pitoiset
69bd1c0c40 radv: restore uploading shaders individually instead of consecutively
The shaders were uploaded consecutively to fit a RGP constraint but
this was more like a workaround. This upload path doesn't work well for
graphics pipeline library and it was the main blocker for GPL caching.

This commit breaks capturing shaders with RGP if the offset between
shaders is too big. Next commit should fix it by using shaders reloc.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21078>
2023-02-10 13:42:14 +00:00
Samuel Pitoiset
26896616c1 radv: stop skipping the cache for compute/raytracing pipelines with GPL
This was a hard solution somewhat.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21198>
2023-02-09 08:11:45 +00:00
Samuel Pitoiset
87e055a700 radv: pass the number of stages to radv_hash_shaders()
This will help for splitting radv_create_shaders().

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20943>
2023-01-30 09:37:52 +00:00
Konstantin Seurer
cdd68cbafd radv/rt: Hash the pipeline key
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>
2022-12-12 18:18:32 +00:00
Illia Abernikhin
aa4ac5ff8b utils: Merge util/debug.* into util/u_debug.* and remove util/debug.*
Rename env_var_as_unsigned() -> debug_get_num_option(), because duplicate
Rename env_var_as_bool() -> debug_get_bool_option(), because duplicate

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7177

Signed-off-by: Illia Abernikhin <illia.abernikhin@globallogic.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19336>
2022-11-02 07:25:39 +00:00
Samuel Pitoiset
d4ec3f21cf Revert "radv: add a pointer to radv_shader_binary in radv_shader"
This is actually not necessary because we compile and upload binaries
directly from libraries with GPL. This introduced random double free
crashes because binaries were potentially freed by concurrent threads.

Root cause found by Ishi.

This reverts commit f8d887527a.

Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19383>
2022-10-31 12:16:38 +00:00
Bas Nieuwenhuizen
f00b54bc9e radv: Rework modification detection of the meta cache.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19047>
2022-10-12 12:09:48 +02:00
Bas Nieuwenhuizen
8231f7eaa1 radv: Use a VkPipelineCache handle for meta shaders.
Prep work for using the common vk caches.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19047>
2022-10-12 12:09:48 +02:00