Add a new environment varible
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS_DYNAMICE_LIST that specifies a text
file containing a list of RO fossilize caches to load. The list file
is modifiable at runtime to allow for loading RO caches after
initialization unlike MESA_DISK_CACHE_READ_ONLY_FOZ_DBS.
The implementation spawns an updater thread that uses inotify to monitor
the list file for modifications, attempting to load new foz dbs added to
the list. Removing files from the list will not evict a loaded cache.
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS_DYNAMIC_LIST takes an absolute path.
The file must exist at initialization for updating to occur.
File names of foz dbs in the list file are new-line separated and take
relative paths to the default cache directory like
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS.
The maximum number of RO foz dbs is kept to 8 and is shared between
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS_DYNAMIC_LIST and
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS.
The intended use case for this feature is to allow prebuilt caches
to be downloaded and loaded asynchronously during app runtime.
Prebuilt caches be large (several GB) and depending on network
conditions would otherwise present extended wait time for caches
to be availible before app launch.
This will be used in Chrome OS.
Signed-off-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19328>
When combining of ro+rw caches is enabled, at first the ro cache should be
looked up and if data isn't found there then rw cache should be checked.
The rw cache checking got lost by accident after the code rebase and there
was no unit test covering this condition. Fix the rw cache looking up and
add the unit test case.
Fixes: 32fe60e8c4 ("util/disk_cache: Support combined foz ro and non-foz rw caches")
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Juston Li <justonli@google.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20595>
Mesa utilizes only one type of cache at a time. This patch enables support
for combined reading from read-only Fossilize cache + non-foz read-write
caches.
From now on, a non-foz read-write caches will first try to retrieve data
from a read-only foz cache if new MESA_DISK_CACHE_COMBINE_RW_WITH_RO_FOZ
environment variable is set to true, otherwise the caching behaviour is
unchanged. The new flag has no effect when MESA_DISK_CACHE_SINGLE_FILE=1,
i.e. when the single-file foz cache is used.
This change allows us to ship a prebuilt RO caches for a certain
applications, while the rest of applications will benefit from the
regular RW caching that supports cache-size limitation. This feature
will be used by ChromeOS.
Usage example #1:
MESA_DISK_CACHE_DATABASE=0
MESA_DISK_CACHE_SINGLE_FILE=0
MESA_DISK_CACHE_COMBINE_RW_WITH_RO_FOZ=1
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS=rocache1,rocache2
Usage example #2:
MESA_DISK_CACHE_DATABASE=1
MESA_DISK_CACHE_SINGLE_FILE=0
MESA_DISK_CACHE_COMBINE_RW_WITH_RO_FOZ=1
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS=rocache1,rocache2
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18551>
Introduce new cache type, the Mesa-DB. This is a single-file read/write
cache that is based on the read-only Fossilize DB cache. Mesa-DB supports
cache size capping. It's a much more efficient cache than the multi-file
cache because Mesa-DB doesn't have the inode overhead. The plan is to make
Mesa-DB the default cache implementation once it will be deemed as stable
and well tested. For now users have to set the new MESA_DISK_CACHE_DATABASE
environment variable in order to active the Mesa-DB cache.
Mesa-DB cache is resilient to corrupted cache files and doesn't require
maintenance from users and developers. The size capping is implemented
by evicting least recently used cache items and compacting the cache
database files with the evicted entries. In order to prevent frequent
compaction of the cache, at minimum a half of cache is evicted when cache
is full.
Acked-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16888>
Mesa caches are compressed by default and we just added option to disable
compression that will be needed by the new Mesa-DB cache. Let's enable
testing of uncompressed caches for the multi-file and foz-db caches in
addition to the compressed to increase test coverage of the new code.
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Suggested-by: Emil Velikov <emil.velikov@collabora.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16888>
Rename MESA_GLSL_CACHE to MESA_SHADER_CACHE, as the on-disk cache can
store not only GLSL but also SPIR-V shaders.
v2:
- Keep old envvar as deprecated (Mike)
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15390>
