In the linear allocator, when a size larger than the minimum
buffer size is allocated, we currently create the new buffer
to fit exactly the requested size.
In that case, don't bother updating the `latest` pointer, since
this newly created buffer is already full. In the worst case,
the current `latest` is full and it would be the same; in the
best case, there's still room for small allocations, so we avoid
wasting space if the next allocations would fit.
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25517>
Not comprehensive, but those were the ones used to work on the
previous linear_alloc changes. Also having a test already
set up lower the barrier to add more tests for future in case of
bugs.
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25280>
The disk_cache_create() now always returns valid cache even when disk
cache is disabled. In a case of disabled cache, the disk cache is NO-OP.
Test whether get/put() work as expected for the disabled cache.
Reviewed-by: Rob Clark <robdclark@chromium.org>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24985>
Previous commit decoupled EGL_ANDROID_blob_cache from the disk cache
and haven't updated the SHADER_CACHE_DISABLE_BY_DEFAULT test-case that
is failing because now cache is always created even if disk cache is
disabled, such cache is NO-OP in this case. Fix the failing test.
Fixes: 39f26642 ("util: Decouple disk cache from EGL_ANDROID_blob_cache")
Reviewed-by: Rob Clark <robdclark@chromium.org>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24985>
Test for disabled cache was removed when we decoupled
EGL_ANDROID_blob_cache from the disk cache because test was failing
since it became outdated. Add the updated test.
Fixes: 39f26642 ("util: Decouple disk cache from EGL_ANDROID_blob_cache")
Reviewed-by: Rob Clark <robdclark@chromium.org>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24985>
Previously rb_tree_search would return the first node encountered, but
that may not be the first node that would be encoutnered by, say,
rb_tree_foreach.
Two test cases are added.
v2: Add more curly braces. Suggested by Faith.
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24856>
Just because the user / system-integrater doesn't want shader disk
cache, doesn't mean they don't want EGL_ANDROID_blob_cache to work.
We've kind of already solved this for the android case, so just
generalize that solution.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9520
Signed-off-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24636>
It uses a poll function that waits for a second hoping for another thread
to catch up, which is not a reliable way to do synchronization. The test
has been spuriously failing merges on a regular basis recently.
This is issue #9222, which I'm leaving open until the author can fix the test.
Fixes: 3b69b67545 ("util/fossilize_db: add runtime RO foz db loading via FOZ_DBS_DYNAMIC_LIST")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24755>
Guard util to be standalone by move subdir('util') just behind subdir('android_stub')
texcompress_rgtc_tmp.h is not a standard c header, do not treat it as source
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19522>
Make the `error` variable static to prevent a clash with
the glibc error() function when LTO is used.
Fixes the LTO build.
Otherwise, it'll fail in the linking phase with a conflict:
```
[411/2321] Linking target src/util/process_test
FAILED: src/util/process_test
c++ -o src/util/process_test src/util/process_test.p/tests_process_test.c.o -flto -Wl,--as-needed -Wl,--no-undefined -Wl,--fatal-warnings -Wl,--start-group src/util/libmesa_util.a src/util/format/libmesa_format.a src/util/libmesa_util_sse41.a src/c11/impl/libmesa_util_c11.a subprojects/perfetto/libperfetto.a /usr/lib/x86_64-linux-gnu/libz.so -pthread -lm -ldl /usr/lib/x86_64-linux-gnu/libunwind.so -Wl,--end-group
mold: error: symbol type mismatch: error
>>> defined in /tmp/process_test.SLc9I6.ltrans0.ltrans.o as STT_OBJECT
>>> defined in /lib/x86_64-linux-gnu/libc.so.6 as STT_FUNC
```
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21511>
Whenever a single file mesa-db cache hits max size limit, a half of cache
is evicted and the cache file is defragmented. The downside of this eviction
strategy is that it causes high disk IO usage during eviction if mesa-db
cache file size is large.
In order to mitigate this downside, we will split mesa-db into multiple
part such that only one part will be evicted at a time. Each part will be
an individual single file mesa-db cache, like a DB shard. The new multipart
mesa-db cache will merge the parts into a single virtual cache.
This patch introduces two new environment variables:
1. MESA_DISK_CACHE_DATABASE_NUM_PARTS:
Controls number of mesa-db cache file parts. By default 50 parts will be
created. The old pre-multipart mesa-db cache files will be auto-removed
if they exist, i.e. Mesa will switch to the new DB version automatically.
2. MESA_DISK_CACHE_DATABASE_EVICTION_SCORE_2X_PERIOD:
Controls the eviction score doubling time period. The evicted DB part
selection is based on cache entries size weighted by 'last_access_time' of
the entries. By default the cache eviction score is doubled for each month
of cache entry age, i.e. for two equally sized entries where one entry is
older by one month than the other, the older entry will have x2 eviction
score than the other entry. Database part with a highest total eviction
score is selected for eviction.
This patch brings x40 performance improvement of cache eviction time using
multipart cache vs a single file cache due to a smaller eviction portions
and more optimized eviction algorithm.
Acked-by: Timothy Arceri <tarceri@itsqueeze.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20256>
The cache_exists() function adds a dummy cache entry that may affect cache
eviction testing results. Remove this entry.
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20256>
The environment variables are persistent and must be explicitly unset by
each cache test, otherwise next test will fail. Add the missing unsets to
the Cache.List test.
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20256>
Allow the loading process to affect driconf option matching without
changing the behavior throughout mesa common code or leaking the name of
the loading process to logs, artifact storage, or in sub-thread naming,
as can be the case with the broader MESA_PROCESS_NAME override.
This new MESA_DRICONF_EXECUTABLE_OVERRIDE takes higher precedence over
MESA_PROCESS_NAME in the case where both are set.
Signed-off-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20779>
Also deprecate GALLIUM_PROCESSS_NAME in favor of MESA_PROCESS_NAME,
while maintaining existing functionality for use cases relying on
GALLIUM_PROCESSS_NAME.
GALLIUM_PROCESSS_NAME takes higher precedence over MESA_PROCESS_NAME in
the case where both are set.
Signed-off-by: Ryan Neph <ryanneph@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20779>
Add a new environment varible
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS_DYNAMICE_LIST that specifies a text
file containing a list of RO fossilize caches to load. The list file
is modifiable at runtime to allow for loading RO caches after
initialization unlike MESA_DISK_CACHE_READ_ONLY_FOZ_DBS.
The implementation spawns an updater thread that uses inotify to monitor
the list file for modifications, attempting to load new foz dbs added to
the list. Removing files from the list will not evict a loaded cache.
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS_DYNAMIC_LIST takes an absolute path.
The file must exist at initialization for updating to occur.
File names of foz dbs in the list file are new-line separated and take
relative paths to the default cache directory like
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS.
The maximum number of RO foz dbs is kept to 8 and is shared between
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS_DYNAMIC_LIST and
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS.
The intended use case for this feature is to allow prebuilt caches
to be downloaded and loaded asynchronously during app runtime.
Prebuilt caches be large (several GB) and depending on network
conditions would otherwise present extended wait time for caches
to be availible before app launch.
This will be used in Chrome OS.
Signed-off-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19328>
When combining of ro+rw caches is enabled, at first the ro cache should be
looked up and if data isn't found there then rw cache should be checked.
The rw cache checking got lost by accident after the code rebase and there
was no unit test covering this condition. Fix the rw cache looking up and
add the unit test case.
Fixes: 32fe60e8c4 ("util/disk_cache: Support combined foz ro and non-foz rw caches")
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Juston Li <justonli@google.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20595>
Mesa utilizes only one type of cache at a time. This patch enables support
for combined reading from read-only Fossilize cache + non-foz read-write
caches.
From now on, a non-foz read-write caches will first try to retrieve data
from a read-only foz cache if new MESA_DISK_CACHE_COMBINE_RW_WITH_RO_FOZ
environment variable is set to true, otherwise the caching behaviour is
unchanged. The new flag has no effect when MESA_DISK_CACHE_SINGLE_FILE=1,
i.e. when the single-file foz cache is used.
This change allows us to ship a prebuilt RO caches for a certain
applications, while the rest of applications will benefit from the
regular RW caching that supports cache-size limitation. This feature
will be used by ChromeOS.
Usage example #1:
MESA_DISK_CACHE_DATABASE=0
MESA_DISK_CACHE_SINGLE_FILE=0
MESA_DISK_CACHE_COMBINE_RW_WITH_RO_FOZ=1
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS=rocache1,rocache2
Usage example #2:
MESA_DISK_CACHE_DATABASE=1
MESA_DISK_CACHE_SINGLE_FILE=0
MESA_DISK_CACHE_COMBINE_RW_WITH_RO_FOZ=1
MESA_DISK_CACHE_READ_ONLY_FOZ_DBS=rocache1,rocache2
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18551>
Getting debug_get_num_option to return int64_t, as long under 64 bit Linux are 64 bit size,
so using fixed int64_t for cross platform consistence, as long under win32 is 32 bit size.
Getting DEBUG_GET_ONCE_FLAGS_OPTION to return uint64_t to getting it to be
consistence with debug_get_flags_option.
DEBUG_GET_ONCE_NUM_OPTION is not accessed in codebase, so add unittest for it, it maybe
used in future, remove it is not consistence
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19554>
According to the deprecation note:
> It's a pure subset of meson.get_external_property, and works strangely
> in host == build configurations, since it would be more accurately
> described as get_host_property.
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19904>
To be more consistent with other environment variables and ensure
better scoping, all environment variables in utrace have now been
prefixed with `MESA_`.
Signed-off-by: Mark Collins <mark@igalia.com>
Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Ack-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18271>
All environment variables involved in utrace usage were very
fragmented and convoluted to decode the meaning of, this commit has
simplified them down into easier to understand flags which directly
indicate the resulting behavior (such as `perfetto` enabling queued
logs rather than needing to set a `queued` flag) while combining
them into a single envvar `GPU_TRACES` and updating existing
terminology in utrace to match up with the new options.
Signed-off-by: Mark Collins <mark@igalia.com>
Reviewed-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Ack-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18271>
This is done by find and replace:
boolean -> bool
TRUE -> true
FALSE -> false
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19649>
Add unit tests for debug_get_bool_option and debug_get_num_option
Merge env_var_as_boolean and debug_get_bool_option and implement
env_var_as_boolean with debug_get_bool_option in a stricter side.
Merge env_var_as_unsigned and debug_get_num_option and implement
env_var_as_unsigned with debug_get_num_option in a stricter side.
Move debug_control, parse_debug_string, parse_enable_string,
comma_separated_list_contains from debug.* to u_debug.*
Main changes:
os_get_option() is used instead of getenv() for env_var_as_boolean
and env_var_as_unsigned;
also debug_get_bool_option() has logic like "true" always if not "false";
env_var_as_boolean() now uses different logic:
if env variable is neither "true" nor "false" returns the default value,
we left the second one; but if you want the behavior to be the same as in
the old version of debug_get_bool_option() use dfault=true
Signed-off-by: Illia Abernikhin <illia.abernikhin@globallogic.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19336>
For hot path, there is only need to a load instruction to load if initialized are true now,
So the extra cost is minimal.
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18323>
Warning message:
../src/util/tests/mesa-sha1_test.cpp:42:1: warning: 'InstantiateTestCase_P_IsDeprecated' is deprecated: INSTANTIATE_TEST_CASE_P is deprecated, please use INSTANTIATE_TEST_SUITE_P [-Wdeprecated-declarations]
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18203>
Introduce new cache type, the Mesa-DB. This is a single-file read/write
cache that is based on the read-only Fossilize DB cache. Mesa-DB supports
cache size capping. It's a much more efficient cache than the multi-file
cache because Mesa-DB doesn't have the inode overhead. The plan is to make
Mesa-DB the default cache implementation once it will be deemed as stable
and well tested. For now users have to set the new MESA_DISK_CACHE_DATABASE
environment variable in order to active the Mesa-DB cache.
Mesa-DB cache is resilient to corrupted cache files and doesn't require
maintenance from users and developers. The size capping is implemented
by evicting least recently used cache items and compacting the cache
database files with the evicted entries. In order to prevent frequent
compaction of the cache, at minimum a half of cache is evicted when cache
is full.
Acked-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16888>
Mesa caches are compressed by default and we just added option to disable
compression that will be needed by the new Mesa-DB cache. Let's enable
testing of uncompressed caches for the multi-file and foz-db caches in
addition to the compressed to increase test coverage of the new code.
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Suggested-by: Emil Velikov <emil.velikov@collabora.com>
Signed-off-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16888>
As the default option for msvc 2019 does support designated initializers
```
../src/util/tests/timespec_test.cpp(302): error C7555: use of designated initializers requires at least '/std:c++20'
../src/util/tests/timespec_test.cpp(303): error C7555: use of designated initializers requires at least '/std:c++20'
../src/util/tests/timespec_test.cpp(312): error C7555: use of designated initializers requires at least '/std:c++20'
../src/util/tests/timespec_test.cpp(313): error C7555: use of designated initializers requires at least '/std:c++20'
```
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15497>
We also miss this function on MSVC. But let's use the functionality in
meson to check for supported functions instead of hand-rolling the list
here.
Fixes: 067023dce2 ("util: Add some unit tests of the half-float conversions.")
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16290>
Make sure that we're handling nans/infs correctly, in particular only
generating quiet nans when given a quiet nan. This catches the previous
qNaN fix.
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16233>
