Commit graph

231 commits

Author SHA1 Message Date
Mauro Rossi
6facb0c08f android: fix libz dynamic library dependencies
Fixes a series of libz related building errors:

target SharedLib: gallium_dri_32
(out/target/prod...SHARED_LIBRARIES/gallium_dri_intermediates/LINKED/gallium_dri.so)
external/elfutils/libelf/elf_compress.c:117: error: undefined reference to 'deflateInit_'
...
external/elfutils/libelf/elf_compress.c:244: error: undefined reference to 'inflateEnd'
clang++: error: linker command failed with exit code 1 (use -v to see
invocation)

Fixes: 85a9b1b "util/disk_cache: compress individual cache entries"
2017-03-04 21:47:26 +00:00
Timothy Arceri
85a9b1b562 util/disk_cache: compress individual cache entries
This reduces the cache size for Deus Ex from ~160M to ~30M for
radeonsi (these numbers differ from Grigori's results below
probably due to different graphics quality settings).

I'm also seeing the following improvements in minimum fps in the
Shadow of Mordor benchmark on an i5-6400 CPU@2.70GHz, with a HDD:

no-cache:                    ~10fps
with-cache-no-compression:   ~15fps
with-cache-and-compression:  ~20fps

Note: The with cache results are from the second run after closing
and opening the game to avoid the in-memory cache.

Since we mainly care about decompression I went with
Z_BEST_COMPRESSION as suggested on irc by Steinar H. Gunderson
who has benchmarked decompression speeds.

Grigori Goronzy provided the following stats for Deus Ex: Mankind
Divided start-up times on a Athlon X4 860k with a SSD:

No Cache                                 215 sec

Cold Cache zlib BEST_COMPRESSION         285 sec
Warm Cache zlib BEST_COMPRESSION         33 sec

Cold Cache zlib BEST_SPEED               264 sec
Warm Cache zlib BEST_SPEED               33 sec

Cold Cache no compression                266 sec
Warm Cache no compression                34 sec

The total cache size for that game is 48 MiB with BEST_COMPRESSION,
56 MiB with BEST_SPEED and 170 MiB with no compression.

These numbers suggest that it may be ok to go with Z_BEST_SPEED
but we should gather some actual decompression times before doing
so. Other options might be to do the compression in a separate
thread, this might allow us to use a higher compression algorithim
such as LZMA.

Reviewed-by: Grigori Goronzy <greg@chown.ath.cx>
Acked-by: Marek Olšák <marek.olsak@amd.com>
2017-03-03 12:09:08 +11:00
Timothy Arceri
5afde61752 util/disk_cache: add support for detecting corrupt cache entries
V2: fix pointer increments for writing/reading crc

Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Grigori Goronzy <greg@chown.ath.cx>
2017-03-03 12:09:08 +11:00
Jonas Pfeil
cd2b55e536 ralloc: Make sure ralloc() allocations match malloc()'s alignment.
The header of ralloc needs to be aligned, because the compiler assumes
that malloc returns will be aligned to 8/16 bytes depending on the
platform, leading to degraded performance or alignment faults with ralloc.

Fixes SIGBUS on Raspberry Pi at high optimization levels.

This patch is not perfect for MSVC, as maybe in the future the alignment
for the most demanding data type might change to more than 8.

v2: Commit message reword/typo fix, and add a bigger explanation in the
    code (by anholt)

Signed-off-by: Jonas Pfeil <pfeiljonas@gmx.de>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Cc: mesa-stable@lists.freedesktop.org
2017-03-02 13:01:45 -08:00
Jason Ekstrand
e647c4fbd9 util/build-id: Return a pointer rather than copying the data
We're about to use the build-id as the starting point for another SHA1
hash in the Intel Vulkan driver, and returning a pointer is far more
convenient.

Reviewed-by: Chad Versace <chadversary@chromium.org>
2017-03-01 15:31:44 -08:00
Kenneth Graunke
63d1ebca3a ralloc: Delete autofree handling.
There was exactly one user of this, and I just removed it.

It also accessed an implicit global context, with no locking.  This
meant that it was only safe if all callers of ralloc_autofree_context()
held the same lock...which is a pretty terrible thing for a utility
library to impose.

Signed-off-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
2017-02-27 15:46:12 -08:00
Vinson Lee
30a4b25efe util/disk_cache: Use backward compatible st_mtime.
Fix Mac OS X build error.

  CC       libmesautil_la-disk_cache.lo
In file included from disk_cache.c:46:
./disk_cache.h:57:20: error: no member named 'st_mtim' in 'struct stat'
   *timestamp = st.st_mtim.tv_sec;
                ~~ ^

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=99918
Fixes: 207e3a6e4b ("util/radv: move *_get_function_timestamp() to utils")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
2017-02-24 16:06:40 -08:00
Emil Velikov
9833488974 util: automake: add sha1/README to the tarball
Suggested-by: Andreas Boll <andreas.boll.dev@gmail.com>
Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
2017-02-24 17:38:16 +00:00
Timothy Arceri
d258055c8b util/disk_cache: fix bug with deleting old cache dirs
If there was more than a single directory in the .cache/mesa dir
then it would only remove one (or none) of the directories.

Apparently Valgrind was also reporting:
Conditional jump or move depends on uninitialised value

Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
2017-02-23 09:20:22 +11:00
Brian Paul
b87ef9e606 util: fix MSVC build issue in disk_cache.h
Windows doesn't have dlfcn.h.  Protect the code in question
with #if ENABLE_SHADER_CACHE test.  And fix indentation.

Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
2017-02-21 20:54:46 -07:00
Timothy Arceri
0441e6bc8b util/disk_cache: create timestamp and gpu_id dirs when MESA_GLSL_CACHE_DIR is used
The make check test is also updated to make sure these dirs are created.

Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
2017-02-22 08:40:14 +11:00
Timothy Arceri
207e3a6e4b util/radv: move *_get_function_timestamp() to utils
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
2017-02-22 08:40:00 +11:00
Jonathan Gray
a042465c21 util/build-id: define ElfW and NT_GNU_BUILD_ID if needed
Define ElfW() and NT_GNU_BUILD_ID if needed as these defines are not
present on at least OpenBSD and FreeBSD.  Fixes the build on OpenBSD.

Fixes: d4fa083e11 ("util: Add utility build-id code.")
Signed-off-by: Jonathan Gray <jsg@jsg.id.au>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
2017-02-20 16:39:24 +00:00
Timothy Arceri
62c90492ef glsl: disable on disk shader cache when running as another user
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
2017-02-17 20:21:22 +11:00
Timothy Arceri
a2bf0954fb util/disk_cache: fix typo in function stub 2017-02-17 15:54:00 +11:00
Timothy Arceri
a3ab09f90f util/disk_cache: check cache exists before calling munmap()
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
2017-02-17 11:18:43 +11:00
Timothy Arceri
512c046edd util/disk_cache: add support for removing old versions of the cache
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
2017-02-17 11:18:43 +11:00
Timothy Arceri
3342ce452c util/disk_cache: allow drivers to pass a directory structure
In order to avoid costly fallback recompiles when cache items are
created with an old version of Mesa or for a different gpu on the
same system we want to create directories that look like this:

./{TIMESTAMP}_{LLVM_TIMESTAMP}/{GPU_ID}

Note: The disk cache util will take a single timestamp string, it is
up to the backend to concatenate the llvm string with the mesa string
if applicable.

Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
2017-02-17 11:18:43 +11:00
Emil Velikov
cadf174866 util/disk_cache: do not allow space in MESA_GLSL_CACHE_MAX_SIZE
No other env var used in mesa allows for space in the variable contents.

Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
Acked-by: Timothy Arceri <tarceri@itsqueeze.com>
2017-02-16 15:22:17 +00:00
Matt Turner
d4fa083e11 util: Add utility build-id code.
Provides the ability to read the .note.gnu.build-id section of ELF
binaries, which is inserted by the --build-id=... flag to ld.

Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Reviewed-by: Chad Versace <chadversary@chromium.org>
2017-02-15 13:59:51 -08:00
Jason Ekstrand
a9a517f530 util: Fix a typo in Makefile.sources
Reviewed-by: Eric Engestrom <eric.engestrom@imgtec.com>
2017-02-15 10:27:42 -08:00
Jason Ekstrand
f76584e7b7 util: Add helpers for iterating over Vulkan extension structs
Reviewed-by: Dave Airlie <airlied@redhat.com>
2017-02-14 16:15:39 -08:00
Timothy Arceri
0cbde643eb util/disk_cache: correctly use stat(3)
I forgot to error check stat() and also I wasn't using the subdir in
is_two_character_sub_directory().

Fixes: d7b3707c61 "util/disk_cache: use stat() to check if entry is a directory"
Reviewed-by: Plamena Manolova <plamena.manolova@intel.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
2017-02-13 10:01:12 +00:00
Timothy Arceri
d7b3707c61 util/disk_cache: use stat() to check if entry is a directory
d_type is not supported on all systems.

Tested-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=97967
2017-02-10 23:50:36 +11:00
Timothy Arceri
a4086bb531 util/disk_cache: error check asprintf()
Fixes: f3d911463e "util/disk_cache: stop using ralloc_asprintf() unnecessarily"

Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Reviewed-by: Eric Engestrom <eric.engestrom@imgtec.com>
2017-02-10 09:25:32 +11:00
Timothy Arceri
f3d911463e util/disk_cache: stop using ralloc_asprintf() unnecessarily
Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2017-02-09 14:11:24 +11:00
Timothy Arceri
4026b45bbc util: add a disk_cache_remove() function
This will be used to remove cache items created with old versions
of Mesa or other invalid cache items from the cache.

V2: rename stub function (cache_* funtions were renamed disk_cache_*)
in master.

Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
2017-02-09 12:22:56 +11:00
Carl Worth
0f60c6616e glsl: Switch to disable-by-default for the GLSL shader cache
The shader cache is expected to be developed incrementally over a
fairly long series of commits. For that period of instability, we
require users to opt into the shader cache by setting:

	MESA_GLSL_CACHE_ENABLE=1

In the future, when the shader cache is complete, we can revert this
commit so that the cache will be on by default.

The user can always disable the cache with
MESA_GLSL_CACHE_DISABLE=1. That functionality is not affected by this
commit, (nor will it be affected by the future revert).

Reviewed-by: Eric Anholt <eric@anholt.net>
2017-01-31 09:51:30 +11:00
Emil Velikov
ea8b2624c8 utils: really remove the __END_DECLS macro
Fixes: d1efa09d34 "util: import sha1 implementation from OpenBSD"
Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
2017-01-18 20:09:57 +00:00
Emil Velikov
9f8dc3bf03 utils: build sha1/disk cache only with Android/Autoconf
Earlier commit imported a SHA1 implementation and relaxed the SHA1 and
disk cache handling, broking the Windows builds.

Restrict things for now until we get to a proper fix.

Fixes: d1efa09d34 "util: import sha1 implementation from OpenBSD"
Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
2017-01-18 20:09:01 +00:00
Emil Velikov
d1efa09d34 util: import sha1 implementation from OpenBSD
At the moment we support 5+ different implementations each with varying
amount of bugs - from thread safely problems [1], to outright broken
implementation(s) [2]

In order to accommodate these we have 150+ lines of configure script and
extra two configure toggles. Whist an actual implementation being
~200loc and our current compat wrapping ~250.

Let's not forget that different people use different code paths, thus
effectively makes it harder to test and debug since the default
implementation is automatically detected.

To minimise all these lovely experiences, import the "100% Public
Domain" OpenBSD sha1 implementation. Clearly document any changes needed
to get building correctly, since many/most of those can be upstreamed
making future syncs easier.

As an added bonus this will avoid all the 'fun' experiences trying to
integrate it with the Android and SCons builds.

v2: Manually expand __BEGIN_DECLS/__END_DECLS and document (Tapani).

Furthermore it seems that some games (or surrounding runtime) static
link against OpenSSL resulting in conflicts. For more information see
the discussion thread [3]

Bugzilla [1]: https://bugs.freedesktop.org/show_bug.cgi?id=94904
Bugzilla [2]: https://bugs.freedesktop.org/show_bug.cgi?id=97967
[3] https://lists.freedesktop.org/archives/mesa-dev/2017-January/140748.html
Cc: Mark Janes <mark.a.janes@intel.com>
Cc: Vinson Lee <vlee@freedesktop.org>
Cc: Tapani Pälli <tapani.palli@intel.com>
Cc: Jonathan Gray <jsg@jsg.id.au>
Tested-by: Jonathan Gray <jsg@jsg.id.au>
Signed-off-by: Emil Velikov <emil.velikov@collabora.com>
Acked-by: Tapani Pälli <tapani.palli@intel.com> (v1)
Acked-by: Jason Ekstrand <jason@jlekstrand.net> (v1)
2017-01-18 19:07:23 +00:00
Marek Olšák
44e9b67229 radeonsi: make fix_fetch 64-bit
v2: add u_bit_consecutive64

Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
2017-01-16 18:07:08 +01:00
Timothy Arceri
0252ba26c5 util: fix list_is_singular()
Currently its dependant on the user calling and checking the result
of list_empty() before using the result of list_is_singular().

Cc: "13.0" <mesa-stable@lists.freedesktop.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2017-01-12 13:58:32 +11:00
Marek Olšák
86514d84e0 util: import CRC32 implementation from gallium
Reviewed-by: Timothy Arceri <timothy.arceri@collabora.com>
2016-11-22 18:05:51 +01:00
Gwan-gyeong Mun
69cc7d90f9 util/disk_cache: close a previously opened handle in disk_cache_put (v2)
We're missing the close() to the matching open().

CID 1373407

v2: Fixes from Emil Velikov's review
    Update the teardown in reverse order of the setup/init.

Cc: "13.0" <mesa-stable@lists.freedesktop.org>
Signed-off-by: Mun Gwan-gyeong <elongbug@gmail.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> (v1)
2016-11-22 15:13:42 +00:00
George Kyriazis
c323180733 mesa: removed redundant #else
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
2016-11-21 12:44:46 -06:00
Jason Ekstrand
9b9fb6d212 util/vk_alloc: Add a vk_zalloc2 helper
Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
2016-11-17 12:03:24 -08:00
Jason Ekstrand
d0646c8015 util/vk_alloc: Ensure NULL is handled correctly in vk_free
Reviewed-by: Dave Airlie <airlied@redhat.com>
2016-11-16 20:07:23 -08:00
Vinson Lee
ed6694d511 util: Fix Clang trivial destructor check.
Check for Clang before GCC.

Clang defines __GNUC__ == 4 and __GNUC_MINOR__ == 2 and matches the GCC
check but not the GCC version for trivial destructor.

Fixes: 98ab905af0 ("mesa: Define introspection macro to determine
whether a type is trivially destructible.")
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=98526
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
2016-11-15 17:35:56 -08:00
Marek Olšák
a6ff2a3378 util/disk_cache: use unambiguous naming
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
2016-11-15 20:22:28 +01:00
Marek Olšák
31727300e1 util: import cache.c/h from glsl
It's not dependent on GLSL and it can be useful for shader caches that don't
deal with GLSL.

v2: address review comments
v3: keep the other 3 lines in configure.ac

Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
2016-11-15 20:22:28 +01:00
Tapani Pälli
ec725dc140 util: use STATIC_ASSERT instead of static_assert
fixes following compilation warnings on Android build:

"warning: implicit declaration of function 'static_assert' is invalid in
C99 [-Wimplicit-function-declaration]"

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Emil Velikov <emil.velikov@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
2016-11-11 07:59:29 +02:00
Brian Paul
5b92008ae2 util: add MSVC HAS_TRIVIAL_DESTRUCTOR implementation
Based on a patch by George Kyriazis but changed to test for
_MSC_VER >= 1800 (Visual Studio 2015).

This fixes the failed CANARY assertion in src/util/ralloc.c:get_header()
on Windows.

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=98595
Tested-by: Brian Paul <brianp@vmware.com>
Signed-off-by: Brian Paul <brianp@vmware.com>
2016-11-09 14:55:10 -07:00
Marek Olšák
6608dbf540 ralloc: add a linear allocator as a child node of ralloc
v2: remove goto, cosmetic changes

Tested-by: Edmondo Tommasina <edmondo.tommasina@gmail.com> (v1)
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
2016-10-31 11:53:38 +01:00
Marek Olšák
acc23b04cf ralloc: remove memset from ralloc_size
only do it in rzalloc_size as it was supposed to be

Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Tested-by: Edmondo Tommasina <edmondo.tommasina@gmail.com>
2016-10-31 11:53:38 +01:00
Marek Olšák
9454f7c0ef ralloc: add DECLARE_RZALLOC_CXX_OPERATORS
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Tested-by: Edmondo Tommasina <edmondo.tommasina@gmail.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Kenneth Graunke <kenneth@whiteacpe.org>
2016-10-31 11:53:38 +01:00
Marek Olšák
330482177c ralloc: don't memset ralloc_header, clear it manually
time GALLIUM_NOOP=1 ./run shaders/private/alien_isolation/ >/dev/null

Before (2 takes):

real    0m8.734s    0m8.773s
user    0m34.232s   0m34.348s
sys     0m0.084s    0m0.056s

After (2 takes):

real    0m8.448s    0m8.463s
user    0m33.104s   0m33.160s
sys     0m0.088s    0m0.076s

Average change in "real" time spent: -3.4%

calloc should only do 2 things compared to malloc:
- check for overflow of "n * size"
- call memset
I'm not sure if that explains the difference.

v2: clear "parent" and "next" in the caller of add_child.

Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net> (v1)
Tested-by: Edmondo Tommasina <edmondo.tommasina@gmail.com> (v1)
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com> (v1)
2016-10-31 11:53:38 +01:00
Vinson Lee
889ee4da05 util: Include string.h in bitscan.h.
Fix build error with clang.

  Compiling src/compiler/glsl/link_varyings.cpp ...
In file included from src/compiler/glsl/link_varyings.cpp:33:
In file included from src/compiler/glsl/glsl_symbol_table.h:34:
In file included from src/compiler/glsl/ir.h:33:
In file included from src/compiler/glsl_types.h:29:
/usr/include/string.h:518:12: error: exception specification in declaration does not match previous declaration
extern int ffs (int __i) __THROW __attribute__ ((__const__));
           ^
src/util/bitscan.h:51:13: note: expanded from macro 'ffs'
            ^
src/util/bitscan.h:96:18: note: previous declaration is here
   const int i = ffs(*mask) - 1;
                 ^
src/util/bitscan.h:51:13: note: expanded from macro 'ffs'
            ^

Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=97952
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Timothy Arceri <timothy.arceri@collabora.com>
2016-10-20 14:54:28 -07:00
Dave Airlie
0cfd428aef vulkan: add vk_alloc.h shared allocation inlines.
vulkan allocation allows for overriding the allocator used,
add some macros for anv/radv to share for this.

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Signed-off-by: Dave Airlie <airlied@redhat.com>
2016-10-19 09:05:26 +10:00
Dave Airlie
78bce52f9a util: move min/max/clamp macros to util macros.h
Although the vulkan drivers include mesa macros.h, for
radv I'd like to move away from that.

Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Signed-off-by: Dave Airlie <airlied@redhat.com>
2016-10-19 09:05:25 +10:00