Commit graph

184774 commits

Author SHA1 Message Date
Marek Olšák
3cc2562ac0 winsys/amdgpu: remove now-redundant amdgpu_bo_slab_entry::real
The pb_slab pointer can be used to get the BO pointer because pb_slab is
inside the BO structure now.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
49bf2545fe winsys/amdgpu: add amdgpu_bo_real_reusable slab for the backing buffer
Add contents of amdgpu_bo_slab into it. This will allow removing the "real"
pointer from amdgpu_bo_slab_entry because "(char*)entry.slab" is now
pointing next to it.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
cf2dc2d512 winsys/amdgpu: don't layer slabs, use only 1 level of slabs, it improves perf
This increases FPS in VP2020/Catia1 by 10-18%!!!!!!!!!!!!!!!!!!!!!!!

I have no rational explanation for this.

In the most extreme case, 8192 256B slab BOs (smallest size) are now
allocated from a single 2MB slab.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
4a078e693e r300,r600,radeon/winsys: always pass the winsys to radeon_bo_reference
This will allow the removal of pb_cache_entry::mgr.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
643f390de5 radeon_winsys: add struct radeon_winsys* parameter into fence_reference
Since the radeon winsys implements fences as buffers, we need radeon_winsys*
to destroy them. This will enable the removal of pb_cache_entry::mgr.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
e847c6e301 gallium/pb_cache: switch time variables to milliseconds and 32-bit type
to decrease pb_cache_entry by 8 bytes.

Add msecs_base_time to offset time == 0 to the creation of pb_cache.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
896c8b67cb gallium/pb_cache: remove pb_cache_entry::end to save space
just compute it at each use

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
523a4f71f2 winsys/amdgpu: stop using pb_buffer::vtbl
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
7752579202 winsys/amdgpu: rename to amdgpu_bo_slab to amdgpu_bo_slab_entry
It's a slab entry. "Slab" is the whole buffer, which is AMDGPU_BO_REAL
if we want to be precise.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
b3c64638b4 iris,zink,winsys/amdgpu: remove unused/redundant slab->entry_size
slab->base has the same field.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
9431c33899 gallium/pb_slab: move group_index and entry_size from pb_slab_entry to pb_slab
This removes 8 bytes from every slab entry, and thus amdgpu_bo_slab.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
5a3bacc376 winsys/amdgpu: reduce wasted memory due to the size tolerance in pb_cache
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Daniel Schürmann
023e78b4d7 aco: add new post-RA scheduler for ILP
Totals from 77247 (97.37% of 79330) affected shaders: (GFX11)

Instrs: 44371374 -> 43215723 (-2.60%); split: -2.64%, +0.03%
CodeSize: 227819532 -> 223188224 (-2.03%); split: -2.06%, +0.03%
Latency: 301016823 -> 290147626 (-3.61%); split: -3.70%, +0.09%
InvThroughput: 48551749 -> 47646212 (-1.87%); split: -1.88%, +0.01%
VClause: 870581 -> 834655 (-4.13%); split: -4.13%, +0.00%
SClause: 1487061 -> 1340851 (-9.83%); split: -9.83%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25676>
2024-01-06 11:30:42 +00:00
Daniel Schürmann
72a5c659d4 aco: form clauses for LDS instructions
No fossil-db changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25676>
2024-01-06 11:30:42 +00:00
Daniel Schürmann
8f16745821 aco: fix should_form_clause() for memory instructions without operands
In particular, this applies to s_memtime and s_memrealtime.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25676>
2024-01-06 11:30:41 +00:00
Vinson Lee
568f61787a ac/rgp: Fix single-bit-bitfield-constant-conversion warning
../src/amd/common/ac_rgp.c:119:48: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
  119 |    header->flags.is_semaphore_queue_timing_etw = 1;
      |                                                ^ ~

Fixes: ed0c852243 ("radv: add initial SQTT files generation support")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26839>
2024-01-05 22:42:58 -08:00
Karol Herbst
1e5bc00715 rusticl/program: add LLVM functions to cache timestamp
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24386>
2024-01-06 03:09:48 +00:00
Karol Herbst
299f949775 rusticl/meson: generate bindings for LLVM
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24386>
2024-01-06 03:09:48 +00:00
Karol Herbst
d72544eea6 ci,rusticl: bump meson req to 1.3.1
See https://github.com/mesonbuild/meson/pull/12620

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24386>
2024-01-06 03:09:48 +00:00
Michael Tretter
d9fd4d7c0d egl/wayland: fix formatting and add trailing comma
Commit 659bace01a ("egl: add a few trailing commas") added a few
trailing commas to the last item in a struct to avoid that commit
e85983d772 ("egl: re-format using clang-format") moves the closing
brace into the previous line.

The wl_callback_listener was missing the comma and the brace was moved
to the previous line, which makes it harder to read the code.

Add the comma and fix the formatting.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26902>
2024-01-06 02:19:02 +00:00
Yonggang Luo
08b6f786ac nouveau: Use align64 instead of ALIGN for 64 bit value parameter
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26868>
2024-01-06 02:05:09 +00:00
Yonggang Luo
06163884b2 nvk: Avoid use align as variable, replace it with alignment
align is a function and when we want use it, the align variable will shadow it
So replace it with alignment

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26868>
2024-01-06 02:05:09 +00:00
Michael Catanzaro
481e737fe0 util: create parents of disk cache directory if needed
Currently the shader cache is broken when running under flatpak-spawn
because the shader cache's parent directory will not exist. For example,
the shader cache directory might be:

/home/mcatanzaro/.var/app/org.gnome.Epiphany.Devel/cache

If /home/mcatanzaro/.var/app/org.gnome.Epiphany.Devel/ does not already
exist, we fail. Let's create the directories recursively, as if by
'mkdir -p', rather than just fail.

Fixes #8294

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25925>
2024-01-06 01:19:29 +00:00
Eric Engestrom
a0fab95bc0 lvp: update symbols that have become aliases for newer ones
All of these have been renamed in the spec (usually by being promoted);
renamed them in our code too.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26494>
2024-01-06 00:49:53 +00:00
Mark Janes
2236dc3481 intel/dev: update workaround definitions to latest defect status
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26898>
2024-01-05 22:51:46 +00:00
Mark Janes
590fe58ef6 intel: remove MTL a0 workarounds
Meteorlake shipped with the b0 stepping.  Remove fixes for hardware
bugs that were corrected prior to the platform release.

Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26898>
2024-01-05 22:51:46 +00:00
Mark Janes
a6a95591aa intel/dev: poison macros for workarounds fixed at a stepping
INTEL_NEEDS_WA macros are valid when a workaround applies to all
platforms which have the GFX_VERx10 versions for the workaround.

Some workarounds were fixed at a stepping after the platform release.
If a workaround applies partially to any platform, then GFX_VERx10
cannot be used to correctly apply the workaround.

This change invalidates INTEL_NEEDS_WA_16014538804 and
INTEL_NEEDS_WA_22014412737, which were fixed for MTL platforms at
stepping b0.  The run-time checks were already present for all uses of
these macros.  Updating the poisoned macros to INTEL_WA_{num}_GFX_VER
compiles out the run-time checks on platforms where they cannot apply.

Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26898>
2024-01-05 22:51:45 +00:00
Mark Janes
7354d3a947 intel/dev: improve descriptions of workaround macros.
Instructions for INTEL_WA_{num}_GFX_VER macros were confusing and
contradicted itself.

Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26898>
2024-01-05 22:51:45 +00:00
Yonggang Luo
d6c258d9ee util: Add align_uintptr and use it treewide to replace ALIGN that works on size_t and uintptr_t
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26866>
2024-01-05 21:54:35 +00:00
Yonggang Luo
aa8ea0f1b9 glsl: Fixes glcpp/tests with mingw/gcc
glcpp mingw failing in the following way:
```
49/56 mesa:compiler+glcpp / glcpp test (unix)     FAIL             3.39s   exit status 1
50/56 mesa:compiler+glcpp / glcpp test (oldmac)   FAIL             3.41s   exit status 1
51/56 mesa:compiler+glcpp / glcpp test (bizarro)  FAIL             3.42s   exit status 1
52/56 mesa:compiler+glcpp / glcpp test (windows)  FAIL             3.45s   exit status 1
```

The test failed because on mingw, the stderr will comes after stdout,
but all the expect files, the stderr is coming first,
so we flush(stderr) first to makesure stderr out before stdout

The failing example:

039-func-arg-obj-macro-with-comma: FAIL
---

+++

@@ -1,3 +1,5 @@

+0:12(21): preprocessor error: Error: macro foo invoked with 2 arguments (expected 1)
+

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26778>
2024-01-05 21:15:58 +00:00
Rhys Perry
ae54cbeb3f nir: remove sad_u8x4
All uses of this can be replaced with msad_4x8.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
2024-01-05 18:55:22 +00:00
Rhys Perry
5fd747a502 radv: enable msad_4x8
This helps some FSR3 shaders.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
2024-01-05 18:55:22 +00:00
Rhys Perry
a339699b5c ac/llvm: implement msad_4x8
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
2024-01-05 18:55:22 +00:00
Rhys Perry
1410735a62 aco: implement msad_4x8
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
2024-01-05 18:55:22 +00:00
Rhys Perry
e86ab8173b nir/algebraic: optimize vkd3d-proton's MSAD
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
2024-01-05 18:55:22 +00:00
Rhys Perry
0477421f7d nir: add msad_4x8
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
2024-01-05 18:55:22 +00:00
Konstantin Seurer
c511b8968a radv: Implement VK_KHR_ray_tracing_position_fetch
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26895>
2024-01-05 18:20:20 +00:00
Rhys Perry
24ef827f71 radv: remove radv_shader_info's cs.subgroup_size
This is the same as wave_size.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26894>
2024-01-05 17:35:48 +00:00
Rhys Perry
59dbe633e3 radv: use CS wave selection for task shaders
This uses wave32 for small workgroups and wave64 when certain subgroup
operations are used.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26894>
2024-01-05 17:35:48 +00:00
Rhys Perry
3009dcd102 aco: correctly set min/max_subgroup_size for wave32-as-wave64
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26894>
2024-01-05 17:35:48 +00:00
Friedrich Vock
1e3541728b radv,aco: Convert 1D ray launches to 2D
Because we use unaligned dispatches, 1D launches only use 8 threads per
wave. Converting to 2D and fixing up launch IDs in the prolog
significantly increases occupancy.

Gives ~30% uplift in Ghostwire Tokyo.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26105>
2024-01-05 17:08:05 +00:00
Juan A. Suarez Romero
12563a527f ci/v3d: add support for rpi5
Add full Vulkan CTS testing for the new V3D 7.1 driver, used in the
Raspberry Pi 5.

So far we add it to run nightly; in future will be added to pre-merge
CI.

Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26705>
2024-01-05 16:10:50 +00:00
Mary Guillemard
b6d828576e zink: Always fill external_only in zink_query_dmabuf_modifiers
Fix piglit.spec@ext_image_dma_buf_import@ext_image_dma_buf_import-modifiers
randomly skipping some tests as external_only content was never initialized.

Cc: mesa-stable

Reviewed-by: default avatarMike Blumenkrantz <michael.blumenkrantz@gmail.com>
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26305>
2024-01-05 13:30:45 +00:00
Mary Guillemard
db0f177edd zink: Initialize pQueueFamilyIndices for image query / create
Fixes: d922850e36 ("zink: break out image/buffer create info structs into helper funcs")

Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26303>
2024-01-05 13:59:49 +01:00
Georg Lehmann
71edf4de5e aco/gfx12: implement broadcast dmask shrink behavior
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26897>
2024-01-05 12:03:54 +00:00
Georg Lehmann
4a6ee2c483 aco: shrink buffer stores with undef/zero components
Buffer stores store 0 like image stores for unspecified components.

Foz-DB Navi21:
Totals from 91 (0.11% of 79330) affected shaders:
Instrs: 63327 -> 63121 (-0.33%)
CodeSize: 315312 -> 314440 (-0.28%); split: -0.28%, +0.00%
VGPRs: 3144 -> 3120 (-0.76%)
Latency: 441424 -> 441300 (-0.03%); split: -0.03%, +0.00%
InvThroughput: 65501 -> 65130 (-0.57%)
Copies: 6197 -> 5999 (-3.20%)
PreVGPRs: 2197 -> 2182 (-0.68%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26897>
2024-01-05 12:03:54 +00:00
Connor Abbott
862df28f6b ir3/legalize: Fix helper propagation with b.any/b.all/getone
We need to set uses_helpers_beginning in order to propagate it to
predecessor blocks.

Fixes: aa322a37fc ("ir3: Implement helper invocation optimization")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26564>
2024-01-05 11:37:09 +00:00
Yonggang Luo
91ec3cc7e2 panfrost: Avoid use align as variable, replace it with other names
align is a function and when we want use it, the align variable will shadow it
So replace it with other names

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26885>
2024-01-05 02:42:05 +00:00
Yonggang Luo
b389bccccd util,compiler: Avoid use align as variable, replace it with other names
align is a function and when we want use it, the align variable will shadow it
So replace it with other names

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26885>
2024-01-05 02:42:05 +00:00
Karol Herbst
5ff33f9905 rusticl: use real buffer for cb0 for drivers prefering
At the moment it's radeonsi and zink.

Consequentially this also fixes data races in zink due to driver internal
reasons.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25946>
2024-01-05 01:26:44 +01:00