Commit graph

182856 commits

Author SHA1 Message Date
Sergi Blanch Torne
11b707de0e ci: disable Collabora's LAVA lab for maintance
This is to inform you of some planned downtime in the LAVA lab as follows:
* Start: 2024-01-08 08:00 UTC
* End: 2024-01-08 14:00 UTC

Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26899>
2024-01-08 07:42:31 +00:00
Christian Gmeiner
1c042537e6 pan/mdg: Use nir_builder for load_sampler_lod_parameters_pan
Found by code inspection as I need something similar for etnaviv.

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26879>
2024-01-08 07:23:40 +00:00
Konstantin Seurer
4d02543853 vtn: Remove transpose(m0)*m1 fast path
This is broken for games that rely on invariant geometry since the usage
of matrices can affect how gl_Position is computed. The fdot fastpath
relied on if and how fdot is lowered for correctness.

Totals from 6578 (7.73% of 85071) affected shaders:
MaxWaves: 147190 -> 147170 (-0.01%)
Instrs: 4451406 -> 4438140 (-0.30%); split: -0.31%, +0.01%
CodeSize: 23553020 -> 23541772 (-0.05%); split: -0.07%, +0.03%
VGPRs: 302304 -> 302328 (+0.01%)
SpillSGPRs: 1309 -> 1329 (+1.53%)
Latency: 22509985 -> 22177164 (-1.48%); split: -1.48%, +0.00%
InvThroughput: 4862795 -> 4842951 (-0.41%); split: -0.41%, +0.01%
VClause: 85035 -> 84998 (-0.04%); split: -0.06%, +0.02%
SClause: 131008 -> 131055 (+0.04%); split: -0.02%, +0.05%
Copies: 298935 -> 298060 (-0.29%); split: -0.71%, +0.41%
PreSGPRs: 266833 -> 267292 (+0.17%); split: -0.85%, +1.03%
PreVGPRs: 249511 -> 249601 (+0.04%)

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9562
cc: mesa-stable

Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26821>
2024-01-07 22:14:36 +00:00
Konstantin Seurer
ba7b08e324 radv/rt: Repurpose radv_ray_tracing_stage_is_compiled
Replace it with radv_ray_tracing_stage_is_always_inlined and use it inside
radv_rt_compile_shaders.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25224>
2024-01-07 21:28:32 +01:00
Konstantin Seurer
73cc952870 radv/sqtt: Avoid duplicate stage check
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25224>
2024-01-07 21:28:26 +01:00
Konstantin Seurer
77b9a6f9e2 radv/rt: Use radv_shader for compiled shaders
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25224>
2024-01-07 21:28:19 +01:00
Konstantin Seurer
59d490b8aa radv/rt: Remove useless assert
If it's NULL, the code will segfault anyways.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25224>
2024-01-07 21:28:11 +01:00
Konstantin Seurer
8198805e1f radv: Skip compiling chit and miss shaders
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25224>
2024-01-07 21:28:06 +01:00
Konstantin Seurer
0f87d406b5 radv/rt: Skip compiling a traversal shader
If we don't need one, we don't compile one.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25224>
2024-01-07 21:28:02 +01:00
Konstantin Seurer
aaa64217ca radv: Add more ray tracing data to the cache
This makes the cache more flexible when it comes to missing stages. This
will be used to skip compiling unused ray tracing stages.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25224>
2024-01-07 21:27:56 +01:00
Konstantin Seurer
a784477269 radv: Don't store library stack sizes
They are already imported in radv_rt_fill_stage_info.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25224>
2024-01-07 21:27:53 +01:00
Konstantin Seurer
92a951db6a radv: Make pipeline cache object data generic
Pipeline cache objects can hold some generic data. Anything concerning
that should not be handled in "common" code paths.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25224>
2024-01-07 21:27:27 +01:00
Marek Olšák
a84729d368 radeonsi/ci: add gfx11 flakes
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
205646cd77 winsys/amdgpu: simplify code using amdgpu_cs_context::chunk_ib
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
44df9517cd winsys/amdgpu: don't use amdgpu_fence::ctx for fence dependencies
The only remaining use of ctx is amdgpu_fence_is_syncobj.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
7ccdcae4b5 winsys/amdgpu: use pipe_reference for amdgpu_ctx refcounting
this is the standard utility for refcounting

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
33980355d4 winsys/amdgpu: implement explicit fence dependencies as sequence numbers
This eliminates redundant fence dependencies if BOs add the same ones.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
6d7a76595d winsys/amdgpu: remove dependency_flags parameter from cs_add_fence_dependency
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
6ac0b4ef05 winsys/amdgpu: rename amdgpu_bo_real::lock to map_lock
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
1e2c02d76b winsys/amdgpu: rename amdgpu_bo_sparse::lock -> commit_lock
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
e1261c77b5 winsys/amdgpu: rename amdgpu_winsys_bo::bo -> bo_handle
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
4d486888ee winsys/amdgpu: rewrite BO fence tracking by adding a new queue fence system
This decreases the time spent in amdgpu_cs_submit_ib from 15.4% to 8.3%
in VP2020/Catia1, which is a decrease of CPU load for that thread by 46%.
Overall, it increases performance by a small number in CPU-bound benchmarks.
The biggest improvement I have seen is VP2020/Catia2, where it increases
FPS by 12%.

It no longer stores pipe_fence_handle references inside amdgpu_winsys_bo.

The idea is to have a global fixed list of queues (only 1 queue per IP
for now) where each queue generates its own sequence numbers (generated
by the winsys, not the kernel). Each queue also has a ring of fences.
The sequence numbers are used as indices into the ring of fences, which
is how sequence numbers are converted to fences.

With that, each BO only has to keep a list of sequence numbers, 1 for each
queue. The maximum number of queues is set to 6. Since the system can
handle integer wraparounds of sequence numbers correctly, we only need
16-bit sequence numbers in BOs to have accurate busyness tracking. Thus,
each BO uses only 12 bytes to represent all its fences for all queues.
There is also a 1-byte bitmask saying which sequence numbers are
initialized.

amdgpu_winsys.h contains the complete description. It has several
limitations that exist to minimize the memory footprint and updating of
BO fences.

Acked-by: Yogesh Mohan Marimuthu <yogesh.mohanmarimuthu@amd.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
b976f8fc1e winsys/amdgpu: compute bo->unique_id at pb_slab_alloc, not at memory allocation
We would compute the unique IDs for 1000 slab entries and then only use
a few, wasting the IDs. Assign the IDs only when we actually need to
return a new buffer.

This decreases the number of collisions we get in amdgpu_lookup_buffer,
and thus the number of times we have to search in the BO list.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
32dae84d43 winsys/amdgpu: allocate 1 amdgpu_bo_slab_entry per cache line
The structure size is exactly 64 bytes, so every entry occupies exactly
1 cache line.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
6d913a2bcc r300,r600,radeonsi: switch to pb_buffer_lean
to remove pb_buffer::vtbl from all buffer structures

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
d2c76c4d77 winsys/radeon: stop using pb_buffer::vtbl
Only the destroy function used it.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
6c4ab02674 gallium/pb_cache: remove pb_cache_entry::buffer
The buffer pointer is always at a constant offset from pb_cache_entry,
so just pass the "offsetof" value to pb_cache and use that to get
the pointer.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
20bf2a06fb gallium/pb_cache: remove pb_cache_entry::mgr
We can just pass it via functions.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
d7de372358 gallium/pb_cache: switch to pb_buffer_lean
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
39c1311766 gallium/pb_buffer: define pb_buffer_lean without vtbl, inherit it by pb_buffer
amdgpu doesn't need vtbl.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
eb19f0daa3 winsys/amdgpu: don't use gpu_address to compute slab entry offset in bo_map
use the code we have in amdgpu_bo_get_va

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
a8e98882ea winsys/amdgpu: remove va (gpu_address) from amdgpu_bo_slab_entry
Keep it only in amdgpu_bo_real and amdgpu_bo_sparse. Slab entries can
compute it from the slab BO and adding their entry index.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
3cc2562ac0 winsys/amdgpu: remove now-redundant amdgpu_bo_slab_entry::real
The pb_slab pointer can be used to get the BO pointer because pb_slab is
inside the BO structure now.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
49bf2545fe winsys/amdgpu: add amdgpu_bo_real_reusable slab for the backing buffer
Add contents of amdgpu_bo_slab into it. This will allow removing the "real"
pointer from amdgpu_bo_slab_entry because "(char*)entry.slab" is now
pointing next to it.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
cf2dc2d512 winsys/amdgpu: don't layer slabs, use only 1 level of slabs, it improves perf
This increases FPS in VP2020/Catia1 by 10-18%!!!!!!!!!!!!!!!!!!!!!!!

I have no rational explanation for this.

In the most extreme case, 8192 256B slab BOs (smallest size) are now
allocated from a single 2MB slab.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
4a078e693e r300,r600,radeon/winsys: always pass the winsys to radeon_bo_reference
This will allow the removal of pb_cache_entry::mgr.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
643f390de5 radeon_winsys: add struct radeon_winsys* parameter into fence_reference
Since the radeon winsys implements fences as buffers, we need radeon_winsys*
to destroy them. This will enable the removal of pb_cache_entry::mgr.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:30 +00:00
Marek Olšák
e847c6e301 gallium/pb_cache: switch time variables to milliseconds and 32-bit type
to decrease pb_cache_entry by 8 bytes.

Add msecs_base_time to offset time == 0 to the creation of pb_cache.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
896c8b67cb gallium/pb_cache: remove pb_cache_entry::end to save space
just compute it at each use

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
523a4f71f2 winsys/amdgpu: stop using pb_buffer::vtbl
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
7752579202 winsys/amdgpu: rename to amdgpu_bo_slab to amdgpu_bo_slab_entry
It's a slab entry. "Slab" is the whole buffer, which is AMDGPU_BO_REAL
if we want to be precise.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
b3c64638b4 iris,zink,winsys/amdgpu: remove unused/redundant slab->entry_size
slab->base has the same field.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
9431c33899 gallium/pb_slab: move group_index and entry_size from pb_slab_entry to pb_slab
This removes 8 bytes from every slab entry, and thus amdgpu_bo_slab.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Marek Olšák
5a3bacc376 winsys/amdgpu: reduce wasted memory due to the size tolerance in pb_cache
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26643>
2024-01-06 20:55:29 +00:00
Daniel Schürmann
023e78b4d7 aco: add new post-RA scheduler for ILP
Totals from 77247 (97.37% of 79330) affected shaders: (GFX11)

Instrs: 44371374 -> 43215723 (-2.60%); split: -2.64%, +0.03%
CodeSize: 227819532 -> 223188224 (-2.03%); split: -2.06%, +0.03%
Latency: 301016823 -> 290147626 (-3.61%); split: -3.70%, +0.09%
InvThroughput: 48551749 -> 47646212 (-1.87%); split: -1.88%, +0.01%
VClause: 870581 -> 834655 (-4.13%); split: -4.13%, +0.00%
SClause: 1487061 -> 1340851 (-9.83%); split: -9.83%, +0.00%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25676>
2024-01-06 11:30:42 +00:00
Daniel Schürmann
72a5c659d4 aco: form clauses for LDS instructions
No fossil-db changes.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25676>
2024-01-06 11:30:42 +00:00
Daniel Schürmann
8f16745821 aco: fix should_form_clause() for memory instructions without operands
In particular, this applies to s_memtime and s_memrealtime.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25676>
2024-01-06 11:30:41 +00:00
Vinson Lee
568f61787a ac/rgp: Fix single-bit-bitfield-constant-conversion warning
../src/amd/common/ac_rgp.c:119:48: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
  119 |    header->flags.is_semaphore_queue_timing_etw = 1;
      |                                                ^ ~

Fixes: ed0c852243 ("radv: add initial SQTT files generation support")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26839>
2024-01-05 22:42:58 -08:00
Karol Herbst
1e5bc00715 rusticl/program: add LLVM functions to cache timestamp
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24386>
2024-01-06 03:09:48 +00:00
Karol Herbst
299f949775 rusticl/meson: generate bindings for LLVM
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24386>
2024-01-06 03:09:48 +00:00