Commit graph

204426 commits

Author SHA1 Message Date
Konstantin Seurer
76031ba53d radv: Optimize the gfx12 encode shader
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
97f6287827 radv: Use the BVH8 format on GFX12
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
95e7343a7d radv/bvh: Add helpers for encoding
The build and update paths can use the same code.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
3af19f336c radv/bvh: Document GFX12 BVH encoding
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
2942e3affb radv/rra: Set rra_accel_struct_header::rtip_level
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
fa99eeb2b4 radv/rra: Move gfx10_3 specific code to a new file
gfx12 needs completely different code and having them in different files
is cleaner.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
9d157173b2 radv: Refactor create_bvh_descriptor
Make it a bit more extendable since GFX12 introduced more fields.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
978e9b670e aco,nir: Add support for new GFX12 ray tracing instructions
Adds image_bvh_dual_intersect_ray and image_bvh8_intersect_ray which can
handle the new BVH format. Both instructions write up to 10 VGPRs so
they need to use a vec16 definition in nir.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Natalie Vock
ee0f784858 aco/ra: Don't consider precolored ops/defs in get_reg_impl
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Natalie Vock
b9e506afd4 aco: Add support for multiple definitions in emit_mimg
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Natalie Vock
f309d76aab aco: Add support for multiple ops fixed to defs
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
fe739a2da2 ac: Add rt_version
rt_version describes which generation of RT capabilities a chip has.
This matches what PAL does.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
c33e598f39 vulkan: Add vk_ir_header::dst_leaf_node_offset
Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
2dee1117b7 vulkan: Add a vk_device parameter to get_encode_key
Useful for selecting different encoding options based on hardware
generation.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Konstantin Seurer
0cc9443e9b util: Add BITSET_EXTRACT
Extracts a <=32 bit range from a bitset.

Reviewed-by: Natalie Vock <natalie.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34273>
2025-04-17 20:20:40 +00:00
Eric Engestrom
c37a468a8a pick-ui: make Backport-to: 25.0 backport to 25.0 *and more recent release branches*
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
It is what developers expect, so make the code match it.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34580>
2025-04-17 20:13:41 +00:00
Jesse Natalie
37e6a8b57f d3d12: Minor fixes to residency algorithm when eviction is needed
Always reset batch_count and batch_memory_size. Proceed to eviction
only if we stopped before filling up the batch.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34562>
2025-04-17 19:51:15 +00:00
Jesse Natalie
565980f3c0 d3d12: Add tc memory throttles
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34562>
2025-04-17 19:51:15 +00:00
Caio Oliveira
43e521f7a5 hk: Don't expect garbage on shared_size
Talking to Alyssa this looks like an artifact of old ways that the
shader was being produced, so get rid of that zeroing.  Add an
assert as a canary for any problems we might be missing.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34139>
2025-04-17 19:13:18 +00:00
Caio Oliveira
33295b2249 spirv, nir: Allow non-Aliased workgroup memory blocks
Allocate space for the aliased region first, then allocate the
non-Aliased blocks in sequence after that.

SPV_KHR_workgroup_memory_explicit_layout extension added support for
having Blocks of workgroup (shared) memory, which include layout
decoration.  For that extension all such blocks must be decorated with
Aliased.

SPV_KHR_untyped_pointers extension lifts that requirement, allowing
blocks that don't alias in workgroup memory.  They are still explicitly
laid out.

The motivation is that untyped pointers provide a different
mechanism to obtain the same effect as the Aliased blocks.  Instead of
having two Aliased variables with different types, have a single
variable and use an untyped pointer with a different type to access it.

This patch is a preparation for supporting untyped pointers.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34139>
2025-04-17 19:13:18 +00:00
Caio Oliveira
fd0a7efb5a spirv, nir: Delay calculation of shared_size when using explicit layout
Move the calculation to nir_lower_vars_to_explicit_types().  This
consolidates the check of shader_info::shared_memory_explicit_layout
in a single place instead of in all drivers.

This is motivated by SPV_KHR_untyped_pointers.  Before that extension
we had essentially two modes for shared memory variables

- No layout decorations in the SPIR-V, and both internal layout and
  driver location was _given by the driver_.

- Explicitly laid out, i.e. they are blocks, and decorated with Aliased.
  Because they all alias, we could assign them driver location directly
  to the start of the shared memory.

With the untyped pointers extension, there's a third option, to be added
by a later commit

- Explicitly laid out, i.e. they are blocks, and NOT decorated
  with Aliased.  Driver location is _given by the driver_.  Blocks
  with and without Aliased can be mixed.

The driver location of multiple blocks that don't alias depend on
alignment that is driver-specific, which we can more easily do from
the nir_lower_vars_to_explicit_types() that already has access to
a function to obtain such value.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> (hk)
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com> (v3dv)
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (anv/hasvk)
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (panvk)
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> (radv)
Reviewed-by: Rob Clark <robdclark@gmail.com> (tu)
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34139>
2025-04-17 19:13:17 +00:00
Eric Engestrom
adfe29cb46 ci: give high priority to post-merge jobs as well
Right now, `deploy-docs` (updating the docs.mesa3d.org website) is the
only job that can exist in that pipeline (along with `alpine/x86_64_build`,
but that job is always a no-op).

Right now this job can get stuck for a long time waiting for a runner,
but it's very short, and we kinda want the website to be updated quickly.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34554>
2025-04-17 18:39:34 +00:00
José Roberto de Souza
a96e280dfe intel: Program XY_FAST_COLOR_BLT::Destination Mocs for gfx12
Copy engine is not used in gfx12 platforms on ANV but that is possible
in Iris.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34560>
2025-04-17 18:11:44 +00:00
Alyssa Rosenzweig
8b068ef6c1 hk: handle HIC with twiddled
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:06 +00:00
Alyssa Rosenzweig
88bdc27342 asahi: let booleans be your guide
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:06 +00:00
Alyssa Rosenzweig
551355d4e5 asahi,hk: factor out zls_control pack helper
makes both drivers a lot more readable, but especially gl

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:06 +00:00
Alyssa Rosenzweig
3a560dd32b asahi: identify ZLS compress load/store bits
obvious in retrospect!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:06 +00:00
Alyssa Rosenzweig
9757185153 hk: plumb ZLS tiling
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:06 +00:00
Alyssa Rosenzweig
454a90eaa8 asahi: plumb ZLS tiling bits
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:06 +00:00
Alyssa Rosenzweig
715f6b3b33 asahi: identify ZLS tiling bits
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:06 +00:00
Alyssa Rosenzweig
0dca602aff asahi: generalize compression check
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:06 +00:00
Alyssa Rosenzweig
72f3dcc8da asahi: generalize tiling checks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:06 +00:00
Alyssa Rosenzweig
c4130af883 asahi: extend tile width/height in texture desc
we need to support up to 16384x16384 for atomics on twiddled images.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:06 +00:00
Alyssa Rosenzweig
f21dc4d0cf asahi: pack sample count in s/w texture descriptor
not needed for non-msaa case, and this lets us free up 2 bits.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:06 +00:00
Alyssa Rosenzweig
8f57b5187f ail: support twiddled
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:05 +00:00
Alyssa Rosenzweig
6fdad684da ail: generalize ail_space_bits
for full twiddling.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:05 +00:00
Alyssa Rosenzweig
e5006dc6ae hk: fill sparse.write with nonzero values
fuzz for bugs.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:05 +00:00
Alyssa Rosenzweig
ffac153bcf hk: reindent/unscope
no functional change, split because the diff is all spacing changes.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:05 +00:00
Alyssa Rosenzweig
86d3489c35 hk: drop FS null checks
these are all dead now.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:05 +00:00
Alyssa Rosenzweig
3ab8ce8579 hk: fix null FS corner cases
this fixes null FS + cull distance/API sample mask, which require a prolog.
fixes upcoming CTS.

Backport-to: 25.1
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:05 +00:00
Alyssa Rosenzweig
d959557669 hk: fix tessellation + clipper queries
fixes upcoming cts

Backport-to: 25.1
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34585>
2025-04-17 17:54:05 +00:00
Rhys Perry
427479c040 aco: remove va_vdst/vm_vsrc/sa_sdst variables
Use the "wait" variable instead.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34529>
2025-04-17 17:28:22 +00:00
Rhys Perry
3d6fa6996c aco: init vm_vsrc/sa_sdst from depctr_wait
fossil-db (navi31):
Totals from 5805 (7.31% of 79377) affected shaders:
Instrs: 14229621 -> 14207115 (-0.16%); split: -0.16%, +0.00%
CodeSize: 75358724 -> 75268624 (-0.12%); split: -0.12%, +0.00%
Latency: 133637034 -> 133624262 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 22067819 -> 22066213 (-0.01%); split: -0.01%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34529>
2025-04-17 17:28:22 +00:00
Rhys Perry
ce2be5ab8e aco: combine VALU lanemask hazard into VALUMaskWriteHazard
This is now basically the same as the original VALUMaskWriteHazard, except
it now considers both VALU and SALU writes.

Now that it's a part of VALUMaskWriteHazard, differences from the original
VALU lanemask workaround are:
- it includes SALU reads after the write
- it includes VALU writes and SALU/VALU reads after the write which are
  not lanemasks
- it combines s_waitcnt_depctr instructions when it's a read after both a
  SALU write and a VALU write
- non-exec VALU SGPR reads reset the SGPRs read by VALU as a lanemask
- exec SGPRs are ignored

resolve_all_gfx11() is also finished.

fossil-db (navi31):
Totals from 21538 (27.13% of 79377) affected shaders:
Instrs: 27628855 -> 27552972 (-0.27%); split: -0.30%, +0.03%
CodeSize: 145968448 -> 145667616 (-0.21%); split: -0.23%, +0.02%
Latency: 209537805 -> 209509519 (-0.01%); split: -0.02%, +0.00%
InvThroughput: 36304270 -> 36301624 (-0.01%); split: -0.01%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12623
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/11480
Backport-to: 25.0
Backport-to: 25.1
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34529>
2025-04-17 17:28:22 +00:00
Mel Henning
eee3c8eab8 nak: Handle idp4 ureg latencies
Fixes: 6b8a4e6bb7 ("nak: Add Turing latency information")
Fixes: 7a01953a39 ("nak: Add Ampere and Ada latency information")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12993
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34563>
2025-04-17 17:10:40 +00:00
Mel Henning
de1ed48325 nak/spill_values: Spill constants across edges if needed
In a previous iteration of the spilling code, we added an extra check to
only spill across edges if the value being spilled is in the W set.
This was due to a misunderstanding of the modeling of S and W in Braun
and Hack.  In the current implementation, we maintain the invariant that
every live value is in at least one of S or W so we don't need that
check but it was left in by mistake.

One exception to this rule was added when we special-cased constant
values.  Now the invariant is that every live value is in S, in W, or is
a constant.  When we made this change, the check we accidentally left in
bit us because now if a value is constant but not in W, it wasn't
getting spilled across the edge.  This can result in a value getting
filled later which was never spilled, leading to undefined values.

Fixes: 7b82e26e3c ("nak: Don't spill/fill const values")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12993
Co-authored-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34563>
2025-04-17 17:10:40 +00:00
Eric Engestrom
8744c98fa9 meson: remove duplicate deprecated for power8 option
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Fixes: c4b305079d ("meson: Simplify the power8 optimization logic")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34565>
2025-04-17 14:43:30 +00:00
Eric Engestrom
b9472db496 meson: remove duplicate deprecated for gallium-xa option
Fixes: cf40099730 ("meson: deprecate gallium-xa")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34565>
2025-04-17 14:43:30 +00:00
Rohan Garg
cbc1ec4f73 anv: re enable compression for CPS surfaces on platforms other than Xe
I accidentally disabled compression on CPS surfaces marked as storage or
color attachment for all platforms, when this should only be limited to
Xe.

Fixes: 80f9b6 ('anv: CPB surfaces that are used as color attachments or for stores cannot be compressed')
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34297>
2025-04-17 14:24:11 +00:00
Rhys Perry
4fcf2eb1d7 aco/gfx12: VOPD src0/1 are src bank compatible if they are the same vgpr
fossil-db (gfx1201):
Totals from 66518 (83.80% of 79377) affected shaders:
Instrs: 36939667 -> 36656685 (-0.77%); split: -0.79%, +0.02%
CodeSize: 220575208 -> 220201764 (-0.17%); split: -0.21%, +0.04%
Latency: 258919732 -> 258137974 (-0.30%); split: -0.35%, +0.05%
InvThroughput: 49911351 -> 49643836 (-0.54%); split: -0.55%, +0.02%
VClause: 788661 -> 788430 (-0.03%); split: -0.04%, +0.01%
SClause: 1176416 -> 1176263 (-0.01%); split: -0.02%, +0.01%
VALU: 18014058 -> 17818119 (-1.09%); split: -1.10%, +0.01%
VOPD: 4926983 -> 5122922 (+3.98%); split: +4.01%, -0.04%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34246>
2025-04-17 14:00:29 +00:00