Commit graph

206033 commits

Author SHA1 Message Date
Marek Olšák
0dc5d649ea winsys/amdgpu: fall back to a normal priority without root in the winsys
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34983>
2025-05-28 10:23:15 +00:00
Marek Olšák
2ef6aa5934 winsys/amdgpu: pass PIPE_CONTEXT_* flags to ctx_create
instead of using our own flags; also REALTIME_PRIORITY is never used,
so the relevant code is removed

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34983>
2025-05-28 10:23:15 +00:00
Marek Olšák
7f441beaf6 winsys/amdgpu: set the priority for gfx user queues
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34983>
2025-05-28 10:23:15 +00:00
Marek Olšák
6785e42511 winsys/amdgpu: add a high priority gfx queue
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34983>
2025-05-28 10:23:15 +00:00
Marek Olšák
59e93b02e0 winsys/amdgpu: add enums for queues using the fence rings
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34983>
2025-05-28 10:23:15 +00:00
Marek Olšák
4bf2a28334 winsys/amdgpu: fix running out of 32bit address space with high FPS
Reproduced with gfxbench5 gl_tess_off.

Fixes: 4d486888ee - winsys/amdgpu: rewrite BO fence tracking by adding a new queue fence system

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34983>
2025-05-28 10:23:14 +00:00
Marek Olšák
d9e681ee3f winsys/amdgpu: use alt_fence for all video queues
It's already used by VCN queues.

This reduces the size of sequence numbers stored per BO.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34983>
2025-05-28 10:23:14 +00:00
Marek Olšák
f75e54453f winsys/amdgpu: print an error when we fail to allocate VA
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34983>
2025-05-28 10:23:14 +00:00
Samuel Pitoiset
8596150ae8 aco: implement bitfield_reverse for types other than 32-bits
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34583>
2025-05-28 09:52:12 +00:00
Daniel Schürmann
5b4d284493 aco/isel: use vector-aligned operands for image_bvh64_intersect_ray
Totals from 93 (0.12% of 79377) affected shaders: (Navi48)
MaxWaves: 1376 -> 1368 (-0.58%)
Instrs: 3583500 -> 3581861 (-0.05%); split: -0.05%, +0.00%
CodeSize: 18792300 -> 18785296 (-0.04%); split: -0.04%, +0.00%
VGPRs: 8652 -> 8592 (-0.69%); split: -1.25%, +0.55%
Latency: 20861347 -> 20834407 (-0.13%); split: -0.17%, +0.04%
InvThroughput: 4032604 -> 4028020 (-0.11%); split: -0.14%, +0.03%
VClause: 90507 -> 90525 (+0.02%); split: -0.01%, +0.03%
Copies: 279429 -> 277839 (-0.57%); split: -0.58%, +0.01%
Branches: 100260 -> 100251 (-0.01%)
PreVGPRs: 8949 -> 8771 (-1.99%)
VALU: 1955635 -> 1954053 (-0.08%); split: -0.08%, +0.00%
SALU: 477347 -> 477329 (-0.00%); split: -0.01%, +0.01%
VOPD: 69 -> 61 (-11.59%)

Totals from 93 (0.12% of 79377) affected shaders: (Navi31)

MaxWaves: 1376 -> 1374 (-0.15%)
Instrs: 3442606 -> 3440344 (-0.07%); split: -0.07%, +0.00%
CodeSize: 17801008 -> 17790476 (-0.06%); split: -0.07%, +0.01%
VGPRs: 8652 -> 8556 (-1.11%); split: -1.25%, +0.14%
Latency: 20590943 -> 20542279 (-0.24%); split: -0.27%, +0.03%
InvThroughput: 3978133 -> 3969497 (-0.22%); split: -0.25%, +0.03%
VClause: 91784 -> 91769 (-0.02%); split: -0.05%, +0.03%
Copies: 277177 -> 275263 (-0.69%); split: -0.70%, +0.01%
Branches: 100098 -> 100092 (-0.01%); split: -0.02%, +0.01%
PreVGPRs: 9021 -> 8843 (-1.97%)
VALU: 2001794 -> 1999893 (-0.09%); split: -0.10%, +0.00%
SALU: 419504 -> 419559 (+0.01%); split: -0.01%, +0.02%
VOPD: 77 -> 64 (-16.88%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:17 +00:00
Rhys Perry
c50f9541e4 aco/tests: Add tests for vector-aligned operands
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:17 +00:00
Daniel Schürmann
b5382faa9c aco/validate: validate register assignment of vector-aligned operands
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:17 +00:00
Daniel Schürmann
9091c3bf5b aco/ra: add affinities for MIMG vector-aligned operands
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:17 +00:00
Daniel Schürmann
fb689f133e aco/ra: handle register assignment of vector-aligned operands
Vector-aligned operands are handled by temporarily allocating
a vector-SSA value for the duration of the instruction.
On completion of the register assignment, the individual
operands are assigned to the reserved register space and,
if necessary, parallelcopies are emitted.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:17 +00:00
Daniel Schürmann
92b1154397 aco/ra: Always rename copy-kill operands, even if the temporary doesn't match
This makes it independent of whether the operand already got renamed or not.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:17 +00:00
Daniel Schürmann
4fad3514a9 aco/ra: only change registers of already handled operands in update_renames()
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:17 +00:00
Daniel Schürmann
51a2e1eb94 aco/ra: don't use kill-flags as indicator in get_reg_create_vector()
We are about to re-use this function for vector-aligned operands.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:17 +00:00
Daniel Schürmann
3d8b355f22 aco/assembler: support vector-aligned operands on MIMG instructions
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:17 +00:00
Daniel Schürmann
8cb1700c74 aco/print_ir: print parenthesis around vector-aligned operands
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:16 +00:00
Daniel Schürmann
6aabcb02a1 aco/print_ir: only print 'lateKill' if requested via print_kill flag
Also only print lateKill for actually killed operands.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:16 +00:00
Daniel Schürmann
a9645fdd89 aco: introduce concept of vector-aligned Operands
Operand::isVectorAligned indicates that the Operand is part of a vector
consisting of multiple operands. Therefore, it must reside in a register
aligned with the next Operand.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:16 +00:00
Daniel Schürmann
a4fa3935fd aco/live_var_analysis: set same lateKill flags for same operands
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:16 +00:00
Daniel Schürmann
ee0ee282b9 aco: simplify Operand() constructor
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34359>
2025-05-28 09:24:16 +00:00
Mary Guillemard
19531a0162 pan/csf: Fix typo in cs_trace_run_idvs2
This should have been a multiply not an add.
Fix an assertion when running in tracing mode on panvk.

Fixes: 79a1d98e1e ("pan/csf: make cs_builder.h usable from c++")
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35176>
2025-05-28 10:12:19 +02:00
Samuel Pitoiset
2ebfa64be7 radv: add radv_disable_hiz_his_gfx12 and enable for Mafia Definitive Edition
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
This is a workaround for random GPU hangs with HiZ/HiS on GFX12
because the correct fix is complex and it will take time to be
implemented properly.

Mafia Definitive Edition is the first known game affected by this.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13222
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35182>
2025-05-28 07:20:26 +00:00
Marek Olšák
35c76bc7f7 nir/tcs_info: use range analysis to determine the range of tess levels
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35195>
2025-05-28 06:46:56 +00:00
Marek Olšák
24c3f30e4a nir/tcs_info: gather which patch outputs are only read/written by invoc 0
Tested thoroughly by a shader test.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35195>
2025-05-28 06:46:56 +00:00
Marek Olšák
a3632d7d88 nir/tcs_info: gather for all patch outputs whether they're written by all invocs
This substantially rewrites the pass. It also makes it easier to read.
Tested thoroughly by a shader test.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35195>
2025-05-28 06:46:56 +00:00
Faith Ekstrand
6b265d9d7d nvk: Use NAK by default on Kepler
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
At this point it's way more correct than codegen.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Faith Ekstrand
a37c50e985 nak/sm20: Encode OpSuLdGa and OpSuStGa
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Lorenzo Rossi
2a5ac439f5 nak/sm20: Encode imadsp
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Faith Ekstrand
2c0957d086 nak/sm20: Encode surface address ops
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Lorenzo Rossi
4d86f95cff nak/sm32: Encode surface address ops
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Lorenzo Rossi
2c0d0bad01 nak: Remove unused intrinsic image_load_raw_nv
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Lorenzo Rossi
ac3fd5768b nak: Add surface address ops
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Faith Ekstrand
82d789d22a nvk: Lower images to addresses on Kepler
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Lorenzo Rossi
bb3f4b86bf nvk: Use nil_su_info for image descriptors on Kepler
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Lorenzo Rossi
e5d4c4934d nak: Add an image address lowering pass
On Kepler, we don't have real surface opcodes. Instead, we have a bunch
of helper ops which make image calculations in the shader reasonably
efficient and we have to lower to that and an address-based load/store.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Faith Ekstrand
0b06feee92 nouveau: Move bitview to a common rust util folder
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Lorenzo Rossi
5fbcdd6e32 nir,nak: Add NV-specific image intrinsics
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Lorenzo Rossi
a27a711c3a nil: Add helpers for filling out Fermi/Kepler surface info
Kepler needs different descriptors for Image storage, these descriptors
are not directly used by the hardware but by lowering instructions
(added in later commits)

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:19 +00:00
Faith Ekstrand
e388f25a2a nil: Align linear image strides to the image alignment
This is a function no-op right.  In the case where we don't have an
explicit_row_stride_B, we use an alignment of 128B which is what we were
aligning the row stride to before.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:18 +00:00
Faith Ekstrand
7a577d2e3a nvk: Drop the union from nvk_buffer_view
The space saved is tiny and not having the union at all makes me way
more comfortable about the modifications we're about to do.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34975>
2025-05-28 01:47:18 +00:00
Adam Jackson
4b1c824b67 vtn/opencl: Handle OpenCLstd_F{Min,Max}_common
Normal fmin doesn't make any promises about NaN, common additionally
doesn't make any promises about infinities. Would be nice to hook that
up to codegen but lowering them to normal works for now.

Cc: mesa-stable
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34941>
2025-05-28 01:08:20 +00:00
Adam Jackson
92f07860a4 vtn: (Silently) handle FunctionParameterAttributeNo{Capture,Write}
Silences a few thousand warnings in sycl/test-e2e

Cc: mesa-stable
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34941>
2025-05-28 01:08:20 +00:00
Adam Jackson
fd7589d775 vtn: Handle SPV_KHR_uniform_group_instructions
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34941>
2025-05-28 01:08:20 +00:00
Caleb Callaway
52db0e1480 intel/compiler: fix SHA generation for shader replace
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35140>
2025-05-27 22:57:19 +00:00
Faith Ekstrand
8ffe0098be nvk: Reserve a sampler for TXF on Kepler
The SPIR-V spec says texelFetch and friends don't take a sampler.
However, on Kepler and earlier hardware, the sampler is read even for
tld.  In particular, the hardware reads the sRGB conversion bit in the
sampler and this can be in an inconsistent state if we haven't
initialized samplers properly.  On Kepler, we should just reserve a
sampler at device creation time and always use that for tld.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35190>
2025-05-27 22:23:37 +00:00
Faith Ekstrand
bfdc95b109 nak: Scalarize non-constant ald/ast on Kepler
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35190>
2025-05-27 22:23:36 +00:00
Faith Ekstrand
f10c42fcc1 bitview: Implement SetFieldU64 for all BitViewMutable
There's no reason why we need to keep overriding this.  There's also
probably not a good reason why this trait exists at all but that's a
problem for another day.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35190>
2025-05-27 22:23:36 +00:00