Commit graph

3080 commits

Author SHA1 Message Date
Faith Ekstrand
caeee413a3 nil: Don't use Fermi bits in the Maxwell null descriptor
Fixes: c8d8e2249a ("nil: Add a helper for populating the null descriptor")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35475>
(cherry picked from commit bbc6f50963)
2025-06-18 17:55:46 +02:00
Mel Henning
eacca4b1ec nak: Don't swap f2fp sources in legalize
The order of these is important.

Fixes: e19871bd6a ("nak: Use F2FP for nir_op_pack_half_2x16_split on SM86+")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12717
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35267>
(cherry picked from commit aae67ab678)
2025-06-04 15:52:48 +02:00
Mel Henning
a77ee5440a nak: Forbid reordering labeled OpNop
Totals:
Static cycle count: 1104322907 -> 1108862573 (+0.41%)

Totals from 111376 (56.68% of 196502) affected shaders:
Static cycle count: 948085895 -> 952625561 (+0.48%)

Fixes: 79d0f8263d ("nak: Add a simple postpass instruction scheduler")
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35141>
(cherry picked from commit 018f4f1c27)
2025-06-04 15:52:48 +02:00
Mel Henning
031c20e9d3 nvk: Call ensure_slm for nvk_cmd_dispatch_shader
Internal shaders can also use slm, so we need to allocate it correctly.

This fixes
dEQP-VK.dgc.ext.compute.misc.max_pc_range_256_full_preprocess_with_execution_set
with NAK_DEBUG=spill

Fixes: 105bdf2e36 ("nvk: Add a helper for dispatching compute shaders")
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35143>
(cherry picked from commit 0e5880ebe4)
2025-06-04 15:52:48 +02:00
Faith Ekstrand
569459a804 nvk: Only allow importing mappable dma-bufs to HOST_VISIBLE types
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35213>
(cherry picked from commit 601cf33c44)
2025-06-04 15:52:48 +02:00
Faith Ekstrand
7e0c8b8efd nouveau/mme: Don't install the HW tests
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35163>
(cherry picked from commit 26ba29f75b)
2025-06-04 15:52:46 +02:00
Mel Henning
4483824a0a nak/spill_values: Follow phis from src to dest
ssa_state_out has the predecessor's SSAValue, so we need look for it in
the phi_src map.

Totals:
CodeSize: 4545122720 -> 4534830176 (-0.23%); split: -0.23%, +0.00%
Number of GPRs: 10963889 -> 10963693 (-0.00%); split: -0.00%, +0.00%
SLM Size: 1855380 -> 1649308 (-11.11%); split: -11.11%, +0.01%
Static cycle count: 1104322907 -> 1093035821 (-1.02%); split: -1.02%, +0.00%
Spills to memory: 480689 -> 139107 (-71.06%)
Fills from memory: 480689 -> 139107 (-71.06%)
Spills to reg: 458804 -> 242139 (-47.22%); split: -47.23%, +0.01%
Fills from reg: 303068 -> 222030 (-26.74%); split: -26.75%, +0.01%
Max warps/SM: 7245516 -> 7245580 (+0.00%)

Totals from 9899 (5.04% of 196502) affected shaders:
CodeSize: 1056727952 -> 1046435408 (-0.97%); split: -0.98%, +0.00%
Number of GPRs: 1666652 -> 1666456 (-0.01%); split: -0.01%, +0.00%
SLM Size: 1107988 -> 901916 (-18.60%); split: -18.61%, +0.01%
Static cycle count: 254942337 -> 243655251 (-4.43%); split: -4.43%, +0.01%
Spills to memory: 480689 -> 139107 (-71.06%)
Fills from memory: 480689 -> 139107 (-71.06%)
Spills to reg: 367784 -> 151119 (-58.91%); split: -58.92%, +0.01%
Fills from reg: 222209 -> 141171 (-36.47%); split: -36.49%, +0.02%
Max warps/SM: 119188 -> 119252 (+0.05%)

Fixes: bcad2add47 ("nak: Add a spilling pass")
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
(cherry picked from commit 6c68c2c3ba)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35244>
2025-06-04 15:52:46 +02:00
Faith Ekstrand
84e66ae44a nvk: Allocate the correct VAB size on Kepler
We were allocating 128 KiB but claimed 256 KiB.  Allocate the right size
and assert that the size matches.

Fixes: 970bd70584 ("nvk: allocate VAB memory area")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35172>
(cherry picked from commit 9fe2a21e93)
2025-06-04 15:52:46 +02:00
Mel Henning
3dedf9bbc1 nak: Fix a perf regression in tex lowering
These lines look like they were mistakenly introduced, and cause a
significant perf hit. Eg. this fix improves the Horizon Zero Dawn
in-game benchamark by ~42% on my ampere machine (5992 pts -> 8517 pts).

Fixes: d16e75e55f ("nak: Lower texture inputs for Kepler B")
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35100>
(cherry picked from commit 9d620fabd2)
2025-06-04 15:52:45 +02:00
Mel Henning
7fc8c5e10f nouveau/headers: Ignore PermissionError in rustfmt
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13172
Fixes: 591b5da4 ("nouveau/headers: Run rustfmt on generated files")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35008>
(cherry picked from commit bfe8340296)
2025-05-20 20:18:07 +02:00
Mel Henning
5d6a3c5862 nouveau/headers: Run rustfmt after file is closed
If we run a subprocess while the file is still open, we may not have
flushed the file contents to disk.

Fixes: 591b5da4 ("nouveau/headers: Run rustfmt on generated files")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35008>
(cherry picked from commit da22094593)
2025-05-20 20:18:07 +02:00
Dave Airlie
05f249ca93 nvk: Fix compute class comparison in dispatch indirect
This works by coincidence rather than design.

Reviewed-by: Mel Henning <mhenning@darkrefraction.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34910>
(cherry picked from commit bd7777aee6)
2025-05-20 20:18:07 +02:00
Faith Ekstrand
349c1e2cf4 nak: Set lower_pack_64_4x16
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Otherwise, these can cause infinite loops in optimization because there
aren't _split variants and the optimizer tries to combine and split
things infinitely.

Reviewed-by: Mel Henning <drawoc@darkrefraction.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34849>
(cherry picked from commit efd1cddbe9)
2025-05-07 09:04:51 +02:00
Mel Henning
4c1447e89b nak: Check that swizzles are none
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
wherever we check that src_mod is none.

This commit simply does:
s/src_mod.is_none()/is_unmodified()/
across all of nak except the definition of is_unmodified() itself.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Fixes: bad23ddb48 ("nak: Add F16 and F16v2 sources")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34794>
(cherry picked from commit 9d1c38ddf1)
2025-05-06 23:40:53 +02:00
Mel Henning
81b877c9a0 nak: Add Src::is_unmodified() helper
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Fixes: bad23ddb48 ("nak: Add F16 and F16v2 sources")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34794>
(cherry picked from commit 6e72f0f81b)
2025-05-06 17:45:13 +02:00
Karol Herbst
84f1dcdc2a nak: fix handling of delays > 15
Fixes: 2b569ecdb6 ("nak: Handle delays > 15")
Acked-by: Mel Henning <mhenning@darkrefraction.com>
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34722>
(cherry picked from commit 9a97a5d577)
2025-05-01 09:39:41 +02:00
Mel Henning
d0627d243d nak: Remove hfma2 src 1 modifiers
This fixes a compilation issue in Marvel Rivals where the legalization
logic and the encoding logic don't line up, which results in an
assertion failure on this instruction:

    r17 = hfma2 r17.xx -r18.xx 0x3c003c00

The fix here is a little overly restrictive because it turns out we
actually do have modifiers for all 3 sources. Those modifiers will
be added in later commits.

Fixes: 567cae69c3 ("nak: Add 16-bits float operations")
Reviewed-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34750>
(cherry picked from commit 1ff7135691)
2025-04-30 14:15:58 +02:00
Faith Ekstrand
6f9e010794 nvk: Maxwell+ is now conformant
This advertises Vulkan 1.4 conformance for Maxwell+ GPUs as well as
enabling Vulkan on them by default.

Backport-to: 25.1
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34582>
(cherry picked from commit d383d78e8c)
2025-04-23 12:21:56 +02:00
Mel Henning
9811e2b7d1 nvk: Override render enable for blits and resolves
Fixes cts tests:

dEQP-VK.conditional_rendering.conditional_ignore.blit_image
dEQP-VK.conditional_rendering.conditional_ignore.blit_image_inverted
dEQP-VK.conditional_rendering.conditional_ignore.resolve_image
dEQP-VK.conditional_rendering.conditional_ignore.resolve_image_inverted

which were introduced in vk-gl-cts commit 4aa277c300

Fixes: 32f2317223 ("nvk: Use meta for doing blits with the 3D hardware")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34644>
(cherry picked from commit 2fc4c98aaf)
2025-04-22 18:44:52 +02:00
Mel Henning
e52fac2e6d nvk: SET_STATISTICS_COUNTER at start of meta_begin
Ideally, begin/end should be roughly symmetric - the initialization
order should be the reverse of the teardown order.

Fixes: 6f85e6b06b ("nvk: Disable statistics around meta ops")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34644>
(cherry picked from commit 52085f2a0e)
2025-04-22 18:44:51 +02:00
Mel Henning
614c26c634 nak: Handle idp4 ureg latencies
Fixes: 6b8a4e6bb7 ("nak: Add Turing latency information")
Fixes: 7a01953a39 ("nak: Add Ampere and Ada latency information")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12993
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34563>
(cherry picked from commit eee3c8eab8)
2025-04-22 01:24:38 +02:00
Mel Henning
53c8864428 nak/spill_values: Spill constants across edges if needed
In a previous iteration of the spilling code, we added an extra check to
only spill across edges if the value being spilled is in the W set.
This was due to a misunderstanding of the modeling of S and W in Braun
and Hack.  In the current implementation, we maintain the invariant that
every live value is in at least one of S or W so we don't need that
check but it was left in by mistake.

One exception to this rule was added when we special-cased constant
values.  Now the invariant is that every live value is in S, in W, or is
a constant.  When we made this change, the check we accidentally left in
bit us because now if a value is constant but not in W, it wasn't
getting spilled across the edge.  This can result in a value getting
filled later which was never spilled, leading to undefined values.

Fixes: 7b82e26e3c ("nak: Don't spill/fill const values")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12993
Co-authored-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34563>
(cherry picked from commit de1ed48325)
2025-04-22 01:24:36 +02:00
Faith Ekstrand
e944636ff7 nak/sm70: Fix the bit74_75_ar_mod assert
It's used for src2, not src0.

Fixes: 40422927dc ("nak: Pass has_mod to all form of src2 requiring it")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33107>
(cherry picked from commit 47fc468944)
2025-04-22 00:04:15 +02:00
Faith Ekstrand
0702e54b55 nak/legalize: Take a RegFile in copy_alu_src_and_lower_ineg()
Fixes: af6093a712 ("nak/legalize: Add a helper for lowering ineg")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33107>
(cherry picked from commit 328112c6bc)
2025-04-22 00:04:14 +02:00
Faith Ekstrand
9fa9cd870f nak/legalize: Take a RegFile in copy_alu_src_and_lower_fmod
Otherwise, we'll screw up uniform GPRs.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33107>
(cherry picked from commit 22a30bfa4f)
2025-04-22 00:04:12 +02:00
Faith Ekstrand
44b01b55d5 nvk: Handle shifted QMD cbuf addrs in indirect command processing
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34443>
2025-04-15 23:38:54 +00:00
Faith Ekstrand
8b2f0be254 nak/qmd: Add QMD version 4.0 for Hopper
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34443>
2025-04-15 23:38:54 +00:00
Faith Ekstrand
a62c59d7e0 nak/qmd: Add support for shifted cbuf addresses
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34443>
2025-04-15 23:38:54 +00:00
Faith Ekstrand
7050a285a3 nak/qmd: Rework cbuf size suffix handling
Addresses will also be shifted starting with QMD version 4.0 and we'll
need something a little more general for that.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34443>
2025-04-15 23:38:54 +00:00
Faith Ekstrand
c8a5086213 nvk: Disable VK_EXT_descriptor_buffer pre-Maxwell
Our strategy of covering the entire address space in buffer views
requires that we be able to create very large buffer views.  The
pre-Maxwell texture unit doesn't allow for this.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34542>
2025-04-15 18:03:53 -05:00
Faith Ekstrand
c135bd6542 nak: Get rid of RegRef::zero
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34538>
2025-04-15 22:17:38 +00:00
Faith Ekstrand
7125b25800 nak/sm70: Add zero_reg() and true_reg() helpers
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34538>
2025-04-15 22:17:38 +00:00
Faith Ekstrand
c4452c5fa4 nak/sm50: Add zero_reg() and true_reg() helpers
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34538>
2025-04-15 22:17:38 +00:00
Faith Ekstrand
32570924cf nak: Move some legalization helpers from sm50 to common code
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34538>
2025-04-15 22:17:38 +00:00
Dave Airlie
6a4c1ac464 nak: add F2FP to sm75 instr latencies
hw_tests found this was missing, copy it from Ampere.

Fixes: 6b8a4e6bb7 ("nak: Add Turing latency information")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34537>
2025-04-15 21:56:55 +00:00
Faith Ekstrand
58321cf2e5 nak: Add stubs for Fermi and Kepler A
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34536>
2025-04-15 16:04:10 -05:00
Faith Ekstrand
c0cd01241a nak: Add stubs for Kepler B
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34536>
2025-04-15 16:04:10 -05:00
Faith Ekstrand
af6093a712 nak/legalize: Add a helper for lowering ineg
This is similar to the helper we have for lowering float modifiers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34536>
2025-04-15 16:04:09 -05:00
Faith Ekstrand
d16e75e55f nak: Lower texture inputs for Kepler B
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34535>
2025-04-15 20:02:52 +00:00
Faith Ekstrand
9c5a0eca95 nak: Add False and True to IntCmpOp
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34535>
2025-04-15 20:02:52 +00:00
Faith Ekstrand
eb4fb70bae nak: Add a plop2 test
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34535>
2025-04-15 20:02:52 +00:00
Faith Ekstrand
e84f210362 nak/hw_tests: Feed predicate/carry sources with 0/1 data
Instead of doing the &1 on the GPU, do it on the CPU.  This saves an
instruction and also makes things easier to debug.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34535>
2025-04-15 20:02:51 +00:00
Faith Ekstrand
fadac25b0c nil: Multiply by array_stride_B instead of adding
Fixes: 5577128c83 ("nil: Rewrite the TIC code in Rust")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34495>
2025-04-12 17:04:40 +00:00
Faith Ekstrand
5c81b3546f nvk/nvkmd: Check the correct flag for the Kepler GART workaround
Fixes: 1db57bb414 ("nvk/nvkmd: Rework memory placement flags")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34495>
2025-04-12 17:04:40 +00:00
Konstantin Seurer
cb31b5a958 clc,libcl: Clean up CL includes
This patch does a couple of things to make CL integration with drivers
as seamless as possible:
- We pull in opencl-c.h and opencl-c-base.h to stop relying on system
  headers.
- Parts of libcl.h are moved to new headers that are incomplete CL-safe
  variants of libc headers.
- A couple of util headers are changed to remove now unnecessary
  __OPENCL_VERSION__ guards and make more headers CL safe.
- Drivers now include src/compiler/libcl and use headers like
  macros.h,u_math.h instead of libcl.h.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33576>
2025-04-11 21:27:37 +00:00
Dave Airlie
0df8c4f20c nvk: add hopper support to nv_push_dump
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34442>
2025-04-10 00:37:45 +00:00
Dave Airlie
725e84974e nvk: add ADA compute class to nv_push_dump
now we have the classes use them in the push dump.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34442>
2025-04-10 00:37:45 +00:00
Faith Ekstrand
a441bd3c7e nvk: Call vk_device_finish() last in nvk_DestroyDevice()
We initialize it first so we should destroy it last.  Otherwise, things
we call in our own destroy code might reference a dead vk_device and
blow up.

Fixes: d2ea532528 ("vulkan: Use syncobj shim")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34444>
2025-04-10 00:18:47 +00:00
Dave Airlie
b5d1b0d7e3 nvk: update nvidia class header files.
This adds Ada/Hopper compute headers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34440>
2025-04-09 22:03:18 +00:00
Faith Ekstrand
18b2bef45a nak: Add a NAK_DEBUG=panic option
This tells it to actually panic instead of unwinding and returning NULL.
I find myself commenting out the unwind code pretty frequently so I can
get GDB to break at the panic.  This should help avoid that extra debug
step.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34435>
2025-04-09 16:14:00 +00:00