Commit graph

207813 commits

Author SHA1 Message Date
Mel Henning
694523e2b9 nak: Implement nir_intrinsic_vote_ieq with OpMatch
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35778>
2025-06-28 16:10:50 +00:00
Mel Henning
10acb44c64 nir: Split lower_vote_eq into int/float versions
Recent nvidia hardware has a native instruction for
nir_intrinsic_vote_ieq but not for nir_intrinsic_vote_feq. So, split
this boolean into two so we can contol the lowering separately for each
instruction.

Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35778>
2025-06-28 16:10:50 +00:00
Mel Henning
00fe8e45a0 nak: Add OpMatch
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35778>
2025-06-28 16:10:49 +00:00
Valentine Burley
ef6273c43c ci: Fix artifact name for jobs with parallel indices
GitLab mangled artifact filenames when `CI_JOB_NAME` contained a slash
(e.g. 1/4), resulting in broken names like `4.zip`.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13424
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35796>
2025-06-28 14:59:58 +00:00
Natalie Vock
e236a731e4 radv/rt: Enable pointer flags on GFX11+
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Allows hardware to do some of the culling work, as well as early-cull
box nodes with CullOpaque/CullNonOpaque ray masks when all children are
(not) opaque.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32417>
2025-06-28 10:31:38 +00:00
Natalie Vock
e82717a5cf radv: Use common helper to set BLAS node pointer flags on gfx11+
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32417>
2025-06-28 10:31:38 +00:00
Natalie Vock
06a06bbe09 radv: Encode child opaqueness information in box nodes
Also, use one reserved field from the header to store the root node's
opaqueness flags. This is used to propagate opaqueness info across the
BLAS/TLAS boundary.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32417>
2025-06-28 10:31:37 +00:00
Natalie Vock
3b1f94d00d radv: Encode child opaqueness information in triangle nodes
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32417>
2025-06-28 10:31:37 +00:00
Natalie Vock
7301e92d49 vulkan/runtime/bvh: Propagate opaqueness information through the BVH
AMD hardware can early-cull box nodes if all leaves are either opaque or
not and the ray flags are set to discard (non-)opaque geometries. This
works even across TLAS/BLAS boundaries.

Propagate info on whether all child nodes are opaque or not through the
BVH to allow RADV to set these flags per box node.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32417>
2025-06-28 10:31:37 +00:00
Marek Olšák
6afa638b18 ac/nir/lower_ngg: rename user_clip_plane_enable_mask -> cull_clipdist_mask
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:26 +00:00
Marek Olšák
814990684d ac/nir/lower_ngg: pack GS outputs and XFB outputs in LDS optimally
This switches the code to the new slot offsets from ac_nir_prerast_out
instead of using a prefix bitmask over outputs_written.

The LDS layout no longer includes these:
- GS: output components that are not written by GS
- VS/TES+XFB: output components that are not written by XFB
- VS/TES+XFB: slots that are not written by XFB (this could be significant)

This is also a cleanup because it unduplicates the bitcounts.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:26 +00:00
Marek Olšák
75b1602c14 ac/nir/lower_ngg_gs: return LDS size from the pass
instead of computing it separately. This is better because
ac_nir_lower_ngg_gs knows the final LDS size anyway, and it will be
easier to modify the size calculation this way.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:26 +00:00
Marek Olšák
d79f28e9b3 ac/nir/lower_ngg: return LDS size for NGG VS and TES from the pass
instead of computing it separately. This is better because
ac_nir_lower_ngg_nogs knows the final LDS size anyway, and it will be
easier to modify the size calculation this way.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:26 +00:00
Marek Olšák
7fe603ad82 radv: call gfx10_get_ngg_info after NIR lowering
It will receive LDS vertex and scratch sizes from NIR lowering passes,
so it can't be before them.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:26 +00:00
Marek Olšák
42e98f115a radv: always use the ngg_lds_layout SGPR
This is a prerequisite for NGG lowering passes to return LDS vertex and
scratch sizes, which will lead to further simplifications. That will
require calling gfx10_get_ngg_info after radv_postprocess_nir, which means
LDS offsets are unknown when the passes are called.

This makes the 2 values no longer compile-time constants.

A later commit will remove NGG_LDS_LAYOUT_SCRATCH_BASE (the passes will
determine it), so only NGG_LDS_LAYOUT_GS_OUT_VERTEX_BASE will come from
an SGPR, though that could be removed too (non-trivially) or handled as
a relocation.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:26 +00:00
Marek Olšák
8346469ec0 ac/nir/lower_ngg_gs: split lower_ngg_gs_intrinsic into gathering and lowering
We need to gather outputs before lowering because lowering requires that we
know the LDS vertex stride, so that we can lower output stores to LDS
stores.

The pass will determine the LDS vertex stride, not drivers.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:26 +00:00
Marek Olšák
84e8e899cd ac/nir: add an option not to gather values in ac_nir_gather_prerast_store_output_info
This will be needed in the next commit.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:26 +00:00
Marek Olšák
ebdd97a993 ac/nir: add LDS layout info for GSVS and XFB to ac_nir_prerast_per_output_info
This will be used to reduce the NGG LDS size for uncompacted GS and XFB
outputs.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:25 +00:00
Marek Olšák
39a9dce5fc ac/nir: add an option to pack clip/cull distance components to remove holes
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:25 +00:00
Marek Olšák
6cd813810e ac/nir: add an option write_pos_to_clip_vertex to clip against POS
This enables emulating clip planes without ClipVertex via clip distances
(max 8) instead of the fixed-func hw (max 6 planes).

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:25 +00:00
Marek Olšák
3dd3f2f889 ac/nir/lower_ngg_gs: build streamout after lowering intrinsics
Streamout will require prerast info, which is gathered by
lower_ngg_gs_intrinsics.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:25 +00:00
Marek Olšák
83dc5917fe ac/nir: lower ClipVertex before all position exports
just code reordering (position exports should be at the end for perf)

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:25 +00:00
Marek Olšák
c9b6a95038 ac/nir: remove the done parameter from ac_nir_export_position
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:25 +00:00
Marek Olšák
7c3760201d ac/nir/lower_ngg: never export edge flags via position exports
It has no effect, but the extra export instructions is unnecessary and
we can't gather the effective number of position exports from NIR if we
insert incorrect exports.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35351>
2025-06-28 08:20:25 +00:00
Lionel Landwerlin
a742b859bd anv: add support for handling wa_18019110168 with gfx-libs
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:35 +00:00
Lionel Landwerlin
fcf4401824 brw: handle wa_18019110168 with independent shader compilation
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:35 +00:00
Lionel Landwerlin
bc8d18aee2 brw: make a helper for vertex attribute offset computation
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:34 +00:00
Lionel Landwerlin
8fabcd754f brw: move primitive_id_index field in fs_msaa
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:34 +00:00
Lionel Landwerlin
6336cf0ea2 brw: store the remapping table for wa_18019110168 in constant data
That way it can be accessed at runtime.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:33 +00:00
Lionel Landwerlin
e1a7eb1718 brw: extract out attribute register remapping
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:33 +00:00
Lionel Landwerlin
5cc66e2c8d anv/brw: move Wa_18019110168 handling to backend
We simplify the implementation by assuming the worse case, copying
entire per-vertex regions if necessary.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:32 +00:00
Lionel Landwerlin
8e7e0ef75a anv: make Wa_18019110168 deal with dynamic provoking vertex
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:32 +00:00
Lionel Landwerlin
f0f4f9c566 brw: fix vertex attribute offset computation
The formula uses scalar indices (4bytes), not slots (16bytes).

We also incorrectly passed a scalar (vertex case) & slot (mesh case)
offset in the push constants. Use slots instead so that the value is
smaller and we can pack more stuff into fs_msaa_flags.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 18bbcf9a63 ("intel: introduce new VUE layout for separate compiled shader with mesh")
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:31 +00:00
Lionel Landwerlin
4b5539a0cb brw: fix set_range on load_per_primitive_output
load intrinsics don't have range

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 18bbcf9a63 ("intel: introduce new VUE layout for separate compiled shader with mesh")
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:31 +00:00
Rob Clark
7fd99c88b9 freedreno/a6xx: The great register renaming
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Align register names to internal docs to avoid having to mentally remap
register names between the names we invented over the years and what
they are actually called.

Signed-off-by: Rob Clark <rob.clark@oss.qualcomm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35803>
2025-06-27 23:08:32 +00:00
Rob Clark
a8c052ca9d freedreno: Rename IBO -> UAV
Internally, adreno uses dx terminology, and calls these UAVs.  Rename to
match.

Signed-off-by: Rob Clark <rob.clark@oss.qualcomm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35803>
2025-06-27 23:08:31 +00:00
Rob Clark
12530fb8df freedreno/registers: Some reg64 conversion
Signed-off-by: Rob Clark <rob.clark@oss.qualcomm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35803>
2025-06-27 23:08:30 +00:00
Rob Clark
f55d8fa62e freedreno/registers: Fix a couple missing variants
Signed-off-by: Rob Clark <rob.clark@oss.qualcomm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35803>
2025-06-27 23:08:29 +00:00
Rob Clark
79d2014d8b freedreno/a6xx: Fix SP_CS_IBO address on a7xx
This moved to accomidate the additional BINDLESS_BASE registers, and we
overlooked that when adding a7xx support.

Signed-off-by: Rob Clark <rob.clark@oss.qualcomm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35803>
2025-06-27 23:08:28 +00:00
Rob Clark
029270f9c1 freedreno/decode: Add a simple util to dump csv reg list
Signed-off-by: Rob Clark <rob.clark@oss.qualcomm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35803>
2025-06-27 23:08:28 +00:00
Faith Ekstrand
f6bdb8cd06 nvk/copy: Set the gob_kind for tiled copies on Blackwell+
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
This is needed on blackwell to deal with kind/bpp interactions.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35807>
2025-06-27 21:52:08 +00:00
Dave Airlie
a428835681 nouveau/headers: add a hacked up clc9b5 dma-copy
This adds some hacked up dma-copy bits needed for blackwell
until nvidia can release the class.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35807>
2025-06-27 21:52:08 +00:00
Faith Ekstrand
6594a11dfe nil: Remove some unused renames from cbindgen.toml
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35807>
2025-06-27 21:52:07 +00:00
Faith Ekstrand
50a0f9bd88 nil: Add a new GOBType for Z24 on Blackwell+
It's a different memory layout from Ada and earlier.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35807>
2025-06-27 21:52:07 +00:00
Faith Ekstrand
f6ff11a594 nil: Reorder the checks in GOBType::choose() again
Match on HW generation first.  Blackwell is so totally different that it
doesn't make sense to share any code with the others.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35807>
2025-06-27 21:52:07 +00:00
Faith Ekstrand
6fbd6a52ca nak: Use fswzadd.ndv on Blackwell+
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35795>
2025-06-27 21:23:25 +00:00
Faith Ekstrand
b4c05a8402 nak: Add a TexDerivMode to OpFSwz and OpFSwzAdd
Even on Blackwell, this is a single bit but we might as well use the
enum for it so it's consistent with tex ops.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35795>
2025-06-27 21:23:25 +00:00
Faith Ekstrand
4ac4bd62c8 nak: Add a new TexDerivMode enum and plumb it through
For most generations, this is just plumbing through a false bit.  But on
Blackwell, we need to set .dxy at least in compute shaders.

Also, we had a bunch of .NDV comments on OpTxd but it has never existed
there on ahy hardware generation, it's just a left-over from trying to
copy+paste from codegen.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35795>
2025-06-27 21:23:24 +00:00
Faith Ekstrand
c6ad70551b nak: Tell NIR to lower invalid implicit LODs
I think NVIDIA hardware more or less does this for us for free but it's
nice to have NIR make sure.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35795>
2025-06-27 21:23:24 +00:00
Faith Ekstrand
13935c955c nak: Add better is_blackwell*() helpers
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35795>
2025-06-27 21:23:24 +00:00