Commit graph

43 commits

Author SHA1 Message Date
Daniel Schürmann
9b1a635bb3 amd/common: merge radv_nir_opt_access_speculate() into ac_nir_flag_smem_for_loads()
One shader is negatively affected, but we save 2 entire iterations over every shader.
This effect is also mitigated with the next commits.

Totals from 1 (0.00% of 79839) affected shaders: (Navi48)

Instrs: 947 -> 958 (+1.16%)
CodeSize: 4728 -> 4732 (+0.08%)
Latency: 20678 -> 20723 (+0.22%)
InvThroughput: 2697 -> 2698 (+0.04%)
SClause: 26 -> 27 (+3.85%)
Copies: 139 -> 145 (+4.32%)
Branches: 46 -> 47 (+2.17%)
VALU: 460 -> 463 (+0.65%)
SALU: 201 -> 204 (+1.49%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37843>
2025-10-14 16:33:12 +00:00
Samuel Pitoiset
08ddf2f878 radv: lower embedded/immutable samplers earlier
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Lowering them earlier right after VTN would allow us to implement
embedded samplers for descriptor heap properly for merged shaders.

Non-immediate samplers are still lowered in
radv_nir_apply_pipeline_layout because they require shader arguments.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37688>
2025-10-07 09:25:28 +00:00
Georg Lehmann
acd879f096 radv: set ACCESS_CAN_SPECULATE for smem buffer loads with known good descriptors
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Foz-DB GFX1201:
Totals from 2872 (3.59% of 80098) affected shaders:
MaxWaves: 78208 -> 78234 (+0.03%); split: +0.21%, -0.18%
Instrs: 6214171 -> 6193701 (-0.33%); split: -0.40%, +0.07%
CodeSize: 33121244 -> 33113692 (-0.02%); split: -0.18%, +0.16%
VGPRs: 151680 -> 152016 (+0.22%); split: -0.25%, +0.47%
SpillSGPRs: 775 -> 776 (+0.13%)
Latency: 46080905 -> 45955331 (-0.27%); split: -0.55%, +0.28%
InvThroughput: 6235954 -> 6250598 (+0.23%); split: -0.25%, +0.48%
VClause: 111125 -> 110955 (-0.15%); split: -0.17%, +0.02%
SClause: 221845 -> 214761 (-3.19%); split: -3.20%, +0.01%
Copies: 501387 -> 488215 (-2.63%); split: -2.96%, +0.33%
Branches: 191455 -> 178574 (-6.73%)
PreSGPRs: 146364 -> 146923 (+0.38%); split: -0.12%, +0.50%
PreVGPRs: 120813 -> 121073 (+0.22%)
VALU: 3139282 -> 3137471 (-0.06%); split: -0.11%, +0.05%
SALU: 1079863 -> 1083158 (+0.31%); split: -0.55%, +0.86%
VMEM: 182255 -> 182247 (-0.00%)
SMEM: 293409 -> 290233 (-1.08%)

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36938>
2025-08-27 09:45:19 +00:00
Samuel Pitoiset
ce83800262 radv: remove unused forwarded declarations of pipeline layout
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36792>
2025-08-18 07:25:34 +00:00
Marek Olšák
439d805291 nir: rename nir_lower_io_to_scalar_early -> nir_lower_io_vars_to_scalar
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35760>
2025-06-26 18:20:49 +00:00
Samuel Pitoiset
59dfa8c2f5 radv: switch to nir_intrinsic_load_input_attachment_coord
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35556>
2025-06-20 06:12:24 +00:00
Georg Lehmann
c3964e87f8 radv: apply fneg/fabs modifiers to wmma
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34396>
2025-04-22 16:08:55 +00:00
Konstantin Seurer
b218c45973 radv: Handle nir_intrinsic_printf
Makes it possible to use printf statements inside glsl meta shaders.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34208>
2025-04-10 19:31:37 +00:00
Timur Kristóf
bf1a968a11 radv: Add bool return value to radv_nir_apply_pipeline_layout.
And stop using it with the deprecated NIR_PASS_V macro.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33609>
2025-02-22 08:54:16 +01:00
Timur Kristóf
e3e2ba4eb5 radv: Add bool return value to radv_nir_lower_abi.
And stop using it with the deprecated NIR_PASS_V macro.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33609>
2025-02-22 08:54:16 +01:00
Konstantin Seurer
af375c6756 radv: Optimize fs builtins using static gfx state
The values of some builtins are known at compile time when the
application creates pipelines with static state.

Stats for graphics pipelines:

Totals from 568 (0.71% of 80536) affected shaders:
MaxWaves: 12364 -> 12502 (+1.12%); split: +1.26%, -0.15%
Instrs: 515696 -> 501182 (-2.81%); split: -2.85%, +0.04%
CodeSize: 2815736 -> 2741464 (-2.64%); split: -2.69%, +0.05%
VGPRs: 29528 -> 29160 (-1.25%); split: -1.71%, +0.46%
SpillSGPRs: 212 -> 215 (+1.42%)
Latency: 5515421 -> 5409125 (-1.93%); split: -2.05%, +0.13%
InvThroughput: 1293512 -> 1277913 (-1.21%); split: -1.27%, +0.06%
VClause: 10570 -> 10295 (-2.60%); split: -2.74%, +0.14%
SClause: 19040 -> 18531 (-2.67%); split: -2.83%, +0.16%
Copies: 37189 -> 35431 (-4.73%); split: -5.31%, +0.58%
Branches: 11391 -> 11070 (-2.82%); split: -2.92%, +0.11%
PreSGPRs: 27848 -> 27313 (-1.92%); split: -1.95%, +0.03%
PreVGPRs: 24847 -> 24106 (-2.98%); split: -3.00%, +0.02%
VALU: 359356 -> 348779 (-2.94%); split: -2.97%, +0.03%
SALU: 59135 -> 57448 (-2.85%); split: -3.11%, +0.26%
VMEM: 14674 -> 14313 (-2.46%)
SMEM: 30901 -> 30342 (-1.81%); split: -1.84%, +0.03%

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32793>
2025-02-17 09:45:15 +00:00
Timur Kristóf
666f10c174 radv: Add missing includes and remove unnecessary NIR includes.
RADV won't compile without the added includes after we
stop including the full nir.h from the VK common functions.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33439>
2025-02-12 22:33:07 +01:00
Samuel Pitoiset
b05a112d92 radv/nir: add cooperative matrix lowering for GFX12
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33378>
2025-02-07 12:06:10 +00:00
Georg Lehmann
ff225dee67 radv: inline radv_nir_lower_poly_line_smooth
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33340>
2025-02-05 11:23:35 +00:00
Timur Kristóf
930243bf36 radv: Reorder potentially per-primitive FS builtins.
There are some FS built-ins that can be per-vertex or
per-primitive depending on whether a mesh shader is used:
primitive ID (implicit in VS), layer and viewport.

However, the HW requires per-primitive FS inputs to be ordered last.
This causes bugs when the same unlinked FS is used together
with VS/TES/GS and MS (with unlinked ESO or fast-linked GPL).

To solve this problem, we reorder the FS inputs so that these
potentially per-primitive inputs go after per-vertex inputs but
before per-primitive inputs.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32220>
2024-11-28 18:14:56 +00:00
Rhys Perry
4383a917e6 radv: optimize VS input load components to constants earlier
This helps some linking optimizations.

fossil-db (navi21):
Totals from 2262 (2.85% of 79395) affected shaders:
MaxWaves: 57680 -> 57738 (+0.10%); split: +0.11%, -0.01%
Instrs: 1061526 -> 1053937 (-0.71%); split: -0.79%, +0.07%
CodeSize: 5766352 -> 5736784 (-0.51%); split: -0.60%, +0.08%
VGPRs: 89376 -> 89000 (-0.42%); split: -0.43%, +0.01%
Latency: 4102938 -> 4059773 (-1.05%); split: -1.14%, +0.08%
InvThroughput: 1105885 -> 1092291 (-1.23%); split: -1.24%, +0.01%
VClause: 18917 -> 18972 (+0.29%); split: -0.12%, +0.41%
SClause: 28839 -> 28115 (-2.51%); split: -3.32%, +0.81%
Copies: 73396 -> 72671 (-0.99%); split: -1.63%, +0.65%
PreSGPRs: 65866 -> 65838 (-0.04%); split: -0.22%, +0.17%
PreVGPRs: 69752 -> 69278 (-0.68%)
VALU: 680351 -> 673489 (-1.01%); split: -1.03%, +0.02%
SALU: 121459 -> 121515 (+0.05%); split: -0.00%, +0.05%
VMEM: 29632 -> 30021 (+1.31%); split: -0.02%, +1.33%
SMEM: 73744 -> 73836 (+0.12%); split: -0.01%, +0.14%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31593>
2024-10-21 15:52:53 +00:00
Samuel Pitoiset
656d7e887a radv: fix lowering the view index to an input varying for FS
When multiview is used and the FS is compiled separately with GPL, the
view index still needs to be lowered, otherwise it's crashing later.

The lowering doesn't need to know the previous stage because ViewIndex
is a global thing (ie. it's neither a per-vertex or a per-primitive
varying).

This fixes recent
dEQP-VK.pipeline.pipeline_library.graphics_library.misc.other.view_index_from_device_index_*_pre_rasterization

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31192>
2024-09-17 06:36:08 +00:00
Georg Lehmann
ca88783318 radv/nir: add a pass to optimize shuffle/booleans dependent only on tid/consts
This pass uses constant folding to determine which invocation is read by shuffle
for each invocation. Then, it detects patterns in the result and uses more
a specialized intrinsic if possible.

For booleans it creates inverse_ballot.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24650>
2024-07-17 15:04:38 +00:00
Samuel Pitoiset
c533a79878 radv: implement VK_KHR_dynamic_rendering_local_read
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27263>
2024-05-07 10:35:04 +00:00
Samuel Pitoiset
87fde606cf radv: use SPDX-License-Identifier
Only files under src/amd/vulkan/** are concerned.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28599>
2024-04-08 07:17:31 +00:00
Samuel Pitoiset
2686cd59df radv: rename radeon_info variables to gpu_info everywhere
Sometimes we might have other info struct, so renaming to gpu_info
removes the confusion.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28500>
2024-04-01 16:35:09 +00:00
Timur Kristóf
8f3cc3cb29 radv: Use mapped driver locations for determining I/O strides.
This will allow us to more accurately determine the
input and output strides, because the I/O locations mapped
by RADV don't match the locations in NIR.
As a result, ESO will use less LDS.

It also fixes the per-patch output stride of tess control
shaders, because previously we omitted tess factors from them.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28021>
2024-03-19 15:01:19 +00:00
Timur Kristóf
a6f270c160 radv: Lower mesh shader draw ID to zero when they have a task shader.
This prevents potential app bugs causing GPU hangs and other issues.
Note that the draw ID is undefined in mesh shaders when also
using a task shader.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27879>
2024-03-07 17:07:24 +00:00
Konstantin Seurer
c925b6019d radv/rt: Lower ray payloads like hit attribs
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27051>
2024-02-02 16:36:15 +00:00
Tatsuyuki Ishi
70b0c5909b radv: Add radv_spirv_to_nir_options that summarize early gfx states.
radv_shader_spirv_to_nir now takes a more minimal struct instead of gfx
states. This struct will be used for NIR cache hashing in the future.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26696>
2024-01-26 11:46:18 +00:00
Samuel Pitoiset
b555f9451c radv: introduce radv_graphics_state_key
This struct only contains graphics related state.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27007>
2024-01-15 11:49:04 +00:00
Samuel Pitoiset
138ce72860 radv/nir: remove useless struct for nir_shader typedef
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27007>
2024-01-15 11:49:04 +00:00
Samuel Pitoiset
18349692d0 radv/nir: pass radv_shader_stage to some radv_nir_xxx() functions
Instead of passing separate parameters for args, layout, info etc.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27007>
2024-01-15 11:49:04 +00:00
Konstantin Seurer
3a69424e09 radv/nir: Add radv_nir_lower_hit_attrib_derefs
Move out the pass so it can be unit tested.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24271>
2023-11-02 15:48:36 +00:00
Bas Nieuwenhuizen
9df4703fbb radv: Add cooperative matrix lowering.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24683>
2023-10-24 13:24:18 +00:00
Samuel Pitoiset
66eaca3a0a radv: introduce radv_shader_layout for per-stage descriptor layout
With pipelines, the shader layout is inherited from the pipeline layout
but with shader objects, the layout is passed through
VkCreateShaderInfoEXT.

This basically replaces uses of radv_pipeline_layout by
radv_shader_layout during shaders compilation. This will avoid
creating a pipeline layout with ESO.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24562>
2023-08-10 08:51:05 +00:00
Samuel Pitoiset
08bfcc12d4 radv: rename radv_pipeline_stage to radv_shader_stage
It's more generic and it will fit shader object just well.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>
2023-07-26 07:44:49 +00:00
Eric Engestrom
8b319c6db8 radv: reformat according to its .clang-format
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23253>
2023-06-16 19:59:52 +00:00
Samuel Pitoiset
7ee8a12a62 radv: add a NIR pass that lower fragment shader barycentric intrinsics
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16742>
2023-06-07 14:40:35 +00:00
Samuel Pitoiset
85cbdba355 radv: add support for smooth lines
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21587>
2023-05-22 07:58:35 +00:00
Timur Kristóf
c0692dbfab radv: Move I/O lowering functions into a new file.
Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>
2023-03-30 05:59:13 +00:00
Timur Kristóf
90bf9ed759 radv: Move radv_nir_export_multiview to new file.
Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>
2023-03-30 05:59:13 +00:00
Timur Kristóf
685dcecb57 radv: Move radv_nir_lower_viewport_to_zero to new file.
Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>
2023-03-30 05:59:13 +00:00
Timur Kristóf
67de4b54fe radv: Move radv_nir_lower_view_index to new file.
Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>
2023-03-30 05:59:13 +00:00
Timur Kristóf
1e2a5858f4 radv: Move radv_nir_lower_intrinsics_early to new file.
Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>
2023-03-30 05:59:13 +00:00
Timur Kristóf
87e7dfb5f8 radv: Move radv_nir_lower_fs_intrinsics to new file.
Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>
2023-03-30 05:59:12 +00:00
Timur Kristóf
838defc5eb radv: Move radv_nir_lower_primitive_shading_rate to new file.
Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>
2023-03-30 05:59:12 +00:00
Timur Kristóf
1978eaf5b2 radv: Move radv_nir_* to a new folder.
Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>
2023-03-30 05:59:12 +00:00