Commit graph

2937 commits

Author SHA1 Message Date
Marek Olšák
ae22da2ff8 ac/nir: lower more loads in ac_nir_lower_intrinsics_to_args instead of drivers
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
ceb6f8fc32 amd: lower load_tess_rel_patch_id/primitive_id/tess_coord and overwrite.. in NIR
The overwrite instruction complicates it a little, which is why these
intrinsics are lowered together.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
61bfb4fa06 amd: lower load_subgroup_invocation in NIR
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
e69f47faee amd: lower load_local_invocation_index in NIR
This is the last intrinsic that needed the LS VGPR bug workaround in ACO
and ac_nir_to_llvm.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
433ca6ba38 ac/nir: extract a load_subgroup_id lowered helper
this will be used in the next commit

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
342dcbdc8b amd: lower load_vertex_id/instance_id and overwrite_vs_arguments in NIR
2 things complicate this:
- overwrite_vs_arguments_amd
- the LS VGPR bug workaround

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
66dd70adc5 amd: lower load_gs_wave_id_amd in NIR
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
923f59c971 amd: lower load_barycentric_at_offset in NIR
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
16ab05fad1 amd: lower load_barycentric_pixel/centroid/sample in NIR
radeonsi needs to preserve interp_mode in the arg load.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
a15e733a81 ac,radeonsi: move load_vector_arg flags to common code
This will be needed by lowering of barycentrics.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
7e83f6ca8b amd: lower load_front_face in NIR
radeonsi must do this after si_lower_nir_abi, which optimizes front_face,
but doesn't lower it.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
6ad5225b2a amd: lower load_frag_shading_rate in NIR
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
6d2e29ff6e amd: lower load_sample_pos in NIR
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
110e474b4f amd: lower load_sample_id in NIR
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
684c8da553 amd: lower load_invocation_id in NIR
ACO can't look for it because it's lowered there.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
d281240c57 amd: lower load_first_vertex/base_instance/draw_id/view_index in NIR
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
0d372b043b amd: lower load_local_invocation_id in NIR
This is based on ACO.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
13cb5c7b72 amd: lower load_frag_coord in NIR
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
58cb155068 amd: lower load_pixel_coord in NIR
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Marek Olšák
85c3b5159a ac/nir: handle disabled PS VGPRs in ac_nir_load_arg_at_offset
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782>
2025-01-02 17:36:55 +00:00
Timur Kristóf
652a0b48bc amd: Set lower_layer_fs_input_to_sysval in common code, not in drivers.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32641>
2025-01-02 14:07:51 +00:00
Timur Kristóf
ed88616a12 ac/nir/ngg: Don't mark multiview layer output as varying.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32641>
2025-01-02 14:07:51 +00:00
Marek Olšák
c21bc65ba7 nir/opt_load_store_vectorize: make hole_size signed to indicate overlapping loads
A negative hole size means the loads overlap. This will be used by drivers
to handle overlapping loads in the callback easily.

Reviewed-by: Mel Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32699>
2025-01-01 00:03:55 +00:00
Georg Lehmann
e112e2b047 nir,amd: optimize front_face ? a : -a
Foz-DB Navi31:
Totals from 3345 (4.21% of 79395) affected shaders:
MaxWaves: 96182 -> 96174 (-0.01%)
Instrs: 3135439 -> 3129508 (-0.19%); split: -0.24%, +0.05%
CodeSize: 16776088 -> 16718048 (-0.35%); split: -0.38%, +0.03%
VGPRs: 190884 -> 190848 (-0.02%); split: -0.03%, +0.01%
Latency: 32624132 -> 32621734 (-0.01%); split: -0.16%, +0.16%
InvThroughput: 5759987 -> 5749957 (-0.17%); split: -0.23%, +0.05%
VClause: 51044 -> 51086 (+0.08%); split: -0.12%, +0.20%
SClause: 103415 -> 103223 (-0.19%); split: -0.64%, +0.45%
Copies: 170398 -> 170555 (+0.09%); split: -0.64%, +0.74%
PreSGPRs: 135567 -> 133887 (-1.24%)
PreVGPRs: 140569 -> 141317 (+0.53%)
VALU: 1959144 -> 1953839 (-0.27%); split: -0.30%, +0.03%
SALU: 217956 -> 217676 (-0.13%); split: -0.20%, +0.07%

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32791>
2024-12-30 22:31:35 +00:00
Timur Kristóf
de2cb4a7d3 ac/nir: Only store params to attribute ring that are varying.
On GFX11+, varying outputs from the last pre-rasterization stage
are implemented by storing the outputs to the so-called
attribute ring.

Make sure to only store them when necessary.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:41 -06:00
Timur Kristóf
13234a8a8a ac/nir: Only export parameters when they are actually varying.
In AMD terminology, varying outputs are implemented by
parameter export instructions on GFX6-10.3 GPUs.

Only emit those when actually necessary.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:38 -06:00
Timur Kristóf
4d6c00944b ac/nir: Only export positions when they are really system values.
In AMD terminology, a system value is implemented by
position export instructions.

Make sure to only emit those when they are needed.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:36 -06:00
Timur Kristóf
f5981e8c0b ac/nir: Split GS output usage masks to varying and sysval masks.
To keep track which output is used for what purpose.

Note that this commit just adds the capability to track this
separately in ac/nir. The drivers will need to be updated
in the future to take advantage of this.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:33 -06:00
Timur Kristóf
92464109e3 ac/nir: Mark when pre-rast output is used as varying or sysval.
In this commit, just collect the info.
It will be taken into use by subsequent commits.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:29 -06:00
Timur Kristóf
cb0671aede ac/nir/ngg: Refactor storing per-primitive primitive ID to attribute ring.
Simplify the code using the helpers introduced in previous commits.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:26 -06:00
Timur Kristóf
edde762b56 ac/nir/ngg: Move emitting GS vertex param exports to if.
On GFX10-10.3 (when no attribute ring is present), only emit
the GS vertex parameter exports on the vertex export threads.
Other threads don't have anything to export.

Move this code around to make it a bit easier to follow.
Also add some comments to better explain what's what.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:23 -06:00
Timur Kristóf
68dbcdd935 ac/nir/ngg: Move wait attr ring workaround for GS to better place.
The call depends on the phis created by create_output_phis so
the code becomes more readable if we move it closer to that.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:20 -06:00
Timur Kristóf
9acc2f2435 ac/nir/ngg: Remove dead code for attribute ring stores.
These are replaced by the new helpers added in previous commits.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:17 -06:00
Timur Kristóf
f528de896e ac/nir/ngg: Refactor export_pos0_wait_attr_ring.
There is no need to create phis in this function anymore,
because they can be already created by create_output_phis before.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:14 -06:00
Timur Kristóf
badbb01c5d ac/nir/ngg: Refactor GS attribute ring stores.
Use the new helper.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:11 -06:00
Timur Kristóf
23c615bde2 ac/nir/ngg: Refactor VS/TES attribute ring stores.
Use the new helper.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:09 -06:00
Timur Kristóf
f38680aa1c ac/nir: Introduce ac_nir_store_parameters_to_attr_ring.
This function is going to be used for storing parameter outputs
to the attribute ring, instead of the current implementation.

It is going to be taken into use in the following commits.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:06 -06:00
Timur Kristóf
c4b45f1ec8 ac/nir: Pass ac_nir_prerast_out to ac_nir_export_position.
In a subsequent	commit,	ac_nir_export_position will
start using other fields from ac_nir_prerast_out.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:04 -06:00
Timur Kristóf
3d291a98c4 ac/nir: Pass ac_nir_prerast_out to ac_nir_export_parameters.
In a subsequent commit, ac_nir_export_parameters will
start using other fields from ac_nir_prerast_out.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:31:01 -06:00
Timur Kristóf
896237b52e ac/nir/ngg: Simplify updating mesh shader output info.
All 64-bit outputs are already lowered to 32-bit.
There is no need to handle them here.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:30:58 -06:00
Timur Kristóf
f460e3a36b ac/nir/ngg: Use ac_nir_prerast_out in mesh shader lowering.
This will help us share more code between the mesh shader lowering
and other passes.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640>
2024-12-28 10:30:53 -06:00
David Rosca
e3d602de98 ac/parse_ib: Parse VCN IB_COMMON_OP_WRITEMEMORY
And more small fixes.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32760>
2024-12-27 08:17:16 +00:00
Qiang Yu
b0c47871ec ac: remove ac_nir_lower_subdword_loads
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32781>
2024-12-27 01:58:38 +00:00
Qiang Yu
21f888a3ed ac,radv: move ac_nir_lower_bit_size_callback to common place
To be used by radeonsi for OpenCL.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32781>
2024-12-27 01:58:38 +00:00
Qiang Yu
5f601361ed ac/nir: lower access for shared and scratch memory
OpenCL may load and store vec16 data, while ACO only
support <=32byte. Radeonsi is going to use
ac_nir_lower_mem_access_bit_sizes() for lowering these
access.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32781>
2024-12-27 01:58:38 +00:00
Marek Olšák
c0e5e8f932 amd: update addrlib
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32687>
2024-12-26 21:02:21 +00:00
Marek Olšák
c6fd69bd5e ac: remove unused code
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32780>
2024-12-26 10:12:43 +00:00
Marek Olšák
de996ac481 radeonsi: kill Z and stencil PS outputs if depth or stencil is disabled
This adds kill_z and kill_stencil flags to the shader PS epilog key, which
removes those outputs if depth or stencil are disabled.

It must be implemented in:
* ACO PS epilog
* LLVM PS epilog
* ac_nir_lower_ps for monolithic shaders

Some of the samplemask code wasn't completely correct, but probably harmless.

Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32713>
2024-12-24 12:02:20 +00:00
Daniel Schürmann
28a214728c ac/lower_ngg: move readlane into break blocks in streamout code generation for gfx12/ACO
This avoids unnecessary shuffle code and s_wait_loadcnt.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32743>
2024-12-21 12:32:25 +00:00
Daniel Schürmann
47227089d6 ac/lower_ngg: move break blocks after loop in streamout code generation for gfx12/ACO
By inverting the break condition, the loop becomes shorter.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32743>
2024-12-21 12:32:25 +00:00