Commit graph

15890 commits

Author SHA1 Message Date
Georg Lehmann
ee0e183700 aco/gfx11.5: select SOPC float instructions
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245>
2024-07-18 08:36:15 +00:00
Georg Lehmann
4bd229ac50 aco/gfx11.5: select SOP2 float instructions
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245>
2024-07-18 08:36:14 +00:00
Georg Lehmann
6affd916b5 aco/gfx11.5: fix s_fmac acc to definition
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245>
2024-07-18 08:36:14 +00:00
Georg Lehmann
a90d4d340c aco/gfx11.5: select SALU float conversions
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245>
2024-07-18 08:36:14 +00:00
Georg Lehmann
4399c7bac3 aco: add aco_opcode::p_s_cvt_f16_f32_rtne
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245>
2024-07-18 08:36:14 +00:00
Georg Lehmann
1efb7754fc aco/gfx11.5: select s_(ceil|floor|trunc|rndne)
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245>
2024-07-18 08:36:14 +00:00
Georg Lehmann
33a719b3e2 aco/gfx11.5: select s_cvt_[ui]32_f32
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245>
2024-07-18 08:36:14 +00:00
Georg Lehmann
343420fd4e aco/gfx12: don't allow vgpr operands for pseudo scalar
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245>
2024-07-18 08:36:14 +00:00
Georg Lehmann
d58d0274a8 aco/gfx12: use trans s_delay_alu for pseudo scalar
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29245>
2024-07-18 08:36:14 +00:00
Samuel Pitoiset
5072848a2f radv/meta: remove useless memset when destroying DCC retile state
There should be only one finish call.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:33 +00:00
Samuel Pitoiset
602bdda3a5 radv/meta: rework creating DCC retile pipelines
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:33 +00:00
Samuel Pitoiset
176befe439 radv/meta: fix potential memleak when creating DCC retile pipelines
If the driver needs to create two different DCC retile pipelines which
is based on the image swizzle, it will just overwrite the existing
layouts.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:33 +00:00
Samuel Pitoiset
8e53114de3 radv/meta: fix potential race condition when creating DCC retile pipelines
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:33 +00:00
Samuel Pitoiset
0906b64724 radv/meta: rework creating copy expand pipelines
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:33 +00:00
Samuel Pitoiset
67d720d115 radv/meta: create fmask copy layouts regardless on-demand
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:33 +00:00
Samuel Pitoiset
ee1bc2e821 radv/meta: rework creating FMASK expand pipelines
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:33 +00:00
Samuel Pitoiset
f1ec223cd2 radv/meta: create fmask expand layouts regardless on-demand
To be consistent with other meta operations.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:33 +00:00
Samuel Pitoiset
4bef832b10 radv/meta: rework creating blit pipelines
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:33 +00:00
Samuel Pitoiset
ad82a338b3 radv/meta: cleanup meta_emit_blit()
Passing the image and the image view is useless.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:33 +00:00
Samuel Pitoiset
89dc316720 radv/meta: move the locking around creating blit pipelines
Only the on-demand path needs to be locked.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:33 +00:00
Samuel Pitoiset
e8683cae01 radv/meta: cleanup radv_device_init_meta_blit_{color,depth,stencil]()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
20be729636 radv/meta: create the fill/copy pipelines on-demand
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
c57987afc7 radv/meta: separate creating the fill/copy pipelines
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
5f2cbc3ab9 radv/meta: cleanup creating the compute depth decompress pipeline
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
b3d9afe44d radv/meta: create the compute depth decompress pipeline on-demand
It was always compiled when the device is created.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
f3e7c7e19f radv/meta: rework creating the gfx depth decompress pipeline
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
b54dc6a29a radv/meta: remove unused parameter to radv_get_depth_pipeline()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
38567300d4 radv/meta: move locking around the gfx depth decompress pipeline
Locking is only needed when the pipeline is created on-demand, so move
that to the caller.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
7c94ed0394 radv/meta: avoid potential NULL deref with the gfx depth decompress pipeline
If the pipeline failed to be created.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
9c1fa23adf radv/meta: remove the depth resummarize operation
This has never been used and if we need at some point, we can just
re-introduce it.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
76bf65b613 radv/meta: rework creating the VRS copy HTILE pipeline
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
f5c743e9e9 radv/meta: fix potential race condition when creating the copy VRS pipeline
This could lead to a race condtion if two command buffers are recorded
at the same time because it's accessing the device meta state.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30233>
2024-07-17 21:55:32 +00:00
Samuel Pitoiset
619bcd3b5c radv: allow to capture with RGP on GFX11_5
It works fine.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30225>
2024-07-17 16:25:19 +00:00
Samuel Pitoiset
e2882ea3e2 ac/rgp: assume GFX11_5 use the same SQTT/RGP versions as GFX11
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30225>
2024-07-17 16:25:19 +00:00
Samuel Pitoiset
664a31bcd1 radv: disable SPM trace on GFX11_5
SPM needs performance counters and they aren't exposed yet.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30225>
2024-07-17 16:25:19 +00:00
Samuel Pitoiset
057c4e3786 radv: expose BufferFloat32AtomicMinMax on GFX11_5
This is supported like GFX11.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30225>
2024-07-17 16:25:19 +00:00
Samuel Pitoiset
d2ae0c9ef8 radv: fix programming DB_RENDER_CONTROL for NULL depth/stencil on GFX11_5
It should be programmed like GFX11.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30225>
2024-07-17 16:25:19 +00:00
Samuel Pitoiset
1c5779250b radv: do not expose ImageFloat32AtomicMinMax on GFX11_5
These opcodes aren't supported on GFX11-11.5.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30225>
2024-07-17 16:25:19 +00:00
Georg Lehmann
eff0998064 radv: use radv_nir_opt_tid_function to create inverse_ballot
Foz-DB Navi21:
Totals from 542 (0.68% of 79395) affected shaders:
Instrs: 617316 -> 616259 (-0.17%); split: -0.19%, +0.02%
CodeSize: 3347852 -> 3320040 (-0.83%); split: -0.85%, +0.02%
VGPRs: 21864 -> 21824 (-0.18%); split: -0.29%, +0.11%
SpillSGPRs: 207 -> 199 (-3.86%)
Latency: 4900847 -> 4895665 (-0.11%); split: -0.11%, +0.01%
InvThroughput: 860278 -> 857272 (-0.35%); split: -0.35%, +0.00%
SClause: 21251 -> 21169 (-0.39%); split: -0.40%, +0.01%
Copies: 57759 -> 58881 (+1.94%); split: -0.06%, +2.00%
Branches: 20854 -> 20365 (-2.34%); split: -2.36%, +0.01%
PreSGPRs: 20785 -> 20774 (-0.05%)
PreVGPRs: 17309 -> 17212 (-0.56%)
VALU: 379885 -> 378180 (-0.45%); split: -0.45%, +0.00%
SALU: 87522 -> 88664 (+1.30%); split: -0.02%, +1.32%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24650>
2024-07-17 15:04:38 +00:00
Georg Lehmann
39de178656 radv: use radv_nir_opt_tid_function for shuffles
The main motivation were open coded clustered inclusive scans
and clustered broadcasts in the gdeflate decompression shader used by
DirectStorage.

Foz-DB Navi21 (only the_last_of_us_part1 is affected):
Totals from 8 (0.01% of 79395) affected shaders:
Instrs: 6230 -> 5438 (-12.71%)
CodeSize: 33376 -> 29148 (-12.67%)
Latency: 77017 -> 72917 (-5.32%)
InvThroughput: 10190 -> 9280 (-8.93%)
Copies: 566 -> 569 (+0.53%)
PreSGPRs: 528 -> 524 (-0.76%)
PreVGPRs: 232 -> 230 (-0.86%)
VALU: 2889 -> 2616 (-9.45%)
SALU: 1748 -> 1491 (-14.70%); split: -14.82%, +0.11%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24650>
2024-07-17 15:04:38 +00:00
Georg Lehmann
ca88783318 radv/nir: add a pass to optimize shuffle/booleans dependent only on tid/consts
This pass uses constant folding to determine which invocation is read by shuffle
for each invocation. Then, it detects patterns in the result and uses more
a specialized intrinsic if possible.

For booleans it creates inverse_ballot.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24650>
2024-07-17 15:04:38 +00:00
Georg Lehmann
2d3f536174 aco,nir: add dpp16_shift_amd intrinsic
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24650>
2024-07-17 15:04:38 +00:00
Alex Deucher
dec4603a21 ac/surface: fix version check for gfx12 DCC
It's 58, not 59.  The original 58 feature was not
ready so it was dropped.

Fixes: 0bb83a4060 ("ac/surface: finish display DCC for gfx12")
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30212>
2024-07-16 16:33:32 +00:00
Josh Simmons
2b99906e5b radv: Fix shader mask for SQ_WGP SPM counters
Signed-off-by: Josh Simmons <josh@nega.tv>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30200>
2024-07-16 16:10:11 +00:00
Daniel Schürmann
6723128e94 aco/spill: Don't add phi definitions to live-in variables
Changes are because we don't add artificial uses to the phi definitions anymore.

Totals from 13 (0.02% of 79395) affected shaders: (GFX10.3)

Instrs: 230510 -> 230285 (-0.10%); split: -0.10%, +0.00%
CodeSize: 1269916 -> 1268760 (-0.09%); split: -0.10%, +0.01%
SpillSGPRs: 2057 -> 2058 (+0.05%)
Latency: 2729731 -> 2723103 (-0.24%)
InvThroughput: 696888 -> 695286 (-0.23%)
VClause: 5795 -> 5768 (-0.47%)
SClause: 6855 -> 6858 (+0.04%)
Copies: 32336 -> 32275 (-0.19%); split: -0.22%, +0.03%
VALU: 151782 -> 151731 (-0.03%); split: -0.04%, +0.01%
SALU: 30766 -> 30758 (-0.03%); split: -0.03%, +0.01%
VMEM: 12157 -> 12078 (-0.65%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
2024-07-16 14:00:49 +00:00
Daniel Schürmann
bb5af6bede aco: remove live-out variables from IR
Since we changed all passes to use the live-in variables,
these are not needed anymore.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
2024-07-16 14:00:49 +00:00
Daniel Schürmann
f86816ca85 aco/print_ir: print live-in instead of live-out variables
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
2024-07-16 14:00:49 +00:00
Daniel Schürmann
043ec096c1 aco/validate: use live-in variables for RA validation
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
2024-07-16 14:00:49 +00:00
Daniel Schürmann
976dd71942 aco/cssa: use live-in variables instead of live-out variables
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
2024-07-16 14:00:49 +00:00
Daniel Schürmann
c146d4b6b6 aco/spill: use live-in variables directly rather than computing them
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30120>
2024-07-16 14:00:49 +00:00