Commit graph

15150 commits

Author SHA1 Message Date
Georg Lehmann
05ca6e2478 amd/common: set COMPUTE_STATIC_THREAD_MGMT_SE2-3 correctly on gfx10-11
There is a hole between SE1 and SE2 occupied by COMPUTE_TMPRING_SIZE.

Fixes: 3c8b48e310 ("ac,radeonsi: add a function to initialize compute preambles")

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29622>
2024-06-08 19:18:53 +00:00
Eric Engestrom
c02329ded1 ci: set a common B2C_JOB_SUCCESS_REGEX with the message that's printed for all jobs
Simpler code, and more reliable against serial corruption because that
message is printed 4 times (vs only once for the other ones).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29608>
2024-06-08 07:16:27 +00:00
Marek Olšák
dc113c418d ac/nir: import the dispatch logic for the universal compute clear/blit shader
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28917>
2024-06-08 05:48:11 +00:00
Marek Olšák
6b15e45908 ac/nir: import the universal compute clear/blit shader
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28917>
2024-06-08 05:48:11 +00:00
Marek Olšák
1becc6953c ac/nir: import the MSAA resolving pixel shader from radeonsi
It has a lot of options for efficiency.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28917>
2024-06-08 05:48:11 +00:00
Marek Olšák
f96bbb64d6 radeonsi: add decision code to select when to use compute blit for performance
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28917>
2024-06-08 05:48:11 +00:00
Marek Olšák
1b9ce2625f ac/nir/lower_ngg: don't use gfx12 xfb defs outside their basic block on gfx11
Move the defs after nir_pop_if and phis and inside the gfx12 branch.

Fixes: 1ea96a47cd - ac/nir/lower_ngg: use voffset in global_atomic_add for xfb

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29564>
2024-06-08 00:11:18 -04:00
Marek Olšák
ea99c3fcb9 amd: update addrlib
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29564>
2024-06-08 00:11:17 -04:00
Marek Olšák
2ea3cb054b ac/surface: pass the correct addrlib handle to Addr3GetPossibleSwizzleModes
Fixes: d22564d29c - ac/surface: add gfx12

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29564>
2024-06-08 00:11:15 -04:00
Eric Engestrom
bd6ace73f3 radv/ci: document navi31 regression from !29235
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29596>
2024-06-07 20:57:01 +00:00
Vignesh Raman
cea3aeefd0 ci: add farm variable for devices in collabora farm
Add farm variable for devices in the collabora farm so that
the LAVA job submitter uses this in structured log files.

Signed-off-by: Vignesh Raman <vignesh.raman@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29583>
2024-06-07 14:32:49 +00:00
Rhys Perry
5297896856 aco: use ac_get_hw_cache_flags()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243>
2024-06-07 13:22:43 +00:00
Rhys Perry
167b6cac45 ac: stop using radeon_info for ac_get_hw_cache_flags
This makes the function easier to use when radeon_info is not available.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243>
2024-06-07 13:22:43 +00:00
Rhys Perry
00eccf524f aco: use GFX12 scope/temporal-hint
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243>
2024-06-07 13:22:42 +00:00
Rhys Perry
b41f0f6cc1 aco: use ac_hw_cache_flags
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243>
2024-06-07 13:22:42 +00:00
Rhys Perry
cdaf269924 aco: inline store_vmem_mubuf/emit_single_mubuf_store
Both of these are only used once.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243>
2024-06-07 13:22:42 +00:00
Rhys Perry
185fa04baa aco/gfx6: set glc for buffer_store_byte/short
For the same reason we set it for image stores. GFX6 has a caching bug
which requires this.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29243>
2024-06-07 13:22:42 +00:00
Samuel Pitoiset
d4ccae739b radv: fix creating unlinked shaders with ESO when nextStage is 0
When nextStage is 0, the driver needs to assume that a stage might be
used with any valid next stages.

Fixes new dEQP-VK.shader_object.binding.*_no_next_stage.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29567>
2024-06-07 12:21:38 +00:00
Friedrich Vock
f1742d36f3 radv/rt: Fix memory leak when compiling libraries
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29579>
2024-06-06 21:56:11 +00:00
Daniel Schürmann
c452a4d1cc aco/ra: use round robin register allocation
Totals from 74681 (94.06% of 79395) affected shaders: (GFX11)

MaxWaves: 2265668 -> 2263546 (-0.09%); split: +0.01%, -0.10%
Instrs: 44941647 -> 44412809 (-1.18%); split: -1.23%, +0.05%
CodeSize: 234173852 -> 232009132 (-0.92%); split: -0.97%, +0.05%
VGPRs: 3033208 -> 3403000 (+12.19%); split: -0.02%, +12.22%
Latency: 305575738 -> 301100302 (-1.46%); split: -1.70%, +0.23%
InvThroughput: 49366070 -> 49020000 (-0.70%); split: -0.91%, +0.21%
VClause: 875748 -> 854930 (-2.38%); split: -2.65%, +0.27%
SClause: 1369614 -> 1327212 (-3.10%); split: -3.43%, +0.33%
Copies: 2887932 -> 2883061 (-0.17%); split: -1.93%, +1.76%
Branches: 885041 -> 885101 (+0.01%); split: -0.01%, +0.02%
VALU: 25218078 -> 25215170 (-0.01%); split: -0.20%, +0.19%
SALU: 4328640 -> 4326052 (-0.06%); split: -0.20%, +0.14%
VOPD: 9129 -> 9611 (+5.28%); split: +7.48%, -2.20%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29235>
2024-06-06 21:02:15 +00:00
Daniel Schürmann
197943ae27 aco/ra: change heuristic to first fit
Totals from 73175 (92.17% of 79395) affected shaders: (GFX11)

MaxWaves: 2217690 -> 2217930 (+0.01%); split: +0.02%, -0.01%
Instrs: 44780731 -> 44784895 (+0.01%); split: -0.14%, +0.15%
CodeSize: 233238960 -> 233255604 (+0.01%); split: -0.11%, +0.12%
VGPRs: 3009116 -> 3007684 (-0.05%); split: -0.29%, +0.24%
Latency: 304320163 -> 304286592 (-0.01%); split: -0.31%, +0.30%
InvThroughput: 49121992 -> 49145025 (+0.05%); split: -0.20%, +0.25%
VClause: 872566 -> 873242 (+0.08%); split: -0.25%, +0.33%
SClause: 1359666 -> 1361640 (+0.15%); split: -0.11%, +0.26%
Copies: 2879649 -> 2881646 (+0.07%); split: -1.13%, +1.20%
Branches: 887102 -> 887093 (-0.00%); split: -0.01%, +0.01%
VALU: 25128240 -> 25128572 (+0.00%); split: -0.12%, +0.12%
SALU: 4328852 -> 4330559 (+0.04%); split: -0.07%, +0.11%
VOPD: 8861 -> 8992 (+1.48%); split: +2.63%, -1.15%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29235>
2024-06-06 21:02:15 +00:00
Daniel Schürmann
d76fc005b6 aco/ra: re-use registers from killed operands
Totals from 77283 (97.34% of 79395) affected shaders: (GFX11)

MaxWaves: 2348498 -> 2348250 (-0.01%); split: +0.01%, -0.02%
Instrs: 45304558 -> 45097367 (-0.46%); split: -0.57%, +0.11%
CodeSize: 235719656 -> 234957768 (-0.32%); split: -0.43%, +0.11%
VGPRs: 3065984 -> 3073244 (+0.24%); split: -0.41%, +0.65%
Latency: 308010576 -> 307008565 (-0.33%); split: -0.85%, +0.52%
InvThroughput: 49560307 -> 49464214 (-0.19%); split: -0.54%, +0.34%
VClause: 881895 -> 879739 (-0.24%); split: -0.78%, +0.53%
SClause: 1388139 -> 1374634 (-0.97%); split: -1.12%, +0.14%
Copies: 2918583 -> 2910434 (-0.28%); split: -1.92%, +1.64%
Branches: 893947 -> 893712 (-0.03%); split: -0.06%, +0.03%
VALU: 25260728 -> 25256766 (-0.02%); split: -0.20%, +0.19%
SALU: 4377750 -> 4373595 (-0.09%); split: -0.17%, +0.07%
VOPD: 8603 -> 9163 (+6.51%); split: +8.54%, -2.03%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29235>
2024-06-06 21:02:15 +00:00
Daniel Schürmann
b054cfe704 aco/ra: move can_write_m0() check into get_reg_specified()
This way, affinities are also covered.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29235>
2024-06-06 21:02:15 +00:00
Daniel Schürmann
8e817cf52b aco/ra: refactor get_reg_simple() with increased stride.
This should avoid some redundant calls.

Totals from 153 (0.19% of 79395) affected shaders: (GFX11)

Instrs: 301717 -> 301687 (-0.01%); split: -0.06%, +0.05%
CodeSize: 1583080 -> 1582988 (-0.01%); split: -0.06%, +0.05%
VGPRs: 10068 -> 10348 (+2.78%)
Latency: 6685446 -> 6685475 (+0.00%); split: -0.11%, +0.11%
InvThroughput: 999241 -> 999316 (+0.01%); split: -0.01%, +0.02%
VClause: 3868 -> 3870 (+0.05%)
Copies: 23752 -> 23769 (+0.07%); split: -0.27%, +0.34%
Branches: 6479 -> 6480 (+0.02%)
VALU: 179290 -> 179307 (+0.01%); split: -0.04%, +0.04%
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29235>
2024-06-06 21:02:15 +00:00
Daniel Schürmann
1b0edf3f33 aco/ra: Fix array access when finding register for subdword variables
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29235>
2024-06-06 21:02:15 +00:00
Daniel Schürmann
5326e033ff aco/ra: fix handling of killed operands in compact_relocate_vars()
Found by inspection.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29235>
2024-06-06 21:02:14 +00:00
Samuel Pitoiset
afa2070c99 radv: initialize compute preambles with the common helper
The PM4 mechanism can emit paired packets on GFX11+ when possible.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29452>
2024-06-06 20:26:47 +00:00
Samuel Pitoiset
3c8b48e310 ac,radeonsi: add a function to initialize compute preambles
Preambles are very similar between RADV and RadeonSI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29452>
2024-06-06 20:26:47 +00:00
Samuel Pitoiset
428601095c ac,radeonsi import PM4 state from RadeonSI
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29452>
2024-06-06 20:26:47 +00:00
Samuel Pitoiset
15fe733703 radv: add a helper to get image VA
Similar to buffer, and less error prone.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29428>
2024-06-06 18:21:33 +00:00
Rhys Perry
4cfb7a0c17 aco: remove support for sub-dword push constants
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29480>
2024-06-06 17:52:05 +00:00
Rhys Perry
e21312018e ac/llvm: remove support for sub-dword push constants
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29480>
2024-06-06 17:52:05 +00:00
Rhys Perry
41c5f71343 radv: lower sub-dword push constants
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29480>
2024-06-06 17:52:05 +00:00
Rhys Perry
69b7fcd775 ac/nir: support lowering of sub-dword push constants
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29480>
2024-06-06 17:52:04 +00:00
Georg Lehmann
3fb1a64918 aco: move s_add_u32 -> s_addk_i32 optimization fully to ra
Having this in one place is better.
When I wrote the old I wasn't aware that checking the kill flag on definitions
is the same as checking zero uses.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29512>
2024-06-06 16:28:23 +00:00
Georg Lehmann
60f3f0fdbb aco/ra: use a switch to check vop2acc instruction support
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29512>
2024-06-06 16:28:23 +00:00
Georg Lehmann
fdc2fb6835 aco: move literal unswizzle opt to RA
Much simpler.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29512>
2024-06-06 16:28:23 +00:00
Georg Lehmann
c63c750380 aco/gfx11+: fix inline constants for v_pk_fmac_f16
On newer hardware, the hi operation reads the lo half of the inline constant.
On older hardware, it reads the hi half (zero).
I tested this on Navi31 for gfx11 and Raphael for gfx10.

Foz-DB Navi31:
Totals from 4 (0.01% of 79395) affected shaders:
CodeSize: 36832 -> 36448 (-1.04%)
Latency: 20362 -> 20334 (-0.14%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29512>
2024-06-06 16:28:23 +00:00
Georg Lehmann
39380d475a aco: add affinities for possible sopk optimizations
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29512>
2024-06-06 16:28:23 +00:00
Georg Lehmann
fac475bc25 aco: rework how affinities for acc operands are determined
Improve accuracy by adding a helper that's also used by
the optimization function.

Foz-DB Navi31:
Totals from 50 (0.06% of 79206) affected shaders:
CodeSize: 126148 -> 126128 (-0.02%); split: -0.05%, +0.04%
Latency: 334049 -> 334060 (+0.00%); split: -0.00%, +0.00%
InvThroughput: 59203 -> 59205 (+0.00%)
Copies: 2011 -> 1998 (-0.65%); split: -0.75%, +0.10%
VALU: 14221 -> 14208 (-0.09%); split: -0.11%, +0.01%

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29512>
2024-06-06 16:28:23 +00:00
Samuel Pitoiset
5b6207b282 radv: only set valid bitfields for CB/DS surfaces address on GFX12
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29566>
2024-06-06 15:42:35 +00:00
Samuel Pitoiset
fe78ad2690 radv: fix emitting VGT_PRIMITIVEID_RESET in the GFX preamble on GFX12
It's a uconfig register.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29566>
2024-06-06 15:42:35 +00:00
Samuel Pitoiset
be3c837c04 radv: update configuring COVERAGE_TO_SHADER_SELECT on GFX12
This bit has been moved to SPI_PS_INPUT_ENA.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29566>
2024-06-06 15:42:35 +00:00
Samuel Pitoiset
27496928e4 radv: update configuring depth clamp enable on GFX12
DISABLE_VIEWPORT_CLAMP has been moved to a new register.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29566>
2024-06-06 15:42:35 +00:00
Rhys Perry
8e475bba61 aco: implement nir_intrinsic_nop_amd and nir_intrinsic_sleep_amd
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29466>
2024-06-06 14:26:52 +00:00
Rhys Perry
1ad05d4ca8 aco: implement nir_atomic_op_ordered_add_gfx12_amd
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29466>
2024-06-06 14:26:52 +00:00
Rhys Perry
0dee5fdd3c aco: don't combine vgpr into writelane src0
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29466>
2024-06-06 14:26:52 +00:00
Rhys Perry
2a4424425a aco/gfx12: fix s_wait_event immediate
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29466>
2024-06-06 14:26:52 +00:00
Rhys Perry
26c981b2b8 ac/nir: skip subgroup_id/local_invocation_index lowering for gfx12
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29466>
2024-06-06 14:26:52 +00:00
Rhys Perry
c651eed1d8 aco/gfx12: implement load_subgroup_id
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29466>
2024-06-06 14:26:52 +00:00