Commit graph

10725 commits

Author SHA1 Message Date
Samuel Pitoiset
da32cbb5c6 aco: fix missing uses of MRT output flags
Fixes regressions on GFX6 and the RAGE2 workaround.

Fixes: a297ac10a4 ("radv,aco: stop lowering FS outputs in NIR")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20154>
2022-12-05 15:01:19 +00:00
Samuel Pitoiset
b051719b05 radv: do not set ZPASS_INCREMENT_DISABLE on GFX11
This field no longer exists.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20090>
2022-12-05 12:13:29 +01:00
Samuel Pitoiset
3ab9218820 radv: fix SPI_SHADER_Z_FORMAT for alpha-to-coverage via MRTZ on GFX11
It should select a 32-bit format with alpha.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20126>
2022-12-05 08:22:28 +00:00
Samuel Pitoiset
a297ac10a4 radv,aco: stop lowering FS outputs in NIR
This was a bad idea because:
- it diverges too much with the fragment shader epilog
- it doesn't allow to implement alpha-to-coverage via MRTZ correctly
- it was supposed to be used by LLVM but this never happened

Reverting this back allows us to fix alpha-to-coverage via MRTZ
on GFX11 easily, including for fragment shader epilogs.

fossils-db (NAVI21):
Totals from 20411 (15.13% of 134913) affected shaders:
VGPRs: 972056 -> 971400 (-0.07%); split: -0.08%, +0.01%
CodeSize: 92284804 -> 92295392 (+0.01%); split: -0.05%, +0.06%
MaxWaves: 465010 -> 465166 (+0.03%); split: +0.03%, -0.00%
Instrs: 17034162 -> 17034963 (+0.00%); split: -0.00%, +0.01%
Latency: 252013190 -> 251971764 (-0.02%); split: -0.03%, +0.02%
InvThroughput: 45859625 -> 45842556 (-0.04%); split: -0.04%, +0.01%
VClause: 324627 -> 324629 (+0.00%); split: -0.03%, +0.03%
SClause: 672918 -> 672826 (-0.01%); split: -0.05%, +0.04%
Copies: 1172126 -> 1158152 (-1.19%); split: -1.20%, +0.01%
Branches: 420602 -> 420604 (+0.00%); split: -0.00%, +0.00%
PreSGPRs: 1025441 -> 1025481 (+0.00%)
PreVGPRs: 861787 -> 860650 (-0.13%); split: -0.17%, +0.03%

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20126>
2022-12-05 08:22:28 +00:00
Samuel Pitoiset
3be728f1d0 aco: fix indexing MRT0 alpha channel for alpha-to-coverage via MRTZ on GFX11
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20126>
2022-12-05 08:22:28 +00:00
Samuel Pitoiset
20856bfe0f aco: always use 32-bit for exporting alpha-to-coverage via MRTZ on GFX11
16-bit isn't possible. Note that this is currently style broken for
compressed formats because the w channel is never written to.

Ported from RadeonSI ('radeonsi/gfx11: fix alpha-to-coverage with
stencil or samplemask export')

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20126>
2022-12-05 08:22:28 +00:00
Samuel Pitoiset
664aa7a37b radv: fix emitting invalid color attachments
Note sure how this happened.

Fixes: 97dc28b177 ("radv: fix configuring COLOR_INVALID on GFX11")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20127>
2022-12-05 07:46:47 +00:00
Konstantin Seurer
ad8de42ce5 radv: Use get_first_non_void_channel more often
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18634>
2022-12-02 22:06:11 +00:00
Konstantin Seurer
6397304519 radv: Only create bvh pipelines when using rt
Saves some time when creating non-rt devices.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20110>
2022-12-02 21:14:00 +00:00
Konstantin Seurer
7fe515f6d4 radv/rra: Get rid of annoying memory aliasing warning
Such cursed behavior is almost non existent in practise. When capturing
a Doom Eternal, this warning spams the output for no reason.

The warning is also unnecessary since we copy acceleration structures
right after building them now.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
e2b7e478a5 radv/rra: Fix setting some offsets
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
79dcacfc04 radv/rra: Refactor rra_fill_accel_struct_header_common
No need to re-do the offset calculation for every field.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
bb6b45e26e radv/rra: Set the metadata size correctly
Fixes: 5749806 ("radv: Add Radeon Raytracing Analyzer trace dumping utilities")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
0e3325dfb6 radv/rra: Remove an obsolete comment
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
94ec359ae5 radv/rra: Defer destroying accel struct data
This allows us to dump acceleration structures that were destroyed
before present.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
ae9c65a552 radv/rra: Copy accel structs directly after build
This is the second step of decoupling acceleration structure dumping
from lifetimes. It also simplifies the logic a bit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
08a85076e5 radv/rra: Introduce radv_rra_accel_struct_data
This will be useful for dumping acceleration structures that were
destroyed before submit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
ff3ba5c74d radv: Add hash_table_foreach to .clang-format
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
David Heidelberg
f7e76eee28 ci/amd: re-enable previously OOM tests
Since we have ZRAM now, we can enable previously failing tests on OOM.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19535>
2022-12-02 13:51:15 +00:00
Bas Nieuwenhuizen
89663828ea aco: Don't use v_lshrrev_b64 for moves on GFX11.
Looking at VOPD things, shifts are not very likely to get dual issued
but plain moves are. Looking at RDNA2 v_lshrrev_b64 are half the perf
of v_mov_b32 (but you need twice as many moves), so on GFX11 this likely
reaches the threshold where moves are faster.

Totals from 68400 (50.70% of 134906) affected shaders:

CodeSize: 275489516 -> 275459536 (-0.01%); split: -0.01%, +0.00%
Instrs: 51775474 -> 51991286 (+0.42%)
Latency: 589884847 -> 589066439 (-0.14%); split: -0.15%, +0.01%
InvThroughput: 127154986 -> 126037619 (-0.88%); split: -0.88%, +0.00%
Copies: 3756157 -> 3976193 (+5.86%)
Branches: 1259604 -> 1260072 (+0.04%)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19633>
2022-12-02 13:25:57 +00:00
Bas Nieuwenhuizen
91fe2a2361 aco: Use more detailed wave64 timing for GFX10+.
Also nabbed some dual issue stuff for GFX11 from LLVM.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19633>
2022-12-02 13:25:57 +00:00
Qiang Yu
2fb1097bac ac/nir/ngg: merge multi stream gs shader queries
Before this commit each stream will emit a query block, now
we merge them to a single block.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20074>
2022-12-02 09:38:07 +00:00
Qiang Yu
6c44d92362 ac/llvm,radeonsi: lower attribute ring intrinsics in nir
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>
2022-12-02 07:34:32 +00:00
Qiang Yu
daaa8ddb8e ac/llvm,radeonsi: lower nir primitive counter add intrinsics
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>
2022-12-02 07:34:31 +00:00
Qiang Yu
bb837bf6ef nir,ac/llvm: add nir_buffer_atomic_add_amd
Used by radeonsi for lower nir_atomic_add_gen/xfb_prim_count_amd.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>
2022-12-02 07:34:31 +00:00
Qiang Yu
7cec2e7520 ac/llvm,radeonsi: lower nir_load_streamout_buffer_amd
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>
2022-12-02 07:34:31 +00:00
Qiang Yu
daf5d30b59 ac/llvm,radeonsi: lower nir_load_user_clip_plane in abi
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>
2022-12-02 07:34:31 +00:00
Qiang Yu
84abc307a5 ac/llvm: remove lowered abi->intrinsic_load() intrinsics
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>
2022-12-02 07:34:31 +00:00
Qiang Yu
e9f08d8193 ac/nir: add ac_nir_unpack_arg
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>
2022-12-02 07:34:31 +00:00
Qiang Yu
8030fbcf16 nir,ac/llvm: add nir_load_smem_buffer_amd
Used by radeonsi to load const buffer.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>
2022-12-02 07:34:31 +00:00
Qiang Yu
73ea7d651a ac/llvm: nir_load_smem_amd support 32bit base address
For radeonsi which use 32bit address in ac_build_load_to_sgpr().

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18010>
2022-12-02 07:34:31 +00:00
Qiang Yu
9b2ec290c4 ac/llvm: remove unused llvm cull
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17109>
2022-12-02 04:37:23 +00:00
Qiang Yu
7e1b804992 radeonsi: implement two lds base load intrinsics
LDS will be accessed starting from esgs_ring which has offset 0.
So ngg_scratch and ngg_emit base address is just the offset from
the esgs_ring base.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17109>
2022-12-02 04:37:23 +00:00
Qiang Yu
3c1ebebeae radeonsi: use nir_lower_gs_intrinsics
Replace some llvm code.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17109>
2022-12-02 04:37:23 +00:00
Rhys Perry
9b6ab40b3b aco: improve do_pack_2x16() with zero constants
We can skip the v_or_b32 or use an instruction smaller than
v_alignbyte_b32.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19933>
2022-12-01 21:43:28 +00:00
Rhys Perry
917cfd587c aco: use v_minmax/v_maxmin opcodes
fossil-db (gfx1100):
Totals from 29868 (22.12% of 135032) affected shaders:
MaxWaves: 741336 -> 741344 (+0.00%)
Instrs: 34624902 -> 34539766 (-0.25%); split: -0.25%, +0.00%
CodeSize: 187196804 -> 187192100 (-0.00%); split: -0.01%, +0.01%
VGPRs: 1816860 -> 1816788 (-0.00%); split: -0.01%, +0.01%
Latency: 502597202 -> 502245627 (-0.07%); split: -0.08%, +0.01%
InvThroughput: 84813176 -> 84586122 (-0.27%); split: -0.28%, +0.01%
VClause: 633826 -> 633749 (-0.01%); split: -0.02%, +0.01%
SClause: 1317738 -> 1317047 (-0.05%); split: -0.06%, +0.01%
Copies: 2130610 -> 2130954 (+0.02%); split: -0.03%, +0.05%
Branches: 766093 -> 765969 (-0.02%); split: -0.02%, +0.00%
PreSGPRs: 1630250 -> 1630034 (-0.01%); split: -0.02%, +0.00%
PreVGPRs: 1590777 -> 1590664 (-0.01%); split: -0.01%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19933>
2022-12-01 21:43:28 +00:00
Rhys Perry
dfbc8e0192 aco: change order in combine_minmax()
Prepare for future optimizations.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19933>
2022-12-01 21:43:28 +00:00
Rhys Perry
ce5838599d aco/gfx11: use v_cvt_i32_i16/v_cvt_u32_u16
fossil-db (gfx1100):
Totals from 52753 (39.07% of 135032) affected shaders:
CodeSize: 153603860 -> 153163384 (-0.29%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19933>
2022-12-01 21:43:28 +00:00
Samuel Pitoiset
ab7f518ed0 radv,driconf: fix static driconf by parsing 00-radv-defaults.conf
Otherwise when xmlconfig is disabled, drirc workarounds aren't applied
with RADV.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7785
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20077>
2022-12-01 16:55:31 +00:00
Qiang Yu
076a333d40 ac/nir/ngg: rename nogs 16bit output mask and var
To represent 16bit outputs more clearly.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19697>
2022-12-01 13:10:35 +00:00
Qiang Yu
abe2e99e9e ac/nir/ngg: gs support 16bit outputs
radeonsi uses 16bit varying slots.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19697>
2022-12-01 13:10:35 +00:00
Qiang Yu
68519891a7 ac/nir/ngg: gs skip check bit size before nir_u2u
nir_u2u do for us.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19697>
2022-12-01 13:10:35 +00:00
Qiang Yu
d3e20e8834 ac/nir/ngg: gs store output use src_type index for type info
More precise type info, can be used for 16bit output streamout
to convert 16bit int/uint/float to 32bit one later.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19697>
2022-12-01 13:10:35 +00:00
Qiang Yu
0cb5ea512f ac/nir/ngg: gs use u_foreach_bit64 to loop all output slots
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19697>
2022-12-01 13:10:35 +00:00
Qiang Yu
13b75594d7 ac/nir/ngg: reduce nogs 16bit output gather space
Max slot number for 16bit output is 16, so no need to use
64 array size for them.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19697>
2022-12-01 13:10:35 +00:00
Bas Nieuwenhuizen
9a311a1891 radv: Remove the old LBVH shader.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19891>
2022-12-01 02:20:48 +00:00
Bas Nieuwenhuizen
5ba950eb14 radv: Switch to new LBVH implementation.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19891>
2022-12-01 02:20:48 +00:00
Bas Nieuwenhuizen
ea159e47a5 radv: Add new LBVH shaders.
Contrary to the previous implementation, this actually implements an LBVH builder.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19891>
2022-12-01 02:20:48 +00:00
Bas Nieuwenhuizen
f531f671ef radv: Handle nodes with 2 invalid children in internal node converter.
Fixes: 682dc5c28e ("radv: Add conversion shader for internal nodes")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19891>
2022-12-01 02:20:48 +00:00
Georg Lehmann
a3beb82cf6 aco: Use wave size specific opcode for s_or in cube map coord code.
Cc: mesa-stable

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20041>
2022-12-01 01:39:27 +00:00