Konstantin Seurer
2a5d8d4cf4
aco: Unify demote and demote_if selection
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24906 >
2023-09-01 07:23:33 +00:00
Konstantin Seurer
9af91edda9
aco: Use bytes() instead of size() in emit_wqm
...
This should get most of the cases that would fail validation.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24906 >
2023-09-01 07:23:33 +00:00
Samuel Pitoiset
e104718c9f
aco: allow separate compilation of NGG shaders
...
Also prevent to emit a long-jump for VS as NGG without a GS.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24907 >
2023-09-01 06:52:40 +00:00
Samuel Pitoiset
bfb39031f1
aco: flag blocks with long-jump as export_end for separate compilation
...
This will allow us to adjust fix_exports().
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24907 >
2023-09-01 06:52:40 +00:00
Qiang Yu
a2ba50aee6
aco: add vs prolog instruction selection for radeonsi
...
Port from llvm si_llvm_build_vs_prolog().
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24712 >
2023-08-31 02:39:15 +00:00
Qiang Yu
3f87413811
aco: prepare fix_ls_vgpr_init_bug to be used by gl vs prolog
...
Prolog does not have nir shader.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24712 >
2023-08-31 02:39:15 +00:00
Qiang Yu
eb4f871034
aco: pass sw_stage when setup_isel_context
...
We are going to add more shader parts.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24712 >
2023-08-31 02:39:15 +00:00
Samuel Pitoiset
f30d47c518
aco: fix emitting TCS epilogs end on GFX9+
...
With merged shaders, the long-jump should be emitted inside the
divergent if (ie. only for TCS invocations) and other non TCS
invocations should just end the program.
This fixes a bunch of failures with CTS by forcing TCS epilogs on
RDNA2.
Not sure how RadeonSI will handle that but maybe doing the merged
wave info thing in epilogs would help.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24832 >
2023-08-29 09:30:07 +00:00
Samuel Pitoiset
8ebb29245c
aco: add support for compiling {VS,TES}+GS separately on GFX9+
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24862 >
2023-08-25 10:22:41 +00:00
Samuel Pitoiset
80177e0296
aco: add support for compiling VS+TCS separately on GFX9+
...
The VS will just jump to the TCS.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697 >
2023-08-25 07:22:04 +00:00
Samuel Pitoiset
aba16211a8
aco: disable shared VGPRs for non-monolithic shaders on GFX9+
...
For unmerged shaders on GFX9+, we would need to jump to the second
shader part which means shared VGPRs can't be enabled.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697 >
2023-08-25 07:22:04 +00:00
Georg Lehmann
6d949e18fd
aco: fix u2f16 with 32bit input
...
The vulkan spec says all conversions are correctly rounded, so if the input
is larger than the largest fp16 value, we need to return MAX_FLOAT/inf
instead of cutting off the msbs.
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24826 >
2023-08-23 12:25:56 +00:00
Rhys Perry
9169fbf83c
aco: clarify bpermute pseudo opcode names
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693 >
2023-08-23 12:36:46 +01:00
Konstantin Seurer
bdec044c88
aco: Do not fixup registers if there are no shader calls
...
Frees up some registers when using monolithic compilation.
Quake II RTX and Control (with monolithic compilation):
Totals from 10 (29.41% of 34) affected shaders:
MaxWaves: 77 -> 98 (+27.27%)
Instrs: 49047 -> 48984 (-0.13%); split: -0.16%, +0.03%
CodeSize: 260420 -> 259880 (-0.21%); split: -0.25%, +0.04%
VGPRs: 1328 -> 1104 (-16.87%)
Latency: 477134 -> 479377 (+0.47%); split: -0.05%, +0.52%
InvThroughput: 137763 -> 114108 (-17.17%)
VClause: 1318 -> 1286 (-2.43%); split: -2.66%, +0.23%
SClause: 1295 -> 1293 (-0.15%); split: -0.54%, +0.39%
Copies: 7838 -> 7782 (-0.71%); split: -0.82%, +0.10%
Branches: 2592 -> 2589 (-0.12%)
PreSGPRs: 874 -> 796 (-8.92%)
PreVGPRs: 1283 -> 1013 (-21.04%)
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809 >
2023-08-22 15:46:29 +00:00
Samuel Pitoiset
a29e2c6fbc
aco: implement create_tcs_jump_to_epilog()
...
This implements jumping from the main TCS to the epilog.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24643 >
2023-08-22 06:10:32 +00:00
Samuel Pitoiset
fc9283938f
aco: adjust TCS epilogs for RADV
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24643 >
2023-08-22 06:10:32 +00:00
Samuel Pitoiset
0c2adc7ada
aco: fix jumping from main TCS to epilog on GFX9+
...
On GFX9+, VS is merged with TCS which means this function is called
twice and the epilog was emitted in both shader parts.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24643 >
2023-08-22 06:10:32 +00:00
Qiang Yu
51a8479a51
aco: use semantic location as io temp index
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24442 >
2023-08-16 02:27:45 +00:00
Qiang Yu
facbd13df1
aco: skip scratch init when no scratch arg provide
...
epilog does not use scratch so has no scratch arg.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24442 >
2023-08-16 02:27:45 +00:00
Qiang Yu
d3333609e6
aco: don't emit s_endpgm for tcs with epilog
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24442 >
2023-08-16 02:27:45 +00:00
Qiang Yu
b41d3e42f7
aco: add tcs epilog generation for radeonsi
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24442 >
2023-08-16 02:27:45 +00:00
Qiang Yu
5cf6f4f9a7
aco: allow tcs with epilog to keep nir store output instruction
...
Used to same tess factor outputs.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24442 >
2023-08-16 02:27:45 +00:00
Qiang Yu
5d05ae5df0
aco: add tcs end regs for epilog usage
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24442 >
2023-08-16 02:27:45 +00:00
Qiang Yu
7c7062f8f9
aco: move jump to epilog out of ic_merged_wave_info
...
TCS may be wrapped with if/else when merged shader. Jump
to epilog or end with regs should not be inside it.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24442 >
2023-08-16 02:27:45 +00:00
Faith Ekstrand
65b6ac8aa4
nir: Rename nir_instr_type_ssa_undef to nir_instr_type_undef
...
We already renamed the type, we just need to rename the enum and the
casting helper functions.
Generated with sed:
sed -i -e 's/nir_instr_type_ssa_undef/nir_instr_type_undef/g' src/**/*.h src/**/*.c src/**/*.cpp
sed -i -e 's/nir_instr_as_ssa_undef/nir_instr_as_undef/g' src/**/*.h src/**/*.c src/**/*.cpp
and two tiny whitespace fixups in lima.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24703 >
2023-08-15 17:44:27 +00:00
Konstantin Seurer
c291f612cb
radv/rt: Rename shader_pc and next_shader
...
The names basically had the same meaning. Changing them to
uniform_shader_addr and shader_addr makes it clear, that
uniform_shader_addr is the jump target and shader_addr is the next
shader executed by this invocation.
The next_ prefix is dropped since both are input and output variables.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23565 >
2023-08-15 11:11:16 +02:00
Faith Ekstrand
4695bebc79
nir: Drop nir_dest
...
Instead, we replace every use of it with nir_def. Most of this commit
was generated by sed:
sed -i -e 's/dest.ssa/def/g' src/**/*.h src/**/*.c src/**/*.cpp
A few manual fixups were required in lima and the nir_legacy code.
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674 >
2023-08-14 21:22:53 +00:00
Faith Ekstrand
6c1d32581a
nir: Drop nir_alu_dest
...
Instead, we replace it directly with nir_def. We could replace it with
nir_dest but the next commit gets rid of that so this avoids unnecessary
churn. Most of this commit was generated by sed:
sed -i -e 's/dest.dest.ssa/def/g' src/**/*.h src/**/*.c src/**/*.cpp
There were a few manual fixups required in the nir_legacy.c and
nir_from_ssa.c as nir_legacy_reg and nir_parallel_copy_entry both have a
similar pattern.
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674 >
2023-08-14 21:22:53 +00:00
Faith Ekstrand
977999d836
nir: Get rid of nir_dest_is_divergent()
...
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674 >
2023-08-14 21:22:53 +00:00
Alyssa Rosenzweig
09d31922de
nir: Drop "SSA" from NIR language
...
Everything is SSA now.
sed -e 's/nir_ssa_def/nir_def/g' \
-e 's/nir_ssa_undef/nir_undef/g' \
-e 's/nir_ssa_scalar/nir_scalar/g' \
-e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \
-e 's/nir_gather_ssa_types/nir_gather_types/g' \
-i $(git grep -l nir | grep -v relnotes)
git mv src/compiler/nir/nir_gather_ssa_types.c \
src/compiler/nir/nir_gather_types.c
ninja -C build/ clang-format
cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \;
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Acked-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585 >
2023-08-12 16:44:41 -04:00
Georg Lehmann
c4f356faf4
aco: always use rtne for fquantize2f16
...
The SPIR-V spec says:
If Value is positive with a magnitude too large to represent as a
16-bit floating-point value, the result is positive infinity.
If Value is negative with a magnitude too large to represent as a
16-bit floating-point value, the result is negative infinity.
This is only the case for rtne v_cvt_f16_f32
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24617 >
2023-08-11 12:37:23 +00:00
Alyssa Rosenzweig
cf8ceb6e0a
aco: Remove is_ssa check
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432 >
2023-08-03 22:40:29 +00:00
Samuel Pitoiset
f4ec2e7bb3
radv,aco: move has_epilog to radv_shader_info
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24404 >
2023-08-02 16:59:18 +00:00
Samuel Pitoiset
f433d39935
aco: add infra for compiling TCS epilogs
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24417 >
2023-08-02 07:29:50 +00:00
Qiang Yu
572625ea6c
aco: extract aco_compile_shader_part from aco_compile_ps_epilog
...
Will be shared with radeonsi tcs epilog and other shader parts build.
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24417 >
2023-08-02 07:29:50 +00:00
Alyssa Rosenzweig
51db19f7a2
nir: Rename scoped_barrier -> barrier
...
sed + ninja clang-format + fix up spacing for common code.
If you are unhappy that I did not manually change the whitespace of your driver,
you need to enable clang-format for it so the formatting would happen
automatically.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24428 >
2023-08-01 23:18:29 +00:00
Georg Lehmann
82920c99a5
aco: fix non constant 16bit bitnz/bitz
...
Fixes: 573e98f34a ("aco: implement nir_op_bitz/bitnz")
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24420 >
2023-08-01 15:54:31 +00:00
Georg Lehmann
1e8d445bcf
aco/gfx11: use v_cmp_class_f16 with opsel for bitnz/bitz
...
Foz-DB GFX11:
Totals from 1607 (1.21% of 132657) affected shaders:
MaxWaves: 36511 -> 36513 (+0.01%)
Instrs: 2412601 -> 2408770 (-0.16%); split: -0.16%, +0.00%
CodeSize: 12434716 -> 12414536 (-0.16%); split: -0.17%, +0.00%
VGPRs: 108872 -> 108860 (-0.01%)
Latency: 27555570 -> 27541897 (-0.05%); split: -0.05%, +0.00%
InvThroughput: 3144382 -> 3140929 (-0.11%); split: -0.11%, +0.00%
SClause: 65762 -> 65770 (+0.01%)
Copies: 127148 -> 127170 (+0.02%); split: -0.06%, +0.08%
PreVGPRs: 96737 -> 96447 (-0.30%)
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24380 >
2023-08-01 07:52:42 +00:00
Timur Kristóf
be11fee2a7
aco: Refactor select_program to smaller functions.
...
This prepares for allowing to compile 1 shader at a time
for merged shader stages.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23874 >
2023-07-28 14:27:29 +00:00
Qiang Yu
599b50b448
aco,radv: replace tess_input_vertices shader info param
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204 >
2023-07-24 01:49:20 +00:00
Bas Nieuwenhuizen
e536d31a46
aco: Fix some constant patterns in 16-bit vec4 construction with s_pack.
...
Fixes: 04e3d7ad93 ("aco: improve nir_op_vec with constant operands")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24286 >
2023-07-22 22:11:15 +00:00
Bas Nieuwenhuizen
2fcf7c7014
aco: fix nir_op_vec8/16 with 16-bit elements.
...
Fixes: 5718347c2b ("aco: implement vec2/3/4 with subdword operands")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24286 >
2023-07-22 22:11:15 +00:00
Timur Kristóf
9fb9e54d69
aco: Remove subgroup_id and num_subgroups intrinsics.
...
These are lowered in NIR now.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24005 >
2023-07-14 12:17:24 +00:00
Timur Kristóf
dc3bbd351a
aco: Fix subgroup_id intrinsic on GFX10.3+.
...
Change this to match how it works in the LLVM backend.
Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24005 >
2023-07-14 12:17:24 +00:00
Rhys Perry
ae48fae658
aco: remove 64-bit integer conversion opcodes
...
These are handled by nir_lower_int64 now.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23926 >
2023-07-03 10:38:28 +00:00
Rhys Perry
25c49e491f
aco,ac/llvm,ac/nir,vtn: unify cube opcodes
...
fossil-db (navi21):
Totals from 17068 (12.79% of 133461) affected shaders:
Instrs: 24743703 -> 24743572 (-0.00%); split: -0.00%, +0.00%
CodeSize: 132579952 -> 132580620 (+0.00%); split: -0.00%, +0.00%
VGPRs: 1227840 -> 1227984 (+0.01%)
Latency: 403180114 -> 403251188 (+0.02%); split: -0.00%, +0.02%
InvThroughput: 75311302 -> 75320892 (+0.01%); split: -0.00%, +0.01%
VClause: 415400 -> 415402 (+0.00%); split: -0.00%, +0.00%
Copies: 1715404 -> 1715258 (-0.01%); split: -0.01%, +0.01%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com> (r600)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23930 >
2023-06-30 15:35:03 +00:00
Georg Lehmann
573e98f34a
aco: implement nir_op_bitz/bitnz
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23298 >
2023-06-29 13:39:30 +00:00
Rhys Perry
58f8e0e2a0
nir,aco: add INCLUDE_HELPERS index to reduce intrinsic
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23621 >
2023-06-27 18:53:50 +00:00
Rhys Perry
c5fc89019b
aco: include helpers in emit_uniform_{reduce,scan}
...
Found by inspection.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23621 >
2023-06-27 18:53:49 +00:00
Vitaliy Triang3l Kuzmin
dda425df09
aco: Implement fragment shader interlock intrinsics
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:04 +00:00