Qiang Yu
599b50b448
aco,radv: replace tess_input_vertices shader info param
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24204 >
2023-07-24 01:49:20 +00:00
Bas Nieuwenhuizen
e536d31a46
aco: Fix some constant patterns in 16-bit vec4 construction with s_pack.
...
Fixes: 04e3d7ad93 ("aco: improve nir_op_vec with constant operands")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24286 >
2023-07-22 22:11:15 +00:00
Bas Nieuwenhuizen
2fcf7c7014
aco: fix nir_op_vec8/16 with 16-bit elements.
...
Fixes: 5718347c2b ("aco: implement vec2/3/4 with subdword operands")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24286 >
2023-07-22 22:11:15 +00:00
Timur Kristóf
9fb9e54d69
aco: Remove subgroup_id and num_subgroups intrinsics.
...
These are lowered in NIR now.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24005 >
2023-07-14 12:17:24 +00:00
Timur Kristóf
dc3bbd351a
aco: Fix subgroup_id intrinsic on GFX10.3+.
...
Change this to match how it works in the LLVM backend.
Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24005 >
2023-07-14 12:17:24 +00:00
Rhys Perry
ae48fae658
aco: remove 64-bit integer conversion opcodes
...
These are handled by nir_lower_int64 now.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23926 >
2023-07-03 10:38:28 +00:00
Rhys Perry
25c49e491f
aco,ac/llvm,ac/nir,vtn: unify cube opcodes
...
fossil-db (navi21):
Totals from 17068 (12.79% of 133461) affected shaders:
Instrs: 24743703 -> 24743572 (-0.00%); split: -0.00%, +0.00%
CodeSize: 132579952 -> 132580620 (+0.00%); split: -0.00%, +0.00%
VGPRs: 1227840 -> 1227984 (+0.01%)
Latency: 403180114 -> 403251188 (+0.02%); split: -0.00%, +0.02%
InvThroughput: 75311302 -> 75320892 (+0.01%); split: -0.00%, +0.01%
VClause: 415400 -> 415402 (+0.00%); split: -0.00%, +0.00%
Copies: 1715404 -> 1715258 (-0.01%); split: -0.01%, +0.01%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com> (r600)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23930 >
2023-06-30 15:35:03 +00:00
Georg Lehmann
573e98f34a
aco: implement nir_op_bitz/bitnz
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23298 >
2023-06-29 13:39:30 +00:00
Rhys Perry
58f8e0e2a0
nir,aco: add INCLUDE_HELPERS index to reduce intrinsic
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23621 >
2023-06-27 18:53:50 +00:00
Rhys Perry
c5fc89019b
aco: include helpers in emit_uniform_{reduce,scan}
...
Found by inspection.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23621 >
2023-06-27 18:53:49 +00:00
Vitaliy Triang3l Kuzmin
dda425df09
aco: Implement fragment shader interlock intrinsics
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:04 +00:00
Vitaliy Triang3l Kuzmin
e0f4b52559
aco: Add Primitive Ordered Pixel Shading waitcnt rules
...
When letting the overlapping waves enter their ordered sections, there must
be no memory accesses to resources which need primitive-ordered access that
are still pending, or there would be a race between the current wave and
the overlapping waves.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Vitaliy Triang3l Kuzmin <triang3l@yandex.ru>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22250 >
2023-06-26 15:58:04 +00:00
Timur Kristóf
05928f4200
aco: Use ac_hw_stage instead of aco-specific HWStage.
...
The new ac_hw_stage is going to be used by drivers as well.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23597 >
2023-06-23 12:49:04 +00:00
Caio Oliveira
59cc77f0fa
compiler: Move from nir_scope to mesa_scope
...
Just moving the enum and performing renames, no behavior change.
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23328 >
2023-06-19 23:29:26 +00:00
Eric Engestrom
6b21653ab4
aco: reformat according to its .clang-format
...
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23253 >
2023-06-16 19:59:52 +00:00
Konstantin Seurer
51cd2965c7
aco/rt: Do not initialize the next shader addr
...
The uniform one is already set and the raygen shader isn't guarded
anymore.
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23545 >
2023-06-10 10:00:27 +00:00
Rhys Perry
08064a5542
aco: mask bits source of s_bfe
...
The s_bfe instructions use 7 bits, not 5 like the NIR opcode requires.
No fossil-db changes (navi21).
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9162
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23464 >
2023-06-08 11:54:17 +01:00
Yonggang Luo
586391720b
util: use uint32_t as the parameter of align function
...
align on negative value doesn't make sense
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23421 >
2023-06-08 06:41:21 +00:00
Daniel Schürmann
defdcd2058
aco: adjust RT prolog for shader functions [disables RT]
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22096 >
2023-06-08 00:37:03 +00:00
Daniel Schürmann
b33c01b00f
radv/rt: set up RT shader args for separate compilation
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22096 >
2023-06-08 00:37:03 +00:00
Daniel Schürmann
393f3426b6
aco: implement select_program_rt()
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22096 >
2023-06-08 00:37:03 +00:00
Daniel Schürmann
f66f274304
aco: implement nir_intrinsic_load_resume_shader_address_amd
...
Similar to p_constaddr but targeting BBs.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22096 >
2023-06-08 00:37:03 +00:00
Daniel Schürmann
03c4b5b0cc
nir,amd: add nir_intrinsic_store_[scalar|vector]_arg_amd to overwrite inputs
...
This intrinsic must only be used at top-level CF in order
to not break SSA properties.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22096 >
2023-06-08 00:37:03 +00:00
Daniel Schürmann
1be3a558f2
radv: add remaining RT shader args for separate compilation
...
Also wrap RT args into struct {} rt for improved consistency
and remove some 'ray_' prefixes.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22096 >
2023-06-08 00:37:03 +00:00
Rhys Perry
b5c31f14a2
aco: remove memory_barrier_buffer implementation
...
This is no longer used.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21624 >
2023-06-07 13:19:42 +00:00
Rhys Perry
21867b45c1
aco: fix has_color_exports=true for mrtz exports
...
V_008DFC_SQ_EXP_NULL is after V_008DFC_SQ_EXP_MRTZ.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Tested-by: Mikhail Gavrilov mikhail.v.gavrilov@gmail.com
Fixes: d3611af389 ("aco: support nir_export_amd with ps targets")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9135
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23371 >
2023-06-05 11:42:21 +00:00
Qiang Yu
2d0e8e0258
aco: use ac_get_image_dim for array check when image intrinsic
...
This is to avoid missing array flag when <=GFX8 and 3D image
in which case is treated as 2D array image.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23094 >
2023-06-02 09:21:59 +00:00
Qiang Yu
ed97cd92dc
aco: implement nir_bindless_image_fragment_mask_load_amd
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Sigend-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23094 >
2023-06-02 09:21:59 +00:00
Qiang Yu
50f9d644e8
aco: implement nir_xfb_counter_sub_amd
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23094 >
2023-06-02 09:21:59 +00:00
Qiang Yu
8d5cc23c18
aco: use gds reg when ordered xfb counter add
...
This is currently only used by radeonsi.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23094 >
2023-06-02 09:21:59 +00:00
Qiang Yu
460b528c9e
aco: implement load buffer with ACCESS_USES_FORMAT_AMD
...
This is used by radeonsi for vs input load and cdna
image load emulation.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23094 >
2023-06-02 09:21:59 +00:00
Qiang Yu
89dab66561
aco: implement two load lds ngg intrininsic for radeonsi
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23094 >
2023-06-02 09:21:58 +00:00
Georg Lehmann
7836260af8
aco: cleanup v_cmp_class usage
...
It's not the best way to check for NaN.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23310 >
2023-05-31 12:56:37 +00:00
Samuel Pitoiset
1947500208
aco: remove nir_intrinsic_load_barycentric_at_sample occurences
...
This is lowered earlier and shouldn't get there.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23307 >
2023-05-31 07:25:46 +00:00
Chia-I Wu
750d641ca6
aco: fix alignment check in emit_load
...
align_offset already takes const_offset into consideration. Remove the
adjustment.
Fixes: 542733dbbf ("aco: add emit_load helper")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9097
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23242 >
2023-05-30 16:02:34 +00:00
Rhys Perry
364ad2815b
aco: implement texture samples with strict WQM coordinates
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22636 >
2023-05-25 16:29:16 +00:00
Rhys Perry
2447156b3c
aco: implement strict_wqm_coord_amd
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22636 >
2023-05-25 16:29:16 +00:00
Rhys Perry
35c133a77b
aco: add MIMG_instruction::strict_wqm
...
This lets us use linear VGPRs for part of the texture sample's address.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22636 >
2023-05-25 16:29:16 +00:00
Rhys Perry
09785e5e1b
radv,radeonsi: use ac_nir_lower_tex
...
fossil-db (navi21):
Totals from 17279 (12.74% of 135636) affected shaders:
MaxWaves: 270015 -> 269991 (-0.01%)
Instrs: 24847385 -> 24843807 (-0.01%); split: -0.02%, +0.00%
CodeSize: 133215364 -> 133198744 (-0.01%); split: -0.02%, +0.01%
VGPRs: 1217632 -> 1217872 (+0.02%); split: -0.00%, +0.02%
Latency: 405347021 -> 404971784 (-0.09%); split: -0.09%, +0.00%
InvThroughput: 75386590 -> 75350344 (-0.05%); split: -0.07%, +0.03%
VClause: 426986 -> 426821 (-0.04%); split: -0.04%, +0.01%
SClause: 966751 -> 966971 (+0.02%); split: -0.01%, +0.03%
Copies: 1738510 -> 1737970 (-0.03%); split: -0.08%, +0.05%
PreSGPRs: 1169070 -> 1169120 (+0.00%); split: -0.00%, +0.00%
PreVGPRs: 1136102 -> 1136183 (+0.01%); split: -0.00%, +0.01%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22636 >
2023-05-25 16:29:16 +00:00
Rhys Perry
1ba73621bc
nir,vtn,aco,ac/llvm: make cube_face_coord_amd more direct
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22636 >
2023-05-25 16:29:16 +00:00
Georg Lehmann
b4c4d28263
aco: emit_wqm on MIMG dst, not operands
...
Now p_wqm always kills its operand, so no movs will be created for it.
Long term we want to remove p_wqm in favor of a Definition flag,
so this is also a step in that direction.
Foz-DB Navi21:
Totals from 45351 (33.63% of 134864) affected shaders:
VGPRs: 2099552 -> 2116192 (+0.79%); split: -0.14%, +0.93%
CodeSize: 179530772 -> 179072104 (-0.26%); split: -0.29%, +0.03%
MaxWaves: 1054740 -> 1052262 (-0.23%); split: +0.10%, -0.33%
Instrs: 33238535 -> 33188347 (-0.15%); split: -0.17%, +0.02%
Latency: 451000471 -> 450869384 (-0.03%); split: -0.11%, +0.08%
InvThroughput: 86026785 -> 86286288 (+0.30%); split: -0.11%, +0.41%
VClause: 633291 -> 623920 (-1.48%); split: -1.91%, +0.43%
SClause: 1436708 -> 1431395 (-0.37%); split: -0.60%, +0.23%
Copies: 2166563 -> 2122592 (-2.03%); split: -2.29%, +0.26%
Branches: 706846 -> 706838 (-0.00%); split: -0.00%, +0.00%
PreSGPRs: 1976162 -> 1976592 (+0.02%)
PreVGPRs: 1797409 -> 1794704 (-0.15%)
MaxWaves regressions in Detroit: Become Human MaxWaves seem to be due
to the scheduler choosing to schedule more aggressively.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22956 >
2023-05-16 19:57:29 +00:00
Marek Olšák
3a043b5739
aco: don't treat ACCESS_NON_READABLE as ACCESS_COHERENT
...
... and expect it to behave like ACCESS_NON_TEMPORAL.
Handling ACCESS_NON_TEMPORAL is sufficient.
This was copied from ac_nir_to_llvm.c.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22770 >
2023-05-12 21:45:44 +00:00
Alyssa Rosenzweig
ce638eafe2
aco,radv: Use unified atomics
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914 >
2023-05-12 20:39:46 +00:00
Timur Kristóf
9b6945bb65
amd: Cleanup old GS intrinsics code.
...
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22690 >
2023-05-04 19:08:59 +00:00
Timur Kristóf
025c1f5174
ac/nir: Emit legacy GS DONE signal in NIR.
...
Legacy GS needs to emit a DONE signal at the end. Do this in NIR
instead of in the backends.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22690 >
2023-05-04 19:08:59 +00:00
Timur Kristóf
f66281c7fb
amd: Add and implement gs_wave_id sysval.
...
Contains a global wave ID of legacy GS waves.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22690 >
2023-05-04 19:08:58 +00:00
Timur Kristóf
c1591bfc28
amd: Add and implement sendmsg_amd intrinsic.
...
This intrinsic is going to be used for simplifying GS code.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22690 >
2023-05-04 19:08:58 +00:00
Marek Olšák
820c50ada3
nir: rename ACCESS_STREAM_CACHE_POLICY -> ACCESS_NON_TEMPORAL and document
...
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22769 >
2023-05-04 01:55:22 +00:00
Qiang Yu
3c59df7318
aco: get scratch addr from symbol for radeonsi
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22727 >
2023-04-28 11:33:28 +08:00
Qiang Yu
5c287290d8
aco,radv: remove unused aco compile options
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22199 >
2023-04-26 03:27:26 +00:00