Commit graph

12996 commits

Author SHA1 Message Date
Rhys Perry
cb096b85ff aco/spill: skip p_branch in process_block
Fixes compilation of a Dead by Daylight shader.

fossil-db (gfx1100):
Totals from 58 (0.04% of 133461) affected shaders:
Instrs: 319824 -> 319421 (-0.13%); split: -0.13%, +0.00%
CodeSize: 1711260 -> 1708744 (-0.15%); split: -0.15%, +0.00%
SpillSGPRs: 2567 -> 2459 (-4.21%)
Latency: 3274930 -> 3274921 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 924106 -> 924105 (-0.00%); split: -0.00%, +0.00%
Copies: 41883 -> 41757 (-0.30%); split: -0.31%, +0.00%
Branches: 9144 -> 9146 (+0.02%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9599
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24896>
2023-08-29 11:53:57 +00:00
Samuel Pitoiset
f30d47c518 aco: fix emitting TCS epilogs end on GFX9+
With merged shaders, the long-jump should be emitted inside the
divergent if (ie. only for TCS invocations) and other non TCS
invocations should just end the program.

This fixes a bunch of failures with CTS by forcing TCS epilogs on
RDNA2.

Not sure how RadeonSI will handle that but maybe doing the merged
wave info thing in epilogs would help.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24832>
2023-08-29 09:30:07 +00:00
Samuel Pitoiset
6463e60fe4 radv: remove the pipeline dependency for emitting VGT_GS_MODE
For shader object.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24865>
2023-08-29 09:10:18 +00:00
Samuel Pitoiset
3f517ff947 radv: fix emitting TCS epilogs if TES and GS are linked on GFX9+
TES would be NULL because everything is merged to GS.
Found by inspection.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24890>
2023-08-29 07:09:51 +00:00
Samuel Pitoiset
01ecaca188 radv: small cleanups in radv_emit_patch_control_points()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24890>
2023-08-29 07:09:51 +00:00
Samuel Pitoiset
9314d5c6df radv: rename tcs_shader to tcs in radv_emit_tcs_epilog_state()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24890>
2023-08-29 07:09:51 +00:00
Samuel Pitoiset
43fe842b92 radv,aco: remove unused clip/cull distances variables
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24883>
2023-08-25 11:13:26 +00:00
Samuel Pitoiset
8ebb29245c aco: add support for compiling {VS,TES}+GS separately on GFX9+
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24862>
2023-08-25 10:22:41 +00:00
Samuel Pitoiset
37aa6d25e1 aco: ensure to initialize exec manually for non-monolithic {VS,TES}/GS on GFX9+
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24862>
2023-08-25 10:22:41 +00:00
Samuel Pitoiset
92015fe3dc radv: preserve shader arguments for non-monolithic {VS,TES}/GS on GFX9+
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24862>
2023-08-25 10:22:41 +00:00
Samuel Pitoiset
c906723009 radv: always declare some arguments for non-monolithic {VS,TES}/GS shaders
When compiling VS/TES and GS separately, we can't know if the GS will
need shader queries, so we have to always declare this argument.
Similar story for the view index.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24862>
2023-08-25 10:22:41 +00:00
Samuel Pitoiset
80177e0296 aco: add support for compiling VS+TCS separately on GFX9+
The VS will just jump to the TCS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:04 +00:00
Samuel Pitoiset
196b355db6 aco: ensure to initialize exec manually for VS as LS on GFX9+
When VS and TCS are compiled separately with shader object on GFX9+.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:04 +00:00
Samuel Pitoiset
aba16211a8 aco: disable shared VGPRs for non-monolithic shaders on GFX9+
For unmerged shaders on GFX9+, we would need to jump to the second
shader part which means shared VGPRs can't be enabled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:04 +00:00
Samuel Pitoiset
4224da6726 radv: preserve shader arguments for non-monolithic VS/TCS on GFX9+
This is more robust than re-creating the function signature in ACO.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:04 +00:00
Samuel Pitoiset
c161337029 ac: allow to mark shader arguments as preserved
These arguments would be used by ACO to generate a function signature
for merged shaders automatically.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
8ba1860fbd radv: add a new shader argument for non-monolithic shaders PC
This will be used to jump from VS to TCS/GS or TES to GS on GFX9+.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
7b4f10b434 radv: always declare some arguments for non-monolithic VS/TCS shaders
For separate VS/TCS compilation on GFX9+, the TCS might be using push
constants but not the VS and we can't know this information when
compiling the VS. Similar logic for the other arguments.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
a4933d2d7f radv: force indirect descriptor sets for non-monolithic shaders
When VS and TCS are compiled separately on GFX9+, we can't know how
many descriptor sets are used for both stages and the function
arguments must match.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
e5d30462c9 radv: do not inline push constants for non-monolithic shaders
It's hard to implement this because the function arguments must match
when eg. VS or TCS are compiled separately on GFX9+.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
34ddde6d63 radv: use info->uses_view_index directly when declaring shader arguments
No need for a separate variable.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
467bf47281 radv: add radv_shader_info::is_monolithic
This will be used to implement shader object on GFX9+.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Benjamin Cheng
dd20ec5655 radv/video: send h264 scaling list in raster order
ITU spec defines the H264 ScalingList{4x4,8x8} in zig-zag order, but
AMD HW wants raster order.

Reviewed-by: Lynne <dev@lynne.ee>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24572>
2023-08-25 03:08:13 +00:00
Benjamin Cheng
d578e4416a radv/video: use vk_video_derive_h264_scaling_list
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Lynne <dev@lynne.ee>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24572>
2023-08-25 03:08:13 +00:00
Alyssa Rosenzweig
465b138f01 treewide: Use nir_shader_intrinsic_pass sometimes
This converts a lot of trivial passes. Nice boilerplate deletion. Via Coccinelle
patch (with a small manual fix-up for panfrost where coccinelle got confused by
genxml + ninja clang-format squashed in, and for Zink because my semantic patch
was slightly buggy).

    @def@
    typedef bool;
    typedef nir_builder;
    typedef nir_instr;
    typedef nir_def;
    identifier fn, instr, intr, x, builder, data;
    @@

    static fn(nir_builder* builder,
    -nir_instr *instr,
    +nir_intrinsic_instr *intr,
    ...)
    {
    (
    -   if (instr->type != nir_instr_type_intrinsic)
    -      return false;
    -   nir_intrinsic_instr *intr = nir_instr_as_intrinsic(instr);
    |
    -   nir_intrinsic_instr *intr = nir_instr_as_intrinsic(instr);
    -   if (instr->type != nir_instr_type_intrinsic)
    -      return false;
    )

    <...
    (
    -instr->x
    +intr->instr.x
    |
    -instr
    +&intr->instr
    )
    ...>

    }

    @pass depends on def@
    identifier def.fn;
    expression shader, progress;
    @@

    (
    -nir_shader_instructions_pass(shader, fn,
    +nir_shader_intrinsics_pass(shader, fn,
    ...)
    |
    -NIR_PASS_V(shader, nir_shader_instructions_pass, fn,
    +NIR_PASS_V(shader, nir_shader_intrinsics_pass, fn,
    ...)
    |
    -NIR_PASS(progress, shader, nir_shader_instructions_pass, fn,
    +NIR_PASS(progress, shader, nir_shader_intrinsics_pass, fn,
    ...)
    )

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24852>
2023-08-24 15:48:02 +00:00
Samuel Pitoiset
112b393766 radv: stop declaring unused SGPR arguments for PS epilogs
ACO no longer requires these arguments.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24838>
2023-08-24 07:21:58 +00:00
Samuel Pitoiset
0004d903d4 radv: fix the per-patch data offset when TES isn't linked with TCS
When TCS and TES aren't linked together and TCS exports unused outputs,
the per-patch data offset needs to be adjusted. This is similar to the
LS-HS vertex stride when VS and TCS aren't linked together.

This fixes a bunch of failures by forcing the driver to use TCS epilogs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24776>
2023-08-24 06:03:12 +00:00
Georg Lehmann
87d13ee73d aco: combine a | ~b to bfi(b, a, -1)
Somehow I missed this when writing the a & ~b patch.

Foz-DB Navi21:
Totals from 1591 (1.20% of 132657) affected shaders:
Instrs: 2316379 -> 2315940 (-0.02%)
CodeSize: 12524240 -> 12528724 (+0.04%); split: -0.00%, +0.04%
Latency: 45393195 -> 45389285 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 8658991 -> 8657944 (-0.01%); split: -0.01%, +0.00%
Copies: 135777 -> 135778 (+0.00%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24505>
2023-08-23 20:06:49 +00:00
Georg Lehmann
6d949e18fd aco: fix u2f16 with 32bit input
The vulkan spec says all conversions are correctly rounded, so if the input
is larger than the largest fp16 value, we need to return MAX_FLOAT/inf
instead of cutting off the msbs.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24826>
2023-08-23 12:25:56 +00:00
Rhys Perry
1d29a1e2fc aco: add adjust_bpermute_dst helper
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>
2023-08-23 12:36:46 +01:00
Rhys Perry
9169fbf83c aco: clarify bpermute pseudo opcode names
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>
2023-08-23 12:36:46 +01:00
Rhys Perry
8a024c985f aco: fix p_bpermute_gfx6's exec save/restore with wave32
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>
2023-08-23 12:36:46 +01:00
Rhys Perry
85957dd6e5 aco: fix p_bpermute_gfx6 with input at non-zero byte
Same as the other bpermute pseudo instructions.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>
2023-08-23 12:36:46 +01:00
Samuel Pitoiset
203b4054f3 aco: rework printing shader stages
To avoid printing "unknown" for shader object when eg. VS and TCS
are compiled separately.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24810>
2023-08-23 09:21:33 +00:00
Samuel Pitoiset
aef257fd15 radv: advertise NV_device_generated_commands_compute
This extension introduces a token for implementing DGC compute, it's
only intended to be used by vkd3d-proton.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Samuel Pitoiset
1a90b7a5da radv: allow DGC on the compute queue
DGC cmdbuf on ACE are executed as IB1 without chaining because IB2
isn't supported on ACE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Samuel Pitoiset
559da06755 radv: implement NV_device_generated_commands_compute
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Samuel Pitoiset
a57fe712f7 radv: prepare radv_prepare_dgc() for DGC compute
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Samuel Pitoiset
aa0ca1e1db radv: prepare radv_get_sequence_size() for DGC compute
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Samuel Pitoiset
bb82a3402a radv: track the pipeline bind point for indirect commands layout
This will be used to implement DGC compute.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Konstantin Seurer
7aee3ba36d radv: Stop updating the stack_size in insert_rt_case
There are two paths that call insert_rt_case:
- Traversal shader: The stack size is ignored.
- Monolithic raygen shader: The stack sizes of the inlined shaders are
                            accounted for in compute_rt_stack_size.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
bdec044c88 aco: Do not fixup registers if there are no shader calls
Frees up some registers when using monolithic compilation.

Quake II RTX and Control (with monolithic compilation):

Totals from 10 (29.41% of 34) affected shaders:
MaxWaves: 77 -> 98 (+27.27%)
Instrs: 49047 -> 48984 (-0.13%); split: -0.16%, +0.03%
CodeSize: 260420 -> 259880 (-0.21%); split: -0.25%, +0.04%
VGPRs: 1328 -> 1104 (-16.87%)
Latency: 477134 -> 479377 (+0.47%); split: -0.05%, +0.52%
InvThroughput: 137763 -> 114108 (-17.17%)
VClause: 1318 -> 1286 (-2.43%); split: -2.66%, +0.23%
SClause: 1295 -> 1293 (-0.15%); split: -0.54%, +0.39%
Copies: 7838 -> 7782 (-0.71%); split: -0.82%, +0.10%
Branches: 2592 -> 2589 (-0.12%)
PreSGPRs: 874 -> 796 (-8.92%)
PreVGPRs: 1283 -> 1013 (-21.04%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
ec708c26ef radv/rt: Split stage initialization and hashing
The dependency chain is: init stages -> compute pipeline key -> hash
stages.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
f3e2900c59 radv/rt: Insert rt_return_amd before lowering shader calls
Also skips running nir_lower_shader_calls for the traversal shader. This
will be used to skip the pass and the rt_return_amd insertion for
monolithic raygen shaders.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
774421f11e radv/rt: Add and use radv_build_traversal
Moves most of the build code to a helper which will be useful for adding
inline traversal.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
2d7965dbff radv/rt: Do not apply stack_ptr for non-recursive stages
stack_ptr is set to 0.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
d174a71db8 radv/rt: Remove some dead code
- call_idx_base was used for resume shaders in the shader call loop
- hit attribs are lowered elsewhere
- stack_size is set in radv_pipeline_rt.c

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Georg Lehmann
9cf6984200 nir: unify lower_find_msb with has_{find_msb_rev,uclz}
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24662>
2023-08-22 12:08:37 +00:00
Georg Lehmann
2ac7e6614a nir: unify lower_bitfield_extract with has_bfe
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24662>
2023-08-22 12:08:37 +00:00
Georg Lehmann
34c3f81614 nir: unify lower_bitfield_insert with has_{bfm,bfi,bitfield_select}
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24662>
2023-08-22 12:08:37 +00:00