Commit graph

12986 commits

Author SHA1 Message Date
Samuel Pitoiset
c906723009 radv: always declare some arguments for non-monolithic {VS,TES}/GS shaders
When compiling VS/TES and GS separately, we can't know if the GS will
need shader queries, so we have to always declare this argument.
Similar story for the view index.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24862>
2023-08-25 10:22:41 +00:00
Samuel Pitoiset
80177e0296 aco: add support for compiling VS+TCS separately on GFX9+
The VS will just jump to the TCS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:04 +00:00
Samuel Pitoiset
196b355db6 aco: ensure to initialize exec manually for VS as LS on GFX9+
When VS and TCS are compiled separately with shader object on GFX9+.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:04 +00:00
Samuel Pitoiset
aba16211a8 aco: disable shared VGPRs for non-monolithic shaders on GFX9+
For unmerged shaders on GFX9+, we would need to jump to the second
shader part which means shared VGPRs can't be enabled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:04 +00:00
Samuel Pitoiset
4224da6726 radv: preserve shader arguments for non-monolithic VS/TCS on GFX9+
This is more robust than re-creating the function signature in ACO.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:04 +00:00
Samuel Pitoiset
c161337029 ac: allow to mark shader arguments as preserved
These arguments would be used by ACO to generate a function signature
for merged shaders automatically.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
8ba1860fbd radv: add a new shader argument for non-monolithic shaders PC
This will be used to jump from VS to TCS/GS or TES to GS on GFX9+.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
7b4f10b434 radv: always declare some arguments for non-monolithic VS/TCS shaders
For separate VS/TCS compilation on GFX9+, the TCS might be using push
constants but not the VS and we can't know this information when
compiling the VS. Similar logic for the other arguments.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
a4933d2d7f radv: force indirect descriptor sets for non-monolithic shaders
When VS and TCS are compiled separately on GFX9+, we can't know how
many descriptor sets are used for both stages and the function
arguments must match.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
e5d30462c9 radv: do not inline push constants for non-monolithic shaders
It's hard to implement this because the function arguments must match
when eg. VS or TCS are compiled separately on GFX9+.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
34ddde6d63 radv: use info->uses_view_index directly when declaring shader arguments
No need for a separate variable.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Samuel Pitoiset
467bf47281 radv: add radv_shader_info::is_monolithic
This will be used to implement shader object on GFX9+.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24697>
2023-08-25 07:22:03 +00:00
Benjamin Cheng
dd20ec5655 radv/video: send h264 scaling list in raster order
ITU spec defines the H264 ScalingList{4x4,8x8} in zig-zag order, but
AMD HW wants raster order.

Reviewed-by: Lynne <dev@lynne.ee>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24572>
2023-08-25 03:08:13 +00:00
Benjamin Cheng
d578e4416a radv/video: use vk_video_derive_h264_scaling_list
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Lynne <dev@lynne.ee>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24572>
2023-08-25 03:08:13 +00:00
Alyssa Rosenzweig
465b138f01 treewide: Use nir_shader_intrinsic_pass sometimes
This converts a lot of trivial passes. Nice boilerplate deletion. Via Coccinelle
patch (with a small manual fix-up for panfrost where coccinelle got confused by
genxml + ninja clang-format squashed in, and for Zink because my semantic patch
was slightly buggy).

    @def@
    typedef bool;
    typedef nir_builder;
    typedef nir_instr;
    typedef nir_def;
    identifier fn, instr, intr, x, builder, data;
    @@

    static fn(nir_builder* builder,
    -nir_instr *instr,
    +nir_intrinsic_instr *intr,
    ...)
    {
    (
    -   if (instr->type != nir_instr_type_intrinsic)
    -      return false;
    -   nir_intrinsic_instr *intr = nir_instr_as_intrinsic(instr);
    |
    -   nir_intrinsic_instr *intr = nir_instr_as_intrinsic(instr);
    -   if (instr->type != nir_instr_type_intrinsic)
    -      return false;
    )

    <...
    (
    -instr->x
    +intr->instr.x
    |
    -instr
    +&intr->instr
    )
    ...>

    }

    @pass depends on def@
    identifier def.fn;
    expression shader, progress;
    @@

    (
    -nir_shader_instructions_pass(shader, fn,
    +nir_shader_intrinsics_pass(shader, fn,
    ...)
    |
    -NIR_PASS_V(shader, nir_shader_instructions_pass, fn,
    +NIR_PASS_V(shader, nir_shader_intrinsics_pass, fn,
    ...)
    |
    -NIR_PASS(progress, shader, nir_shader_instructions_pass, fn,
    +NIR_PASS(progress, shader, nir_shader_intrinsics_pass, fn,
    ...)
    )

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24852>
2023-08-24 15:48:02 +00:00
Samuel Pitoiset
112b393766 radv: stop declaring unused SGPR arguments for PS epilogs
ACO no longer requires these arguments.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24838>
2023-08-24 07:21:58 +00:00
Samuel Pitoiset
0004d903d4 radv: fix the per-patch data offset when TES isn't linked with TCS
When TCS and TES aren't linked together and TCS exports unused outputs,
the per-patch data offset needs to be adjusted. This is similar to the
LS-HS vertex stride when VS and TCS aren't linked together.

This fixes a bunch of failures by forcing the driver to use TCS epilogs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24776>
2023-08-24 06:03:12 +00:00
Georg Lehmann
87d13ee73d aco: combine a | ~b to bfi(b, a, -1)
Somehow I missed this when writing the a & ~b patch.

Foz-DB Navi21:
Totals from 1591 (1.20% of 132657) affected shaders:
Instrs: 2316379 -> 2315940 (-0.02%)
CodeSize: 12524240 -> 12528724 (+0.04%); split: -0.00%, +0.04%
Latency: 45393195 -> 45389285 (-0.01%); split: -0.01%, +0.00%
InvThroughput: 8658991 -> 8657944 (-0.01%); split: -0.01%, +0.00%
Copies: 135777 -> 135778 (+0.00%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24505>
2023-08-23 20:06:49 +00:00
Georg Lehmann
6d949e18fd aco: fix u2f16 with 32bit input
The vulkan spec says all conversions are correctly rounded, so if the input
is larger than the largest fp16 value, we need to return MAX_FLOAT/inf
instead of cutting off the msbs.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24826>
2023-08-23 12:25:56 +00:00
Rhys Perry
1d29a1e2fc aco: add adjust_bpermute_dst helper
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>
2023-08-23 12:36:46 +01:00
Rhys Perry
9169fbf83c aco: clarify bpermute pseudo opcode names
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>
2023-08-23 12:36:46 +01:00
Rhys Perry
8a024c985f aco: fix p_bpermute_gfx6's exec save/restore with wave32
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>
2023-08-23 12:36:46 +01:00
Rhys Perry
85957dd6e5 aco: fix p_bpermute_gfx6 with input at non-zero byte
Same as the other bpermute pseudo instructions.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24693>
2023-08-23 12:36:46 +01:00
Samuel Pitoiset
203b4054f3 aco: rework printing shader stages
To avoid printing "unknown" for shader object when eg. VS and TCS
are compiled separately.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24810>
2023-08-23 09:21:33 +00:00
Samuel Pitoiset
aef257fd15 radv: advertise NV_device_generated_commands_compute
This extension introduces a token for implementing DGC compute, it's
only intended to be used by vkd3d-proton.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Samuel Pitoiset
1a90b7a5da radv: allow DGC on the compute queue
DGC cmdbuf on ACE are executed as IB1 without chaining because IB2
isn't supported on ACE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Samuel Pitoiset
559da06755 radv: implement NV_device_generated_commands_compute
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Samuel Pitoiset
a57fe712f7 radv: prepare radv_prepare_dgc() for DGC compute
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Samuel Pitoiset
aa0ca1e1db radv: prepare radv_get_sequence_size() for DGC compute
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Samuel Pitoiset
bb82a3402a radv: track the pipeline bind point for indirect commands layout
This will be used to implement DGC compute.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24275>
2023-08-23 06:05:39 +00:00
Konstantin Seurer
7aee3ba36d radv: Stop updating the stack_size in insert_rt_case
There are two paths that call insert_rt_case:
- Traversal shader: The stack size is ignored.
- Monolithic raygen shader: The stack sizes of the inlined shaders are
                            accounted for in compute_rt_stack_size.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
bdec044c88 aco: Do not fixup registers if there are no shader calls
Frees up some registers when using monolithic compilation.

Quake II RTX and Control (with monolithic compilation):

Totals from 10 (29.41% of 34) affected shaders:
MaxWaves: 77 -> 98 (+27.27%)
Instrs: 49047 -> 48984 (-0.13%); split: -0.16%, +0.03%
CodeSize: 260420 -> 259880 (-0.21%); split: -0.25%, +0.04%
VGPRs: 1328 -> 1104 (-16.87%)
Latency: 477134 -> 479377 (+0.47%); split: -0.05%, +0.52%
InvThroughput: 137763 -> 114108 (-17.17%)
VClause: 1318 -> 1286 (-2.43%); split: -2.66%, +0.23%
SClause: 1295 -> 1293 (-0.15%); split: -0.54%, +0.39%
Copies: 7838 -> 7782 (-0.71%); split: -0.82%, +0.10%
Branches: 2592 -> 2589 (-0.12%)
PreSGPRs: 874 -> 796 (-8.92%)
PreVGPRs: 1283 -> 1013 (-21.04%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
ec708c26ef radv/rt: Split stage initialization and hashing
The dependency chain is: init stages -> compute pipeline key -> hash
stages.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
f3e2900c59 radv/rt: Insert rt_return_amd before lowering shader calls
Also skips running nir_lower_shader_calls for the traversal shader. This
will be used to skip the pass and the rt_return_amd insertion for
monolithic raygen shaders.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
774421f11e radv/rt: Add and use radv_build_traversal
Moves most of the build code to a helper which will be useful for adding
inline traversal.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
2d7965dbff radv/rt: Do not apply stack_ptr for non-recursive stages
stack_ptr is set to 0.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Konstantin Seurer
d174a71db8 radv/rt: Remove some dead code
- call_idx_base was used for resume shaders in the shader call loop
- hit attribs are lowered elsewhere
- stack_size is set in radv_pipeline_rt.c

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24809>
2023-08-22 15:46:29 +00:00
Georg Lehmann
9cf6984200 nir: unify lower_find_msb with has_{find_msb_rev,uclz}
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24662>
2023-08-22 12:08:37 +00:00
Georg Lehmann
2ac7e6614a nir: unify lower_bitfield_extract with has_bfe
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24662>
2023-08-22 12:08:37 +00:00
Georg Lehmann
34c3f81614 nir: unify lower_bitfield_insert with has_{bfm,bfi,bitfield_select}
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24662>
2023-08-22 12:08:37 +00:00
Friedrich Vock
bfb55d0266 ac/sqtt,radv/sqtt: Add and use marker for separate RT compilation
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24371>
2023-08-22 11:33:11 +00:00
Friedrich Vock
3d3d5c4bc3 radv/sqtt: Handle separately-compiled RT pipelines
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24371>
2023-08-22 11:33:11 +00:00
Friedrich Vock
1cd9525b18 radv/sqtt: Write LDS size metadata in code objects
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24371>
2023-08-22 11:33:11 +00:00
Friedrich Vock
7809fb9e49 radv/sqtt: Unregister records based on hash
RT pipelines have multiple hashes used in records, so don't always use
the pipeline hash.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24371>
2023-08-22 11:33:11 +00:00
Friedrich Vock
3ed4cca883 radv/sqtt: Move record filling to helper function
RT shaders construct records differently, but this piece of code is
common to all types of pipelines.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24371>
2023-08-22 11:33:11 +00:00
Friedrich Vock
b4a704b42a ac/rgp: Add metadata for separate-compiled RT stages
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24371>
2023-08-22 11:33:11 +00:00
Friedrich Vock
0c4e92bf3e ac/rgp: Write lds_size metadata
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24371>
2023-08-22 11:33:11 +00:00
Friedrich Vock
be0e3e8e09 ac/sqtt,radv: Split internal and API hash in PSO correlations
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24371>
2023-08-22 11:33:11 +00:00
Friedrich Vock
d5f1c9fb4b ac/msgpack: make fixstrs a const char
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24371>
2023-08-22 11:33:10 +00:00
Samuel Pitoiset
a29e2c6fbc aco: implement create_tcs_jump_to_epilog()
This implements jumping from the main TCS to the epilog.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24643>
2023-08-22 06:10:32 +00:00