Commit graph

13547 commits

Author SHA1 Message Date
Samuel Pitoiset
f4ec2e7bb3 radv,aco: move has_epilog to radv_shader_info
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24404>
2023-08-02 16:59:18 +00:00
Samuel Pitoiset
f433d39935 aco: add infra for compiling TCS epilogs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24417>
2023-08-02 07:29:50 +00:00
Qiang Yu
572625ea6c aco: extract aco_compile_shader_part from aco_compile_ps_epilog
Will be shared with radeonsi tcs epilog and other shader parts build.

Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24417>
2023-08-02 07:29:50 +00:00
Samuel Pitoiset
ac99fbe591 aco: add aco_shader_info::tcs::has_epilog
This will be used by both RADV and RadeonSI to jump from the main TCS
to the epilog.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24417>
2023-08-02 07:29:50 +00:00
Samuel Pitoiset
ac40924a3b radv: allow to use fixed IO locations for VS<->TCS<->TES without linking
For shader objects.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24408>
2023-08-02 06:54:09 +00:00
Samuel Pitoiset
ec1e11ab23 amd,radeonsi: move si_shader_io_get_unique_index_patch() to common code
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24408>
2023-08-02 06:54:09 +00:00
Alyssa Rosenzweig
51db19f7a2 nir: Rename scoped_barrier -> barrier
sed + ninja clang-format + fix up spacing for common code.

If you are unhappy that I did not manually change the whitespace of your driver,
you need to enable clang-format for it so the formatting would happen
automatically.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24428>
2023-08-01 23:18:29 +00:00
David Rosca
ad9d39a5ba ci/amd: Skip VAAPI CreateSurfacesWithConfigAttribs/1121 test
It now times out with surfaces being allocated non-interlaced
by default due to a slower path being taken in clear_render_target.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24361>
2023-08-01 21:58:27 +00:00
Mike Blumenkrantz
4d8c53b070 radv: bump max xfb output to 128
this is the number of components supported for streamout

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24285>
2023-08-01 20:47:34 +00:00
Georg Lehmann
82920c99a5 aco: fix non constant 16bit bitnz/bitz
Fixes: 573e98f34a ("aco: implement nir_op_bitz/bitnz")
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24420>
2023-08-01 15:54:31 +00:00
Danylo Piliaiev
261df5fb3c radv: Use common nir_vk_is_not_xfb_output
Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24318>
2023-08-01 10:32:02 +02:00
Danylo Piliaiev
81407797b9 radv: fix unused non-xfb shader outputs not being removed
It was not taken into account that without Offset decoration
the output is not written into XFB.

Aside from eliminating more outputs this change prevents gl_PerVertex
builtins generated by glslang from being kept alive in case when XFB
is enabled. Keeping such outputs alive may upset a driver.

VUID-StandaloneSpirv-Offset-04716:
    "Only variables or block members in the output interface decorated
    with Offset can be captured for transform feedback, and those
    variables or block members must also be decorated with XfbBuffer
    and XfbStride, or inherit XfbBuffer and XfbStride decorations from
    a block containing them"

Additional info about glslang behavior could be found at:
 https://github.com/KhronosGroup/glslang/issues/1526

Fixes: e95531e101
("radv: fix gathering XFB info if there is dead outputs")

Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24318>
2023-08-01 10:30:59 +02:00
Georg Lehmann
1e8d445bcf aco/gfx11: use v_cmp_class_f16 with opsel for bitnz/bitz
Foz-DB GFX11:
Totals from 1607 (1.21% of 132657) affected shaders:
MaxWaves: 36511 -> 36513 (+0.01%)
Instrs: 2412601 -> 2408770 (-0.16%); split: -0.16%, +0.00%
CodeSize: 12434716 -> 12414536 (-0.16%); split: -0.17%, +0.00%
VGPRs: 108872 -> 108860 (-0.01%)
Latency: 27555570 -> 27541897 (-0.05%); split: -0.05%, +0.00%
InvThroughput: 3144382 -> 3140929 (-0.11%); split: -0.11%, +0.00%
SClause: 65762 -> 65770 (+0.01%)
Copies: 127148 -> 127170 (+0.02%); split: -0.06%, +0.08%
PreVGPRs: 96737 -> 96447 (-0.30%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24380>
2023-08-01 07:52:42 +00:00
Georg Lehmann
1659d982c3 aco: combine a & ~b to bfi(b, 0, a)
Foz-DB Navi21:
Totals from 905 (0.68% of 132657) affected shaders:
Instrs: 1223583 -> 1221016 (-0.21%); split: -0.22%, +0.01%
CodeSize: 6567272 -> 6567064 (-0.00%); split: -0.04%, +0.03%
SpillSGPRs: 1231 -> 1223 (-0.65%)
SpillVGPRs: 829 -> 823 (-0.72%); split: -1.45%, +0.72%
Latency: 40952209 -> 40946230 (-0.01%); split: -0.02%, +0.01%
InvThroughput: 9411929 -> 9397932 (-0.15%); split: -0.17%, +0.02%
VClause: 29108 -> 29112 (+0.01%); split: -0.04%, +0.05%
Copies: 105272 -> 105221 (-0.05%); split: -0.28%, +0.23%
Branches: 29330 -> 29329 (-0.00%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24383>
2023-08-01 07:15:28 +00:00
Rhys Perry
76232d6724 radv: correctly skip MRT output NaN fixup for meta shaders
radv_nir_compiler_options::enable_mrt_output_nan_fixup is only used for
epilogs, these days.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: 290c3d360e ("aco,radv: lower outputs to exports when nir for monolithic ps")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9414
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24352>
2023-07-31 17:31:00 +00:00
Rhys Perry
e88c077096 radv: workaround WWZ exporting index=1 through location=1
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Fixes: 290c3d360e ("aco,radv: lower outputs to exports when nir for monolithic ps")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9232
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24354>
2023-07-31 11:38:14 +00:00
Timur Kristóf
be11fee2a7 aco: Refactor select_program to smaller functions.
This prepares for allowing to compile 1 shader at a time
for merged shader stages.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23874>
2023-07-28 14:27:29 +00:00
Samuel Pitoiset
0d75fc8e42 radv: remove radv_shader_info::tes::num_linked_patch_inputs
It's never used.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24346>
2023-07-28 08:21:40 +00:00
Samuel Pitoiset
f04577b789 radv: add support for dynamic TCS vertices out for TES
With shader object, if TES is compiled without a TCS, the number of
TCS vertices out might not be known at compile time and it needs to be
loaded from a user SGPR.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24346>
2023-07-28 08:21:40 +00:00
Samuel Pitoiset
faa756b3ba radv: copy the number of TCS vertices out to TES shader info
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24346>
2023-07-28 08:21:40 +00:00
Samuel Pitoiset
e855c7dd3d radv: stop checking if patch control points is dynamic everywhere
Check that the values are non-zero instead.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24346>
2023-07-28 08:21:40 +00:00
Samuel Pitoiset
a50cec9e18 radv: use a packed user SGPR for the TES state
It only contains the number of tessellation patches for now, but it
will be used to pass the number of TCS vertices out for shader object.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24346>
2023-07-28 08:21:40 +00:00
Samuel Pitoiset
7ee74881d9 radv: prevent linking TCS<->TES when TES is NULL
This can happen with shader object.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24346>
2023-07-28 08:21:40 +00:00
Samuel Pitoiset
54414a2121 radv: initialize tcs.tes_{patch}_inputs_read to a default value
For shader object when a TCS is not linked to a TES at compile time.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24346>
2023-07-28 08:21:40 +00:00
Samuel Pitoiset
afa5b83152 radv: do not always copy the number of tess patches to TES
This is only needed when the number of patch control points is known
at compile time. Adding a check makes it less confusing.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24346>
2023-07-28 08:21:40 +00:00
Samuel Pitoiset
f1b98f32c4 radv: stop lowering patch vertices for TES
This intrinsic is replaced during ABI lowering later.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24346>
2023-07-28 08:21:40 +00:00
Samuel Pitoiset
40a693e056 radv: stop copying some NIR info fields from TES to TCS
They aren't used, only TES needs to know them.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24346>
2023-07-28 08:21:40 +00:00
Samuel Pitoiset
76cc85ebb9 radv: compute the legacy GS info earlier
This allows geometry shaders to work with shader object on GFX6-8
because the workgroup size is the wave size. We will need different
tweaks for NGG but that's for later.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24333>
2023-07-28 08:55:48 +02:00
Samuel Pitoiset
329907178e radv: use next stage to determine if primID/clip dist should be exported
More shader object friendly.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24333>
2023-07-28 08:55:12 +02:00
Samuel Pitoiset
48fc29e075 radv: use next_stage to determine if the layer should be exported
More shader object friendly.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24333>
2023-07-28 08:55:12 +02:00
Samuel Pitoiset
8ab8145aa1 radv: add support for VS/TES as ES without shaders IO linking
This implements fixed IO location for VS/TES with GS. This is currently
unused because everything is linked with GPL or monolithic pipelines,
but this will be used for shader object.

Tested by running full CTS after disabling NIR IO linking for VS/TES.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24328>
2023-07-28 06:51:24 +00:00
Samuel Pitoiset
2e7ea0fc24 radv: track whether inputs/outputs are linked per shader stage
With shader object, it's possible to compile shaders without any
linking.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24328>
2023-07-28 06:51:24 +00:00
Benjamin Cheng
3629b43822 radv/video: use app provided hevc scaling list order
This partially reverts commit da54b578.

Vulkan spec defers the definition of these lists to the H265 ITU spec,
which defines the scaling lists to be in "up-right diagonal scan order"
already.

Fixes: da54b578 ("radv/video: fix hevc scaling lists.")
Reviewed-by: Lynne <dev@lynne.ee>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24356>
2023-07-28 12:35:26 +10:00
Samuel Pitoiset
670bd70fa6 radv: emulate GEOMETRY_SHADER_INVOCATIONS query on RDNA1-2
The number of geometry shader invocations is correctly counted by the
hardware for both NGG and the legacy GS path but it increments for
NGG VS/TES because they are merged with GS, but it shouldn't. Fix this
by emulating the number of geometry shader invocations.

This fixes piglit/bin/arb_query_buffer_object-qbo and recent
dEQP-VK.query_pool.statistics_query.gs_invocations_no_gs.* failures
with NGG.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24231>
2023-07-27 09:15:22 +02:00
Samuel Pitoiset
b3aeaee5eb radv: implement nir_intrinsic_atomic_add_gs_invocation_count_amd
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24231>
2023-07-27 09:13:11 +02:00
Samuel Pitoiset
2e0893cf8b radv: rename RADV_SHADER_QUERY_PIPELINE_STAT_OFFSET
To RADV_SHADER_QUERY_GS_PRIM_EMIT_OFFSET.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24231>
2023-07-27 09:13:11 +02:00
Samuel Pitoiset
50709863ac radv: simplify the NGG vs legacy pipelinestat query path
NGG is enabled by default on RDNA1-2 but the driver might fallback to
legacy GS for some reasons, like XFB. On these generations, the number
of generated primitives by GS needs to be emulated from the NGG shader
because the hw doesn't increment the related pipelinestat counter.

In order to support NGG and legacy GS with that query (remember that
we can't know pipelines when starting/ending queries), we used to
reserve 2x 64-bit counters to store the GDS results, and the results
were accumulated.

Now that legacy GS also uses GDS counters, we can simplify this path
and overwrite the pipelinestat counter directly instead of having two
separate counters.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24231>
2023-07-27 09:13:11 +02:00
Samuel Pitoiset
f13ff5e70e radv: enable pipelinestat query emulation for legacy GS
This will allow us to fix a bug with the number of geometry shader
invocations which increase for NGG VS or TES but shouldn't. And also
for simplifying the NGG vs legacy query path.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24231>
2023-07-27 09:13:11 +02:00
Samuel Pitoiset
96b97ed527 radv: declare the shader query user SGPR for emulating GS counters
This user SGPR is only declared on chips that support NGG but might
fallback to legacy GS for some reasons, like XFB. It will be used to
emulate GS counters from shaders.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24231>
2023-07-27 09:13:11 +02:00
Samuel Pitoiset
e1f8cfc2b2 radv: rename NGG query state to be more generic
To use emulated GS counters for legacy GS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24231>
2023-07-27 09:13:11 +02:00
Rhys Perry
37fbfa655a aco: insert s_nop before VGPR deallocation
A hazard requires this, apparently.

https://reviews.llvm.org/D155681

fossil-db (gfx1100):
Totals from 23175 (17.36% of 133461) affected shaders:
Instrs: 41240100 -> 41263275 (+0.06%)
CodeSize: 211858524 -> 211951224 (+0.04%)
Latency: 469738570 -> 469738576 (+0.00%)
InvThroughput: 58013998 -> 58013999 (+0.00%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24320>
2023-07-26 13:58:22 +00:00
Samuel Pitoiset
08bfcc12d4 radv: rename radv_pipeline_stage to radv_shader_stage
It's more generic and it will fit shader object just well.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>
2023-07-26 07:44:49 +00:00
Samuel Pitoiset
090d88247d radv: cleanup pipeline compute emit helpers
Merge both functions together and rename the function.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>
2023-07-26 07:44:49 +00:00
Samuel Pitoiset
fdec88bd7c radv: rework determining the NGG stage without a graphics pipeline
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>
2023-07-26 07:44:49 +00:00
Samuel Pitoiset
174816019f radv: simplify lowering NGG GS intrinsics
The is_ngg field is already set correctly for GS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>
2023-07-26 07:44:49 +00:00
Samuel Pitoiset
70dbe011bb radv: rename graphics pipeline linking helpers
There is no pipeline dependency.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>
2023-07-26 07:44:49 +00:00
Samuel Pitoiset
697d4d4b03 radv: move removing all varyings when the FS is a noop
This allows us to remove one more pipeline dependency.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>
2023-07-26 07:44:49 +00:00
Samuel Pitoiset
5da9f38c53 radv: stop passing radv_graphics_pipeline to radv_fill_shader_info()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>
2023-07-26 07:44:49 +00:00
Samuel Pitoiset
a7fdcc3b22 radv: rework considering force VRS without relying on graphics pipeline
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>
2023-07-26 07:44:49 +00:00
Samuel Pitoiset
9d89b29a80 radv: set next_stage to MESA_SHADER_NONE if there is no FS
This follows the same convention as shader object where the last stage
would have nextStage to 0. This will allow more refactoring.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24313>
2023-07-26 07:44:49 +00:00