Commit graph

8070 commits

Author SHA1 Message Date
Daniel Schürmann
bdbf873b0f nir: remove redundant passes from nir_opt_if()
These are now covered by nir_opt_loop():
- opt_if_loop_last_continue()
- opt_merge_breaks()
- opt_if_loop_terminator()

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24940>
2024-01-03 20:48:05 +00:00
Daniel Schürmann
a3ed36da1a treewide: replace calls to nir_opt_trivial_continues() with nir_opt_loop()
Totals from 850 (1.11% of 76636) affected shaders: (RADV, GFX11)
MaxWaves: 18134 -> 18130 (-0.02%)
Instrs: 3011298 -> 3008585 (-0.09%); split: -0.17%, +0.08%
CodeSize: 15836804 -> 15841972 (+0.03%); split: -0.09%, +0.12%
VGPRs: 63580 -> 63604 (+0.04%)
SpillSGPRs: 966 -> 1148 (+18.84%); split: -0.83%, +19.67%
Latency: 36102291 -> 30186144 (-16.39%); split: -16.41%, +0.02%
InvThroughput: 9058100 -> 7011821 (-22.59%); split: -22.61%, +0.02%
VClause: 65369 -> 65364 (-0.01%); split: -0.03%, +0.02%
SClause: 100309 -> 100305 (-0.00%); split: -0.04%, +0.04%
Copies: 335658 -> 336472 (+0.24%); split: -0.70%, +0.94%
Branches: 110806 -> 108945 (-1.68%); split: -1.94%, +0.26%
PreSGPRs: 73476 -> 73934 (+0.62%); split: -0.25%, +0.87%
PreVGPRs: 58809 -> 58840 (+0.05%); split: -0.01%, +0.06%

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24940>
2024-01-03 20:48:04 +00:00
Samuel Pitoiset
7ea48145a1 radv: stop clearing CMASK to 0xcc when FMASK is present on GFX9
This is incorrect because clearing CMASK to 0xCC should only happen
for fast clears with DCC and FMASK/CMASK.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26634>
2024-01-03 13:58:06 +00:00
Samuel Pitoiset
1714f7aff4 radv: disable FMASK for MSAA images with layers on GFX9
It seems that FMASK can be interleaved and RADV doesn't support this
properly, so let's disable it.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26634>
2024-01-03 13:58:06 +00:00
Samuel Pitoiset
10ce61e16c radv: advertise VK_KHR_maintenance6
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26757>
2024-01-03 11:24:27 +00:00
Samuel Pitoiset
db9816fd66 radv: add support for NULL index buffer
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26757>
2024-01-03 11:24:27 +00:00
Samuel Pitoiset
744cb98bc6 radv: add support for version 2 of all descriptor binding commands
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26757>
2024-01-03 11:24:27 +00:00
Samuel Pitoiset
5f63624701 radv: rename RADV_GRAPHICS_STAGES to RADV_GRAPHICS_STAGE_BITS
For consistency with RADV_RT_STAGE_BITS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26757>
2024-01-03 11:24:27 +00:00
Samuel Pitoiset
17081334ab radv: add support for VkBindMemoryStatusKHR
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26757>
2024-01-03 11:24:27 +00:00
Samuel Pitoiset
845a666e0f radv: move radv_BindImageMemory2() to radv_image.c
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26757>
2024-01-03 11:24:27 +00:00
Samuel Pitoiset
f30fa9dec8 radv: move radv_rt_{common,shader} files to nir/
These files contains NIR lowering code for RT.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26851>
2024-01-03 09:40:29 +00:00
Samuel Pitoiset
02c5365ffa radv: make a couple of NIR RT functions as static
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26851>
2024-01-03 09:40:29 +00:00
Samuel Pitoiset
685c4b6337 radv: move radv_{emulate,enable}_rt() to radv_physical_device.c
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26851>
2024-01-03 09:40:29 +00:00
Eric Engestrom
7e8db6aedf meson: always define {,DRAW_}LLVM_AVAILABLE one way or the other
With the usual benefits of `#if` instead of `#ifdef` (mostly the fact
that typos can be build failures instead of silently being interpreted
as if 0).

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3863>
2023-12-24 10:01:39 +00:00
Friedrich Vock
b2067001d4 radv/rt: bsearch inlined shaders
When there are lots of inlined shaders, going over each one and checking
if the call index matches becomes expensive. Instead, use a
binary-search-like selection to skip most of the checks.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26380>
2023-12-21 12:39:06 +00:00
Timur Kristóf
4d93aac74d radv: Use correct plane and binding index with SDMA.
This lets us support multi-planar images properly on transfer queues.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26353>
2023-12-20 13:22:26 +00:00
Timur Kristóf
ab4720691c radv: Clean up SDMA chunked copy info struct.
Remove redundant fields.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26353>
2023-12-20 13:22:26 +00:00
Timur Kristóf
7fe899a3b6 radv: Use SDMA surface structs for determining unaligned buffer copies.
This removes a bunch of redundant calculations, thus making the code
less error-prone.

Also move the row pitch calculation to a separate function.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26353>
2023-12-20 13:22:26 +00:00
Timur Kristóf
dab4863396 radv: Pass radv_sdma_surf from copy functions to SDMA.
This makes it possible to gather the surface information
only once and pass it to the various SDMA functions, therefore
the code can be less error-prone.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26353>
2023-12-20 13:22:26 +00:00
Timur Kristóf
85fa749c63 radv: Refactor and simplify SDMA surface info functions.
This makes it possible to call the function just once instead
of calling different functions repeatedly.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26353>
2023-12-20 13:22:26 +00:00
Timur Kristóf
a21cba6799 radv: Unify SDMA surface struct for linear and tiled images.
Using just one struct for both types of images (and buffers)
will simplify a lot of code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26353>
2023-12-20 13:22:26 +00:00
Timur Kristóf
65dfdd3fff radv: Move SDMA function and struct declarations to a new header.
Very few parts of RADV actually need the SDMA functions, so
moving them to a separate header makes the driver cleaner and
also improves compilation time when SDMA functions change.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26353>
2023-12-20 13:22:26 +00:00
Bas Nieuwenhuizen
07ad6fd34a radv: Use correct writemask for cooperative matrix ordering.
Not expecting this to actually fix anything externally visible,
but reduces some invalid usage when the resulting vector is
not 16 elements long (e.g. the C/result matrix).

Fixes: 9df4703fbb ("radv: Add cooperative matrix lowering.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26768>
2023-12-20 11:02:30 +00:00
Bas Nieuwenhuizen
eaf61adea5 radv: Add option to clear LDS at the end of a shader.
Only shaders which explicitly allow shared memory are included for
now. The pass is very late to avoid optimizations removing the stores
and to ensure the clear gets added after MS outputs get loaded from LDS.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26679>
2023-12-20 09:15:45 +00:00
Samuel Pitoiset
b5153693cc radv: move image view related code to radv_image_view.c
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26713>
2023-12-19 09:48:35 +00:00
Samuel Pitoiset
1db86d93f2 radv: move buffer view related code to radv_buffer_view.c
More code isolation.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26713>
2023-12-19 09:48:35 +00:00
Samuel Pitoiset
02282fe482 radv: move SI_GS_PER_ES to radv_constants.h
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26713>
2023-12-19 09:48:35 +00:00
Samuel Pitoiset
d057cfc8f6 radv: move more shader related declarations to radv_shader.h
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26713>
2023-12-19 09:48:35 +00:00
Samuel Pitoiset
0481723cf0 radv: move radv_depth_clamp_mode to radv_cmd_buffer.c
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26713>
2023-12-19 09:48:35 +00:00
Samuel Pitoiset
c0d7ef1e58 radv: move more descriptor related declarations to radv_descriptor_set.h
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26713>
2023-12-19 09:48:34 +00:00
Samuel Pitoiset
a080390c3c radv: move radv_image_is_renderable() to radv_image.c
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26713>
2023-12-19 09:48:34 +00:00
Samuel Pitoiset
8a5106f91a radv: move RADV_HASH_SHADER_xxx flags to radv_pipeline.c
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26713>
2023-12-19 09:48:34 +00:00
Samuel Pitoiset
9d15390635 radv: move meta declarations to radv_meta.h
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26713>
2023-12-19 09:48:34 +00:00
Samuel Pitoiset
939936a328 radv: remove radv_get_tess_output_topology() declaration
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26713>
2023-12-19 09:48:34 +00:00
Konstantin Seurer
662f86c533 radv/bvh: Stop emitting leaf nodes inside the encoder
Avoids unnecessary copies.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26725>
2023-12-18 19:17:21 +00:00
Konstantin Seurer
8fca54a754 radv: Add more offsets acceleration_structure_layout
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26725>
2023-12-18 19:17:21 +00:00
Tatsuyuki Ishi
533ec9843e radv: Precompute shader max_waves.
Doing it at bind-time causes a 1.4% overhead (among all driver calls) in
Overwatch 2. !24502 mentions that it can be precomputed in case overhead
is a concern, so do it here.

max_waves is stored directly in the radv_shader struct, because
ac_shader_config conforms to LLVM ABI and we cannot add anything custom,
and radv_shader_info needs to be determined from NIR only.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26692>
2023-12-18 14:31:25 +00:00
Tatsuyuki Ishi
1161f22c27 radv: Move up radv_get_max_waves, radv_get_max_scratch_waves.
To avoid forward declaration.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26692>
2023-12-18 14:31:25 +00:00
Tatsuyuki Ishi
e444908d65 radv: Simplify shader config assignment.
We don't hash this struct so direct assignment here is OK.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26692>
2023-12-18 14:31:24 +00:00
Samuel Pitoiset
4353b0ad72 radv: move emitting the fb mip tail workaround when rendering begins
It doesn't have to be emitted in the draw path.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26589>
2023-12-18 13:06:29 +00:00
Samuel Pitoiset
7dd7e551b1 radv: stop checking FMASK for the fb mip tail workaround
Vulkan doesn't allow mipmaps with MSAA images, so checking for FMASK
shouldn't have any effect.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26589>
2023-12-18 13:06:29 +00:00
Samuel Pitoiset
57efe44f43 radv: add missing HTILE support for fb mip tail workaround
PAL also applies to depth/stencil images with HTILE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26589>
2023-12-18 13:06:29 +00:00
Friedrich Vock
f9a394b753 radv/rt: Initialize unused children in PLOC early-exit
Bad things happen when these aren't initialized.

Fixes GPU hangs when loading Avatar: Frontiers of Pandora savegames.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26712>
2023-12-15 17:52:00 +00:00
Samuel Pitoiset
768c737273 radv: remove some declared but unused functions/macros
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26682>
2023-12-15 08:40:24 +00:00
Samuel Pitoiset
2f79ed1831 radv: stop asserting some image create info fields
Vulkan Validation Layers already check that and these assertions never
caught anything.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26682>
2023-12-15 08:40:23 +00:00
Samuel Pitoiset
1ef5feac5e radv: fix binding partial depth/stencil views with dynamic rendering
With dynamic rendering, it's allowed to begin rendering with depth or
stencil only but still with a depth/stencil format. The test below
checks that unbound part of ds isn't modified, if depth is bound and
stencil not and vice versa.

This fixes a recent CTS
dEQP-VK.dynamic_rendering.primary_cmd_buff.basic.partial_binding_depth_stencil.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25350>
2023-12-14 12:54:23 +00:00
Samuel Pitoiset
98ea540158 radv: add support for MRT compaction with PS epilogs
Now that PS epilogs are always compiled during cmdbuf recording, we
have all information to enable MRT compaction, for optimal performance.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26667>
2023-12-14 09:51:26 +01:00
Samuel Pitoiset
8760b7fab7 radv: cleanup radv_pipeline_generate_ps_epilog_key()
This has no effects because key->spi_shader_col_format isn't used when
the graphics pipeline needs to compile a PS epilog.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26663>
2023-12-14 08:18:46 +00:00
Samuel Pitoiset
60e6e3f8e6 radv: cleanup gathering PS info with/without PS epilogs
To make it clear that some fields aren't used with PS epilogs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26663>
2023-12-14 08:18:46 +00:00
Samuel Pitoiset
a39c3920fb radv: cleanup ac_nir_lower_ps options
To make it clear that some fields aren't used with PS epilogs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26663>
2023-12-14 08:18:46 +00:00