Commit graph

6839 commits

Author SHA1 Message Date
Samuel Pitoiset
d9e68f4e40 radv: pass a radv_shader to radv_emit_dispatch_packets()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22102>
2023-03-24 12:43:51 +00:00
Samuel Pitoiset
d07c81d87e radv: move cs_regalloc_hang_bug to radv_shader_info
This is more like a shader property.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22102>
2023-03-24 12:43:51 +00:00
Timur Kristóf
e743ddbbb4 radv: Enable mesh shading on GFX11.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21409>
2023-03-23 11:49:35 +00:00
Timur Kristóf
36edbebe8b radv: Adjust mesh draw packets for GFX11.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21409>
2023-03-23 11:49:35 +00:00
Timur Kristóf
9c1571bc56 radv: Include per-prim params in NUM_INTERP on GFX11.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21409>
2023-03-23 11:49:35 +00:00
Timur Kristóf
500f3e0bde radv: Use PRIM_ATTR for PS inputs on GFX11.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21409>
2023-03-23 11:49:35 +00:00
Timur Kristóf
09b81c7570 radv: Add extra offset to per-prim params.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21409>
2023-03-23 11:49:35 +00:00
Timur Kristóf
552614e15a radv: Use per-prim params in has_param_exports.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21409>
2023-03-23 11:49:35 +00:00
Timur Kristóf
d0f25109f6 radv: Add per-prim attributes to ring_attr stride.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21409>
2023-03-23 11:49:35 +00:00
Timur Kristóf
7cc9a72913 radv: Use new mesh shading packet defines.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21409>
2023-03-23 11:49:35 +00:00
Samuel Pitoiset
d4bce9508b radv: use common GetBufferMemoryRequirements2()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22047>
2023-03-22 07:50:04 +00:00
Alyssa Rosenzweig
968c5c6334 radv: Use umod_imm
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22010>
2023-03-22 06:18:18 +00:00
Samuel Pitoiset
cb5701eed9 radv: implement VK_KHR_map_memory2
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22032>
2023-03-21 23:08:03 +00:00
Samuel Pitoiset
cd59db8c46 radv: pass radv_shader to radv_shader_need_indirect_descriptor_sets()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:13 +00:00
Samuel Pitoiset
0d95660a03 radv: pass radv_ray_tracing_pipeline to radv_rt_pipeline_compile()
The base.base is redundant but this will allow us to move the shaders
array out of radv_pipeline.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:13 +00:00
Samuel Pitoiset
88137cd710 radv: rework dumping shaders when a GPU hang is reported
Preliminary work for moving the shaders array outside of radv_pipeline.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:13 +00:00
Samuel Pitoiset
444c5887c2 radv: pass radv_shader to radv_dump_shader_stats()
Preliminary work for moving the shaders array outside of radv_pipeline.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:13 +00:00
Samuel Pitoiset
0514c015e6 radv: use a separate compute path in radv_flush_constants()
radv_get_shader() will be a radv_graphics_pipeline only function.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:12 +00:00
Samuel Pitoiset
e7bbd610b8 radv: pass shader/base_reg to radv_emit_userdata_address()
Preliminary work for moving the shaders array outside of radv_pipeline.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:12 +00:00
Samuel Pitoiset
1e3cac8d5e radv: pass shader/base_reg to radv_emit_inline_push_consts()
Preliminary work for moving the shaders array outside of radv_pipeline.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:12 +00:00
Samuel Pitoiset
c2e71729af radv: pass shader/base_reg to radv_emit_descriptor_pointers()
Preliminary work for moving the shaders array outside of radv_pipeline.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:12 +00:00
Samuel Pitoiset
d7c5b6fb94 radv: replace radv_lookup_user_sgpr() by radv_get_user_sgpr()
radv_get_user_sgpr() no longer relies on radv_pipeline which is
another step for moving the shaders array outside of it.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:12 +00:00
Samuel Pitoiset
4066e3a951 radv: stop using radv_get_shader_shader() for task shaders
radv_get_shader() should only be used for VS or TES, no need to add
another indirection for task shaders. While we are at it, rename
compute_shader to task_shader.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:12 +00:00
Samuel Pitoiset
5082b6b034 radv: add support for caching PS epilogs
For PS epilogs created at link time because libraries are still not
cached.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:59 +00:00
Samuel Pitoiset
228afe1a2f radv: make radv_shader_part_create() non-static
It will be used when creating shaders from the cache.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:59 +00:00
Samuel Pitoiset
83c20b95dd radv: allow to return the PS epilog binary to the pipeline
To add it to the shaders cache.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:59 +00:00
Samuel Pitoiset
eba315d2bf radv: upload prologs/epilogs as part of radv_shader_part_create()
Don't need to duplicate this code.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:59 +00:00
Samuel Pitoiset
aa15d64949 radv: store the total radv_shader_part_binary size
Similar to radv_shader. This will be used for the shaders cache.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:59 +00:00
Samuel Pitoiset
aec9fd394f radv: store spi_shader_col_format to radv_shader_part_binary
For PS epilogs in the cache.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:58 +00:00
Samuel Pitoiset
a455e0c99f radv: stop storing the binary as part of radv_shader_part
It's unnecessary to keep a pointer to the binary.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:58 +00:00
Samuel Pitoiset
2803b786f0 radv: remove set but never used num_preserved_sgprs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:58 +00:00
Samuel Pitoiset
f2aa0e80a0 radv: enable fullyCoveredFragmentShaderInputVariable on GFX9+
For vkd3d-proton Tier 3.

It's passing CTS and vkd3d-proton tests.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8311
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21497>
2023-03-21 08:44:09 +00:00
Samuel Pitoiset
8e84251cc7 radv: implement fullyCoveredFragmentShaderInputVariable
1 means INNER_COVERAGE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21497>
2023-03-21 08:44:09 +00:00
Samuel Pitoiset
ebbb0d3f1b radv: enable SAMPLE_COVERAGE_ENA if the fully covered built-in is used
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21497>
2023-03-21 08:44:09 +00:00
Samuel Pitoiset
2a02f567aa radv: lower nir_intrinsic_load_fully_covered
The sample coverage VGPR input would be the inner coverage and 0 means
it's uncovered.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21497>
2023-03-21 08:44:09 +00:00
Benjamin Cheng
e57caf9893 radv: initialize cmd_buffer upload list earlier
When cs allocation fails in radv_create_cmd_buffer,
radv_destroy_cmd_buffer is called before returning
VK_ERROR_OUT_OF_HOST_MEMORY. At that point, the upload list is not
initalized yet, so SIGSEGV will occur when trying to iterate through the
upload bo list. Initialize the upload list earlier to avoid this.

Signed-off-by: Benjamin Cheng <ben@bcheng.me>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22016>
2023-03-21 08:06:24 +00:00
Konstantin Seurer
deb537de3c radv/rt: Handle load_constant instructions when inlining shaders
Fixes the following tests:
dEQP-VK.ray_query.builtin.rayqueryterminate.ahit.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.ahit.triangles,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.call.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.call.triangles,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.chit.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.chit.triangles,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.miss.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.miss.triangles,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.rgen.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.rgen.triangles,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.sect.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.sect.triangles,Fail

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8647
Fixes: fda262f ("radv/rt: move Ray Tracing shader creation into separate file")
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22002>
2023-03-20 19:04:34 +00:00
Samuel Pitoiset
d750ad19fd radv: fix NGG streamout with VS and GPL on GFX11
With GPL it's not possible to know the primitive topology when
compiling the pre-rasterization stages. For NGG, we use the maximum
number of vertices per prim and rely on the hardware to ignore the
extra bits for points/lines.

Though, this can't work for NGG streamout because the number of
vertices per prim is used to compute a streamout offset. The only
way to solve this is to pass the number of vertices per prim through
a new user SGPR.

This fixes a bunch of streamout tests with Zink/RADV on GFX11.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21833>
2023-03-20 17:47:03 +00:00
Samuel Pitoiset
0badfd8b20 radv: add helpers for destroying various pipeline types
Much cleaner than having a single function for everything.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21894>
2023-03-20 13:56:32 +00:00
Samuel Pitoiset
abfdc06b01 radv: rename RADV_PIPELINE_LIBRARY to RADV_PIPELINE_RAY_TRACING_LIB
This seems more consistent with graphics pipeline libraries and it
avoids any confusion.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21894>
2023-03-20 13:56:32 +00:00
Konstantin Seurer
0f18bb4076 radv: Fix inserting stack_size into the cache
Fixes: 3e03fe4 ("radv/rt: move stack_sizes into radv_ray_tracing_module")
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21969>
2023-03-18 14:57:51 +00:00
Konstantin Seurer
3887f64dc3 radv: Fix loading stack_size from the cache
Fixes: 3e03fe4 ("radv/rt: move stack_sizes into radv_ray_tracing_module")
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21969>
2023-03-18 14:57:51 +00:00
Tatsuyuki Ishi
22d6556a4b radv: Fix missing wait of GS copy shader upload for dmashaders.
Fixes: 0cde42a506 ("radv: Wait for shader uploads asynchronously.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21985>
2023-03-18 03:04:15 +00:00
Friedrich Vock
89590c1d84 radv: Add RT shader stage names for executable properties
Now that we use raygen shaders, we also need to support RT stages for
executable properties.

Fixes: f123d65e9f ("radv/rt: use prolog for raytracing shaders")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21960>
2023-03-16 21:28:03 +00:00
Tatsuyuki Ishi
0cde42a506 radv: Wait for shader uploads asynchronously.
This introduces tracking of the required semaphore values in pipelines,
which is then propagated to cmd_buffers on bind. Each queue also keeps
track the maximum count it has waited for, so that we can avoid the waiting
overhead once all the shaders are loaded and referenced.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>
2023-03-16 18:02:57 +00:00
Tatsuyuki Ishi
a8c5fd3b1b radv: Upload shaders to invisible VRAM on small BAR systems.
Following PAL's implementation, this patch avoids allocating shader code
buffers in BAR and use SDMA to upload them to invisible VRAM
directly.

For some games like HZD, shaders can take as much as 400MB, which exceeds
the non-resizable BAR size (256MB) and cause inconsistent spilling
behavior. The kernel will normally move these to invisible VRAM on its own,
but there are a few cases that it does not reliably happen. This patch does
the moving explicitly in the driver to ensure predictable results.

In this patch, we upload the shaders synchronously; so the shader will be
ready as soon as vkCreate*Pipeline returns. A following patch will make
this asynchronous and don't block until we see a use of the pipeline.

As a side effect, when SQTT is used we now store the shaders on a cacheable
buffer which would speed up writing the trace to the disk.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>
2023-03-16 18:02:57 +00:00
Tatsuyuki Ishi
3b258ae2d9 radv: Introduce sdma_copy_buffer for GFX7+.
Helper salvaged from radeonsi (before SDMA removal).

This will be used for driver internal submissions to DMA shaders from GTT
to invisible VRAM.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>
2023-03-16 18:02:56 +00:00
Tatsuyuki Ishi
d4fb3db748 radv: Use radeon_cmdbuf for sdma_copy_image.
For consistency with the sdma_copy_buffer helper that will be added next.

As a general justification, SDMA commands require little state tracking and
using radeon_cmdbuf makes it more suitable for driver internal use.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>
2023-03-16 18:02:56 +00:00
Jesse Natalie
f8566533ea radv: Fix returning an expression from a void function
Fixes: d5de56bf ("radv: add RT shader args")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21932>
2023-03-16 16:07:14 +00:00
Tatsuyuki Ishi
9faaff4561 radv/rt: Don't upload the prolog twice.
radv_shader_create already calls radv_shader_binary_upload.

Fixes: 4b92a53285 ("radv: add radv_create_rt_prolog()")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21945>
2023-03-16 12:27:21 +00:00