Commit graph

6818 commits

Author SHA1 Message Date
Samuel Pitoiset
d7c5b6fb94 radv: replace radv_lookup_user_sgpr() by radv_get_user_sgpr()
radv_get_user_sgpr() no longer relies on radv_pipeline which is
another step for moving the shaders array outside of it.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:12 +00:00
Samuel Pitoiset
4066e3a951 radv: stop using radv_get_shader_shader() for task shaders
radv_get_shader() should only be used for VS or TES, no need to add
another indirection for task shaders. While we are at it, rename
compute_shader to task_shader.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21878>
2023-03-21 20:36:12 +00:00
Samuel Pitoiset
5082b6b034 radv: add support for caching PS epilogs
For PS epilogs created at link time because libraries are still not
cached.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:59 +00:00
Samuel Pitoiset
228afe1a2f radv: make radv_shader_part_create() non-static
It will be used when creating shaders from the cache.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:59 +00:00
Samuel Pitoiset
83c20b95dd radv: allow to return the PS epilog binary to the pipeline
To add it to the shaders cache.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:59 +00:00
Samuel Pitoiset
eba315d2bf radv: upload prologs/epilogs as part of radv_shader_part_create()
Don't need to duplicate this code.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:59 +00:00
Samuel Pitoiset
aa15d64949 radv: store the total radv_shader_part_binary size
Similar to radv_shader. This will be used for the shaders cache.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:59 +00:00
Samuel Pitoiset
aec9fd394f radv: store spi_shader_col_format to radv_shader_part_binary
For PS epilogs in the cache.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:58 +00:00
Samuel Pitoiset
a455e0c99f radv: stop storing the binary as part of radv_shader_part
It's unnecessary to keep a pointer to the binary.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:58 +00:00
Samuel Pitoiset
2803b786f0 radv: remove set but never used num_preserved_sgprs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21897>
2023-03-21 13:53:58 +00:00
Samuel Pitoiset
f2aa0e80a0 radv: enable fullyCoveredFragmentShaderInputVariable on GFX9+
For vkd3d-proton Tier 3.

It's passing CTS and vkd3d-proton tests.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8311
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21497>
2023-03-21 08:44:09 +00:00
Samuel Pitoiset
8e84251cc7 radv: implement fullyCoveredFragmentShaderInputVariable
1 means INNER_COVERAGE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21497>
2023-03-21 08:44:09 +00:00
Samuel Pitoiset
ebbb0d3f1b radv: enable SAMPLE_COVERAGE_ENA if the fully covered built-in is used
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21497>
2023-03-21 08:44:09 +00:00
Samuel Pitoiset
2a02f567aa radv: lower nir_intrinsic_load_fully_covered
The sample coverage VGPR input would be the inner coverage and 0 means
it's uncovered.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21497>
2023-03-21 08:44:09 +00:00
Benjamin Cheng
e57caf9893 radv: initialize cmd_buffer upload list earlier
When cs allocation fails in radv_create_cmd_buffer,
radv_destroy_cmd_buffer is called before returning
VK_ERROR_OUT_OF_HOST_MEMORY. At that point, the upload list is not
initalized yet, so SIGSEGV will occur when trying to iterate through the
upload bo list. Initialize the upload list earlier to avoid this.

Signed-off-by: Benjamin Cheng <ben@bcheng.me>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22016>
2023-03-21 08:06:24 +00:00
Konstantin Seurer
deb537de3c radv/rt: Handle load_constant instructions when inlining shaders
Fixes the following tests:
dEQP-VK.ray_query.builtin.rayqueryterminate.ahit.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.ahit.triangles,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.call.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.call.triangles,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.chit.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.chit.triangles,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.miss.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.miss.triangles,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.rgen.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.rgen.triangles,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.sect.aabbs,Fail
dEQP-VK.ray_query.builtin.rayqueryterminate.sect.triangles,Fail

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8647
Fixes: fda262f ("radv/rt: move Ray Tracing shader creation into separate file")
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22002>
2023-03-20 19:04:34 +00:00
Samuel Pitoiset
d750ad19fd radv: fix NGG streamout with VS and GPL on GFX11
With GPL it's not possible to know the primitive topology when
compiling the pre-rasterization stages. For NGG, we use the maximum
number of vertices per prim and rely on the hardware to ignore the
extra bits for points/lines.

Though, this can't work for NGG streamout because the number of
vertices per prim is used to compute a streamout offset. The only
way to solve this is to pass the number of vertices per prim through
a new user SGPR.

This fixes a bunch of streamout tests with Zink/RADV on GFX11.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21833>
2023-03-20 17:47:03 +00:00
Samuel Pitoiset
0badfd8b20 radv: add helpers for destroying various pipeline types
Much cleaner than having a single function for everything.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21894>
2023-03-20 13:56:32 +00:00
Samuel Pitoiset
abfdc06b01 radv: rename RADV_PIPELINE_LIBRARY to RADV_PIPELINE_RAY_TRACING_LIB
This seems more consistent with graphics pipeline libraries and it
avoids any confusion.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21894>
2023-03-20 13:56:32 +00:00
Konstantin Seurer
0f18bb4076 radv: Fix inserting stack_size into the cache
Fixes: 3e03fe4 ("radv/rt: move stack_sizes into radv_ray_tracing_module")
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21969>
2023-03-18 14:57:51 +00:00
Konstantin Seurer
3887f64dc3 radv: Fix loading stack_size from the cache
Fixes: 3e03fe4 ("radv/rt: move stack_sizes into radv_ray_tracing_module")
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21969>
2023-03-18 14:57:51 +00:00
Tatsuyuki Ishi
22d6556a4b radv: Fix missing wait of GS copy shader upload for dmashaders.
Fixes: 0cde42a506 ("radv: Wait for shader uploads asynchronously.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21985>
2023-03-18 03:04:15 +00:00
Friedrich Vock
89590c1d84 radv: Add RT shader stage names for executable properties
Now that we use raygen shaders, we also need to support RT stages for
executable properties.

Fixes: f123d65e9f ("radv/rt: use prolog for raytracing shaders")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21960>
2023-03-16 21:28:03 +00:00
Tatsuyuki Ishi
0cde42a506 radv: Wait for shader uploads asynchronously.
This introduces tracking of the required semaphore values in pipelines,
which is then propagated to cmd_buffers on bind. Each queue also keeps
track the maximum count it has waited for, so that we can avoid the waiting
overhead once all the shaders are loaded and referenced.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>
2023-03-16 18:02:57 +00:00
Tatsuyuki Ishi
a8c5fd3b1b radv: Upload shaders to invisible VRAM on small BAR systems.
Following PAL's implementation, this patch avoids allocating shader code
buffers in BAR and use SDMA to upload them to invisible VRAM
directly.

For some games like HZD, shaders can take as much as 400MB, which exceeds
the non-resizable BAR size (256MB) and cause inconsistent spilling
behavior. The kernel will normally move these to invisible VRAM on its own,
but there are a few cases that it does not reliably happen. This patch does
the moving explicitly in the driver to ensure predictable results.

In this patch, we upload the shaders synchronously; so the shader will be
ready as soon as vkCreate*Pipeline returns. A following patch will make
this asynchronous and don't block until we see a use of the pipeline.

As a side effect, when SQTT is used we now store the shaders on a cacheable
buffer which would speed up writing the trace to the disk.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>
2023-03-16 18:02:57 +00:00
Tatsuyuki Ishi
3b258ae2d9 radv: Introduce sdma_copy_buffer for GFX7+.
Helper salvaged from radeonsi (before SDMA removal).

This will be used for driver internal submissions to DMA shaders from GTT
to invisible VRAM.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>
2023-03-16 18:02:56 +00:00
Tatsuyuki Ishi
d4fb3db748 radv: Use radeon_cmdbuf for sdma_copy_image.
For consistency with the sdma_copy_buffer helper that will be added next.

As a general justification, SDMA commands require little state tracking and
using radeon_cmdbuf makes it more suitable for driver internal use.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16271>
2023-03-16 18:02:56 +00:00
Jesse Natalie
f8566533ea radv: Fix returning an expression from a void function
Fixes: d5de56bf ("radv: add RT shader args")
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21932>
2023-03-16 16:07:14 +00:00
Tatsuyuki Ishi
9faaff4561 radv/rt: Don't upload the prolog twice.
radv_shader_create already calls radv_shader_binary_upload.

Fixes: 4b92a53285 ("radv: add radv_create_rt_prolog()")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21945>
2023-03-16 12:27:21 +00:00
Daniel Schürmann
f123d65e9f radv/rt: use prolog for raytracing shaders
Co-authored-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:30 +00:00
Friedrich Vock
bea022d1f6 radv/rt: Add shader config combination/postprocessing utils
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:30 +00:00
Friedrich Vock
0569b350ed radv: Emit RT shader VA user SGPR
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:30 +00:00
Daniel Schürmann
a16df842a6 radv: compile rt_prolog
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:30 +00:00
Daniel Schürmann
4b92a53285 radv: add radv_create_rt_prolog()
Co-authored-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:30 +00:00
Daniel Schürmann
c38b8678c9 radv: add RT shader handling to radv_postprocess_config
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:29 +00:00
Daniel Schürmann
3f03eebf04 radv: add RT stages to radv_get_shader_name()
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:29 +00:00
Daniel Schürmann
650f386bdd radv: handle RT stages in radv_nir_shader_info_pass()
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:29 +00:00
Daniel Schürmann
d5de56bf59 radv: add RT shader args
Co-authored-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:29 +00:00
Timur Kristóf
6185e4f2ff aco, radv: Remove VS IO information from ACO.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>
2023-03-15 14:54:28 +00:00
Timur Kristóf
2f78700f89 radv: Remove VS inputs code from LLVM backend.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>
2023-03-15 14:54:28 +00:00
Timur Kristóf
27c8131978 radv: Apply swizzle and alpha adjust in radv_nir_lower_vs_inputs.
Deal with VS input related things in a single pass instead of
having two different passes.

Fossil DB stats on Rembrandt (GFX10.3):

Totals from 174 (0.13% of 134913) affected shaders:
VGPRs: 7736 -> 7520 (-2.79%)
CodeSize: 354004 -> 353604 (-0.11%); split: -0.17%, +0.06%
MaxWaves: 4196 -> 4248 (+1.24%)
Instrs: 65228 -> 65139 (-0.14%); split: -0.19%, +0.06%
Latency: 265823 -> 265728 (-0.04%); split: -0.12%, +0.08%
InvThroughput: 84629 -> 84644 (+0.02%); split: -0.08%, +0.10%
VClause: 1618 -> 1606 (-0.74%); split: -0.93%, +0.19%
SClause: 1382 -> 1379 (-0.22%); split: -0.36%, +0.14%
Copies: 5586 -> 5566 (-0.36%); split: -0.55%, +0.20%
PreSGPRs: 4994 -> 5037 (+0.86%); split: -0.10%, +0.96%
PreVGPRs: 4948 -> 4955 (+0.14%); split: -0.04%, +0.18%

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>
2023-03-15 14:54:27 +00:00
Timur Kristóf
a46acdbc3f radv: Lower non-dynamic VS inputs in NIR.
Add a new RADV specific NIR pass which lowers VS input
loads to AMD specific buffer load instructions.
We do this because we want to remove the RADV specific
VS input handling from the shader compiler back-ends.

Fossil DB stats on Rembrandt (GFX10.3):

Totals from 32507 (24.09% of 134913) affected shaders:
VGPRs: 1245512 -> 1245344 (-0.01%); split: -0.35%, +0.34%
SpillSGPRs: 1068 -> 1102 (+3.18%)
CodeSize: 90333192 -> 90327232 (-0.01%); split: -0.07%, +0.06%
MaxWaves: 881816 -> 881388 (-0.05%); split: +0.23%, -0.28%
Instrs: 17264710 -> 17264562 (-0.00%); split: -0.09%, +0.09%
Latency: 87300501 -> 86586480 (-0.82%); split: -1.07%, +0.25%
InvThroughput: 13700046 -> 13685931 (-0.10%); split: -0.20%, +0.10%
VClause: 361520 -> 361301 (-0.06%); split: -1.32%, +1.26%
SClause: 441018 -> 441505 (+0.11%); split: -0.54%, +0.65%
Copies: 1371477 -> 1373838 (+0.17%); split: -0.57%, +0.75%
Branches: 496639 -> 496611 (-0.01%); split: -0.01%, +0.00%
PreSGPRs: 1122956 -> 1122663 (-0.03%); split: -0.09%, +0.06%
PreVGPRs: 976051 -> 995717 (+2.01%); split: -0.12%, +2.14%

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>
2023-03-15 14:54:27 +00:00
Yogesh Mohan Marimuthu
2b04d6cada radv: set RADEON_SURF_NO_TEXTURE flag in radv_get_surface_flags()
In vkCreateImage() if block comrpessed format and VK_IMAGE_TILING_LINEAR is
used, then the app crashes in vega gpu.

This is because addrlib does not support BC + linear as from function
ValidateSwModeParams(). From Marek Olšák the addrlib behaviour is correct.

In pal driver, flags.texture is not set in DetermineSurfaceFlags() function
if BC + linear. pal driver does it because it is expected that the
BC + linear image is only used as transfer resource.

This patch sets RADEON_SURF_NO_TEXTURE flag if usage is not
VK_IMAGE_USAGE_SAMPLED_BIT and and VK_IMAGE_USAGE_STORAGE_BIT.
flags.texture flag is not set if RADEON_SURF_NO_TEXTURE and this fixes
the crash.

v1: set NO_TEXTURE if not SAMPLED or STORAGE (Marek Olšák)

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21422>
2023-03-15 12:25:33 +00:00
Dave Airlie
040dc89b95 radv/trace: don't attempt to emit trace on non-graphics/compute queues
This will just break on video queues

Fixes: 9477f117f4 ("radv/video: add initial frameworking.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21671>
2023-03-15 05:08:49 +00:00
Dave Airlie
17d19a3c07 radv/video: fix h265 decoding sizes.
Fixes the misrendering/hangs on hevc content.

Fixes: 6c3c242361 ("radv/video: add h265 decode UVD support")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21671>
2023-03-15 05:08:49 +00:00
Dave Airlie
67a606c611 radv/video: fix used for reference flags.
These weren't getting programmed properly for interlaced videos

Fixes: 3e2c768aa8 ("radv/vcn: enable dynamic dpb tier 2 for h264/h265 on navi21+")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21671>
2023-03-15 05:08:49 +00:00
Dave Airlie
30ccf31aae radv/video: fix h264 frame heights when field images are in use
This was breaking MBAFF decoding.

Fixes: 8a29291dbe ("radv/video: add h264 support for uvd")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21671>
2023-03-15 05:08:49 +00:00
Daniel Schürmann
f6a36190a1 radv/rt: Fix any_hit scratch variables.
We have to make sure not to change call_data locations as well.

Fixes: 481f78ab93 ('radv/rt: place any-hit scratch vars after intersection scratch vars')
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21876>
2023-03-14 11:57:02 +00:00
Konstantin Seurer
ecf29228d0 radv/sqtt: Skip dumping pipeline libraries
They don't have any shaders which can lead to crashes when dumping
them.

Fixes: 2e04aeb ("radv: capture RT pipelines from the SQTT layer")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21852>
2023-03-13 20:05:49 +00:00
Samuel Pitoiset
4d03bf0f9d radv: allow to cache optimized (LTO) pipelines with GPL
This should be working now, except PS epilogs that are still not
added to the cache.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21834>
2023-03-13 13:35:24 +00:00