Commit graph

7472 commits

Author SHA1 Message Date
Daniel Schürmann
8740fb0026 radv/meta: disable disk cache for meta shaders
Meta shaders are already stored in a separate cache file,
inserting them into the disk cache is unnecessary.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23909>
2023-07-11 07:36:53 +00:00
Friedrich Vock
e034ba1c44 radv/rt: Miss rays that hit the triangle's v edge
The hardware seems to do this as well. Avoids invoking hit shaders twice
at shared edges.
Fixes the fails in watertightness tests on emulated RT.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24063>
2023-07-10 18:31:30 +00:00
Friedrich Vock
a096cf784f radv/rt: Enable exact on software intersection functions
These functions need exact output, otherwise watertightness can't be
guaranteed in some cases.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24063>
2023-07-10 18:31:30 +00:00
Erik Faye-Lund
6d42711043 radv: do not rely on constant-folding
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24004>
2023-07-10 16:06:40 +02:00
Friedrich Vock
d92520e19c Revert "Revert "radv: Enable ray tracing pipelines by default""
This reverts commit 49b400423c.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23978>
2023-07-08 10:10:15 +00:00
Friedrich Vock
816ed055ac Revert "radv/rt: Enable RT pipelines on GFX10_3+ excluding vangogh"
This reverts commit 7f3874981d.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23978>
2023-07-08 10:10:15 +00:00
Jesse Natalie
511116d630 radv: Fix label name
Reviewed-by: Eric Engestrom <eric@igalia.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Acked-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24042>
2023-07-07 17:40:38 +00:00
Friedrich Vock
4880c827d6 radv: Re-enable RT pipeline capture/replay handles
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:25 +00:00
Friedrich Vock
169583a4ad radv/rt: Rework radv_GetRayTracingCaptureReplayShaderGroupHandlesKHR
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:25 +00:00
Friedrich Vock
fccf6fbeec radv/rt: Replay shader allocations according to capture/replay handle
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:25 +00:00
Friedrich Vock
3e9bd821f1 radv/rt: Associate capture/replay handles with stages
For stages where the capture/replay handle is only known after compiling
and uploading the shader, the shader needs to be relocated to the VA
corresponding to the capture/replay address.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:25 +00:00
Friedrich Vock
eee0068943 radv/rt: Only compare the non-recursive capture/replay handle
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:25 +00:00
Friedrich Vock
59d269c48e radv: Add radv_rt_capture_replay_handle
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:25 +00:00
Friedrich Vock
e3bd54d2a8 radv: Add support for creating capture/replay shaders
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:25 +00:00
Friedrich Vock
4f192b9af4 radv: Split up implementation of radv_shader_create
This will make it easy to re-use the split-up parts for creating
replayed shaders.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:25 +00:00
Friedrich Vock
51f2fa1a5e radv: Break up radv_shader_nir_to_asm
radv_shader_nir_to_asm actually had 3 functions: compiling the NIR to
asm, uploading the shaders and generating debug info for them.
This reduces the functionality of radv_shader_nir_to_asm to only compile
NIR to asm. Uploading the shader and generating debug info is split into
separate functions.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:25 +00:00
Friedrich Vock
878a731c77 radv: Add radv_shader_reupload
Used for relocating RT shaders with capture/replay.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:24 +00:00
Friedrich Vock
744357477e radv: Add utilities to serialize and deserialize shader allocation info
Can be used to capture/replay an arbitrary sequence of shader
allocations while preserving VAs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:24 +00:00
Friedrich Vock
d23e41de6c radv: Add option to allocate shaders in replayable VA range
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:24 +00:00
Friedrich Vock
ec9f5b7777 radv: Move shader arena allocation to a separate function
The arena size is also needed for capture/replay.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:24 +00:00
Friedrich Vock
91241014e8 radv: Add radv_shader_free_list
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23516>
2023-07-05 15:58:24 +00:00
Joshua Ashton
59cf3702c4 radv: Remove unused pipeline param from radv_generate_pipeline_key
Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23912>
2023-07-04 21:05:16 +01:00
Joshua Ashton
5ca8390e5f radv: Advertise VK_EXT_pipeline_robustness
Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23912>
2023-07-04 21:05:09 +01:00
Joshua Ashton
c9ac960b6a radv: Implement VK_EXT_pipeline_robustness
Funnel the data from the pNext into the new pipeline key members for buffer robustness

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23912>
2023-07-04 21:04:38 +01:00
Joshua Ashton
3e269c7a4e radv: Rename radv_required_subgroup_info to radv_shader_stage_key
Additionally, prefix the members that are subgroup related with subgroup_

We will use this structure to store pipeline robustness information.

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23912>
2023-07-04 21:04:34 +01:00
Joshua Ashton
2c6125c1c5 radv: Split and move buffer robustness to shader key
Will be used by pipeline robustness.

There is also loads of space in the first dword.

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23912>
2023-07-04 21:04:30 +01:00
Joshua Ashton
3e36b47b96 radv: Rename radv_nir_compiler_options::robust_buffer_access to robust_buffer_access_llvm
This is only used by LLVM, make that clearer.

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23912>
2023-07-04 21:04:21 +01:00
Joshua Ashton
f52ebc8b40 radv: Refactor buffer robustness to an enum
Will be taken advantage of with pipeline robustness.

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23912>
2023-07-04 21:04:10 +01:00
Yonggang Luo
0fbec6dd79 radv: Use nir_remove_non_entrypoints in radv_shader.c
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23902>
2023-07-03 21:45:35 +00:00
Konstantin Seurer
20beebb041 amd: Move ac_hw_stage to its own file
Otherwise ACO has to include ac_shader_util.h which also includes NIR.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23906>
2023-07-03 21:12:45 +00:00
Connor Abbott
d322a80170 radv: Fix radv_pipeline_is_blend_enabled
This was relying on cb being NULL instead of just gracefully handling
it, and it will stop being NULL once we start tracking attachment count
as state. Moreover is was broken in the case where only the blend enable
is dynamic.

Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22301>
2023-07-03 19:51:05 +00:00
Tatsuyuki Ishi
94b92718d9 radv: Enable weak reference cache for device->mem_cache.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23878>
2023-07-03 17:59:07 +00:00
Daniel Schürmann
3bd72eec1e radv: reference pipeline cache object in radv_pipeline
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23878>
2023-07-03 17:59:07 +00:00
Konstantin Seurer
1ea963c00b radv: Use nir_builder_at
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23883>
2023-07-03 15:21:37 +00:00
Rhys Perry
b3b9b22c95 radv: call nir_lower_int64 later
I would rather this be after several of these complicated lowering passes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23926>
2023-07-03 10:38:27 +00:00
Rhys Perry
38cff03e58 radv: use nir_lower_conv64
This has a more accurate i2f/u2f implementation.

fossil-db (navi21):
Totals from 342 (0.26% of 133461) affected shaders:
MaxWaves: 10480 -> 10472 (-0.08%)
Instrs: 273455 -> 274019 (+0.21%); split: -0.02%, +0.22%
CodeSize: 1545020 -> 1546852 (+0.12%); split: -0.11%, +0.23%
VGPRs: 9528 -> 9552 (+0.25%)
SpillSGPRs: 553 -> 592 (+7.05%); split: -2.71%, +9.76%
SpillVGPRs: 821 -> 811 (-1.22%); split: -1.71%, +0.49%
Latency: 3837590 -> 3822989 (-0.38%); split: -0.52%, +0.14%
InvThroughput: 1284512 -> 1277012 (-0.58%); split: -0.77%, +0.19%
VClause: 6480 -> 6486 (+0.09%); split: -0.15%, +0.25%
SClause: 6640 -> 6645 (+0.08%); split: -0.02%, +0.09%
Copies: 45858 -> 46160 (+0.66%); split: -0.10%, +0.76%
Branches: 8736 -> 8741 (+0.06%); split: -0.01%, +0.07%
PreSGPRs: 14320 -> 14323 (+0.02%)
PreVGPRs: 8545 -> 8551 (+0.07%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9275
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23926>
2023-07-03 10:38:27 +00:00
Timur Kristóf
bd43d9e3d9 radv: Enable required subgroup size on mesh/task.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23925>
2023-07-03 10:49:11 +02:00
Timur Kristóf
34ace6688a radv: Use required subgroup info for graphics shaders.
We plan to allow setting this on some graphics shader stages.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23925>
2023-07-03 10:49:00 +02:00
Timur Kristóf
8982bd6045 radv: Refactor required subgroup size in pipeline key.
This is to allow setting required subgroup size and
full subgroups on more than just the compute stage.

Use an enum (not the actual subgroup size integer)
so that we can have some bits reserved there for
future use.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23925>
2023-07-03 10:47:39 +02:00
Samuel Pitoiset
b544a6d6c3 radv: emit PA_SC_SCREEN_SCISSOR_BR with the actual fb extent
For some reasons, this register is needed for RGP to report actual
render/depth targets size instead of 0 for both width/height. It
doesn't seem to have any other effects.

Gitlab: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9169
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23921>
2023-06-30 06:38:53 +00:00
Juston Li
33ee59af1d radv: fix incorrect size for primitives generated query
Primitives generated queries write 1 integer, the primitives-generated
count that is incremented every time a primitive emitted to that stream
reaches the transform feedback stage.

Fixes: 1ebf463a5a ("radv: implement VK_EXT_primitives_generated_query")
Signed-off-by: Juston Li <justonli@google.com>
Reviewed-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23915>
2023-06-29 16:58:50 +00:00
Georg Lehmann
130fbda71b radv: set has_bit_test for aco
Foz-DB Navi21:
Totals from 15285 (11.52% of 132657) affected shaders:
VGPRs: 1019136 -> 1019000 (-0.01%); split: -0.02%, +0.01%
SpillSGPRs: 10889 -> 10909 (+0.18%)
SpillVGPRs: 901 -> 914 (+1.44%); split: -0.89%, +2.33%
CodeSize: 103578640 -> 103523220 (-0.05%); split: -0.08%, +0.03%
MaxWaves: 259782 -> 259820 (+0.01%)
Instrs: 19247383 -> 19223764 (-0.12%); split: -0.15%, +0.02%
Latency: 323877613 -> 323684655 (-0.06%); split: -0.10%, +0.04%
InvThroughput: 62505295 -> 62386541 (-0.19%); split: -0.21%, +0.02%
VClause: 366162 -> 366136 (-0.01%); split: -0.03%, +0.02%
SClause: 786505 -> 785527 (-0.12%); split: -0.22%, +0.10%
Copies: 1348920 -> 1349209 (+0.02%); split: -0.26%, +0.29%
Branches: 456331 -> 456324 (-0.00%); split: -0.01%, +0.00%
PreSGPRs: 849542 -> 849402 (-0.02%); split: -0.02%, +0.01%
PreVGPRs: 925300 -> 924678 (-0.07%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23298>
2023-06-29 13:39:30 +00:00
Erik Faye-Lund
a6da9d7147 amd: use imm-helpers
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23855>
2023-06-29 07:08:19 +00:00
Joshua Ashton
68b9ad0ba7 radv: Do not enable robustness for push constants with robustBufferAccess2
There is no spec text requiring this behaviour, it is only for buffers.

Signed-off-by: Joshua Ashton <joshua@froggi.es>

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23885>
2023-06-28 20:49:30 +00:00
Konstantin Seurer
1e2f647fbb radv/rt: Hash stages using radv_hash_shaders
The hash also depends on the radv_pipeline_key as well as the flags. The
pipeline layout will also play a role when we implement inline
descriptor sets and push constants.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23747>
2023-06-28 19:45:25 +00:00
Konstantin Seurer
de1092e256 radv/rt: Fix caching non-recursive stages
The hash used for insertion is calculated in a different way than the
hash used for lookup.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23747>
2023-06-28 19:45:25 +00:00
Samuel Pitoiset
3f7ea95bc9 radv: inline more values in radv_emit_fb_ds_state()
These are no longer adjusted.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23887>
2023-06-28 13:21:44 +00:00
Samuel Pitoiset
5010ab8fff radv: stop emitting TILE_SURFACE_ENABLE for the ZRANGE_PRECISION workaround
The only case that matters is when the fb is emitted, but HTILE is
already disabled there using DB_RENDER_CONTROL.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23887>
2023-06-28 13:21:44 +00:00
Samuel Pitoiset
7b8c6cedcf radv: allow NV_device_generated_commands with RADV_DEBUG=noibs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23791>
2023-06-28 06:34:20 +00:00
Samuel Pitoiset
277b2afd70 radv/amdgpu: add support for executing DGC cmdbuf with RADV_DEBUG=noibs
This contains some preliminary work to be able to execute DGC cmdbuf
on the compute queue because IB2 doesn't exist.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23791>
2023-06-28 06:34:20 +00:00