Commit graph

6088 commits

Author SHA1 Message Date
Friedrich Vock
f502b3aab3 radv/rt: Dispatch internal converter indirectly
Preparation for using the converter with PLOC.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
49c0995918 radv/rt: Fix internal converter synchronization
Fixes: e83e4faf ("radv: Only emit parents from parents that actually end up in the tree.")
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
fa578f280e radv: Add radv_indirect_unaligned_dispatch
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
030a1f6843 radv: Use a struct for AABBs
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Bas Nieuwenhuizen
ccf0a69e05 radv: Make the number of internal nodes be written on the GPU.
Opens the door of algorithms with a variable number of nodes.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Bas Nieuwenhuizen
0e23df959e radv: Add BVH IR header.
To include GPU state passed between stages but not in a node.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
37525c11d1 radv: Rename emulated float helpers
Use only conversion functions now.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Samuel Pitoiset
eae2867122 radv: move nir_opt_idiv_const/nir_lower_idiv after NGG lowering
NGG streamout lowering creates some idiv instructions that need to be
lowered.

No fossil-db results because it's currently broken.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19364>
2022-10-28 18:19:57 +00:00
Samuel Pitoiset
e2fcbd4a37 radv/llvm: fix dual source blending on GFX11
Untested but this should be similar to RadeonSI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19367>
2022-10-28 17:03:37 +00:00
Samuel Pitoiset
d172fc1fca radv: fix VRS limit when attachmentFragmentShadingRate is disabled
Can be reproduced on GFX10.3 with RADV_DEBUG=nohiz.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19374>
2022-10-28 16:30:29 +00:00
Samuel Pitoiset
c41997f29f radv: fix suspending/resuming pipeline statistics queries with GDS
This probably doesn't fix anything in practice because GDS is only
used for the number of generated primitives by GS and meta operations
don't use GS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19348>
2022-10-28 08:12:30 +00:00
Alyssa Rosenzweig
941c37c085 nir/lower_idiv: Remove imprecise_32bit_lowering
NIR has two implementations of lower_idiv, keyed on the
imprecise_32bit_lowering flag. This flag is misleading: the results when
setting this flag "imprecise", they're completely wrong for some values.
If a backend has a native implementation of umul_high, the correct path
isn't that much more expensive. If it doesn't, it's substantially slower
for highp integer divison... but in practice, non-constant highp integer
division is pretty rare.

After a painful migration of the tree, this code path has no more users.
Remove it so nobody else gets the bright idea of using it again.

Closes: #6555
Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19303>
2022-10-27 19:37:14 +00:00
Rhys Perry
93fb84237f ac/nir: add ac_nir_lower_ngg_options
These signatures were getting ridiculous.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19340>
2022-10-27 13:31:40 +00:00
Daniel Schürmann
c80137fcba radv/rt: overwrite hit args with undef in case of a miss
This helps some variable coalescing.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19188>
2022-10-27 09:45:39 +00:00
Daniel Schürmann
f4270b7659 radv/rt: create traversal shader independent from main shader
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19188>
2022-10-27 09:45:39 +00:00
Iago Toral Quiroga
9deef4cde6 vulkan/runtime: include robustness info when hashing a shader stage
Suggested by Jason Ekstrand.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18883>
2022-10-27 08:17:11 +00:00
Qiang Yu
e536d0fe4b ac/nir/ngg,radv: move LDS layout calculation out of nir ngg lowering
Use lds base load intrinsics in nir ngg lowering to get layout, left
its calulation to driver.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18832>
2022-10-27 07:35:01 +00:00
Qiang Yu
54eea0e393 ac/nir/ngg: pass primitive_id_location as param for nogs lower
radeonsi need to use packed driver location for all outputs,
while radv need to use VARYING_SLOT_*. To meet both drivers'
needs.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18832>
2022-10-27 07:35:01 +00:00
Lionel Landwerlin
53a0804146 radv: tweak lower_shader_calls parameters
On Q2RTX shaders :

MaxWaves: 62 -> 69 (+11.29%)
Instrs: 41626 -> 41575 (-0.12%); split: -0.27%, +0.15%
CodeSize: 224960 -> 223740 (-0.54%); split: -0.62%, +0.08%
VGPRs: 800 -> 704 (-12.00%)
Scratch: 75776 -> 70656 (-6.76%)
Latency: 922219 -> 977997 (+6.05%)
InvThroughput: 212154 -> 201746 (-4.91%); split: -5.54%, +0.64%
VClause: 1120 -> 1155 (+3.12%); split: -1.88%, +5.00%
SClause: 1148 -> 1144 (-0.35%); split: -0.70%, +0.35%
Copies: 5840 -> 5788 (-0.89%); split: -0.94%, +0.05%
PreVGPRs: 753 -> 651 (-13.55%)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>
2022-10-26 12:53:26 +00:00
Lionel Landwerlin
1d10d17817 nir/lower_shader_calls: add an option structure for future optimizations
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16556>
2022-10-26 12:53:25 +00:00
Rhys Perry
382831c986 radv,nir: add intrinsics for streamout and GS copy shaders
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19302>
2022-10-25 17:35:08 +00:00
Qiang Yu
188a7f9226 ac/nir/ngg: add query param to ac_nir_lower_ngg_gs
radeonsi may disable it. gfx_level will also be used by latter
vertex param export when gfx11.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17457>
2022-10-25 12:58:43 +00:00
Qiang Yu
a119a6464f nir,ac,radv: add primitive count add intrinsics
radeonsi use shader buffer, but radv use gds for the query
result storage.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17457>
2022-10-25 12:58:43 +00:00
Samuel Pitoiset
e18f76d890 radv: disable dual source blending in more situations
According to PAL, there is more restrictions that RADV doesn't have.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19278>
2022-10-25 12:22:34 +00:00
Qiang Yu
7ee0b8b8df ac/nir/ngg,radv: use different counters for shader queries
VK_QUERY_TYPE_PRIMITIVES_GENERATED_EXT should count for each
stream.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7409
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19015>
2022-10-25 02:42:52 +00:00
Qiang Yu
83643e4dc8 nir,ac/nir/ngg,radv: split shader_query_enabled_amd
For used by different counter.

Vulkan:
1. VK_QUERY_PIPELINE_STATISTIC_GEOMETRY_SHADER_PRIMITIVES_BIT,
   sum generated primitives of all 4 streams when GS.
2. VK_QUERY_TYPE_PRIMITIVES_GENERATED_EXT, count generated
   primitives for all 4 streams when VS/TES/GS.
3. VK_QUERY_TYPE_TRANSFORM_FEEDBACK_STREAM_EXT, count generated
   and streamout primitives for all 4 streams when VS/TES/GS.

OpenGL:
1. GL_GEOMETRY_SHADER_PRIMITIVES_EMITTED_ARB, sum generated
   primitives for all 4 streams when GS.
2. GL_PRIMITIVES_GENERATED, count generated primitives for all 4
   streams when VS/TES/GS.
3. GL_TRANSFORM_FEEDBACK_PRIMITIVES_WRITTEN, count streamout
   primitives for all 4 streams when VS/TES/GS.

pipeline_stat_query_enabled_amd is for Vulkan 1 and OpenGL 1.
xfb_query_enabled_amd is for Vulkan 2/3 and OpenGL 2/3.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19015>
2022-10-25 02:42:52 +00:00
Qiang Yu
1dcbf25757 radv: split active_pipeline_gds_queries
For different enabling of pipeline stat and prims gen.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19015>
2022-10-25 02:42:52 +00:00
Qiang Yu
0bbe8029b6 radv: count gen_prims_queries_enabled
User can enable/disable multi VK_QUERY_TYPE_PRIMITIVES_GENERATED_EXT
queries with same or different index.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19015>
2022-10-25 02:42:52 +00:00
Georg Lehmann
31c1cb06ad radv: Use available dot product instructions on gfx11.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19114>
2022-10-24 19:07:16 +00:00
Konstantin Seurer
ac01f09d57 radv/rt: Load instance id and custom index on demand
Stats for Quae II RTX:

57fps -> 57fps

Totals from 7 (14.00% of 50) affected shaders:
VGPRs: 800 -> 784 (-2.00%)
CodeSize: 217868 -> 218308 (+0.20%)
MaxWaves: 62 -> 63 (+1.61%)
Instrs: 40384 -> 40420 (+0.09%); split: -0.01%, +0.10%
Latency: 866315 -> 870692 (+0.51%)
InvThroughput: 199189 -> 196595 (-1.30%); split: -1.75%, +0.45%
VClause: 1058 -> 1077 (+1.80%)
SClause: 1126 -> 1130 (+0.36%)
Copies: 5787 -> 5772 (-0.26%); split: -0.40%, +0.14%
PreVGPRs: 764 -> 750 (-1.83%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19159>
2022-10-24 14:39:25 +00:00
Samuel Pitoiset
cf7ada973a radv: disable shaderBufferFloat64AtomicMinMax on GFX11
buffer_atomic_{fmin,fmax}_x2 aren't supported on this GPU apparently.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19268>
2022-10-24 12:40:33 +00:00
Samuel Pitoiset
7298bc1810 radv: disable {shader,sparse}ImageFloat32AtomicMinMax on GFX11
image_atomic_{fmin,fmax} aren't supported on this GPU apparently.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19268>
2022-10-24 12:40:33 +00:00
Bas Nieuwenhuizen
514a9b3b75 radv: Use correct primgrp size for gfx11.
From radeonsi, found by inspection.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19222>
2022-10-24 09:08:42 +00:00
Bas Nieuwenhuizen
9d782e5294 radv: Improve gfx11 clear colors for DCC.
When I wrote the function I did unorms wrong. Let's copy radeonsi
harder.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19222>
2022-10-24 09:08:42 +00:00
Samuel Pitoiset
152b90efcd aco,radv/llvm: do not export parameters on GFX11
They will be exported through the attribute ring instead.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19216>
2022-10-24 07:55:06 +02:00
Samuel Pitoiset
c5bf63753d ac/nir,radv/llvm: add support for the attribute ring
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19216>
2022-10-24 07:55:06 +02:00
Samuel Pitoiset
fee5e379f4 radv/llvm: do not set COMPR for exports on GFX11
Use 0x3 channel mask instead.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19216>
2022-10-24 07:53:59 +02:00
Georg Lehmann
125741dbae nir/opt_algebraic: Optimize various find_msb_rev patterns.
From dxvk, dxil-spirv, fxc, dxc and others.

Totals from 177 (0.13% of 134913) affected shaders:
CodeSize: 1079504 -> 1059872 (-1.82%)
Instrs: 195381 -> 192269 (-1.59%)
Latency: 3664137 -> 3631951 (-0.88%)
InvThroughput: 599479 -> 585675 (-2.30%)

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18951>
2022-10-22 11:57:33 +02:00
Georg Lehmann
d57f5c9cac radv,aco: Lower uclz in NIR.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18951>
2022-10-22 11:57:23 +02:00
Konstantin Seurer
b5ecb5388e radv/rt: Fix setting tmax for opaque AABBs
data->vars->tmax will be overwritten by inner_vars.tmax so we have to
set inner_vars.tmax instead. This probably doesn't affect anything in
practise.

cc: mesa-stable

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19207>
2022-10-22 08:55:58 +00:00
Samuel Pitoiset
897adcf7d7 radv: disable AMD_shader_explicit_vertex_parameter on GFX11
Not supported by ACO.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19230>
2022-10-21 16:42:12 +00:00
Samuel Pitoiset
c18e39ebca radv: disable VK_AMD_texture_gather_bias_lod on GFX11
image_gather4_b_o no longer exists apparently, so this is likely
broken.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19230>
2022-10-21 16:42:12 +00:00
Samuel Pitoiset
59be3b8a8f radv: advertise VK_EXT_extended_dynamic_state3
Some tricky dynamic states still need to be implemented for full ds3
support with Zink, I will implement them later.

Pass dEQP-VK.pipeline.*extended_dynamic_state* on NAVI21.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18882>
2022-10-21 15:56:52 +00:00
Samuel Pitoiset
e48c0fbd8f radv: add support for dynamic depth clamp enable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18882>
2022-10-21 15:56:52 +00:00
Samuel Pitoiset
31f946f4d2 radv: add support for dynamic provoking vertex mode
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18882>
2022-10-21 15:56:52 +00:00
Samuel Pitoiset
e26d3cd9e8 radv: add support for dynamic depth clip negative one to one
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18882>
2022-10-21 15:56:52 +00:00
Samuel Pitoiset
fbed3aed4a radv: add support for dynamic conservative rasterization mode
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18882>
2022-10-21 15:56:52 +00:00
Samuel Pitoiset
96282ceb9a radv: add support for dynamic depth clip enable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18882>
2022-10-21 15:56:52 +00:00
Samuel Pitoiset
8682e09c33 radv: add support for dynamic sample mask
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18882>
2022-10-21 15:56:52 +00:00
Samuel Pitoiset
7c38d94261 radv: add support for dynamic alpha to coverage enable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18882>
2022-10-21 15:56:52 +00:00