Commit graph

10457 commits

Author SHA1 Message Date
Rhys Perry
6113ee650a aco/gfx11: fix FS input loads in quad-divergent control flow
This is not ideal and it would be great to somehow make it better some
day.

fossil-db (gfx1100):
Totals from 5208 (3.86% of 135032) affected shaders:
MaxWaves: 127058 -> 126962 (-0.08%); split: +0.01%, -0.09%
Instrs: 3983440 -> 4072736 (+2.24%); split: -0.00%, +2.24%
CodeSize: 21872468 -> 22230852 (+1.64%); split: -0.00%, +1.64%
VGPRs: 206688 -> 206984 (+0.14%); split: -0.05%, +0.20%
Latency: 37447383 -> 37491197 (+0.12%); split: -0.05%, +0.17%
InvThroughput: 6421955 -> 6422348 (+0.01%); split: -0.03%, +0.03%
VClause: 71579 -> 71545 (-0.05%); split: -0.09%, +0.04%
SClause: 148289 -> 147146 (-0.77%); split: -0.84%, +0.07%
Copies: 259011 -> 258084 (-0.36%); split: -0.61%, +0.25%
Branches: 101366 -> 101314 (-0.05%); split: -0.10%, +0.05%
PreSGPRs: 223482 -> 223460 (-0.01%); split: -0.21%, +0.20%
PreVGPRs: 184448 -> 184744 (+0.16%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19370>
2022-11-01 12:42:43 +00:00
Rhys Perry
16d2c7ad55 aco/gfx11: perform FS input loads in WQM
fossil-db (gfx1100):
Totals from 48184 (35.68% of 135032) affected shaders:
MaxWaves: 1131876 -> 1131960 (+0.01%); split: +0.05%, -0.04%
Instrs: 36755466 -> 36782290 (+0.07%); split: -0.04%, +0.11%
CodeSize: 200812068 -> 200915348 (+0.05%); split: -0.04%, +0.09%
VGPRs: 2163980 -> 2163828 (-0.01%); split: -0.15%, +0.14%
Latency: 484174459 -> 484341018 (+0.03%); split: -0.06%, +0.09%
InvThroughput: 87941284 -> 87944874 (+0.00%); split: -0.04%, +0.04%
VClause: 652984 -> 653085 (+0.02%); split: -0.09%, +0.10%
SClause: 1510995 -> 1528832 (+1.18%); split: -0.40%, +1.58%
Copies: 1997689 -> 2001857 (+0.21%); split: -0.49%, +0.69%
Branches: 676629 -> 676584 (-0.01%); split: -0.02%, +0.01%
PreSGPRs: 2033070 -> 2036725 (+0.18%)
PreVGPRs: 1903922 -> 1903897 (-0.00%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 3730be9873 ("aco: mostly implement FS input loads on GFX11")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19370>
2022-11-01 12:42:43 +00:00
Rhys Perry
3da4fe9c6d aco: fix typo in branch lowering
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: aadb7aef01 ("aco: add VINTERP instruction format")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19370>
2022-11-01 12:42:43 +00:00
Samuel Pitoiset
c2ff7c3eb2 radv: enable VK_EXT_transform_feedback with NGG streamout on GFX11
Untested but it mostly work on GFX10.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19435>
2022-11-01 11:28:45 +01:00
Samuel Pitoiset
75faaac895 radv: allow to enable NGG streamout with RADV_PERFTEST=ngg_streamout
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19435>
2022-11-01 11:28:43 +01:00
Samuel Pitoiset
4621c4936b radv: enable all possible features with NGG streamout
Tested by forcing NGG streamout on GFX10.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19435>
2022-11-01 11:28:41 +01:00
Samuel Pitoiset
d467ff7b0e radv: set the correct buffer size for NGG streamout
If the size is passed through CmdBindTransformFeedback() uses that.

This partially fixes dEQP-VK.transform_feedback.simple.multiquery_1
by reporting the correct number of primitives written (the computation
is based on the buffer size). There is still a bug around GDS offsets
that will be fixed later.

Tested on GFX10.3 by forcing NGG streamout.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19433>
2022-11-01 10:41:36 +01:00
Samuel Pitoiset
76555a4777 radv: fix primitives generated query with NGG only
According to the AMD registers database, SAMPLE_STREAMOUTSTATS no
longer exists on GFX11. This fixes primitives generated query if only
the NGG path is used. Tested on GFX10.3 by forcing NGG everywhere.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19410>
2022-11-01 07:26:31 +00:00
Samuel Pitoiset
8dafda4a07 radv: suspend/resume primitives generated query separately for legacy/NGG
With NGG only, only the GDS query counter will be incremented.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19410>
2022-11-01 07:26:31 +00:00
Samuel Pitoiset
e83ec595df radv: use 32-bit GDS counters for primitives generated query with NGG
Do not need to use 64-bit operations.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19410>
2022-11-01 07:26:31 +00:00
Samuel Pitoiset
1e925ce565 radv: reword an incorrect comment about primitives generated query
This is not the XFB counter.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19410>
2022-11-01 07:26:31 +00:00
Konstantin Seurer
1c7d208302 radv/rt: Set vars.arg in the traversal shader
Fixes: f4270b7 ("radv/rt: create traversal shader independent from main shader")
Closes: #7591
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19402>
2022-10-31 17:25:55 +00:00
Michel Dänzer
6fd1414016 radeonsi/ci: Test piglit quick_shader profile as well
It's useful for catching regressions, e.g. it would have caught
https://gitlab.freedesktop.org/mesa/mesa/-/issues/7567 .

For the radeonsi-stoney-gl:amd64 job, we can use the "gpu" profile,
which contains quick_gl and quick_shader.

For the radeonsi-raven-piglit-gl:amd64 job, that would almost double
the runtime, and bring it dangerously close to the 30 minutes timeout.
Instead, split it up into radeonsi-raven-piglit-quick_gl:amd64 and
radeonsi-raven-piglit-quick_shader:amd64 jobs.

v2:
* Rebase, drop spec@arb_tessellation_shader@execution flakes and add
  one crash instead.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19378>
2022-10-31 16:58:26 +00:00
Rhys Perry
b7ea47ede6 radv,aco: don't use lower_to_fragment_fetch_amd on GFX11+
FMask doesn't exist on GFX11. Have txf_ms take the fragment_fetch_amd
path.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19375>
2022-10-31 16:26:30 +00:00
Rhys Perry
b9a3d8b0ab radv/gfx11: don't create texop_samples_identical
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19375>
2022-10-31 16:26:30 +00:00
Rhys Perry
d31e5c8efc radv/gfx11: don't create fmask copy/expand pipelines
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19375>
2022-10-31 16:26:30 +00:00
Rhys Perry
140cefe95a ac/nir: lower gfx11 vertex parameter exports
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19228>
2022-10-31 14:33:43 +00:00
Rhys Perry
14a1925727 aco: don't split swizzled store_buffer_amd on GFX9+
This isn't necessary.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19228>
2022-10-31 14:33:43 +00:00
Rhys Perry
e6d26cb288 nir,ac/nir,aco,radv: replace has_input_*_amd with more general intrinsics
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19228>
2022-10-31 14:33:43 +00:00
Samuel Pitoiset
9f692f5aa6 radv: do not unconditionally disable NGG streamout lowering in NIR
This is still always disabled because use_ngg_streamout is FALSE
but it will be turned on at some point.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Samuel Pitoiset
d653ca4c52 radv: lower NIR intrinsics for NGG streamout
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Samuel Pitoiset
3bafe6e0bc radv: call nir_io_add_intrinsic_xfb_info() after IO lowering
This is needed for NGG streamout which gets the XFB info directly
from intrinsics.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Samuel Pitoiset
ed865440db radv/llvm: prevent emitting streamout outputs for NGG
They are directly emitted from NIR.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Samuel Pitoiset
fa287c99ed aco: remove invalid assertions for NGG streamout
Streamout outputs are directly emitted from NIR now.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Samuel Pitoiset
db7ffa4006 aco: implement NIR intrinsics for NGG streamout
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Samuel Pitoiset
fdc212bd7b aco: create a new builder variant for ds_add_rtn
This instruction can use 1 definition and 3 operands.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Bas Nieuwenhuizen
5d04064642 radv: Handle attribute ring intrinsic correctly with LLVM.
Again, if we don't set progress to false we get fun stuff.

Fixes: 8bf1aa1b76 ("radv: add lowering for nir_intrinsic_load_ring_attr_{offset}_amd")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19404>
2022-10-31 13:17:04 +00:00
Bas Nieuwenhuizen
45ff58cfd1 radv: Handle GSVS ring intrinsic correctly with LLVM.
If we don't set progress to false we get a mess as a replacement is
still attempted.

Fixes: 382831c986 ("radv,nir: add intrinsics for streamout and GS copy shaders")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19404>
2022-10-31 13:17:04 +00:00
Bas Nieuwenhuizen
ec9d71498e radv: Use correct types for loading the rings with LLVM.
Ring descriptors are v4i32, not i8.

Fixes: cb117cdc96 ("radv/llvm: use ac_build_gep0_type to get args types")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19404>
2022-10-31 13:17:04 +00:00
Samuel Pitoiset
d4ec3f21cf Revert "radv: add a pointer to radv_shader_binary in radv_shader"
This is actually not necessary because we compile and upload binaries
directly from libraries with GPL. This introduced random double free
crashes because binaries were potentially freed by concurrent threads.

Root cause found by Ishi.

This reverts commit f8d887527a.

Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19383>
2022-10-31 12:16:38 +00:00
Georg Lehmann
2eac571d61 aco: Use opsel for the third operand.
Foz-DB Navi21:
Totals from 2 (0.00% of 134913) affected shaders:
CodeSize: 7788 -> 7772 (-0.21%)
Instrs: 1305 -> 1303 (-0.15%)
Latency: 7175 -> 7163 (-0.17%)
InvThroughput: 2082 -> 2078 (-0.19%)
Copies: 57 -> 55 (-3.51%)

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19380>
2022-10-31 09:54:01 +00:00
Samuel Pitoiset
25e311e9d3 radv: implement transform feedback queries with NGG streamout
The control bit is written to the upper bits because GDS counters
are 32-bits only, this allows to re-use the existing query shader.

Tested on GFX10.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19325>
2022-10-31 08:22:29 +00:00
Bas Nieuwenhuizen
78519987b9 radv: Speculatively tune RT pipelines for GFX11.
With ACO not supporting VOPD and the high number of SALU instructions,
we're likely better off using wave64 until we can actually benchmark
this and fix these issues.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19288>
2022-10-31 02:39:34 +00:00
Bas Nieuwenhuizen
9369b40725 radv: Use PLOC for BVH building
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Bas Nieuwenhuizen
271865373e radv: Add PLOC shader
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
14dfb6035f radv: Add REF as a typename macro to .clang-format
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
0c0f179037 radv: Add global sync utilities
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
608fa1bd25 radv/rt: Track number of inactive leaf nodes
To avoid emitting nodes with only invalid children in PLOC.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
f502b3aab3 radv/rt: Dispatch internal converter indirectly
Preparation for using the converter with PLOC.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
49c0995918 radv/rt: Fix internal converter synchronization
Fixes: e83e4faf ("radv: Only emit parents from parents that actually end up in the tree.")
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
fa578f280e radv: Add radv_indirect_unaligned_dispatch
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
030a1f6843 radv: Use a struct for AABBs
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Bas Nieuwenhuizen
ccf0a69e05 radv: Make the number of internal nodes be written on the GPU.
Opens the door of algorithms with a variable number of nodes.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Bas Nieuwenhuizen
0e23df959e radv: Add BVH IR header.
To include GPU state passed between stages but not in a node.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Friedrich Vock
37525c11d1 radv: Rename emulated float helpers
Use only conversion functions now.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19292>
2022-10-30 19:48:46 +00:00
Marek Olšák
0ac37b595a nir: add nir_intrinsic_optimization_barrier_vgpr_amd for LLVM
We need this for the MSAA resolve shader.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19243>
2022-10-29 18:38:33 +00:00
Rhys Perry
7fa50ced14 aco: insert waitcnt before/after ds_ordered_count
The LLVM backend does this when lowering ordered_xfb_counter_add_amd. I
guess there is some missing dependency checking or something.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19345>
2022-10-28 21:50:05 +00:00
Rhys Perry
ea8ddf5c26 aco: add storage_gds
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19345>
2022-10-28 21:50:05 +00:00
Samuel Pitoiset
eae2867122 radv: move nir_opt_idiv_const/nir_lower_idiv after NGG lowering
NGG streamout lowering creates some idiv instructions that need to be
lowered.

No fossil-db results because it's currently broken.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19364>
2022-10-28 18:19:57 +00:00
Samuel Pitoiset
e2fcbd4a37 radv/llvm: fix dual source blending on GFX11
Untested but this should be similar to RadeonSI.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19367>
2022-10-28 17:03:37 +00:00