Commit graph

6142 commits

Author SHA1 Message Date
Samuel Pitoiset
0bf06400be radv: consider allocated command buffers in the initial state.
The Vulkan spec says:
    "When a command buffer is allocated, it is in the initial state.
     Some commands are able to reset a command buffer (or a set of
     command buffers) back to this state from any of the executable,
     recording or invalid state. Command buffers in the initial state
     can only be moved to the recording state, or freed."

Because the status wasn't initialized, it was implicitly set to
RADV_CMD_BUFFER_STATUS_INVALID and that triggered a reset for newly
allocated command buffers.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19506>
2022-11-07 12:02:36 +01:00
Marek Olšák
8956682810 amd: rename enums ARCTURUS -> MI100, ALDEBARAN -> MI200
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19477>
2022-11-06 17:20:39 -05:00
Konstantin Seurer
c37357f4e0 radv/rra: Remove some node type validation
Node types can only be invalid for certain acceleration structure types.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19544>
2022-11-06 17:03:52 +00:00
Konstantin Seurer
868107e16c radv/rra: Improve validation message formatting
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19544>
2022-11-06 17:03:52 +00:00
Konstantin Seurer
07bf85c2ec radv/rra: Rename rra_accel_struct_validation_fail
...to rra_validation_fail since it is used quite often.
Shortening the name should improve readability.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19544>
2022-11-06 17:03:52 +00:00
Konstantin Seurer
e5b3efe582 radv/rt: Restore prev barycentrics when rejecting hits
Closes: #6348
cc: mesa-stable

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19534>
2022-11-06 11:36:10 +00:00
Bas Nieuwenhuizen
9a8453d07e radv: Use compares for node type in traversal.
The HW has no bit test instruction, so we change 3 pairs of and+cmp
to a single and + 3 cmps, saving 2 VALU instructions.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19454>
2022-11-05 16:38:48 +00:00
Samuel Pitoiset
06adf6ad11 radv: invalidate L2 instead of only writeback L2 when using DCC stores
It seems INV_L2 is the right thing to do, especially for RDNA2 chips
with non-coherent RBs (NAVI22 is one of these). This fixes DCC
corruption.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6476
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7507
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19516>
2022-11-04 22:40:11 +00:00
Samuel Pitoiset
26c8fedc1b radv: make the GDS/GDS OA buffer objects resident
GDS is used for NGG queries/streamout (GFX10+ only) and the BOs were
only added to the graphics queue because compute doesn't need them.
Though, the kernel emits a GDS switch when a queue submission doesn't
use GDS. That means that submitting jobs on the compute queue without
GDS can reset the state of the graphics queue and lead to GPU hangs.

The only viable solution for now is to make the GDS BOs resident to
avoid resetting the state between queues. This shouldn't introduce
more syncs between queues because GDS BOs are similar for both.

This fixes a GPU hang with Warhammer Chaosbane during loading time and
possibly some spurious random GPU hangs. Note that this GPU hang was
workarounded on the Steam side with RADV_DEBUG=nongg.

Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19466>
2022-11-04 21:52:35 +00:00
Samuel Pitoiset
e891e84f4b radv: fix setting MIN_LOD for texture descriptors on GFX11
Found by inspection because the MIN_LOD bits were moved.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19496>
2022-11-04 06:43:04 +00:00
Marek Olšák
ca29d9a67c ac: fix IB parsing for Stoney
We need to pass the family to register parsing functions.

Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19419>
2022-11-04 00:42:08 +00:00
Marek Olšák
a54c8cdaf2 ac/gpu_info: simplify how has_accelerated_dot_product is set
Reviewed-by: Mihai Preda <mhpreda@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19419>
2022-11-04 00:42:08 +00:00
Friedrich Vock
95ed033066 radv: Use spirv1.5 instead of vulkan1.2
Ubuntu 20.04 ships a glslangValidator that doesn't know about vulkan1.2 yet.

Fixes: 27186537 ("radv: Add PLOC shader")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19479>
2022-11-03 22:58:47 +00:00
Konstantin Seurer
6da2320a92 radv/rra: Print invalid node types
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19465>
2022-11-03 21:25:04 +00:00
Samuel Pitoiset
fab87b0f41 radv: fix fallback for extreme geometry with tessellation on GFX11
It would assert anyways. Found by inspection.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19495>
2022-11-03 11:27:53 +00:00
Bas Nieuwenhuizen
d876ddc920 radv: Make the compute scratch waves per SE as well.
Fixes: 278e533ec9 ("radv: update scratch buffer registers on GFX11")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19488>
2022-11-03 10:03:44 +00:00
Bas Nieuwenhuizen
b8865ad046 radv: Fix compute scratch buffer emission.
Copied wrong from radeonsi. The registers following the scratch
buffer address are the shader rsrc1/rsrc2. Not the user SGPR0
containing the ring resource word 1.

Fixes: 278e533ec9 ("radv: update scratch buffer registers on GFX11")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19488>
2022-11-03 10:03:44 +00:00
Bas Nieuwenhuizen
32cf10e17a radv: Style cleanup.
Now have a local info ptr.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19488>
2022-11-03 10:03:44 +00:00
Rhys Perry
254f4fa573 radv: increase gfx1100/gfx1101 physical vgprs
https://reviews.llvm.org/D134522

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18825>
2022-11-02 17:09:32 +00:00
Rhys Perry
a71d068fd0 radv/llvm: fix GS shaders on GFX8/9
6698753cdb switched our GS output stores to use MUBUF.

The stride doesn't matter for the ESGS descriptor (because idxen=false and
the index stride is 64), but this fixes it anyway.

This also changes ACO to use MUBUF store too, since MTBUF doesn't seem to
work correctly with an invalid data format in the descriptor.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Fixes: 6698753cdb ("ac/llvm: don't use tbuffer_store as a fallback for swizzled stores")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18885>
2022-11-02 12:48:01 +00:00
Illia Abernikhin
aa4ac5ff8b utils: Merge util/debug.* into util/u_debug.* and remove util/debug.*
Rename env_var_as_unsigned() -> debug_get_num_option(), because duplicate
Rename env_var_as_bool() -> debug_get_bool_option(), because duplicate

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7177

Signed-off-by: Illia Abernikhin <illia.abernikhin@globallogic.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19336>
2022-11-02 07:25:39 +00:00
Kenneth Graunke
fde99747e9 nir: Drop infer_non_readable option for nir_opt_access()
Everybody sets it to true now, and the only reason for the option to
exist was to work around a bug that's now been fixed.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19162>
2022-11-02 03:42:04 +00:00
Samuel Pitoiset
53a8dd1d42 radv: move GDS counters after reserved GDS offsets for streamout
Otherwise, queries might return invalid data because they used
the same offsets as NGG streamout.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19442>
2022-11-01 19:47:07 +00:00
Samuel Pitoiset
0bccf8f773 radv: use defines instead of magic values for GDS counters offset
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19442>
2022-11-01 19:47:07 +00:00
Samuel Pitoiset
d2f3e01f61 radv/amdgpu: add a kernel GDS management workaround for -ENOMEM
The kernel keeps returning -ENOMEM if multiple processes allocate GDS,
this always happen while running VKCTS. This solution is loosely based
on RadeonSI, except that it includes a timeout of 1s to exit the loop.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19440>
2022-11-01 18:23:27 +00:00
Samuel Pitoiset
7e3be0b3ce radv: assign IO var locations for VS/TES with NGG streamout
NGG streamout lowering requires nir_shader::num_outputs to be set to
the total number of outputs in order to compute the pervertex LDS size
correctly. This is wasting LDS memory but it's currently the only viable
solution.

This fixes a bunch of dEQP-VK.transform_feedback.* failures.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19436>
2022-11-01 16:47:13 +00:00
Samuel Pitoiset
c2ff7c3eb2 radv: enable VK_EXT_transform_feedback with NGG streamout on GFX11
Untested but it mostly work on GFX10.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19435>
2022-11-01 11:28:45 +01:00
Samuel Pitoiset
75faaac895 radv: allow to enable NGG streamout with RADV_PERFTEST=ngg_streamout
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19435>
2022-11-01 11:28:43 +01:00
Samuel Pitoiset
4621c4936b radv: enable all possible features with NGG streamout
Tested by forcing NGG streamout on GFX10.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19435>
2022-11-01 11:28:41 +01:00
Samuel Pitoiset
d467ff7b0e radv: set the correct buffer size for NGG streamout
If the size is passed through CmdBindTransformFeedback() uses that.

This partially fixes dEQP-VK.transform_feedback.simple.multiquery_1
by reporting the correct number of primitives written (the computation
is based on the buffer size). There is still a bug around GDS offsets
that will be fixed later.

Tested on GFX10.3 by forcing NGG streamout.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19433>
2022-11-01 10:41:36 +01:00
Samuel Pitoiset
76555a4777 radv: fix primitives generated query with NGG only
According to the AMD registers database, SAMPLE_STREAMOUTSTATS no
longer exists on GFX11. This fixes primitives generated query if only
the NGG path is used. Tested on GFX10.3 by forcing NGG everywhere.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19410>
2022-11-01 07:26:31 +00:00
Samuel Pitoiset
8dafda4a07 radv: suspend/resume primitives generated query separately for legacy/NGG
With NGG only, only the GDS query counter will be incremented.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19410>
2022-11-01 07:26:31 +00:00
Samuel Pitoiset
e83ec595df radv: use 32-bit GDS counters for primitives generated query with NGG
Do not need to use 64-bit operations.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19410>
2022-11-01 07:26:31 +00:00
Samuel Pitoiset
1e925ce565 radv: reword an incorrect comment about primitives generated query
This is not the XFB counter.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19410>
2022-11-01 07:26:31 +00:00
Konstantin Seurer
1c7d208302 radv/rt: Set vars.arg in the traversal shader
Fixes: f4270b7 ("radv/rt: create traversal shader independent from main shader")
Closes: #7591
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19402>
2022-10-31 17:25:55 +00:00
Rhys Perry
b7ea47ede6 radv,aco: don't use lower_to_fragment_fetch_amd on GFX11+
FMask doesn't exist on GFX11. Have txf_ms take the fragment_fetch_amd
path.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19375>
2022-10-31 16:26:30 +00:00
Rhys Perry
b9a3d8b0ab radv/gfx11: don't create texop_samples_identical
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19375>
2022-10-31 16:26:30 +00:00
Rhys Perry
d31e5c8efc radv/gfx11: don't create fmask copy/expand pipelines
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19375>
2022-10-31 16:26:30 +00:00
Rhys Perry
140cefe95a ac/nir: lower gfx11 vertex parameter exports
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19228>
2022-10-31 14:33:43 +00:00
Rhys Perry
e6d26cb288 nir,ac/nir,aco,radv: replace has_input_*_amd with more general intrinsics
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19228>
2022-10-31 14:33:43 +00:00
Samuel Pitoiset
9f692f5aa6 radv: do not unconditionally disable NGG streamout lowering in NIR
This is still always disabled because use_ngg_streamout is FALSE
but it will be turned on at some point.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Samuel Pitoiset
d653ca4c52 radv: lower NIR intrinsics for NGG streamout
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Samuel Pitoiset
3bafe6e0bc radv: call nir_io_add_intrinsic_xfb_info() after IO lowering
This is needed for NGG streamout which gets the XFB info directly
from intrinsics.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Samuel Pitoiset
ed865440db radv/llvm: prevent emitting streamout outputs for NGG
They are directly emitted from NIR.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Bas Nieuwenhuizen
5d04064642 radv: Handle attribute ring intrinsic correctly with LLVM.
Again, if we don't set progress to false we get fun stuff.

Fixes: 8bf1aa1b76 ("radv: add lowering for nir_intrinsic_load_ring_attr_{offset}_amd")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19404>
2022-10-31 13:17:04 +00:00
Bas Nieuwenhuizen
45ff58cfd1 radv: Handle GSVS ring intrinsic correctly with LLVM.
If we don't set progress to false we get a mess as a replacement is
still attempted.

Fixes: 382831c986 ("radv,nir: add intrinsics for streamout and GS copy shaders")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19404>
2022-10-31 13:17:04 +00:00
Bas Nieuwenhuizen
ec9d71498e radv: Use correct types for loading the rings with LLVM.
Ring descriptors are v4i32, not i8.

Fixes: cb117cdc96 ("radv/llvm: use ac_build_gep0_type to get args types")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19404>
2022-10-31 13:17:04 +00:00
Samuel Pitoiset
d4ec3f21cf Revert "radv: add a pointer to radv_shader_binary in radv_shader"
This is actually not necessary because we compile and upload binaries
directly from libraries with GPL. This introduced random double free
crashes because binaries were potentially freed by concurrent threads.

Root cause found by Ishi.

This reverts commit f8d887527a.

Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19383>
2022-10-31 12:16:38 +00:00
Samuel Pitoiset
25e311e9d3 radv: implement transform feedback queries with NGG streamout
The control bit is written to the upper bits because GDS counters
are 32-bits only, this allows to re-use the existing query shader.

Tested on GFX10.3.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19325>
2022-10-31 08:22:29 +00:00
Bas Nieuwenhuizen
78519987b9 radv: Speculatively tune RT pipelines for GFX11.
With ACO not supporting VOPD and the high number of SALU instructions,
we're likely better off using wave64 until we can actually benchmark
this and fix these issues.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19288>
2022-10-31 02:39:34 +00:00