Commit graph

6262 commits

Author SHA1 Message Date
Marek Olšák
d6acdbd935 radeonsi: implement and use compute-based DCC decompression on gfx9-10
DCC_DECOMPRESS doesn't work. Instead of trying to figure out why,
use a compute blit where the load is compressed and the store is
uncompressed.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4761>
2020-04-30 22:27:31 +00:00
Bas Nieuwenhuizen
85fe0e551f radv: Fix implicit sync with recent allocation changes.
the implicit sync flag gets set at the beginning at the function,
but I used = instead of |= later.

Fixes: bec9285027 "radv: Stop using memory type indices."
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4814>
2020-04-29 21:03:39 +00:00
Marek Olšák
22a4cb4937 ac: enable displayable DCC on Navi12 & Navi14
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4697>
2020-04-29 14:53:25 +00:00
Marek Olšák
3b45631d7a ac/surface: validate that DCC is enabled correctly on gfx9+
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4697>
2020-04-29 14:53:25 +00:00
Marek Olšák
5e31e4b697 ac/surface: add code for gfx10 displayable DCC
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4697>
2020-04-29 14:53:25 +00:00
Marek Olšák
e2fbba7720 ac/surface: move non-displayable DCC to the end of the buffer
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4697>
2020-04-29 14:53:25 +00:00
Marek Olšák
a3dc7fffbb ac/surface: don't compute DCC if it's unsupported by DCN on gfx9+
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4697>
2020-04-29 14:53:25 +00:00
Marek Olšák
5d785f99b7 ac/surface: match get_display_flag() with expectations for is_displayable
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4697>
2020-04-29 14:53:25 +00:00
Marek Olšák
3dc2ccc14c ac/surface: replace RADEON_SURF_OPTIMIZE_FOR_SPACE with !FORCE_SWIZZLE_MODE
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4697>
2020-04-29 14:53:25 +00:00
Marek Olšák
f6d87ec8a9 ac/surface: remove RADEON_SURF_TC_COMPATIBLE_HTILE and assume it's always set
So that drivers can enable it without worrying how the texture was
allocated.

v2: reworked the mechanism, hopefully fixes now
    added Bas Nieuwenhuizen's diff to fix radv

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> (v1)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4697>
2020-04-29 14:53:25 +00:00
Marek Olšák
25d3cc293e ac/surface: rename micro tile mode enums like gfx10 uses them
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4697>
2020-04-29 14:53:25 +00:00
Timur Kristóf
e4e1a0ac13 radv: Use smaller esgs_itemsize for ACO.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4388>
2020-04-29 11:51:04 +00:00
Timur Kristóf
ee5f04c9c9 aco: Use new default driver locations.
The way the new locations are set up has much fewer gaps
between each I/O slot, so this results in a massive reduction
in the LDS usage of tessellation shaders.

Totals (GFX10):
VGPRS: 3976792 -> 3974864 (-0.05 %)
Code Size: 260552784 -> 260532860 (-0.01 %) bytes
LDS: 48723 -> 30179 (-38.06 %) blocks
Max Waves: 1053407 -> 1053583 (0.02 %)

Totals from affected shaders (1407 shaders on GFX10):
SGPRS: 59144 -> 59216 (0.12 %)
VGPRS: 63024 -> 61096 (-3.06 %)
Code Size: 2695508 -> 2675584 (-0.74 %) bytes
LDS: 47109 -> 28565 (-39.36 %) blocks
Max Waves: 12999 -> 13175 (1.35 %)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4388>
2020-04-29 11:51:04 +00:00
Timur Kristóf
efa4976709 radv: Use new linking helper to set default driver locations.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4388>
2020-04-29 11:51:04 +00:00
Timur Kristóf
7056714f50 aco: Set config->lds_size when TES or VS is running on HW ESGS.
This doesn't fix anything, just reports the LDS size used by
merged ESGS shaders, such as vertex_geometry_gs and
tess_eval_geometry_gs.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4388>
2020-04-29 11:51:04 +00:00
Timur Kristóf
baa46878d4 aco: Calculate workgroup size of legacy GS.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4388>
2020-04-29 11:51:04 +00:00
Timur Kristóf
fdbb296853 aco: Remember VS/TCS output driver locations.
Instead of relying on calling shader_io_get_unique_index repeatedly,
remember the which output driver location corresponds to which
varying slot.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4388>
2020-04-29 11:51:04 +00:00
Timur Kristóf
ab07c4ea70 aco: Use context variables instead of calculating TCS inputs/outputs.
VS needs the number of TCS inputs, and TES needs the number of TCS
outputs.

It is error-prone to repeat those calculations in both instruction
selection and setup. Just set them in one place instead.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4388>
2020-04-29 11:51:04 +00:00
Timur Kristóf
fd0248c37b radv: Refactor calculate_tess_lds_size and get_tcs_num_patches.
Previously these functions needed the bit mask of the TCS outputs
and patch outputs written, and concluded the number of outputs
from that.

Now, they take the number of outputs and patch outputs instead.
This will allow the backend compiler to better optimize the
LDS layout.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4388>
2020-04-29 11:51:04 +00:00
Rhys Perry
9392ddab43 aco: consider blocks unreachable if they are in the logical cfg
unreachable was true if the last block is unreachable in the linear cfg,
but it should also be true if it is unreachable in the logical cfg.

Fixes dEQP-VK.graphicsfuzz.for-with-ifs-and-return

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: 8d8c864beb
    ('aco: improve check for unreachable loop continue blocks')

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4764>
2020-04-29 11:07:09 +00:00
Samuel Pitoiset
0e6afbbe56 radv: advertise VK_EXT_robustness2
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4775>
2020-04-29 07:29:54 +00:00
Samuel Pitoiset
0f1ead7b53 radv: handle NULL vertex bindings
With VK_EXT_robustness2, an element of pBuffers can be NULL.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4775>
2020-04-29 07:29:54 +00:00
Samuel Pitoiset
c1ef225d18 radv: handle NULL descriptors
All fields must be zero, otherwise the HW hangs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4775>
2020-04-29 07:29:54 +00:00
Samuel Pitoiset
60cc065c7d aco: fix adjusting the sample index with FMASK if value is negative
The SPIR-V spec doesn't say explicitly that the sample index
must be an unsigned integer.

This fixes crashes with some new VK_EXT_robustness2 tests.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4775>
2020-04-29 07:29:54 +00:00
Samuel Pitoiset
a112ec4c11 aco: fix nir_texop_texture_samples with NULL descriptors
With VK_EXT_robustness2, descriptors can be NULL and the number of
samples returned by nir_texop_texture_samples should be 0.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4775>
2020-04-29 07:29:54 +00:00
Samuel Pitoiset
aa94213781 ac/llvm: fix nir_texop_texture_samples with NULL descriptors
With VK_EXT_robustness2, descriptors can be NULL and the number of
samples returned by nir_texop_texture_samples should be 0.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4775>
2020-04-29 07:29:54 +00:00
Bas Nieuwenhuizen
9248b04528 radv: Expose 4G element texel buffers.
Old value seems to be copied from anv.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4787>
2020-04-29 07:05:04 +00:00
Rhys Perry
3ee3ad561a aco: fix vgpr nir_op_vecn with sgpr operands
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
c5eda3c746 aco: improve clamped integer addition disassembly workaround
Make it work with 16-bit and GFX10.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
4ed83e2f94 aco: add various GFX10 int16 opcodes
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
43f2ba39ef aco: fix sub-dword overwrite check in RA validator
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
cca8d6ce06 aco: fix sub-dword out-of-bounds check in RA validator
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
307aca83a2 aco: add missing adjust_max_used_regs()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
99ca96fbf5 aco: improve RA for uneven p_split_vector
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
24116a8a56 aco: don't recurse in sub-dword get_reg_simple()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
09c584caeb aco: split self-intersecting copies instead of swapping
Example situation:
v1 = {v0.hi, v1.lo}
v0.hi = v1.hi

The 4-byte copy's definition is completely used, but swapping it makes no
sense. We have to split it to generate correct code:
swap(v0.hi, v1.lo)
swap(v0.hi, v1.hi)

Found in dEQP-VK.spirv_assembly.type.vec3.i16.constant_composite_vert

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
be4a34966c aco: fix neighboring register check in get_reg_simple()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
fb59ed6bb9 aco: check alignment of non-subdword registers in get_reg_specified()
When splitting a v6b vector into v1 and v2b components, we should ensure
the v1 definition doesn't start at the upper half.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
916cc3e231 aco: make RegisterFile::block() take a regclass
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Samuel Pitoiset
0549fba3cc radv: advertise VK_AMD_memory_overallocation_behavior
Doom Eternal explicitly allows overallocation via this extension
but that shouldn't change anything because it's the default RADV
behavior.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4785>
2020-04-28 21:03:26 +00:00
Samuel Pitoiset
5832f2b8a3 radv: track memory heaps usage if overallocation is explicitly disallowed
By default, RADV supports overallocation by the sense that it doesn't
reject an allocation if the target heap is full.

With VK_AMD_overallocation_behaviour, apps can disable overallocation
and the driver should account for all allocations explicitly made by
the application, and reject if the heap is full.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4785>
2020-04-28 21:03:26 +00:00
Samuel Pitoiset
32035cca3f radv: remove unused radv_device_memory::map_size field
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4785>
2020-04-28 21:03:25 +00:00
Samuel Pitoiset
523e9603d3 radv: enable FMASK for color attachments only
The reason behind this is that FMASK requires CMASK and also that
FMASK for non color attachments looks unnecessary. It's currently
much easier to add this simple check because the driver tries to
always enable DCC first and if we enable FMASK only if CMASK, we
might loose some FMASK compressions.

This helps fixing some new robustness2 tests which fails because
only FMASK is enabled.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4783>
2020-04-28 17:23:05 +02:00
Bas Nieuwenhuizen
7262c743dc radv: Determine memory type for import based on fd.
This would be necessary for an application to figure out if the
memory was allocated using a memory type with VK_MEMORY_PROPERTY_PROTECTED_BIT.

It also allows one to determine VRAM vs. GTT etc.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4751>
2020-04-28 15:45:03 +02:00
Bas Nieuwenhuizen
f30983be3a radv/winsys: Add function to get domains/flags from fd.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4751>
2020-04-28 15:45:00 +02:00
Bas Nieuwenhuizen
bec9285027 radv: Stop using memory type indices.
Lots of extra coding was involved in managing them.

And for protected memory I was thinking of making a function that
goes from domain+flags to memory types, which can reuse this array.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4751>
2020-04-28 15:44:56 +02:00
Bas Nieuwenhuizen
4a8d172d3f radv: Use actual memory type count for setting app-visible bitset.
Otherwise we might make a bitset that is too large.

Cc: <mesa-stable@lists.freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4751>
2020-04-28 15:44:27 +02:00
Samuel Pitoiset
7a0a6a7180 radv: do not expose GTT as device local memory mostly for APUs
On APUs, the memory is unified (all heaps are equally fast) and
apps should count all memory heaps together. But some games like
Id Tech games (Youngblood and such) don't manage memory correctly
on APUs and they spill everything when one VRAM heap is full.

Instead of spilling buffers, they should just allocate new buffers
in the second heap but it seems like these games are confused if
two memory heaps have the DEVICE_LOCAL_BIT set.

This is probably a first step towards better memory management on
APUs but there is still some work to do if we want to run most apps
with a small dedicated VRAM (256MB or so).

This gives a huge boost for Id Tech games on APUs, and doesn't
seem to reduce Feral games performance.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4771>
2020-04-27 22:41:41 +00:00
Bas Nieuwenhuizen
cbeda7f78e radv: Add WSI buffers to BO list only if they can be used.
Also reverse the BO list removal loop. This way typical WSI usage
should find the entry in O(active swapchains) iterations, which
should not be a performance issues. Tested with Doom(2106) which
found the entry in 1 iteration every time.

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4306>
2020-04-27 18:01:24 +00:00
Samuel Pitoiset
42b1696ef6 ac,radeonsi: fix compilations issues with LLVM 11
Latest LLVM replaced LLVMVectorTypeKind.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2826
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4755>
2020-04-27 17:13:36 +00:00