Commit graph

8034 commits

Author SHA1 Message Date
Bas Nieuwenhuizen
0dd0f6cf75 radv: Don't invalidate VCACHE after clear_htile_mask.
radv_src_access_flush sets all the required flags (which doesn't include VCACHE. The
flush after write is implicit. The invalidate happens for any user that needs it
with the radv_dst_access_flush).

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12274>
2021-10-03 12:16:55 +00:00
Bas Nieuwenhuizen
c6b8702eb4 radv: Fix Android build for common functions.
Fixes: 9fc16b66d0 ("radv: use common vkGetPhysicalDevice{Image}FormatProperties()")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5328
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12764>
2021-10-03 00:36:58 +00:00
Marek Olšák
edc8a4a037 ac/surface: enable DCC image stores for all displayable DCC on gfx10.3
Co-authored-by: Joshua Ashton <joshua@froggi.es>
Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13153>
2021-10-02 22:56:48 +00:00
Joshua Ashton
fccdebd64d radv: Use common DCC image store check
We need to keep RADV and RadeonSI on the same page about this due to modifiers.

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13153>
2021-10-02 22:56:48 +00:00
Joshua Ashton
e6fcf65578 ac/surface: Add helper for checking if a surface supports DCC Image stores
We need to keep RADV and RadeonSI on the same page about this due to modifiers.

Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13153>
2021-10-02 22:56:48 +00:00
Marek Olšák
923c535ee8 ac/surface: don't overwrite DCC settings for imported buffers
Fixes: 0f6251b31f - ac/surface: use DCC compatible with image stores for < 4K resolutions

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13120>
2021-10-01 16:15:40 -04:00
Tapani Pälli
814d6b626f radv: remove feature checks from device creation
This is already handled by vk_device_init(); drivers no longer
need to do it themselves.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12867>
2021-10-01 17:37:02 +00:00
Samuel Pitoiset
2aa705ec87 radv: determine the VS output parameters in the shader info pass
This can be determined earlier instead of duplicating code in both
compiler backends.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13119>
2021-10-01 17:11:39 +00:00
Samuel Pitoiset
89f1117c62 radv: set export_clip_dists for the GS copy shader
This is needed for the next change to correctly compute the VS
output parameters from the shader info pass.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13119>
2021-10-01 17:11:39 +00:00
Timur Kristóf
c53a1dff32 radv: Don't declare ngg_gs_state when there is no API GS.
This shader arg is only used when the pipeline has API GS,
so it is useless to declare it otherwise.

Also remove unused code from radv_declare_shader_args.
NGG is handled by the GS code path so these were never
actually used.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13129>
2021-10-01 15:46:12 +00:00
Timur Kristóf
6ca66808b5 aco: Fix determining whether any culling is enabled.
Use 0xB instead of 0x00FFFFFF - this allows to jump over the culling
code when no actual culling is enabled but the ngg_cull_face_is_ccw
flag is set.

Fixes: 182d9b1e60
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13129>
2021-10-01 15:46:12 +00:00
Timur Kristóf
c13a8d20f7 aco: Fix small primitive precision.
This is a mistake. It should use ngg_culling_settings
instead of ngg_gs_state.

Fixes: 182d9b1e60
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13129>
2021-10-01 15:46:12 +00:00
Bas Nieuwenhuizen
0b92d4ec98 radv: Add GPU serialization of acceleration structures.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12840>
2021-10-01 13:43:28 +00:00
Bas Nieuwenhuizen
5a45e2e04d radv: Add CPU serialization of acceleration structures.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12840>
2021-10-01 13:43:28 +00:00
Bas Nieuwenhuizen
0bf77a3fd1 radv: Add GPU copying of acceleration structures.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12840>
2021-10-01 13:43:28 +00:00
Bas Nieuwenhuizen
c7e0a8a6f7 radv: Add CPU copying of acceleration structures.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12840>
2021-10-01 13:43:28 +00:00
Bas Nieuwenhuizen
76fcd50e14 radv: Add GPU copy/serialization/deserialization shader.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12840>
2021-10-01 13:43:28 +00:00
Bas Nieuwenhuizen
6def6ba04e radv: Add acceleration structure queries.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12840>
2021-10-01 13:43:28 +00:00
Bas Nieuwenhuizen
817b4a9aca radv: Add copy/serialization info to accel struct headers.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12840>
2021-10-01 13:43:28 +00:00
Bas Nieuwenhuizen
c814e42c12 radv: Add an indirect dispatch struct to the header.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12840>
2021-10-01 13:43:28 +00:00
Bas Nieuwenhuizen
237a979419 radv: Add an internal indirect dispatch command.
Avoiding the entire buffer dance.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12840>
2021-10-01 13:43:28 +00:00
Bas Nieuwenhuizen
8500e9e1f0 radv: Add DMA buffer update function for internal use.
Only split out the DMA part because that doesn't need the BO.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12840>
2021-10-01 13:43:28 +00:00
Tony Wasserka
fa655b6f70 aco/spill: Use std::unordered_map for spills_entry
fossil-db on Navi14:
Totals from 305 (0.20% of 150305) affected shaders:
CodeSize: 5498340 -> 5498328 (-0.00%)
Instrs: 1009992 -> 1009989 (-0.00%)
Latency: 33922644 -> 33923018 (+0.00%)
InvThroughput: 9302963 -> 9303151 (+0.00%)
VClause: 19004 -> 19001 (-0.02%)

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
3fad5efd15 aco/spill: Use unordered_map for spills_exit
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
7368f16fe8 aco/spill: Clarify use of long-lived references by adding const
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
a3ac3b231f aco/spill: Reduce allocations in next_uses_per_block
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
a6bcda6a01 aco/spill: Change worklist to a single integer
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
e60a70dde0 aco/spill: Store remat list in an std::unordered_map instead of std::map
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
4a78a05247 aco/spill: Replace an std::map to booleans with std::set
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
112babc697 aco/spill: Reduce redundant std::map lookups
The previous code checked for element containment first and then performed
another map lookup upon element access. Instead, map::find can be used to
retrieve an iterator usable for element access with no extra lookup needed.

Furthermore, pure containment checks have been simplified using map::count.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
05163fd4f4 aco/spill: Avoid copying current_spills when not needed
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
6650799ea5 aco/spill: Prefer unordered_map over map for next use distances
This changes the iteration order of next use distances, so some "random"
changes to shader metrics are expected.

fossil-db for Navi14:
Totals from 1239 (0.82% of 150305) affected shaders:
SpillSGPRs: 10559 -> 10562 (+0.03%); split: -0.05%, +0.08%
SpillVGPRs: 1632 -> 1863 (+14.15%)
CodeSize: 19321468 -> 19304164 (-0.09%); split: -0.09%, +0.01%
Instrs: 3593957 -> 3591647 (-0.06%); split: -0.07%, +0.01%
Latency: 103120695 -> 102475647 (-0.63%); split: -0.63%, +0.01%
InvThroughput: 23897614 -> 23575320 (-1.35%); split: -1.36%, +0.02%
VClause: 66406 -> 66943 (+0.81%); split: -0.01%, +0.81%
SClause: 118559 -> 118548 (-0.01%)
Copies: 310871 -> 308950 (-0.62%); split: -0.69%, +0.08%
Branches: 123386 -> 123413 (+0.02%); split: -0.00%, +0.03%

These numbers mostly come from parallel-rdp ubershaders. Small changes are
also found in the rdr2 and rage2 shader metrics, whereas others are not
significantly affected.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
4453bce770 aco/spill: Replace vector<map> with vector<vector> for local_next_use
While adding/removing elements is faster with std::map, the cost of container
copies (and the involved memory allocations) vastly outweigh that benefit in
this usage pattern.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
92d7a6ab1c aco/spill: Avoid destroying local next use maps over-eagerly
Recreating these maps in a later block requires allocating fresh memory.
Instead, by never shrinking the containing vector in the first place,
previously allocated map memory is now re-used.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
df6c395095 aco/spill: Persist memory allocations of local next use maps
The function local_next_uses allocated one next-use map per instruction in
the current block. Since the function is called once per block, this caused
an excessive amount of memory allocations to take place.

With this change, the memory for next-use maps is allocated only once and
then reused for later blocks.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
387b315ea0 aco/spill: Avoid copying next_use maps more often than needed
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
276da301e6 aco/spill: Replace map[] with map::insert
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
b183bdeabf aco/spill: Remove unused container
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
0812d440c7 aco: Use std::vector for the underlying container of std::stack
By default, std::stack uses std::deque to allocate its elements, which has
poor cache efficiency. std::vector makes appending elements more expensive
(due to potential reallocations), but in the changed contexts the element
count should always be low anyway.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Tony Wasserka
f81eb2a827 aco/spill: Avoid unneeded copies when iterating over maps
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11925>
2021-10-01 09:39:13 +00:00
Emmanuel Gil Peyrot
c238faf746 radv: Allow building when LLVM isn’t enabled
Now that ACO is considered feature-complete, it can be nice to avoid the
huge LLVM dependency when one only wants a Vulkan driver.

This patch allows radv to be built without LLVM.  The two features which
get disabled are RADV_DEBUG=llvm and shader disassembly.  The latter
is an issue for debugging, so I added a warning that this configuration
is unsupported.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11319>
2021-10-01 10:40:18 +02:00
Tony Wasserka
b70e551a51 aco/tests: Assert that the requested IR is actually provided
In particular, assembly will not be provided if no disassembler is available
for the given GPU architecture.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11319>
2021-10-01 10:40:18 +02:00
Tony Wasserka
3c1802accd radv: Disable shader disassembly when no disassembler is available
ACO relies on LLVM to disassemble AMD shaders for ISAs newer than GFX7,
so disassembly needs to be skipped when LLVM is not enabled.

For vkGetPipelineExecutableInternalRepresentationsKHR and vkGetShaderInfoAMD,
the disassembly will not be reported anymore if it can't be generated.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11319>
2021-10-01 10:40:18 +02:00
Tony Wasserka
d5ac15c0e4 radv: Build code which depends on LLVM only when enabled
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11319>
2021-10-01 10:40:18 +02:00
Emmanuel Gil Peyrot
d0ec3582af radv: Support shader compilation without LLVM dependencies
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11319>
2021-10-01 10:40:18 +02:00
Emmanuel Gil Peyrot
02ee0a0a7f radv: Support device initialization without LLVM dependencies
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11319>
2021-10-01 10:40:18 +02:00
Tony Wasserka
a3e339853a aco: Extend set of supported GPUs that can be disassembled with CLRX
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11319>
2021-10-01 10:40:18 +02:00
Tony Wasserka
ef48887a9e aco: Separate LLVM/CLRX asm printers more cleanly
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11319>
2021-10-01 10:40:18 +02:00
Tony Wasserka
73e82f94c9 radv: Rename radv_shader_helper.h to radv_llvm_helper.h
This better reflects that the functions declared in this header are only
available with LLVM.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11319>
2021-10-01 10:40:18 +02:00
Martin Roukala (néé Peres)
7ef8f1fa46 radv/ci: mark some tests as flaky on gfx9
These tests have been flaky on vega10/renoir. We don't have time to look
into it just yet, so let's just mark them as such for now!

I am also including raven, as it likely has the same issue!

v2: add dEQP-VK.api.object_management.multithreaded_per_thread_resources.image_2d
v3:
 - don't skip the test, mark them as flakes (Emma Anholt)

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: Martin Roukala (néé Peres) <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13118>
2021-10-01 08:21:39 +00:00