Commit graph

10226 commits

Author SHA1 Message Date
Konstantin Seurer
7485660d12 radv: Fix incorrect build info indexing
Fixes: 8666b1f ("radv: Build acceleration structures using BVH IR")
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18928>
2022-10-03 18:23:39 +00:00
Karol Herbst
876ea2f83b ac/llvm: fix load_ubo for vectors with more than 4 elements
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18802>
2022-10-01 13:36:46 +00:00
Karol Herbst
70d375100c ac/llvm: fix load/store_shared for vectors with more than 4 elements
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18802>
2022-10-01 13:36:46 +00:00
Karol Herbst
6eb3226b71 ac/llvm: fix load_const for vectors with more than 4 elements
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18802>
2022-10-01 13:36:46 +00:00
Karol Herbst
d6d39920a0 ac/llvm: support non 32 bit sized workgroup ids
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18802>
2022-10-01 13:36:46 +00:00
Rhys Perry
7e54fea373 aco: fix assembler.gfx11.vinterp test
This was missed. I guess CI doesn't have a recent enough LLVM for these
tests.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17710>
2022-09-30 20:57:02 +00:00
Rhys Perry
4544490df0 aco: limit hard clauses to 63 instructions
See https://reviews.llvm.org/D127391

fossil-db (gfx1100):
Totals from 4 (0.00% of 161689) affected shaders:
Latency: 24545 -> 24539 (-0.02%)
InvThroughput: 102867 -> 102835 (-0.03%)

fossil-db (navi10):
Totals from 4 (0.00% of 161220) affected shaders:
Latency: 25969 -> 25959 (-0.04%)
InvThroughput: 112917 -> 112869 (-0.04%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17710>
2022-09-30 20:57:02 +00:00
Rhys Perry
a406b36d30 aco/gfx11: update form_hard_clauses
See https://reviews.llvm.org/D127391

fossil-db (gfx1100):
Totals from 116 (0.07% of 161689) affected shaders:
Instrs: 124719 -> 124664 (-0.04%); split: -0.06%, +0.02%
CodeSize: 731660 -> 731440 (-0.03%); split: -0.04%, +0.01%
Latency: 2771695 -> 2771671 (-0.00%); split: -0.00%, +0.00%
InvThroughput: 1050309 -> 1050312 (+0.00%)
VClause: 3731 -> 3779 (+1.29%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17710>
2022-09-30 20:57:02 +00:00
Rhys Perry
2930317cea aco/gfx11: deallocate VGPRs at the end of the shader
fossil-db (gfx1100):
Totals from 65987 (40.81% of 161689) affected shaders:
Instrs: 57123207 -> 57199947 (+0.13%)
CodeSize: 308402500 -> 308709460 (+0.10%)
Latency: 680527139 -> 680527160 (+0.00%)
InvThroughput: 131620026 -> 131620045 (+0.00%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17710>
2022-09-30 20:57:02 +00:00
Rhys Perry
6407d783ea aco: update sendmsg enum from LLVM
Add GFX11 enums and some new ones that apparently existed before.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17710>
2022-09-30 20:57:02 +00:00
Rhys Perry
7cecc81683 aco/gfx11: fix s_waitcnt printing
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17710>
2022-09-30 20:57:02 +00:00
Rhys Perry
7b21af3f51 aco: improve wait_imm unpack
Add GFX11 support and use wait_imm::unset_counter. Looping in the waitcnt
pass was probably broken on GFX11 because of this.

fossil-db (gfx1100):
Totals from 899 (0.56% of 161689) affected shaders:
Instrs: 1319368 -> 1319179 (-0.01%)
CodeSize: 7124640 -> 7123884 (-0.01%)
Latency: 26554304 -> 26404606 (-0.56%)
InvThroughput: 9032485 -> 8978773 (-0.59%); split: -0.59%, +0.00%

No navi10 fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17710>
2022-09-30 20:57:02 +00:00
Rhys Perry
39a6067635 aco/gfx11: swap ds_cmpst_* data operands
According to an LLVM comment, these are swapped in GFX11.

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17710>
2022-09-30 20:57:02 +00:00
Rhys Perry
2cdb3e4b6b aco: add VMEMtoScalarWriteHazard tests
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18270>
2022-09-30 11:44:38 +00:00
Rhys Perry
4c327dcfbc aco: improve printing of s_waitcnt_depctr
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18270>
2022-09-30 11:44:38 +00:00
Rhys Perry
27305680e2 aco: improve printing of sgpr_null
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18270>
2022-09-30 11:44:38 +00:00
Rhys Perry
a1022dbf07 aco: use some helpers in GFX10 hazard workarounds
No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18270>
2022-09-30 11:44:38 +00:00
Rhys Perry
a4f6acece1 aco: improve VMEMtoScalarWriteHazard s_waitcnt mitigation
fossil-db (navi10):
Totals from 171 (0.13% of 134563) affected shaders:
Instrs: 69348 -> 69144 (-0.29%)
CodeSize: 381112 -> 380296 (-0.21%)
Latency: 451534 -> 451387 (-0.03%)
InvThroughput: 88692 -> 88686 (-0.01%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18270>
2022-09-30 11:44:38 +00:00
Rhys Perry
2bd16256a6 aco: fix VMEMtoScalarWriteHazard s_waitcnt mitigation
It doesn't make sense for a "s_waitcnt vmcnt(0)" to affect a store or DS
instruction.

LLVM checks for "s_waitcnt vmcnt(0) lgkmcnt(0) expcnt(0)" but ignores
s_waitcnt_vscnt (which I assume is a bug).

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: bcf94bb933 ("aco: properly recognize that s_waitcnt mitigates VMEMtoScalarWriteHazard")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18270>
2022-09-30 11:44:38 +00:00
Samuel Pitoiset
f5c50f496c radv: enable the unified heap on APUs for Red Dead Redemption 2
This seems to resolve the memory explosion without hurting performance.
This workaround is only applied for native Vulkan.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18884>
2022-09-30 11:01:50 +02:00
Samuel Pitoiset
0974b67a6e radv: add a new drirc option to enable a unified heap on APUs
This option (radv_enable_unified_heap_on_apu) allows to force the
driver to expose only one heap of VRAM because some games seem to
perform better with that.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18884>
2022-09-30 11:01:48 +02:00
Samuel Pitoiset
c8765c5244 radv: ignore shader stages that don't need to be imported with GPL
Fixes an assertion with vkoverhead.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18869>
2022-09-29 16:22:50 +00:00
Samuel Pitoiset
bc0ff36885 radv: fix initializing the blend state for NULL render pass with GPL
If a library with only the vertex input interface is created, this
would crash.

This fixes segfault with vkoverhead.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18869>
2022-09-29 16:22:50 +00:00
Samuel Pitoiset
6000e3f534 radv: fix initializing the attribute mask for VS input state and GPL
attribute_mask was unused when I first introduced this but now it's
used again.

This fixes a bunch of GPL regressions.

Fixes: 0feab7b9cf ("radv: prepare the VS input state for prologs created with GPL")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18862>
2022-09-29 16:02:41 +00:00
Samuel Pitoiset
a63a31a47b radv: remove NULL checks when binding a graphics pipeline
The driver no longer bind NULL graphics pipelines, so these checks
are useless.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18873>
2022-09-29 14:44:47 +00:00
Jason Ekstrand
10bc2cd3ae vulkan: Rename viewport_state::negative_one_to_one
This makes it a bit clearer what it's for.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18878>
2022-09-29 05:00:10 +00:00
Konstantin Seurer
7c6f407d4c radv: Add and use AS and scratch layout structs
The memory layout logic is duplicated between
radv_GetAccelerationStructureBuildSizesKHR and
radv_CmdBuildAccelerationStructuresKHR. This patch adds a helper that
computes the scratch and acceleration structure memory layout for a
given build configuration.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18842>
2022-09-28 12:28:01 +00:00
Samuel Pitoiset
bad75ed3f1 radv: fix emitting RBPLUS state when MRTs are compacted
The RBPLUS state needs the non-compacted SPI_SHADER_COL_FORMAT value,
otherwise the state is wrongly emitted if there is holes.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7348
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7319
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7365
Fixes: 8fcb4aa0eb ("radv: compact MRTs to save PS export memory space")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18818>
2022-09-28 11:48:16 +00:00
Bas Nieuwenhuizen
1dd5d94f65 radv: Do an early check of distance in triangle culling.
Culls like 99% of the triangles that are culled at all.

Reduces VALU usage in Q2RTX traversal by ~8%, though doesn't look
like VALU is a bottleneck at this point ...

For Control we get a ~5% reduction in VALU usage, but similarly it
doesn't look like a bottleneck.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18830>
2022-09-28 11:08:28 +00:00
Daniel Schürmann
97850c0bf0 aco/opt_value_numbering: use monotonic_allocator for unordered_map
This patch also changes the rename map to unordered.
Roughly halves the time spent on CSE in ACO.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
b39d2168a7 aco: implement allocator_traits for monotonic_allocator<T>
For easier usage, this patch also adds aliases for std::map
and std::unordered_map using this allocator.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
a128d444cb aco: use monotonic_buffer_resource for instructions
As monotonic_buffer_resource is not thread-safe,
we use a thread_local instance which gets allocated once.

This change reduces the compile time spent in ACO by
approximately 10%.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
15b3cc73bf aco: implement custom memory resource
This basic allocator implements an arena allocation strategy
and cannot free individual allocations.
It is intended for very fast memory allocations in situations
where memory is used to build up a few objects and then is
released all at once.

This class mimics std::pmr::monotonic_buffer_resource.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
0b76e22a96 aco: simplify operands_offset calculation in create_instruction()
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
David Heidelberg
a1d34d8c95 ci/amd: move radeonsi in amd directory
Deduplicate jobs, make easier to use GL+VK scenarios.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17938>
2022-09-27 11:59:28 +00:00
Samuel Pitoiset
29305a89b0 radv: fix handling multiview with GPL
When multiview is used, the last pre-rasterization stage should export
the layer to the fragment shader. With GPL, the next stage is
implicitly FS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>
2022-09-27 11:38:07 +00:00
Samuel Pitoiset
0c6f52a999 radv: fix handling primitive ID and clip/cull distances with GPL
When compiling the pre-rasterization stages with GPL, we can't know if
the fragment shader reads the primitive ID or the clip/cull distances
as inputs and we have to export them unconditionally.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>
2022-09-27 11:38:06 +00:00
Samuel Pitoiset
630bb53e68 radv: fix handling ViewportIndex with GPL
When the fragment shader reads the viewport index as input and the last
pre-rasterization stage doesn't export it, it should be implicitly
zero (ie. first viewport). When the next stage is known, it's already
lowered in NIR, but with GPL we can't because we don't know if the FS
reads it. To fix this use AC_EXP_PARAM_DEFAULT_VAL_000.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>
2022-09-27 11:38:06 +00:00
Samuel Pitoiset
515847b2a7 radv: rework multiview lowering in NIR slightly
This will be used to export multiview unconditionally with GPL.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>
2022-09-27 11:38:06 +00:00
Friedrich Vock
8666b1ffbf radv: Build acceleration structures using BVH IR
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
1c0ab3ba67 radv: Create pipelines for conversion meta shaders
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
45a66ab30b radv: Add conversion shader for leaf nodes
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
682dc5c28e radv: Add conversion shader for internal nodes
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
3e6033394e radv: Add BVH IR types
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
ca8061c2a7 radv: Split CmdBuildAccelerationStructuresKHR into different functions
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
c039a132d6 radv: Rename internal node shader to lbvh_internal
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Rhys Perry
3730be9873 aco: mostly implement FS input loads on GFX11
Quad-divergent CF and vertex selection doesn't work, but should at least
prevent crashes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:57 +00:00
Rhys Perry
826ed52174 aco/tests: add GFX11 assembly tests
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:57 +00:00
Rhys Perry
48c8c25e68 aco: omit read-only memory_sync_info when printing
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:57 +00:00
Rhys Perry
aadb7aef01 aco: add VINTERP instruction format
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:56 +00:00