Commit graph

10200 commits

Author SHA1 Message Date
Konstantin Seurer
7c6f407d4c radv: Add and use AS and scratch layout structs
The memory layout logic is duplicated between
radv_GetAccelerationStructureBuildSizesKHR and
radv_CmdBuildAccelerationStructuresKHR. This patch adds a helper that
computes the scratch and acceleration structure memory layout for a
given build configuration.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18842>
2022-09-28 12:28:01 +00:00
Samuel Pitoiset
bad75ed3f1 radv: fix emitting RBPLUS state when MRTs are compacted
The RBPLUS state needs the non-compacted SPI_SHADER_COL_FORMAT value,
otherwise the state is wrongly emitted if there is holes.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7348
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7319
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7365
Fixes: 8fcb4aa0eb ("radv: compact MRTs to save PS export memory space")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18818>
2022-09-28 11:48:16 +00:00
Bas Nieuwenhuizen
1dd5d94f65 radv: Do an early check of distance in triangle culling.
Culls like 99% of the triangles that are culled at all.

Reduces VALU usage in Q2RTX traversal by ~8%, though doesn't look
like VALU is a bottleneck at this point ...

For Control we get a ~5% reduction in VALU usage, but similarly it
doesn't look like a bottleneck.

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18830>
2022-09-28 11:08:28 +00:00
Daniel Schürmann
97850c0bf0 aco/opt_value_numbering: use monotonic_allocator for unordered_map
This patch also changes the rename map to unordered.
Roughly halves the time spent on CSE in ACO.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
b39d2168a7 aco: implement allocator_traits for monotonic_allocator<T>
For easier usage, this patch also adds aliases for std::map
and std::unordered_map using this allocator.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
a128d444cb aco: use monotonic_buffer_resource for instructions
As monotonic_buffer_resource is not thread-safe,
we use a thread_local instance which gets allocated once.

This change reduces the compile time spent in ACO by
approximately 10%.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
15b3cc73bf aco: implement custom memory resource
This basic allocator implements an arena allocation strategy
and cannot free individual allocations.
It is intended for very fast memory allocations in situations
where memory is used to build up a few objects and then is
released all at once.

This class mimics std::pmr::monotonic_buffer_resource.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
0b76e22a96 aco: simplify operands_offset calculation in create_instruction()
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
David Heidelberg
a1d34d8c95 ci/amd: move radeonsi in amd directory
Deduplicate jobs, make easier to use GL+VK scenarios.

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17938>
2022-09-27 11:59:28 +00:00
Samuel Pitoiset
29305a89b0 radv: fix handling multiview with GPL
When multiview is used, the last pre-rasterization stage should export
the layer to the fragment shader. With GPL, the next stage is
implicitly FS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>
2022-09-27 11:38:07 +00:00
Samuel Pitoiset
0c6f52a999 radv: fix handling primitive ID and clip/cull distances with GPL
When compiling the pre-rasterization stages with GPL, we can't know if
the fragment shader reads the primitive ID or the clip/cull distances
as inputs and we have to export them unconditionally.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>
2022-09-27 11:38:06 +00:00
Samuel Pitoiset
630bb53e68 radv: fix handling ViewportIndex with GPL
When the fragment shader reads the viewport index as input and the last
pre-rasterization stage doesn't export it, it should be implicitly
zero (ie. first viewport). When the next stage is known, it's already
lowered in NIR, but with GPL we can't because we don't know if the FS
reads it. To fix this use AC_EXP_PARAM_DEFAULT_VAL_000.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>
2022-09-27 11:38:06 +00:00
Samuel Pitoiset
515847b2a7 radv: rework multiview lowering in NIR slightly
This will be used to export multiview unconditionally with GPL.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18699>
2022-09-27 11:38:06 +00:00
Friedrich Vock
8666b1ffbf radv: Build acceleration structures using BVH IR
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
1c0ab3ba67 radv: Create pipelines for conversion meta shaders
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
45a66ab30b radv: Add conversion shader for leaf nodes
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
682dc5c28e radv: Add conversion shader for internal nodes
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
3e6033394e radv: Add BVH IR types
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
ca8061c2a7 radv: Split CmdBuildAccelerationStructuresKHR into different functions
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Friedrich Vock
c039a132d6 radv: Rename internal node shader to lbvh_internal
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18769>
2022-09-26 22:25:22 +00:00
Rhys Perry
3730be9873 aco: mostly implement FS input loads on GFX11
Quad-divergent CF and vertex selection doesn't work, but should at least
prevent crashes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:57 +00:00
Rhys Perry
826ed52174 aco/tests: add GFX11 assembly tests
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:57 +00:00
Rhys Perry
48c8c25e68 aco: omit read-only memory_sync_info when printing
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:57 +00:00
Rhys Perry
aadb7aef01 aco: add VINTERP instruction format
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:56 +00:00
Rhys Perry
55cd74d468 aco: add LDSDIR instruction format
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:56 +00:00
Rhys Perry
a7a9aad14d aco: limit GFX11 to 128 VGPRs for now
See https://reviews.llvm.org/D128054

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:56 +00:00
Rhys Perry
4e55b5b851 aco: update assembler for GFX11
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:56 +00:00
Rhys Perry
077dd12ac6 aco/gfx11: don't use more than 1 NSA dword
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:56 +00:00
Rhys Perry
d8d99c3c4f aco: add GFX11 opcode numbers
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:56 +00:00
Rhys Perry
2f74df7117 aco: fix assembly of MUBUF-to-LDS loads
These have an implicit m0 read and don't write VGPRs.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:56 +00:00
Rhys Perry
78779fd63d aco: add reg() helper to assembler
SGPR encoding is slightly different on GFX11.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:56 +00:00
Rhys Perry
7a1b522148 aco: rename Interp_instruction to VINTRP_instruction
These is clearer since GFX11 adds another interpolation format.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
2022-09-26 14:49:56 +00:00
Yonggang Luo
091249dff4 aco: Fixes compiling error about char8_t with c++20
The error is:
../mesa/src/amd/compiler/aco_register_allocation.cpp:382:7: error: no matching function for call to 'printf'
      printf(u8"☐");

Fixes: 209a89e51d ("aco: Convert to use u8 literal for Unicode character to fixes msvc warning")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7318

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Tested-by: Marcus Seyfarth <m.seyfarth@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18796>
2022-09-26 13:28:33 +00:00
Daniel Schürmann
8ace685432 radv/rt: inline radv_rt_pipeline_create_() helper into radv_rt_pipeline_create()
This saves some re-allocation and will help in future with RT shader functions.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18755>
2022-09-26 13:03:44 +00:00
Daniel Schürmann
40366a3aaf radv/rt: create separate radv_rt_pipeline struct
inherited from radv_compute_pipeline to contain all RT-related information.
This will make it easier to transition to RT shader functions.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18755>
2022-09-26 13:03:44 +00:00
Mike Blumenkrantz
007c6b1dd2 radv: use direct access to last_vgt_api_stage_locs for sgpr emission
radv_lookup_user_sgpr is heavy, stop using it

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18808>
2022-09-26 11:40:26 +00:00
Mike Blumenkrantz
351c4ed6d1 radv: store pointer to sgprs for last vertex stage
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18808>
2022-09-26 11:40:26 +00:00
Marcin Ślusarz
0337f449a6 radv: use nir_shader_instructions_pass in radv_nir_lower_ycbcr_textures
Changes:
- nir_metadata_preserve(..., nir_metadata_all) is called when pass doesn't
  make progress

Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12282>
2022-09-26 11:13:03 +00:00
Mike Blumenkrantz
a54d996463 radv: ALWAYS_INLINE radv_is_streamout_enabled()
v2 by Timur Kristóf:
- Use ALWAYS_INLINE instead of just inline.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18807>
2022-09-26 10:51:25 +00:00
Mike Blumenkrantz
051594fb7f radv: ALWAYS_INLINE radv_flush_descriptors
B I G  P E R F

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18807>
2022-09-26 10:51:25 +00:00
Mike Blumenkrantz
876f7f60ac radv: ALWAYS_INLINE radv_after_draw
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18807>
2022-09-26 10:51:25 +00:00
Samuel Pitoiset
7985a9df9a radv: enable NGG culling unconditionally for GPL but disable it dynamically
When compiling the pre-rasterization stages we don't know the primitive
topology, but we still want to enable NGG culling for performance. To
achieve that, NGG culling is enabled unconditionally when the topology
is unknown and disabled dynamically for points or lines.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18776>
2022-09-26 09:28:14 +02:00
Samuel Pitoiset
e7f6786d59 radv: use the maximum number of vertices per primitives for NGG with GPL
The hw will ignore the extra bits when points/lines are drawn, so this
just works as-is.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18776>
2022-09-26 09:28:14 +02:00
Samuel Pitoiset
e364670e83 radv: determine the last VGT api stage also for GPL
When compiling the pre-rasterization stages, we need to know this.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18776>
2022-09-26 09:28:14 +02:00
Samuel Pitoiset
e053f6feb3 radv: remove useless gfx10_ngg_info::enable_vertex_grouping
It's always TRUE and this will simplify future changes.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18776>
2022-09-26 09:28:14 +02:00
Samuel Pitoiset
efd83e9627 radv: allow to build the pre-rasterization stages in a library
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18672>
2022-09-26 07:25:50 +00:00
Samuel Pitoiset
456543e6d8 radv: determine the last VGT api stage from the active_stages bitfield
With GPL, we can get binaries from libs directly, so this would fail.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18672>
2022-09-26 07:25:50 +00:00
Samuel Pitoiset
c08ba6a76c radv: import the GS copy shader from a library if present
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18672>
2022-09-26 07:25:50 +00:00
Samuel Pitoiset
2fd3b0bceb radv: do not free the GS copy shader binary if created from a library
Similar to other shader stages.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18672>
2022-09-26 07:25:50 +00:00
Konstantin Seurer
bf4a2b374f radv: Use scalar layout for BDA references
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18794>
2022-09-25 19:44:45 +00:00