Commit graph

9611 commits

Author SHA1 Message Date
Timur Kristóf
c4d60df380 radv: Refactor view index emit to use a per-stage function.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf
99b7521504 radv: Refactor radv_emit_inline_push_consts to work with radeon_cmdbuf.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf
19fcf5a4cb radv: Refactor radv_emit_descriptor_pointers to work with radeon_cmdbuf.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf
21e6d596ed radv: Refactor radv_emit_userdata_address to work with radeon_cmdbuf.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf
aadb3d69d1 radv: Refactor some CP DMA functions to work with radeon_cmdbuf.
Allow emitting these packets without a radv_cmd_buffer object.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Timur Kristóf
e10cbb5d98 radv: Refactor predication for compute queues.
Initialize the inverted predication VA only when it is used
for the first time.

This is needed to get conditional rendering work correctly with
task shaders because the internal compute cmdbuf may not exist
yet when conditional rendering starts.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16531>
2022-07-20 19:00:30 +00:00
Georg Lehmann
333f056edf radv, aco: Don't lower 16bit isign.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17440>
2022-07-20 14:31:15 +00:00
Georg Lehmann
4be6164ac9 radv: Remove ineg from lower_bit_size_callback callback.
It's always lowered to isub.

Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17440>
2022-07-20 14:31:15 +00:00
Georg Lehmann
b96126ee95 radv,aco: Don't lower and vectorize 16bit iabs.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17440>
2022-07-20 14:31:15 +00:00
Rhys Perry
37dda55f46 aco: remove unnecessary exp_cnt increments
update_counters() already does this.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17257>
2022-07-20 14:04:15 +00:00
Vinson Lee
7384aab3ee aco: Initialize spill_ctx members in constructor.
Fix defects reported by Coverity Scan.

Uninitialized scalar field (UNINIT_CTOR)
uninit_member: Non-static class member sgpr_spill_slots is not
initialized in this constructor nor in any functions that it calls.
uninit_member: Non-static class member vgpr_spill_slots is not
initialized in this constructor nor in any functions that it calls.

Fixes: 7d34044908 ("aco: refactor VGPR spill/reload lowering")
Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17583>
2022-07-20 13:39:12 +00:00
Rhys Perry
0fb3aa005f radv: don't update misaligned_mask in CmdBindVertexBuffers2
This can't do it in the loop because it doesn't easily know what
attributes use a binding.

We could do it in a separate loop, but there's no point, especially since
zink does CmdSetVertexInputEXT() after CmdBindVertexBuffers2().

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Fixes: c335a4d70e ("radv: dynamically calculate misaligned_mask for dynamic vertex input")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17521>
2022-07-19 20:40:20 +00:00
Rhys Perry
1d019d2ab7 radv: fix vbo_bound_mask indexing
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6812
Fixes: 1b8bdecf6e ("radv: add a mask of bound descriptor buffers for dynamic vertex input")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17521>
2022-07-19 20:40:20 +00:00
Jason Ekstrand
b510ee0d22 Use vk_foreach_struct_const where needed
We're about to make it so that the compiler warns/errors if you use the
wrong iterator macro.  Fix up a bunch of places where someone used the
wrong one before we break anything.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17630>
2022-07-19 19:55:17 +00:00
Samuel Pitoiset
e840ba9ed8 aco: requires Exact for p_jump_to_epilog
Otherwise, in presence of p_exit_early_if the main FS will always
jump to the PS epilog regardless the exact mask.

This fixes dEQP-VK.draw.renderpass.shader_invocation.helper_invocation
and few vkd3d-proton regressions when PS epilogs are forced.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17617>
2022-07-19 17:52:36 +00:00
Daniel Schürmann
ac39e7bf23 aco: fix assertion in insert_exec_mask
The exec mask might also be of type mask_type_loop.

Fixes: d068eb53e8 ('aco/insert_exec_mask: optimize top-level transition to exact before demote')
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17402>
2022-07-19 16:56:37 +00:00
Daniel Schürmann
6de68c5dca aco: Avoid live-range splits in Exact mode
Because the data register of atomic VMEM instructions
is shared between src and dst, it might be necessary
to create live-range splits during RA.
Make the live-range splits explicit in WQM mode.

Totals from 7 (0.01% of 134913) affected shaders: (GFX10.3)
Latency: 17209 -> 17210 (+0.01%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15347>
2022-07-19 16:30:49 +00:00
Daniel Schürmann
f12eb5c213 aco: avoid unnecessary copies in emit_wqm()
No fossil-db changes.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15347>
2022-07-19 16:30:49 +00:00
Konstantin Seurer
d4ca5942be ac/llvm: Remove load_vertex_id handling
Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17539>
2022-07-19 13:26:09 +00:00
Konstantin Seurer
83ccc810b4 aco: Remove dead nir_intrinsic_load_vertex_id case
This intrinsic is lowered in NIR.

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17539>
2022-07-19 13:26:09 +00:00
Konstantin Seurer
f90babb567 radv: Use nir_gen_rect_vertices
Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17535>
2022-07-19 12:47:30 +00:00
Samuel Pitoiset
203afc9351 radv: disable viewport depth clamping only when necessary
When the application uses depth values outside of the [0.0,1.0] range
with VK_EXT_depth_range_unrestricted, or when explicitly disabled.
Otherwise, the driver can clamp to [0.0,1.0] internally for optimal
performance.

From the Vulkan spec "in 28.10.1. Depth Clamping and Range Adjustment":

    "If depth clamping is not enabled and zf is not in the range [0, 1]
    and either VK_EXT_depth_range_unrestricted is not enabled, or the
    depth attachment has a fixed-point format, then zf is undefined
    following this step."

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16349>
2022-07-19 09:00:27 +00:00
Konstantin Seurer
fc26fbde3d vulkan: Common vk_format_get_component_bits
RADV and PowerVR use the same implementation.
Turnip does use a slightly modified version but the helper only has one
use -> just inline it and get rid of turnip's vk_format.h.

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17515>
2022-07-18 22:14:06 +00:00
Samuel Pitoiset
5ee5c73d2d radv: implement PS epilogs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
270cc39648 aco: add support for compiling PS epilogs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
8d13392969 aco: refactor export_fs_mrt_color() for PS epilogs preparation
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
df8fb721a5 radv,aco: rename radv_aco_build_prolog to radv_aco_build_shader_part
Will be re-used for PS epilogs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
897561b7b9 aco: add aco_postprocess_shader() helper
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
d9ffff09b0 aco: prevent adding DONE/VM to the last export if the FS has an epilog
If the fragment shader exports MRTZ and the epilog some color exports,
DONE/VM should be added to the last export.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
2784bfe93f aco: do not abort if the FS doesn't export anything but has an epilog
The main fragment shader can only export MRTZ (if present) and the
epilog will export colors.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
a6dff6caa1 aco: emit p_jump_to_epilog if the main fragment shader has an epilog
MRTZ is still exported from the main shader.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
8bdcc20815 aco: add new pseudo instruction p_jump_to_epilog
The first operand of this new pseudo-instruction is a 64-bit SGPR for
the continue PC, followed by a variable list of fixed VGPRS for the
color exports which are the PS epilog inputs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
0fd3754c26 radv: add a function that declares PS epilog shader arguments
The PS epilog would be a "normal" compiled shader using RA, etc. It
will declare up to 8x4 VGPRs for all color exports.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
a38db1a94e radv: declare a new user SGPR arg in FS for the epilog PC
The main FS would have to jump to the PC of the PS epilog. Given that
shaders are allocated in the 32-bit addr space, one user SGPR is fine.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
0db7a0b6e8 radv,aco: introduce {radv,aco}_ps_epilog_key
To pass the necessary pipeline information for compiling PS epilogs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Samuel Pitoiset
eee098486a radv,aco: track if a fragment shader needs an epilog
This is currently disabled but it will be used for testing first,
and then for graphics pipeline libraries.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Konstantin Seurer
0580910aa9 radv: Only set rt stack size for dynamic stacks
When using a static callable stack, the required scratch has already
been allocated.
Dynamic stacks are located at the end of scratch memory
and are allocated on demand using radv_set_rt_stack_size.
Static stacks live at the start of scratch memory and are allocated in
create_rt_shader by setting scratch_size.

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-By: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17579>
2022-07-18 12:05:35 +00:00
Qiang Yu
eeaf0b1888 ac/nir/ngg: add a barrier before prim id export
When culling enabled, it will use LDS space, which overlap with
the prim id export.

Fixes: e97f0463a8 ("ac/nir: Implement NGG deferred attribute culling in NIR.")
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17593>
2022-07-18 09:50:09 +00:00
Qiang Yu
0b7ef846b3 ac/nir/ngg: fix nogs culling scratch size
Should be in bytes not dwords.

Fixes: e97f0463a8 ("ac/nir: Implement NGG deferred attribute culling in NIR.")
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17593>
2022-07-18 09:50:09 +00:00
Timur Kristóf
5050db0b55 radv: Remove trailing whitespace introduced by DGC commits.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17580>
2022-07-18 08:23:41 +00:00
Timur Kristóf
b732285312 radv: Only initialize DGC state when DGC is enabled.
This function causes a crash with RADV_DEBUG=llvm and this commit
works around that crash.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17580>
2022-07-18 08:23:41 +00:00
Arvind Yadav
8adbd2a964 ac/llvm: Implement nir_intrinsic_load_point_coord_maybe_flipped opcodes
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15117>
2022-07-16 07:08:10 -04:00
Arvind Yadav
689559d10f ac/llvm : Adding Number of all interpolated inputs in ac_shader_abi
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15117>
2022-07-16 07:08:10 -04:00
Tatsuyuki Ishi
648731e2bd radv: Only set pstate for the first hw_ctx.
We used to do it for every queue, which was duplicate work as pstate is
per-device. It could also cause trouble when multiple hw_ctx are created as
the call will succeed for only one of them and the rest will return -EBUSY.

Simplify and fix this by only setting for the first non-null hw_ctx.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17541>
2022-07-15 15:08:07 +00:00
Bas Nieuwenhuizen
2d2591bbb7 radv: Expose VK_NV_device_generated_commands.
Closes: #6736

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17269>
2022-07-15 14:45:13 +00:00
Bas Nieuwenhuizen
05bf39238b radv: Add stub for vkCmdBindPipelineShaderGroupNV.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17269>
2022-07-15 14:45:13 +00:00
Bas Nieuwenhuizen
3f09bd5a0e radv: Implement CmdExecuteGeneratedCommandsNV.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17269>
2022-07-15 14:45:13 +00:00
Bas Nieuwenhuizen
d7bd9db9fc radv: Implement DGC cmdbuffer generation.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17269>
2022-07-15 14:45:13 +00:00
Bas Nieuwenhuizen
37a619f517 radv: Implement DGC generated command layout structure.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17269>
2022-07-15 14:45:13 +00:00
Bas Nieuwenhuizen
0c7bb92a78 radv: Add DGC meta shader.
This generated the cmd and upload buffers.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17269>
2022-07-15 14:45:13 +00:00