When using a secondary CS without IB2, we have to memcpy the
contents into the primary CS. Use these flags to improve perf
by preventing memcpy from VRAM.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
The new submit code path should now be able to handle gang submit
even when chaining is not enabled.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
In a gang submit, the maximum number of IBs is per IP type,
and is different for each queue. Let's respect that.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
Remove a useless extra function call, and rename fallback to
internal because it is the only code path left.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
This commit consolidates the queue submit code paths into one.
Now we always allocate BOs for every CS, but when IBs aren't
allowed, we simply submit every BO to the kernel.
A microbenchmark done by Bas indicated that submitting more IBs to
the kernel only adds a negligible overhead. Additionally, this
allows us to stop copying the command buffer contents in system
memory and get rid of a lot of legacy code.
In order to be able to submit every BO, we make sure to add the
last BO to the old_ib_buffers array on cs_finalize. This also
necessitates some changes in cs_execute_secondary and other
functions.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
This will still make it so that RADV_DEBUG=hang will only submit
one command buffer at a time, but otherwise let's pass all CS
objects into one submission and let the winsys split them if
necessary.
The winsys can do a better job at splitting them because
radv_queue has no knowledge of IBs and ignores chaining in the
splitting logic.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
In a gang submit, the follower (typically ACE) and leader
(typically GFX) can have synchronization between each other.
We must ensure that these end up in the same submission,
otherwise we can deadlock the GPU.
We rely on radv_queue here to order follower before the leader
in the submitted CS array.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
When submitting a CS whose IP type doesn't support chaining,
let's submit all the IBs that this CS allocated.
This is going to be better than the sysmem code path because it
doesn't require the winsys to memcpy the contents of the command
buffers.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
Some new features, such as perf counters and gang submit use
postambles. Implement these properly.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
When splitting a larger submission into several smaller ones,
the flushing at the beginning of the initial preambles is not
really necessary, so it'll be better to use the continue
preambles for this purpose.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
For feature parity with initial preambles.
Previously, continue preambles were for GFX6 only, but this is
about to change in the next commits.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
Currently, radv_queue already splits submissions but we want to
change this and be able to split them in the winsys code as well.
Necessary because we want to split based on number of actual
IBs not number of command buffers, but radv_queue is not
aware of IBs.
Note that this commit does not actually take this new split into
use yet, that will be done in a following commit when it is ready,
this is why we set the max IB count higher than radv_queue here.
This commit is the first step in making "fallback" the default and
only submission code path.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
They have the same meaning, but use_ib makes more sense because it
also takes the gfx_level into account.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
These are useful when debugging the submission code.
- Ensure correct IB alignment
- Ensure we don't submit empty IBs
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
Shouldn't matter in practice because the kernel will likely
give us a page-aligned BO, but better to specify it just
in case.
Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
This disallows IB BOs on eg. SDMA queues which was previously
mistakenly left out.
Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
The number of IBs per submit isn't infinite, it depends on the IP type
(ie. some initial setup needed for a submit) and the packet size.
It can be calculated according to the kernel source code as:
(ring->max_dw - emit_frame_size) / emit_ib_size
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
Skipping the continue preamble can allow other processes to mess
up some registers set by the current process.
Originally, we could omit generating the continue preamble when
no shader rings were used, because the register initialization
happened at the beginning of every main cmdbuf. However, this
isn't the case anymore.
Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
The base SGPR and the number of SGPRs can be equal but it was incorrect
because one VS can have draw_id and one can have base_instance. Fix
this by invalidating the vertex user SGPRs unconditionally.
Though they should also be invalidated after executing secondaries,
otherwise nothing is invalidated if the same pipeline is bind to the
primary again.
This fixes dEQP-VK.dynamic_rendering.primary_cmd_buff.random.seed*.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21652>
Replaces the RADV pipeline cache with an implementation
based on the common vk_pipeline_cache.
We use a dual-layer approach with two types of cache entries.
1. radv_shader:
- serialized as radv_shader_binary
- uses SHA1 of the binary as key
2. radv_pipeline_cache_object:
- contains pointers to associated radv_shaders
- serialized as list of SHA1
- uses the pipeline hash as key
In combination with single-file disk-cache, this reduces the cache size by ~60%.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
This function takes a radv_shader_binary and writes it to the
disk cache before creating and returning a radv_shader cache entry.
The key of the cache entry is the full SHA1 hash of the binary.
This way, we will be able to deduplicate identical shaders.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
It was only read by RRA which can infer it from the parenbt internal
node.
Change in average build time (Control):
84.69471 ms -> 84.25319 ms
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22400>
is_gs_copy_shader and is_trap_handler_shader were cleared in
radv_init_shader_args. This restores the original behaviour.
Fixes: 67635bb ("radv: zero-initialize radv_shader_args right before declaring them")
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22382>
Avoid recompiling some RADV files when something changes in NIR.
Also clean up a few other includes.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22241>
Don't recompile entire ACO when something changes in NIR.
Instead, only use some headers which are actually needed,
include these in ACO files instead of relying on nir.h to
include them.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22241>
We don't want to rely on any NIR structures in ACO, because
we would like to avoid the need to include nir.h in aco_ir.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22241>
