In order to reduce the number of shader builds after pipeline creation
(that ideally shouldn't happen) we pre-generate two shader variants at
pipeline creation time. In addition to the default one, that set the
return size for all texture to 16 bit, we build another variant
setting the return size for all textures to 32-bit. cmd buffer selects
the latter if any of the textures requires 32bit.
So we are using an all 16-bit return size or an all 32-bit return size
variants. This could be slightly improved by pre-generating return
size combinations if the texture number is below a threshold. But that
would require more space, and bigger pipeline creation time, so would
need to be evaluated.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
This also includes being able to serialize them as part of
GetPipelineCacheData and to deserialize it as part of
CreatePipelineCache.
So now we can also upload the assembly of the variant as part of the
PipelineCache creation.
Note that from all this the tricky part was the prog_data
serialization. v3d_compile allocates and fill a new prog_data, with
rzalloc. Among other things because it also allocates internally the
uniform list. So we needed to replicate that when deserializating the
prog_data. Ideally we would like to avoid that, and allocate as much
resources as possible using vk_alloc, but that would mean a somewhat
deep change on the v3d_compiler, that we want to avoid as much
possible for now.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
We should always save state on a push before starting a meta operation,
even if we don't have a pipeline, since dynamic state can be set at any
time directly on the command buffer. Similarly, we should always restore
it if the pop after the meta operation signals that it has written any
state, not only if we have a graphics pipeline to restore.
Fixes a rendering artifact in VkQuake.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
Since vkCmdClearAttachments executes inside a render pass, we would
benefit from converting it to a draw within the current subpass job to
improve batching and avoid expensive tile load/store operations.
This can dramatically improve performance for applications using this
command, however, we can only use this if we are clearing the base
layers of framebuffer attachments, since otherwise we would need to
use layered rendering, which we don't support yet.
This improves vkQuake3 performance dramatically (almost 100%
performance improvement at 1080p), which calls this twice per frame.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
This bit is helpful if we already need to store the buffer, but otherwise
we should not emit the store only to get the clear, we can use the global
clear packet for that and save us an expensive tile buffer store operation.
Also, we have not been using the per-buffer clear bit for depth/stencil
stores since "v3dv: fix depth/stencil clears on hardware", so we should
never been considering clearing needs to flag stores.
This improves vkQuake performance somewhere between 5%-15% by allowing
us to skip the store of the depth attachment.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
This gets vkQuake to render correctly, which creates render passes with
stencil load operations even when the depth/stencil attachment format
doesn't have a stencil aspect. While this is a bit weird, it seems to
be allowed by the spec:
"If the format has depth and/or stencil components, loadOp and storeOp
apply only to the depth data, while stencilLoadOp and stencilStoreOp
define how the stencil data is handled."
In our case we were not ignoring it and this was causing that we emitted a
Z buffer load that seemed to clobber the Z clear, preventing all draw calls
from passing the depth test.
While we are at it, also change the depth/stencil store operation (which
was already handling this scenario correctly) to use the format of the
render pass attachment description rather than the underlying image
format.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
So far we have been getting away with finishing the current job in the
presence of a pipeline barrier and relying on the RCL serialization,
but of course this is not always enough.
This patch addresses synchronization across different GPU units
(i.e. draw indirect after compute), as well as cases where we need to
sync before binning.
Fixes CTS failures in:
dEQP-VK.synchronization.op.single_queue.barrier.*
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
The order in which we emit the attributes is relevant, since
GL_SHADER_STATE_ATTRIBUTE_RECORD packets don't include an explicit
attribute index. This means that we need to emit them in driver
location order, since the compiler uses that location to compute
attribute offsets in the VPM.
Fixes ~1300 CTS tests in:
dEQP-VK.pipeline.vertex_input.multiple_attributes.out_of_order.*
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
For some reason, CTS expects E5B9G9R9 and B10G11R11 with
transparent black border clamping produce alpha 1 instead of 0.
Since border color takes precedence over the texture state swizzle,
the only way to fix this is to lower the texture swizzle in the shader
to set alpha to 1.
Fixes:
dEQP-VK.pipeline.sampler.view_type.*b10g11r11*clamp_to_border_transparent_black
dEQP-VK.pipeline.sampler.view_type.*e5b9g9r9*.clamp_to_border_transparent_black
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
Currently, we end the current job whenever the user emits a
pipeline barrier, but we then expect to have a valid job when
we emit a draw call.
If by the time we have to emit a draw call we don't have a valid
job, we need to create one by resuming execution of the current
subpass.
Fixes some tests in:
dEQP-VK.renderpass.suballocation.attachment_allocation.input_output.*
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
The first attribute must be active if using builtins.
This fixes a lot of simulator crashes for vertex input CTS tests.
It should be noted that some of these tests still fail after this
fix though, so there may be some other bug.
Fixes crashes in:
dEQP-VK.pipeline.vertex_input.multiple_attributes.*
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
If the meta operation did not change descriptor state then we should keep it,
not reset it.
Fixes:
dEQP-VK.fragment_operations.early_fragment.early_fragment_tests_stencil
dEQP-VK.fragment_operations.early_fragment.no_early_fragment_tests_stencil
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
We are treating them as a special case of texture, so the commit is
mostly about integrating them with the existing
SAMPLER/SAMPLER_IMAGE/COMBINED_IMAGE_SAMPLER infrastructure.
This commit doesn't use in any special way the render pass
information, including the dependencies, so it is possible that we
would need to do something else. But this commit gets several CTS
tests, and two Sascha Willem Vulkan demos, so let's start with this
commit and handle any other use case for following commits.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
When rasterization is disabled there are a number of CreateInfo
structs that should be ignored. We were managing this correctly
for some cases, but not all of them. Specifically, viewport state
must be ignored and we weren't doing that.
Fixes:
dEQP-VK.api.descriptor_set.descriptor_set_layout_lifetime.graphics
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
The hardware can't do this, so we need to record a CPU job that will
map the indirect buffer at queue submission time, read the dispatch
parameters and then submit a regular dispatch.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
New jobs need to re-emit all state. Typically, this is achieved
by resetting all dirty state flags when we start a new job, but
for index buffers we were not using a dirty bit because we always
emit them immediately. This patch adds the bit and only tries
to skip index buffer state if the bit is not dirty, which will
ensure that we will always emit it for new jobs.
This fixes a regression in the shadowmapping demo from Sascha Willems
introduced with "v3dv: try harder to skip emission of redundant state".
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
Heavily based on the already existing for the v3d OpenGL driver, but
without references, and with some extra OOM checks (Vulkan CTS has
several OOM tests).
With this commit v3dv_bo_alloc and v3dv_bo_free became frontends to
the bo_cache. The former tries to get a BO from the cache if possible,
and the latter stores the BO on the cache if possible. The former also
adds a new parameter to point if the BO to allocate is private.
As v3d we are only caching private BOs, those created by the driver
for internal use (like CLs, tile_alloc, etc). They are the ones with
the highest change of being reused (for example, CL BOs are always
4KB, so they can always be reused). User-created BOs can have any
size, including some very large ones for buffers and images, which
makes them far less likely to be reused and would add a lot of memory
pressure if we decided to cache them.
In any case, in practice, we found that we could get a performance
improvement by caching also user-created BOs, but that would need more
care and an analysis to decide which ones makes sense. Would also
require to change how the cached BOs are stored by size. Right now
there are an array of list_head, that doesn't work well with big
BOs. If done, that would be handled on a separate commit.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
We had done all the plumbing for this but EZ can be disabled in 3 places
and we were never setting the enable bit in the configuration bits packet.
Also, configuration bits must not enable EZ if this has been disabled in
the RCL for the whole frame, which we do if we don't have a depth
attachment at all.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
Otherwise cloned BO lists point to the original list objects and not
the cloned ones, and that will confuse anything that tries to iterate over
them, such as list_length(), leading to infinite loops.
Fixes (in debug mode):
dEQP-VK.api.command_buffers.render_pass_continue
In that test we clone a full CL job from a secondary, and without this,
the BO lists in its CL lists will point to the bo_list field in the
original job, leading to an infinite loop as we assert the expected size
of these lists at queue submit time in handle_cl_job.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
Asserting on them makes it easier to identify applications and tests that
try to use unimplemented features.
Also, there are some APIs that relate to optional features we don't
(or can't) support, such as sparse, so for these we just provide
the trivial implementation.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
We were assuming that if the command buffer state doesn't have any
attachments (as per the attachment count) the attachment state array
should not be allocated, however, during meta operations it is
possible that the attachment state grows (since meta operations can
emit render passes of their own). In that case, we would grow the
state for the meta operation but then pop the previous attachment
count and we would leak the state.
An example of that is a secondary command buffer which has no
attachment state by default since it doesn't execute a render pass
begin, but that executes one in a meta operation (for
vkCmdClearAttachments for example).
Fix this by making the attachment count an allocation count instead
and not popping it once we finish a meta operation. Also, always free
the state so long as there is a valid pointer, and assert that the
allocated count is not zero in that case.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
The main change we are introducing here is that now we allow secondary
command buffers that execute in a render pass to have a job list with
more than one job.
The main issue with vkCmdClearAttachments is that we currently need
this to spawn multiple jobs to clear multilayered framebuffers, as we
need to setup a different 2D framebuffer for each layer to clear and
therefore emit a different RCL for each. We could avoid this
completely by used layered rendering with the "clear rect" path to
redirect the clear rects to appropriate layers of the primary
framebuffer, however, our hardware only supports layered rendering
with geometry shaders, which we don't support at present.
Because vkCmdClearAttachments relies on having framebuffer state
available (something we would not need if we used the geometry shader
implementation), if this is not available in the secondary we need to
postpone emission of the command until the secondary is executed
inside a primary. We do this by using a new CPU job
V3DV_JOB_TYPE_CPU_CLEAR_ATTACHMENTS that is processed during
vkCmdExecuteCommands by calling vkCmdClearAttachments directly in the
primary.
As a consequence of these changes, it is now possible that a secondary
command buffer that runs inside a render pass have any kind of job in
its job list, including partial CLs that need to be branched to and
full CLs that need to be submitted to the GPU as is, so we introduced
a new GPU job type V3DV_JOB_TYPE_GPU_CL_SECONDARY to identify partial
CLs.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
Event waits can be safely moved before a render pass start, since
event setting and resetting commands cannot happen inside one. We
don't need to go that far, but we can use this to record the wait
in its own separate job and then execute this job before the
binning commands recorded in the secondary command buffer when
we execute the secondary into a primary.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
There are basically two types of scenarios to consider:
- Secondary command buffers that run inside a render pass.
- Secondary command buffers that run outside a render pass.
For the former we want to record their commands into a binning command
list that we can branch to when executed into a primary command
buffer. This means this kind of command buffers don't spawn new jobs,
just the default one where they record the binning commands which
won't include the frame setup, which will be provided by the primary
they will be executed in.
For the latter we don't require anything special, we just record as
many jobs as we need as usual and link that job list from the primary
job list when executed.
This handles most scenarios except:
- vkCmdWaitForEvents
- VkCmdClearAttachments
Both of these can spawn new jobs inside a render pass, which is not
what we want for secondary command buffers. We will address this is
follow-up patches.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
We should always do this. So far we have been getting away with this
because we overallocate at v3dv_job_start_frame, but that won't do
for secondary command buffers for example, it is also unreliable
if CLs grow past that initial allocation.
In the future, we might want to fix our emit macros so they do the
allocation check implicitly, which would simplify the code and would
make this process a lot less error prone.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
During a command buffer reset we call cmd_buffer_init(), which will
add the command buffer to the pool, so make sure we remove it first
and that we use a safe iterator when resetting a pool.
Fixes:
dEQP-VK.api.command_buffers.pool_reset_reuse
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
In order to properly check (and possibly compile) shader variants we
need a pipeline and a compatible descriptor set. So far we were trying
to do that check as early as possible, so we were trying to do it at
CmdBindPipeline or CmdBindDescriptorSets, and a combination of dirty
flags. This showed to not cover all the corners cases, and made the
code complex, as needed to handle cases where the descriptors were not
yet available, and return early. The latter also meant that we were
running several checks that failed in the middle.
This commit moves the variant check to CmdDraw, when we should have a
pipeline and compatible descriptor sets, and simplifies and makes more
strict the existing code.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
This reverts a previous half-attempt at an implementation of events
using a BO to hold the event state, and provides a full
implementation. V3D doesn't have any built-in GPU functionality to
wait on any kind of events, so we need to implement this in the driver
an therefore we no longer need to use a BO for the event state.
Instead, we implement GPU waits by using a CPU job for the wait
operation that spawns a wait thread if the wait operation doesn't have
all its events signaled by the time it is processed. To implement the
semantics of the wait correctly, any jobs in the same command buffer
that come after the wait will not be emitted until the wait thread
completes.
If a submit spawns any wait threads for a command buffer we can't
signal any semaphores for it until all the wait threads complete and
we know that all the jobs for those command buffers have been
submitted. The same applies to the submit fence, if present.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
This is generally very difficult to handle properly everywhere, but
at least this is good enough to make the few CTS tests for this happy.
Fixes (on Rpi4):
dEQP-VK.wsi.xlib.swapchain.simulate_oom.*
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
There is a hw bug by which the only way to clear the depth/stencil
tile buffers is to emit a clear of all tile buffers, so if we have
to do any such clears, we just emit a single clear of all tile
buffers and skip doing any per-buffer clears, even for color buffers,
since they would be redundant.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
We were declaring the destroy callback function as taking a pointer for the
vulkan object handle and relying on an implicit conversion to the Vulkan
handle type, however that would be incorrect on 32-bit platforms, where
non-dispatchable Vulkan objects (the kind that we may allocate privately during
command buffer recording), are defined as uint64_t, so the signature of the
destry callback type doesn't match the signature of the actual Vulkan
function, leading to bogus results. Fix that by using uint64_t instead.
This fixes compilation warnings and also crashes in some tests when
compiling and executing natively in Rpi4.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
We were pre-packing the constants from the pipeline state and then
always emitting that at draw time, ignoring dynamic state. This makes
it so we don't prepack at pipeline creation time and we always emit
the correct constants directly the command buffer dynamic state.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6766>
