No need to include the same BO multiple times in the long-lived ringbuffer
object's list of relocs to be added to the submit.
Improves non-TC drawoverhead -test 9 (8 tex updates) throughput by 1.4901%
+/- 0.8705% (n=20)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11697>
On drawoverhead -test 9 (8 texture changes), this saves us 172kb of
memory. That's only ~1% of the GEM memory while the test is running, but
more importantly it saves us 29% of the gem BO allocations.
non-TC drawoverhead -test 9 (8 texture change) throughput 0.449019% +/-
0.336296% (n=100), but this gets better as we get better suballocation
density.
Note that this means that all fd_ringbuffer_new_object calls can now
return data aligned to 64 bytes, instead of 4k. We may find that we need
to increase it if some of our objects (tex consts, sampler consts, etc.)
require more alignment than that. But, this may help non-drawoverhead
perf if any of our RB objects have a cache in front of them (indirect
consts?) and we don't have most of our data in the same cache set any
more.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11697>
this better enables object-specific (e.g., context) queues where the owner
of the queue will always be needed and various pointers will be passed in
for tasks
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11312>
Going beyond 0x100000 results in hangs, however I found that the
last 0x100000 packet just doesn't get executed. Thus the real limit is
0x0FFFFF. At least this is true for a6xx.
This could be tested by appending nops to the cmdstream and placing
e.g. CP_INTERRUPT at the end, at any position other than being
0x100000 packet it results in a hang.
Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10786>
This provides the upper layer (gallium, etc) a way to ensure that
rendering involving the bo has been flushed all the way to the kernel.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10530>
Move the submit ioctl to it's own thread to unblock the driver thread
and let it move on to the next frame.
Note that I did experiment with doing the append_bo() parts
synchronously on the theory that we should be more likely to hit the
fast path if we did that part of submit merging before the bo was
potentially re-used in the next batch/submit. It helped some things
by a couple percent, but hurt more things.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
A more direct solution would be for bo's to have a reference to the
device. But bo's are ref/unrefd more frequently.
This avoids async submits unrefing a bo after the device handle-
table is freed.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
For submits flushed with (a) no required fence, and (b) no externally
visible effects (ie. imported/exported bo), we can defer flushing the
submit and merge it into a later submit.
This is a bit more work in userspace, but it cuts down the number of
submit ioctls. And a common case is that later submits overlap in the
bo's used (for example, blit upload to a buffer, which is then used in
the following draw pass), so it reduces the net amount of work needed
to be done in the kernel to handle the submit ioctl.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/19
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
For deferred submits, we still need to do the prep steps immediately,
but the ioctl construction can be deferred.. prepare for that by
splitting the prep out.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
Now that we have some bo state tracking for userspace fences, we can
build on this to add a way for the pipe implementation to defer a submit
flush in order to merge submits into a single ioctl.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
Move everything into a struct assocated with the pipe_fence_handle, so
that the drm layer can fill in the seqn/fd fences directly.
This will give us a comvenient place to insert a util_queue_fence in the
next commit.
While we're at it, extract the uint32_t fence (previously called
'timestamp' in place, a kgsl legacy) into a struct that encapsulates
both the kernel fence and the userspace fence.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
In the common case, a bo will have no more than a single fence attached.
We can inline the storage to avoid a separate allocation in this case.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
Add a per-fd_pipe fence "timeline" so we can detect cases where we don't
need to call into the kernel to determine if a fd_bo is still busy.
This reuses table_lock, rather than introducing a per-bo lock to protect
fence state updates because (a) the common / hotpath pattern is to
update fences on a lot of objects, but checking the fence state of a
single object is less common, and (b) because we already hold the table
lock in common spots where we need to check the bo's fence state (ie.
allocations from the bo-cache).
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
There are a couple cases where we want to use _NOSYNC, but at the same
time we want to ensure that rendering related to a bo is actually
flushed.
This doesn't do anything yet, but when we start deferring/merging
submits we'll need a way to trigger anything deferred to flush.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
Most of them were actually unused. The memory type (KMEM vs SMI) only
applied to very old a2xx era devices that had a small/fast stacked
memory (SMI) vs normal memory (KMEM). And the cache flags are ignored
(ie. everything is writecombine), but we can add new cache flags later
when they actually do something.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10444>
Get this out of the way first to avoid some register push/pop. Only
reloc->bo is needed after writing the address into cmdstream, so this
turns msm_submit_append_bo() into a tail call.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9581>
"OBJECT" rb's are long lived, and generating them is not a hotpath, but
relocs to "STREAMING" rb's are a hot path. But we can decouple these.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9581>
In particular I wanted the nr_foo increment to be after assignment..
mostly just to track down a potential race. (This wasn't it, but I
like this color for the bikeshed better.)
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7342>
Since we are not tracking reloc flags per submit, we can just construct
this table at flush time, rather than using a second growable table that
is in sync with msm_submit->bos.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7342>
In general, rings are not shared across contexts/threads. But this
can happen with texture stateobjs, which can be invalidated by other
contexts.
And while we're here, lets convert the rest of freedreno/drm to
u_atomic
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7342>
debug_printf() isn't terribly great in multi-threaded situations.. but
since we now have a simple util/log.h, which even plays nicely with
logcat on android, lets use that instead.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7342>
This is a bit over-paranoid, and can cause drm device fd leaks if there
is a bo leak somewhere. Which is kind of a worse outcome.
This "fixes" a fd leak seen in:
dEQP-EGL.functional.query_context.get_current_display.*
dEQP-EGL.functional.query_context.get_current_context.*
dEQP-EGL.functional.query_context.get_current_display.*
(Still tracking down the root leak)
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6900>
This uses a meson builtin to handle -fvisibility=hidden. This is nice
because we don't need to track which languages are used, if C++ is
suddenly added meson just does the right thing.
Acked-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4740>
Otherwise we can MADVISE(WONTNEED) a bo that someone else is still
using. We already handled that in the dma-buf and flink-name export
paths.
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5204>
Older kernels did not support `MSM_INFO_GET_IOVA`. But this is only
required for (a) clover (ie. `fd_set_global_binding()`) and drm paths
that are limited to newer kernels. So move the location of the assert
to fix new userspace on old kernels.
Fixes: c9e8df61dc ("freedreno: Initialize the bo's iova at creation time.")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Tested-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5081>
"softpin" mode was introduced in the same kernel as the 'DUMP' flag. So
if we are using the legacy non-softpin path, clear the dump flag. OTOH
the 'DUMP' flag isn't quite so needed on older kernels, since we would
get all cmdstream, even SDS stateobjs, dumped regardless, as they would
have cmd table entries.
Fixes: b2c23b1e48 ("freedreno: Mark all ringbuffer BOs as to be dumped on crash.")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Tested-by: Ilia Mirkin <imirkin@alum.mit.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5081>
For the piglit drawoverhead case, 5/18 of the objects' relocs were
duplicated. We can dedupe them at object create time (since objects are
long-lived) and avoid repeated relocation work at emit time.
nohw drawoverhead program statechange throughput 2.34082% +/- 0.645832%
(n=10).
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5020>
