Commit graph

354 commits

Author SHA1 Message Date
Marek Olšák
b1237ecb32 zink: don't set u_resource_vtbl
nothing uses it

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10659>
2021-05-21 17:38:04 +00:00
Mike Blumenkrantz
d517996ae5 zink: add even more validation for linear images before creation
nvidia claims format feature support for various flags but then doesn't
actually support them for a 3D image, only 2D, meaning that an extra step
is necessary before creating a linear image

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10823>
2021-05-17 12:55:17 +00:00
Mike Blumenkrantz
df6ab67e99 zink: populate images with u_blitter if transfer_dst isn't available
this seems incredibly unlikely, but now it's handled

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10823>
2021-05-17 12:55:17 +00:00
Mike Blumenkrantz
6a67315d8a zink: don't multiply cube array image layers
this whole thing seems broken, but this part in particular is super broken

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10824>
2021-05-17 12:42:31 +00:00
Mike Blumenkrantz
0f04a90ea3 zink: emit some barriers out of renderpass where possible
when emitting barriers, we don't need to end the renderpass in some cases

this handles the case of emitting barriers for new resources which have
never before been accessed and thus do not require synchronization at a
specific point in the batch

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10700>
2021-05-14 13:15:07 +00:00
Mike Blumenkrantz
8e2ac24482 zink: stop overwriting buffer map pointers for stream uploader
this breaks the driver!

the uploader always maps its own pointer, so modifying that at any
point just explodes things later

Fixes: d179c5d28e ("zink: implement threaded context")

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10787>
2021-05-13 21:37:15 +00:00
Mike Blumenkrantz
27a4ba52db zink: unify resource rebinding
this improves handling for shader images

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10699>
2021-05-11 22:39:50 -04:00
Erik Faye-Lund
9618d5700c zink: check for error when binding memory
If binding fails, we don't really have a valid image. So let's make sure
we free up the memory sooner rather than later.

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10689>
2021-05-10 16:38:04 +00:00
Mike Blumenkrantz
9e5d927d22 zink: remove compute cruft from resource mapping
this used to sync the compute batch, now it does nothing

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10707>
2021-05-10 01:12:36 +00:00
Mike Blumenkrantz
edd06995d7 zink: never use LINEAR for VK_EXT_4444_formats
we shouldn't be reading these back, and some drivers lie about what features
are supported in linear tiling anyway

Reviewed-by: Joshua Ashton <joshua@froggi.es>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10702>
2021-05-07 21:13:46 +00:00
Mike Blumenkrantz
c90246b682 zink: implement sparse buffer creation/mapping
sparse buffers are not cpu-readable, so any mapping requires that they
use a staging buffer, either the stream uploader for writes or a manual
copy for readback

future work here should attempt to resolve two perf issues:
* sparse allocations should be allocated dynamically using a suballocator
  or some other, more useful strategy
* readback shouldn't allocate a huge staging buffer

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10509>
2021-05-05 02:46:06 +00:00
Mike Blumenkrantz
480b2b6508 zink: minor refactoring of buffer map for read case
make this a little more flexible for the non-DONTBLOCK case

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10509>
2021-05-05 02:46:06 +00:00
Mike Blumenkrantz
e34dc0840b zink: use cached memory for staging resources
I think at one point before staging resource flagging was less reliable
this method made sense, but now it's worse

Fixes: 6ff6d01c37 ("zink: don't use cached mem for staging resources")

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10363>
2021-04-23 03:31:34 +00:00
Mike Blumenkrantz
82710b8fc4 Revert "zink: force scanout sync when mapping scanout resource"
This reverts commit 874535752b.

Reviewed-by: Adam Jackson <ajax@redhat.com>

Fixes: 874535752b ("zink: force scanout sync when mapping scanout resource")

Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10358>
2021-04-21 14:18:17 +00:00
Mike Blumenkrantz
874535752b zink: force scanout sync when mapping scanout resource
this is just for unit tests where the scanout object is redundant and
the only time a flush occurs is from stalling on readback

Fixes: 104603fa76 ("zink: create separate linear tiling image for scanout")

Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10239>
2021-04-20 14:29:38 +00:00
Michel Dänzer
2928c21eb7 Convert most remaining free-form fall-through comments to FALLTHROUGH
One exception is src/amd/addrlib/, for which -Wimplicit-fallthrough is
explicitly disabled.

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Juan A. Suarez <jasuarez@igalia.com>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10220>
2021-04-15 16:01:22 +00:00
Erik Faye-Lund
1cf6b8d461 zink: fall back from cached to non-cached memory
This fixes basic rendering on top of V3DV, which doesn't seem to expose
the cached memory we expect and love.

Fixes: 598dc3dca4 ("zink: use cached memory for all resources when possible")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10230>
2021-04-14 14:16:16 +00:00
Mike Blumenkrantz
e52712a653 zink: disable mutable formats for zs formats and scanout images
swapchain images are never going to be used as texture views, and zs formats
aren't compatible with any other formats

this enables implicit modifiers in some cases, and more work can be done in the future
to eliminate mutable format usage to further improve performance

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10180>
2021-04-12 15:37:46 +00:00
Mike Blumenkrantz
3622df7ab2 zink: flag anv for mesa image create wsi
the image creation wsi is used, but the memory creation wsi isn't

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10180>
2021-04-12 15:37:46 +00:00
Mike Blumenkrantz
104603fa76 zink: create separate linear tiling image for scanout
rendering onto a linear-tiled image is unbelievably slow if any sort of
blending is enabled, so instead always render to optimal tiling and then
copy to linear for scanout

this doubles performance for now and can be deleted in its entirety along
with the rest of the related hacks once real wsi support is implemented

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10180>
2021-04-12 15:37:46 +00:00
Mike Blumenkrantz
17b5ca5869 zink: don't lose existing pNext when using wsi_image_create_info in image creation
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10138>
2021-04-09 15:54:23 +00:00
Mike Blumenkrantz
0dfc076743 zink: only unmap PIPE_MAP_ONCE in synchronous mode
this flag cannot be used to infer that a transfer_map call will be matched
by a transfer_unmap call when tc reordering is active

fixes #4600

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10114>
2021-04-09 13:41:44 +00:00
Mike Blumenkrantz
f19946ca6e zink: stop unmapping resources
it turns out there's not actually a requirement that resources be unmapped,
which means that a ton of overhead can be saved both in the unmap codepath
(the cpu overhead here is pretty insane) and then also when mapping cached
resource memory, as the map can now be added to the cache for immediate reuse

as seen in radeonsi

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9980>
2021-04-08 03:34:49 +00:00
Mike Blumenkrantz
421fbb5f5e zink: flag DYNAMIC resources as coherent
the contents of these may change frequently, so we should skip all the
flush/invalidate

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9983>
2021-04-06 12:29:56 +00:00
Mike Blumenkrantz
6ff6d01c37 zink: don't use cached mem for staging resources
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9983>
2021-04-06 12:29:56 +00:00
Mike Blumenkrantz
a565ead64a zink: only use host mem for staging resources with linear tiling
VK_EXT_4444_formats strikes again

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9985>
2021-04-05 22:24:32 +00:00
Mike Blumenkrantz
6433661cda zink: use 2 variant to check image format props during create
just to be pedantic

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9984>
2021-04-05 22:14:26 +00:00
Mike Blumenkrantz
3fa7ce4d46 zink: use better usage flags for staging resources
enforce linear bind more logically and mark with more accurate usage

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9982>
2021-04-05 21:58:08 +00:00
Mike Blumenkrantz
d4e2493639 zink: avoid cached memory allocations when not requested
don't rely on driver orderings to not pick this

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9981>
2021-04-04 22:22:20 -04:00
Mike Blumenkrantz
ace0b16e8e zink: use VkSubresourceLayout::depthPitch as layer_stride when mapping 3D imgs
this is not only more correct according to vk spec, it avoids having a 0-sized
layer_stride, which totally breaks the transfer map

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9969>
2021-04-01 13:43:22 +00:00
Mike Blumenkrantz
a084d012a9 zink: handle PIPE_MAP_DONTBLOCK for buffer read maps
this is mostly used by queries

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9962>
2021-04-01 12:57:51 +00:00
Mike Blumenkrantz
d179c5d28e zink: implement threaded context
this includes:
* async buffer mapping/replacement
* async queue submission
* async (threaded) gallium flush handling

the main churn here is from handling async gallium flushes, which involves
creating multiple gallium fences (zink_tc_fence) for each zink fence (zink_fence).
a tc fence may begin waiting for completion at any time, even before the zink_fence
has had its cmdbuf(s) submitted, so handling this type of desync ends up needing
almost a complete rewrite of the existing queue architecture

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9935>
2021-03-31 20:16:05 -04:00
Mike Blumenkrantz
598dc3dca4 zink: use cached memory for all resources when possible
this massively speeds up memory access across the driver, specifically
for pbo-related operations

it does require that mapped memory is manually invalidated/flushed, however, and
we also need to manually align host-visible memory to be able to do that

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9884>
2021-03-30 17:27:15 -04:00
Mike Blumenkrantz
1998e216b6 zink: track coherent resource objects
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9884>
2021-03-30 17:11:43 -04:00
Mike Blumenkrantz
0731861748 zink: add locking for resource maps
vulkan prohibits multiple maps on a resource, so these have to be locked
down

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9885>
2021-03-30 15:12:58 +00:00
Mike Blumenkrantz
00dc0036bb zink: flatten out buffer creation usage flags codepath
GL doesn't know what it's doing, so we just need all the flags all the time

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9788>
2021-03-28 22:17:57 -04:00
Mike Blumenkrantz
c96b20970f zink: toggle between linear/optimal tiling during image creation
if we would be creating an unusable image, don't

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9788>
2021-03-28 22:17:57 -04:00
Mike Blumenkrantz
aac88f49e5 zink: check image format props before creating image
ensure that these attrs are viable before failing silently later on

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9788>
2021-03-28 22:17:57 -04:00
Mike Blumenkrantz
7edc3f50c1 zink: add color output bit and/or use linear tiling for sampled images
these need to have some way to get data onto them

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9788>
2021-03-28 22:17:57 -04:00
Mike Blumenkrantz
0a0fb85fbe zink: incrementally add image usage flags based on device caps
these should be provided, but if not...

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9788>
2021-03-28 22:17:57 -04:00
Mike Blumenkrantz
69cdb6b776 zink: create separate vk image/buffer objects for shader image use
the STORAGE_TEXEL and STORAGE_IMAGE bits can't be accurately applied due
to opengl allowing all resources to be used everywhere, so instead we can
create a separate object on demand which is used only by shaders and gets
extra barriers inferred along with the base object to avoid desync whenever
it is used

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9788>
2021-03-28 22:17:57 -04:00
Mike Blumenkrantz
d922850e36 zink: break out image/buffer create info structs into helper funcs
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9788>
2021-03-28 22:17:57 -04:00
Mike Blumenkrantz
7892b18acf zink: remove unnecessary flush during image maps
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9786>
2021-03-24 16:45:41 +00:00
Mike Blumenkrantz
6c35b4c771 zink: remove handling for resource flushing between compute/gfx batches
this is no longer a thing

Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9786>
2021-03-24 16:45:41 +00:00
Mike Blumenkrantz
3d94144d38 zink: add buffer_subdata hook
this just sets some flags to help optimize mapping

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9790>
2021-03-24 16:11:30 +00:00
Mike Blumenkrantz
8dd8754238 zink: explicitly use stream uploader for staging buffers
this requires adding an offset member to the transfer struct

Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9789>
2021-03-24 16:00:28 +00:00
Mike Blumenkrantz
12dfba1951 zink: unify gfx and compute batches
now that batches aren't limited and flushing is less costly, there's no
reason to keep these separate

the primary changes here are removing the zink_queue enum and collapsing
related arrays which used it as an index, e.g., zink_batch_usage into single
members

remaining future work here will include removing synchronization flushes which
are no longer necessary and (eventually) removing batch params from a number of
functions since there is now only ever a single batch

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9765>
2021-03-24 03:26:20 +00:00
Mike Blumenkrantz
e7362a9cbd zink: move zink_flush_compute() users to zink_flush_queue()
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9726>
2021-03-23 17:14:05 +00:00
Mike Blumenkrantz
e4e753333b zink: cache transfer maps
vk spec disallows mapping memory regions more than once, but we always
map the full memory range, so we can just refcount and store the pointer
onto the resource object for reuse to avoid issues here

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9725>
2021-03-23 14:23:17 +00:00
Mike Blumenkrantz
230748d614 zink: break out buffer mapping part of zink_transfer_map
no functional changes, but this is going to get much more complex in
the near future

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9725>
2021-03-23 14:23:17 +00:00