Commit graph

136385 commits

Author SHA1 Message Date
Mike Blumenkrantz
a5984bdf78 zink: fix arrays of texel buffer descriptors
these need to pass a constructed array of buffers, not a single buffer per
descriptor

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
929a748401 zink: avoid hashing states without descriptors
this is unnecessary hashing which decreases the accuracy of the states

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
72a06746bf zink: remove image layouts from descriptor states
these are always the same value, so there's no need to track them

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
5700cbfa1a zink: always use VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL for sampler bindings
if a resource is used multiple times with different samplers then this can result
in conflicting layouts, so it's better to just use the general case here

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
e4421c4b82 zink: don't create descriptor barrier hash tables for cached descriptor set
this won't be used, so there's no point in allocating it

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
38e6e44262 zink: remove struct zink_descriptor_resource from descriptor updating
this can be inferred from a resource's barrier usage, and then by merging
the batch-tracking for resources into the barrier loop, this deduplicates
calls for resources which are bound for multiple stages as well as removing
another iteration during descriptor updating

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
4a3b344b23 zink: directly use resource count from pool instead of accumulating every time
this is a surprisingly impactful speedup

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
201cd1a8c0 zink: track resource count on descriptor pool object
we can do this once here instead of accumulating the count all over later

also remove the debug value on the set which duplicated this

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
d9793a8a60 zink: reuse descriptor barriers across draws
if we aren't invalidating the descriptor set then we can safely reuse its
barriers to avoid doing any sort of hashing during descriptor updating

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
c55e2fb59c zink: move descriptor binding out of the update codepath
we won't always need to update sets that we bind

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
dc020db723 zink: pre-size descriptor transition hash table
this avoids costly rehashing

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
80d9f3aa34 zink: simplify check for knowing whether descriptor updating is needed
if a program has at least one pool object set then it will have descriptors

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
fe31566fbb zink: remove intermediate func for descriptor set getting
we can simplify this a bit by just getting the objects in the function

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
141ff0e091 zink: allow reuse of zink_descriptor_pools between programs
now that we've split these into their own objects, we can reuse them across
programs with matching binding layouts for a given type, giving even more value to
the descriptor set cache by increasing reuse

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
ee40550b87 zink: move descriptor type to pool object from set
this is a little less memory usage

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
58500979a8 zink: track number of sets currently allocated per descriptor pool
if we're going to be sharing pools across programs, it's no longer sufficient
to check the hash populations, so we need a separate counter to know when we have
to flush

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
871022dc84 zink: change program pointer on struct zink_descriptor_set to pool pointer
the program using the set isn't very relevant now that there's the intermediate
pool struct which stores the sets

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
817de3c776 zink: break out all the descriptor pool/layout stuff into a new struct
by pushing these into a separate struct, we can eventually reuse them between
programs to make the descriptor set caches even more valuable

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
3d4e6c641b zink: break out descriptor stuff into new files
this is getting to be enough code that it's getting to be a hassle to
keep with the program stuff

also rename a couple of the moved functions

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
eee488abad zink: deduplicate VkWriteDescriptorSet setup
no functional changes

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
ce0524f009 zink: break out image descriptor updating
all the descriptor updates are now broken out, so update_descriptors is now just
a dispatch for calling the other update functions

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
eb375dc939 zink: break out sampler descriptor updating
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
e94c2d84b3 zink: break out ssbo descriptor updating
by the power of grayskull, this is now readable

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
e4076d557f zink: split out ubo descriptor updating
now that we're doing incremental updates, we can refactor this to be much
more readable by having separate functions for each type of descriptor update

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
2c83ea13d2 zink: add VkPipelineLayout to zink_program meta struct
this lets us simplify a little more code

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
0248e6b969 zink: don't double iterate all the per-batch sets on reset
we're already iterating once here, so we can definitely skip iterating again
and just do the removals inlined

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
74a0badb91 zink: add flag for recycled descriptor sets
this lets us skip a hash lookup in the last_set fastpath

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
3f561bbc83 zink: improve descriptor cache invalidation
we can pass the offset of the 'invalid' flag directly to the resources
to let them run through and invalidate their sets in time for us to detect
it when we recycle the set during batch reset and throw it onto our allocation array

additionally, by adding refs for the actual objects used in a descriptor set, we can
ensure that our cache is as accurate as possible

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
1a9f3ba2d7 zink: only batch-reference the program in use once per descriptor update
now that we're guaranteed to know what our batch is earlier, we can move this
referencing around to reduce the number of hash lookups we'll perform here

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
9df3147264 zink: move descriptor set allocation near the top of update_descriptors
no functional changes, but it moves the only point in the function where a
flush can occur out of the way which lets us get the batch we'll be using
immediately

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
514b5ced97 zink: add program pointer to desc set struct
we generally want to avoid this, but it lets us skip a lot of hash lookups,
which is a performance hit

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
b4105e8ac0 zink: actually flag all used resources as used during update_descriptors
num descriptors != num bound resources, so this needs to be a separate count/index

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
40be7f6353 zink: store last-used descriptor set for each type of set for quick reuse
we can avoid some hash lookups this way, and we can also avoid putting the null
descriptor sets back into the array since we know they'll always be the last-used
set

this also helps our null set reuse be more explicit since we never have to put these sets
back into an array or anything

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
8c908ee6a4 zink: pre-hash sampler views and states
this greatly reduces the amount of on-demand hashing that we have to do,
as now in worst case we'll be hashing 2x uint32_t per sampler descriptor
vs 2x vulkan object pointer (sometimes a uint64_t) and a uint32_t

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
fa77cc2fbb zink: add a null sampler view descriptor hash to the screen
pre-hashing this saves us time later since we can just reuse it

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
67f9038b85 zink: introduce descriptor states
this aims to track the states of descriptors so that we can do more incremental
updating

it also enables the descriptor cache to be more robust by providing the incremental
data as the key

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
6d233e74ad zink: use dynamic offsets for first ubo
this lets us avoid invalidating the ubo descriptor state, which reduces our
cache overhead

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
d4304ab0d2 zink: split descriptor sets based on usage
this uses multiple descriptor sets so that we can perform more incremental
updating and increase the value that we get from our cache by only invalidating
one state at a time

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
c5ce20e97a zink: move surface refs to the end of descriptor updating
keeping descriptor stuff with descriptor stuff

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
d06c2ae21f zink: reorder descriptor barrier applying during updating
just putting this with the rest of the non-descriptor stuff

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
761a04a464 zink: move streamout to draw_vbo
this isn't part of descriptor updating

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
b10a3be45e zink: add second level cache for descriptor sets
this lets us store sets that are valid but not currently used so that we
can either prolongue a cache entry's lifetime or cannibalize a valid entry
if necessary to avoid needing to allocate more sets

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:53 +00:00
Mike Blumenkrantz
6c4c995836 zink: add caching for descriptor sets
this is a lot of churn that more or less amounts to hashing the descriptor
state during draw and then performing lookups with this to determine whether
we can reuse an existing descriptor set instead of allocating one

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:52 +00:00
Mike Blumenkrantz
840ea21fa1 zink: add scaling factor for descriptor set bucket allocations
now descriptor sets allocate in increasingly large batches based on how many
sets a program has allocated, multiplying by 10 any time the sets hit a power of
10, e.g., if 100 sets are allocated, we now allocate in batches of 100

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:52 +00:00
Mike Blumenkrantz
bd78710d21 zink: add bucket allocating for descriptor sets
now instead of allocating a single descriptorset at a time, we allocate
a defined count of descriptorsets (currently 10) at once and keep a separate
array of allocated-and-unused sets that we can pop sets off of

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:52 +00:00
Mike Blumenkrantz
4461a871b8 zink: remove flushes for batch descriptor use
now that we do all our tracking and flushing per-program, we can throw
out the batch-based flushing and let our descriptors free-range

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:52 +00:00
Mike Blumenkrantz
d21e88719a zink: even better handling for descriptor oom
in addition to ensuring that all our batches stay under the max size by cycling
them whenever we get too many active descriptors going, we now do per-program
descriptor pools, so we can do some limiting there as well to ensure that we
aren't letting any one program hog all the resources

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:52 +00:00
Mike Blumenkrantz
4ec1d5e70c zink: add helper function for cycling a batch
this saves a bit of typing here and there

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:52 +00:00
Mike Blumenkrantz
7ae8509673 zink: use more precise sizing for descriptor pools
now that these are on the program structs, we can size the pools optimally
based on the set layouts

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:52 +00:00
Mike Blumenkrantz
9b3e2ba30c zink: move descriptor set alloc function to zink_program.c
try to keep things organized...

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9348>
2021-03-12 16:32:52 +00:00