fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2026-05-09 04:38:03 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
193056 commits 66 branches 934 tags 955 MiB
Commit graph

193056 commits

This branch
This branch
All branches
Author SHA1 Message Date
Faith Ekstrand
05e213f03e nvk/lower_descriptors: Add a descriptor_type_is_ubo/ssbo() helper 
This is a small behavioral change as we are no longer double-counting
inline uniform data loads when determining cbuf priority.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:46 +00:00
Faith Ekstrand
903fb6f74a nvk: Make nvk_min_cbuf_alignment() inline 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:46 +00:00
Faith Ekstrand
cbe62813a1 nvk: Rename nvk_cmd_buffr_get_cbuf_descriptor() 
This makes it clear that it returns an address, not a descriptor.  That
distinction will matter soon.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:46 +00:00
Faith Ekstrand
59303584e3 nvk: Align buffer descriptors 
In theory, the app is supposed to do this for us but this gives us a bit
of protection against potential GPU faults.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:46 +00:00
Faith Ekstrand
5685de8795 nvk: Split write_[dynamic_]buffer_desc into UBO and SSBO variants 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:46 +00:00
Faith Ekstrand
6387ae7dfb nvk: Split SSBO and UBO address formats 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:46 +00:00
Faith Ekstrand
dc7b08c41a nak: Implement nir_intrinsic_ldcx_nv 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:46 +00:00
Faith Ekstrand
851b3ddd05 nak: Lower non-uniform ldcx_nv to global loads 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:46 +00:00
Faith Ekstrand
e05cb967e7 nir: Add nir_foreach_block_in_cf_node_safe() iterators 
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:46 +00:00
Faith Ekstrand
7b5856ebe9 nak: Implement [un]pin_cx_handle_nv 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:46 +00:00
Faith Ekstrand
12b79f814b nak: Implement r2ur_nv 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:46 +00:00
Faith Ekstrand
dc99d9b2df nvk,nak: Switch to nir_intrinsic_ldc_nv 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
b107240474 nir: Add some new _nv intrinsics 
The ldc_nv and ldcx_nv intrinsics correspond to the index and bindless
forms of NVIDIA's LDC instruction, respectively.  ldc_nv is pretty much
load_ubo without some of the unnecessary constant bits while ldcx_nv
takes a 64-bit bindless handle instead of an index.  The other two give
us a little control over register allocation at the NIR level to ensure
that LDCX handles are placed in uniform registers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
ab84cf11c7 nak/copy_prop: Don't propagate bindless cbufs into non-uniform blocks 
We can propagate within a non-uniform block just fine but not across
them because that might change live registers in unpredictable ways.
The real boundary here is that we can't propagate across an OpPin but
that's a lot harder to express.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
06fc2d018e nak/legalize: Bindless cbufs must be pinned in non-uniform blocks 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
c8e25b45fb nak/legalize: Allow pinned uniform vectors in non-uniform blocks 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
2279c2dd65 nak: Add OpPin and OpUnpin 
These act as a vector OpCopy, except that copy-prop can't see through
them and the destination of OpPin gets pinned in the register file and
is unallowed to move.  Of course, we have to be careful with these
because spilling can't spill them, either.  If we have too many live
pinned values at the same time, spilling or RA may fail.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
718ef00ca4 nak/ra: Add a concept of pinned registers to RegAllocator 
Unlike the pinned set in VecRegAllocator which exists for the duration
of an instruction, registers which are pinned in the main allocator are
pinned until the register is freed.  The pinned set in VecRegAllocator
is initialized to a copy of the one in the main register allocator.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
049e7ce920 nak/ra: Rename PinnedRegAllocator to VecRegAllocator 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
b1dbe42343 nak/ra: Pull searching for unused/unpinned regs into a helper 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
5aab57e1b5 nak/ra: Handle bindless CBufs 
This is done by adding a couple of helpers that we use throughout the RA
pass which abstract reading an SSA value from a source and writing a
register to that source.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
a8f8e441f5 nak/bitset: Add an iterator 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
82776f3882 nak/calc_instr_deps: Account for bindless CBufs 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
0c0cb4b9e9 nak/dce: Account for bindless CBuf handles 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
40a5b83cb3 nak/sm70: Properly encode bindless cbufs 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
d09d3f5246 nak/from_nir: Emit uniform instructions when !divergent 
The really tricky case here is phis, which may have a uniform def even
though some of the srcs are non-uniform.  This happens because of the
restriction elsewhere that requires UGPRs and UPreds to only ever be
written in uniform control-flow.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
3dfd92888a nak: Add a UniformBuilder 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
ab8a4d1940 nak/from_nir: Clean up phi annotations 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
b013d54e4f nak/lower_cf: Flag phis as convergent when possible 
Because we go in and out of SSA, all the phis get re-created and the new
phis will default to divergent.  This little pass attempts to prove as
many of the phis convergent as possible.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
06902bf52e nak: Convert to LCSSA before divergence analysis 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
3528a0760c nak/lower_cf: Track block divergence 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
0782087b8b nak/lower_cf: Parent scopes are never NULL 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
29aad97279 nak/copy_prop: Don't propagate UBOs into uniform instructions 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
5406cfc7fe nak/copy_prop: Rewrap a couple comments 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
81288bfad7 nak: Add a opt_uniform_instrs() pass 
This both lowers away invalid uniform instructions and tries to optimize
things a bit so we don't end up with more R2UR than necessary.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
be91c321c9 nak/calc_instr_deps: Add latencies for uniform instructions 
We know this is wrong.  In many cases, they're faster than warp
instructions, sometimes with a latency as low as 2.  However, there seem
to be a bunch of exceptions we don't understand and it's better to be
more concervative and have correct shaders.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
2d4e445099 nak/calc_instr_deps: Rewrite calc_delays() again 
This time we take into account WaR and WaW dependencies and not just RaW
dependencies.  The NVIDIA ISA is actually quite dynamic and the not
everything is nicely pipelined such that writes always happen at
consistent cycles.  There are exact rules, of course, but we don't know
what those are so we need to make some worst-case assumptions.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
434af5b98b nak/calc_instr_deps: Rename a couple variables 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
b47b8643b7 nak/legalize: Explicitly ignore OpPhiSrcs and OpPhiDsts 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
66a5608c11 nak/legalize: Uniform instructions can't have cbuf sources 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
7efc113bfe nak/legalize: Copy uniform vectors in non-uniform control-flow 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
e4df28ade8 nak/legalize: Ensure all SSA values for a given ref are in the same file 
It's possible for copy propagation to put a GPR and a UGPR into the same
vector source in an instruction.  Hardware registers, however, are
either GPRs or UGPRs and never a mix.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
caf033b142 nak/legalize: Handle uniform sources in warp instructions 
UGPRs in warp instructions are treated more like cbufs than GPRs.
You're only allowed to have one and it has to share space with the
possible cbuf or immediate.  This means we need to treat them as a "not
a register" case for warp instructions but as a register for uniform
instructions.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
6ad49ca7d0 nak/legalize: Patch a RegFile through to copy helpers 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
c83593b07e nak/legalize: Fix imad and ffma legalization on SM50 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
d9422a0897 nak/legalize: Be more precise about shfl and out 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
c1203ef5d1 nak/legalize: Drop some pointless plop3 logic 
There is no restrictions on plop3 in terms of what sources can be
immediates vs. registers.  Also, since we fold constants, all of this
code is dead.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
37b55ee34f nak/legalize: Fold immediate sources before instructions 
This way, if we insert a copy to move the immediate to a GPR, the
immediate we place in the copy is also folded.

Fixes: 85462f7455 ("nak: Legalize immediates with source modifiers")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
00c6244ca2 nak/sm70: Implement a bunch of uniform ops on SM75+ 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00
Faith Ekstrand
7359c214b7 nak/sm70: Fix encoding of fadd/fsetp and friends with UGPRs 
UGPRs are treated more like CBufs or immediates so we want to use the
default encoding in the UGPR case and only use the reg encoding in the
zero or GPR case.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591>
 2024-06-13 20:43:45 +00:00