Commit graph

181332 commits

Author SHA1 Message Date
Faith Ekstrand
26748d6fbf nak: Make calc_max_live a function of the Liveness trait
If we rework things a bit to compute live-out hash sets internally to
the pass we can make it work with just the information provided by the
trait.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Faith Ekstrand
80d58b3521 nak: Handle dead destinations in RA
It's possible for a destination component to be killed immediately if
it's never used.  To handle this, we need to track killed destinations
as well as killed sources.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Faith Ekstrand
865d6c303f nak: Require Rust 1.70.0
This is when std::sync::OnceLock gets stabilized which we require for
parsing debug options.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Faith Ekstrand
71c5bbc5a8 nak: Break guts of liveness into traits
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Faith Ekstrand
f1cb99d06b nak: Separate the CFG from liveness
One could argue that this should go in the function and not be an
on-demand analysis pass but we can do that later.  For now, we just
break it out into a separate data structure.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Faith Ekstrand
497046412e nak: Allocate the minimum number of GPRs.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Faith Ekstrand
d9c606fb08 nak/ra: Pass a PerRegFile num_regs into the allocator
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Faith Ekstrand
7b3d62e51a nak: Print to stderr
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Dave Airlie
2a7df0a03e nak: fix backtrace crash running computeheadless
Doesn't fix the example, but stops the crash.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Daniel Almeida
d7d330754b nak: add support for nir_op_unpack_half_2x16_split_{x|y}
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Daniel Almeida
59c05e16e4 nak: add support for packhalf2x16_split
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Daniel Almeida
1f10cdbbbe nak: add support for findmsb,findlsb
These all map into OpBFind, which is sourced from PTX and NV50.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Daniel Almeida
e887c4d07a nak: add support for nir_op_bitfield_reverse
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Daniel Almeida
d27e271ba5 nak: Add support for nir_op_bitcount
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Daniel Almeida
d133603c7d nak: add support for nir_op_isign
Fixes dEQP-VK.glsl.builtin.function.common.sign.*

Notice that ISel support appears not to be wired up in the old compiler, so
ISetP + Sel was chosen.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Daniel Almeida
22b4d1c559 nak: change ishl data type to I32
This matches NV50 as getDType always returns nv50_ir::TYPE_S32 for
nir_op_ishl

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Daniel Almeida
30824fa6a5 nak: more lowerings
Add more lowerings based on nv50.

Fixes
dEQP-VK.glsl.builtin.precision.ldexp.highp.scalar

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Faith Ekstrand
e03cc1f542 nak: Add some documentation for SSA values
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:11 +00:00
Daniel Almeida
11ca78f923 nak: run nir_lower_frexp and nir_opt_algebraic_late
These two are needed to get dEQP-VK.glsl.builtin.precision.frexp.* to pass.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Daniel Almeida
394bd770bc nak: add support for floor, ceil and trunc
These instructions are not supported and this shows when running the CTS.

Add support for them.

Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
4dd277e233 nak: Rework RA a bit
Instead of tracking pinned things in the register allocator, we split
the register allocator into RegAllocator and PinnedRegAllocator.  The
RegAllocator struct only allows for very simple single-SSA allocations
and frees. It tracks locations of everything, what's used, etc. but
otherwise knows nothing about pinning or vectors.

The new PinnedRegAllocator struct wraps a RegAllocator by taking a
mutable reference to it.  It provides support for pinning and all the
vector stuff.  To destroy a PinnedRegAllocator, finish() is called which
re-places any evicted SSA values and populates an OpParCopy with any
needed copies.  Because PinnedRegAllocator owns a mutable reference to
the RegAllocator, it's impossible to mix uses of PinnedRegAllocator and
RegAllocator.  This ensures that, for as long as the pinned version
exists, nothing can be allocated which migh escape the pinning.

This fixes a bunch of corner cases when register pressure gets tight.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
c9a6073754 nak: Use pcopy.push() in RA
This version guarantees that the source and destination vector stay in
sync.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
c0e6d80063 nak: Use num_regs instead of max_reg in RA
Since the top value is always the default register, we don't need to
worry about overflowing a u8.  What we do need to worry about is
register files with zero registers which is a thing pre-Turing for
uniform register files.  Use num instead of max so we don't end up
subtracting 1 from 0.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
19806cc9e4 nak: Initialize RA with only live registers
While coping the entire RA struct from the predecessor is probably
faster, it may contain values which are not live in the current block.
We could purge those but the iteration gets tricky with Rust.  It's far
clearer and more rust-friendly to deal with it as an initialization
problem.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
ec47843cca nak: Record register pressure in liveness
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
7c98dd8bfd nak: Add a PerRegFile helper struct
Sometimes it's useful to have something that's per-register-file like we
do for register allocation.  Since this is generally useful, add a
struct for it.  In future, it might be neat to pull in the enum_map
crate which basically does this generically.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
b661279e6e nak: Rework liveness to add next-use information
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
f67fecf196 nak: Compute liveness in reverse block order
If we walk the blocks forwards, we end up having a minimum O(n^2)
algorithm because everything has to propagate bottom to top.  Looping
over the blocks bacwards ensures that all the liveness information is
propgated in the first pass in the absence of back edges.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
82f0c6cf77 nak: Use the builder in some lowering passes
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
2d69a2c1d7 nak: Add a builder
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
063c9f41fa nak: Work in terms of bits for type sizes
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
ced6b44ba6 nak: Follow memcpy semantics with OpParCopy
Destination first followed by source.  Otherwise, we'll screw ourselves
up endlessly.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Daniel Almeida
831d1ee2d6 nak: Do not allocate vectors needlessly in optimization passes
All passes allocate a Vec per Instr during map(). This is wasteful,
because most instances of map()  produce a single instruction (by
mapping one instruction to another instruction) or no instructions at
all.

In such cases, they return an empty Vec, or a Vec with a single entry.

Rework the signatures so that a Vec is only when mapping one instruction
into many.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Daniel Almeida
5014b4697d nak: Heap-allocate Instrs
Heap-allocate Instrs to avoid copying them around whenever they are
mutated by a pass. This lowers the amount of copies in detriment of
cache-locality.

Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
fa0891d37c nak: Implement nir_op_b2b1 and nir_op_b2b32
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
3bf0882ea8 nak: Implement indirect UBO loads
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
0ff7fbf093 nak: Rework cbufs a bit
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
86684fa9aa nak: Legalize everything
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
9fcc0eaa8a nak: Fix multisampled textureing
Needed to handle txf_ms a few places and force LOD_MODE_ZERO

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
cc88a1c78e nak: Add a Pred struct move the enum to PredRef
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
38c9d6dfbb nak: Add accum predicates to Op[FI]Setp
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
c6edf43f70 nak: Delete unused imports and dead code
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
d48bea6638 nak: Fold fsat into FAdd/FFma/FMul
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
4c3a5b16e9 nak: Rework source modifiers instructions a bit
Instead of I/D/FMov instructions, just use [DF]Add instead. For ineg, we
add a new INeg instruction which we can lower to IADD3 later.  The
reason for this is that IAdd3 is complicated and makes detecting an ineg
rather annoying.  Also, if we ever bring NAK up on older hardware, not
all hardware has IAdd3 and INeg will be lowerable everywhere.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
8ba1429216 nak: DCE things with constant false predicates
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
818ec3242b nak: Optimize OpLop3 and OpPLop3
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
59f6d657f3 nak: Fix the 2nd predicate on LOP3
I put it in the wrong spot.  Also, it's a source so we should use
set_pred_src() for it.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
4367e0786b nak: Implement image atomics
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
82e5b0dd93 nak: Implement global/shared_atomic_comp_swap
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00
Faith Ekstrand
d359c64b2a nak: Implement nir_intrinsic_shared_atomic_*
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:10 +00:00