Commit graph

181332 commits

Author SHA1 Message Date
Mary Guillemard
1a2bed819d nak: Fix ALD/AST encoding for vtx and offset
The bit offsets were inverted compared to gv100 codegen.

Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
66c9c40f68 nak: Handle modifiers in dedup_srcs() in opt_lop()
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
1b6962eb7e nak: Provide more detail when printing IR after passes
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
5654366a55 nak: Fold source modifiers in legalize
We were smashing them to None later but didn't actually take them into
account in fold_lop_src() to ensure the smash was valid.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
6da06bf7d7 nak/nir: Return one sample for gl_SampleMaskIn[0] when sample shading
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
0e393f0d58 nak/nir: Interpolate gl_PointCoord
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
062735c1cc nak/nir: Add a load_frag_w helper
Also, rework things a bit.  The one functional change is that we now put
1/w in gl_FragCoord.w which should fix a CTS test.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
8ec753749a nak/nir: Fix sample vs. pixel input interpolation
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
1ab35afc26 nak/nir: Move frag_coord/sample_pos lowering to FS input lowering
It monkeys about with interpolation so it makes more sense to keep all
that stuff together.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
460a0d8a4c nak/nir: Plumb the FS key into lower_fs_input_intrin
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Mary Guillemard
580b98f353 nak: Gather global memory usage for ShaderInfo
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Mary Guillemard
0afc6fa880 nak: Add for_each_instr in Shader
Allows to visit each instructions without remapping every functions.

Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
e9bad677af nak: Plumb uses_kill through from nak_from_nir
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
274d521c22 nak: Take components into account in load_*input
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
543eebb272 nak: Handle interpolate_at_offset
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
f5c41151f2 nak/nir: Use nir_shader_intrinsics_pass for FS inputs
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Mary Guillemard
04911df940 nak: Add support for fddx and fddy
This uses SHFL in combination with FSWZADD.

Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
e174fc9ab3 nak/nir: Fix helper invocations
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:13 +00:00
Faith Ekstrand
4e14cefa70 nak: Add support for gl_FrontFace
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
2c91587ac5 nak: Saturate depth writes
Otherwise, they get may clipped away.  This seems wrong in light of
unrestricted depth but it fixes a bunch of CTS tests for now so let's go
with it.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
6abbef04b0 nvk/nak: Translate our new FS flags from NAK to nvk_shader
Eventually, we want to drop all this garbage from nvk_shader and just
use nak_shader_info directly but, for now, it's less disruptive to just
translate from new to old.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
34cc225070 nak: Plumb more FS info through to the C API
These are needed for state setup, not the shader header so we have to
plumb them all the way through to the driver.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
0ec283083c nak: Re-plumb compute shader info
Put the stuff in nak_shader_info in nak.h into a union so we can have
other stage info exported as well.  Then plumb local_size and smem_size
through ShaderInfo like we do for a bunch of the FS things.  While we're
shuffling things around, pull nak_shader_info::cs into a union.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
9eeda9dd6d nak: Rework FS outputs again
This time we map them to a consistent output address space like we do
for all other I/O and system values and do the remap in nak_from_nir.
This lets us know very precise usage information and more robustly build
the OMask in the shader header.  We also handle location_frac now.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
7ce8470bd4 nak: Stash a ShaderInfo in ShaderFromNir
We're going to want to gather data into this struct as we go.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
3cdc26c157 nak: Pull sm, num_gprs, and tls_size into a ShaderInfo struct
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
141137ad14 nak: Use count_attribute_slots for FS input var sizes
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
64168e58cd nvk,nak: Plumb through the zs_self_dep key bit
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
c54f622a93 nak: Set TLS size properly in the shader header
We were only accounting for load/store_scratch before, not spilling.
Pull the value from the Shader, that one's accurate.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
cbb05f0e6b nak: Implement discard and demote
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
24c2728ca5 nak: Implement load_sample_id and load_sample_mask_in
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
595f03957c nak/nir: Several FS output fixes
1. Lower arrays to elements
 2. Multiply locations by 4 to get GPR numbers
 3. Fix dual-src blend
 4. Sample mask and depth both consume slots if either is used

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
ea750f1ffd nak: Handle empty OpFSOut
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
ecf5c4c13b nak: Properly handle OpFSOut in RA and liveness
That one's a bit magic because it exists to ensure that each output GPR
ends up in the right register at the end of the shader.  We tried to
handle it as a simple lowering before but it's easier and safer to
handle it directly in RA and liveness.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
c47488341e nak: Use load_interpolated_input for frag_coord
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Dave Airlie
0e327167a2 nak: do perspective divide for interp none as well
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
ab80e1189f nak: Add support for centroid and sample interp modes
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Dave Airlie
850668c942 nak: make ipa encoding match the order in codegen gv100
78 has interp mode
76 has sample mode

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
dadc79d96f nak: Handle flat FS inputs
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
ea6057deb4 nvk: Use sysvals for fragcoord etc. with NAK
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
073b2257dd nak: Emit FS_OUT before EXIT
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
a07aa29f61 nak: Re-work Instr::get_latency()
We re-order and re-arrange the whole thing by instruction type.  Also,
instead of returning an Option<u32>, have a has_fixed_latench() method
to check the instruction and then get_dst_latench() to get the latency
from instruction launch to the given destination index being available.
This lets us handle predicates properly which have a different number of
cycles for some reason.  Oh, it's now just as correct as the estimates
in nv50_ir_target_gm107.cpp.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
3ad3f1376c nak: Rework calc_delay.rs
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
2288b33a48 nak: Rework the barrier insert pass
This is mostly a refactor but it does tuck in a three functional
changes:

 1. We no lonter insert read barriers for sources that we immediately
    overwrite in the instruction.

 2. We track read and write barriers separately so we don't wait for
    read-after-read hazards but do for write-after-read.

 3. Wait on all barriers in branch instructions

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
91ea23103a nak: Use the right number of predicates in RegTracker
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
57dde7f217 nak: Implement Index[Mut] for RegTracker
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Mary Guillemard
405a9ccca8 nak: Make PRMT selection a Src
On SM50, because of immediate form limitations to 20 bits, we need to
use a register to use PRMT.

Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Mary Guillemard
c38c456270 nak: sm75: Fix panic when encoding MUFU with SQRT and TANH
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
1b869d2db5 nak: Rename OpBFind to OpFlo
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00
Faith Ekstrand
eb10a778fd nak: Fix instruction ordering in nak_ir.rs
I'm trying to stick to the ordering in the NVIDIA docs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
2023-11-14 00:48:12 +00:00