Faith Ekstrand
23e1f3c373
nvk: Use nak_shader_info natively
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:13 +00:00
Faith Ekstrand
c074ea6215
nak: Handle the num_gpr offsetting inside nak
...
This makes the thing in the nak_shader_info exactly the thing that gets
plugged into the hardware. Makes the driver a bit simpler.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:13 +00:00
Faith Ekstrand
d8551cd328
nak: Add a writes_layer bit to nak_shader_info::vtg
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
a232050204
nak: Move clip, cull, and XFB into a nak_shader_info.vtg
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
440adf7970
nak: Properly prefix nak_xfb_info
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
4e6e814f5e
nak: Rename TLS to SLM
...
Shader Local Memory is what NVIDIA calls it in the shader header docs as
well as the command stream headers. Better to be consistent even if it
gets my Intel brain confused. (Intel uses SLM for shared memory.)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
a946071546
nvk: Use nak_fs_key instead of rolling our own
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
0f086401e3
nvk: Move even more lowering into nvk_codegen.c
...
At this point, we're fully trusting NAK to do its own lowering and we
only lower stuff in nvk_shader.c if it's relevant for Vulkan. This also
assumes that NAK is already doing the right thing everywhere.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
67bb8e8165
nvk: Move the guts of nvk_compile_nir() to nvk_codegen.c
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
0405f494e8
nvk: Move the optimization loop to the nvk_codegen.c
...
We also call it from nak_preprocess_nir and lower var copies there. NAK
should already be doing this for us.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
7f8fbacb8a
nvk: Move a bunch of codegen-specific lowering to helpers
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
c3a44f6264
nvk: Add a codegen helper for nir_shader_compiler_options
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
845e7d2911
nvk: Only lower outputs to temporaries
...
Also, move it up to right after we parse the SPIR-V and remove some now
unnecessary clean-up passes.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
26bb5f4972
nak/nir: Lower indirect FS inputs
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
e507d70333
nvk: Handle load_first_vertex in nvk_nir_lower_descriptors()
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
82061b1b9d
nvk: Only advertise VK_KHR_shader_terminate_invocation if using NAK
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26197 >
2023-11-15 02:24:12 +00:00
Faith Ekstrand
618bdb8571
nak: Rework FS input interpolation
...
This gives FS I/O the same treatment as we did for vertex attributes in
that we now have a NIR intrinsic which pretty closely matches the
hardware and we lower to that before going into NAK. This gives us a
bit more control in the NIR.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26181 >
2023-11-14 16:38:03 +00:00
Faith Ekstrand
d3c5688cf5
nak: Plumb the nak_compiler through to lower_fs_input_intrin
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26181 >
2023-11-14 16:38:02 +00:00
Faith Ekstrand
f5ba0751e2
nak: Make encode_sm75 a method of Shader
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26181 >
2023-11-14 16:38:02 +00:00
Faith Ekstrand
a6376705e4
nak: Make ALD/AST.PHYS a boolean
...
The generic flags field was originally copied from codegen but a boolean
makes way more sense.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26181 >
2023-11-14 16:38:02 +00:00
Faith Ekstrand
8e00ee6fe8
nak: Drop OpAtomCas in favor of OpAtom with atom_op == CmpExch
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26181 >
2023-11-14 16:38:02 +00:00
Faith Ekstrand
ea453b373d
nak: Fix copy-prop for OpPLop3 sources
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26181 >
2023-11-14 16:38:02 +00:00
Faith Ekstrand
a65518b625
nvk: Free NAK shaders
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26181 >
2023-11-14 16:38:02 +00:00
Faith Ekstrand
ade73d2e7f
nvk: Stop asserting 11-bit storage image handles
...
Now that NAK is the default for Turing+, we can just chalk any storage
image descriptor handle overruns up to codegen bugs. We could plumb
shader stages all the way through to here and only assert when codegen
is in use but that's a lot of work just for an assert.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
92cb4cc7f6
nvk: Default to NAK on Turing+
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
M Henning
f2495f47a7
nvk: Use load_global_constant for ubo loads
...
and support load_global_constant in nak
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
M Henning
1d7e051a7f
nak: Add MemOrder::Constant
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
M Henning
2a02a0d0b3
nak: Bind nir_intrinsic_access
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
M Henning
dfb6260483
nak: Specify MemScope on MemOrder::Strong
...
The scope isn't relevant on MemOrder::Weak. In fact, ptxas refuses
any input that specifies a scope on weak loads. So, don't make those
combinations representable in our IR.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
e43e8f3f78
nak: Require meson 1.3.0 and clean up a couple bits
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
7feba721c5
nak: Rely on Rust 1.73 for next_multiple_of() and div_ceil()
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
c7c73d6d17
nvk: Enable subgroups features
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
4bd9c99ca0
nak: Document a bit in encode_lds()
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
0177373e2d
nak: Insert an OpNop after OpBar
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
e111bfdb5c
nak: Emit MemBar before Bar
...
This is what NVIDIA does. I'm not sure why this order but it seems
necessary to pass some of the 1.1 memory model tests.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
bc58620e1e
nak: Use the simplified BAR.SYNC encoding
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
4f2f9c6ab3
nak: Use strong ordering for Image load/store
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
ec2c866a77
nak: Emit CCtl in barriers with acq/rel semantics
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
8781acba18
nak: Allow 1-component image load/store
...
We get these when we get an atomic image load/store
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
ee11b473b5
nak: Implement read_invocation and shuffle_*
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
edf015eb7f
nak: Fix the encoding of OpShfl
...
We weren't handling Zero. Also, we need to mask immediates or else the
encoder blows up. The hardware automatically masks them when they come
in as sources but when we get immediates, they're not guaranteed to fit
in the bitfield.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
5dd7a76c8b
nak: Implement vote and ballot
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
3df9065296
nak/nir: Zero-pad subgroup masks
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
37de14e2e2
nak/nir: Allow boolean vote_ieq
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
143d88dcc3
nak: Lower subgroup_id and num_subgroups
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
42a305416a
nak: Use nir_shader_intrinsics_pass for system values
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
ce40d81d46
nak: Call nir_lower_subgroups()
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
bf8642e984
nak: Remove unnecessary control barriers
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
3468f70ca1
nak: Use barriers for re-convergence
...
Now that we're getting total re-convergence through barriers, we can
drop our OpWarpSyncs in front of barriers and derivatives
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00
Faith Ekstrand
22468439ff
nak: Handle control-flow barriers
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998 >
2023-11-14 00:48:15 +00:00