Commit graph

537 commits

Author SHA1 Message Date
Jesse Natalie
8dd44c7e72 microsoft/compiler: Skip POS for io compaction
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31942>
2024-12-24 05:54:07 -05:00
Jesse Natalie
128caf94ca microsoft/compiler: Put holes in driver_location based on I/O variable sizes
DXIL requires that each I/O variable has a unique semantic name, but when
dealing with semantics that take up multiple slots, that variable implicitly
takes up multiple names. So when assigning driver_location, we need to do
the same.

That means also updating outputs and patch constants to have a mapping from
driver_location to a compacted index, since the metadata arrays *can't* have
holes.

This would be simpler if we could hang it off the nir_variable but there's
not really any free fields to be able to do that. We only need this compacted
mapping inside the DXIL backend anyway so we can just store the array in the
module.

Tested-by: Benjamin Otte <otte@gnome.org>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12128
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32047>
2024-12-02 22:40:39 +00:00
Rhys Perry
45c1280d2c nir_lower_mem_access_bit_sizes: pass access to callback
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31904>
2024-11-13 12:59:26 +00:00
Rhys Perry
61752152f7 nir_lower_mem_access_bit_sizes: add nir_mem_access_shift_method
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31904>
2024-11-13 12:59:26 +00:00
Georg Lehmann
cba575f4df nir: always emit ddx intrinsics
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31014>
2024-10-17 09:50:19 +00:00
Marek Olšák
02923e237d nir: add hole_size parameter into the vectorize callback
It will be used to allow merging loads with a hole between them.

Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29398>
2024-10-15 05:50:24 +00:00
Jesse Natalie
44cc67e659 microsoft/compiler: Move nir_lower_undef_to_zero out of the optimization loop
Otherwise after https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7710
I'm getting fighting between this pass and nir_opt_if.

Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31003>
2024-09-03 21:53:24 +00:00
Jesse Natalie
81a00a48bc microsoft/compiler: Handle subgroup size 4 as a required wave size
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30876>
2024-08-29 03:30:31 +00:00
Alyssa Rosenzweig
50c0dbfd70 dxil: switch to derivative intrinsics
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30568>
2024-08-08 23:06:36 +00:00
David Heidelberg
68215332a8 build: pass licensing information in SPDX form
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Dylan Baker <dylan.c.baker@intel.com>
Acked-by: Eric Engestrom <eric@igalia.com>
Acked-by: Daniel Stone <daniels@collabora.com>
Signed-off-by: David Heidelberg <david@ixit.cz>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29972>
2024-06-29 12:42:49 -07:00
Alyssa Rosenzweig
dd85b50d18 treewide: use nir_break_if
Via Coccinelle patch and some manual hunk editing:

    @@
    expression b, E;
    @@

    -nir_push_if(b, E);
    -{
    -nir_jump(b, nir_jump_break);
    -}
    -nir_pop_if(b, NULL);
    +nir_break_if(b, E);

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29877>
2024-06-26 19:07:35 +00:00
Alyssa Rosenzweig
da752ed7c1 treewide: use nir_def_replace sometimes
Two Coccinelle patches here. Didn't catch nearly as much as I would've liked but
it's a start.

Coccinelle patch:

    @@
    expression intr, repl;
    @@

    -nir_def_rewrite_uses(&intr->def, repl);
    -nir_instr_remove(&intr->instr);
    +nir_def_replace(&intr->def, repl);

Coccinelle patch:

    @@
    identifier intr;
    expression instr, repl;
    @@

    nir_intrinsic_instr *intr = nir_instr_as_intrinsic(instr);
    ...
    -nir_def_rewrite_uses(&intr->def, repl);
    -nir_instr_remove(instr);
    +nir_def_replace(&intr->def, repl);

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Juan A. Suarez Romero <jasuarez@igalia.com> [broadcom]
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com> [lima]
Reviewed-by: Christian Gmeiner <cgmeiner@igalia.com> [etna]
Reviewed-by: Pavel Ondračka <pavel.ondracka@gmail.com> [r300]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29817>
2024-06-21 15:36:56 +00:00
Alyssa Rosenzweig
15257b65c6 treewide: use nir_metadata_control_flow
Via Coccinelle patch:

    @@
    @@

    -nir_metadata_block_index | nir_metadata_dominance
    +nir_metadata_control_flow

...plus some manual fixups for call sites missed by coccinelle.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Juan A. Suarez Romero <jasuarez@igalia.com> [broadcom]
Acked-by: Vasily Khoruzhick <anarsoul@gmail.com> [lima]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29745>
2024-06-17 16:28:14 -04:00
Daniel Schürmann
9b1a748b5e nir: remove nir_intrinsic_discard
The semantics of discard differ between GLSL and HLSL and
their various implementations. Subsequently, numerous application
bugs occurred and SPV_EXT_demote_to_helper_invocation was written
in order to clarify the behavior. In NIR, we now have 3 different
intrinsics for 2 things, and while demote and terminate have clear
semantics, discard still doesn't and can mean either of the two.

This patch entirely removes nir_intrinsic_discard and
nir_intrinsic_discard_if and replaces all occurences either with
nir_intrinsic_terminate{_if} or nir_intrinsic_demote{_if} in the
case that the NIR option 'discard_is_demote' is being set.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27617>
2024-06-17 19:37:16 +00:00
Daniel Schürmann
f3d8bd18dd nir: introduce discard_is_demote compiler option
This new option indicates that the driver emits the same
code for nir_intrinsic_discard and nir_intrinsic_demote.
Otherwise, it is assumed that discard is implemented as
terminate.

spirv_to_nir uses this option in order to directly emit
nir_demote in case of OpKill.

RADV GFX11:
Totals from 3965 (4.99% of 79439) affected shaders:
MaxWaves: 119418 -> 119424 (+0.01%); split: +0.03%, -0.03%
Instrs: 1608753 -> 1620830 (+0.75%); split: -0.18%, +0.93%
CodeSize: 8759152 -> 8785152 (+0.30%); split: -0.18%, +0.48%
VGPRs: 152292 -> 149232 (-2.01%); split: -2.37%, +0.36%
Latency: 9162314 -> 10033923 (+9.51%); split: -0.46%, +9.97%
InvThroughput: 1491656 -> 1493408 (+0.12%); split: -0.10%, +0.22%
VClause: 21424 -> 21452 (+0.13%); split: -0.31%, +0.44%
SClause: 53598 -> 55871 (+4.24%); split: -2.15%, +6.39%
Copies: 90553 -> 90462 (-0.10%); split: -2.91%, +2.81%
Branches: 16283 -> 16311 (+0.17%)
PreSGPRs: 113993 -> 113254 (-0.65%); split: -1.84%, +1.19%
PreVGPRs: 110951 -> 108914 (-1.84%); split: -2.08%, +0.24%
VALU: 963192 -> 963167 (-0.00%); split: -0.01%, +0.01%
SALU: 87926 -> 90795 (+3.26%); split: -2.92%, +6.18%
VMEM: 25937 -> 25936 (-0.00%)
SMEM: 110012 -> 109799 (-0.19%); split: -0.20%, +0.01%

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27617>
2024-06-17 19:37:15 +00:00
Karol Herbst
d22f936019 nir: remove workgroup_id_zero_base
This removes the need for drivers to handle both versions. The base will
get added once in nir_lower_system_values when converting from deref to
intrinsic and will be replaced by a zero for users not supporting it.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26800>
2024-04-24 20:18:49 +00:00
Karol Herbst
3217838fef nir: remove global_invocation_id_zero_base
This removes the need for drivers to handle both versions. The base will
get added once in nir_lower_system_values when converting from deref to
intrinsic and will be replaced by a zero for users not supporting it.

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26800>
2024-04-24 20:18:49 +00:00
Mike Blumenkrantz
38c5a722c2 microsoft/compiler: set compact_arrays in compiler options
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28554>
2024-04-05 17:56:28 +00:00
Jesse Natalie
607720151f microsoft/compiler: Handle base vertex/instance sysvals as DXIL intrinsics
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28383>
2024-04-05 00:25:41 +00:00
Jesse Natalie
c21355260b microsoft/compiler: Add a fractional var mask for variable sorting
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28535>
2024-04-04 20:29:11 +00:00
Jesse Natalie
33735585a9 d3d12: Lower uniforms to UBO by nir options
Ubo0 is tricky. It exists if there were any uniforms when
lower_uniforms_to_ubo was run. If we try to run that ourselves,
it might be too late and DCE/remove_dead_variables might've been run,
which removed the uniforms and their accesses, without decrementing
num_uniforms. So we have no good way of knowing whether to declare
ubos from [0, N] or [1, N]. In practice this probably doesn't make
much of a difference but the logic is there so ¯\_(ツ)_/¯

If we use the nir option, then dead code isn't run, and num_uniforms
is a true indicator of whether ubo0 exists or not.

Note that this means we are no longer running this pass for internal
shaders that don't come from the GLSL compiler, so various places are
updated to query the nir info bit that's set by running this pass.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28535>
2024-04-04 20:29:11 +00:00
Jesse Natalie
9702ddccc2 microsoft/compiler: Don't duplicate work from gather_info in var sorting
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28535>
2024-04-04 20:29:10 +00:00
Jesse Natalie
ed757b010a microsoft/compiler: Improve linking helpers
* Take masks instead of shaders
* Be more aggressive at removing sysvals
* Handle loading from outputs

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28535>
2024-04-04 20:29:10 +00:00
Jesse Natalie
d4a15a048b microsoft/compiler: Expect front-facing var as an input
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28535>
2024-04-04 20:29:10 +00:00
Jesse Natalie
4180cf39f0 microsoft/compiler: Simplify I/O component type enum handling
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28535>
2024-04-04 20:29:10 +00:00
Jesse Natalie
230bed221e microsoft/compiler: Move kill-unused/undefined varying pass from spirv to common
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28535>
2024-04-04 20:29:10 +00:00
Jesse Natalie
b6b5514a5b microsoft/compiler: When sorting variables, put unused variables last
These variables will only be used for xfb

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28535>
2024-04-04 20:29:10 +00:00
Sil Vilerino
0dd871442b d3d12: Fix leak dxil_module::serialized_dependency_table
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28546>
2024-04-03 14:50:28 +00:00
Jesse Natalie
a6658e3dc6 microsoft/compiler: Simplify code emitting CL globals
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26803>
2024-03-26 20:12:41 +00:00
Jesse Natalie
6fc8d0147e microsoft/compiler: Don't store static-indexing handles that are dynamically emitted
These handles can be emitted in control flow, which means that the handle
might be in a block which does not dominate a block that's processed
later on, which results in incorrect DXIL if we try to reference it.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26803>
2024-03-26 20:12:41 +00:00
Jesse Natalie
267ae85a72 microsoft/compiler: Disable GS streams workaround for validator 1.8
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28339>
2024-03-25 19:11:35 +00:00
Jesse Natalie
811bed8a23 microsoft/compiler: domainLocation component index needs to be i8
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28339>
2024-03-25 19:11:35 +00:00
Jesse Natalie
ba17f5ca6a microsoft/compiler: Remove code after discard/terminate in later optimization steps
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27996>
2024-03-05 21:40:09 +00:00
Jesse Natalie
4ccbaa2cd8 microsoft/compiler: Remove deref load/store/atomic ops that statically go out of array bounds
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27919>
2024-03-01 20:49:17 +00:00
Jesse Natalie
705e646e38 microsoft/compiler: Fix SM6.6 non-bindless handle annotation for UAV counter
Counter bool is 8, not 6.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27708>
2024-02-21 03:39:22 +00:00
Jesse Natalie
1314c5b357 microsoft/compiler: Fix wave size control for SM6.8+
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27624>
2024-02-15 00:29:50 +00:00
Jesse Natalie
e21774eb0a microsoft/compiler: Fix wave size control for SM6.6+
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27624>
2024-02-15 00:29:50 +00:00
Jesse Natalie
0daad70f9f microsoft/compiler: For emulating scan, ensure all threads are active when reading cross-lane
HLSL docs say WaveReadLaneAt is undefined if the target lane is inactive. This makes
sense since the target lane may need to *send* the data, rather than it being pulled
by the calling lane. So don't early-out on the loop, iterate through the whole wave
on all threads and read the cross-lane data before branching.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27624>
2024-02-15 00:29:50 +00:00
Jesse Natalie
a8758afabd microsoft/compiler: Relax assert for SPIR-V barriers
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27531>
2024-02-13 17:53:42 +00:00
Jesse Natalie
94e0091190 microsoft/compiler: Only use simplified subgroup ID algorithm for compute
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27528>
2024-02-08 20:41:58 +00:00
Jesse Natalie
003d2da2dc microsoft/compiler: Add a pass for promoting ACCESS_COHERENT on loads/stores
DXIL doesn't have instruction-level coherency. We have 3 options:
1. Promote the instruction to an atomic instruction. We can only do this
   for 32-bit or 64-bit ops.
2. If using bindless, declare the local resource declaration as globally-coherent.
3. If not using bindless, add globally-coherent to the global resource declaration.

This pass does all 3 of these, stopping at the intrinsic level for supported types
of atomics, otherwise assigning to the global resource declaration, which will be
unused if we're doing bindless, where instead we'll get it from the instruction.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27348>
2024-02-02 01:19:51 +00:00
Jesse Natalie
b74cd405d3 microsoft/compiler: Respect ACCESS_COHERENT in UAV variable data
DXIL has a globally-coherent field for UAVs. When emitting UAV metadata
based on a resource variable, respect the relevant bit in the var data.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5628
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27348>
2024-02-02 01:19:51 +00:00
Jesse Natalie
5cdcb7134a microsoft/compiler: Use double pack/unpack instead of int for reduce ops on doubles
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27313>
2024-01-29 17:46:11 +00:00
Jesse Natalie
beb82c836d microsoft/compiler: Add feature flags for new comparison sampling ops
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27200>
2024-01-24 21:38:49 +00:00
Karol Herbst
f2b7c4ce29 nir: rework and fix rotate lowering
No driver supports urol/uror on all bit sizes. Intel gen11+ only for 16
and 32 bit, Nvidia GV100+ only for 32 bit. Etnaviv can support it on 8,
16 and 32 bit.

Also turn the `lower` into a `has` option as only two drivers actually
support `uror` and `urol` at this momemt.

Fixes crashes with CL integer_rotate on iris and nouveau since we emit
urol for `rotate`.

v2: always lower 64 bit

Fixes: fe0965afa6 ("spirv: Don't use libclc for rotate")
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by (Intel and nir): Ian Romanick <ian.d.romanick@intel.com>

Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27090>
2024-01-22 10:27:44 +00:00
Jesse Natalie
90bf4b630b microsoft/compiler: Handle comparison bias/gradient sampling
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27160>
2024-01-20 00:07:15 +00:00
Jesse Natalie
733b7fe831 microsoft/compiler: Declare shader model 6.8 / validator 1.8
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27160>
2024-01-20 00:07:15 +00:00
Daniel Schürmann
bdbf873b0f nir: remove redundant passes from nir_opt_if()
These are now covered by nir_opt_loop():
- opt_if_loop_last_continue()
- opt_merge_breaks()
- opt_if_loop_terminator()

Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24940>
2024-01-03 20:48:05 +00:00
Yonggang Luo
0210b554d6 treewide: Replace the include of nir_types.h with glsl_types.h
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26753>
2023-12-30 15:08:11 +00:00
Faith Ekstrand
adafcacf15 microsoft: Stop claiming dot_4x8_sat support
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26533>
2023-12-06 23:15:33 +00:00