Every case but SHADER_OPCODE_SEND and SHADER_OPCODE_BARRIER will be
lowered to SEND before register allocation happens. And the barrier
send has a null destination, so the restriction doesn't apply.
Note that this hack is for Gfx9 only, so we don't need to worry about
Xe3's SHADER_OPCODE_SEND_GATHER feature.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34040>
We used to have other opcodes as well, but we've since transitioned
entirely to logical send lowering prior to register allocation.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34040>
A number of places emit monolithic sends, where the second payload is
empty. Some places were using a BAD_FILE register, while others were
specifying the hardware ARF null register. Switch to BAD_FILE for
consistency - this is usually what we do for "source isn't present".
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34040>
The vulkan runtime doesn´t store this parameter in the dynamic state
(since it's not a dynamic state). Just capture it at compile time and
leave on the wm_prog_data.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36665>
It's not only for GL, change to a generic name.
Use command:
find . -type f -not -path '*/.git/*' -exec sed -i 's/\bgl_shader_stage\b/mesa_shader_stage/g' {} +
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36569>
These are identical, pull them into a helper so we only have one place
to fix bugs.
(Marked fixes because the next two patches depend on the refactor.)
Fixes: ec2e8bc33f ("intel/compiler: Avoid copy propagating large registers into EOT messages")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36577>
Setting variable names currently always uses ralloc, but the new
nir_variable_* helpers will mostly eliminate ralloc/malloc in a later
commit.
This just updates all places that touch nir_variable names to use the new
helpers.
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36538>
Despite the information in "Overview of Memory Access" (57046), the L3
seems to be smarter on Xe2+. See 4aa3b2d3ad ("anv: LNL+ doesn't need
the special flush for sparse").
The behavior is the same both with vm_bind and TR-TT.
v2: Add some comments (Caio).
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36150>
The comment was pasted from the commit message that added it. Remove
the parts that only make sense in the commit message, not in the final
code.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36150>
The residencyNonResidentStrict property requires that writes to
unbound memory be ignored and reads return zero. We need this
property, otherwise vkd3d will claim we don't support DX12.
If a shader writes to a variable associated with an unbound memory
region (i.e., mapped to a null tile), reads it back (in the same
shader) and expects the value be 0 instead of what is wrote, it has to
use the 'volatile' access qualifier to the variable associated with
the access, otherwise the compiler will be allowed to optmize things
and use the non-zero value. This is explained in the "Accessing
Unbound Regions" section of the Vulkan spec.
Our hardware adds an extra problem on top of the above. BSpec page
"Overview of Memory Access" (47630, 57046) says:
"If a read from a Null tile gets a cache-hit in a
virtually-addressed GPU cache, then the read may not return
zeroes."
So, when we detect this type of access, we have to turn off the
caching.
There's a proposed Vulkan CTS test that does exactly the above.
No shaders on shader_db seem to be using 'volatile'.
v2:
- Reorder commit order
- Rewrite commit message
v3:
- Rework the patch after Caio pointed out the interaction with
'coherent'.
- Remove previous R-B tags due to the patch differences.
v4:
- Rework the patch and commit message again after further
discussions.
v5:
- Check for atomic first so we don't regress DG2 atomic tests.
Fixes future test: dEQP-VK.sparse_resources.buffer.ssbo.read_write.sparse_residency_non_resident_strict
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36150>
The GLSL spec says (among other things):
"When a volatile variable is read, its value must be re-fetched from
the underlying memory, even if the shader invocation performing the
read had previously fetched its value from the same memory. When a
volatile variable is written, its value must be written to the
underlying memory, even if the compiler can conclusively determine
that its value will be overwritten by a subsequent write."
The SPIR-V spec says (among other things):
"Accesses to volatile memory cannot be eliminated, duplicated, or
combined with other accesses."
So in this commit we make sure that both writes and reads marked as
volatile can't be affected by CSE.
v2: Reorder patches in the series.
Credits-to: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> (v1)
Reviewed-by: Iván Briano <ivan.briano@intel.com> (v1)
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36150>
We seem to be ignoring the 'volatile' keyword coming from the shaders.
Record this in MEMORY_LOGICAL_FLAGS so we can use it later.
Credits-to: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Iván Briano <ivan.briano@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36150>
Most of the time with nir_def_rewrite_uses_after, you want to rewrite after the
replacement. Make that the default thing to be more ergonomic and to drop
parent_instr uses.
We leave nir_def_rewrite_uses_after_instr defined if you really want the old
signature with an arbitrary after point.
Via Coccinelle patch:
@@
expression a, b;
@@
-nir_def_rewrite_uses_after(a, b, b->parent_instr)
+nir_def_rewrite_uses_after_def(a, b)
Followed by a bunch of sed.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36489>
NIR is going to use exec_node/list without the C++ code, and may switch to
a different linked list implementation in the future.
GLSL is going to use ir_exec_node/list, which we want to keep private
for GLSL, so that we can change it easily.
Thus, it's better to fork the C++ version of list.h for Intel.
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36425>
In the C23 standard unreachable() is now a predefined function-like
macro in <stddef.h>
See https://android.googlesource.com/platform/bionic/+/HEAD/docs/c23.md#is-now-a-predefined-function_like-macro-in
And this causes build errors when building for C23:
-----------------------------------------------------------------------
In file included from ../src/util/log.h:30,
from ../src/util/log.c:30:
../src/util/macros.h:123:9: warning: "unreachable" redefined
123 | #define unreachable(str) \
| ^~~~~~~~~~~
In file included from ../src/util/macros.h:31:
/usr/lib/gcc/x86_64-linux-gnu/14/include/stddef.h:456:9: note: this is the location of the previous definition
456 | #define unreachable() (__builtin_unreachable ())
| ^~~~~~~~~~~
-----------------------------------------------------------------------
So don't redefine it with the same name, but use the name UNREACHABLE()
to also signify it's a macro.
Using a different name also makes sense because the behavior of the
macro was extending the one of __builtin_unreachable() anyway, and it
also had a different signature, accepting one argument, compared to the
standard unreachable() with no arguments.
This change improves the chances of building mesa with the C23 standard,
which for instance is the default in recent AOSP versions.
All the instances of the macro, including the definition, were updated
with the following command line:
git grep -l '[^_]unreachable(' -- "src/**" | sort | uniq | \
while read file; \
do \
sed -e 's/\([^_]\)unreachable(/\1UNREACHABLE(/g' -i "$file"; \
done && \
sed -e 's/#undef unreachable/#undef UNREACHABLE/g' -i src/intel/isl/isl_aux_info.c
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36437>
It's mostly used for SEND messages and fully defines the register data
(that's its purpose after all).
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36457>
The delivered values in the coarse pixel size are 0 when coarse pixel
dispatch is disabled and that is screwing up our half pixel offset
adjustment.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36457>
Xe2 adds two more flags registers. We barely use the second flags
register on previous platforms, so the omission was not previously
noticed.
There are several efforts in progress that will add using of more flags
registers.
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35415>
At this point, using the per-register granularity will only help in
conjuction with fragment shader discard (which is implemented using f1).
v2: Loop restructuring and code cleanups. Suggested by Curro.
v3: Only apply Wa on Gfx12.5+. Suggested by Curro.
v4: Also apply to implicit flag reads. Suggested by Curro. This version
affects a *lot* more shaders (10,936 on Meteor Lake shader-db versus
4,482 before). The results are still very much in the 🤷 territory.
v5: Add missing dependency. I thought I got them all the previous
time. :( Noticed by Curro.
shader-db:
Lunar Lake
total cycles in shared programs: 886315282 -> 886391040 (<.01%)
cycles in affected programs: 204907250 -> 204983008 (0.04%)
helped: 1 / HURT: 6716
LOST: 0
GAINED: 1
Meteor Lake and DG2 had similar results. (Meteor Lake shown)
total cycles in shared programs: 883774789 -> 883921507 (0.02%)
cycles in affected programs: 481836784 -> 481983502 (0.03%)
helped: 4 / HURT: 10936
LOST: 3
GAINED: 7
fossil-db:
Lunar Lake
Totals:
Cycle count: 32600441334 -> 32601862658 (+0.00%); split: -0.00%, +0.00%
Totals from 90283 (11.44% of 789260) affected shaders:
Cycle count: 17265933202 -> 17267354526 (+0.01%); split: -0.00%, +0.01%
Meteor Lake and DG2 had similar results. (Meteor Lake shown)
Totals:
Cycle count: 26477292677 -> 26480321805 (+0.01%); split: -0.00%, +0.01%
Max dispatch width: 8010440 -> 8010984 (+0.01%)
Totals from 132952 (14.71% of 903925) affected shaders:
Cycle count: 15349555348 -> 15352584476 (+0.02%); split: -0.00%, +0.02%
Max dispatch width: 1085416 -> 1085960 (+0.05%)
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35415>
