SPV_KHR_bfloat16 requires a small set of operations,
since it doesn't support all the arithmetic ops.
This patch adds conversions to/from Float32 and also
the necessary ops (bfdot, bffma, bfmul) to implement
SpvOpDot using the same lowering approach than the
Float32 counterpart.
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34105>
In float pointing rules adding +0.0f preserves all values except
for -0.0f, so what we want here is to add -0.0f. In the future
we should add proper support for float immediates in the assembler.
Fixes: fafdd24285 ("intel/executor: Update bfloat example")
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34105>
We were wrongly lowering all add/sub operations with saturation on 8-bit
values on v11+.
This fixes CTS failures on
"dEQP-VK.spirv_assembly.instruction.compute.opudotaccsatkhr.*" and
likely more apps.
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Fixes: d79a31bf81 ("pan/bi: Lower removed instructions in algebraic on v11+")
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34743>
This probably doesn't fix anything in practice. I don't think this path is
ever taken for SGPRs except for pseudo-scalar transcendental instructions,
and those don't have any precolored operands/definitions.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34343>
Aligning these can create situations where register allocation is
impossible. Only pseudo-instructions can use these, and they don't require
any alignment.
I'm not sure if these temporaries actually happen in practice. This
probably only affects the get_reg()'s compact_relocate_vars fallback path,
which doesn't usually happen for SGPRs.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34343>
D16 MIMG or pseudo-scalar transcendental instructions might give lower
limits to the maximum register available for their definitions, so just
try to place them earlier.
This is also part of fixing compact_relocate_vars with aligned NPOT
def/killed-op space (the second part is the later commit which changes
get_stride()).
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34343>
This gives more details on the hardware depth bias implementation details.
Signed-off-by: Mary Guillemard <mary.guillemard@collabora.com>
Reviewed-by: Lars-Ivar Hesselberg Simonsen <lars-ivar.simonsen@arm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34745>
The necessary bits have been added to the bifrost/valhall backend,
so we can advertise support for VK_EXT_shader_demote_to_helper_invocation
now.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Lars-Ivar Hesselberg Simonsen <lars-ivar.simonsen@arm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34676>
Bifrost/Valhall implementation is actually close to the demote behavior,
where discarded threads will continue execution until explicitly
terminated.
Given we'll need to support the demote behavior for Vulkan, let's use
nir_lower_terminate_to_demote(), which solves the problem, and allows
for extra dead-code elimination when a termination point is guaranteed
to be taken by all threads.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Lars-Ivar Hesselberg Simonsen <lars-ivar.simonsen@arm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34676>
The lane source passed to CLPER must be wrap-invariant, and we currently
violate this constraint in the !abs && !coarse case. So let's rework
the code to use a XOR modifier on the current lane instead, and adjust
the sign of the (righ - left) operation afterwards.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Lars-Ivar Hesselberg Simonsen <lars-ivar.simonsen@arm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34676>
This is already supported, we just need to toggle the switch.
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Lars-Ivar Hesselberg Simonsen <lars-ivar.simonsen@arm.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34676>
The src and dst queue family indices in an image memory barrier may
contain arbitrary values that can be ignored unless both are different.
This fixes a crash in upcoming CTS tests.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34691>
These stages are for the jobs that are skipped in merge pipelines,
automatically run in nightly pipelines, and are available to run
manually in other pipelines.
None of these ever run in post-merge pipelines.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34590>
While all the arm32 jobs are testing GL, there are both GL and VK arm64
jobs. This is required to be able to use the rootfs from the test-*
containers.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34451>
Add new job definitions using the new debian/x86_64-test-video container,
and convert the radeonsi-raven-va and radeonsi-raven-vaapi-fluster jobs
to use the rootfs derived from this container.
Now that we are no longer downloading the Fluster vectors at runtime, the
parallelsim of the radeonsi-raven-vaapi-fluster job can be greatly
decreased.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34451>
Most of the churn in this commit is changing unit tests that were
testing things that are now invalid.
shader-db:
All Intel platforms had similar results. (Lunar Lake shown)
total instructions in shared programs: 17122204 -> 17122669 (<.01%)
instructions in affected programs: 120669 -> 121134 (0.39%)
helped: 0 / HURT: 124
total cycles in shared programs: 895602370 -> 895613210 (<.01%)
cycles in affected programs: 17868974 -> 17879814 (0.06%)
helped: 35 / HURT: 85
fossil-db:
All Intel platforms had similar results. (Lunar Lake shown)
Totals:
Instrs: 210736518 -> 210743769 (+0.00%)
Cycle count: 30377733040 -> 30377699060 (-0.00%); split: -0.00%, +0.00%
Max live registers: 66056852 -> 66056966 (+0.00%)
Totals from 1505 (0.21% of 706776) affected shaders:
Instrs: 1890151 -> 1897402 (+0.38%)
Cycle count: 48397408 -> 48363428 (-0.07%); split: -0.11%, +0.04%
Max live registers: 256821 -> 256935 (+0.04%)
Reviewed-by: Matt Turner <mattst88@gmail.com>
Fixes: 020b0055e7 ("i965/fs: Propagate conditional modifiers from compares to adds")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34509>
When I originally wrote that code, I didn't understand what a jerk NaN
can be.
v2: Remove the brw_type_is_uint stuff. This function is currently only
called for float types. In a later commit, integer types will be
supported but only for NZ and Z conditions. Noticed by Matt.
shader-db:
All Intel platforms had similar results. (Lunar Lake shown)
total instructions in shared programs: 17122197 -> 17122204 (<.01%)
instructions in affected programs: 1691 -> 1698 (0.41%)
helped: 0 / HURT: 4
total cycles in shared programs: 895602484 -> 895602370 (<.01%)
cycles in affected programs: 912964 -> 912850 (-0.01%)
helped: 2 / HURT: 2
fossil-db:
All Intel platforms had similar results. (Lunar Lake shown)
Totals:
Instrs: 210736388 -> 210736518 (+0.00%)
Cycle count: 30377728900 -> 30377733040 (+0.00%); split: -0.00%, +0.00%
Totals from 130 (0.02% of 706776) affected shaders:
Instrs: 169911 -> 170041 (+0.08%)
Cycle count: 18021210 -> 18025350 (+0.02%); split: -0.00%, +0.02%
Reviewed-by: Matt Turner <mattst88@gmail.com>
Fixes: 020b0055e7 ("i965/fs: Propagate conditional modifiers from compares to adds")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34509>
The previous commit converts any NOT that might have been affected by
this path into a simple MOV. Those MOVs are handled by other paths.
No shader-db or fossil-db changes on any Intel platform.
v2: Fix a bad squash. Changes that were accidentally in this commit were
supposed to be in the previous commit. Noticed by Ivan.
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34509>
On Xe platforms, many fragment shaders have patterns like:
asr(8) g21<2>W g1.2<0,1,0>W 15D
...
mov(8) g11<1>UW g21<16,8,2>UW
...
not.nz.f0.0(8) null<1>D g11<8,8,1>W
Converting the NOT.NZ to MOV.Z enables copy propagation to eliminate the
original MOV. Then cmod propagation is able to eliminate the
NOT-converted-to-MOV.
It might be possible to cover this case by adding more opcodes to the
list NOT can propagate to. The next commit will show that just
converting to MOV is a better approach anyway.
v2: Fix a bad squash. Changes that were supposed to be in this commit
were accidentally in the next commit. Noticed by Ivan.
shader-db:
Meteor Lake, DG2, and Tiger Lake had similar results. (Meteor Lake shown)
total instructions in shared programs: 20069804 -> 20065167 (-0.02%)
instructions in affected programs: 592450 -> 587813 (-0.78%)
helped: 2300 / HURT: 0
total cycles in shared programs: 884534032 -> 884496201 (<.01%)
cycles in affected programs: 13064194 -> 13026363 (-0.29%)
helped: 1285 / HURT: 790
LOST: 18
GAINED: 15
fossil-db:
Meteor Lake, DG2, and Tiger Lake had similar results. (Meteor Lake shown)
Totals:
Instrs: 234506495 -> 234468664 (-0.02%)
Cycle count: 24444825202 -> 24445710703 (+0.00%); split: -0.01%, +0.01%
Max live registers: 42349793 -> 42349789 (-0.00%)
Max dispatch width: 7131344 -> 7131744 (+0.01%); split: +0.05%, -0.04%
Totals from 16673 (2.07% of 805781) affected shaders:
Instrs: 6497669 -> 6459838 (-0.58%)
Cycle count: 435877770 -> 436763271 (+0.20%); split: -0.54%, +0.74%
Max live registers: 1122972 -> 1122968 (-0.00%)
Max dispatch width: 151528 -> 151928 (+0.26%); split: +2.19%, -1.92%
No shader-db or fossil-db on any other Intel platforms.
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34509>
brw_opt_constant_fold_instruction can either do nothing or replace the
instruction with a MOV of an immediate value. Previously each opcode
case would perform this replacement, and code at the bottom of the
function would verify the results.
It is much simpler if each opcode case calculates a result in a brw_reg,
and code at the bottom of the function performs the replacement. There
are two outlier cases that cannot use this pattern: MAD and
BROADCAST. These cases simply return directly from the switch-statement
after performing the replacement.
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34707>
In multi-layered framebuffer LRZ also has several layers and binning
pass needs to write depth to a correct layer, so binning VS needs
VARYING_SLOT_LAYER.
Fixes: 9775b33d0f ("tu: Enable GMEM with layered rendering")
Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34728>
The hardware expects textureGrad (txd) derivatives to be specified as
offset coordinates from the base texture coordinate, rather than raw
gradients.
This change fixes the majority of the
dEQP-GLES3.functional.shaders.texture_functions.texturegrad.* tests on
GC7000. Remaining failures are due to shadow sampler handling, which
will be addressed separately.
Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34704>
This is already supported, we just need to toggle the switch.
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34695>
