This patch also replaces lower_negate with lower_ineg / lower_fneg.
The fneg semantics have been clarified as of Version 1.5, Revision 1
of the SPIR-V specification, which means that the previous lowering
to fsub is not a viable solution anymore, and is replaced with
lowering to fmul(x, -1.0).
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6597>
If multiple rules could match, the rule that appears first in the file
is used.
Only Tiger Lake and Ice Lake are affected. Other platforms either have
a LRP instruction or can't run any shaders from shader-db that would
benefit.
v2: Fix issues created when this commit was rebased on top of
3c8934a644 ("nir/algebraic: add flrp patterns for 16 and 64 bits").
Noticed by Caio.
Tiger Lake and Ice Lake had similar results.
total instructions in shared programs: 20908672 -> 20908661 (<.01%)
instructions in affected programs: 419 -> 408 (-2.63%)
helped: 5
HURT: 0
helped stats (abs) min: 1 max: 3 x̄: 2.20 x̃: 3
helped stats (rel) min: 1.85% max: 3.19% x̄: 2.49% x̃: 2.65%
95% mean confidence interval for instructions value: -3.56 -0.84
95% mean confidence interval for instructions %-change: -3.24% -1.73%
Instructions are helped.
total cycles in shared programs: 473513940 -> 473513793 (<.01%)
cycles in affected programs: 7176 -> 7029 (-2.05%)
helped: 12
HURT: 0
helped stats (abs) min: 5 max: 22 x̄: 12.25 x̃: 12
helped stats (rel) min: 0.84% max: 3.24% x̄: 2.09% x̃: 1.80%
95% mean confidence interval for cycles value: -15.43 -9.07
95% mean confidence interval for cycles %-change: -2.57% -1.61%
Cycles are helped.
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6358>
This prevents other transformations from converting them to 'a != 0'.
For example, both of these transformations can do this:
(('~flt', 0.0, ('fabs', a)), ('fne', a, 0.0)),
(('~flt', ('fneg', ('fabs', a)), 0.0), ('fne', a, 0.0)),
Both fsign(fabs(NaN)) and fsign(fneg(fabs(NaN))) should produce zero,
but, since 'NaN != 0.0' is true, cascading these transformations could
cause them to generate 1.0 or -1.0 respecively.
No shader-db or fossil-db changes on any Intel platform.
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6358>
No shader-db or fossil-db changes on any Intel platform.
v2: Add a coding line to fix SCons build problems caused by the ±
character.
Fixes: 25bfba3335 ("nir/algebraic: Recognize open-coded copysign(1.0, a)")
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6358>
This prevents some fossil-db regressions in "spir-v: Mark floating point
comparisons exact".
v2: Note that the patterns and replacements produce the same value when
isnan(b). Suggested by Caio.
v3: Use C99 isfinite() instead of (obsolete) BSD finite(). Fixes
various Windows builds.
No fossil-db changes on any Inetl platform, Vega, or Polaris10.
All Intel platforms had similar results. (Tiger Lake shown)
total instructions in shared programs: 20908670 -> 20908672 (<.01%)
instructions in affected programs: 69 -> 71 (2.90%)
helped: 0
HURT: 1
total cycles in shared programs: 473515288 -> 473513940 (<.01%)
cycles in affected programs: 4942 -> 3594 (-27.28%)
helped: 2
HURT: 0
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6358>
This also prevents some fossil-db regressions in "spir-v: Mark floating
point comparisons exact".
v2: Mark the fmin / fmax in the replacement exact to prevent other
optimizations from ruining the NaN-clensing property of the fmin / fmax.
Suggested by Rhys. Don't assume that constants are not NaN because some
components of a vector might be NaN while others are numbers. Noticed
by Rhys. This causes ~8 more shaders in Age of Wonders III (dxvk) to
regress on cycles (not instructions) by less than 1% when "spir-v: Mark
floating point comparisons exact" is applied. This difference is too
small to care.
All Intel platforms had similar results. (Tiger Lake shown)
total instructions in shared programs: 20908668 -> 20908670 (<.01%)
instructions in affected programs: 9196 -> 9198 (0.02%)
helped: 10
HURT: 5
helped stats (abs) min: 1 max: 2 x̄: 1.40 x̃: 1
helped stats (rel) min: 0.02% max: 5.41% x̄: 2.20% x̃: 2.16%
HURT stats (abs) min: 2 max: 6 x̄: 3.20 x̃: 3
HURT stats (rel) min: 2.44% max: 16.67% x̄: 9.39% x̃: 12.50%
95% mean confidence interval for instructions value: -1.22 1.49
95% mean confidence interval for instructions %-change: -2.08% 5.41%
Inconclusive result (value mean confidence interval includes 0).
total cycles in shared programs: 473515330 -> 473515288 (<.01%)
cycles in affected programs: 67146 -> 67104 (-0.06%)
helped: 10
HURT: 7
helped stats (abs) min: 1 max: 36 x̄: 15.90 x̃: 17
helped stats (rel) min: 0.01% max: 1.29% x̄: 0.66% x̃: 0.89%
HURT stats (abs) min: 1 max: 48 x̄: 16.71 x̃: 4
HURT stats (rel) min: 0.08% max: 1.94% x̄: 0.87% x̃: 0.19%
95% mean confidence interval for cycles value: -13.88 8.94
95% mean confidence interval for cycles %-change: -0.56% 0.49%
Inconclusive result (value mean confidence interval includes 0).
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6358>
The reordered are automatically considered by nir_algebraic rules for
commutative instructions.
No shader-db or fossil-db changes on any Intel platform.
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6358>
I originally noticed that 3b30814791 ("nir/algebraic: Optimize 1-bit
Booleans") caused this pattern no longer be matched by incorrectly
replacing b@32 with b@1. Making that correct had no effect on
shader-db. When this pattern originally was added, it only affected 4
shaders, so it's not worth the effort to debug further.
This reverts commit f50400cc80.
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6358>
The original comment was a little terse and a little incorrect. The
rearrangements are fine w.r.t. NaN. However, they produce incorrect
results if one operand is +Inf and the other is -Inf.
A later commit, "nir/algebraic: Add some compare-with-zero optimizations
that are exact", will add some more patterns here. It may be reasonable
to squash this commit (forward) into that commit.
v2: Fix some incorrect comparisons operators in the comment (<= vs >=).
Add commentary that subtraction works like addition w.r.t. NaN. Both
noticed / suggested by Caio.
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6358>
This commit only documents the current behavior, even if that behavior
is not the behavior preferred by the relevant specs.
In SPIR-V, there are two flavors of the sign instruction, and each lives
in an extended instruction set. The GLSL.std.450 FSign instruction is
defined as:
Result is 1.0 if x > 0, 0.0 if x = 0, or -1.0 if x < 0.
This also matches the GLSL 4.60 definition.
However, the OpenCL.ExtendedInstructionSet.100 sign instruction is
defined as:
Returns 1.0 if x > 0, -0.0 if x = -0.0, +0.0 if x = +0.0, or -1.0 if
x < 0. Returns 0.0 if x is a NaN.
There are two differences. Each treats -0.0 differently, and each also
treats NaN differently. Specifically, GLSL.std.450 FSign does not
define any specific behavior for NaN.
There has been some discussion in Khronos about the NaN behavior of
GLSL.std.450 FSign. As part of that discussion, I did some research
into how we treat NaN for nir_op_fsign, and this commit just captures
some of those notes.
v2: Document the expected behavior of nir_op_fsign more thoroughly.
Suggested by Rhys. Note that the current implementation of constant
folding does not produce the expected result for NaN. Suggested by
Caio.
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> [v1]
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6358>
As part of this series, it removes the need for float->double conversion,
just to be able to print a single float.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8254>
In commit 00b28a50b2, Marek extended
a number of optimizations that had been 32-bit specific to work on
other bit-sizes.
Most optimizations preserve the data type across the transformation.
In other words, an optimization which generates e.g. fp64 operations
only does so when the source expression also contains fp64 operations.
These transformations are fine with respect to lowering, because we
will lower away all expressions that would trigger the search portion
of the expression, and so we'd never apply those rules.
However, a few of the rules create new operations that run afoul of
lowering passes. For example,
('bcsel', a, 1.0, 0.0) => ('b2f', a)
where the result is a double would simply be a selection between two
different 64-bit constants. The replacement expression, on the other
hand, involves a nir_op_b2f64 ALU operation. If we're run after
nir_lower_doubles, then it may not be legal to generate such an
expression anymore (at least without running lowering again, which we
don't do today).
Regressions due to this are blocking the 20.3 release, so for now, we
take the easy route and simply disallow those few rules when doing full
softfp64 lowering, which fixes the immediate problem. But it doesn't
solve the long-term problem in an extensible manner.
In the future, we may want to add a `lowered_alu_ops` bitfield to the
NIR shader, and as lowering passes are run, mark them as taboo. Then,
we could have each algebraic transformation track which operations it
creates in the replacement expression. With both of those in place,
nir_replace_instr could compare the transformation's list of ALU ops
against `lowered_alu_ops` and implicitly skip rules that generate
forbidden ALU operations.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3504
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7841>
All these instructions replicate the result of a N-component dot-product
to a vec4. Naming them fdot_replicatedN gives the impression that are
some sort of abstract dot-product that replicates the result to a vecN.
They also deviate from fdph_replicated... which nobody would reasonably
consider naming fdot_replicatedh.
Naming these opcodes fdotN_replicated more closely matches what they
are, and it matches the pattern of fdph_replicated.
I believe that the only reason these opcodes were named this way was
because it simplified the implementation of the binop_reduce function in
nir_opcodes.py. I made some fairly simple changes to that function, and
I think the end result is ok.
The bulk of the changes come from the sed rename:
sed --in-place -e 's/fdot_replicated\([234]\)/fdot\1_replicated/g' \
$(grep -r 'fdot_replicated[234]' src/)
v2: Use a named parameter to binop_reduce instead of using
isinstance(name, str). Suggested by Jason.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5725>
Before 8e1b75b330 ("nir/algebraic: optimize iand/ior of (n)eq zero") this
optimization didn't need the use of umax/umin. VC4 HW supports only signed
integer max/min operations.
lower_umin and lower_umax are added to allow enabling previous optimizations
behaviour for this cases.
Fixes: 8e1b75b330 ("nir/algebraic: optimize iand/ior of (n)eq zero")
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7083>
This reverts commit 939ddf3f67.
Intel has a separate pass for fusing FFMAs selectively. We split
these flags in commit 1b72c31e1f and
the reasoning still stands. The patch being reverted was just a
cleanup, so there should be no issue with reverting it.
Acked-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6849>
v2: move *2*mp patterns to the end of late_optimizations
v3: remove ftrunc from the optimizations to fix:
dEQP-GLES3.functional.shaders.builtin_functions.common.modf.vec2_lowp_vertex
Reviewed-by: Rob Clark <robdclark@chromium.org> (v1)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6283>
It would be nice if we could do swizzling of an expression on the
replacement side so that we could have a single ieq/ine of the vector
after CSE. However, if you do want vector operations, nir_opt_vectorize()
does just fine.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6567>
If a is Nan, fsat(NaN) is expected to be 0 and some optimizations
should be marked as inexact.
Fixes a GPU hang with Death Stranding and RADV/ACO (RADV/LLVM
isn't affected because it lowers fsat).
No fossils-db change.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3368
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6519>
These consist of the variations nir_op_{i|u|f}{min|max|med}3 which are either
lowered in the backend (LLVM) anyway or can be recombined by the backend (ACO).
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6421>
Microsoft's DXIL is based on LLVM, which doesn't have an integer ABS
opcode, but instead needs it lowered to NEG + MAX. We need to do this
with an option, to prevent an already existing optimization rule from
undoing this.
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5211>
It was always fneu but naming it fne causes confusion from time to time. So
lets rename it. Later we also want to add other unordered and fne, this is
a smaller preparation for that.
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6377>
