Indirect addressing(vx1 and vxh) not supported with UB/B datatype for
src0, so we need to change the data type for both dest and src0.
This fixes following tests cases on Xe2+
- dEQP-VK.spirv_assembly.instruction.compute.8bit_storage.push_constant_8_to_16*
- dEQP-VK.spirv_assembly.instruction.compute.8bit_storage.push_constant_8_to_32*
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29316>
We can do CSEL on F, HF, *W, and *D on Gfx11+. Gfx9 can only do F.
We can lower unsupported types to CMP+CSEL, allowing us to use CSEL
in the IR and not worry about the limitations.
Rework: (Sagar)
- Update validation pass for CSEL
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29316>
Reorganize the code to make clearer all the lowering cases:
(a) Single invocation workgroup. Index and IDs are all zero.
(b) Local ID provided by hardware.
(c) Local Index provided by the hardware. Depending on the case this
might not be the final local index, e.g. heuristics for tile.
(d) Neither provided by the hardware.
Case (c) is new and supported by Mesh/Task shaders. At the moment the
nir_lower_compute_system_values handle lowering of LocalID for
Task/Mesh, but a later patch will flip that on ANV.
This will make the Task/Mesh use the same lowering as Compute shaders.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29828>
Looks like some of the tests uses the bias which does not fit into half
float parameter, so it's better to use float param for sample_b.
If we have cube arrays, we anyway combine BIAS and array index properly
so we don't have to worry about the first parameter.
This fixes: GTF-GL46.gtf21.GL3Tests.texture_lod_bias.texture_lod_bias_clamp_m_g_M
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29533>
This has no impact on the generated shaders, but does have a small
(positive) impact on the amount of time spent in shader compilation.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29126>
Following https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28957,
some Xe2 code paths started triggering asserts.
In the cases fixed by this patch, it was because of the assert added
to brw_type_larger_of() in cf8ed9925f ("intel/brw: Make a helper for
finding the largest of two types"), and then brw_type_larger_of() is
used in 674e89953f. (For example, the assert was triggering when the
SHL types differed between D and UD.)
Fixes: 674e89953f ("intel/brw: Use new builder helpers that allocate a VGRF destination")
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29925>
When the destination of both instructions is NULL and the conditional
modifier matches, operands_match (by way of instructions_match) will
only test the first two operands. This can result in bad CSE
happening.
This is a very, very narrow edge case.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29848>
This one is a bit more complex in that we need to handle 3-source
commutative opcodes. But it's also quite useful:
fossil-db results on Alchemist (A770):
Instrs: 151659750 -> 150164959 (-0.99%); split: -0.99%, +0.01%
Cycles: 12822686329 -> 12574996669 (-1.93%); split: -2.05%, +0.12%
Subgroup size: 7589608 -> 7589592 (-0.00%)
Send messages: 7375047 -> 7375053 (+0.00%); split: -0.00%, +0.00%
Loop count: 46313 -> 46315 (+0.00%); split: -0.01%, +0.01%
Spill count: 110184 -> 54670 (-50.38%); split: -50.79%, +0.41%
Fill count: 213724 -> 104802 (-50.96%); split: -51.43%, +0.47%
Scratch Memory Size: 9406464 -> 3375104 (-64.12%); split: -64.35%, +0.23%
Our older Shadow of the Tomb Raider fossil is particularly helped with
over a 90% reduction in scratch access (spills, fills, and scratch
size). However, benchmarking in the actual game shows no change in
performance. We're thinking the game's shaders have been updated since
our capture.
Ian noted that there was a bug here where we'd accidentally CSE two ADD3
instructions with null destinations and different src[2] that couldn't
be dead code eliminated due to conditional mods. However, this is only
a bug in the new cse_defs pass so we don't need to nominate this for
stable branches.
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29848>
This implements a replacement for the previous implementation of
nir_intrinsic_load_barycentric_at_sample that relied on the Pixel
Interpolator shared function, since it's going to be removed from the
hardware from Xe2 onwards.
This implementation simply looks up the X/Y offsets of each sample
index on the table provided in the PS thread payload by using indirect
addressing, then does the actual interpolation by recursing into
emit_pixel_interpolater_alu_at_offset() introduced in the previous
commit.
Note that even though this is only immediately useful on Xe2+
platforms there's no reason why it shouldn't work on earlier
platforms, as long as we have the sample X/Y offsets available in the
thread payload.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29847>
This implements a replacement for the previous implementation of
nir_intrinsic_load_barycentric_at_offset that relied on the Pixel
Interpolator shared function, since it's going to be removed from the
hardware from Xe2 onwards.
That's okay since we can get all the primitive setup information
needed for interpolation at an arbitrary coordinate: We use the X/Y
offset relative to the "X/Y Start" coordinates from the thread payload
order to evaluate the plane equations also provided in the thread
payload for each barycentric coordinate of each polygon. The
evaluation of the barycentric plane equations (and the RHW plane
equation for perspective-correct interpolation) uses the accumulator
and MAD/MAC for ALU efficiency, but that means we need to manually
split instructions to fit the width of the accumulator. The division
and scaling for perspective-correct interpolation is also now done in
the shader if necessary.
Note that even though this is only immediately useful on Xe2+, the
thread payload numbers are filled out for older platforms, and the EU
restrictions of previous Xe platforms are taken into account, mostly
for the purposes of testing and performance evaluation.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29847>
Floating-point offsets work fine in combination with the
floating-point arithmetic we're about to lower these intrinsics into,
and they require less instructions than converting to fixed-point and
then back. No reason to take the precision/range hit nor the extra
instructions.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29847>
The ALU-based implementation of the barycentric interpolation
intrinsics introduced by a subsequent commit will require some
primitive setup information not delivered in the PS thread payload
unless explicitly requested:
- "Source Depth and/or W Attribute Vertex Deltas" if a
perspective-correct interpolation mode is used -- Note that this is
already requested for CPS interpolation, we just need to enable it
in more cases.
- "Perspective Bary Planes" if a perspective-correct interpolation
mode is used.
- "Non-Perspective Bary Planes" if a non-perspective-corrected
interpolation mode is used.
- "Sample offsets" if any at_sample interpolation is used so the
coordinate offsets of the sample can be calculated.
This ALU implementation of barycentric interpolation will only be
needed for *_at_offset and *_at_sample interpolation, since the fixed
function hardware still computes barycentrics for us at the current
sample coordinates, only the cases that previously relied on the Pixel
Interpolator shared function need to be re-implemented with ALU
instructions, since that shared function will no longer exist on Xe2
hardware.
Thanks to Rohan for a bugfix of the uses_sample_offsets calculation,
this patch includes his fix squashed in.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29847>
It turns out the problem I was trying to catch in be4fa59a72
("intel/brw: Clear write_accumulator flag when changing the
destination") also came from the DPAS lowering pass itself. Checking for
invalid uses of the feature in fs_validate helped detect the problem.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28834>
The original goal was to get rid of a bunch of the magic constants
sprinkled through the function. Once I did that, I realized that there
was a lot my symmertry between the row-major and column-major paths
possible.
It's +6 lines of code, but about 15 of those lines are comments
explaining things that were not obvious in the original code.
v2: Save duplicated condition in a variable with a meaningful
name. Suggested by Caio.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28834>
Even though the hardware does not naively support these configurations,
there are many potential benefits to advertising them. These
configurations can theoretically use half the memory bandwidth for loads
and stores. For large matrices, that can be the limiting in performance.
The current implementation, however, has a number of significant
problems.
The conversion from float16 to float32 is performed in the driver during
conversion from NIR. As a result, many common usage patterns end up
doing back-to-back conversions to and from float16 between matrix
multiplications (when the result of one multiplication is used as the
accumulator for the next).
The float16 version of the matrix waste half the possible register
space. Each float16 value sits alone in a dword. This is done so that
the per-invocation slice of an 8x8 float16 result matrix and an 8x8
float32 result matrix will have the same number of elements. This makes
it possible to do straightforward implementations of all the unary_op
type conversions in NIR.
It would be possible to perform N:M element type conversions in the
backend using specialized NIR intrinsics. However, per #10961, this
would be very, very painful. My hope is that, once a suitable resolution
for that issue can be found, support for these configs can be restored.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28834>
There are a lot of places where we add 0 to an offset. Avoiding
generating this can save us algebraic + copy_propagation later.
Cuts compile time in Borderlands 3 by -0.590631% +/- 0.170108% (n=25).
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29849>
Instead of replicating the whole thing in macros, just make an alu2()
function and use that in the wrappers. It ought to get inlined anyway.
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29849>
Wait and see if we actually have a candidate for saturate propagation
before requesting liveness info. Saves the calculation in the case
where we have nothing to do.
Cuts compile time in Borderlands 3 by -0.304754% +/- 0.194162% (n=25).
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29849>
This intrinsic is pretty much equivalent to
load_global_constant_uniform_block_intel, it just has a predicate. If
the predicate is always true we can turn into into the other.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29663>
When the surface handles are generated with exec_all, we can avoid
emitting the workaround.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29663>
The default case should lead us to the next rematerialization block so
this is useless.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29663>
Some of the instructions we don't need to rematerialize because we
already know they are executed with NoMask so we can use their
destination without reemitting them again.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29663>
The r0.5 thread payload register contains Surface State Offset bits
[27:6] as bits [31:10], so we need to shift the register right by 4 in
order to get the surface state offset expected in ExBSO mode, which is
the only extended descriptor encoding supported by the UGM shared
function for SS addressing on Xe2+.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29543>
We no longer need to reserve registers for constructing spill/fill
messages. We have split sends and construct message headers in new
temporary registers with a very short lifespan which are simply added
to the existing interference graph as new nodes and allocated via the
normal mechanism.
This means that when we need to spill for the first time, we can avoid
discarding and recomputing the entire interference graph. We also avoid
needing to recreate all spill candidate information once ra_allocate()
fails, because the graph remains valid, and none of the existing nodes
had any changes to their interference. The existing spill candidates
remain valid.
This will slightly help improve compile time when needing to spill.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25811>
