We need to unref the device *after* submit cleanup, so that
fd_submit_sp_destory() can still reference the device.
Fixes: d558cb664a ("freedreno/drm: Submit should hold ref to device")
Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27445>
Follow the blob and optimize subgroup operation using brcst.active and
getlast when supported.
The transformation consists of two parts. First, a NIR transform
replaces subgroup operations with a sequence of new brcst_active_ir3
intrinsics followed by a new [type]_clusters_ir3 intrinsic (where type
can be reduce, inclusive_scan, or exclusive_scan).
The brcst_active_ir3 intrinsic is lowered directly to a brcst.active
instruction. The other intrinsics get lowered to a new macro
(OPC_SCAN_CLUSTERS_MACRO) which later gets emitted as a loop (using
getlast/getone) that iterates all clusters and produces the requested
scan result.
OPC_SCAN_CLUSTERS_MACRO has a number of optional arguments. First, since
the exclusive scan result is not a natural by-product of the loop but
has to be calculated explicitly, its destination is optional. This is
necessary since adding it unconditionally will produce unused
instructions that won't be DCE'd anymore at this point. Second, when
performing 32b MUL_U reductions (that expand to multiple instructions),
an extra scratch register is necessary.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/6387
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26950>
Moves to shared registers keep the value written by the lowest active
fiber. This means that read_first.macro can simply be turned into a mov
and doesn't need a getone block.
Signed-off-by: Job Noorman <jnoorman@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26950>
This is spurious and it looks like we should be able to uses non-zero
base level everytime on GFX6-8 but it doesn't always work.
This fixes the remaining CTS failures on GFX6-8.
Cc: mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26290>
This pass will later also serve as a way to accurately insert physical
edges, which is the original motivation. However it also lets us put
branchstack handling on a more solid footing.
There was an off-by-one in the old branchstack handling because it
didn't consider that a single if-else actually has two reconvergence
points active at the same time, so it undercounted the branchstack by 1
for pretty much every shader. We change the HW formula to produce the
same result, which now makes it much more sensible.
We can also delete the physical predecessor handling in ir3_legalize,
because it was only needed to handle (jp) which is now handled earlier.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22072>
We were relying on it in RA to tell us whether we could give more
registers to the shader mostly "for free" (because occupancy is bounded
by the branchstack), but it turns out it was actually 0 so we weren't
taking advantage of it.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22072>
There are two problems with shared register allocation at the moment:
1. We weren't modelling physical edges correctly, and once we do, the
current hack in RA for handling them won't work correctly. This means
live-range splitting doesn't work. I've tried various strategies but
none of them seems to fix this.
2. Spilling of shared registers to non-shared registers isn't
implemented.
Spilling of shared regs is significantly simpler than spilling
non-shared regs, because (1) spilling and unspilling is significantly
cheaper, just a single mov, and (2) we can swap "stack slots" (actually
non-shared regs) so all the complexity of parallel copy handling isn't
necessary. This means that it's much easier to integrate RA and
spilling, while still using the tree-scan framework, so that we can
spill instead of splitting live ranges. The other issue, of phi nodes
with physical edges, we can handle by spilling those phis earlier. For
this to work, we need to accurately insert physical edges based on
divergence analysis or else every phi node would involve physical edges,
which later commits will accomplish.
This commit adds a shared register allocation pass which is a
severely-cut-down version of RA and spilling. Everything to do with live
range splitting is cut from RA, and everything to do with parallel copy
handling and for spilling we simply always spill as long as soon as we
encounter a case where it's necessary. This could be improved,
especially the spilling strategy, but for now it keeps the pass simple
and cuts down on code duplication. Unfortunately there's still some
shared boilerplate with regular RA which seems unavoidable however.
The new RA requires us to redo liveness information, which is
significantly expensive, so we keep the ability of the old RA to handle
shared registers and only use the new RA when it may be required: either
something potentially requiring live-range splitting, or a too-high
shared register limit.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22072>
It seems we missed an error-case that got introduced in OpenGL 4.4.
While this error doesn't *technically* exist as-is in OpenGL ES before
version 3, neither does 3D textures. And while OES_texture_3D introduces
it to OpenGL ES 2.0 without adding the same error for ETC2 textures,
that is likely an omission in the spec; 3D ETC2 texture was never a
thing.
This fixes a regression in the confidential Khronos CTS, specifically
GL46.gtf42.GL3Tests.texture_storage.texture_storage_compressed_texture_data
Fixes: 652a898d316 ("mesa/main: add support for EXT_texture_storage")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10545
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Tested-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27428>
some passes (e.g., opt_shrink_vector) operate on the assumption that
sparse tex ops have a certain number of components and then remove components
and unset the sparse flag if they can optimize out the sparse usage
zink's sparse ops do not have the standard number of components, which
causes such passes to make incorrect assumptions and tag them as
not being sparse, which breaks everything
fix#10540
Fixes: 0d652c0c8d ("zink: shrink vectors during optimization")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27414>
the util function here takes a bitmask of memory type indices, not properties.
rename the function and correct the usage
fixes sparse on nvidia blob
Fixes: c71287e70c ("zink: correct sparse bo mem_type_idx placement")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27414>
With a VS prolog, we end up with 3 long jumps (VS prolog->VS->TCS->TCS
epilog), super annoying.
The shaders config must also be combined between VS and TCS.
This is for VK_EXT_shader_object.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27336>
When VS or TCS are compiled separately on GFX9+, the shader input args
must match. This is implemented using a complete separate path, it's
duplicated but it seems cleaner than adding a ton of checks here and
there.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27336>
With shader objects, both stages might not exist and if the src stage
doesn't, this will copy garbage data because it's unitialized.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27336>
This will be optimized during shader info linking if TCS is present.
The main motivation for this change is ESO because the next stage
might not exist.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27396>
When the state isn't dynamic, the patch control points value must
greater than 0. Having a separate field isn't necessary.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27396>
Since SIMD8 no longer exists, the SIMD modes enums have different names
and different values.
v2 (Francisco Jerez): Rebase on 07b9bfacc7 ("intel/compiler: Move
logical-send lowering to a separate file").
v3: Update brw_disasm.c with SIMD descriptions.
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27305>
The extra assertions are just there to help validate
pack_lod_and_array_index (in nir_lower_tex.c).
v2: Split got_lod_or_bias into two variables. This simplifies some
changes that Sagar is working on. Suggested by Sagar.
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27305>
Note: a future commit will expand the sampler message type to the 6 bits
used on Xe2.
v2 (Francisco Jerez): Rebase on 07b9bfacc7 ("intel/compiler: Move
logical-send lowering to a separate file").
v3: Drop XE2_SAMPLER_MESSAGE_SAMPLE_BIAS_MLOD as it does not actually
exist. This resulted in some bigger changes in brw_disasm.c. Noticed
by Sagar.
v4: Now that XE2_SAMPLER_MESSAGE_SAMPLE_MLODc conflicts with
GFX7_SAMPLER_MESSAGE_SAMPLE_GATHER4_PO_C, the determination of
min_lod_is_first must include devinfo->ver or previous platforms will
break.
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27305>
Message types are expanded to 6-bit encoding now. 5 bits are still the
same field from the Sampler Message Descriptor. The most significant bit
is now bit 31 of the Sampler Message Descriptor. The messages that have
'1 in bit 6 are only to support programmable offsets and those would
require message header. If a sampler type shows only 5 bits encoding, it
is implied bit 6 equal to 0 and there is no requirement for header.
v2 (idr): Trivial formatting changes.
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27305>
This small refactor simplifies a later commit that will optionally emit
some opcodes before the switch (as is already done with the shadow
comparitor).
v2 (Francisco Jerez): Rebase on 07b9bfacc7 ("intel/compiler: Move
logical-send lowering to a separate file").
v3 (Jordan): SHADER_OPCODE_TXL => SHADER_OPCODE_TXL_LZ (was
SHADER_OPCODE_TXF_LZ).
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27305>
The Bspec also says, "The table below describes the SIMD modes which
are supported. SIMD32 and SIMD64 are used for media-type operations
only." Perhaps this commit should just add
if (devinfo->ver >= 20)
return 16;
instead.
v2: Use reg_unit in get_sampler_lowered_simd_width. Suggested by Sagar.
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27305>
