Commit graph

177607 commits

Author SHA1 Message Date
Konstantin Seurer
df710fe695 radv/rt: Enable monolithic pipelines
Store can_inline inside the stages to avoid rerunning the analysis pass
for library stages.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21929>
2023-09-10 11:40:12 +00:00
Konstantin Seurer
f2514e75f0 radv/rt: Add monolithic raygen lowering
Ray traversal is inlined to allow for constant folding and avoid
spilling.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21929>
2023-09-10 11:40:12 +00:00
Konstantin Seurer
e039e3cd76 radv/rt: Store NIR shaders separately
In order to compile monolithic shaders with pipeline libraries, we need
to keep the NIR around for inlining recursive stages.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21929>
2023-09-10 11:40:12 +00:00
Mike Blumenkrantz
39fca243bb nir/inline_uniforms: fix oob access with nir_find_inlinable_uniforms
the array dimensionality needs to match nir_add_inlinable_uniforms even if
only the first member is used

Fixes: 0c0fb216dd ("nir/inline_uniforms: Allow possibility of more than one UBO")

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25063>
2023-09-09 16:40:46 +00:00
Mike Blumenkrantz
94941de25b zink: delete all psiz=1.0 stores if maintenance5 is present
this frees up an output location woooo

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24782>
2023-09-09 15:44:45 +00:00
Mike Blumenkrantz
23df2bf41b zink: slightly refactor psiz deletion during linking
no functional changes

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24782>
2023-09-09 15:44:45 +00:00
Eric Engestrom
b5c2e91e4a Revert "ci: taking igalia farm offline"
This reverts commit a69ffbd08a.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25136>
2023-09-09 14:08:23 +00:00
Mike Blumenkrantz
6d5174974a zink: use HIC for image subdata when possible
this has a lot of caveats:
* extension must be supported
* resource must have usage bit set
* resource must not have any pending batch usage
* resource must be in supported layout

if all of these conditionals pass, then HIC can be used for direct image subdata

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:44 +00:00
Mike Blumenkrantz
f24891269d zink: check/use suboptimal HIC during ici init
this allows implicit use of HIC where possible while rejecting it when
it would cause performance loss

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:44 +00:00
Mike Blumenkrantz
e006a3d8bc zink: use some return codes for check_ici errors
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:44 +00:00
Mike Blumenkrantz
4feb37d629 zink: fix some off-by-one indentation
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:44 +00:00
Mike Blumenkrantz
9907573d98 zink: add a fixup method for extra driver props
some extensions have "extra" props which need the get_count -> get_prop_array
dance, and codegen is too stupid to figure this out (and probably always will be)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:44 +00:00
Mike Blumenkrantz
9ff5026100 zink: disable HIC without resizable BAR
this otherwise ooms the system

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:44 +00:00
Mike Blumenkrantz
f95134468a zink: move mem type detection up in file
no functional changes

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:44 +00:00
Mike Blumenkrantz
521800cf59 zink: hook up VK_EXT_host_image_copy
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:44 +00:00
Mike Blumenkrantz
5edcab3385 lavapipe: don't advertise UNDEFINED layout for HIC
this is illegal

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:44 +00:00
Mike Blumenkrantz
943909514e lavapipe: handle VkHostImageCopyDevicePerformanceQueryEXT
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:44 +00:00
Mike Blumenkrantz
87fa46c10d zink: use VkFormatProperties3
but wrap it in a smaller type to save some space

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:44 +00:00
Mike Blumenkrantz
e02441b000 zink: simplify redundant is_buffer check
it's in the params

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24775>
2023-09-09 13:02:43 +00:00
Mike Blumenkrantz
738eb0d78c ci: bump VVL to 1.3.263
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24925>
2023-09-09 11:29:36 +00:00
Jordan Justen
ddc3c18e4a intel/dev: Update device string for MTL PCI ID 0x7d55
Ref: bspec 55420
Cc: mesa-stable
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Acked-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25073>
2023-09-09 07:00:30 +00:00
Faith Ekstrand
bb91e0306c nvk: Invalidate the texture cache in PipelineBarrier
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25135>
2023-09-09 05:17:05 +00:00
Faith Ekstrand
dff769e2bd nvk: Set the discard bit for Z/S self-deps
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25135>
2023-09-09 05:17:05 +00:00
Faith Ekstrand
35e0989779 nvk: Don't add a dummy attachment when gl_SampleMask is written
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25135>
2023-09-09 05:17:05 +00:00
Ian Romanick
8ce4d7a08d intel/compiler: Don't evict for workgroup-scope fences
Flushing and invalidating caches isn't necessary for workgroup scope
fences.  In fact, the DP_FLUSH_TYPE docs (BSpec 54041) say:

   "If the fence scope is Local or Threadgroup, HW ignores the flush
    type and operates as if it was set to None(no flush)"

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24842>
2023-09-09 04:41:25 +00:00
Ian Romanick
5eddf60e56 intel/compiler: Combine control barriers with identical memory semantics
This prevents the second barrier generating a spurious, identical fence
message as the first barrier.

fossil-db stats on Alchemist:

   Totals:
   Instrs: 196513342 -> 196512777 (-0.00%); split: -0.00%, +0.00%
   Cycles: 14271426028 -> 14271404569 (-0.00%); split: -0.00%, +0.00%
   Send messages: 8021892 -> 8021770 (-0.00%)

   Totals from 46 (0.01% of 653252) affected shaders:
   Instrs: 76761 -> 76196 (-0.74%); split: -0.75%, +0.01%
   Cycles: 2027946 -> 2006487 (-1.06%); split: -1.45%, +0.39%
   Send messages: 7589 -> 7467 (-1.61%)

Nothing in shader-db was affected.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24842>
2023-09-09 04:41:25 +00:00
Kenneth Graunke
9f98f20c58 anv: Use nir_opt_barrier_modes() to drop unnecessary barriers
fossil-db stats on Alchemist:

   Totals:
   Instrs: 196514947 -> 196513342 (-0.00%); split: -0.00%, +0.00%
   Cycles: 14271450761 -> 14271426028 (-0.00%); split: -0.00%, +0.00%
   Send messages: 8022316 -> 8021892 (-0.01%)

   Totals from 43 (0.01% of 653252) affected shaders:
   Instrs: 98558 -> 96953 (-1.63%); split: -1.63%, +0.00%
   Cycles: 15867801 -> 15843068 (-0.16%); split: -0.17%, +0.02%
   Send messages: 8997 -> 8573 (-4.71%)

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24842>
2023-09-09 04:41:24 +00:00
Kenneth Graunke
2b14618daa glsl: Use nir_opt_barrier_modes() to drop unnecessary barriers
iris shader-db stats on Alchemist:

    total instructions in shared programs: 23150249 -> 23142733 (-0.03%)
    instructions in affected programs: 157322 -> 149806 (-4.78%)
    helped: 105
    HURT: 2
    helped stats (abs) min: 2 max: 821 x̄: 71.61 x̃: 15
    helped stats (rel) min: 0.13% max: 27.56% x̄: 6.21% x̃: 2.35%
    HURT stats (abs)   min: 1 max: 2 x̄: 1.50 x̃: 1
    HURT stats (rel)   min: 0.18% max: 0.23% x̄: 0.20% x̃: 0.20%
    95% mean confidence interval for instructions value: -101.99 -38.50
    95% mean confidence interval for instructions %-change: -7.59% -4.58%
    Instructions are helped.

    total sends in shared programs: 1036916 -> 1035366 (-0.15%)
    sends in affected programs: 15274 -> 13724 (-10.15%)
    helped: 108 / HURT: 0
    helped stats (abs) min: 1 max: 162 x̄: 14.35 x̃: 3
    helped stats (rel) min: 0.88% max: 33.83% x̄: 9.81% x̃: 5.05%
    95% mean confidence interval for sends value: -20.79 -7.92
    95% mean confidence interval for sends %-change: -11.66% -7.95%

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24842>
2023-09-09 04:41:24 +00:00
Kenneth Graunke
5754461f05 dxil: Set UAV_FENCE_THREAD_GROUP any time global isn't required
With the new nir_opt_barrier_modes() pass, we may encounter control
barriers with no memory modes set, such as:

   @barrier () (execution_scope=WORKGROUP, memory_scope=WORKGROUP, mem_semantics=ACQ|REL, mem_modes=0)

The DXIL validator documentation [1] mentions an
INSTR.BARRIERMODENOMEMORY validation rule:

   "sync must include some form of memory barrier - _u (UAV) and/or
    _g (Thread Group Shared Memory). Only _t (thread group sync) is
    optional."

We were generating a dx.op.barrier instruction with only one flag,
DXIL_BARRIER_MODE_SYNC_THREAD_GROUP.  This seems to run afoul of the
above validator rule.  So, this patch adjusts the code generator to
set DXIL_BARRIER_MODE_UAV_FENCE_THREAD_GROUP too, whenever
UAV_FENCE_GLOBAL isn't required.

[1] https://github.com/microsoft/DirectXShaderCompiler/blob/main/docs/DXIL.rst

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24842>
2023-09-09 04:41:24 +00:00
Kenneth Graunke
fb3e37a014 virgl, nir_to_tgsi: Add a hack for promoting partial memory barriers
Most drivers will want nir_opt_barrier_modes() to optimize out
unnecessary memory barrier modes.  However, virgl has to translate
back to GLSL, which means it can really only handle partial memory
barriers in compute shaders today, because there isn't a proper
way to express them otherwise.  Just ask nir_to_tgsi to promote
these back to full barriers as a workaround.

See KHR-GL43.shader_storage_buffer_object.advanced-readWrite-case1
on virpipe-on-gl as a case where this hack is needed.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24842>
2023-09-09 04:41:24 +00:00
Kenneth Graunke
dd92fd8fcc lavapipe: Don't delete control barriers
Control barriers still need to do synchronization even if there are no
associated memory barrier modes.

Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24842>
2023-09-09 04:41:24 +00:00
Kenneth Graunke
fc0aaa81ee nir: Reduce the scope of shared memory barriers
Originally written by Ian Romanick for the Intel backend, but ported
to the new nir_opt_barrier_modes() common optimization pass.  Ian's
original explanation and commit message follows:

Shared memory only exists within a workgroup, so synchronizing it beyond
workgroup scope is nonsense.

Basically every SPIR-V compiler generates operations like

    OpMemoryBarrier(/*Memory*/Device,
                    /*Semantics*/AcquireRelease | WorkgroupMemory)

This is suggested in numerous places, including
https://github.com/KhronosGroup/GLSL/blob/master/extensions/khr/GL_KHR_vulkan_glsl.txt.
Even Mesa's glsl_to_nir pass does this. This advice, which has been
copy-and-pasted everywhere, is contrary to issue 13 in the original
GL_ARB_compute_shader spec:

   "Since shared memory is only accessible to threads within a single
    work group, memoryBarrierShared() also only requires synchronization
    with other threads in the same work group."

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24842>
2023-09-09 04:41:24 +00:00
Kenneth Graunke
7dd897e1cd nir: Add an optimization pass to reduce barrier modes
Many shaders issue full memory barriers, which may need to synchronize
access to images, SSBOs, shared local memory, or global memory.
However, many of them only use a subset of those memory types - say,
only SSBOs.

Shaders may also have patterns such as:

   1. shared local memory access
   2. barrier with full variable modes
   3. more shared local memory access
   4. image access

In this case, the barrier is needed to ensure synchronization between
the various shared memory operations.  Image reads and writes do also
exist, but they are all on one side of the barrier, so it is a no-op for
image access.  We can drop the image mode from the barrier here too.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24842>
2023-09-09 04:41:24 +00:00
Kenneth Graunke
1c3706fc28 nir: Fix function parameter indentation in nir_opt_barriers.c
The first parameter should be on the first line, and any subsequent
lines should line up.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24842>
2023-09-09 04:41:24 +00:00
Mike Blumenkrantz
17a35412dc zink: re-rework i/o variable handling to make having variables entirely optional
old variables are now only used for copying names if possible, which should
make it possible for zink to process shaders which have no variables at all

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:44 +00:00
Mike Blumenkrantz
8b4904405e zink: use right function to get src_type in eliminate_io_wrmasks
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:44 +00:00
Mike Blumenkrantz
459b49a174 zink: add a new linker pass to handle mismatched i/o components
this is the inverted version of rewrite_read_as_0 which tests for mismatched
component i/o on a given location and rewrites the inputs to zero if the
producer shader didn't write to the component

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:44 +00:00
Mike Blumenkrantz
9af2f17937 zink: create new vars without copying existing ones
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:44 +00:00
Mike Blumenkrantz
86668052dd zink: use explicit sizing for builtins when creating variables
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:44 +00:00
Mike Blumenkrantz
53dab1cf40 zink: use MAX_PATCH_VERTICES directly for arrayed io var sizing
no functional changes

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:44 +00:00
Mike Blumenkrantz
e81048a0e2 zink: use explicit stride from types instead of copying old_var stride
should be no functional changes

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:44 +00:00
Mike Blumenkrantz
3c422ba518 zink: simplify an arrayed io check during variable creation
no functional changes

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:44 +00:00
Mike Blumenkrantz
dcc1d115f9 zink: use nir_io_semantics::num_slots for indirect var creation
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:43 +00:00
Mike Blumenkrantz
a91d920a63 zink: delete some bindless io lowering code
now that variables are pre-converted this is no longer necessary

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:43 +00:00
Mike Blumenkrantz
7fdc74b078 zink: fix typing on bindless io lowering
with lowered io this should always be an ivec2

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:43 +00:00
Mike Blumenkrantz
a3ce422a99 zink: reorder bindless io lowering
should be no functional changes

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:43 +00:00
Mike Blumenkrantz
e87b24719f zink: set is_xfb=false for all i/o variables
this can affect streamout generation, even though it so far hasn't

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24950>
2023-09-09 04:15:43 +00:00
Mike Blumenkrantz
20cbc6b9a0 zink: ci updates
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24962>
2023-09-09 03:44:05 +00:00
Mike Blumenkrantz
39b814b2a6 zink: handle multi-plane implicit sync
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24962>
2023-09-09 03:44:05 +00:00
Mike Blumenkrantz
d4f8ad27f2 zink: handle implicit sync for dmabufs
this adds explicit queue transitions to FOREIGN at the end of the batch
for all written-to dmabufs, then also adds signal/wait semaphores
using the dmabuf fds

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24962>
2023-09-09 03:44:05 +00:00