Commit graph

6729 commits

Author SHA1 Message Date
Kenneth Graunke
0510a947ba anv: Fix dynamic primitive topology for tess on Gfx7.x too
Commit 24342e499b changed how primitive
topology is handled on Gfx8+ but missed updating the Gfx7.x code.

As a result, tests which previously used topologies like PATCHLIST_3
instead started using bogus ones like LINESTRIP_ADJ.  This caused a
GPU hangs in a bunch of Vulkan conformance tests involving tessellation.

This fixes those hangs.

Fixes: 24342e499b ("anv: fix dynamic primitive topology for tess")
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11434>
2021-06-17 02:00:32 +00:00
Yiwei Zhang
357e7ae30b anv: fix build errors after commit 8b7ff78
Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Roman Stratiienko <r.stratiienko@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11373>
2021-06-16 19:55:48 +00:00
Mike Blumenkrantz
24342e499b anv: fix dynamic primitive topology for tess
this needs to use the pre-converted topology using tess state patch control points

Fixes: f6fa4a8000 ("anv: add support for dynamic primitive topology change")

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11332>
2021-06-16 13:45:15 +00:00
Jason Ekstrand
7bf0c762c3 anv: Agressively no-op Flush/InvalidateMappedMemoryRanges
This has two steps.  First, for each range we look at the memory object
and see if it actually needs flushing before we start throwing CLFLUSH
instructions.  Second, we look at the whole list of types on device
initialization and decide whether or not we need CLFLUSH at all.  The
first part should speed up atom chips a bit since we're currently
CLFLUSHing everything even when we don't need to.  The second isn't
needed on most of today's parts because we base it on !has_llc but it is
needed for discrete parts.  It's also over-all cleaner.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Tested-by: Felix DeGrood <felix.j.degrood@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11364>
2021-06-15 21:00:37 +00:00
Felix DeGrood
a49b145e8d anv: Replace DC Flush with HDC Pipeline Flush
HDC Pipeline Flush is the correct method for flushing HDC
pipeline on Gfx12+ HW. Continue using DC Flush for earlier HW.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>
2021-06-15 12:57:42 +00:00
Felix DeGrood
ef70388a3a anv: Add ANV_PIPE_HDC_PIPELINE_FLUSH_BIT
Gfx12+ PIPE_CONTROL bit for flushing HDC cache and memory
transactions to L3 cache.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>
2021-06-15 12:57:42 +00:00
Felix DeGrood
82952deb8b anv: Only flush Tile Cache on VK_ACCESS_HOST_R/W
Tile Cache flush flushes all Color/Depth values from L3 cache
to memory in Unified Cache mode. This is only required when
CPU access is required.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>
2021-06-15 12:57:42 +00:00
Felix DeGrood
6f26a51f47 anv: remove unnecessary Tile Cache flushes
On Gfx12+, flushing tile cache ensures color/depth values are
globally visible, but that's expensive.  Most operations only
need values to be GT-visible which can be achieved with depth
or rt flush.  Remove a bunch of unnecessary Tile Cache flushes.
Fast clears and slow depth clears still require Tile Cache flush.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>
2021-06-15 12:57:42 +00:00
Felix DeGrood
a7bb74db7b anv: Remove Tile Cache flush from SBA, Pipe Select
Tile Cache flushing not required for State Base Address or
Pipe Select instructions.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>
2021-06-15 12:57:42 +00:00
Felix DeGrood
1da9ff047e anv: Clear all pending stall after pipe flush
Was only clearing CS stalls after emitting pending pipe
controls.  Need to clear all stalls.

Cc: mesa-stable
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>
2021-06-15 12:57:42 +00:00
Felix DeGrood
fc5cb54008 anv: Add debug messages for DEBUG_PIPE_CONTROL
Enable with INTEL_DEBUG=pc.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>
2021-06-15 12:57:42 +00:00
Felix DeGrood
6c345ddbe4 anv: Cache VB/IB in L3$ for Gfx12
Gfx12 enables caching of Vertex and Index Buffers in L3.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9834>
2021-06-15 12:57:42 +00:00
Dave Airlie
f3630548f1 crocus: initial gallium driver for Intel gfx 4-7
This is a gallium driver for the Intel gfx 4-7 GPUs.

It was initially cloned from the iris driver by Ilia Mirkin,
then I ported over large reams of code from i965 until it worked.

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11146>
2021-06-14 06:34:05 +10:00
Dave Airlie
8da92b5c0a intel/compiler: add flag to indicate edge flags vertex input is last
965 and the mesa st disagree on how vertex elements are ordered when
edgeflags are involved. 965 wants them in gl_vert_attrib order,
but gallium supplies the edgeflag as the last vertex element regardless.

This adds a flag which is enabled for gen4/5 to denote that the
edgeflag is at the end. When we reap 965 later we can resolve this
better.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11146>
2021-06-14 06:05:18 +10:00
Dave Airlie
5f03570eaa intel: reorder base program key.
This gets hashed a lot, this reduces the size of this, and the other
keys by a small amount

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11146>
2021-06-14 06:05:17 +10:00
Dave Airlie
4de8ba15b4 intel/decode: handle gen4/5 WM state fragment shaders
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11146>
2021-06-14 06:05:17 +10:00
Yiwei Zhang
a0122385f0 anv: fix Android WSI VkFence
Fix an obvious one-liner bug.

Signed-off-by: Yiwei Zhang <zzyiwei@chromium.org>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11344>
2021-06-13 06:12:31 +00:00
Jason Ekstrand
e23b55c3f0 i965: Use nir_lower_passthrough_edgeflags
Now that there's a common NIR pass, there's no point in us doing this in
the back-end anymore.  In order to use this pass in i965, we do have to
make one tiny change.  Gallium runs the pass after assigning input and
output locations and so needs the pass to respect those locations and
num_inputs.  i965, however, runs it before any location assignment or
I/O lowering so we don't care.  We do, however, need the pass to succeed
with num_inputs == 0 because we set that later.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11313>
2021-06-11 21:19:06 +00:00
Jason Ekstrand
00dc4e0d83 intel/isl: Use a 4D physical total extent for size calculations
With Yf and Ys tiling, everything is actually four dimensional because
we can have multiple depth or multisampled array slices in the same
tile.  This commit just enhances the calculations so they can handle it.

Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11330>
2021-06-11 20:51:21 +00:00
Jason Ekstrand
dc764916d9 intel/isl: Make tile logical extents four dimensional
Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11330>
2021-06-11 20:51:21 +00:00
Jason Ekstrand
a4dafe1fad intel/isl: Make the offset helpers four dimensional
We need to do this in order to handle Yf and Ys tiling because they use
a four-dimensional tile instead of laying everything out in two
dimensions.

v2 (Jason Ekstrand):
 - Update functions added since v1:
    - isl_surf_get_image_range_B_tile
    - blorp_can_hiz_clear_depth
    - get_image_offset_el

Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com> (v1)
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11330>
2021-06-11 20:51:21 +00:00
Jason Ekstrand
3f7f6b878c intel/blorp: Use isl_surf_get_image_offset_B_tile_el in ccs_ambiguate
Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11330>
2021-06-11 20:51:21 +00:00
Jason Ekstrand
0eb3d0a8df intel/isl: Add a isl_surf_get_image_offset_B_tile_el helper
Reviewed-by: Topi Pohjolainen <topi.pohjolainen@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11330>
2021-06-11 20:51:21 +00:00
Jason Ekstrand
ddf970af88 anv/blorp: Optimize addresses/relocations when ANV_ALWAYS_SOFTPIN
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11236>
2021-06-08 22:53:22 +00:00
Jason Ekstrand
c7e1488037 anv: Optimize anv_address_physical when ANV_ALWAYS_SOFTPIN
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11236>
2021-06-08 22:53:22 +00:00
Jason Ekstrand
921bd2d1c7 anv: Fast-path surface relocs when we have softpin
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11236>
2021-06-08 22:53:22 +00:00
Jason Ekstrand
6afc3f97b6 anv: Make anv_batch_emit_reloc inline and optimize SKL+
This should drop the CPU overhead of processing buffers on SKL+ by
dropping some of the logic contained in anv_reloc_list_add() whenever we
have enough compile-time information to know we have softpin.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11236>
2021-06-08 22:53:22 +00:00
Jason Ekstrand
9802a0d7ca anv: Add a helper to add a BO to the batch list without a reloc
The relocation list currently serves two purposes.  One is for
relocations on older non-softpin platforms.  The second is to keep track
of driver-managed BOs which are used by the given command buffer.  We
going to need a mechanism to add BOs to the command buffer without doing
a relocation into the batch.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11236>
2021-06-08 22:53:22 +00:00
Jason Ekstrand
7e8c28383c anv: Handle OOM in the pinned path in anv_reloc_list_add
Fixes: b3c0b1b218 "anv: Use a bitset for tracking residency"
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11236>
2021-06-08 22:53:22 +00:00
Jason Ekstrand
a63e97e09a anv: Make use_softpin compile-time in genX code
Whenever we have the GFX_VERx10 macro available, we can make use_softpin
a compile-time thing for everything but Broadwell and Cherryview.  This
should save us some CPU cycles especially on SKL+.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11236>
2021-06-08 22:53:22 +00:00
Jason Ekstrand
04ccfeae98 anv: Require softpin on Gen8+
Softpin was added to i915 in

    commit 506a8e87d8d2746b9e9d2433503fe237c54e4750
    Author: Chris Wilson <chris@chris-wilson.co.uk>
    Date:   Tue Dec 8 11:55:07 2015 +0000

        drm/i915: Add soft-pinning API for execbuffer

which was included in Linux 4.5.  It's been over 5 years so it's
probably reasonable to make it a hard requirement.

Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11236>
2021-06-08 22:53:22 +00:00
Caio Marcelo de Oliveira Filho
7c1c9e935e anv: Support workgroup memory in other shaders
Mesh and Task shaders can use workgroup memory, so generalize its
handling in anv by moving it from anv_pipeline_compile_cs() to
anv_pipeline_lower_nir().

Update Pipeline Statistics accordingly.

Reviewed-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11230>
2021-06-08 11:30:39 -07:00
Caio Marcelo de Oliveira Filho
8af6766062 nir: Move workgroup_size and workgroup_variable_size into common shader_info
Move it out the "cs" sub-struct, since these will be used for other
shader stages in the future.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11225>
2021-06-08 09:23:55 -07:00
Caio Marcelo de Oliveira Filho
b5f6fc442c nir: Move zero_initialize_shared_memory into common shader_info
Move it out the "cs" sub-struct, since the bit will be used for other
shader stages in the future.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11225>
2021-06-08 09:23:55 -07:00
Rhys Perry
1cbcfb8b38 nir, nir/algebraic: add byte/word insertion instructions
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3151>
2021-06-08 08:57:42 +00:00
Dave Airlie
c04f20e7e0 intel/isl: add blend enable flag to gen4/5
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10655>
2021-06-08 12:35:52 +10:00
Dave Airlie
c25a5a832a intel/isl: add levels and minimum array element to null fill
gen4/5 needs these to avoid gpu hangs around matching depth/null
surfaces

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10655>
2021-06-08 12:35:43 +10:00
Dave Airlie
cb152e79f2 intel/isl: convert null surface fill to a struct.
Suggested by Jason, pre-convert this to a struct so it can
be expanded for gen4/5 crocus support

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10655>
2021-06-08 12:35:40 +10:00
Dave Airlie
6989afea58 intel/isl: decrease isl_format_layouts size by 36k
This drops
-0000000000011e90 R isl_format_layouts
+0000000000008f48 R isl_format_layouts

I think that's about 36k.

Thanks to Jason for suggesting PACKED

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11232>
2021-06-08 11:52:25 +10:00
Caio Marcelo de Oliveira Filho
c8a7bd0dc8 nir: Rename WORK_GROUP (and similar) to WORKGROUP
Be consistent with other usages in Vulkan and SPIR-V, and the recently
added workgroup_size field.

Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11190>
2021-06-07 22:34:42 +00:00
Caio Marcelo de Oliveira Filho
a71a780598 nir: Rename nir_intrinsic_load_local_group_size to nir_intrinsic_load_workgroup_size
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11190>
2021-06-07 22:34:42 +00:00
Caio Marcelo de Oliveira Filho
430d2206da compiler: Rename local_size to workgroup_size
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11190>
2021-06-07 22:34:42 +00:00
Jason Ekstrand
9e0fd49858 intel/fs/ra: Fix payload node setup for SIMD16 on Gen4-5
Since 40e1d798c6, we are now using physical register numbers for
everything which makes it all simpler.  In particular, we no longer need
the special case for setting up the payload for SIMD16 on Gen4-5.  This
fixes a pile of piglit tests on ILK and similar.

Fixes: 40e1d798c6 "intel/fs: Use ra_alloc_contig_reg_class()..."
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11221>
2021-06-07 16:52:19 +00:00
Marcin Ślusarz
2ebf4e984b intel/disasm: remove useless space after "("
Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11070>
2021-06-07 08:46:11 +00:00
Marcin Ślusarz
daba2894ff intel/disasm: decode/describe more send messages
Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11070>
2021-06-07 08:46:11 +00:00
Jason Ekstrand
b742f73913 intel/isl: Fix isl_format_is_valid
The format enum space isn't necessarily contiguous so we can't assume
that if it's in the table it's valid.  We need to check something.

Fixes: ed6e586562 "intel: properly constify isl_format_layouts"
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11191>
2021-06-04 16:16:44 -05:00
Eric Anholt
cf33316ec0 intel/vec4: Use ra_alloc_contig_reg_class() to reduce RA overhead.
We go from 1672 RA regs to the real 128 HW regs.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9437>
2021-06-04 19:08:57 +00:00
Eric Anholt
40e1d798c6 intel/fs: Use ra_alloc_contig_reg_class() to speed up RA.
By using the new class type, we don't need to make 1928 different
registers to represent each contigous reg size starting from the actual
128 HW register, or have a mapping between RA regs and HW base regs.  With
the number of regs reduced, and the fast q computation when using the new
classes, we no longer need to compute our own q.

This drops the FS RA initialization time on my CFL system from about 1ms to
50us.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9437>
2021-06-04 19:08:57 +00:00
Eric Anholt
95d41a3525 ra: Use struct ra_class in the public API.
All these unsigned ints are awful to keep track of.  Use pointers so we
get some type checking.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9437>
2021-06-04 19:08:57 +00:00
Adam Jackson
ed6e586562 intel: properly constify isl_format_layouts
Putting a const char * in the struct means it's a pointer that has to be
resolved at rtld time, which means it can be in .data.rel.ro but not
.rodata like you'd hope. Fix this with the usual string table trick.

Cuts about 20k (-80k read-write +60k read-only) and ~280 relocations
from the gallium driver.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11168>
2021-06-04 18:32:33 +00:00