Commit graph

160627 commits

Author SHA1 Message Date
Mark Janes
c8f563b633 anv: Allocate buffers with write-combined local memory
Marginally improves DG1 performance (< 1%)

v2: Only on local mem (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18841>
2022-09-28 09:51:31 +00:00
Mark Janes
5c62ad34b6 anv: Use WC mapped local memory for block pool BO
Improve DG1 performance:

  Fallout: +7%
  Talos:  +15%

v2: Don't drop SNOOP (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18841>
2022-09-28 09:51:31 +00:00
Mark Janes
755b413ffa anv: Track BOs that need a write-combined mapping
v2: simplify logic a bit (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18841>
2022-09-28 09:51:31 +00:00
Daniel Schürmann
97850c0bf0 aco/opt_value_numbering: use monotonic_allocator for unordered_map
This patch also changes the rename map to unordered.
Roughly halves the time spent on CSE in ACO.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
b39d2168a7 aco: implement allocator_traits for monotonic_allocator<T>
For easier usage, this patch also adds aliases for std::map
and std::unordered_map using this allocator.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
a128d444cb aco: use monotonic_buffer_resource for instructions
As monotonic_buffer_resource is not thread-safe,
we use a thread_local instance which gets allocated once.

This change reduces the compile time spent in ACO by
approximately 10%.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
15b3cc73bf aco: implement custom memory resource
This basic allocator implements an arena allocation strategy
and cannot free individual allocations.
It is intended for very fast memory allocations in situations
where memory is used to build up a few objects and then is
released all at once.

This class mimics std::pmr::monotonic_buffer_resource.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Daniel Schürmann
0b76e22a96 aco: simplify operands_offset calculation in create_instruction()
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18112>
2022-09-28 09:25:20 +00:00
Kenneth Graunke
a778f3edd3 blorp: Fix typo in blorp_xy_block_copy_blt
suppotred -> supported (Thanks to Tapani for catching this.)

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>
2022-09-28 08:41:35 +00:00
Kenneth Graunke
759d51c367 blorp: Implement blitter clears via XY_FAST_COLOR_BLT
Vulkan transfer queues need this functionality.  A lot of the code is
pretty similar to what we have for XY_BLOCK_COPY_BLT.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>
2022-09-28 08:41:35 +00:00
Kenneth Graunke
5f4ad65daf blorp: Make blitter_supports_aux accessible from multiple files.
We'll want it in blorp_clear.c shortly.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>
2022-09-28 08:41:35 +00:00
Kenneth Graunke
98bd984977 intel/genxml: Add XY_FAST_COLOR_BLT
We'll need to use this for VkCmdFillBuffer on transfer queues.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15183>
2022-09-28 08:41:35 +00:00
Eric Engestrom
7a33997434 ci: bump mold to 1.5
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18838>
2022-09-28 07:56:47 +00:00
Eric Engestrom
bcc5cb4a1b ci: unexport local variable (and fix formatting)
Signed-off-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18838>
2022-09-28 07:56:47 +00:00
Jason Ekstrand
647773a094 intel/devinfo: DG2 supports ray-tracing
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jason Ekstrand
4c0dbe6420 anv: Advertise ray-tracing on DG2
Also disable ray-tracing support if with_intel_vk_rt is not set.

Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jordan Justen
9c6ed40925 meson: Define with_intel_vk_rt based on with_intel_clc
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jordan Justen
f7c82c72c4 meson: Deprecate vulkan-rt-drivers intel
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jordan Justen
e565d1e4ea anv/meson: Use anv_flags and anv_cpp_flags in genX compiles
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Lionel Landwerlin
64e8b0d255 anv: use the right dispatch size for tracing shaders
We assumed the trampoline shader would always be SIMD8.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Zhang, Jianxun <jianxun.zhang@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Lionel Landwerlin
044cf4e75d anv: bump client visible address heap to 32GiB
Some raytracing tests are allocating lots of buffer and because of our
2Mb alignment restriction on local memory, we're running our of VMA...

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Zhang, Jianxun <jianxun.zhang@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Lionel Landwerlin
5ad803840d anv: setup scratch space correctly for RT shaders
Things are a bit confusing because we use the term "scratch" for 2
different things :
  * the buffer for register allocation spilling
  * the buffer for storing live values between splitted shaders around shader calls

Here we're fixing the missing register allocation spilling buffer.

v2: update comments (Caio)
    fix scratch bo size computation with pipeline libraries (Lionel)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jason Ekstrand
f3ddfd81b4 anv: Build BVHs on the GPU with GRL
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jason Ekstrand
dc1aedef2b intel/grl: Parse GRL files and generate C
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jason Ekstrand
20f291c5f9 anv/grl: Add a helper for dispatching our pre-built kernels
v2: Use the default pipeline cache (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jason Ekstrand
639665053f anv/grl: Build OpenCL kernels
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Iván Briano
37663438f7 anv/grl: Add a GRL file parser
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jason Ekstrand
5f948503e4 anv: Import GRL
GRL, or Graphics Library for Ray-tracing is a library we share with the
Windows drivers for doing BVH builds on the GPU.  It consists of a few
headers shared between CL and C code, a bunch of CL kernels, and some
GRL meta-kernels in their own format.

Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Lionel Landwerlin
49976b23e9 anv: add new command buffer space allocation
To be used for acceleration structure building.

v2: fix missing u_vector_finish
    Free all BOs

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jason Ekstrand
6c76ceb613 anv: Add support for OpenCL-style kernel dispatch
v2: Use brw_cs_get_dispatch_info() (Lionel)
    Merge barrier fixes (Lionel)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jason Ekstrand
eb249f125d anv: Add support for compiling OpenCL-style kernels
v2: remove unused definitions

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jason Ekstrand
6ad1a5b57a anv: Add extern "C" guards
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Lionel Landwerlin
23c7142cd6 anv: disable SIMD16 for RT shaders
Since divergence is a lot more likely in RT than compute, it makes
sense to limit ourselves to SIMD8.

The trampoline shader defaults to SIMD16 since this one is uniform.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:37 +00:00
Jason Ekstrand
5814436159 anv: Set up the memory-backed FIFO buffer
v2: Fix incorrect goto (Caio)
    Comment 3DSTATE_BTD programming (Caio)

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Jason Ekstrand
93526c89c5 anv: Implement VK_KHR_pipeline_library
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Jason Ekstrand
4388b8b253 anv: Add an anv_address_map helper
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Jason Ekstrand
191fab68b0 anv/formats: Advertise ACCELERATION_STRUCTURE_VERTEX_BUFFER_BIT
v2: Only expose the bit when ray tracing is supported.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Lionel Landwerlin
3beb088490 intel/mi_builder: allow half GP registers for dereferencing
Some of the GRL metakernels will generate 64bit value in a register,
then use only half of that as the last operation on that value.

v2: Add comment (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Jason Ekstrand
24a4a3bbbb intel/mi_builder: Add a helper for incrementing reference counts
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Jason Ekstrand
b7296e11ae intel/mi_builder: add a way to reserve a register
Will be useful for GRL metakernels.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Lionel Landwerlin
a5e5e6aa0f genxml: add missing no duplicate anyhit flag
This mirrors the VK_GEOMETRY_NO_DUPLICATE_ANY_HIT_INVOCATION_BIT_KHR
enum of VkGeometryFlagBitsKHR. Purely here for documentation.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Lionel Landwerlin
8fc7a98e31 intel/fs: disable split_array_vars on opencl kernels
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Lionel Landwerlin
57593c5395 intel/nir: disable assert on async stack id
This can be accessed from :
   - RT shaders
   - CS trampoline shader

We missed the second part here.

Fixes: 0465714790 ("intel/nir/rt: add more helpers for ray queries")
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Lionel Landwerlin
8d580de4a9 intel/nir: fix potential invalid function impl ptr usage
We keep the nir_builder::impl value around, but we've run some passes
that might have change the main function.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 96fde5518b ("intel/rt: Add a helper to create the raygen trampoline shader")
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Lionel Landwerlin
1ffd28149f intel/nir: fixup preserved metadata in rayquery lowering
Another case of not clearing the metadata correctly.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: c78be5da30 ("intel/fs: lower ray query intrinsics")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Lionel Landwerlin
9dba8d8aa1 intel/fs: take a builder arg for resolve_source_modifiers()
There will be situations where we will want to use a local builder
rather than the one associated with NIR->backend translation.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Lionel Landwerlin
649cdc617f intel/nir: reuse rt helper
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Lionel Landwerlin
57f1e95102 intel/rt: fix procedural primitive ID access
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Jason Ekstrand
aea88f16df intel/fs: SEL_EXEC uses the integer pipe for 64-bit stuff
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00
Jason Ekstrand
c80c0ed943 intel/fs: Always use integer types for indirect MOVs
There's a new Gen12.5 restriction which forbids using the VxH or Vx1 on
the floating-point pipe.

Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16970>
2022-09-28 05:38:36 +00:00