Commit graph

178336 commits

Author SHA1 Message Date
Caio Oliveira
3d7e5ec758 spirv: Expose some memory related functions in vtn_private.h
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23825>
2023-09-28 07:35:02 +00:00
Caio Oliveira
af3eb80afa nir: Handle cooperative matrix in various passes
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23825>
2023-09-28 07:35:02 +00:00
Caio Oliveira
3105d516d0 nir: Add new intrinsics for Cooperative Matrix
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23825>
2023-09-28 07:35:02 +00:00
Caio Oliveira
2d0f4f2c17 compiler/types: Add support for Cooperative Matrix types
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23825>
2023-09-28 07:35:02 +00:00
Paulo Zanoni
b75da97a1d anv: enable sparse resources by default
This of course only applies to xe.ko. There is no reason to keep it
disabled by default.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23045>
2023-09-28 06:16:40 +00:00
Paulo Zanoni
7e2d8cced3 anv/sparse: add INTEL_DEBUG=sparse
This pollutes stderr a lot, but I've used it countless times while
developing this code.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23045>
2023-09-28 06:16:40 +00:00
Paulo Zanoni
2bdd01187d anv/sparse: get ready to issue a single vm_bind ioctl per non-opaque bind
Game testing shows it's common for this operation to result in
multiple bind regions, so try to use a single ioctl when we can.

Actual testing reveals 136 shader-related tests fail when we actually
do this, so for now keep doing a single bind per ioctl while leaving a
very easy way to the desired behavior when we figure this out.

It should also be possible to go even higher-level and do this at the
anv_queue_submit_sparse_bind_locked() layer, but that should happen in
future commits.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23045>
2023-09-28 06:16:40 +00:00
Paulo Zanoni
6368c1445f anv/sparse: add the initial code for Sparse Resources
This giant patch implements a huge chunk of the Vulkan Sparse
Resources API. I previously had this as a nice series of many smaller
patches that evolved as the xe.ko added more features, but once I was
asked to squash some of the major reworks I realized I wouldn't be
able easily rewrite history, so I just squased basically the whole
series into a giant patch. I may end up splitting this again later if
I find a way to properly do it.

If we want to support the DX12 API through vkd3d we need to support
part of the the Sparse Resources API. If we don't, a bunch of Steam
games won't work.

For now we only support the xe.ko backend, but the vast majority of
the code is KMD-independent and so an i915.ko implementation would use
most of what's here, just extending the part that binds and unbinds
memory.

v2+: There's no way to sanely track the version history of this patch
in this commit message. Please refer to Gitlab.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23045>
2023-09-28 06:16:40 +00:00
Paulo Zanoni
e4598f0eea intel/isl: simplify the check for maximum surface size
The only thing that changes between these 3 checks is the size.

This entire patch was suggested by Kenneth Graunke, I just converted
his gitlab comment to a git commit.

Credits-to: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23045>
2023-09-28 06:16:40 +00:00
Paulo Zanoni
0de5d142e8 intel/isl: add ISL_SURF_USAGE_SPARSE_BIT
Vulkan Sparse resources have their own set of rules, so here we try to
make ISL aware of them through ISL_SURF_USAGE_SPARSE_BIT.

The big deal here is when some image ends up not using Tile64 nor
TileYs. Previously Ys was not supported on TGL at all, and Tile64 did
not have support for 3D. Now we still have some formats that end up
not being used with either Tile64 and Ys, but need to support Sparse
on them (e.g., YUV on Tile64). In the future we may have new tiling
formats or hardware restrictions that would force this case to happen
again.

So here we do some adjustments so we can make sparse work with other
tiling formats, although with limited functionality (e.g., those
formats may be restricted to opaque binds, and certainly don't support
the standard block shapes).

v2: before we had Ys support, we had defined TGL's block size as 4k.
v3: move the size_B chunk to before nte notify_failure() checks (Ken).

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23045>
2023-09-28 06:16:40 +00:00
Faith Ekstrand
968cefbff1 nvk: Cache NIR shaders
We can't cache shader binaries just yet but this at least lets us cache
the output of spirv_to_nir and the initial optimize.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25443>
2023-09-28 03:55:53 +00:00
Faith Ekstrand
cdbd86c176 nvk: Add a default pipeline cache
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25443>
2023-09-28 03:55:53 +00:00
Faith Ekstrand
abe52a6d03 nvk: Re-structure early shader compilation a bit
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25443>
2023-09-28 03:55:53 +00:00
Faith Ekstrand
a4f8fd9dd5 nvk: Hook up the disk cache
This won't actually do much yet because we don't have pipeline caches
yet but it turns on the infrastructure.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25443>
2023-09-28 03:55:53 +00:00
Faith Ekstrand
d08df319ca nvk: Store a 20-bit driver_build_sha in nvk_instance
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25443>
2023-09-28 03:55:53 +00:00
Timothy Arceri
1780102923 nir: fix typo in comment
The variable is unused or dead, not used.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25414>
2023-09-28 01:54:43 +00:00
antonino
76d150674b vulkan: Handle vkSetDebugUtilsObjectNameEXT on WSI objects
Some WSI objects don't extend `vk_object_base` therefore they need
special handling.

Fixes: 3c87618d35 ("vulkan: Handle vkGet/SetPrivateDataEXT on Android swapchains")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24752>
2023-09-28 01:23:55 +00:00
antonino
5fe289d741 vulkan: Extend vkGet/SetPrivateDataEXT handling to VkSurface
VkSurface is handled by WSI and it doesn't extend `vk_object_base` so it
needs special handling.

Fixes: 3c87618d35 ("vulkan: Handle vkGet/SetPrivateDataEXT on Android swapchains")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24752>
2023-09-28 01:23:55 +00:00
antonino
710d478066 vulkan: Extend vkGet/SetPrivateDataEXT handling to all platforms
Non-android platforms use mesa WSI, however some WSI object still don't
extend `vk_object_base` so they still need special handling.

Fixes: 3c87618d35 ("vulkan: Handle vkGet/SetPrivateDataEXT on Android swapchains")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24752>
2023-09-28 01:23:55 +00:00
Dave Airlie
c45ae052f0 llvmpipe/cs: migrate cs image handle to common jit code.
This moves some of the code over, and uses the generic paths.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25416>
2023-09-28 00:41:04 +00:00
Dave Airlie
2c74f825f4 llvmpipe/cs: migrate to generic jit texture from pipe code.
This moves some cs specific bits to the generic code and uses it.

Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25416>
2023-09-28 00:41:04 +00:00
Marcin Ślusarz
ea92bd8d44 intel/compiler: mask GS URB handles at thread payload construction
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Marcin Ślusarz
815eee10e0 intel/compiler/mesh: implement IO for xe2
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Marcin Ślusarz
ee4214de6e intel/compiler/mesh: fix position of output URB handle for xe2
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Francisco Jerez
7f3dc4505d intel/fs: Delete manual 'inst->mlen' calculations from all uses of logical URB reads.
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Francisco Jerez
53d1d793cb intel/fs: Delete manual 'inst->mlen' calculations from all uses of logical URB writes.
Rework:
 * Marcin: update emit_urb_indirect_vec4_write

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Francisco Jerez
34a2c9ce35 intel/fs: Specify number of data components of logical URB writes via control immediate.
This is what most logical SEND messages do when they take a variable
number of components.  'inst->mlen' is expected to be zero for logical
SEND opcodes, which are expected to behave like plain arithmetic
operations, so certain automated transformations (like SIMD lowering)
can manipulate them without opcode-specific special-casing.

Guessing the number of components from 'inst->mlen' has other
disadvantages, because it requires duplicating the logic that infers
the message payload size in every use of the instruction -- Instead we
can just do the computation once during logical send lowering.  In
addition on LNL platform this causes the 'inst->mlen' field of URB
writes to have units inconsistent with every other SEND instruction,
which is likely to lead to confusion and bugs down the road.

Rework:
 * Marcin: update emit_urb_indirect_vec4_write

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Francisco Jerez
74c9973c0b intel/fs/xe2+: Fix URB writes with 0 data components.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Caio Oliveira
c89597085a intel/compiler/xe2: Update TCS ICP handle code to support SIMD16
Rework:
 * Use ffs(grf_size_bytes) (s-b Ken)

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Caio Oliveira
f0fcb778b4 intel/compiler/xe2: Fix URB writes in TCS
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Caio Oliveira
0c03018abf intel/compiler/xe2: URB fence uses LSC now
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Ian Romanick
623465e26d intel/compiler/xe2: Update fs_visitor::emit_urb_writes to not assume SIMD8
v2: Account for 512b physical registers which causes the URB handle to be in FIXED_GFR 2 instead of 1.

XXX - Use fs_builder::vgrf() instead of open-coded dispatch_width calculations.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Kenneth Graunke
4fffdbbfa2 intel/fs: Fix Xe2 URB read/lowering with per-slot offsets
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Jordan Justen
c28539a2fe intel/compiler: Use enum xe2_lsc_cache_load on xe2
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Jordan Justen
8d27b327f2 intel/compiler: Add enum xe2_lsc_cache_load
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Jordan Justen
09fc9ff114 intel/compiler: Use enum xe2_lsc_cache_store on xe2
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Jordan Justen
c54eff2e03 intel/compiler: Add enum xe2_lsc_cache_store
Rework:
 * Rohan: Fix enum value for L1WB_L3WB
 * Fix write-through comments (Ken)

Ref: bspec 71167
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Marcin Ślusarz
1245020282 intel/compiler: add initial support for URB_LOGICAL_SRC_CHANNEL_MASK to lower_urb_write_logical_send_xe2
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Marcin Ślusarz
9c90377962 intel/compiler: add lsc_msg_desc_wcmask
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Ian Romanick
feec9166cd intel/compiler/xe2: Handle new URB write messages
Rework:
 * idr v1: Fix compilation error.
 * idr v2: Add support for per-channel offsets.
 * idr v3: get_lowered_simd_width is 16 on Xe2+.
 * idr v4: Add disassembly support.  Add validation support.
 * Sqaushed in changes Marcin Ślusarz's patches:
   * "intel/compiler: skip adding 0 to payload address"
   * "intel/compiler/xe2: drop masking off top 8 bits of URB handle"

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Ian Romanick
fa53a7d241 intel/compiler/xe2: Handle new URB read messages
Rework:
 * Sqaushed in changes Marcin Ślusarz's patches:
   * "intel/compiler: skip adding 0 to payload address"
   * "intel/compiler/xe2: drop masking off top 8 bits of URB handle"

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25195>
2023-09-27 23:57:25 +00:00
Konstantin Seurer
4552e594f1 vulkan: Remove vk_get_physical_device_core_1_*_feature_ext
It's unused.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24773>
2023-09-27 23:02:29 +00:00
Konstantin Seurer
05796b29f5 hasvk: Use the common GetPhysicalDeviceFeatures2 implementation
Reviewed-by: Julia Tatz <tatz.j@northeastern.edu>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24773>
2023-09-27 23:02:29 +00:00
Rhys Perry
65afc8bebf nir/algebraic: optimize u2u32(a >> 32)
fossil-db (navi21):
Totals from 352 (0.44% of 79330) affected shaders:
Instrs: 271816 -> 271240 (-0.21%); split: -0.28%, +0.07%
CodeSize: 1546520 -> 1544448 (-0.13%); split: -0.23%, +0.09%
SpillVGPRs: 832 -> 827 (-0.60%); split: -1.08%, +0.48%
Latency: 4037120 -> 4021748 (-0.38%); split: -0.41%, +0.03%
InvThroughput: 1369540 -> 1362066 (-0.55%); split: -0.59%, +0.04%
VClause: 6476 -> 6471 (-0.08%); split: -0.12%, +0.05%
SClause: 6798 -> 6794 (-0.06%)
Copies: 44828 -> 44630 (-0.44%); split: -0.89%, +0.45%
Branches: 8845 -> 8844 (-0.01%); split: -0.05%, +0.03%
PreSGPRs: 14684 -> 14659 (-0.17%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25409>
2023-09-27 22:13:01 +00:00
Rhys Perry
bcdac65ca3 nir/lower_int64: fix find_lsb(0)
If the high 32 bits were zero, this would be umin(find_lsb(lo), 31). This
evaluates to 31 if lo is also zero, instead of -1.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Fixes: 9293d8e64b ("nir: Add find_lsb lowering to nir_lower_int64.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25409>
2023-09-27 22:13:01 +00:00
wangra
e90f9d8e21 tu/kgsl: Fix bitfield of DITHER_MODE_MRT6
The enum `adreno_rb_dither_mode` needs 2 bits, change the `high` to 13 to make sure 2 bits are used for `DITHER_MODE_MRT6`

Fixes: e03259974e ("freedreno: Generate headers from xml files")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25439>
2023-09-27 21:45:40 +00:00
Jordan Justen
b5eb96384c anv: Print warning that Xe2 is not supported rather than failing
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25411>
2023-09-27 21:11:18 +00:00
Jordan Justen
30ba269070 anv: Build for Xe2
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25411>
2023-09-27 21:11:18 +00:00
Jordan Justen
65684b0c7f anv: Disable Ray Tracing on xe2 until our compiler supports Xe2 RT
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25411>
2023-09-27 21:11:18 +00:00
Jordan Justen
3f1b4dd434 anv/blorp: Use anv_genX to set device->blorp.exec
Signed-off-by: Jordan Justen <jordan.l.justen@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25411>
2023-09-27 21:11:18 +00:00