Commit graph

184774 commits

Author SHA1 Message Date
Bas Nieuwenhuizen
748b7f80ef radv: Move sparse binding into a dedicated queue.
1) This better reflects the reality that we only have one timeline
   of sparse binding changes.

2) Allows making it a threaded queue from the start in prep of
   explicit sync stuff.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16935>
2023-11-29 17:37:37 +00:00
Bas Nieuwenhuizen
00faefa08e radv: Remove the sparse binding queue from coherent images.
Never access the image on the queue family, so no need.

(Technically not sure if this is needed for Vulkan, somewhat of
 a backstop in case apps do it)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16935>
2023-11-29 17:37:37 +00:00
Bas Nieuwenhuizen
6ff98f9313 radv: Add implementation of cmd buffers for a sparse binding queue.
None of the commands are allowed on these ...

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16935>
2023-11-29 17:37:37 +00:00
Alessandro Astone
4f48a140ac asahi: Use the compat version of qsort_r
Not all platforms define qsort_r, util_qsort_r takes care of that.

CC: mesa-stable
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25553>
2023-11-29 17:01:09 +00:00
José Roberto de Souza
c9e41f25a1 anv: Add heaps for Xe KMD in platforms without LLC
As Xe KMD don't support WB + 0 way coherency, so this are the only two
memory types possible for integrated GPUs without LLC in Xe KMD.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25462>
2023-11-29 14:57:42 +00:00
José Roberto de Souza
1a0d3504d5 anv: Fill PAT fields in Xe KMD gem_create and vm_bind uAPIs
Unlike i915, Xe KMD needs the cache parameter in gem_create
then during vm bind it request the PAT index that matches previous
parameter.
The PAT index selected could have more memory caracteristics that KMD
don't need to know.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25462>
2023-11-29 14:57:42 +00:00
José Roberto de Souza
99ae565af2 anv: Prepare anv_device_get_pat_entry() for discrete GPUs
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25462>
2023-11-29 14:57:42 +00:00
José Roberto de Souza
eb18a92ef9 iris: Fill PAT fields in Xe KMD gem_create and vm_bind uAPIs
Unlike i915, Xe KMD needs the cache parameter in gem_create
then during vm bind it request the PAT index that matches previous
parameter.
The PAT index selected could have more memory caracteristics that KMD
don't need to know.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25462>
2023-11-29 14:57:42 +00:00
José Roberto de Souza
d26bd29ab4 iris: Prepare iris_heap_to_pat_entry() for discrete GPUs
Xe KMD requires PAT information for discrete GPUs as well.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25462>
2023-11-29 14:57:42 +00:00
José Roberto de Souza
05b3967ddc intel: Enable has_set_pat_uapi for Xe
Xe KMD requires that all platforms supported by it set PAT information.
This will be implemented in Iris and ANV in the next patches.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25462>
2023-11-29 14:57:42 +00:00
José Roberto de Souza
183fd14143 intel: Sync xe_drm.h
Sync xe_drm.h with commit ebe27e42c0a2 ("drm/xe/uapi: support pat_index selection with vm_bind").

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25462>
2023-11-29 14:57:42 +00:00
José Roberto de Souza
500e037661 intel: Add PAT entries for gfx12 and newer
Xe KMD requires PAT for all platforms so here adding PAT entries to
all platforms supported by Xe KMD.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25462>
2023-11-29 14:57:42 +00:00
José Roberto de Souza
d491742d19 anv: Add support all possible cached and coherent memory types
This changes allow us to support HOST_COHERENT, HOST_CACHED and
HOST_COHERENT + HOST_CACHED memory types for platforms that has
the PAT uAPI.

Be aware that Xe KMD will not be able to support cached only memory
types, anv_xe_physical_device_init_memory_types() will reflect that
but internal usage should not allocate
VK_MEMORY_PROPERTY_HOST_CACHED_BIT only memory, hence the assert
added.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25462>
2023-11-29 14:57:42 +00:00
José Roberto de Souza
3baab9bb38 anv: Rename ANV_BO_ALLOC_SNOOPED to ANV_BO_ALLOC_HOST_CACHED_COHERENT
Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25462>
2023-11-29 14:57:42 +00:00
Alyssa Rosenzweig
d50d9eccad ac,radv,radeonsi: use common 1D texture lowering
It was pulled from ac.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26397>
2023-11-29 14:04:15 +00:00
Erik Faye-Lund
ba2bbe21b4 ci: opt-out panfrost from clang-format
In 0e481bf463 ("ci: Opt out asahi from clang-format"), Alyssa
recommended other drivers to follow suit. Since Panfrost originates from
Alyssa, and I doubt any other of the developers particularly cares too
much about this, let's follow her recommendation.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26062>
2023-11-29 13:01:08 +00:00
Jan Beich
112093f9e2 intel: make CLOCK_BOOTTIME optional for non-Linux
src/intel/common/xe/intel_gem.c:71:9: error: use of undeclared identifier 'CLOCK_BOOTTIME'
   case CLOCK_BOOTTIME:
        ^

Fixes: ae0df368a8 ("intel/common: Add intel_gem_read_correlate_cpu_gpu_timestamp()")
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26392>
2023-11-29 10:14:01 +00:00
Jan Beich
5c32c41f65 intel: make CLOCK_TAI optional for non-Linux
src/intel/common/xe/intel_gem.c:72:9: error: use of undeclared identifier 'CLOCK_TAI'
   case CLOCK_TAI:
        ^

Fixes: ae0df368a8 ("intel/common: Add intel_gem_read_correlate_cpu_gpu_timestamp()")
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26392>
2023-11-29 10:14:01 +00:00
Tapani Pälli
ec43c20182 anv: implement dummy blit for Wa_16018063123
Insert a dummy blit prior to MI_ARB_CHECK, MI_SEMAPHORE_WAIT,
MI_FLUSH_DW submitted on the copy engine.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26209>
2023-11-29 08:09:06 +00:00
Tapani Pälli
889d16a43a iris: implement dummy blit for Wa_16018063123
Insert a dummy blit prior to MI_ARB_CHECK, MI_SEMAPHORE_WAIT,
MI_FLUSH_DW submitted on the copy engine.

Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26209>
2023-11-29 08:09:06 +00:00
Konstantin Seurer
827b0fa1ef nir/lower_vars_to_scratch: Remove all unused derefs
If the shader passed to nir_lower_vars_to_scratch contains some unused
derefs to a variable that will be lowered, validation will fail because
the variable is not part of the shader after the pass.

cc: mesa-stable

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26271>
2023-11-29 06:49:48 +00:00
Faith Ekstrand
f9c7f33613 nvk: Enable 8 and 16-bit integer types
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26348>
2023-11-29 02:01:35 +00:00
Faith Ekstrand
bda208665f nak: Handle non-DW-aligned UBO loads
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26348>
2023-11-29 02:01:35 +00:00
Faith Ekstrand
ed535c9d98 nak: Implement integer conversions
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26348>
2023-11-29 02:01:35 +00:00
Faith Ekstrand
3b3f251471 nak: Implement more int/float conversions
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26348>
2023-11-29 02:01:35 +00:00
Faith Ekstrand
9e84e9e44b nak: Add base support for 8 and 16-bit types
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26348>
2023-11-29 02:01:35 +00:00
Faith Ekstrand
082c986614 nak: Fix int8/16 lowering
Move it into nak_postprocess_nir, after lower_mem_access_bit_sizes.
Also, fix the callback for comparison and conversion ops.

For conversion ops, we don't want to lower any of them right now. We'll
need to lower some 64-bit conversions eventually but we'll figure out
those details when we get to implementing real 64-bit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26348>
2023-11-29 02:01:35 +00:00
Faith Ekstrand
dc5cc847dd nak: Implement nir_op_extract_*
This should make a lot of bit twiddling more efficient since NIR can
optimize certain shifts patterns to extract and we can implement it with
a single PRMT instruction.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26348>
2023-11-29 02:01:35 +00:00
Faith Ekstrand
c0d0ce4c6b nak: Rework OpPrmt a bit
This renames `selection` to `sel`, adds a source type for sel, adds the
PrmtMode for selecting permute mode, and adds proper legalization.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26348>
2023-11-29 02:01:35 +00:00
Faith Ekstrand
7b6103ccc8 nak: Add a builder helper for OpPrmt
The builder also has some extra smarts in it for avoiding the PRMT when
the operation happens to exactly select one of the two sources.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26348>
2023-11-29 02:01:35 +00:00
Faith Ekstrand
d5693a590e nak: Drop the PrmtSelection stuff
It's just a very fancy and verbose way to build an immediate.  I don't
think it's really gaining us all that much, especially when the
semantics are documented in the PTX docs.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26348>
2023-11-29 02:01:35 +00:00
Lionel Landwerlin
7dff232c09 intel/ds: add trace of buffer markers
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14924>
2023-11-29 01:16:22 +00:00
Kenneth Graunke
c8e122a738 anv: Implement rudimentary VK_AMD_buffer_marker support
This provides a basic implementation of VK_AMD_buffer_marker: we can
write the 32-bit markers from within a command buffer.  Unfortunately,
our hardware has several limitations that make this difficult to
implement well:

   1. We don't have insight into when specific stages finish (i.e.
      all geometry shaders are done, but pixel rasterization may
      still be occurring).

   2. We cannot perform pipelined writes of 32-bit values to arbitrary
      memory locations.  PIPE_CONTROL::Write Immediate Value would be
      the obvious way to implement this, but it only supports 64-bit
      values, and the extension doesn't allow us to do that.  We instead
      use MI_STORE_DATA_IMM to write 32-bit values, but this requires
      hard stalls.

Despite those limitations, the extension may still be useful for tools
to debug GPU hangs.  We hope to offer another extension in the future
which offers similar functionality but is more efficient on our GPUs.

v2: Updated by Lionel Landwerlin to fix a number of flushing and
    cache coherency issues with these writes.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14924>
2023-11-29 01:16:22 +00:00
Juston Li
746da376e4 venus: implement vkGet[Device]ImageSparseMemoryRequirements
These were left unimplemented despite sparse support being added back to
venus in a55d26b5 ("venus: add back sparse binding support")

Same as vn_GetPhysicalDeviceSparseImageFormatProperties2, venus sparse
support requires queues that also support transfer so any sparse-only
queues are filtered out. If a device only supports sparse with
sparse-only queues, sparse features are disabled and these functions
return count of 0.

Fixes: a55d26b566 ("venus: add back sparse binding support")

Signed-off-by: Juston Li <justonli@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26297>
2023-11-29 00:22:26 +00:00
Faith Ekstrand
a0355dfcfb nvk: Advertise Vulkan 1.1 on Turing+
We actually trigger on whether or not NAK is used for everything.  If
so, we claim 1.1, otherwise claim 1.0.  We need NAK for subgroup ops and
other advanced shader features in later Vulkan versions.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26383>
2023-11-28 23:52:27 +00:00
Faith Ekstrand
93acce99f3 vulkan: Default override patch version to VK_HEADER_VERSION
The patch version shouldn't matter but, on the off chance it does, we
don't want to be advertising non-existent versions.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26383>
2023-11-28 23:52:27 +00:00
Konstantin Seurer
11897376c7 radv/rt: Skip null checks for small case counts
The individual cases make sure the sbt_idx is not null implicitly
because the handles are always != 0.

Totals from 60 (22.56% of 266) affected shaders:
Instrs: 47841 -> 47655 (-0.39%)
CodeSize: 255028 -> 253460 (-0.61%)
Latency: 1179658 -> 1225311 (+3.87%); split: -0.02%, +3.89%
InvThroughput: 224122 -> 232851 (+3.89%); split: -0.02%, +3.92%
Copies: 12049 -> 12043 (-0.05%); split: -0.37%, +0.32%
Branches: 3312 -> 3290 (-0.66%)
PreSGPRs: 3494 -> 3472 (-0.63%)

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25089>
2023-11-28 22:58:35 +00:00
Konstantin Seurer
fe674f67b1 radv/rt: Use a helper for inlining non-recursive stages
So we don't have to write the same logic multiple times.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25089>
2023-11-28 22:58:35 +00:00
Sagar Ghuge
3f747fcbfc iris: Init aux map state for compute engine
We need to write the aux map address during the batch initialization for
compute engine as well otherwise we would run into gpu hang with
compression enabled.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10194

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26342>
2023-11-28 21:27:59 +00:00
Alyssa Rosenzweig
d5e0901fd5 agx: fix 1D texture sampling
fixes texwrap 1d bordercolor cases.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26377>
2023-11-28 20:32:03 +00:00
Alyssa Rosenzweig
e14633fa7d nir/lower_tex: Add 1D lowering
From amd/common.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26377>
2023-11-28 20:32:03 +00:00
Caio Oliveira
5de5a0d475 intel/compiler: Don't use fs_visitor::bld in thread payload classes
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26301>
2023-11-28 19:53:51 +00:00
Caio Oliveira
2d6240ab14 intel/compiler: Don't use fs_visitor::bld in fs_reg_alloc
Just set up the builder without relying on the pre-existing one.  Moves
one step close to remove bld from fs_visitor.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26301>
2023-11-28 19:53:51 +00:00
Caio Oliveira
f55867b56c intel/compiler: Don't use fs_visitor::bld in tests
Tests create their own fs_builder now.  Moves one step closer to remove
bld from fs_visitor.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26301>
2023-11-28 19:53:51 +00:00
Caio Oliveira
9540259e1c intel/compiler: Prefer ctor/dtors in some Google Tests
Per Google Test FAQ recommendation, prefer consutrctors and destructors
unless there's a need to use SetUp/TearDown.

We will take advantage of this later to initialize an fs_builder.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26301>
2023-11-28 19:53:51 +00:00
José Roberto de Souza
d0db92b52d iris: Check for maximum allowed priority in Xe KMD
Previous code was settting whatever priority was requested even if
Xe KMD would not allow it causing warnings in dmesg:

xe 0000:00:02.0: [drm:exec_queue_set_priority [xe]] Ioctl argument check failed at drivers/gpu/drm/xe/xe_exec_queue.c:235: value > xe_exec_queue_device_get_max_priority(xe)
xe 0000:00:02.0: [drm:xe_exec_queue_set_property_ioctl [xe]] Ioctl argument check failed at drivers/gpu/drm/xe/xe_exec_queue.c:912: ret

Now it will query the maximum allowed priority and set the priority
closed to what application requested.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26325>
2023-11-28 18:17:45 +00:00
José Roberto de Souza
6a245e4eea intel: Share function to do device query in Xe KMD
A "dance" is required with this uAPI, first we need to ask KMD what is
the size of the giving query id, then memory needs to be allocated to
match that size and then query again with the memory address set and
at this time Xe KMD will copy the query data to memory.

This dance was being duplicated in xe_engine_get_info() and
anv_xe_physical_device_get_parameters() and the next patch will also
use it in Iris, so here adding it common/xe and re-using as much
as possible.

There is one more implementation of this function in intel/dev but
due to how libs are linked intel/dev can't depend on to intel/common.

Signed-off-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26325>
2023-11-28 18:17:45 +00:00
Marek Olšák
c5e37e7c39 radeonsi: only expose GL_AMD_performance_monitor on gfx7-10.3
It's only implemented for those generations.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26262>
2023-11-28 17:34:26 +00:00
Marek Olšák
c7729effa6 radeonsi: group equal CAP cases
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26262>
2023-11-28 17:34:26 +00:00
Marek Olšák
7a7285bc69 gallium: add PIPE_CAP_PERFORMANCE_MONITOR for GL_AMD_performance_monitor
Use a CAP instead of guessing it. radeonsi won't expose it.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26262>
2023-11-28 17:34:26 +00:00