Commit graph

60188 commits

Author SHA1 Message Date
Mike Blumenkrantz
47d9eaa0f1 zink: flag batch usage on swapchain images
while swapchains themselves are protected against early deletion
during presentation, there is nothing protecting them from
deletion while they are rendering if a swapchain updates
while rendering but before presentation

to address this, add batch usage to swapchains which can be
checked during pruning to ensure a rendering swapchain isn't
pruned

Fixes: dc8c9d2056 ("zink: prune old swapchains on present")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22962>
2023-05-16 00:54:46 +00:00
Mike Blumenkrantz
7ce82f1dec zink: set higher prio on dedicated memory allocations
this should guarantee that e.g., swapchain type images aren't paged
out

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22855>
2023-05-16 00:11:12 +00:00
Mike Blumenkrantz
5dd63a69da zink: hook up some memory extensions
enabling VK_EXT_pageable_device_local_memory guarantees that host memory
allocations will not consume device-local memory and enables overallocation
of device memory when paging can be done

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22855>
2023-05-16 00:11:12 +00:00
Mike Blumenkrantz
b88006fb9f zink: slightly simplify bda allocation chaining
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22855>
2023-05-16 00:11:12 +00:00
Mike Blumenkrantz
cab466816f lavapipe: EXT_attachment_feedback_loop_layout_dynamic_state
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22964>
2023-05-15 23:15:23 +00:00
Mike Blumenkrantz
8d58fa5787 zink: only add feedback loop usage bit if extension is supported
cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22960>
2023-05-15 22:52:57 +00:00
Mike Blumenkrantz
468554804c zink: ignore no-op image copies
rare, but it happens and is illegal

affects:
GTF-GL46.gtf30.GLCoverage.CoverageGL30

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22960>
2023-05-15 22:52:57 +00:00
Mike Blumenkrantz
864ccc7a92 zink: reorder some image copy code
no functional changes, just making other fixes easier to see

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22960>
2023-05-15 22:52:57 +00:00
Mike Blumenkrantz
7466c6fbf2 zink: reject blits where src/dst is 3D and dst/src z!=0
this is technically illegal even though it works everywhere,
though future spec changes may make it legal

affects KHR-GLES3.copy_tex_image_conversions.required.texture3d_cubemap_negz

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22960>
2023-05-15 22:52:57 +00:00
Mike Blumenkrantz
cc13c96b33 zink: reorder some native blit code
no functional changes, but this will make it more convenient to
reject certain blits

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22960>
2023-05-15 22:52:57 +00:00
Mike Blumenkrantz
58532057c5 zink: destroy current batch state after all other batch states
some resources may not be destroyed immediately and may instead be
queued for deletion onto the current batch state, so ensure that the
current state is the last one to be destroyed so that all deferred resources
are also destroyed

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23033>
2023-05-15 20:51:40 +00:00
Mike Blumenkrantz
cd0454646f zink: don't leak swapchain readback semaphores
these are considered usable after the queue goes idle, so add them
back into the cache

Fixes: e9f18f64b9 ("zink: also cache swapchain semaphores")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23033>
2023-05-15 20:51:40 +00:00
Alyssa Rosenzweig
6c90fe189f r600: Use unified atomics
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23026>
2023-05-15 20:32:20 +00:00
Nanley Chery
a0e51fcc4e iris: Init CCS_E to COMPRESSED_NO_CLEAR for XeHP
Use COMPRESSED_NO_CLEAR for the initial CCS aux state instead of
COMPRESSED_CLEAR. This removes a dependency on the initial clear color,
meaning that some resolves related to clear color management are now
avoided.

In the Car Chase benchmark, this avoids all 50 CCS resolves. These only
happen during the warm-up phase of the benchmark, so I'm not sure there
is an impact on FPS. This was tested on a DG2 in small-BAR mode.

Reviewed-by: Jianxun Zhang <jianxun.zhang@intel.com>
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22857>
2023-05-15 19:54:02 +00:00
norablackcat
1404c180e9 rusticl: implement cl_khr_pci_bus_info
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23022>
2023-05-15 18:34:41 +00:00
Lionel Landwerlin
952a523abb intel: switch over to unified atomics
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23004>
2023-05-15 16:32:21 +00:00
Erik Faye-Lund
cef751c86a zink: do not open-code memcpy
There's a lot of optimized memcpy implementations out there, let's use
them instead of manually copying.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23027>
2023-05-15 14:45:40 +00:00
Erik Faye-Lund
93682f9f4d zink: clean up tcs_vertices_out_word handling
At this point, we already have the index of the declaration itself in
the tcs_vertices_out_word variable, so we only need to add the offset
from the start of the exec_modes buffer.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23027>
2023-05-15 14:45:40 +00:00
Erik Faye-Lund
e6edce2f2a zink: fix bad indent
This was indented too much

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23027>
2023-05-15 14:45:40 +00:00
Konstantin Seurer
723922b6f1 gallium/nir: Handle unified atomics in nir_to_tgsi_info
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23016>
2023-05-15 14:15:02 +00:00
Mike Blumenkrantz
b563bfb4ee zink: add some ci flakes
roundup from recent ci jobs

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23021>
2023-05-15 12:07:16 +00:00
antonino
71107b6dc8 zink: don't create invalid inputs in zink_create_quads_emulation_gs
The helper was creating input locations for some builtin bariables.

This caused validation errors in zink because those builtins can't be
used as input.

Fixes: e2220ee55e ("zink: filled quad emulation gs generation function")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22871>
2023-05-15 11:04:41 +00:00
antonino
474d93719e zink: handle interface blocks in copy_vars
Fixes: edaf49160e ("zink: fix array copying in pv lowering")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22871>
2023-05-15 11:04:41 +00:00
antonino
a4113fd021 zink: don't replace non generated gs
Zink replaced the gs emulation shader when the primitive type changes,
however it didn't check whether the gs being replaced was generated.

Fixes: eedbf9046e ("zink: handle switching between primitives")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22871>
2023-05-15 11:04:41 +00:00
Qiang Yu
d1dd36a74e radeonsi: be able to use aco compiler for mono ps
Need to set AMD_DEBUG=useaco environment variable.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:11 +00:00
Qiang Yu
288adae512 radeonsi: fixup sampler desc for tg4 in nir
For ACO which won't do this for us. But we still can't
remove the same code in llvm because non-uniform sampler
is keept as index in nir.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:11 +00:00
Qiang Yu
521cbcb588 ac/llvm,radeonsi: enable lower_array_layer_round_even
ACO need this to be done in nir. Remove the llvm round code
because both radv and radeonsi do this in nir for both aco
and llvm.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:11 +00:00
Qiang Yu
38e064b674 radeonsi: clamp shadow texture reference in nir for aco
This is ported from the LLVM ac_shader_abi->clamp_shadow_reference
code.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:11 +00:00
Qiang Yu
b5409131ef radeonsi: pass use_aco to ac_nir_lower_ps
For dual source blend code emition in aco.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:11 +00:00
Qiang Yu
7180b16afc radeonsi: adjust ps args for aco
aco need explicite args including PS arg compaction and
scratch_offset.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:11 +00:00
Qiang Yu
474ddeffe6 radeonsi: resolve aco scratch addr symbols
Used for scratch buffer operation and reg spill when aco.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:11 +00:00
Qiang Yu
7aac3508dc radeonsi: add symbols to si_shader_binary
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:11 +00:00
Qiang Yu
6a360e4a71 radeonsi: add initial aco compile code
Only for monolithic PS.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:11 +00:00
Qiang Yu
91c91bb972 radeonsi: lower non uniform texture access when aco
aco need all resource have been lowered to descriptor.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:11 +00:00
Qiang Yu
f859436b55 radeonsi: add has_non_uniform_tex_access shader info
Can be used to skip nir_lower_non_uniform_access pass.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
563bdcc7fc radeonsi: lower vector const to scalar at last for aco
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
e252d87816 radeonsi: lower some 64bit ops aco does not support
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
9bc1fb4c07 ac/llvm,radeonsi: lower nir_fpow for aco and llvm
aco does not implement fpow, need nir to lower it
first. llvm will do by itself in the same way, so
we always lower fpow in nir now.

Remove the llvm fpow implementation that has special
handling for the muliplication. It's not used any
more and does not match GLSL spec as fpow(0,0)=NaN
but here we get 0.

There's some pixel changes for gl-radeonsi-stoney:
  ror-default 2 (no tolerance), 0 (1% tol.)

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
19a8626f86 ac/llvm,radeonsi: lower some pack/unpack ops not supported by aco
aco only support the split vertion of these instructions.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
fb2d0fb4a2 ac/llvm,radeonsi: lower ineg in nir
aco does not implement it.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
3fae161ff2 ac/llvm,radeonsi: lower txf offset in nir
aco will complain if txf has offset. Not if other
texture ops.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
f13f9044db ac/llvm,radeonsi: lower fsin/fcos in nir
ACO only support nir_fsin/cos_amd.

There's some pixel changes for gl-radeonsi-stoney trace.
Different pixels:
  furmark  61 (no tolerance), 0 (1% tol.)
  gimark   93867 (no tolerance), 888 (1% tol.)
  tessmark 39 (no tolerance), 0 (1% tol.)

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
f9d54b1d36 ac/llvm,radeonsi: lower idiv in nir
aco does not implement these idiv ops.

nir_lower_idiv is for idiv ops <= 32bit and ported from
llvm amdgpu, so llvm do the same.

nir_lower_divmod64 is for 64bit idiv ops.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
5fa06828b4 tgsi_to_nir: call nir_lower_int64 when required
Use case: radeonsi will generate internal tgsi shader
with 64bit udiv instruction, and we want all 64bit udiv
to be lowered in nir by lower_int64_options.

For GLSL shaders, this is done in glsl to nir, so we do
the same for tgsi here.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
636f628206 radeonsi: remove ps vgpr index save when args init
They will be set by ac_get_fs_input_vgpr_cnt() later anyway.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
1eddf5934b radeonsi: support print raw shader binary
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
ff29502df2 radeonsi: support raw shader binary upload
Only monolithic shader.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
f3997a3ca7 radeonsi: add a raw shader binary type
It's the output of ACO compiler. To share the si_shader_binary
struct with ELF type:
  * add a type field to indicate RAW or ELF
  * rename elf_buffer/size to code_buffer/size

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
83a920dfb9 radeonsi: init spi ps input shader config when aco
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00
Qiang Yu
f954aa1624 radeonsi: pack spi ps input fixup to a function
To be shared with ACO spi ps input construction.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22573>
2023-05-15 02:01:10 +00:00