Commit graph

128923 commits

Author SHA1 Message Date
Marek Olšák
fd6bbdcf59 radeonsi: use staging buffer uploads for most VRAM buffers
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6948>
2020-10-01 16:29:46 +00:00
Marek Olšák
701f7ae9d2 radeonsi: move si_set_active_descriptors_for_shader into si_update_common_shader_state
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6948>
2020-10-01 16:29:46 +00:00
Marek Olšák
f5912c6d32 radeonsi: kill disabled clip distances and planes at per-channel granularity
Apps often enable only 1 plane for gl_ClipVertex, which means 1 scalar
clip distance.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6948>
2020-10-01 16:29:46 +00:00
Marek Olšák
30c3b2c0b6 radeonsi: simplify NGG culling enablement and add radeonsi_shader_culling option
Add a vertex count threshold into si_shader_selector to simplify
the draw_vbo code.

The new option is supposed to be used in 00-mesa-defaults.conf and should be
tweaked for best performance unlike the AMD_DEBUG experimental options.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6948>
2020-10-01 16:29:46 +00:00
Sagar Ghuge
b02bef01c8 intel/blorp: Conditionally clear full surface depth and stencil
We should set "Full Surface Depth and Stencil Clear" field of WM_HZ_OP
3DSTATE packet, only when application requires the entire depth surface
to be cleared.

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6549>
2020-10-01 16:23:10 +00:00
Jason Ekstrand
d5849bc840 anv: Skip HiZ and CCS ambiguates which preceed fast-clears
This gets rid of multiple HiZ ambiguate operations per frame in
Witcher 3.

v2:
- Fix typo (Tapani)

Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6549>
2020-10-01 16:23:10 +00:00
Jason Ekstrand
e9d5ec342d anv: Use more temp vars in cmd_buffer_begin_subpass
This is a mostly cosmetic change but there is one subtle functional
issue:  If we ever render to a 3D depth image, we are now handling the
base layer and number of layers correctly.  I'm not sure rendering to 3D
depth is even allowed but we can theoretically handle it now.

Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6549>
2020-10-01 16:23:10 +00:00
Jason Ekstrand
7c92e413af anv: Allow HiZ clears for multi-view
Now that we're enabling HiZ on multi-layer images, there's no reason why
we can't enable HiZ clears for multi-view.  The only reason I can think
of why we didn't before was because no one thought to and the old code
didn't.  Enabling this means that an attachment will get HiZ cleared if
and only if att_state->fast_clear.

Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6549>
2020-10-01 16:23:10 +00:00
Eleni Maria Stea
03af98abe2 radeonsi: support for external buffers (ext_external_objects)
So far, the callback to create a resource from a memory object had code
for importing textures only. Modified it to allow importing buffers too.

Fixes the following piglit tests:
- ext_external_objects/vk-buf-exchange
- ext_external_objects/vk-pix-buf-update-errors
- ext_external_objects/vk-vert-buf-update-errors
- ext_external_objects/vk-vert-buf-reuse

v2: Used si_alloc_buffer_struct instead of CALLOC
v3: Fixed indentation issue, removed free in case of unsuccessful
allocation, joined two if conditions together

Signed-off-by: Eleni Maria Stea <estea@igalia.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6364>
2020-10-01 15:35:07 +00:00
Samuel Pitoiset
df63491594 radv/aco: lower IO for all stages outside of ACO
Lowering IO for VS, TCS, TES and GS still have to be done for LLVM.

No fossils-db change on NAVI10.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6897>
2020-10-01 14:58:25 +00:00
Samuel Pitoiset
2c322514f3 radv: gather output usage mask from store_output for VS, TES and GS
IO are now lowered before the shader info pass is called and the
output usage masks have to be gathered from store_output instead.

This is currently only used by ACO.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6897>
2020-10-01 14:58:25 +00:00
Connor Abbott
79368ab302 ttn: Fix number of components for IF/UIF
NIR if statements only take one component, but TGSI registers are vec4.
We're supposed to compare the x component, per
https://docs.mesa3d.org/gallium/tgsi.html#opcode-IF.

Fixes: f103bded ("ttn: Use nir control flow insertion helpers")
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Tested-by: Leo Liu <leo.liu@amd.com>
Closes: #3585
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6956>
2020-10-01 15:47:07 +02:00
Samuel Pitoiset
b00a023f1e ac/nir: fix nir_intrinsic_shared_atomic_fadd
This was completely broken.

Fixes dEQP-VK.glsl.atomic_operations.add_float32_compute_shared.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6936>
2020-10-01 06:38:42 +00:00
Samuel Pitoiset
8227b08c08 ac/llvm: fix invalid use of unreachable in ac_build_atomic_rmw()
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6936>
2020-10-01 06:38:42 +00:00
Samuel Pitoiset
892e74d2f7 radv: fix gathering writes_memory for global store/atomic operations
Because global operations are lowered before the shader info pass now
we have to adjust the gathering code.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3578
Fixes: 1588644543 ("radv: lower deref operations for global memory for both backends")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6934>
2020-10-01 08:14:18 +02:00
Dave Airlie
e94fd4cc65 lavapipe: rename vallium to lavapipe
Just a cooler name, and a lot easier to search for. thanks Marek

Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6921>
2020-10-01 00:23:40 +00:00
Olsak, Marek
5e8791a0bf radeonsi: Fix dead lock with aux_context_lock in si_screen_clear_buffer.
After disable SDMA on Arcturus(gfx9), dead lock with aux_context_lock is
detected since si_screen_clear_buffer is called recursively before
release lock.

The call trace is:
si_clear_render_target->si_compute_clear_render_target->
si_launch_grid_internal->si_launch_grid->si_emit_cache_flush->
si_prim_discard_signal_next_compute_ib_start->u_suballocator_alloc->
si_resource_create->si_buffer_create->si_alloc_resource->
si_screen_clear_buffer->simple_mtx_lock->
si_sdma_clear_buffer->si_pipe_clear_buffer->
si_clear_buffer->si_compute_do_clear_or_copy->
si_launch_grid_internal->si_launch_grid->si_emit_cache_flush->
si_prim_discard_signal_next_compute_ib_start->u_suballocator_alloc->
si_resource_create->si_buffer_create->si_alloc_resource->
si_screen_clear_buffer->simple_mtx_lock

Fixes: 07a49bf597 "radeonsi: disable SDMA on gfx9"
Signed-off-by: James Zhu <James.Zhu@amd.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6941>
2020-10-01 00:06:29 +00:00
Eric Engestrom
90e42f87ac add one last 20.1 release to coincide with expected 20.2.1
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6943>
2020-09-30 20:20:40 +00:00
Eric Engestrom
fe16e40974 docs: update calendar and link releases notes for 20.1.9
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6943>
2020-09-30 20:20:40 +00:00
Eric Engestrom
00d87db89b docs: add release notes for 20.1.9
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6943>
2020-09-30 20:20:40 +00:00
Eric Anholt
49ec863e83 freedreno/ir3: Enable the i/o vectorizer on UBOs.
This will merge loads of UBO components together into vec4 loads.  At the
same time, it improves the alignment information on our loads, fixing the
regression from the vec3 loads fix.

shader-db results:
total instructions in shared programs: 12829370 -> 8755851 (-31.75%)
total cat6 in shared programs: 145840 -> 97027 (-33.47%)

Overall results from before the vec3 fix:
total instructions in shared programs: 8019997 -> 8755851 (9.18%)
total cat6 in shared programs: 87683 -> 97027 (10.66%)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6612>
2020-09-30 19:53:43 +00:00
Eric Anholt
e3f4655805 nir: Make nir_lower_ubo_vec4() handle non-vec4-aligned loads.
It turns out I had missed a case in my enumeration of why everything
currently was vec4-aligned.

Fixes a simple testcase of loading from a vec3[2] array in freedreno with
IR3_SHADER_DEBUG=nouboopt.

Initial shader-db results look devastating:

total instructions in shared programs: 8019997 -> 12829370 (59.97%)
total cat6 in shared programs: 87683 -> 145840 (66.33%)

Hopefully this will recover once we introduce the i/o vectorizer, but that
was blocked on getting the vec3 case fixed.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6612>
2020-09-30 19:53:43 +00:00
Eric Anholt
618556a8cb nir: Drop the high_offset argument to the load_store_vectorizer filter.
Nothing uses it, and it's not clear to me what it provides over
alignment/num_components/bit_size.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6612>
2020-09-30 19:53:43 +00:00
Eric Anholt
5f757bb95c nir: Make the load_store_vectorizer provide align_mul + align_offset.
It was passing an encoding of the two that wasn't good for ensuring "Don't
combine loads that would make us straddle a vec4 boundary" for
nir_lower_ubo_vec4.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6612>
2020-09-30 19:53:43 +00:00
Eric Anholt
9c5a793dc7 nir/gl_nir_lower_buffers: Set up align_mul/offset on UBOs.
nir_lower_to_explicit_io will give us good alignments if we have the
cast's alignment information known, and it's trivial: Just the offset of
the UBO variable that is at the base of the deref.  Otherwise, explicit io
assumes the load is aligned just to the size of a scalar value in it.

The change in freedreno is in the noise.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6612>
2020-09-30 19:53:43 +00:00
Eric Anholt
ffbfc1ec0e nir/nir_lower_uniforms_to_ubo: Set better alignments on our new instructions.
The change on freedreno is in the noise.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6612>
2020-09-30 19:53:43 +00:00
Eric Anholt
c88c89ff3e nir: Print the alignment information on casts.
I wanted it for debugging GL alignment.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6612>
2020-09-30 19:53:43 +00:00
Eric Anholt
6c1c571440 nir: Document a bit about how align_mul/offset work.
Introduces a #define for the maximum valid align_mul that's used in the
load_store_vectorizer tests (currently, though it will be used more soon).

Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6612>
2020-09-30 19:53:43 +00:00
Eric Anholt
bd60e31c83 freedreno/ir3: Make sure we run the opt loop after lowering UBOs to vec4.
The lowering pass may introduce vector bcsels that we need to scalarize
back out.  It's unusual to have UBOs and not get any lowered to push
constants, so the flag was usually set anyway.

Fixes: 2b25240993 ("freedreno/ir3: Replace our custom vec4 UBO intrinsic
with the shared lowering.")

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6612>
2020-09-30 19:53:43 +00:00
Jonathan Marek
8dc8922af2 turnip: implement legacy API functions separately
Move legacy API functions to a separate file, and implement them by calling
the new API functions, like tu_CreateRenderPass was already doing.

Signed-off-by: Jonathan Marek <jonathan@marek.ca>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6920>
2020-09-30 17:02:55 +00:00
Jason Ekstrand
25ebd7f90f Revert "nir/lower_goto_if: Add a route::outside set"
This reverts commit d57573dcd4.  The
actual bug was an issue with prev_frontiers which has been properly
fixed in the previous commit.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6750>
2020-09-30 16:46:11 +00:00
Jason Ekstrand
57c9fc3cba nir/lower_goto_ifs: Always include level dom_frontiers in prev_frontier
When we come in from some other level or from the parent, we need to
ensure that the reach set is in prev_frontier but we also need to
consider the dominance frontier of our level.  Otherwise, we may end up
leaving out possible blocks when computing the reach of a level.

Acked-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6750>
2020-09-30 16:46:11 +00:00
Jason Ekstrand
7749983658 nir/lower_goto_ifs: Add asserts for SSA forks
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6750>
2020-09-30 16:46:11 +00:00
Jason Ekstrand
dc010cb74e nir/lower_goto_ifs: Use rzalloc
In particular, SSA forks weren't always getting properly initialized
which was causing asserts to fail.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6750>
2020-09-30 16:46:11 +00:00
Jason Ekstrand
fa3c38ceb3 spirv: Only run repair_ssa if structured
We shouldn't need it if we're unstructured and the pass assumes
structure so attempting to run it will assert-fail.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6750>
2020-09-30 16:46:11 +00:00
Jason Ekstrand
719c68016a nir/dominance: Use _mesa_set_clear instead ofhand-rolling it
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6750>
2020-09-30 16:46:11 +00:00
Jason Ekstrand
b6a4172f10 nir/lower_goto_ifs: Don't destroy SSA form in the process
There are two issues here:

 1. If there are any phi nodes, we'll make complete hash of them.  This
    isn't likely actually a problem because spirv_to_nir doesn't
    generate any actual phi nodes today.  However, if we start doing any
    other passes before this, we may have a problem.

 2. Even without phi nodes, we may still break SSA form.  This can
    happen if we ever have to stick a block inside a conditional to
    satisfy weird CFG constraints.  Doing so can cause it to no longer
    look like it dominates some of its uses even though, at runtime,
    it's guaranteed to be run first.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6750>
2020-09-30 16:46:11 +00:00
Jason Ekstrand
6f134a622b nir/validate: Improve the validation of blocks
This commit adds a number of new validation checks:

 1. We now check that every block pointer in the IR points to a block
    that actually exists in a block list that's reachable from the
    nir_function_impl.

 2. We assert that nir_function_impl::body is non-empty

 3. We assert that the start block has no predecessors.  This is
    important because we tend to put run-once code there.

 4. We now validate some stuff on the end block.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6750>
2020-09-30 16:46:11 +00:00
Marek Olšák
4fb2eddfdf gallium/util: remove empty file u_half.h
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6774>
2020-09-30 16:28:24 +00:00
Marek Olšák
7a1deb16f8 gallium/util: remove redundant util_float_to_half_rtz
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6774>
2020-09-30 16:28:24 +00:00
Marek Olšák
2b6a172343 util: remove util_float_to_half and util_half_to_float wrappers
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6774>
2020-09-30 16:28:24 +00:00
Marek Olšák
5af81393e4 util: move util_half_to_float code into _mesa_half_to_float_slow
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6774>
2020-09-30 16:28:24 +00:00
Marek Olšák
87900afe5b util: implement f16c - fast half<->float conversions
This also happens to fix bptc-float-modes on llvmpipe.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6774>
2020-09-30 16:28:24 +00:00
Jason Ekstrand
7dbb1f7462 nir/cf: Better handle intra-block splits
In the case where end was a instruction-based cursor, we would mix up
our blocks and end up with block_begin pointing after the second split.
This causes a segfault as the cf_node list walk at the end of the
function never terminates properly.  There's also a possibility of
mix-up if begin is an instruction-based cursor which was found by
inspection.

Fixes: fc7f2d2364 "nir/cf: add new control modification API's"
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Acked-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-30 15:47:51 +00:00
Jason Ekstrand
5e2e882270 nir: Disallow goto and goto_if in clone and [de]serialize
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-30 15:47:51 +00:00
Jason Ekstrand
9a48ed84ec nir/copy_propagate: Copy-prop into jump conditions
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-30 15:47:51 +00:00
Connor Abbott
7f0cd6f153 nir/opt_if: Use early returns in opt_if_merge()
We would've had to add yet another level of indentation, or duplicated
finding the if conditions in the next commit. Refactor this function to
use early returns like our other optimizations, so that this isn't an
issue.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-30 15:47:51 +00:00
Connor Abbott
656e428ff4 nir/opt_if: Remove open-coded nir_ssa_def_rewrite_uses()
So that we don't have to change these two places later.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-30 15:47:51 +00:00
Connor Abbott
c6f871b62e nir/lower_returns: Use nir control flow insertion helpers
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-30 15:47:51 +00:00
Connor Abbott
f103bded0b ttn: Use nir control flow insertion helpers
As a side effect, we can delete the whole control flow stack thing.

v2 (Jason Ekstrand):
 - Drop the ttn_if helper and just inline it in the two uses

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-30 15:47:51 +00:00