Commit graph

140962 commits

Author SHA1 Message Date
Jason Ekstrand
cdb88f67dc anv/apply_pipeline_layout: Refactor descriptor chasing code
This makes things a bit more generic for use in the next commit.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:58 +00:00
Jason Ekstrand
0aa3d68206 anv: Use nir_shader_instructions_pass in apply_pipeline_layout
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:58 +00:00
Jason Ekstrand
bfe92b83db anv: Use load_global_constant for shader constants
NIR can do a bit better job optimizing this version.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:58 +00:00
Jason Ekstrand
1ce3660a5a intel/fs,rt: Add a predicate to load_global_const_block
This allows us to do bounds checked A64 block load without the it being
counted as control-flow by NIR.  This means that NIR optimizations like
CSE will be able to work on these the same as a regular load.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:58 +00:00
Eric Anholt
2407952ec9 ci/bare-metal: Restart a run on intermittent kernel lockups.
Since enabling SMP on db820c and cranking up how many tests we run, we've
been seeing lockups like this a couple of times a week.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9655>
2021-03-17 17:13:22 +00:00
Rob Clark
5d2c9fd161 freedreno/drm: Avoid unitialized timestamp in submit fail
Saw a flood of "waiting on invalid fence" with a completely bogus
looking fence # in a log of a rather strange low-memory crash.  Not
sure if it is coming from memory corruption in userspace, but if a
submit ioctl is failing due to failed allocation (or other reason)
we would get left with random stack garbage as the fence #.  Let's
not have that as a potential problem.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9638>
2021-03-17 16:36:37 +00:00
Rhys Perry
5bc100eb2d aco: use a single instruction for uadd32_sat() on GFX8
fossil-db (GFX8):
Totals from 8 (0.01% of 147787) affected shaders:
SGPRs: 352 -> 368 (+4.55%)
CodeSize: 49576 -> 48788 (-1.59%)
Instrs: 9487 -> 9318 (-1.78%)
Latency: 49935 -> 49607 (-0.66%)
InvThroughput: 138493 -> 137443 (-0.76%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>
2021-03-17 15:33:34 +00:00
Rhys Perry
3decb52c82 aco: use uadd32_sat() helper for nir_op_uadd_sat
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>
2021-03-17 15:33:31 +00:00
Rhys Perry
590de30093 aco: implement 64-bit VGPR {u,i}find_msb
This can be created by subgroupBallotFindMSB().

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4458
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>
2021-03-17 15:33:22 +00:00
Marek Olšák
32eb74e1e1 ac/gpu_info: fix more non-coherent RB and GL2 combinations
It ignored non-harvested chips with a non-power-of-two memory bus.

Fixes: abed921ce7 - amd: add support for Navy Flounder

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9568>
2021-03-17 14:40:54 +00:00
Erik Faye-Lund
d4bcb58caf zink: fix free of ralloced pointer
When we alloc with ralloc, we also need to free with it.

But let's take a step back here; we don't just need to use ralloc, we
also need to destroy all other screen-resources. So let's call the
destructor here instead.

Fixes: 2643f9ed28 ("zink: ralloc screen objects")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9647>
2021-03-17 13:08:10 +00:00
Erik Faye-Lund
539036a990 zink: fix emulation of no mipfilter
This approach is taken from the Vulkan spec[1], where a robust maxLod
of 0.25 is proposed instead of 0.0.

This has the effect of allowing room for both minification and
magnification filters, yet still rounding down to the right miplevel in
the end.

[1]: https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSamplerCreateInfo.html#_description

Fixes: 8d46e35d16 ("zink: introduce opengl over vulkan")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9649>
2021-03-17 12:58:12 +00:00
Timur Kristóf
ed7c6e46e7 aco: Delete superfluous tess and ESGS I/O code.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
08fb6904ec radv/llvm: Delete superfluous tess and ESGS I/O code.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
16021e3bae radv/llvm: Only store TCS outputs where they are really needed.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
540168fd15 radv: Use new, NIR-based I/O lowering.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
1958381c9a radv: Reorder some NIR optimizations in preparation for the I/O changes.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
b3a16c0e19 radv: Fill some tess shader info earlier.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
52219ad3a0 radv: Determine tcs_in_out_eq in radv_pipeline instead of the compiler.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
e1ee17249a radv: Calculate tess patches and LDS use outside the backend compilers.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
a736ea5dc6 radv: Save I/O usage data to both shader infos for merged stages.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
c564a452fc radv: Lower IO and set driver locations earlier.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
3185cb7dbf ac: Add NIR passes to lower ES->GS I/O to memory accesses.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
bf966d1c1d ac: Add NIR passes to lower VS->TCS->TES I/O to memory accesses.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
14ad82b4e9 ac/llvm: Emit more efficient code for load_shared.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
d5a79dbcf8 ac/llvm: Add constant offset to load/store_shared.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
00c014aab2 ac/llvm: Make sure to always emit integer comparison for nir_op_ieq.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
252b5d5ecd ac/llvm: Make shared loads/stores work correctly for non-CS stages.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
6e7b1cd251 ac/llvm: Implement new Geometry Shader intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
20a2801011 ac/llvm: Implement the new tessellation intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
60114f3865 ac/llvm: Implement AMD-specific buffer load/store intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
582229585b aco: Implement new Geometry Shader intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
5c95b32c6e aco: Implement the new tessellation I/O related NIR intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
e10e74a7af aco: Implement new buffer load/store intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
4c5c610f1d nir: Add AMD-specific Geometry Shader related intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
38df949f98 nir: Add tessellation related AMD-specific intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
744dc74078 nir: Add nir_opt_offsets to fold const adds into load/store offsets.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
eee3435757 nir: Add AMD-specific buffer load/store intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
c2a81ebe19 nir: Add default unsigned upper bound configuration.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
8ebb8d31af nir: Add unsigned upper bound for TCS load_invocation_id.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
9fbfafb57a nir: Shrink vectors for load_shared.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
084863bb5d nir: Fix unsigned upper bound of local_invocation_index for non-CS stages.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
132171dc4e nir: Add a few more algebraic optimizations to help address calculation.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
9f9b0f583b nir: Add nir_builder helper for I/O address offset calculations.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
f6f68d5cf1 nir: Add new nir_builder helpers for iadd with no_unsigned_wrap.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Rhys Perry
5bc42ce579 nir: Don't update base in vectorize_loads()
The offset is already updated with consideration to the base above under
"/* update the offset */".

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Rhys Perry
c580c3f9c7 aco/tests: add test for waNsaCannotFollowWritelane
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>
2021-03-17 12:31:05 +00:00
Rhys Perry
502a073552 aco: fix NSA following writelane
No fossil-db changes on GFX10.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: c353895c92 ("aco: use non-sequential addressing")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>
2021-03-17 12:31:05 +00:00
Rhys Perry
298d400e5c aco/tests: add test for NSAToVMEMBug
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>
2021-03-17 12:31:05 +00:00
Rhys Perry
194f3e4c69 aco: fix NSA MIMG followed by MUBUF/MTBUF
No fossil-db changes on GFX10.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Fixes: c353895c92 ("aco: use non-sequential addressing")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>
2021-03-17 12:31:05 +00:00