Commit graph

136668 commits

Author SHA1 Message Date
Jason Ekstrand
e13246e053 anv/apply_pipeline_layout: Add some switch statements
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:59 +00:00
Jason Ekstrand
f872a26991 anv/apply_pipeline_layout: Plumb through a UBO address format
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:59 +00:00
Jason Ekstrand
2b8f76b128 anv/apply_pipeline_layout: Move bounds checking later for index/offset
Instead of doing the array check at the load_vulkan_resource_index
intrinsic, stuff it in the vec2 and handle it at load_vulkan_descriptor
time.  This allows the bounds check to take any re-index intrinsics into
account.  This only affects variablePointers + SSBOs + Gen7.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:59 +00:00
Jason Ekstrand
2beba9dd5a anv/apply_pipeline_layout: Run DCE between the early and late passes
This allows us to ignore UBOs in the late code going forward.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:59 +00:00
Jason Ekstrand
a7fe687bde anv/apply_pipeline_layout: Lower UBO loads in the early pass
We're about to enable bindless UBOs via A64 memory access like we do for
SSBOs.  In order to prevent 100% of UBOs from hitting that path, we
enable them in the early lowering.  This way we'll still get binding
table-based UBO access for any non-bindless ones.  In particular, we
need this for UBO pushing to work.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:59 +00:00
Jason Ekstrand
799a931d12 anv/apply_pipeline_layout: Rework the early pass index/offset helpers
Rewrite them all to work on an index/offset vec2 instead of some only
returning the index.  This means SSBO size handling is a tiny bit more
complicated but it will also mean we can use them for descriptor buffers
properly.

This also fixes a bug where we weren't bounds-checking re-index
intrinsics because we applied the bounds check at the tail of the
recursion and not at the beginning.

Fixes: 3cf78ec2bd "anv: Lower some SSBO operations in..."
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:58 +00:00
Jason Ekstrand
cdb88f67dc anv/apply_pipeline_layout: Refactor descriptor chasing code
This makes things a bit more generic for use in the next commit.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:58 +00:00
Jason Ekstrand
0aa3d68206 anv: Use nir_shader_instructions_pass in apply_pipeline_layout
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:58 +00:00
Jason Ekstrand
bfe92b83db anv: Use load_global_constant for shader constants
NIR can do a bit better job optimizing this version.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:58 +00:00
Jason Ekstrand
1ce3660a5a intel/fs,rt: Add a predicate to load_global_const_block
This allows us to do bounds checked A64 block load without the it being
counted as control-flow by NIR.  This means that NIR optimizations like
CSE will be able to work on these the same as a regular load.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8635>
2021-03-17 17:49:58 +00:00
Eric Anholt
2407952ec9 ci/bare-metal: Restart a run on intermittent kernel lockups.
Since enabling SMP on db820c and cranking up how many tests we run, we've
been seeing lockups like this a couple of times a week.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9655>
2021-03-17 17:13:22 +00:00
Rob Clark
5d2c9fd161 freedreno/drm: Avoid unitialized timestamp in submit fail
Saw a flood of "waiting on invalid fence" with a completely bogus
looking fence # in a log of a rather strange low-memory crash.  Not
sure if it is coming from memory corruption in userspace, but if a
submit ioctl is failing due to failed allocation (or other reason)
we would get left with random stack garbage as the fence #.  Let's
not have that as a potential problem.

Signed-off-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9638>
2021-03-17 16:36:37 +00:00
Rhys Perry
5bc100eb2d aco: use a single instruction for uadd32_sat() on GFX8
fossil-db (GFX8):
Totals from 8 (0.01% of 147787) affected shaders:
SGPRs: 352 -> 368 (+4.55%)
CodeSize: 49576 -> 48788 (-1.59%)
Instrs: 9487 -> 9318 (-1.78%)
Latency: 49935 -> 49607 (-0.66%)
InvThroughput: 138493 -> 137443 (-0.76%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>
2021-03-17 15:33:34 +00:00
Rhys Perry
3decb52c82 aco: use uadd32_sat() helper for nir_op_uadd_sat
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>
2021-03-17 15:33:31 +00:00
Rhys Perry
590de30093 aco: implement 64-bit VGPR {u,i}find_msb
This can be created by subgroupBallotFindMSB().

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4458
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9598>
2021-03-17 15:33:22 +00:00
Marek Olšák
32eb74e1e1 ac/gpu_info: fix more non-coherent RB and GL2 combinations
It ignored non-harvested chips with a non-power-of-two memory bus.

Fixes: abed921ce7 - amd: add support for Navy Flounder

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9568>
2021-03-17 14:40:54 +00:00
Erik Faye-Lund
d4bcb58caf zink: fix free of ralloced pointer
When we alloc with ralloc, we also need to free with it.

But let's take a step back here; we don't just need to use ralloc, we
also need to destroy all other screen-resources. So let's call the
destructor here instead.

Fixes: 2643f9ed28 ("zink: ralloc screen objects")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9647>
2021-03-17 13:08:10 +00:00
Erik Faye-Lund
539036a990 zink: fix emulation of no mipfilter
This approach is taken from the Vulkan spec[1], where a robust maxLod
of 0.25 is proposed instead of 0.0.

This has the effect of allowing room for both minification and
magnification filters, yet still rounding down to the right miplevel in
the end.

[1]: https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSamplerCreateInfo.html#_description

Fixes: 8d46e35d16 ("zink: introduce opengl over vulkan")
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9649>
2021-03-17 12:58:12 +00:00
Timur Kristóf
ed7c6e46e7 aco: Delete superfluous tess and ESGS I/O code.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
08fb6904ec radv/llvm: Delete superfluous tess and ESGS I/O code.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
16021e3bae radv/llvm: Only store TCS outputs where they are really needed.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
540168fd15 radv: Use new, NIR-based I/O lowering.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
1958381c9a radv: Reorder some NIR optimizations in preparation for the I/O changes.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
b3a16c0e19 radv: Fill some tess shader info earlier.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
52219ad3a0 radv: Determine tcs_in_out_eq in radv_pipeline instead of the compiler.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
e1ee17249a radv: Calculate tess patches and LDS use outside the backend compilers.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
a736ea5dc6 radv: Save I/O usage data to both shader infos for merged stages.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
c564a452fc radv: Lower IO and set driver locations earlier.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
3185cb7dbf ac: Add NIR passes to lower ES->GS I/O to memory accesses.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
bf966d1c1d ac: Add NIR passes to lower VS->TCS->TES I/O to memory accesses.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
14ad82b4e9 ac/llvm: Emit more efficient code for load_shared.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
d5a79dbcf8 ac/llvm: Add constant offset to load/store_shared.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
00c014aab2 ac/llvm: Make sure to always emit integer comparison for nir_op_ieq.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
252b5d5ecd ac/llvm: Make shared loads/stores work correctly for non-CS stages.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
6e7b1cd251 ac/llvm: Implement new Geometry Shader intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
20a2801011 ac/llvm: Implement the new tessellation intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
60114f3865 ac/llvm: Implement AMD-specific buffer load/store intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
582229585b aco: Implement new Geometry Shader intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
5c95b32c6e aco: Implement the new tessellation I/O related NIR intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
e10e74a7af aco: Implement new buffer load/store intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
4c5c610f1d nir: Add AMD-specific Geometry Shader related intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
38df949f98 nir: Add tessellation related AMD-specific intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
744dc74078 nir: Add nir_opt_offsets to fold const adds into load/store offsets.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
eee3435757 nir: Add AMD-specific buffer load/store intrinsics.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
c2a81ebe19 nir: Add default unsigned upper bound configuration.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
8ebb8d31af nir: Add unsigned upper bound for TCS load_invocation_id.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
9fbfafb57a nir: Shrink vectors for load_shared.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
084863bb5d nir: Fix unsigned upper bound of local_invocation_index for non-CS stages.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
132171dc4e nir: Add a few more algebraic optimizations to help address calculation.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
9f9b0f583b nir: Add nir_builder helper for I/O address offset calculations.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00