Commit graph

819 commits

Author SHA1 Message Date
Timur Kristóf
fdbb296853 aco: Remember VS/TCS output driver locations.
Instead of relying on calling shader_io_get_unique_index repeatedly,
remember the which output driver location corresponds to which
varying slot.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4388>
2020-04-29 11:51:04 +00:00
Timur Kristóf
ab07c4ea70 aco: Use context variables instead of calculating TCS inputs/outputs.
VS needs the number of TCS inputs, and TES needs the number of TCS
outputs.

It is error-prone to repeat those calculations in both instruction
selection and setup. Just set them in one place instead.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4388>
2020-04-29 11:51:04 +00:00
Timur Kristóf
fd0248c37b radv: Refactor calculate_tess_lds_size and get_tcs_num_patches.
Previously these functions needed the bit mask of the TCS outputs
and patch outputs written, and concluded the number of outputs
from that.

Now, they take the number of outputs and patch outputs instead.
This will allow the backend compiler to better optimize the
LDS layout.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4388>
2020-04-29 11:51:04 +00:00
Rhys Perry
9392ddab43 aco: consider blocks unreachable if they are in the logical cfg
unreachable was true if the last block is unreachable in the linear cfg,
but it should also be true if it is unreachable in the logical cfg.

Fixes dEQP-VK.graphicsfuzz.for-with-ifs-and-return

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: 8d8c864beb
    ('aco: improve check for unreachable loop continue blocks')

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4764>
2020-04-29 11:07:09 +00:00
Samuel Pitoiset
60cc065c7d aco: fix adjusting the sample index with FMASK if value is negative
The SPIR-V spec doesn't say explicitly that the sample index
must be an unsigned integer.

This fixes crashes with some new VK_EXT_robustness2 tests.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4775>
2020-04-29 07:29:54 +00:00
Samuel Pitoiset
a112ec4c11 aco: fix nir_texop_texture_samples with NULL descriptors
With VK_EXT_robustness2, descriptors can be NULL and the number of
samples returned by nir_texop_texture_samples should be 0.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4775>
2020-04-29 07:29:54 +00:00
Rhys Perry
3ee3ad561a aco: fix vgpr nir_op_vecn with sgpr operands
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
c5eda3c746 aco: improve clamped integer addition disassembly workaround
Make it work with 16-bit and GFX10.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
4ed83e2f94 aco: add various GFX10 int16 opcodes
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
43f2ba39ef aco: fix sub-dword overwrite check in RA validator
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
cca8d6ce06 aco: fix sub-dword out-of-bounds check in RA validator
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
307aca83a2 aco: add missing adjust_max_used_regs()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
99ca96fbf5 aco: improve RA for uneven p_split_vector
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
24116a8a56 aco: don't recurse in sub-dword get_reg_simple()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
09c584caeb aco: split self-intersecting copies instead of swapping
Example situation:
v1 = {v0.hi, v1.lo}
v0.hi = v1.hi

The 4-byte copy's definition is completely used, but swapping it makes no
sense. We have to split it to generate correct code:
swap(v0.hi, v1.lo)
swap(v0.hi, v1.hi)

Found in dEQP-VK.spirv_assembly.type.vec3.i16.constant_composite_vert

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
be4a34966c aco: fix neighboring register check in get_reg_simple()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
fb59ed6bb9 aco: check alignment of non-subdword registers in get_reg_specified()
When splitting a v6b vector into v1 and v2b components, we should ensure
the v1 definition doesn't start at the upper half.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
916cc3e231 aco: make RegisterFile::block() take a regclass
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
eeccb1a941 aco: lower 8/16-bit integer arithmetic
dEQP-VK.spirv_assembly.type.* passes with the features and extensions
enabled.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4707>
2020-04-24 20:03:59 +01:00
Rhys Perry
bcd9467d5c aco: improve sub-dword emit_split_vector() with sgprs
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
a3dc1441f0 aco: clobber scc in s_bfe_u32 in get_alu_src()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
78389f4cbc aco: handle undef p_create_vector operands in the optimizer
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
deea4b7c5a aco: vectorize global loads/stores
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
7db7206631 aco: allow 8/16-bit shared loads
These should work now

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
48b7beb7b0 aco: add and use get_buffer_store_op() helper
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
936b70c8cf aco: refactor visit_store_scratch() to use new helpers
Should support 8/16-bit stores now

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
18817041f7 aco: refactor visit_store_global() to use new helpers
Should support 8/16-bit stores now

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
c7bd69b3ae aco: refactor visit_store_ssbo() to use new helpers
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
f75c830433 aco: refactor store_vmem_mubuf() to use new helpers
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
98b4cc7110 aco: refactor store_lds() to use new helpers
It should also work correctly for 8/16-bit stores

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
562353e1f1 aco: add helpers for splitting stores
split_store_data() splits a vector and p_as_uniforms it if needed.

scan_write_mask()/advance_write_mask() are similar to
u_bit_scan_consecutive_range(), but makes it easier to only clear part of
the range and will also give ranges for zero'd bits.

split_buffer_store() is a helper for splitting VMEM/SMEM stores.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
211a9f2057 aco: use emit_load helper for VMEM/SMEM loads
Also implements 8/16-bit loads for scratch/global.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
57e6886f98 aco: refactor load_lds to use new helpers
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
542733dbbf aco: add emit_load helper
This helper is used for recombining split loads, passing the result to
p_as_uniform, aligning the offset down and shifting it right if needed and
handling large constant offsets.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
b77d638e1b aco: add and use RegClass::get() helper
Eventually, we'll probably want to replace the current
RegClass(type, size) constructor with this.

This has a functional change in that get_reg_class() now creates v1/v2
instead of v4b/v8b.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Rhys Perry
69b92db131 aco: be more careful about using SMEM for load_global
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4639>
2020-04-24 18:52:54 +00:00
Timur Kristóf
62ff2ff808 aco: Move s_setprio to correct place after the gs_alloc_req.
Previously the setprio was inside the branch, so it would only reset
the priority on the first wave, but not the others.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>
2020-04-24 17:58:57 +00:00
Timur Kristóf
277f37d036 aco: Use 24-bit multiplication for NGG wave id and thread id.
Both of them should always fit 24 bits anyway.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>
2020-04-24 17:58:57 +00:00
Timur Kristóf
eafc1e7365 aco: Use 24-bit multiplication in TCS I/O
The TCS inputs and outputs must always fit into the LDS,
which implies that their addresses also always fit 24 bits.

On AMD GPUs, 24-bit multiplication is much faster than 32-bit
multiplication, so we can take the opportunity to use that
for TCS I/O instead.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>
2020-04-24 17:58:57 +00:00
Timur Kristóf
64332a0937 aco: Const correctness for aco_print_ir.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>
2020-04-24 17:58:57 +00:00
Timur Kristóf
0c0691d43e aco: Const correctness for get_barrier_interaction.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>
2020-04-24 17:58:57 +00:00
Timur Kristóf
f321dc33c8 aco: Abort when RA can't find a register.
Previously, it was just unreachable, which means it will generate
invalid shaders when it encounters a situation when it can't allocate
registers for eg. a large load.

This commit makes it slightly easier to notice such problems without
triggering a GPU hang.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>
2020-04-24 17:58:57 +00:00
Timur Kristóf
f2e7aee244 aco: Increase barrier_count to 7 to include barrier_barrier.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>
2020-04-24 17:58:57 +00:00
Timur Kristóf
25775d346c aco: Only store TCS outputs to VMEM when they are read by TES.
Totals from affected shaders (GFX10):
Code Size: 10832 -> 10736 (-0.89 %) bytes

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4536>
2020-04-24 17:58:57 +00:00
Rhys Perry
ede1c171c5 aco: fix outdated label_vec from p_create_vector labelling
Fixes random dEQP-VK.transform_feedback.fuzz.* crashes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 2dc550202e
    ('aco: copy-propagate p_create_vector copies of vectors')

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4730>
2020-04-24 12:21:15 +00:00
Rhys Perry
665250e830 aco: fix v_or(s_lshl) and v_add(s_lshl) optimizations
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Fixes: d1621834f3
    ('aco: combine VALU and SALU into various VOP3 instructions')

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2822
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4717>
2020-04-24 08:55:19 +00:00
Rhys Perry
0d9fe0405f aco: improve code for 32-bit isign
No shader-db changes on Navi.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4667>
2020-04-23 12:39:33 +00:00
Rhys Perry
d1621834f3 aco: combine VALU and SALU into various VOP3 instructions
shader-db (Navi):
Totals from 2916 (2.28% of 127638) affected shaders:
SGPRs: 184427 -> 184283 (-0.08%); split: -0.10%, +0.02%
VGPRs: 143520 -> 143640 (+0.08%); split: -0.00%, +0.09%
CodeSize: 14913548 -> 14913288 (-0.00%); split: -0.00%, +0.00%
MaxWaves: 26034 -> 26012 (-0.08%)
Instrs: 2935435 -> 2930960 (-0.15%); split: -0.15%, +0.00%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4667>
2020-04-23 12:39:33 +00:00
Rhys Perry
607fb4153d aco: move call to store_output_to_temps in store_ls_or_es_output earlier
Skips get_intrinsic_io_basic_offset()

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4667>
2020-04-23 12:39:33 +00:00
Rhys Perry
b497b774a5 aco: remove copy in load_input_from_temps()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4667>
2020-04-23 12:39:33 +00:00