Commit graph

62 commits

Author SHA1 Message Date
Daniel Schürmann
b98a4d4dd7 aco: refactor GPR limit calculation
This patch delays the calculation of GPR limits in order to
precisely incorporate extra registers (VCC etc.) and shared VGPRs.

Additionally, the allocation granularity is used to set the config.
This has some effect on the reported SGPR stats.

Totals (Navi10):
SGPRs: 6971787 -> 17753642 (+154.65%)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8921>
2021-02-12 19:00:18 +00:00
Rhys Perry
e115b01948 aco: return references in instruction cast methods
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8595>
2021-01-22 14:12:33 +00:00
Rhys Perry
1d245cd18b aco: use format-check methods
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8595>
2021-01-22 14:12:32 +00:00
Rhys Perry
70dbcfa1c9 aco: use instruction cast methods
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8595>
2021-01-22 14:12:32 +00:00
Rhys Perry
441ead5fb3 aco: remove Format::{VOP3A,VOP3B}
These are really the same as Format::VOP3.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8595>
2021-01-22 14:12:32 +00:00
Daniel Schürmann
21a7bea342 aco/validate: relax subdword restrictions
This affects constants/SGPRs on GFX6-8 and
the operand regClass of SDWA instructions.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8260>
2021-01-21 11:05:36 +00:00
Daniel Schürmann
77c9629046 aco/validate: ensure that Operand and Definition size matches for parallelcopies
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8260>
2021-01-21 11:05:36 +00:00
Daniel Schürmann
8fb66187ec aco/validate: validate that p_create_vector operands are aligned unless they are subdword operands
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8260>
2021-01-21 11:05:36 +00:00
Rhys Perry
c353895c92 aco: use non-sequential addressing
fossil-db (GFX10.3):
Totals from 70493 (50.57% of 139391) affected shaders:
SGPRs: 4232624 -> 4231808 (-0.02%); split: -0.09%, +0.07%
VGPRs: 2831772 -> 2764740 (-2.37%); split: -2.53%, +0.17%
CodeSize: 225584412 -> 225048740 (-0.24%); split: -0.44%, +0.21%
MaxWaves: 875319 -> 878837 (+0.40%); split: +0.44%, -0.04%
Instrs: 43157803 -> 42496421 (-1.53%); split: -1.54%, +0.01%
Cycles: 1656380132 -> 1641532056 (-0.90%); split: -0.94%, +0.04%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8523>
2021-01-20 16:46:54 +00:00
Rhys Perry
faf3e9a27f aco: move VADDR to the end of the operand list
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8523>
2021-01-20 16:46:54 +00:00
Daniel Schürmann
2caba08c1a aco: fix VOP3P assembly, VN and validation
aco/opcodes: rename v_pk_fma_mix* -> v_fma_mix*
and add modifier capabilities for VOP3P.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6680>
2021-01-13 17:46:55 +00:00
Rhys Perry
962c917cea aco: move MIMG VDATA to its own operand
We will want both a VDATA operand and a sampler for some TFE/LWE MIMG
instructions.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7775>
2021-01-08 14:27:07 +00:00
Tony Wasserka
b841b4fde8 aco: Add tests for subdword register allocation
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7461>
2020-12-29 18:57:10 +00:00
Rhys Perry
2d12991e01 aco: use FALLTHROUGH macro
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7844>
2020-12-01 10:28:36 +00:00
Tony Wasserka
2bb8874320 aco: Fix -Wshadow warnings
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7430>
2020-11-20 09:29:19 +00:00
Tony Wasserka
1a1099c54f aco: Fix format string used when raising validation errors
Validation errors mention the pretty-printed instruction including
operands with the reserved % character, which caused vasprintf to
expect more format arguments than aco provided.

Fixes: c2b1978aa4 ("aco: rework the way various compilation/validation errors are reported")
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7442>
2020-11-05 17:56:18 +00:00
Timur Kristóf
8d6246205a aco: Add some validation for PSEUDO_REDUCTION instructions.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7232>
2020-10-28 21:47:22 +01:00
Rhys Perry
6db5fbf9f2 aco: allow literals on sub-dword p_parallelcopy
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7216>
2020-10-27 15:24:38 +00:00
James Park
af8d488ea5 util,ac,aco,radv: Cross-platform memstream API
POSIX memstream is not available on Windows.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7143>
2020-10-19 03:37:42 -07:00
Rhys Perry
1a652244e4 aco: implement 16-bit literals
We can copy any value into a 16-bit subregister with a 3 dword
v_pack_b32_f16 on GFX10 or a v_and_b32+v_or_b32 on GFX9.

Because the generated code can depend on the register assignment and to
improve constant propagation, Builder::copy creates a p_create_vector in
the case of sub-dword literals.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7111>
2020-10-15 11:33:42 +00:00
Daniel Schürmann
9185b7c069 aco: add validation rules for p_split_vector
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6777>
2020-10-14 15:31:38 +00:00
Samuel Pitoiset
408195ec53 aco: remove useless occurences of radv_nir_compiler_options
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7061>
2020-10-14 15:09:34 +00:00
Rhys Perry
d2c18b7bf3 aco: use bit vectors for liveness sets
This seems to be much faster than hash sets. When compiling pipelines from
5 games, live_var_analysis takes about a third the time it used to and
fossilize-replay is ~1.77% faster.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6733>
2020-09-21 13:47:28 +00:00
Rhys Perry
e394ff9b5f aco: fix validation of sub-dword parallel-copies
Only the operands with a sub-dword definition need to be checked.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6662>
2020-09-09 15:00:45 +00:00
Samuel Pitoiset
a6146aa598 aco: validate that SMEM operands can use fixed registers
To fix a validation error when loading the scalar tma buffer
descriptor because it's not a temp but a fixed reg (tma_lo/tma_hi).

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6384>
2020-08-24 11:08:24 +00:00
Samuel Pitoiset
58817bda8b aco: fix file leak in ra_fail()
Fixes: c2b1978aa4 ("aco: rework the way various compilation/validation errors are reported")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6399>
2020-08-20 14:42:07 +00:00
Samuel Pitoiset
e901b901cb radv,aco: report ACO errors/warnings back via VK_EXT_debug_report
To help developers.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6318>
2020-08-20 08:15:08 +02:00
Samuel Pitoiset
c2b1978aa4 aco: rework the way various compilation/validation errors are reported
The upcoming change will allow to report all ACO errors (or warnings)
directly to the app via VK_EXT_debug_report. This is similar to what
we already do for reporting various SPIRV->NIR errors.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6318>
2020-08-20 08:15:06 +02:00
Samuel Pitoiset
bc723dfda7 aco: rename DEBUG_VALIDATE to DEBUG_VALIDATE_IR
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6318>
2020-08-20 08:15:04 +02:00
Daniel Schürmann
1b3be07b5f aco: ensure readfirstlane subdword operands are always dword aligned
Cc: 20.1 <mesa-stable@lists.freedesktop.org>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6024>
2020-07-23 18:18:36 +00:00
Rhys Perry
bf4b377b9b aco: make validate() usable in tests
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6013>
2020-07-21 19:38:43 +00:00
Samuel Pitoiset
83d2a73b73 aco: improve validation checks for readlane/writelane
This allows literals for the lane select on GFX10+. The doc says
that is should be a SGPR or a constant but VOP3 on GFX10+ allows
literals.

Some later validation code checks if literals are allowed anyways.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5010>
2020-06-22 11:24:27 +00:00
Rhys Perry
abfe28a6bb aco: fix validation of opsel when set for the definition
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5521>
2020-06-17 15:48:35 +00:00
Rhys Perry
de7c6950b3 aco: fix sub-dword opsel/sdwa checks
These should all check if the operand has a regclass. The opsel check
should also be skipped post-RA.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5504>
2020-06-17 10:57:17 +00:00
Daniel Schürmann
8006feda09 aco: don't allow SGPRs on logical phis
aco_validate() is called after phi lowering, now.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5496>
2020-06-16 14:46:19 +01:00
Rhys Perry
575b431c80 aco: validate sub-dword pseudo instructions
No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5245>
2020-06-15 18:24:22 +00:00
Rhys Perry
edf863d1d2 aco: use Info::definition_size instead of definition's regclass
16-bit abs/neg creates v_xor_b32/v_and_b32 with v2b definitions. These
instructions never do partial writes without SDWA.

No shader-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5040>
2020-06-10 15:05:11 +00:00
Rhys Perry
d9cfb8ad48 aco: validate instructions reading/writing upper halves/bytes
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5040>
2020-06-10 15:05:11 +00:00
Samuel Pitoiset
af7e2c6133 aco: validate 8-bit/16-bit VGPR operands for readfirstlane/readlane/writelane
I would expect it to just work as intended and other solutions,
like v_and_b32 to make sure the upper bits are 0, might have some
overhead.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4494>
2020-05-21 15:06:48 +00:00
Daniel Schürmann
51f4b22fee aco: don't allow unaligned subdword accesses on GFX6/7
There are no SDWA instructions which means that only
full registers can be accessed.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5070>
2020-05-21 12:07:40 +00:00
Samuel Pitoiset
34f2c4dc6a aco: validate v_interp_*_f16 as VOP3 instructions instead of VINTRP
16-bit interp instructions are considered VINTRP by the compiler
but they are emitted as VOP3 by the assembler.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4966>
2020-05-19 17:05:05 +00:00
Rhys Perry
43f2ba39ef aco: fix sub-dword overwrite check in RA validator
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Rhys Perry
cca8d6ce06 aco: fix sub-dword out-of-bounds check in RA validator
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4772>
2020-04-28 23:16:55 +00:00
Daniel Schürmann
edc2b57ac1 aco: allocate full register for subdword definitions if HW doesn't support it
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4573>
2020-04-22 18:23:23 +00:00
Rhys Perry
e18711cda3 aco: fix p_extract_vector validation
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4507>
2020-04-14 10:49:12 +00:00
Daniel Schürmann
3c0c28a1ff aco: validate RA of subdword assignments
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4002>
2020-04-03 23:13:15 +01:00
Daniel Schürmann
799bb10328 aco: validate uninitialized operands
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4002>
2020-04-03 23:13:15 +01:00
Daniel Schürmann
9374659426 aco: validate register alignment of subdword operands and definitions
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4002>
2020-04-03 23:13:15 +01:00
Daniel Schürmann
ad4e104bb9 aco: validate p_create_vector with subdword elements properly
Also allows for undef operands

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4002>
2020-04-03 23:13:15 +01:00
Rhys Perry
9915af5ca1 aco: print and validate opsel
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>

Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-By: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4002>
2020-04-03 23:13:15 +01:00