Commit graph

6262 commits

Author SHA1 Message Date
Samuel Pitoiset
729bdc0d70 aco: fix f2i64/f2u64 with sgprs if the exponent computation overflow
This fixes f16->{i64,u64} conversions for +0/-0.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4509>
2020-04-13 11:51:17 +00:00
Daniel Schürmann
38622de2ec aco: make some reg_file helpers private and fix their uses
Fixes various subdword RA issues

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4492>
2020-04-10 07:19:27 +00:00
Daniel Schürmann
331794495e aco: rename aco_lower_bool_phis() -> aco_lower_phis()
We also lower subdword phis, now.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4492>
2020-04-10 07:19:27 +00:00
Daniel Schürmann
1d41521b16 aco: lower subdword phis with SGPR operands
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4492>
2020-04-10 07:19:27 +00:00
Daniel Schürmann
a39df3bfce aco: don't constant-propagate into subdword PSEUDO instructions
PSEUDO instructions are lowered using SDWA, and thus,
cannot take literals and before GFX9 cannot take constants
at all. As the in-register representation differs between
32bit and 16bit floats, we first need to ensure correct
behavior.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4492>
2020-04-10 07:19:27 +00:00
Daniel Schürmann
1de18708cb aco: ensure correct bit representation of subdword constants
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4492>
2020-04-10 07:19:27 +00:00
Daniel Schürmann
637f45f390 aco: setup subdword regclasses for ssa_undef & load_const
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4492>
2020-04-10 07:19:27 +00:00
Samuel Pitoiset
67b567d0d0 aco: implement nir_op_b2f16/nir_op_i2f16/nir_op_u2f16
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
3119f978e5 aco: implement 16-bit comparisons
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
ccf8e23f59 aco: implement 16-bit nir_op_fmax3/nir_op_fmin3/nir_op_fmed3
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
981ced07a5 aco: implement 16-bit nir_op_ldexp
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
55537ed9d3 aco: implement 16-bit nir_op_f2i32/nir_op_f2u32
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
68339ff7a7 aco: implement 16-bit nir_op_bcsel
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
0646562a17 aco: implement 16-bit nir_op_fsign
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
6793ae1c5e aco: implement 16-bit nir_op_fsat
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
0ecca65d11 aco: implement 16-bit nir_op_fmul
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
b0c60999bc aco: implement 16-bit nir_op_fcos/nir_op_fsin
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
9be4be515f aco: implement 16-bit nir_op_fsub/nir_op_fadd
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
b0b637ca17 aco: implement 16-bit nir_op_fabs/nir_op_fneg
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
acc5912786 aco: implement 16-bit nir_op_fmax/nir_op_fmin
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
66d5bfb09a aco: implement 16-bit nir_op_ffloor/nir_op_fceil
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
c097c9f20c aco: implement 16-bit nir_op_fsqrt/nir_op_frcp/nir_op_frsq
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
26ed9fb79e aco: implement 16-bit nir_op_ftrunc/nir_op_fround_even
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
ee96181ad9 aco: implement 16-bit nir_op_fexp2/nir_op_flog2
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:05 +02:00
Samuel Pitoiset
b8486041df aco: implement 16-bit nir_op_ffract
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:04 +02:00
Samuel Pitoiset
a8b45d7034 aco: implement 16-bit nir_op_frexp_sig/nir_op_frexp_exp
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4452>
2020-04-10 08:05:04 +02:00
Daniel Schürmann
d22e2b3bd0 aco: RA - move all std::function objects into proper functions
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Daniel Schürmann
5351fee56a aco: move all needed helper containers to ra_ctx
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Daniel Schürmann
2ae27b96ef aco: change live_out variables to std::unordered_set
Improves performance of live_var_analysis for larger shaders

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Daniel Schürmann
acc10a7e51 aco: change some std::map to std::unordered_map in register_allocation
This improves compile times slightly for larger shaders

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Daniel Schürmann
69b6069dd2 aco: refactor try_remove_trivial_phi() in RA
Minor refactoring to avoid some pointer chasing.
This patch also changes the live_out argument to be
passed by reference to avoid an unnecessary copy.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Daniel Schürmann
b66f474121 aco: improve speed of live_var_analysis
by merging live_sgprs and live_vgprs sets.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Daniel Schürmann
09850e0a94 aco: during RA only insert into renames table if a variable got renamed
This improves the speed of register allocation.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Daniel Schürmann
48a74b6815 aco: replace assignment hashmap by std::vector in register allocation
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Daniel Schürmann
ba482c2e5f aco: improve register assignment when live-range splits are necessary
When finding a good place for a register, we can ignore
killed operands.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Daniel Schürmann
fb5a7902f2 aco: improve hashing for value numbering
An improved hashing greatly reduces the number of collisions,
and thus, increases the speed for lookups in the hash table.
The hash function now uses Murmur3 written by Austin Appleby.

This patch also pre-reserves space for the hashmap to avoid rehashing.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Daniel Schürmann
c99107ece0 aco: add explicit padding for all Instruction sub-structs
This patch also adds static_asserts on the size of Instructions
to ensure no internal padding is present.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Daniel Schürmann
7f962a9362 aco: guarantee that Temp fits in 4 bytes
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4130>
2020-04-09 15:08:57 +00:00
Samuel Pitoiset
2d8453e6e6 radv: allow TC-compat HTILE with GENERAL outside of render loops
This gives +8% with Wolfeinstein Youngblood on my Vega64, and
according to someone else, it also improves performance with Doom
2016 and Wolfenstein 2 (and probably other ID Tech games).

This improvement is because Youngblood uses GENERAL for the main
depth-only pass and TC-compat HTILE is now enabled with GENERAL if
we know that we are outside of a render loop. This obviously also
reduces the number of HTILE decompressions from/to GENERAL.

Note that Youngblood violates the Vulkan spec regarding render loops
because they are only allowed with input attachments. Expect possible
rendering issues if apps use render loops with the wrong way (ie.
without input attachmens) because HTILE might not be coherent if
a depth-stencil texture is sampled and rendered in the same draw.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2704
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4391>
2020-04-09 12:10:37 +00:00
Samuel Pitoiset
4de84c8cbd radv: only enable TC-compat HTILE for images readable by a shader
If no texture fetches happen it's useless to enable TC-compat HTILE.

Because the driver currently doesn't support TC-compat HTILE for
storage images we don't have to check.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4497>
2020-04-09 11:55:59 +00:00
Samuel Pitoiset
63f07a3047 radv: only expose fp16 control features for chips with double rate fp16
This disables all fp16 shader control features on GFX8 because only
GFX9+ supports double rate packed math.

This improves consistency regarding other AMD Vulkan drivers.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4453>
2020-04-09 13:35:08 +02:00
Samuel Pitoiset
1e4bd1de98 radv: only expose storageInputOutput16 for chips with double rate fp16
This feature allows to use both 16-bit integers and 16-bit floats
as inputs/outputs.

This disables storageInputOutput16 on GFX8 because only GFX9+ supports
double rate packed math.

This improves consistency regarding other AMD Vulkan drivers.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4453>
2020-04-09 13:35:08 +02:00
Samuel Pitoiset
1d74c6565d radv: only expose shaderFloat16 for chips with double rate fp16
This disables shaderFloat16 on GFX8 because only GFX9+ supports
double rate packed math.

This improves consistency regarding other AMD Vulkan drivers and
it makes no sense to enable that feature without packed math.

This also reduces performance with Wolfeinstein Youngblood if
fp16 is forced enabled on GFX8, while it's similar on GFX9.

We might re-introduce that feature in the future with ACO support
if it ends up being faster and correct.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4453>
2020-04-09 13:34:36 +02:00
Samuel Pitoiset
a3113e07b9 ac,radv: add ac_gpu_info::has_double_rate_fp16
Only GFX9+ support double rate packed math instructions.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4453>
2020-04-09 13:30:54 +02:00
Bas Nieuwenhuizen
a7e2efa7c9 radv: Use correct buffer count with variable descriptor set sizes.
Fixes dEQP-VK.binding_model.descriptorset_random.sets16.noarray.ubolimitlow.sbolimitlow.imglimitlow.iublimitlow.frag.ialimitlow.0

CC: <mesa-stable@lists.freedesktop.org>
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/2607
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4489>
2020-04-08 15:26:50 +00:00
Bas Nieuwenhuizen
bb7e44a23d radv: Whitespace fixup.
Review comment that I did, but forgot to git add before amending ...

From https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4334

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4488>
2020-04-08 11:18:18 +00:00
Samuel Iglesias Gonsálvez
8b42d26132 radv: set sparseAddressSpaceSize to RADV_MAX_MEMORY_ALLOCATION_SIZE
Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4487>
2020-04-08 11:03:35 +00:00
Samuel Iglesias Gonsálvez
cc678c9ce9 radv: check buffer size in vkCreateBuffer()
Fixes:

   dEQP-VK.api.buffer.basic.size_max_uint64

Signed-off-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4487>
2020-04-08 11:03:35 +00:00
Bas Nieuwenhuizen
a3682670c8 radv: Consider maximum sample distances for entire grid.
The other pixels in the grid might have samples with a larger
distance than the (0,0) pixel.

Fixes dEQP-VK.pipeline.multisample.sample_locations_ext.verify_location.samples_8_packed
when CTS is compiled with clang.

CC: <mesa-stable@lists.freedesktop.org>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4480>
2020-04-08 10:53:33 +00:00
Samuel Pitoiset
9f005f1f85 radv: enable lowering of GS intrinsics for the LLVM backend
This replaces emit_vertex with:
   if (vertex_count < max_vertices) {
      emit_vertex_with_counter vertex_count ...
      vertex_count += 1
   }

Which is exactly what NIR->LLVM was doing but at NIR level. This
pass is already called by ACO.

pipeline-db changes on GFX10:
Totals from affected shaders:
SGPRS: 1952 -> 1912 (-2.05 %)
VGPRS: 2112 -> 2044 (-3.22 %)
Code Size: 189368 -> 185620 (-1.98 %) bytes
Max Waves: 494 -> 491 (-0.61 %)

No pipeline-db changes on other generations.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4182>
2020-04-08 08:24:05 +02:00