Commit graph

276 commits

Author SHA1 Message Date
Samuel Pitoiset
1947500208 aco: remove nir_intrinsic_load_barycentric_at_sample occurences
This is lowered earlier and shouldn't get there.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23307>
2023-05-31 07:25:46 +00:00
Rhys Perry
2447156b3c aco: implement strict_wqm_coord_amd
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22636>
2023-05-25 16:29:16 +00:00
Alyssa Rosenzweig
ce638eafe2 aco,radv: Use unified atomics
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22914>
2023-05-12 20:39:46 +00:00
Rhys Perry
5db64fcc8c aco: use apply_nuw_to_ssa() with load_smem_amd
fossil-db (navi21):
Totals from 107 (0.08% of 135636) affected shaders:
Instrs: 389667 -> 389425 (-0.06%); split: -0.06%, +0.00%
CodeSize: 2050380 -> 2049440 (-0.05%); split: -0.05%, +0.00%
Latency: 3738053 -> 3737313 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 599889 -> 599790 (-0.02%); split: -0.02%, +0.00%
SClause: 16120 -> 15920 (-1.24%)
Copies: 29823 -> 29809 (-0.05%); split: -0.07%, +0.02%
PreSGPRs: 6856 -> 6849 (-0.10%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22553>
2023-04-19 19:29:48 +00:00
Qiang Yu
9cd3aa173a aco: implement nir_bindless_image_atomic_inc/dec_wrap
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22525>
2023-04-19 08:39:46 +00:00
Timur Kristóf
80d8e8d828 aco: Remove setup_*_variables and add setup_lds_size instead.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21935>
2023-04-12 17:30:02 +00:00
Timur Kristóf
4ae46840cd ac, aco, radv: Clarify LDS size on GFX6, and NGG shaders.
This information was wrong in some places, let's fix it now.

GFX6:
The GPU has 64KB LDS, but only 32KB is usable by a workgroup.

NGG:
There was some misinformation about NGG only being able to
address 32 KB LDS, it turns out this is actually not true
and it can address the full 64K.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21935>
2023-04-12 17:30:02 +00:00
Daniel Schürmann
2fee99a36c aco: implement load_ray_launch_{id|size}
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:30 +00:00
Daniel Schürmann
c7c68e1193 aco: move rt_dynamic_callable_stack_base_amd to VGPR
In future, we will use a VGPR arg for that between RT stages.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:30 +00:00
Daniel Schürmann
a33b9d43d8 aco: add RT stage enums
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21780>
2023-03-16 01:40:29 +00:00
Timur Kristóf
6185e4f2ff aco, radv: Remove VS IO information from ACO.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>
2023-03-15 14:54:28 +00:00
Timur Kristóf
477cb943f6 aco: Implement load_typed_buffer_amd.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>
2023-03-15 14:54:27 +00:00
Timur Kristóf
87de5b2b9e aco: Don't include headers from radv.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21696>
2023-03-08 04:39:18 +00:00
Timur Kristóf
a0141c6308 aco, radv: Don't use radv_shader_args in aco.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21696>
2023-03-08 04:39:18 +00:00
Georg Lehmann
ee47cc8256 amd,nir: remove byte_permute_amd intrinsic
It's unused and if we ever want to use it again we should make it an alu
opcode instead.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21445>
2023-02-22 20:13:52 +00:00
Daniel Schürmann
2bb369dd8d nir: add assertions that loops don't have a Continue Construct
Hoping that I didn't miss any, this *should* add assertions
to all functions and passes which explicitly handle 'nir_loop'.

Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13962>
2023-02-21 10:41:11 +00:00
Qiang Yu
ed419f46aa aco: remove early_rast wait insert
It's done in nir position export.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20691>
2023-02-03 12:27:44 +00:00
Timur Kristóf
c644461b71 radv, aco, ac: Implement pack_half_2x16_rtz_split.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15838>
2023-01-26 12:24:24 +00:00
Rhys Perry
068c84f275 aco: add support for fp32 addition atomics
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19810>
2023-01-17 17:39:15 +00:00
Rhys Perry
810ced93f3 aco: align scratch size during assembly
This lets us use less scratch if both VGPR spilling and scratch intrinsics
are used.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20534>
2023-01-09 21:46:13 +00:00
Rhys Perry
c9846158cd aco/gfx11: reduce scratch allocation alignment
fossil-db (gfx1100):
Totals from 112 (0.08% of 134574) affected shaders:
Scratch: 1513472 -> 1455360 (-3.84%)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20534>
2023-01-09 21:46:13 +00:00
Rhys Perry
bd30adf89d aco: apply NUW to additions for scratch access
fossil-db (navi21):
Totals from 52 (0.04% of 135636) affected shaders:
Instrs: 79036 -> 78567 (-0.59%)
CodeSize: 431188 -> 427984 (-0.74%)
Latency: 1318142 -> 1313821 (-0.33%)
InvThroughput: 293842 -> 292836 (-0.34%)
VClause: 2555 -> 2361 (-7.59%); split: -8.06%, +0.47%
Copies: 8746 -> 8767 (+0.24%); split: -0.11%, +0.35%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20117>
2022-12-06 15:23:38 +00:00
Rhys Perry
3a96977542 radv,aco: remove old GS copy shader code
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18898>
2022-11-29 14:28:11 +00:00
Rhys Perry
e6d26cb288 nir,ac/nir,aco,radv: replace has_input_*_amd with more general intrinsics
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19228>
2022-10-31 14:33:43 +00:00
Samuel Pitoiset
fa287c99ed aco: remove invalid assertions for NGG streamout
Streamout outputs are directly emitted from NIR now.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Samuel Pitoiset
db7ffa4006 aco: implement NIR intrinsics for NGG streamout
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19317>
2022-10-31 13:48:39 +00:00
Georg Lehmann
361b47b1f0 aco: Implement signed idot instructions on GFX11.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19114>
2022-10-24 19:07:16 +00:00
Samuel Pitoiset
df997cf47d aco: remove unused isel_context::tcs_num_patches
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18278>
2022-09-01 17:02:16 +00:00
Rhys Perry
c06a5a5ebd radv,aco: use pipe_format for static vertex input state
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17894>
2022-08-30 19:02:11 +00:00
Daniel Schürmann
3d6ea4f666 aco: use std::vector::reserve() more often
This removes the majority of vector re-allocations.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18105>
2022-08-30 16:03:26 +00:00
Samuel Pitoiset
ee5b9bcc57 radv: stop duplicating radv_vs_output_info
Only the last vertex stage needs to access this.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18210>
2022-08-26 14:07:09 +00:00
Georg Lehmann
b96126ee95 radv,aco: Don't lower and vectorize 16bit iabs.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17440>
2022-07-20 14:31:15 +00:00
Konstantin Seurer
83ccc810b4 aco: Remove dead nir_intrinsic_load_vertex_id case
This intrinsic is lowered in NIR.

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17539>
2022-07-19 13:26:09 +00:00
Samuel Pitoiset
270cc39648 aco: add support for compiling PS epilogs
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
2022-07-18 18:40:02 +00:00
Rhys Perry
48578713b7 radv,aco,ac/llvm: use nir_op_f{sin,cos}_amd
This lets NIR optimize the multiplication, particularly sin/cos(a * #b).

fossil-db (Sienna Cichlid):
Totals from 12306 (7.58% of 162293) affected shaders:
MaxWaves: 224814 -> 224834 (+0.01%)
Instrs: 17365273 -> 17338758 (-0.15%); split: -0.16%, +0.00%
CodeSize: 93478488 -> 93354912 (-0.13%); split: -0.14%, +0.01%
VGPRs: 752080 -> 752072 (-0.00%); split: -0.00%, +0.00%
SpillSGPRs: 8440 -> 8410 (-0.36%)
Latency: 200402154 -> 200279405 (-0.06%); split: -0.06%, +0.00%
InvThroughput: 37588077 -> 37545545 (-0.11%); split: -0.11%, +0.00%
VClause: 293863 -> 293874 (+0.00%); split: -0.03%, +0.03%
SClause: 619539 -> 619064 (-0.08%); split: -0.09%, +0.01%
Copies: 1151591 -> 1151641 (+0.00%); split: -0.04%, +0.05%
Branches: 506434 -> 506437 (+0.00%); split: -0.00%, +0.00%
PreSGPRs: 877609 -> 877517 (-0.01%); split: -0.01%, +0.00%
PreVGPRs: 711938 -> 711940 (+0.00%); split: -0.00%, +0.00%

fossil-db (LLVM, Sienna Cichlid):
Totals from 4377 (3.59% of 121873) affected shaders:
SGPRs: 358960 -> 359176 (+0.06%); split: -0.18%, +0.25%
VGPRs: 319832 -> 319720 (-0.04%); split: -0.18%, +0.15%
SpillSGPRs: 46983 -> 47007 (+0.05%); split: -0.99%, +1.04%
CodeSize: 30872812 -> 30764512 (-0.35%); split: -0.39%, +0.04%
MaxWaves: 73814 -> 73904 (+0.12%); split: +0.25%, -0.13%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10587>
2022-07-07 22:18:08 +00:00
Konstantin Seurer
16585664cd radv: vkCmdTraceRaysIndirect2KHR
This changes the trace rays logic to always use
VkTraceRaysIndirectCommand2KHR and implements
vkCmdTraceRaysIndirect2KHR. I renamed the
load_sbt_amd to sbt_base_amd and moved the SBT
load lowering from ACO to NIR.

Note that we can not just upload one pointer to
all the trace parameters because that would
be incompatible with traceRaysIndirect.

Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16430>
2022-06-08 20:20:21 +00:00
Georg Lehmann
1d815548ab radv, aco: Packed usub_sat/isub_sat.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13895>
2022-06-01 17:09:25 +00:00
Samuel Pitoiset
0c8a07f25d aco: remove unnecessary intrinsics that are lowered at the ABI level
Fixes: f553076eaf ("aco: Remove now-superfluous intrinsics.")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16577>
2022-05-19 06:49:07 +00:00
Dave Airlie
a179e1aede aco/radv: drop radv_nir_compiler_options from aco.
Add a new aco input and options structs, then convert from radv
pieces on submit.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16521>
2022-05-17 06:15:25 +00:00
Marek Olšák
39800f0fa3 amd: change chip_class naming to "enum amd_gfx_level gfx_level"
This aligns the naming with PAL.

Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Pierre-Eric Pellou-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16469>
2022-05-13 14:56:22 -04:00
Konstantin Seurer
b30f96dd93 radv,aco: Use ray_launch_size_addr
Signed-off-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15712>
2022-05-12 15:04:31 +00:00
Dave Airlie
04c07a2413 aco/radv: convert to aco shader info at the radv level.
This removes the radv shader info type from aco completely.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>
2022-05-11 19:07:11 +00:00
Dave Airlie
8cfd8420ab aco: convert vs and so info over to aco structs.
This renames the vs to vp (vertex pipeline) on the way past.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>
2022-05-11 19:07:11 +00:00
Dave Airlie
9bd89af1bc aco/info: reduce the gs ring info to what is needed.
Only one member was being used.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>
2022-05-11 19:07:11 +00:00
Dave Airlie
a2701bfdb8 aco: move info pointer to a copy.
This is just setup to move this to a different struct later.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16342>
2022-05-11 19:07:11 +00:00
Rhys Perry
c883abda76 aco: implement load_shared2_amd/store_shared2_amd
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13778>
2022-04-13 23:08:07 +00:00
Rhys Perry
7478b00c7c aco: remove old global access intrinsics
No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14124>
2022-04-13 16:23:35 +00:00
Rhys Perry
3e9517c757 aco: implement _amd global access intrinsics
fossil-db (Sienna Cichlid):
Totals from 7 (0.01% of 134621) affected shaders:
VGPRs: 760 -> 776 (+2.11%)
CodeSize: 222000 -> 222044 (+0.02%); split: -0.01%, +0.03%
Instrs: 40959 -> 40987 (+0.07%); split: -0.01%, +0.08%
Latency: 874811 -> 886609 (+1.35%); split: -0.00%, +1.35%
InvThroughput: 437405 -> 443303 (+1.35%); split: -0.00%, +1.35%
VClause: 1242 -> 1240 (-0.16%)
SClause: 1050 -> 1049 (-0.10%); split: -0.19%, +0.10%
Copies: 4953 -> 4973 (+0.40%); split: -0.04%, +0.44%
Branches: 1947 -> 1957 (+0.51%); split: -0.05%, +0.56%
PreVGPRs: 741 -> 747 (+0.81%)

fossil-db changes seem to be noise.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14124>
2022-04-13 16:23:35 +00:00
Georg Lehmann
141ca78634 radv, aco: Packed iadd_sat/uadd_sat.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15421>
2022-03-28 20:02:52 +00:00
Georg Lehmann
50f585254c aco: Implement scalar iadd_sat.
Signed-off-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15421>
2022-03-28 20:02:52 +00:00