Commit graph

677 commits

Author SHA1 Message Date
Marek Olšák
c9ea9e96a7 ac/llvm: simplify extracting an element in get_image_coords
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28607>
2024-04-12 22:20:14 -04:00
Rhys Perry
35f9318cee ac/llvm: implement mqsad_4x8 and shfr
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26251>
2024-04-05 11:01:39 +00:00
Timur Kristóf
e68ab8651e ac/llvm, radeonsi: Handle tess_rel_patch_id in common code.
We'll need to clean this up later, but for now it's better to
have it in common code than in RadeonSI.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28425>
2024-03-30 21:56:20 +01:00
Marek Olšák
a60b9eb17c ac/llvm: remove remnants of gfx10 NGG streamout
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27952>
2024-03-22 21:58:02 +00:00
Marek Olšák
6773595ed0 nir: rename AMD XFB intrinsics to *_gfx11_amd
to indicate it's only for gfx11.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27952>
2024-03-22 21:58:02 +00:00
Patrick Lerda
0fd907fc7b ac/llvm,radeonsi: fix memory leaks triggered by ac_nir_translate() errors
For instance, this issue is triggered with
"piglit/bin/glslparsertest tests/spec/arb_bindless_texture/compiler/images/arith-bound-image.frag pass 3.30 GL_ARB_bindless_texture GL_ARB_shader_image_load_store":
Direct leak of 176 byte(s) in 1 object(s) allocated from:
    #0 0x7f84c3fbe9a7 in calloc (/usr/lib64/libasan.so.6+0xb19a7)
    #1 0x7f84ba7e0801 in ac_nir_translate ../src/amd/llvm/ac_nir_to_llvm.c:4391
    #2 0x7f84ba53fdf4 in si_llvm_translate_nir ../src/gallium/drivers/radeonsi/si_shader_llvm.c:759
    #3 0x7f84ba542bb7 in si_llvm_compile_shader ../src/gallium/drivers/radeonsi/si_shader_llvm.c:836
    #4 0x7f84ba337b8e in si_compile_shader ../src/gallium/drivers/radeonsi/si_shader.c:2874
    #5 0x7f84ba43a7c1 in si_init_shader_selector_async ../src/gallium/drivers/radeonsi/si_state_shaders.cpp:3176
    #6 0x7f84b81c3448 in util_queue_thread_func ../src/util/u_queue.c:309
    #7 0x7f84b821ea6a in impl_thrd_routine ../src/c11/impl/threads_posix.c:67
    #8 0x7f84c2fea38a  (/lib64/libc.so.6+0x8438a)

Direct leak of 136 byte(s) in 1 object(s) allocated from:
    #0 0x7f84c3fbff57 in operator new(unsigned long) (/usr/lib64/libasan.so.6+0xb2f57)
    #1 0x7f84b1a5f749 in LLVMCreateBuilderInContext (/usr/local/lib64/libLLVM-17.so+0xc84749)
    #2 0x7f84ba7817b0 in ac_llvm_context_init ../src/amd/llvm/ac_llvm_build.c:54
    #3 0x7f84ba542b7a in si_llvm_context_init ../src/gallium/drivers/radeonsi/si_shader_llvm.c:120
    #4 0x7f84ba542b7a in si_llvm_compile_shader ../src/gallium/drivers/radeonsi/si_shader_llvm.c:832
    #5 0x7f84ba337b8e in si_compile_shader ../src/gallium/drivers/radeonsi/si_shader.c:2874
    #6 0x7f84ba43a7c1 in si_init_shader_selector_async ../src/gallium/drivers/radeonsi/si_state_shaders.cpp:3176
    #7 0x7f84b81c3448 in util_queue_thread_func ../src/util/u_queue.c:309
    #8 0x7f84b821ea6a in impl_thrd_routine ../src/c11/impl/threads_posix.c:67
    #9 0x7f84c2fea38a  (/lib64/libc.so.6+0x8438a)

Indirect leak of 176 byte(s) in 1 object(s) allocated from:
    #0 0x7f84c3fbe7ef in __interceptor_malloc (/usr/lib64/libasan.so.6+0xb17ef)
    #1 0x7f84b81b9b3f in ralloc_size ../src/util/ralloc.c:118
    #2 0x7f84b81b9fee in rzalloc_size ../src/util/ralloc.c:152
    #3 0x7f84b81b9fee in rzalloc_array_size ../src/util/ralloc.c:232
    #4 0x7f84b81b05c7 in _mesa_hash_table_init ../src/util/hash_table.c:163
    #5 0x7f84b81b05c7 in _mesa_hash_table_create ../src/util/hash_table.c:186
    #6 0x7f84ba7e06ae in ac_nir_translate ../src/amd/llvm/ac_nir_to_llvm.c:4381
    #7 0x7f84ba53fdf4 in si_llvm_translate_nir ../src/gallium/drivers/radeonsi/si_shader_llvm.c:759
    #8 0x7f84ba542bb7 in si_llvm_compile_shader ../src/gallium/drivers/radeonsi/si_shader_llvm.c:836
    #9 0x7f84ba337b8e in si_compile_shader ../src/gallium/drivers/radeonsi/si_shader.c:2874
    #10 0x7f84ba43a7c1 in si_init_shader_selector_async ../src/gallium/drivers/radeonsi/si_state_shaders.cpp:3176
    #11 0x7f84b81c3448 in util_queue_thread_func ../src/util/u_queue.c:309
    #12 0x7f84b821ea6a in impl_thrd_routine ../src/c11/impl/threads_posix.c:67
    #13 0x7f84c2fea38a  (/lib64/libc.so.6+0x8438a)

Indirect leak of 176 byte(s) in 1 object(s) allocated from:
    #0 0x7f84c3fbe7ef in __interceptor_malloc (/usr/lib64/libasan.so.6+0xb17ef)
    #1 0x7f84b81b9b3f in ralloc_size ../src/util/ralloc.c:118
    #2 0x7f84b81b9fee in rzalloc_size ../src/util/ralloc.c:152
    #3 0x7f84b81b9fee in rzalloc_array_size ../src/util/ralloc.c:232
    #4 0x7f84b81b05c7 in _mesa_hash_table_init ../src/util/hash_table.c:163
    #5 0x7f84b81b05c7 in _mesa_hash_table_create ../src/util/hash_table.c:186
    #6 0x7f84ba7e06e4 in ac_nir_translate ../src/amd/llvm/ac_nir_to_llvm.c:4382
    #7 0x7f84ba53fdf4 in si_llvm_translate_nir ../src/gallium/drivers/radeonsi/si_shader_llvm.c:759
    #8 0x7f84ba542bb7 in si_llvm_compile_shader ../src/gallium/drivers/radeonsi/si_shader_llvm.c:836
    #9 0x7f84ba337b8e in si_compile_shader ../src/gallium/drivers/radeonsi/si_shader.c:2874
    #10 0x7f84ba43a7c1 in si_init_shader_selector_async ../src/gallium/drivers/radeonsi/si_state_shaders.cpp:3176
    #11 0x7f84b81c3448 in util_queue_thread_func ../src/util/u_queue.c:309
    #12 0x7f84b821ea6a in impl_thrd_routine ../src/c11/impl/threads_posix.c:67
    #13 0x7f84c2fea38a  (/lib64/libc.so.6+0x8438a)

Indirect leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x7f84c3fbe7ef in __interceptor_malloc (/usr/lib64/libasan.so.6+0xb17ef)
    #1 0x7f84b81b9b3f in ralloc_size ../src/util/ralloc.c:118
    #2 0x7f84b81b046c in _mesa_hash_table_create ../src/util/hash_table.c:182
    #3 0x7f84ba7e06e4 in ac_nir_translate ../src/amd/llvm/ac_nir_to_llvm.c:4382
    #4 0x7f84ba53fdf4 in si_llvm_translate_nir ../src/gallium/drivers/radeonsi/si_shader_llvm.c:759
    #5 0x7f84ba542bb7 in si_llvm_compile_shader ../src/gallium/drivers/radeonsi/si_shader_llvm.c:836
    #6 0x7f84ba337b8e in si_compile_shader ../src/gallium/drivers/radeonsi/si_shader.c:2874
    #7 0x7f84ba43a7c1 in si_init_shader_selector_async ../src/gallium/drivers/radeonsi/si_state_shaders.cpp:3176
    #8 0x7f84b81c3448 in util_queue_thread_func ../src/util/u_queue.c:309
    #9 0x7f84b821ea6a in impl_thrd_routine ../src/c11/impl/threads_posix.c:67
    #10 0x7f84c2fea38a  (/lib64/libc.so.6+0x8438a)

Indirect leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x7f84c3fbe7ef in __interceptor_malloc (/usr/lib64/libasan.so.6+0xb17ef)
    #1 0x7f84b81b9b3f in ralloc_size ../src/util/ralloc.c:118
    #2 0x7f84b81b046c in _mesa_hash_table_create ../src/util/hash_table.c:182
    #3 0x7f84ba7e06ae in ac_nir_translate ../src/amd/llvm/ac_nir_to_llvm.c:4381
    #4 0x7f84ba53fdf4 in si_llvm_translate_nir ../src/gallium/drivers/radeonsi/si_shader_llvm.c:759
    #5 0x7f84ba542bb7 in si_llvm_compile_shader ../src/gallium/drivers/radeonsi/si_shader_llvm.c:836
    #6 0x7f84ba337b8e in si_compile_shader ../src/gallium/drivers/radeonsi/si_shader.c:2874
    #7 0x7f84ba43a7c1 in si_init_shader_selector_async ../src/gallium/drivers/radeonsi/si_state_shaders.cpp:3176
    #8 0x7f84b81c3448 in util_queue_thread_func ../src/util/u_queue.c:309
    #9 0x7f84b821ea6a in impl_thrd_routine ../src/c11/impl/threads_posix.c:67
    #10 0x7f84c2fea38a  (/lib64/libc.so.6+0x8438a)

SUMMARY: AddressSanitizer: 920 byte(s) leaked in 6 allocation(s).

Fixes: d92d35c9db ("ac/llvm: add a return value to ac_nir_translate")
Signed-off-by: Patrick Lerda <patrick9876@free.fr>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28099>
2024-03-21 00:51:19 +00:00
Marek Olšák
e589833ee1 ac/llvm: fix SSBO bounds checking by using raw instead of struct opcodes
Setting vindex != NULL (even if it's 0) selects a struct.buffer.load opcode,
which causes LLVM to look for "index * stride + offset" in voffset and
moves "index" to vindex (i.e. not 0 anymore), but the bounds checking
(OOB_SELECT) is set to ignore vindex. Setting vindex = NULL selects
a raw.buffer.load opcode.

Fixes: 6b573c00c9 - ac/nir: use ac_build_buffer_load() for SSBO load operations
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10794

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28119>
2024-03-12 23:00:00 +00:00
Georg Lehmann
37a15ba53a ac/llvm: implement as_uniform and ballot_relaxed
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27116>
2024-01-19 20:13:34 +00:00
Konstantin Seurer
220c912080 ac/llvm: Enable helper invocations for quad OPs
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/9239
cc: mesa-stable

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27110>
2024-01-18 21:00:06 +00:00
Rhys Perry
ae54cbeb3f nir: remove sad_u8x4
All uses of this can be replaced with msad_4x8.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
2024-01-05 18:55:22 +00:00
Rhys Perry
a339699b5c ac/llvm: implement msad_4x8
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
2024-01-05 18:55:22 +00:00
Marek Olšák
5cfd659276 ac/llvm: remove code for converting txd from 1D to 2D because NIR does it
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26547>
2023-12-13 16:26:17 +00:00
Kai Wasserbäch
bc7e363f8e fix: ac/llvm: LLVM 18: remove useless passes, partially removed upstream
Upstream removed llvm::createLoopSinkPass() in commit
<b9975cec0e>
and there is no useful alternative except moving to the new pass
manager.

On top of that, the usage of this optimisation pass and
PromoteMemoryToRegisterPass were just useless, according to the
upstream developer of the commit named above. Therefore the easiest
solution is, as him, Marek and Dave suggested, to just remove these two
passes from the pipeline for now.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10192
Reference: https://github.com/llvm/llvm-project/pull/72811
Reference: b9975cec0e
Suggested-by: Dave Airlie <airlied@redhat.com>
Suggested-by: Aiden Grossman <agrossman154@yahoo.com>
Suggested-by: Marek Olšák <maraeo@gmail.com>
Signed-off-by: Kai Wasserbäch <kai@dev.carbon-project.org>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26336>
2023-11-24 14:11:09 +00:00
Qiang Yu
dbbf566588 aco,ac/llvm,radeonsi: lower f2f16 to f2f16_rtz in nir
No need to handle f2f16 specially for OpenGL, and we can vectorize
f2f16 when using ACO.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25990>
2023-11-20 02:20:17 +00:00
Qiang Yu
5696790420 ac/llvm: remove nir_op_*2*mp ops handling
They have been lowered to *2*16 in nir.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25990>
2023-11-20 02:20:17 +00:00
Connor Abbott
387e698bde amd: Implement quad_vote intrinsics
Co-authored-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/218>
2023-11-17 09:45:40 +00:00
Marek Olšák
e7c8d32bfc ac/gpu_info,llvm: trivial cosmetic changes
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26095>
2023-11-07 19:27:43 +00:00
Samuel Pitoiset
d390cd7c5d ac/nir: remove dead code in nir_intrinsic_xfb_counter_{add,sub}_amd
This code path is only used by GFX11 now.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25903>
2023-10-30 08:51:51 +00:00
Samuel Pitoiset
f0abdaea9f amd/llvm,aco,radv: implement NGG streamout with GDS_STRMOUT registers on GFX11
According to RadeonSI, this is required for preemption, user queues,
and we only have to wait for VS after streamout which should be more
performant.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25284>
2023-10-12 16:58:22 +00:00
Marek Olšák
c7e08acd12 ac/llvm: fix flat PS input corruption
Fixes: 0a54fbb5b4 - radeonsi/gfx11: interp changes for 32bit

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25492>
2023-09-30 23:08:47 +00:00
Marek Olšák
2d928c7fee ac/llvm: set !fpmath 3.0 for llvm.sqrt
LLVM will be changed to increase precision with additional instructions,
which we don't want.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25492>
2023-09-30 23:08:46 +00:00
Marek Olšák
8e97d291a8 ac/llvm: replace removed amdgcn.ldexp for LLVM 18
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25393>
2023-09-28 16:51:33 +00:00
Marek Olšák
6060b656f8 gallivm: fix build with LLVM 18
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25388>
2023-09-26 05:45:17 +00:00
Caio Oliveira
3988d901ac meson: Remove unnecessary inc_compiler mentions
The inc_compiler should come as part of idep_compiler, idep_nir or
idep_nir_headers dependency.

Acked-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com> (v3dv)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25314>
2023-09-22 14:52:50 +00:00
Connor Abbott
c93bcb32fe amd: Use inverse ballot intrinsic if available
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25123>
2023-09-20 14:41:18 +00:00
Konstantin Seurer
4c168635f8 ac/llvm: Use float types for float atomics
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25010>
2023-09-14 15:39:39 +00:00
Konstantin Seurer
60e7b1c69c ac/llvm: Use the correct return type for uadd_carry and usub_borrow
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25010>
2023-09-14 15:39:39 +00:00
Konstantin Seurer
3ae0562c23 ac/llvm: Fix typed loads with 16bit formats
For some reason, LLVM can't handle those. Emit a 32bit load and type
conversion instead,

Fixes: 22ca8c8 ("ac/llvm: Implement typed buffer load intrinsic.")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25010>
2023-09-14 15:39:38 +00:00
Marek Olšák
3040aa2e26 ac/llvm: don't convert undef to 0 because nir_opt_undef does it now
TOTALS FROM AFFECTED SHADERS (29663/58918)
  Code Size: 39163724 -> 37842360 (-3.37 %) bytes
  Max Waves: 394813 -> 396334 (0.39 %)
  Outputs: 84616 -> 84616 (0.00 %)
  Patch Outputs: 0 -> 0 (0.00 %)

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24059>
2023-09-06 03:24:16 +00:00
Karol Herbst
a19f98a134 ac: drop 64 bit handling for cl workgroup intrinsics
Signed-off-by: Karol Herbst <git@karolherbst.de>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24905>
2023-08-30 07:04:33 +00:00
Marek Olšák
a2bf30961f ac: implement AMD_FORCE_FAMILY properly, remove SI_FORCE_FAMILY
This sets radeon_info for the forced family correctly.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24759>
2023-08-19 19:36:55 +00:00
Faith Ekstrand
65b6ac8aa4 nir: Rename nir_instr_type_ssa_undef to nir_instr_type_undef
We already renamed the type, we just need to rename the enum and the
casting helper functions.

Generated with sed:

    sed -i -e 's/nir_instr_type_ssa_undef/nir_instr_type_undef/g' src/**/*.h src/**/*.c src/**/*.cpp
    sed -i -e 's/nir_instr_as_ssa_undef/nir_instr_as_undef/g' src/**/*.h src/**/*.c src/**/*.cpp

and two tiny whitespace fixups in lima.

Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24703>
2023-08-15 17:44:27 +00:00
Faith Ekstrand
4695bebc79 nir: Drop nir_dest
Instead, we replace every use of it with nir_def.  Most of this commit
was generated by sed:

   sed -i -e 's/dest.ssa/def/g' src/**/*.h src/**/*.c src/**/*.cpp

A few manual fixups were required in lima and the nir_legacy code.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 21:22:53 +00:00
Faith Ekstrand
6c1d32581a nir: Drop nir_alu_dest
Instead, we replace it directly with nir_def.  We could replace it with
nir_dest but the next commit gets rid of that so this avoids unnecessary
churn.  Most of this commit was generated by sed:

   sed -i -e 's/dest.dest.ssa/def/g' src/**/*.h src/**/*.c src/**/*.cpp

There were a few manual fixups required in the nir_legacy.c and
nir_from_ssa.c as nir_legacy_reg and nir_parallel_copy_entry both have a
similar pattern.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 21:22:53 +00:00
Faith Ekstrand
80a1836d8b nir: Get rid of nir_dest_bit_size()
We could add a nir_def_bit_size() helper but we use ->bit_size about 3x
as often as nir_dest_bit_size() today so that's a major Coccinelle
refactor anyway and this doesn't make it much worse.  Most of this
commit was generated byt the following semantic patch:

    @@
    expression D;
    @@

    <...
    -nir_dest_bit_size(D)
    +D.ssa.bit_size
    ...

Some manual fixup was needed, especially in cpp files where Coccinelle
tends to give up the moment it sees any interesting C++.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 21:22:53 +00:00
Alyssa Rosenzweig
09d31922de nir: Drop "SSA" from NIR language
Everything is SSA now.

   sed -e 's/nir_ssa_def/nir_def/g' \
       -e 's/nir_ssa_undef/nir_undef/g' \
       -e 's/nir_ssa_scalar/nir_scalar/g' \
       -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \
       -e 's/nir_gather_ssa_types/nir_gather_types/g' \
       -i $(git grep -l nir | grep -v relnotes)

   git mv src/compiler/nir/nir_gather_ssa_types.c \
          src/compiler/nir/nir_gather_types.c

   ninja -C build/ clang-format
   cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \;

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Acked-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:44:41 -04:00
Alyssa Rosenzweig
a8013644a1 nir: Drop nir_alu_src::{negate,abs}
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:28 +00:00
Alyssa Rosenzweig
ab0d878932 treewide: Remove more is_ssa asserts
Stuff Coccinelle missed.

   sed -i -e '/assert(.*\.is_ssa)/d' $(git grep -l is_ssa)
   sed -i -e '/ASSERT.*\.is_ssa)/d' $(git grep -l is_ssa)

+ a manual fixup to restore the assert for parallel copy lowering.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:28 +00:00
Alyssa Rosenzweig
5fead24365 treewide: Drop is_ssa asserts
We only see SSA now.

Via Coccinelle patch:

    @@
    expression x;
    @@

    -assert(x.is_ssa);

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-03 22:40:28 +00:00
Yonggang Luo
cd3ea02da0 ac/radv: decouple radv vulkan driver and compiler from gallium
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24464>
2023-08-03 09:45:42 +00:00
Alyssa Rosenzweig
51db19f7a2 nir: Rename scoped_barrier -> barrier
sed + ninja clang-format + fix up spacing for common code.

If you are unhappy that I did not manually change the whitespace of your driver,
you need to enable clang-format for it so the formatting would happen
automatically.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24428>
2023-08-01 23:18:29 +00:00
Rhys Perry
0cb26f31fe ac/llvm: skip ballot zext for 32-bit dest with wave32-as-wave64
NGG lowering can ballots with a 32-bit destination even if the ballot bit
size is 64.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24140>
2023-07-17 11:21:35 +00:00
Rhys Perry
47c9dea6ac ac/llvm: fix wave32 ac_build_mbcnt_add with 64-bit mask
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Cc: mesa-stable
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24140>
2023-07-17 11:21:35 +00:00
Rhys Perry
85d1159a2a ac/llvm: fix AC_TM_CHECK_IR
This was using the wrong pass.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Fixes: 3f272fd15e ("ac/llvm: fix build with LLVM 17")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24140>
2023-07-17 11:21:34 +00:00
Timur Kristóf
7ad9416c61 ac/llvm: Remove subgroup_id and num_subgroups intrinsics.
We expect that these will be lowered in NIR now.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24005>
2023-07-14 12:17:24 +00:00
Rhys Perry
25c49e491f aco,ac/llvm,ac/nir,vtn: unify cube opcodes
fossil-db (navi21):
Totals from 17068 (12.79% of 133461) affected shaders:
Instrs: 24743703 -> 24743572 (-0.00%); split: -0.00%, +0.00%
CodeSize: 132579952 -> 132580620 (+0.00%); split: -0.00%, +0.00%
VGPRs: 1227840 -> 1227984 (+0.01%)
Latency: 403180114 -> 403251188 (+0.02%); split: -0.00%, +0.02%
InvThroughput: 75311302 -> 75320892 (+0.01%); split: -0.00%, +0.01%
VClause: 415400 -> 415402 (+0.00%); split: -0.00%, +0.00%
Copies: 1715404 -> 1715258 (-0.01%); split: -0.01%, +0.01%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Reviewed-by: Gert Wollny <gert.wollny@collabora.com> (r600)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23930>
2023-06-30 15:35:03 +00:00
Caio Oliveira
59cc77f0fa compiler: Move from nir_scope to mesa_scope
Just moving the enum and performing renames, no behavior change.

Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23328>
2023-06-19 23:29:26 +00:00
Alyssa Rosenzweig
d8a45db591 ac/llvm: Drop memory_barrier_buffer impl
Both radeonsi and radv use scoped barriers, so this should not be possible to
hit.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23191>
2023-06-13 16:36:10 +00:00
Qiang Yu
5f52f8a6ba ac/llvm,radeonsi: lower nir_load_user_data_amd in abi
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23540>
2023-06-13 03:41:02 +00:00
Karol Herbst
31fb75a08b ac/llvm: set +cumode for radeonsi
radeonsi switched over to CU wavefront execution mode, but didn't tell
LLVM. This can lead to shaders requiring too many VGPRs to be executed in
CU mode and so cause GPU resets.

Pass along +cumode to LLVM so it properly spills VGPRs.

Fixes: 9d7eab2ab1 ("radeonsi: don't enable WGP_MODE because of high cost of workgroup mem coherency")
Signed-off-by: Karol Herbst <git@karolherbst.de>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23569>
2023-06-11 08:25:45 +00:00