Timur Kristóf
5aa39253cb
nir: Rename nir_get_io_vertex_index_src and include per-primitive I/O.
...
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466 >
2021-11-16 07:46:55 +00:00
Marek Olšák
9151ac3531
ac,radeonsi: cull small lines in the shader using the diamond exit rule
...
It also splits clip_half_line_width into X and Y components for tighter
view culling.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700 >
2021-11-16 02:11:46 +00:00
Marek Olšák
f8a0aa6852
radeonsi: fix view culling for wide lines
...
We need to cull wide lines as quads, but only for view culling.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700 >
2021-11-16 02:11:45 +00:00
Marek Olšák
5ee2965283
ac/llvm: accept primitives whose face culling determinant is Inf or NaN
...
Based on https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13299/diffs
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13380 >
2021-10-19 12:49:06 +00:00
Marek Olšák
efaab0ec50
ac/llvm: add helper ac_build_is_inf_or_nan
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13380 >
2021-10-19 12:49:06 +00:00
Marek Olšák
5e8f76b713
ac/llvm: use fmac instead of mul+sub in face culling
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13380 >
2021-10-19 12:49:05 +00:00
Jason Ekstrand
956199e870
nir: s/nir_var_mem_image/nir_var_image/g
...
We typically use nir_var_mem_* for stuff that has an explicit byte-based
memory layout. Images are opaque.
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13386 >
2021-10-16 03:47:10 +00:00
Jason Ekstrand
cd49706cb1
amd/llvm/nir: Add support for nir_var_mem_image
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4743 >
2021-10-15 14:58:55 +00:00
Samuel Pitoiset
a6298b1bc9
radv: remove unnecessary radv_shader_info:num_inline_push_consts
...
This can be determined directly from the user SGPR loc.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13149 >
2021-10-08 11:37:19 +00:00
Rhys Perry
2564fbfc57
ac/llvm: fix image_samples with null descriptors
...
Fixes various dEQP-VK.robustness.robustness2.* tests.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13251 >
2021-10-08 10:27:11 +00:00
Rhys Perry
1fb63367a8
radv,aco: don't include FMASK in the storage descriptor
...
We perform a FMASK expand when transitioning to GENERAL or TRANSFER_DST
layout, so storage images always have an identity FMASK.
radeonsi doesn't appear to expand the FMASK for read-only storage images,
so the sample index adjustment is still needed there.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12214 >
2021-10-07 15:36:39 +00:00
Rhys Perry
91a5c9fdf2
ac/nir: return 0x76543210 for NULL FMASK fetch
...
This can replace several v_cndmask_b32 with a single v_cndmask_b32.
fossil-db (Sienna Cichlid):
Totals from 1169 (0.78% of 150170) affected shaders:
SGPRs: 82032 -> 81984 (-0.06%); split: -0.19%, +0.13%
VGPRs: 68368 -> 68280 (-0.13%); split: -0.30%, +0.18%
SpillSGPRs: 8754 -> 8609 (-1.66%); split: -4.06%, +2.40%
CodeSize: 5460132 -> 5446308 (-0.25%); split: -0.42%, +0.17%
MaxWaves: 24456 -> 24462 (+0.02%); split: +0.03%, -0.01%
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12214 >
2021-10-07 15:36:39 +00:00
Rhys Perry
225fe37c14
nir: add _amd suffix to fragment_mask_fetch and fragment_fetch texops
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12214 >
2021-10-07 15:36:39 +00:00
Marek Olšák
f00d3e2909
radeonsi: implement shader-based culling for lines
...
This helps some viewperf subtests.
Only view XY culling is done. Edgeflags are always disabled with lines.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13048 >
2021-09-28 17:30:06 +00:00
Marek Olšák
e7e0b90c94
ac/llvm: rename ac_cull_triangle -> ac_cull_primitive
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13048 >
2021-09-28 17:30:06 +00:00
Marek Olšák
25026076ab
ac/llvm: pass cull options into cull_bbox directly
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13048 >
2021-09-28 17:30:06 +00:00
Bas Nieuwenhuizen
8ca54b4d38
radv: Support nir_intrinsic_load_global_constant.
...
SPIR-V parsing can result in some direct constant usage for shader
records. Lower this early to a global based intrinsic so that it
doesn't interfere with the later 32-bit offset based constants
for scratch usage.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592 >
2021-09-21 01:53:39 +00:00
Samuel Pitoiset
7a25451752
ac/llvm: implement nir_intrinsic_global_atomic_{fmin,fmax}
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12716 >
2021-09-15 14:10:42 +00:00
Samuel Pitoiset
9deee80f51
ac/llvm: implement nir_intrinsic_shared_atomic_{fmin,fmax}
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12716 >
2021-09-15 14:10:42 +00:00
Samuel Pitoiset
efe40c98ed
ac/llvm: implement nir_intrinsic_ssbo_atomic_{fmin,fmax}
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12716 >
2021-09-15 14:10:42 +00:00
Samuel Pitoiset
cf3e31fd11
ac/llvm: implement nir_intrinsic_image_deref_atomic_{fmin,fmax}
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12716 >
2021-09-15 14:10:42 +00:00
Marek Olšák
b330c7cb2a
radeonsi: use a trick to extract and pack edgeflags using fewer instructions
...
This removes 4 instructions from the prim export packing.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12343 >
2021-09-14 15:24:11 +00:00
Samuel Pitoiset
09cf2584bd
ac/llvm: fix build with LLVM 14
...
AttributeList::hasAttribute() has been replaced by
hasParamAttr()/hasRetAttr().
All supported LLVM versions have hasParamAttr().
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5320
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12715 >
2021-09-13 10:04:30 +02:00
Marek Olšák
69e96cfc0d
ac,radv: remove unused inputs array and VS input code
...
The previous commit stopped using "inputs".
"load_layer_id" has always been broken and it was probably unused anyway.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570 >
2021-09-07 17:51:41 +00:00
Marek Olšák
3fb229e010
ac,radeonsi: load VS inputs at the call site of nir_intrinsic_load_input
...
to match ACO
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570 >
2021-09-07 17:51:41 +00:00
Marek Olšák
bce7c7f3fc
ac/llvm: implement nir_intrinsic_elect
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570 >
2021-09-07 17:51:41 +00:00
Marek Olšák
e0f07483d0
ac/llvm: implement nir_intrinsic_overwrite_*_arguments_amd
...
This should work if the intrinsics are not called from conditional blocks.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570 >
2021-09-07 17:51:41 +00:00
Marek Olšák
1e178f7a37
ac: make ac_shader_abi::inputs an array instead of a pointer
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570 >
2021-09-07 17:51:41 +00:00
Marek Olšák
6df5f268db
ac: remove needless parameters from ac_shader_abi::emit_outputs
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570 >
2021-09-07 17:51:41 +00:00
Marek Olšák
2e95ad1433
ac/llvm: implement a bunch of NIR AMD intrinsics for NGG
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570 >
2021-09-07 17:51:41 +00:00
Marek Olšák
a33602b1f9
ac/llvm: remove load_tess_coord callback
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570 >
2021-09-07 17:51:41 +00:00
Rhys Perry
859790ba54
ac/llvm: implement udot_4x8/sdot_4x8/udot_2x16/sdot_2x16 opcodes
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617 >
2021-09-03 13:21:27 +00:00
Rhys Perry
d6619d0a01
ac/llvm,radv: implement uadd_sat/iadd_sat
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617 >
2021-09-03 13:21:27 +00:00
Rhys Perry
f7cdd49a09
ac/llvm: implement nir_op_pack_32_4x8
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617 >
2021-09-03 13:21:27 +00:00
Samuel Pitoiset
72b6b02e09
ac/llvm: fix huge alignment when loading from shared memory
...
LLVM doesn't support huge alignments, also it can optimize the shared
loads, so it's unecessary to emit better (but broken) LLVM IR.
Fixes a bunch of crashes with RADV_DEBUG=llvm,checkir.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12616 >
2021-08-31 09:56:27 +02:00
Samuel Pitoiset
e72c2e36e0
ac/llvm: adjust assertion for nir_intrinsic_terminate
...
Fixes dEQP-VK.spirv_assembly.instruction.terminate*.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12616 >
2021-08-31 09:56:27 +02:00
Rhys Perry
0460f01fdc
ac/llvm: implement v2f16 fsat
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12143 >
2021-08-02 10:02:51 +00:00
Jason Ekstrand
a7b429e8ec
amd: Don't handle nir_tex_src_ms_mcs
...
It's an intel-specific texture source and will never be seen on AMD.
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11775 >
2021-07-23 15:53:57 +00:00
Jason Ekstrand
e83fe65cd8
radv,radeonsi: Do cube size divide-by-6 lowering in NIR
...
No point in carrying all this code around twice each in two back-ends.
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12005 >
2021-07-22 14:22:35 -05:00
Marek Olšák
d9173b3b46
ac/llvm: rework how negative W affects culling to not call accept_func twice
...
Always execute the bbox code regardless of negative W, and then simply
use || to discard the result if any W is negative. This is expected to be
rare. (it only happens when a primitive intersects the near plane)
This allows us to eliminate the else statement, which is no longer
executed for accepted primitives with negative W, which are the only
primitives that needed the else branch.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11754 >
2021-07-08 18:37:41 +00:00
Marek Olšák
74b33a8c37
ac/llvm: don't return a status from ac_cull_triangle because it's unused
...
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11754 >
2021-07-08 18:37:41 +00:00
Pierre-Eric Pelloux-Prayer
b564ec7769
ac/llvm: call the callback in all return paths of ac_cull_triangle
...
Fixes: 12d2df15f1 ("ac/llvm: add a callback to ac_cull_triangle to generate code in inner-most block")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4959
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11625 >
2021-06-29 23:38:21 +02:00
Marek Olšák
d3fddf63ec
ac/llvm: don't set skip-uniform-regions to fix atomic.cmpswap
...
This fixes SSBO atomic cmpswap tests and transform feedback order piglit tests.
Fixes: 3bb65c0670 - ac: force enable -structurizecfg-skip-uniform-regions for LLVM 11
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3884
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11517 >
2021-06-22 22:13:07 +00:00
Jason Ekstrand
f0f713960b
nir,amd: Suffix nir_op_cube_face_coord/index with _amd
...
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11463 >
2021-06-21 09:03:34 -05:00
Marek Olšák
12d2df15f1
ac/llvm: add a callback to ac_cull_triangle to generate code in inner-most block
...
This will reduce jumps in culling code.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11486 >
2021-06-20 01:22:01 -04:00
Timur Kristóf
1e49018ced
amd: Add extra source to the mbcnt_amd NIR intrinsic.
...
The v_mbcnt instructions can take an extra source that they add to
the result. This is not exposed in SPIR-V but we now expose it in NIR.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11072 >
2021-06-09 16:48:51 +00:00
Rhys Perry
7d76b07d6b
ac/llvm: implement byte/word extract/insert instructions
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3151 >
2021-06-08 08:57:43 +00:00
Caio Marcelo de Oliveira Filho
c8a7bd0dc8
nir: Rename WORK_GROUP (and similar) to WORKGROUP
...
Be consistent with other usages in Vulkan and SPIR-V, and the recently
added workgroup_size field.
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11190 >
2021-06-07 22:34:42 +00:00
Caio Marcelo de Oliveira Filho
a71a780598
nir: Rename nir_intrinsic_load_local_group_size to nir_intrinsic_load_workgroup_size
...
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11190 >
2021-06-07 22:34:42 +00:00
Marek Olšák
13acbaecd8
radeonsi: rewrite the prefix sum computation for shader culling
...
Instead of storing the vertex mask per wave into LDS and then computing
the prefix sum, store 8-bit bitcounts (vertex counts) of the vertex masks
into LDS. This allows us to compute the sum using v_sad_u8, which computes
a sum of 4 i8vec4 components in one instruction.
Each i8vec4 of vertex counts is loaded in parallel threads (one dword
per thread) instead of all being loaded in thread 0, and readlane copies
them to SGPRs instead of readfirstlane.
LDS is no longer initialized before culling. Instead, the counts for
inactive waves are masked with AND later.
Incorrect old comments are also fixed.
This change removes 80 bytes from the code size, and it allows increasing
the workgroup size from 128 to 256. (which is the main motivation for this)
Now changing the workgroup size with wave64 has no effect on the code size.
Switching to wave32 with 8 waves even generates slightly smaller code than
wave64 with 4 waves.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10813 >
2021-05-25 16:15:44 +00:00