Commit graph

300 commits

Author SHA1 Message Date
Timur Kristóf
5aa39253cb nir: Rename nir_get_io_vertex_index_src and include per-primitive I/O.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13466>
2021-11-16 07:46:55 +00:00
Marek Olšák
9151ac3531 ac,radeonsi: cull small lines in the shader using the diamond exit rule
It also splits clip_half_line_width into X and Y components for tighter
view culling.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>
2021-11-16 02:11:46 +00:00
Marek Olšák
f8a0aa6852 radeonsi: fix view culling for wide lines
We need to cull wide lines as quads, but only for view culling.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13700>
2021-11-16 02:11:45 +00:00
Marek Olšák
5ee2965283 ac/llvm: accept primitives whose face culling determinant is Inf or NaN
Based on https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13299/diffs

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13380>
2021-10-19 12:49:06 +00:00
Marek Olšák
efaab0ec50 ac/llvm: add helper ac_build_is_inf_or_nan
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13380>
2021-10-19 12:49:06 +00:00
Marek Olšák
5e8f76b713 ac/llvm: use fmac instead of mul+sub in face culling
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13380>
2021-10-19 12:49:05 +00:00
Jason Ekstrand
956199e870 nir: s/nir_var_mem_image/nir_var_image/g
We typically use nir_var_mem_* for stuff that has an explicit byte-based
memory layout.  Images are opaque.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13386>
2021-10-16 03:47:10 +00:00
Jason Ekstrand
cd49706cb1 amd/llvm/nir: Add support for nir_var_mem_image
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4743>
2021-10-15 14:58:55 +00:00
Samuel Pitoiset
a6298b1bc9 radv: remove unnecessary radv_shader_info:num_inline_push_consts
This can be determined directly from the user SGPR loc.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13149>
2021-10-08 11:37:19 +00:00
Rhys Perry
2564fbfc57 ac/llvm: fix image_samples with null descriptors
Fixes various dEQP-VK.robustness.robustness2.* tests.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13251>
2021-10-08 10:27:11 +00:00
Rhys Perry
1fb63367a8 radv,aco: don't include FMASK in the storage descriptor
We perform a FMASK expand when transitioning to GENERAL or TRANSFER_DST
layout, so storage images always have an identity FMASK.

radeonsi doesn't appear to expand the FMASK for read-only storage images,
so the sample index adjustment is still needed there.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12214>
2021-10-07 15:36:39 +00:00
Rhys Perry
91a5c9fdf2 ac/nir: return 0x76543210 for NULL FMASK fetch
This can replace several v_cndmask_b32 with a single v_cndmask_b32.

fossil-db (Sienna Cichlid):
Totals from 1169 (0.78% of 150170) affected shaders:
SGPRs: 82032 -> 81984 (-0.06%); split: -0.19%, +0.13%
VGPRs: 68368 -> 68280 (-0.13%); split: -0.30%, +0.18%
SpillSGPRs: 8754 -> 8609 (-1.66%); split: -4.06%, +2.40%
CodeSize: 5460132 -> 5446308 (-0.25%); split: -0.42%, +0.17%
MaxWaves: 24456 -> 24462 (+0.02%); split: +0.03%, -0.01%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12214>
2021-10-07 15:36:39 +00:00
Rhys Perry
225fe37c14 nir: add _amd suffix to fragment_mask_fetch and fragment_fetch texops
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12214>
2021-10-07 15:36:39 +00:00
Marek Olšák
f00d3e2909 radeonsi: implement shader-based culling for lines
This helps some viewperf subtests.
Only view XY culling is done. Edgeflags are always disabled with lines.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13048>
2021-09-28 17:30:06 +00:00
Marek Olšák
e7e0b90c94 ac/llvm: rename ac_cull_triangle -> ac_cull_primitive
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13048>
2021-09-28 17:30:06 +00:00
Marek Olšák
25026076ab ac/llvm: pass cull options into cull_bbox directly
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13048>
2021-09-28 17:30:06 +00:00
Bas Nieuwenhuizen
8ca54b4d38 radv: Support nir_intrinsic_load_global_constant.
SPIR-V parsing can result in some direct constant usage for shader
records. Lower this early to a global based intrinsic so that it
doesn't interfere with the later 32-bit offset based constants
for scratch usage.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12592>
2021-09-21 01:53:39 +00:00
Samuel Pitoiset
7a25451752 ac/llvm: implement nir_intrinsic_global_atomic_{fmin,fmax}
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12716>
2021-09-15 14:10:42 +00:00
Samuel Pitoiset
9deee80f51 ac/llvm: implement nir_intrinsic_shared_atomic_{fmin,fmax}
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12716>
2021-09-15 14:10:42 +00:00
Samuel Pitoiset
efe40c98ed ac/llvm: implement nir_intrinsic_ssbo_atomic_{fmin,fmax}
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12716>
2021-09-15 14:10:42 +00:00
Samuel Pitoiset
cf3e31fd11 ac/llvm: implement nir_intrinsic_image_deref_atomic_{fmin,fmax}
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12716>
2021-09-15 14:10:42 +00:00
Marek Olšák
b330c7cb2a radeonsi: use a trick to extract and pack edgeflags using fewer instructions
This removes 4 instructions from the prim export packing.

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12343>
2021-09-14 15:24:11 +00:00
Samuel Pitoiset
09cf2584bd ac/llvm: fix build with LLVM 14
AttributeList::hasAttribute() has been replaced by
hasParamAttr()/hasRetAttr().

All supported LLVM versions have hasParamAttr().

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5320
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Michel Dänzer <mdaenzer@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12715>
2021-09-13 10:04:30 +02:00
Marek Olšák
69e96cfc0d ac,radv: remove unused inputs array and VS input code
The previous commit stopped using "inputs".

"load_layer_id" has always been broken and it was probably unused anyway.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Marek Olšák
3fb229e010 ac,radeonsi: load VS inputs at the call site of nir_intrinsic_load_input
to match ACO

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Marek Olšák
bce7c7f3fc ac/llvm: implement nir_intrinsic_elect
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Marek Olšák
e0f07483d0 ac/llvm: implement nir_intrinsic_overwrite_*_arguments_amd
This should work if the intrinsics are not called from conditional blocks.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Marek Olšák
1e178f7a37 ac: make ac_shader_abi::inputs an array instead of a pointer
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Marek Olšák
6df5f268db ac: remove needless parameters from ac_shader_abi::emit_outputs
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Marek Olšák
2e95ad1433 ac/llvm: implement a bunch of NIR AMD intrinsics for NGG
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Marek Olšák
a33602b1f9 ac/llvm: remove load_tess_coord callback
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12570>
2021-09-07 17:51:41 +00:00
Rhys Perry
859790ba54 ac/llvm: implement udot_4x8/sdot_4x8/udot_2x16/sdot_2x16 opcodes
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>
2021-09-03 13:21:27 +00:00
Rhys Perry
d6619d0a01 ac/llvm,radv: implement uadd_sat/iadd_sat
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>
2021-09-03 13:21:27 +00:00
Rhys Perry
f7cdd49a09 ac/llvm: implement nir_op_pack_32_4x8
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12617>
2021-09-03 13:21:27 +00:00
Samuel Pitoiset
72b6b02e09 ac/llvm: fix huge alignment when loading from shared memory
LLVM doesn't support huge alignments, also it can optimize the shared
loads, so it's unecessary to emit better (but broken) LLVM IR.

Fixes a bunch of crashes with RADV_DEBUG=llvm,checkir.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12616>
2021-08-31 09:56:27 +02:00
Samuel Pitoiset
e72c2e36e0 ac/llvm: adjust assertion for nir_intrinsic_terminate
Fixes dEQP-VK.spirv_assembly.instruction.terminate*.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12616>
2021-08-31 09:56:27 +02:00
Rhys Perry
0460f01fdc ac/llvm: implement v2f16 fsat
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12143>
2021-08-02 10:02:51 +00:00
Jason Ekstrand
a7b429e8ec amd: Don't handle nir_tex_src_ms_mcs
It's an intel-specific texture source and will never be seen on AMD.

Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11775>
2021-07-23 15:53:57 +00:00
Jason Ekstrand
e83fe65cd8 radv,radeonsi: Do cube size divide-by-6 lowering in NIR
No point in carrying all this code around twice each in two back-ends.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12005>
2021-07-22 14:22:35 -05:00
Marek Olšák
d9173b3b46 ac/llvm: rework how negative W affects culling to not call accept_func twice
Always execute the bbox code regardless of negative W, and then simply
use || to discard the result if any W is negative. This is expected to be
rare. (it only happens when a primitive intersects the near plane)

This allows us to eliminate the else statement, which is no longer
executed for accepted primitives with negative W, which are the only
primitives that needed the else branch.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11754>
2021-07-08 18:37:41 +00:00
Marek Olšák
74b33a8c37 ac/llvm: don't return a status from ac_cull_triangle because it's unused
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11754>
2021-07-08 18:37:41 +00:00
Pierre-Eric Pelloux-Prayer
b564ec7769 ac/llvm: call the callback in all return paths of ac_cull_triangle
Fixes: 12d2df15f1 ("ac/llvm: add a callback to ac_cull_triangle to generate code in inner-most block")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4959
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11625>
2021-06-29 23:38:21 +02:00
Marek Olšák
d3fddf63ec ac/llvm: don't set skip-uniform-regions to fix atomic.cmpswap
This fixes SSBO atomic cmpswap tests and transform feedback order piglit tests.

Fixes: 3bb65c0670 - ac: force enable -structurizecfg-skip-uniform-regions for LLVM 11
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3884

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11517>
2021-06-22 22:13:07 +00:00
Jason Ekstrand
f0f713960b nir,amd: Suffix nir_op_cube_face_coord/index with _amd
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11463>
2021-06-21 09:03:34 -05:00
Marek Olšák
12d2df15f1 ac/llvm: add a callback to ac_cull_triangle to generate code in inner-most block
This will reduce jumps in culling code.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11486>
2021-06-20 01:22:01 -04:00
Timur Kristóf
1e49018ced amd: Add extra source to the mbcnt_amd NIR intrinsic.
The v_mbcnt instructions can take an extra source that they add to
the result. This is not exposed in SPIR-V but we now expose it in NIR.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11072>
2021-06-09 16:48:51 +00:00
Rhys Perry
7d76b07d6b ac/llvm: implement byte/word extract/insert instructions
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3151>
2021-06-08 08:57:43 +00:00
Caio Marcelo de Oliveira Filho
c8a7bd0dc8 nir: Rename WORK_GROUP (and similar) to WORKGROUP
Be consistent with other usages in Vulkan and SPIR-V, and the recently
added workgroup_size field.

Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11190>
2021-06-07 22:34:42 +00:00
Caio Marcelo de Oliveira Filho
a71a780598 nir: Rename nir_intrinsic_load_local_group_size to nir_intrinsic_load_workgroup_size
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Acked-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11190>
2021-06-07 22:34:42 +00:00
Marek Olšák
13acbaecd8 radeonsi: rewrite the prefix sum computation for shader culling
Instead of storing the vertex mask per wave into LDS and then computing
the prefix sum, store 8-bit bitcounts (vertex counts) of the vertex masks
into LDS. This allows us to compute the sum using v_sad_u8, which computes
a sum of 4 i8vec4 components in one instruction.

Each i8vec4 of vertex counts is loaded in parallel threads (one dword
per thread) instead of all being loaded in thread 0, and readlane copies
them to SGPRs instead of readfirstlane.

LDS is no longer initialized before culling. Instead, the counts for
inactive waves are masked with AND later.

Incorrect old comments are also fixed.

This change removes 80 bytes from the code size, and it allows increasing
the workgroup size from 128 to 256. (which is the main motivation for this)

Now changing the workgroup size with wave64 has no effect on the code size.
Switching to wave32 with 8 waves even generates slightly smaller code than
wave64 with 4 waves.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10813>
2021-05-25 16:15:44 +00:00