Commit graph

13547 commits

Author SHA1 Message Date
Samuel Pitoiset
3ff58049b5 radv: implement AMD_shader_early_and_late_fragment_tests
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19738>
2022-12-14 08:16:27 +00:00
Ian Romanick
eb76cee9f8 nir: Eliminate nir_op_i2b
There are a lot of optimizations in opt_algebraic that match ('ine', a,
0), but there are almost none that match i2b.  Instead of adding a huge
pile of additional patterns (including variations that include both ine
and i2b), always lower i2b to a != 0.

At this point in the series, it should be impossible for anything to
generate i2b, so there /should not/ be any changes.

The failing test on d3d12 is a pre-existing bug that is triggered by
this change.  I talked to Jesse about it, and, after some analysis, he
suggested just adding it to the list of known failures.

v2: Don't rematerialize i2b instructions in dxil_nir_lower_x2b.

v3: Don't rematerialize i2b instructions in zink_nir_algebraic.py.

v4: Fix zink-on-TGL CI failures by calling nir_opt_algebraic after
nir_lower_doubles makes progress.  The latter can generate b2i
instructions, but nir_lower_int64 can't handle them (anymore).

v5: Add back most of the hunk at line 2125 of nir_opt_algebraic.py. I
had accidentally removed the f2b(bf2(x)) optimization.

v6: Just eliminate the i2b instruction.

v7: Remove missed i2b32 in midgard_compile.c. Remove (now unused)
emit_alu_i2orf2_b1 function from sfn_instr_alu.cpp. Previously this
function was still used. 🤷

No shader-db changes on any Intel platform.

All Intel platforms had similar results. (Ice Lake shown)
Instructions in all programs: 141165875 -> 141165873 (-0.0%)
Instructions helped: 2

Cycles in all programs: 9098956382 -> 9098956350 (-0.0%)
Cycles helped: 2

The two Vulkan shaders are helped because of the "new" (('b2i32',
('ine', ('ubfe', a, b, 1), 0)), ('ubfe', a, b, 1)) algebraic pattern.

Acked-by: Jesse Natalie <jenatali@microsoft.com> [earlier version]
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev> [earlier version]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>
2022-12-14 06:23:21 +00:00
Ian Romanick
7a5e9df39d nir: Use nir_i2b wrapper everywhere instead of using nir_i2b1 directly
No shader-db or fossil-db changes on any Intel platform.

v2: Add missed i2b1 in ir3_nir_opt_preamble.c.

v3: Add missed i2b1 in ac_nir_lower_ngg.c.

Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Acked-by: Jesse Natalie <jenatali@microsoft.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Tested-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15121>
2022-12-14 06:23:21 +00:00
Konstantin Seurer
687a82d2d3 radv/rt: Handle no-null shader flags
If those flags are set, we can assume that idx is not 0.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20286>
2022-12-13 23:30:28 +00:00
Marek Olšák
afcbccb078 ac/llvm: implement ACCESS_USE_FORMAT_AMD as buffer_load/store_format
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>
2022-12-13 20:33:05 +00:00
Marek Olšák
5307480f4e ac/llvm: unify load_buffer_amd and store_buffer_amd code
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>
2022-12-13 20:33:05 +00:00
Marek Olšák
716ac4a55d nir: replace IS_SWIZZLED flag with ACCESS_IS_SWIZZLED_AMD
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>
2022-12-13 20:33:05 +00:00
Marek Olšák
7998c3bdd3 nir: remove redundant SLC_AMD in favor of ACCESS_STREAM_CACHE_POLICY
ACCESS_STREAM_CACHE_POLICY was added to map to SLC for AMD.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>
2022-12-13 20:33:05 +00:00
Marek Olšák
c0d69b40bc nir: add nir_texop_sampler_descriptor_amd
We'll use it to query the min/mag filter in the shader.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19422>
2022-12-13 20:33:05 +00:00
Samuel Pitoiset
b4f940913d radv: do not remove the PointSize built-in for polygon mode as points
Determine if it can be removed when generating the graphics pipeline
key because of dynamic states.

Fixes new CTS dEQP-VK.rasterization.polygon_as_large_points.* and also
spec@!opengl 1.1@polygon-offset with Zink/RADV.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20073>
2022-12-13 15:01:31 +00:00
Qiang Yu
d2ac1b18df ac/nir: add ac_nir_lower_legacy_gs
For legacy (non-NGG) GS to lower outputs to memory stores and add
shader query when required.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>
2022-12-13 11:43:53 +08:00
Qiang Yu
0e52a4faf0 ac/nir: move ngg_gs_shader_query to a common function
To be shared by NGG GS and legacy GS. Legacy GS need this when
GFX10 which mix use NGG and legacy GS. For example when streamout
is enabled, it uses legacy GS, otherwise uses NGG GS. So legacy
GS also need to update query emulation which is a sum of NGG and
legacy GS results.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>
2022-12-13 11:43:49 +08:00
Qiang Yu
9a6416b374 nir,ac/llvm,radv: add stream id index to nir_load_ring_gsvs_amd
For used by legacy GS to store output to different ring according
to stream id.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>
2022-12-13 11:43:45 +08:00
Qiang Yu
0ea589ec69 ac/llvm,radv,radeonsi: pass instruction to intrinsic_load abi
For simple intrinsic which also need other fields to translate
to LLVM like stream_id.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>
2022-12-13 11:43:40 +08:00
Qiang Yu
603e68ba36 ac/nir,radeonsi: replace llvm ngg gs invocation query with nir
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20156>
2022-12-13 01:26:42 +00:00
Qiang Yu
fd240f759f nir,radv,radeonsi: add nir_atomic_add_gs_invocation_count_amd
For shader query emulation.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20156>
2022-12-13 01:26:42 +00:00
Bas Nieuwenhuizen
92959061a9 radv: Improved export conflict bug workaround.
Based on f129db911b ("radeonsi/gfx11: use a better workaround for the export conflict bug")
which claims better performance.

I couldn't be bothered to do the refactor to check the sample count with
dynamic sample counts, so this is just conservative there.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20273>
2022-12-12 19:59:13 +00:00
Friedrich Vock
568fa71ef8 radv: Handle NULL miss shaders
Fixes reflections in DOOM Eternal.

Fixes: 85580faa ("radv: Add ray traversal loop.")
Closes: #6210
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20270>
2022-12-12 18:52:59 +00:00
Friedrich Vock
276236ed6d radv: Change shaderGroupBaseAlignment to RADV_RT_HANDLE_SIZE
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20270>
2022-12-12 18:52:59 +00:00
Konstantin Seurer
9492b84a68 radv: Add a driconf option to force non uniform texture sampling
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>
2022-12-12 18:18:32 +00:00
Konstantin Seurer
cdd68cbafd radv/rt: Hash the pipeline key
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>
2022-12-12 18:18:32 +00:00
Konstantin Seurer
48ae92ceea radv/rt: Propagate radv_pipeline_key
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>
2022-12-12 18:18:32 +00:00
Konstantin Seurer
7b5ac0ae2a radv/rt: Work around incorrect NOT_DEFERRED handling in Portal RTX
Allows Portal RTX to reach the main menu.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>
2022-12-12 18:18:32 +00:00
Mikhail Korolev
c147a35644 radv: fix assertion on gpu hang detection
fixes assert in RADV_DECL_PIPELINE_DOWNCAST when bound pipline is a compute
pipeline

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20276>
2022-12-12 12:55:07 +00:00
Samuel Pitoiset
13f39da71a radv: fix hashing descriptor set layout
Shouldn't have pointers.

Fixes: 19f8d33876 ("radv: Use vk_descriptor_set_layout")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20250>
2022-12-12 07:33:21 +00:00
Bas Nieuwenhuizen
efa4e9568b radv: Use correct watermark for early loop exit.
The previous check assumed the stack starts at offset=0, which isn't
necessarily true for ray queries.

Note that this didn't cause correctness issues, just made an optimization
not apply. Found when I accidentally made this load-bearing in a
refactor.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20265>
2022-12-11 18:51:29 +00:00
Bas Nieuwenhuizen
f0d6a1a685 radv: Rename stack_base to stack_low_watermark.
Better covers the purpose.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20265>
2022-12-11 18:51:29 +00:00
Rhys Perry
a89755d179 radv: fix task payload lowering when shared_memory_explicit_layout=true
If shared_memory_explicit_layout=true, we would have skipped lowering task
payload variables to explicit types.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>
2022-12-09 20:56:52 +00:00
Rhys Perry
e4060752e2 radv: fix mesh shaders with null winsys
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>
2022-12-09 20:56:52 +00:00
Rebecca Mckeever
64d7385e61 radv: Delete VK_KHR_device_group provided entrypoints
Delete radv_CmdDispatch, radv_CmdSetDeviceMask, and
radv_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>
2022-12-09 14:07:00 -06:00
Rhys Perry
c872e339a1 radv: remove some unnecessary 64-bit IO handling
nir_lower_io() lowers these to 32-bit.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20137>
2022-12-09 17:30:24 +00:00
Rhys Perry
6a5b615ab1 radv: fix streamout with different streams in the same varying slot
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20137>
2022-12-09 17:30:24 +00:00
Rhys Perry
20e670d060 aco/ra: don't swap create_vector operand with definition blocker for SGPRs
There is no SGPR swap instruction, we always need 3 XORs.

fossil-db (navi21):
Totals from 76 (0.06% of 135636) affected shaders:
Instrs: 58400 -> 58347 (-0.09%); split: -0.10%, +0.01%
CodeSize: 312580 -> 312368 (-0.07%); split: -0.08%, +0.01%
Latency: 843333 -> 843180 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 126431 -> 126412 (-0.02%)
Copies: 4008 -> 3955 (-1.32%); split: -1.47%, +0.15%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20240>
2022-12-09 15:58:43 +00:00
Rhys Perry
a05dd58309 aco/ra: don't swap p_create_vector operand with definition blocker for scc
SCC is 1-bit, and we can't copy a 32-bit value into it.

Fixes dEQP-VK.spirv_assembly.type.scalar.i32.iequal_tesse with
ACO_DEBUG=noopt.

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 9476986e6f ("aco/ra: special-case get_reg_for_create_vector_copy()")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20240>
2022-12-09 15:58:43 +00:00
Samuel Pitoiset
25a5d7ac0d radv: fix re-emitting the RB+ state when dynamic color write mask change
Found by inspection.

Fixes: a92d1d13c5 ("radv: add support for dynamic color write mask")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20204>
2022-12-09 07:37:00 +00:00
Konstantin Seurer
4c83a96f16 radv/rt: Discard metadata after lowering intersection shaders
We inserted some enstructions and control flow. Therefore, metadata is
not valid anymore.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19866>
2022-12-09 07:07:10 +00:00
Konstantin Seurer
79b7ed61a8 radv/rt: Lower hit attributes to registers
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19866>
2022-12-09 07:07:10 +00:00
Konstantin Seurer
a598263800 radv/rt: Move load_hit_attrib_amd to the top
Make sure to load hit attributes before a trace_ray call can overwrite them.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19866>
2022-12-09 07:07:10 +00:00
Konstantin Seurer
9f03e9c286 radv: Add nir_foreach_variable_with_modes to .clang-format
Yet another loop macro.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19866>
2022-12-09 07:07:10 +00:00
Samuel Pitoiset
011a0b97b2 radv,aco: move radv_ps_epilog_key to the graphics pipeline key
To avoid redundant structs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20199>
2022-12-08 13:28:00 +00:00
Samuel Pitoiset
67c9497435 radv: fix PS epilogs with RADV_PERFTEST=pswave32
This option is rarely or even never used but it was broken. While we
are at it, remove radv_ps_epilog_key::wave32 because the wave size
can only be changed globally for PS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20199>
2022-12-08 13:28:00 +00:00
Samuel Pitoiset
b7f49de625 radv,aco: use 8-bit for color_is_int{8,10} everywhere
Do not need 32-bits because there is only up to 8 MRTs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20199>
2022-12-08 13:28:00 +00:00
Samuel Pitoiset
9079bd821c radv,aco: rename color output related fields for consistency
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20199>
2022-12-08 13:28:00 +00:00
Tatsuyuki Ishi
327c906424 aco: Migrate RA to use std::optional
The use of std::optional simplifies expressions and would be useful for some
upcoming RA tweaks.

C++17 has been available since the merge of rusticl and should be safe to use as
far as packaging is concerned.

A few style choices are:
- Testing for emptiness uses implicit bool conversion.
- Constructing an empty value uses {}.
- Constructing a filled value uses the implicit conversion constructor.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20125>
2022-12-08 12:08:01 +00:00
Bas Nieuwenhuizen
513442dc32 aco: Add s_delay_alu support for GFX11+
Roughly copied from LLVM. This facilitates better ALU usage by
switching between waves when there is an ALU stall, which isn't
automatic anymore on GFX11.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19743>
2022-12-07 22:05:25 +00:00
Bas Nieuwenhuizen
cd3bf56ace aco: Add helper to get cycle info for an instruction.
For use in s_delay_alu tracking

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19743>
2022-12-07 22:05:25 +00:00
Bas Nieuwenhuizen
352e492c7b aco: Add isTrans helper.
For the s_delay_alu tracking.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19743>
2022-12-07 22:05:25 +00:00
Samuel Pitoiset
bdb7fd69d6 radv: mark extendedDynamicState3SampleLocationsEnable as not required by Zink
Only dynamic blend equation support is still required.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19924>
2022-12-07 16:43:17 +01:00
Samuel Pitoiset
06a65be3d4 radv: advertise extendedDynamicState3LineRasterizationMode
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19924>
2022-12-07 16:43:15 +01:00
Samuel Pitoiset
1009276312 radv: add support for dynamic line rasterization mode
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19924>
2022-12-07 16:43:13 +01:00