Commit graph

10787 commits

Author SHA1 Message Date
Qiang Yu
d2ac1b18df ac/nir: add ac_nir_lower_legacy_gs
For legacy (non-NGG) GS to lower outputs to memory stores and add
shader query when required.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>
2022-12-13 11:43:53 +08:00
Qiang Yu
0e52a4faf0 ac/nir: move ngg_gs_shader_query to a common function
To be shared by NGG GS and legacy GS. Legacy GS need this when
GFX10 which mix use NGG and legacy GS. For example when streamout
is enabled, it uses legacy GS, otherwise uses NGG GS. So legacy
GS also need to update query emulation which is a sum of NGG and
legacy GS results.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>
2022-12-13 11:43:49 +08:00
Qiang Yu
9a6416b374 nir,ac/llvm,radv: add stream id index to nir_load_ring_gsvs_amd
For used by legacy GS to store output to different ring according
to stream id.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>
2022-12-13 11:43:45 +08:00
Qiang Yu
0ea589ec69 ac/llvm,radv,radeonsi: pass instruction to intrinsic_load abi
For simple intrinsic which also need other fields to translate
to LLVM like stream_id.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20158>
2022-12-13 11:43:40 +08:00
Qiang Yu
603e68ba36 ac/nir,radeonsi: replace llvm ngg gs invocation query with nir
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20156>
2022-12-13 01:26:42 +00:00
Qiang Yu
fd240f759f nir,radv,radeonsi: add nir_atomic_add_gs_invocation_count_amd
For shader query emulation.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20156>
2022-12-13 01:26:42 +00:00
Bas Nieuwenhuizen
92959061a9 radv: Improved export conflict bug workaround.
Based on f129db911b ("radeonsi/gfx11: use a better workaround for the export conflict bug")
which claims better performance.

I couldn't be bothered to do the refactor to check the sample count with
dynamic sample counts, so this is just conservative there.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20273>
2022-12-12 19:59:13 +00:00
Friedrich Vock
568fa71ef8 radv: Handle NULL miss shaders
Fixes reflections in DOOM Eternal.

Fixes: 85580faa ("radv: Add ray traversal loop.")
Closes: #6210
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20270>
2022-12-12 18:52:59 +00:00
Friedrich Vock
276236ed6d radv: Change shaderGroupBaseAlignment to RADV_RT_HANDLE_SIZE
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20270>
2022-12-12 18:52:59 +00:00
Konstantin Seurer
9492b84a68 radv: Add a driconf option to force non uniform texture sampling
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>
2022-12-12 18:18:32 +00:00
Konstantin Seurer
cdd68cbafd radv/rt: Hash the pipeline key
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>
2022-12-12 18:18:32 +00:00
Konstantin Seurer
48ae92ceea radv/rt: Propagate radv_pipeline_key
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>
2022-12-12 18:18:32 +00:00
Konstantin Seurer
7b5ac0ae2a radv/rt: Work around incorrect NOT_DEFERRED handling in Portal RTX
Allows Portal RTX to reach the main menu.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20243>
2022-12-12 18:18:32 +00:00
Mikhail Korolev
c147a35644 radv: fix assertion on gpu hang detection
fixes assert in RADV_DECL_PIPELINE_DOWNCAST when bound pipline is a compute
pipeline

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20276>
2022-12-12 12:55:07 +00:00
Samuel Pitoiset
13f39da71a radv: fix hashing descriptor set layout
Shouldn't have pointers.

Fixes: 19f8d33876 ("radv: Use vk_descriptor_set_layout")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20250>
2022-12-12 07:33:21 +00:00
Bas Nieuwenhuizen
efa4e9568b radv: Use correct watermark for early loop exit.
The previous check assumed the stack starts at offset=0, which isn't
necessarily true for ray queries.

Note that this didn't cause correctness issues, just made an optimization
not apply. Found when I accidentally made this load-bearing in a
refactor.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20265>
2022-12-11 18:51:29 +00:00
Bas Nieuwenhuizen
f0d6a1a685 radv: Rename stack_base to stack_low_watermark.
Better covers the purpose.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20265>
2022-12-11 18:51:29 +00:00
Rhys Perry
a89755d179 radv: fix task payload lowering when shared_memory_explicit_layout=true
If shared_memory_explicit_layout=true, we would have skipped lowering task
payload variables to explicit types.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>
2022-12-09 20:56:52 +00:00
Rhys Perry
e4060752e2 radv: fix mesh shaders with null winsys
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19597>
2022-12-09 20:56:52 +00:00
Rebecca Mckeever
64d7385e61 radv: Delete VK_KHR_device_group provided entrypoints
Delete radv_CmdDispatch, radv_CmdSetDeviceMask, and
radv_GetDeviceGroupPeerMemoryFeatures so that the vk_common_*
versions will be used instead. This will avoid repeated code.

Signed-off-by: Rebecca Mckeever <rebecca.mckeever@collabora.com>
Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20218>
2022-12-09 14:07:00 -06:00
Rhys Perry
c872e339a1 radv: remove some unnecessary 64-bit IO handling
nir_lower_io() lowers these to 32-bit.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20137>
2022-12-09 17:30:24 +00:00
Rhys Perry
6a5b615ab1 radv: fix streamout with different streams in the same varying slot
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20137>
2022-12-09 17:30:24 +00:00
Rhys Perry
20e670d060 aco/ra: don't swap create_vector operand with definition blocker for SGPRs
There is no SGPR swap instruction, we always need 3 XORs.

fossil-db (navi21):
Totals from 76 (0.06% of 135636) affected shaders:
Instrs: 58400 -> 58347 (-0.09%); split: -0.10%, +0.01%
CodeSize: 312580 -> 312368 (-0.07%); split: -0.08%, +0.01%
Latency: 843333 -> 843180 (-0.02%); split: -0.02%, +0.00%
InvThroughput: 126431 -> 126412 (-0.02%)
Copies: 4008 -> 3955 (-1.32%); split: -1.47%, +0.15%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20240>
2022-12-09 15:58:43 +00:00
Rhys Perry
a05dd58309 aco/ra: don't swap p_create_vector operand with definition blocker for scc
SCC is 1-bit, and we can't copy a 32-bit value into it.

Fixes dEQP-VK.spirv_assembly.type.scalar.i32.iequal_tesse with
ACO_DEBUG=noopt.

No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 9476986e6f ("aco/ra: special-case get_reg_for_create_vector_copy()")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20240>
2022-12-09 15:58:43 +00:00
Samuel Pitoiset
25a5d7ac0d radv: fix re-emitting the RB+ state when dynamic color write mask change
Found by inspection.

Fixes: a92d1d13c5 ("radv: add support for dynamic color write mask")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20204>
2022-12-09 07:37:00 +00:00
Konstantin Seurer
4c83a96f16 radv/rt: Discard metadata after lowering intersection shaders
We inserted some enstructions and control flow. Therefore, metadata is
not valid anymore.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19866>
2022-12-09 07:07:10 +00:00
Konstantin Seurer
79b7ed61a8 radv/rt: Lower hit attributes to registers
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19866>
2022-12-09 07:07:10 +00:00
Konstantin Seurer
a598263800 radv/rt: Move load_hit_attrib_amd to the top
Make sure to load hit attributes before a trace_ray call can overwrite them.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19866>
2022-12-09 07:07:10 +00:00
Konstantin Seurer
9f03e9c286 radv: Add nir_foreach_variable_with_modes to .clang-format
Yet another loop macro.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19866>
2022-12-09 07:07:10 +00:00
Samuel Pitoiset
011a0b97b2 radv,aco: move radv_ps_epilog_key to the graphics pipeline key
To avoid redundant structs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20199>
2022-12-08 13:28:00 +00:00
Samuel Pitoiset
67c9497435 radv: fix PS epilogs with RADV_PERFTEST=pswave32
This option is rarely or even never used but it was broken. While we
are at it, remove radv_ps_epilog_key::wave32 because the wave size
can only be changed globally for PS.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20199>
2022-12-08 13:28:00 +00:00
Samuel Pitoiset
b7f49de625 radv,aco: use 8-bit for color_is_int{8,10} everywhere
Do not need 32-bits because there is only up to 8 MRTs.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20199>
2022-12-08 13:28:00 +00:00
Samuel Pitoiset
9079bd821c radv,aco: rename color output related fields for consistency
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20199>
2022-12-08 13:28:00 +00:00
Tatsuyuki Ishi
327c906424 aco: Migrate RA to use std::optional
The use of std::optional simplifies expressions and would be useful for some
upcoming RA tweaks.

C++17 has been available since the merge of rusticl and should be safe to use as
far as packaging is concerned.

A few style choices are:
- Testing for emptiness uses implicit bool conversion.
- Constructing an empty value uses {}.
- Constructing a filled value uses the implicit conversion constructor.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20125>
2022-12-08 12:08:01 +00:00
Bas Nieuwenhuizen
513442dc32 aco: Add s_delay_alu support for GFX11+
Roughly copied from LLVM. This facilitates better ALU usage by
switching between waves when there is an ALU stall, which isn't
automatic anymore on GFX11.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19743>
2022-12-07 22:05:25 +00:00
Bas Nieuwenhuizen
cd3bf56ace aco: Add helper to get cycle info for an instruction.
For use in s_delay_alu tracking

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19743>
2022-12-07 22:05:25 +00:00
Bas Nieuwenhuizen
352e492c7b aco: Add isTrans helper.
For the s_delay_alu tracking.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19743>
2022-12-07 22:05:25 +00:00
Samuel Pitoiset
bdb7fd69d6 radv: mark extendedDynamicState3SampleLocationsEnable as not required by Zink
Only dynamic blend equation support is still required.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19924>
2022-12-07 16:43:17 +01:00
Samuel Pitoiset
06a65be3d4 radv: advertise extendedDynamicState3LineRasterizationMode
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19924>
2022-12-07 16:43:15 +01:00
Samuel Pitoiset
1009276312 radv: add support for dynamic line rasterization mode
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19924>
2022-12-07 16:43:13 +01:00
Samuel Pitoiset
f992e663c7 radv: advertise extendedDynamicState3RasterizationSamples
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19924>
2022-12-07 16:43:11 +01:00
Samuel Pitoiset
a2fe4e9b04 radv: add support for dynamic rasterization samples
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19924>
2022-12-07 16:43:09 +01:00
Samuel Pitoiset
c88d4037aa radv: store the number of color/ds samples to the rendering state
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19924>
2022-12-07 16:43:07 +01:00
Samuel Pitoiset
12f26b5e6d radv: fix guardband if the polygon mode is points or lines
If points or lines are drawn using the polygon mode, the guardband
should be adjusted for large points/lines.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20185>
2022-12-06 19:30:51 +00:00
Rhys Perry
bd30adf89d aco: apply NUW to additions for scratch access
fossil-db (navi21):
Totals from 52 (0.04% of 135636) affected shaders:
Instrs: 79036 -> 78567 (-0.59%)
CodeSize: 431188 -> 427984 (-0.74%)
Latency: 1318142 -> 1313821 (-0.33%)
InvThroughput: 293842 -> 292836 (-0.34%)
VClause: 2555 -> 2361 (-7.59%); split: -8.06%, +0.47%
Copies: 8746 -> 8767 (+0.24%); split: -0.11%, +0.35%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20117>
2022-12-06 15:23:38 +00:00
Rhys Perry
381de3c809 aco: more carefully apply constant offsets into scratch accesses
Death stranding does scratch_arr[80-idx]. This doesn't seem to work if we
try to combine the subtraction into the access.

fossil-db (navi21):
Totals from 52 (0.04% of 135636) affected shaders:
Instrs: 78560 -> 79036 (+0.61%)
CodeSize: 427940 -> 431188 (+0.76%)
Latency: 1313809 -> 1318142 (+0.33%)
InvThroughput: 292833 -> 293842 (+0.34%)
VClause: 2361 -> 2555 (+8.22%); split: -0.51%, +8.73%
Copies: 8767 -> 8746 (-0.24%); split: -0.35%, +0.11%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Fixes: 0e783d687a ("aco: use scratch_* for scratch load/store on GFX9+")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7735
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20117>
2022-12-06 15:23:38 +00:00
Rhys Perry
fae8c75ad0 radv/rt: set additions for scratch access as NUW
No fossil-db changes.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20117>
2022-12-06 15:23:38 +00:00
Marek Olšák
b75b9d6324 ac/llvm: set invariant.load on image loads with ACCESS_CAN_REORDER
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146>
2022-12-06 13:27:16 +00:00
Marek Olšák
de3fcc77c7 ac/llvm: inline ac_get_load_intr_attribs
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146>
2022-12-06 13:27:16 +00:00
Marek Olšák
ce860953a6 ac/llvm: rename attribute enums
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146>
2022-12-06 13:27:16 +00:00