Commit graph

6280 commits

Author SHA1 Message Date
Marek Olšák
d3a03d24fc ac/llvm: don't set "readnone" on non-memory intrinsics
It's illegal and LLVM always knows which intrinsics don't read memory.
This started failing IR validation with LLVM 16.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20146>
2022-12-06 13:27:16 +00:00
Samuel Pitoiset
a42f8d49c3 radv: use LATE_Z for depth/stencil attachments used in feedback loops
To make sure shader invocations read the correct values.

Fixes dEQP-VK.rasterization.rasterization_order_attachment_access.*.samples_*.multi_draw_barriers

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19728>
2022-12-06 10:34:11 +01:00
Samuel Pitoiset
86c3b9df12 radv: apply register changes for SQTT on GFX11
Based on the register db, most of them moved.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Acked-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20089>
2022-12-06 08:19:24 +01:00
Konstantin Seurer
b49e7cb3f4 radv: Add missing radeon_check_space() in radv_cmd_buffer_after_draw()
Also adds a missing assert that cdw <= cdw_max.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20149>
2022-12-05 17:19:01 +00:00
Konstantin Seurer
d86f60d21e radv: Use wave32 for ray queries inside compute shaders
Results in a 6% performance improvement with Quake II RTX.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20143>
2022-12-05 16:46:26 +00:00
Samuel Pitoiset
da32cbb5c6 aco: fix missing uses of MRT output flags
Fixes regressions on GFX6 and the RAGE2 workaround.

Fixes: a297ac10a4 ("radv,aco: stop lowering FS outputs in NIR")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20154>
2022-12-05 15:01:19 +00:00
Samuel Pitoiset
b051719b05 radv: do not set ZPASS_INCREMENT_DISABLE on GFX11
This field no longer exists.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20090>
2022-12-05 12:13:29 +01:00
Samuel Pitoiset
3ab9218820 radv: fix SPI_SHADER_Z_FORMAT for alpha-to-coverage via MRTZ on GFX11
It should select a 32-bit format with alpha.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20126>
2022-12-05 08:22:28 +00:00
Samuel Pitoiset
a297ac10a4 radv,aco: stop lowering FS outputs in NIR
This was a bad idea because:
- it diverges too much with the fragment shader epilog
- it doesn't allow to implement alpha-to-coverage via MRTZ correctly
- it was supposed to be used by LLVM but this never happened

Reverting this back allows us to fix alpha-to-coverage via MRTZ
on GFX11 easily, including for fragment shader epilogs.

fossils-db (NAVI21):
Totals from 20411 (15.13% of 134913) affected shaders:
VGPRs: 972056 -> 971400 (-0.07%); split: -0.08%, +0.01%
CodeSize: 92284804 -> 92295392 (+0.01%); split: -0.05%, +0.06%
MaxWaves: 465010 -> 465166 (+0.03%); split: +0.03%, -0.00%
Instrs: 17034162 -> 17034963 (+0.00%); split: -0.00%, +0.01%
Latency: 252013190 -> 251971764 (-0.02%); split: -0.03%, +0.02%
InvThroughput: 45859625 -> 45842556 (-0.04%); split: -0.04%, +0.01%
VClause: 324627 -> 324629 (+0.00%); split: -0.03%, +0.03%
SClause: 672918 -> 672826 (-0.01%); split: -0.05%, +0.04%
Copies: 1172126 -> 1158152 (-1.19%); split: -1.20%, +0.01%
Branches: 420602 -> 420604 (+0.00%); split: -0.00%, +0.00%
PreSGPRs: 1025441 -> 1025481 (+0.00%)
PreVGPRs: 861787 -> 860650 (-0.13%); split: -0.17%, +0.03%

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20126>
2022-12-05 08:22:28 +00:00
Samuel Pitoiset
664aa7a37b radv: fix emitting invalid color attachments
Note sure how this happened.

Fixes: 97dc28b177 ("radv: fix configuring COLOR_INVALID on GFX11")
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20127>
2022-12-05 07:46:47 +00:00
Konstantin Seurer
ad8de42ce5 radv: Use get_first_non_void_channel more often
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18634>
2022-12-02 22:06:11 +00:00
Konstantin Seurer
6397304519 radv: Only create bvh pipelines when using rt
Saves some time when creating non-rt devices.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20110>
2022-12-02 21:14:00 +00:00
Konstantin Seurer
7fe515f6d4 radv/rra: Get rid of annoying memory aliasing warning
Such cursed behavior is almost non existent in practise. When capturing
a Doom Eternal, this warning spams the output for no reason.

The warning is also unnecessary since we copy acceleration structures
right after building them now.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
e2b7e478a5 radv/rra: Fix setting some offsets
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
79dcacfc04 radv/rra: Refactor rra_fill_accel_struct_header_common
No need to re-do the offset calculation for every field.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
bb6b45e26e radv/rra: Set the metadata size correctly
Fixes: 5749806 ("radv: Add Radeon Raytracing Analyzer trace dumping utilities")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
0e3325dfb6 radv/rra: Remove an obsolete comment
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
94ec359ae5 radv/rra: Defer destroying accel struct data
This allows us to dump acceleration structures that were destroyed
before present.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
ae9c65a552 radv/rra: Copy accel structs directly after build
This is the second step of decoupling acceleration structure dumping
from lifetimes. It also simplifies the logic a bit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Konstantin Seurer
08a85076e5 radv/rra: Introduce radv_rra_accel_struct_data
This will be useful for dumping acceleration structures that were
destroyed before submit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20047>
2022-12-02 16:48:07 +00:00
Samuel Pitoiset
ab7f518ed0 radv,driconf: fix static driconf by parsing 00-radv-defaults.conf
Otherwise when xmlconfig is disabled, drirc workarounds aren't applied
with RADV.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/7785
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20077>
2022-12-01 16:55:31 +00:00
Bas Nieuwenhuizen
9a311a1891 radv: Remove the old LBVH shader.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19891>
2022-12-01 02:20:48 +00:00
Bas Nieuwenhuizen
5ba950eb14 radv: Switch to new LBVH implementation.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19891>
2022-12-01 02:20:48 +00:00
Bas Nieuwenhuizen
ea159e47a5 radv: Add new LBVH shaders.
Contrary to the previous implementation, this actually implements an LBVH builder.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19891>
2022-12-01 02:20:48 +00:00
Bas Nieuwenhuizen
f531f671ef radv: Handle nodes with 2 invalid children in internal node converter.
Fixes: 682dc5c28e ("radv: Add conversion shader for internal nodes")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19891>
2022-12-01 02:20:48 +00:00
Lionel Landwerlin
863516dd63 radv: enable lower shader call vectorizing
Totals from 5 (71.43% of 7) affected shaders:
MaxWaves: 48 -> 50 (+4.17%)
Instrs: 32012 -> 32070 (+0.18%)
CodeSize: 172672 -> 172932 (+0.15%)
VGPRs: 512 -> 496 (-3.12%)
Latency: 715333 -> 715279 (-0.01%); split: -0.03%, +0.02%
InvThroughput: 149540 -> 146150 (-2.27%); split: -2.29%, +0.02%
VClause: 900 -> 897 (-0.33%)
Copies: 4173 -> 4199 (+0.62%); split: -0.12%, +0.74%
Branches: 1512 -> 1511 (-0.07%)
PreVGPRs: 475 -> 469 (-1.26%)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20075>
2022-11-30 11:13:13 +00:00
Samuel Pitoiset
96332b3433 radv: stop emitting R_00B8A0_COMPUTE_PGM_RSRC3 from the CS preamble
It will be always emitted as part of the compute pipeline.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20054>
2022-11-30 09:05:13 +00:00
Samuel Pitoiset
80072df824 radv: disable VRS entirely on GFX11
Based on registers, VRS changed a lot and it's unclear how to program
it. This disable VK_KHR_fragment_shading_rate, VRS flat shading and
RADV_FORCE_VRS.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20054>
2022-11-30 09:05:13 +00:00
Samuel Pitoiset
5d552b4f6c radv: do not enable NGG culling on GFX11
RadeonSI disables it as well. It's really unclear if it will help or
not (eg. NGG culling never helped on GFX10).

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20054>
2022-11-30 09:05:13 +00:00
Samuel Pitoiset
84a7138d3c radv: do not enable DCC for MSAA images without FMASK
I don't know how this is supposed to work, especially for fast clears
because CMASK should be cleared to 0xC but FMASK implies CMASK.
This fixes a bunch of MSAA test failures on GFX10.3 with
RADV_DEBUG=nofmask.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20054>
2022-11-30 09:05:13 +00:00
Samuel Pitoiset
9b637aa9a1 radv: set INTERPOLATE_COMP_Z to 0 on GFX11
Ported from RadeonSI to fix a EQAA bug.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20054>
2022-11-30 09:05:13 +00:00
Samuel Pitoiset
62715a6d03 radv: set missing SPI_SHADER_PGM_xxx registers on GFX11
Found by inspection.

Cc: 22.3 mesa-stable
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20054>
2022-11-30 09:05:13 +00:00
Samuel Pitoiset
07dc402a3f radv: inline gfx10_emit_streamout_{begin,end}()
Instead of having 2 different paths.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19632>
2022-11-29 16:50:13 +00:00
Rhys Perry
ae521aaa2a radv: compile GS copy shader after geometry shader
This affects the pipeline feedback durations: GS copy shader compilation
is now included in the GS compilation duration.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18898>
2022-11-29 14:28:11 +00:00
Rhys Perry
0cb48ec3b7 radv,aco: remove old streamout code
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18898>
2022-11-29 14:28:11 +00:00
Rhys Perry
3a96977542 radv,aco: remove old GS copy shader code
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18898>
2022-11-29 14:28:11 +00:00
Rhys Perry
17bd2721e6 radv,aco: implement GS copy shaders using NIR
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18898>
2022-11-29 14:28:11 +00:00
Rhys Perry
3ab471a87d radv: make radv_use_llvm_for_stage device parameter const
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18898>
2022-11-29 14:28:11 +00:00
Rhys Perry
12becb8839 radv: lower streamout in NIR
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18898>
2022-11-29 14:28:11 +00:00
Rhys Perry
19d0403594 radv,aco: export legacy vertex outputs in NIR
This new behaviour will let us insert exports in GS copy shader control
flow.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18898>
2022-11-29 14:28:11 +00:00
Samuel Pitoiset
7e287609e3 radv: use HTILE for VRS image only on GFX10.3
Based on registers, GFX11 no longer uses HTILE.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19962>
2022-11-29 12:19:40 +00:00
Samuel Pitoiset
e3d3fb2e69 radv: do not clear VRS_HTILE_ENCODING on GFX11
This field no longer exists.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19962>
2022-11-29 12:19:40 +00:00
Rhys Perry
974358a8c1 radv/llvm: add clip distance outputs manually if they're missing
It's possible that undef is written to clip/cull distance outputs and
they're eliminated, and we never set any position export to done=1 because
outinfo->pos_exports was calculated with the expectation that clip/cull is
exported.

Eliminating the export and fixing the done=1 bit hangs because the
hardware is still expecting clip/cull distances.

Fixes dEQP-VK.rasterization.provoking_vertex.transform_feedback.first.line_list
hang with LLVM.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19200>
2022-11-28 19:18:50 +00:00
Samuel Pitoiset
aa545a0012 radv: reduce CPU overhead when emitting streamout descriptors
Only the last VGT stage can have streamout.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19723>
2022-11-28 17:59:51 +00:00
Samuel Pitoiset
db7dcc4567 radv: only install 00-radv-defaults.conf if xmlconfig is used
RADV has its own drirc file. Make sure it also uses the xmlconfig
enable option.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Alyssa Rosenzweig <alyssa@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20029>
2022-11-28 16:19:47 +00:00
Samuel Pitoiset
a6d7d2a525 radv: emit DB_RENDER_OVERRIDE2 as part of the framebuffer
The number of samples from the image has to match the number of samples
from the pipeline anyways.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19906>
2022-11-28 15:20:59 +00:00
Konstantin Seurer
c5d91ab254 radv/rt: Check space before emitting descriptors
Found by inspection.

cc: mesa-stable

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20020>
2022-11-27 18:45:41 +00:00
Konstantin Seurer
0966fb2c10 radv/ray_queries: Fix AABB handling
AABB intersections always have to be committed manually.
-> We don't have to copy opaque ones to closest.

It's also invalid to query t for candidate AABBs.

Totals from 7 (14.29% of 49) affected shaders:
CodeSize: 171008 -> 169672 (-0.78%)
Instrs: 32499 -> 32250 (-0.77%); split: -0.78%, +0.01%
Latency: 418859 -> 414759 (-0.98%); split: -0.98%, +0.00%
InvThroughput: 89182 -> 88251 (-1.04%); split: -1.05%, +0.00%
VClause: 602 -> 599 (-0.50%)
SClause: 837 -> 835 (-0.24%)
Copies: 4804 -> 4802 (-0.04%); split: -0.35%, +0.31%
Branches: 1593 -> 1585 (-0.50%)
PreSGPRs: 567 -> 566 (-0.18%)

Fixes: 3f72061 ("radv/rq: Use the common traversal helper")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19698>
2022-11-27 12:57:02 +01:00
Konstantin Seurer
409556f048 radv/ray_queries: Move some initialization outside a likely branch
Gets rid of some copies.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19698>
2022-11-27 12:57:02 +01:00
Bas Nieuwenhuizen
a97a6d0f0e radv: reserve space for the scissor in vkCmdBeginRendering.
Fixes: c7d0d328d5 ("radv: Set the window scissor to the render area, not framebuffer")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20014>
2022-11-26 14:35:54 +00:00