The structure will grow in later commits.
The major change is that the preheader and exit blocks are replaced
by tracking just the innermost optimized nir_loop * and getting the
predecessor and successor blocks out of it.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41220>
ac_nir_optimize_outputs() might pack user varyings into the color
built-ins. If this happens we skip adding clamping to the
components that contain the user varying.
This change also fixes a second bug where a color built-in can be
packed into a non-color slot and was no longer being clamped.
Fixes: 3777a5d7 ("radeonsi: assign param export indices before compilation")
Closes: #14443
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40594>
No intended functional change.
This prevents possible breakage due to DCE removing input loads followed
by nir_shader_gather_info updating input masks and changing the result of
ac_nir_get_io_driver_location after PS input register contents are already
determined.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41175>
This is similar to
75ede9d9bc ("intel/brw: track last successful pass and leave the loop early")
except that it uses the common nir helpers.
Note that I've also marked nir_opt_peephole_select as NOT_IDEMPOTENT
because I'm skeptical that it actually is idempotent. This differs from
both brw and radv.
I'm also marking gcm as not idempotent because it isn't idempotent in
practice on one of the shaders in my shader-db:
2bf4ba7133/fossils/blender
pipeline hash 0e972f8e349af903
This is about a 4% geomean compile time speedup on my local collection
of shaders.
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41118>
This moves from deprecated stage_id/hw_queue_id to per-context
stage_iid/hw_queue_iid, which leads to separate timelines per app.
There are several benefits to this:
- Different driver versions could be used by different apps and perfetto
won't confuse tracepoints.
- Tracepoints from different apps may not align perfectly, so previously
we got a fair amount of weird vertical ordering of tracepoints.
The downside is that info is spread across several timelines multiplied
by queues, but I think that's better since it is easier to understand
which tracepoints correspond to which app.
The changes are mostly copied from radeon/intel perfetto integration.
This also fixes app_event emission along the way, previously
debug_marker_stage was called _before_ SEQ_INCREMENTAL_STATE_CLEARED.
Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41105>
Add latest maintenance extensions required by Android Vulkan
requirements, except VK_KHR_maintenance5 which enables dynamic rendering
on ANGLE and causes issues with some cuttlefish targets.
Test: CI
Reviewed-by: Aaron Ruby <aruby@qnx.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41210>
On my Arc A380 (DG2), this more than doubles the performance of Jeff
Bolz's cooperative matrix benchmark. With llama.cpp modified to use
cooperative matrix on DG2, performance is improved by 37%.
Closes: #15311
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Tested-by: Matt Corallo <git@bluematt.me>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41172>
The next commit will cause some very specific phis to not be lowered to
scalar, and that's the reason the callback is used instead of
nir_lower_all_phis_to_scalar.
It's worth noting that the comment in nir_lower_phis_to_scalar.c
specifically calls out Deus Ex as the reason some phis should not be
lowered. At least on current BRW, zero shaders from Deus Ex trace were
affected for spills or fills on any Intel platform.
shader-db:
All Intel platforms had similar results. (Lunar Lake shown)
total instructions in shared programs: 17050005 -> 17051449 (<.01%)
instructions in affected programs: 41032 -> 42476 (3.52%)
helped: 29 / HURT: 159
total cycles in shared programs: 876411976 -> 876433702 (<.01%)
cycles in affected programs: 1455550 -> 1477276 (1.49%)
helped: 40 / HURT: 150
fossil-db:
All Intel platforms had similar results. (Lunar Lake shown)
Totals:
Instrs: 916599633 -> 916694854 (+0.01%); split: -0.00%, +0.01%
CodeSize: 14705971792 -> 14708302384 (+0.02%); split: -0.00%, +0.02%
Send messages: 40870114 -> 40870113 (-0.00%)
Cycle count: 102360965889 -> 102364169753 (+0.00%); split: -0.00%, +0.01%
Spill count: 3460669 -> 3460240 (-0.01%)
Fill count: 4988325 -> 4987891 (-0.01%)
Max live registers: 192914542 -> 192918153 (+0.00%); split: -0.00%, +0.00%
Max dispatch width: 48848112 -> 48848128 (+0.00%)
Non SSA regs after NIR: 141633613 -> 141671589 (+0.03%); split: -0.00%, +0.03%
Totals from 5713 (0.28% of 2010434) affected shaders:
Instrs: 5215921 -> 5311142 (+1.83%); split: -0.09%, +1.91%
CodeSize: 88940784 -> 91271376 (+2.62%); split: -0.20%, +2.82%
Send messages: 284751 -> 284750 (-0.00%)
Cycle count: 275671864 -> 278875728 (+1.16%); split: -0.74%, +1.90%
Spill count: 857 -> 428 (-50.06%)
Fill count: 845 -> 411 (-51.36%)
Max live registers: 667776 -> 671387 (+0.54%); split: -0.86%, +1.40%
Max dispatch width: 160416 -> 160432 (+0.01%)
Non SSA regs after NIR: 1127904 -> 1165880 (+3.37%); split: -0.10%, +3.47%
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Tested-by: Matt Corallo <git@bluematt.me>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41172>
Enable hardware decode/encode capabilities for VCN 5.3.0 by
configuring the supported codec list. This allows vainfo to
properly enumerate available codec capabilities.
Signed-off-by: Suresh Guttula <suresh.guttula@amd.com>
Reviewed-by: David Rosca <david.rosca@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41202>
Enable VK_KHR_performance_query on GFX11 (RDNA3 / RDNA3.5) now that the
selector tables and packet emission are in place.
Tested on Strix Halo with dEQP-VK.query_pool.performance_query.* (6 pass,
6 not-supported for the allowCommandBufferQueryCopies cases).
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41157>
GFX11 reorganizes the shader perfcounter blocks: wave counts move from
SQ to the SQG registers (still mapped as the SQ block in ac/), while
per-instruction counters move from SQ to the new SQ_WGP block.
Add GFX11-specific selector enums using the new block assignments and
branch radv_query_perfcounter_descs to select them on GFX11+. GL2C,
GL1C, and TCP selectors are unchanged between GFX10.3 and GFX11.
The "Instructions" (total count) counter is dropped on GFX11 as there
is no direct SQ_WGP equivalent for INSTS_ALL.
Selector indices verified against gpu_performance_api's
gpa_hw_counter_gfx11.cc.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41157>
Some perfcounter blocks (e.g. SQ_WGP on GFX11) define num_spm_modules
but have no select1 register array. Skip the select1 loop when the
array is NULL.
This is a prerequisite for enabling performance queries on GFX11.
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41157>
Casting a negative float to uint64_t is undefined behavior. GCC 15 with
-O2 produces 0xFFFFFFFFFFFFFFFF for (uint64_t)(-32767.5f), causing snorm
clear values to be packed incorrectly (e.g. 0xFFFF instead of 0x8001 for
snorm16 -1.0). This results in wrong DCC comp-to-single clear colors and
~966 CTS snorm multisample_resolve test failures.
Fix by casting through int64_t first, which is well-defined (truncation
toward zero) and preserves the two's complement bit pattern.
Fixes: 585c25be1e ("radv: fix color conversions for normalized uint/sint formats")
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41164>
For invalid streams tile cols and rows may be higher than 64.
This would overwrite data after the height_in_sbs array, but since
the maximum amount of bytes overwritten is bound by the maximum
supported decode resolution, this can't overwrite any important
fields and thus won't cause any observable issue.
As this can only happen with invalid streams, it still won't decode
correctly with this fixed.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/work_items/15290
Reviewed-by: Benjamin Cheng <benjamin.cheng@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41016>
This should only consider valid entries, not loop over the entire array.
In addition the array size was wrong before.
Fixes: 779edc0759 ("frontends/va: Correctly derive HEVC StCurrBefore, StCurrAfter and LtCurr")
Reviewed-by: Benjamin Cheng <benjamin.cheng@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41016>
When r300_pick_vertex_shader switches to a WPOS variant, it only dirtied
rs_block_state, leaving vs_state with a stale code size. This caused
cs_count warnings (offset of -4 for one extra VS instruction) but was
mostly harmless since the emitted packet stream still used the current
shader.
Factor the VS code dirtying out of r300_bind_vs_state into a helper and
call it when selecting a new variant too.
Fixes: 806dcf9db7 ("r300: only output wpos in vertex shaders when needed")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41200>
r300_simple_msaa_resolve used to patch srcsurf->pitch with the resolve
destination's tiling bits before passing the surface to the blitter.
That worked when set_framebuffer_state kept the same pipe_surface
pointer, so r300_get_nonnull_cb returned the patched object.
After the de-pointerization, r300_framebuffer_init creates a fresh
r300_surface from the pipe_surface template, discarding the pitch
modification. The hardware then uses the MSAA source tiling for
R300_RB3D_COLORPITCH0, leading to corruption.
Move the tiling override into r300_emit_fb_state and override the tiling
bits of COLORPITCH from the destination surface at emit time.
Fixes: 2eb45daa9c ("gallium: de-pointerize pipe_surface")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/work_items/15303
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41092>
Sampling coeffs with trilinear filtering will output 2x sets of data.
Whether bilinear or trilinear filtering is in use can't be determined
without checking state words, so unconditionally reserve 2x to avoid
clobbering output regs.
Fixes: 7df32ba09d ("pco: initial texture/sampler compiler support")
Signed-off-by: Simon Perretta <simon.perretta@imgtec.com>
Acked-by: Frank Binns <frank.binns@imgtec.com>
Tested-by: Icenowy Zheng <zhengxingda@iscas.ac.cn>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41051>
The Vulkan spec says:
"After a call to vkCmdExecuteGeneratedCommandsEXT, command buffer
state will become undefined according to the tokens executed. This
table specifies the relationship between tokens used and state
invalidation."
The application must re-bind the states that are updated using DGC.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41159>
Only if DGC emits an indexed draw without providing the index buffer
as part of the tokens. This avoids emitting useless packets.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41159>
Save protected radv_queue in device->queues_protected so it can be
relased in radv_destroy_device.
Without this, device->queues[] will point to a new queues to record
the queue created according to the queueCreateInfo. When queueCreateInfo
include both protected and unprotected queue info, the device->queues[]
will be created twice and record one queue at each time. So we will lose
either protected queue or unprotected queue which created first.
Signed-off-by: Julia Zhang <Julia.Zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40619>
Create encrypted fence_bo and eop_bug_bo when the radv_cmd_buffer is
created from a protected pool which is marked with flag
VK_COMMAND_POOL_CREATE_PROTECTED_BIT
Signed-off-by: Julia Zhang <Julia.Zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40619>
Pass secure flag to radv_update_preamble_cs() if this queue is created
with vk flag VK_QUEUE_PROTECTED_BIT and create encrypted tess/ge ring
BOs and compute_scratch_bo according to this secure flag.
Signed-off-by: Julia Zhang <Julia.Zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40619>
