Add infrastructure to reconstruct if's. Later in the series, this will let us
hoist loads from inside uniform if's without speculating. For now, it lets us
handle phi's in nir_opt_preamble in a straightforward way.
Results on AGX are good:
total instructions in shared programs: 1504730 -> 1499674 (-0.34%)
instructions in affected programs: 153673 -> 148617 (-3.29%)
helped: 496
HURT: 0
Instructions are helped.
total bytes in shared programs: 10287768 -> 10238284 (-0.48%)
bytes in affected programs: 1113724 -> 1064240 (-4.44%)
helped: 496
HURT: 0
Bytes are helped.
total halfregs in shared programs: 452669 -> 452049 (-0.14%)
halfregs in affected programs: 14825 -> 14205 (-4.18%)
helped: 152
HURT: 99
Halfregs are helped.
total threads in shared programs: 16469504 -> 16470784 (<.01%)
threads in affected programs: 8960 -> 10240 (14.29%)
helped: 10
HURT: 0
Threads are helped.
Results on ir3 is a bit more of a wash but still should be a win overall: The
regression in moves seems scary, but the cost model already accounts for them as
evidenced by instruction count coming out ahead.
total instructions in shared programs: 3108750 -> 3105993 (-0.09%)
instructions in affected programs: 317367 -> 314610 (-0.87%)
helped: 675
HURT: 242
Instructions are helped.
total nops in shared programs: 673152 -> 675048 (0.28%)
nops in affected programs: 74551 -> 76447 (2.54%)
helped: 353
HURT: 347
Inconclusive result (%-change mean confidence interval includes 0).
total non-nops in shared programs: 2435598 -> 2430945 (-0.19%)
non-nops in affected programs: 232664 -> 228011 (-2.00%)
helped: 816
HURT: 38
Non-nops are helped.
total mov in shared programs: 78201 -> 84011 (7.43%)
mov in affected programs: 10726 -> 16536 (54.17%)
helped: 60
HURT: 781
Mov are HURT.
total cov in shared programs: 74964 -> 74906 (-0.08%)
cov in affected programs: 273 -> 215 (-21.25%)
helped: 17
HURT: 0
Cov are helped.
total dwords in shared programs: 6716814 -> 6748726 (0.48%)
dwords in affected programs: 879778 -> 911690 (3.63%)
helped: 12
HURT: 948
Dwords are HURT.
total full in shared programs: 193210 -> 193212 (<.01%)
full in affected programs: 278 -> 280 (0.72%)
helped: 12
HURT: 22
Inconclusive result (value mean confidence interval includes 0).
total constlen in shared programs: 493632 -> 494816 (0.24%)
constlen in affected programs: 19904 -> 21088 (5.95%)
helped: 9
HURT: 306
Constlen are HURT.
total cat0 in shared programs: 742476 -> 745046 (0.35%)
cat0 in affected programs: 84455 -> 87025 (3.04%)
helped: 277
HURT: 489
Cat0 are HURT.
total cat1 in shared programs: 153303 -> 159059 (3.75%)
cat1 in affected programs: 17810 -> 23566 (32.32%)
helped: 69
HURT: 780
Cat1 are HURT.
total cat2 in shared programs: 1144508 -> 1140731 (-0.33%)
cat2 in affected programs: 121284 -> 117507 (-3.11%)
helped: 841
HURT: 0
Cat2 are helped.
total cat3 in shared programs: 942098 -> 934804 (-0.77%)
cat3 in affected programs: 87140 -> 79846 (-8.37%)
helped: 855
HURT: 1
Cat3 are helped.
total cat4 in shared programs: 65261 -> 65249 (-0.02%)
cat4 in affected programs: 42 -> 30 (-28.57%)
helped: 12
HURT: 0
Cat4 are helped.
total sstall in shared programs: 237311 -> 241281 (1.67%)
sstall in affected programs: 33755 -> 37725 (11.76%)
helped: 179
HURT: 493
Sstall are HURT.
total (ss) in shared programs: 58166 -> 58795 (1.08%)
(ss) in affected programs: 4535 -> 5164 (13.87%)
helped: 35
HURT: 664
(ss) are HURT.
total systall in shared programs: 503784 -> 503805 (<.01%)
systall in affected programs: 3170 -> 3191 (0.66%)
helped: 16
HURT: 13
Inconclusive result (value mean confidence interval includes 0).
total (sy) in shared programs: 27261 -> 27259 (<.01%)
(sy) in affected programs: 76 -> 74 (-2.63%)
helped: 8
HURT: 5
Inconclusive result (value mean confidence interval includes 0).
total waves in shared programs: 439848 -> 439872 (<.01%)
waves in affected programs: 160 -> 184 (15.00%)
helped: 12
HURT: 0
Waves are helped.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24011>
The way backends walk NIR when translating. This will make it easy to filter
can_move based on the parent control flow.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24011>
It can be beneficial to move phi nodes, even though they can often be coalesced.
Model this cost so nir_opt_preamble can make good decisions about hoisting phi
nodes (and by extension, if-statements) into the preamble.
At this point in the series, this has no effect, but it will avoid certain
shader-db regressions associated with the nir_opt_preamble changes later in the
series.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24011>
Speculating these loads is safe, but nir_opt_preamble doesn't know that. Set the
ACCESS bits appropriately to let it know.
This will avoid any code gen regression from the nir_opt_preamble change.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24011>
Determining whether it is safe to hoist a load instruction out of control flow
depends on complex hardware and driver details. Rather than encoding this as
knobs in every NIR pass that wants to do so (notably nir_opt_preamble and
nir_opt_peephole_select), add a per-load ACCESS flag for backends to set.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24011>
The type is called vk_object_base, not vk_vk_objet_base... This should
fix the cross-referencing of this type.
Fixes: f6d4641433 ("vulkan,docs: Document vk_instance")
Reviewed-by: Karmjit Mahil <Karmjit.Mahil@imgtec.com>
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25634>
Right now, S3 always returns something, so we need to check
the content-type .
Acked-by: Emma Anholt <emma@anholt.net>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25606>
The way we init render pass related structures is dangerous with when
structs are not zero initialized - too easy to miss a field. There
were already at least two issues with it.
Signed-off-by: Danylo Piliaiev <dpiliaiev@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25592>
This allows us to pack the is_if boolean into the bottom bit of the parent
pointer, eliminating the boolean and hence shrinking the nir_src by 8 bytes (due
to the extra 63 bits of padding incurred in the old layout).
Because all access is forced through helpers now, this is a local change.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671>
It is undefined behaviour in C to read a different member of a union than was
written. Nothing in-tree should be using this behaviour with the nir_src union:
nir_if should never be read as nir_instr and vice versa. Assert this.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671>
First, we need to give the parent_instr field a unique name to be able to
replace with a helper. We have parent_instr fields for both nir_src and
nir_def, so let's rename nir_src::parent_instr in preparation for rework.
This was done with a combination of sed and manual fix-ups.
Then we use semantic patches plus manual fixups:
@@
expression s;
@@
-s->renamed_parent_instr
+nir_src_parent_instr(s)
@@
expression s;
@@
-s.renamed_parent_instr
+nir_src_parent_instr(&s)
@@
expression s;
@@
-s->parent_if
+nir_src_parent_if(s)
@@
expression s;
@@
-s.renamed_parent_if
+nir_src_parent_if(&s)
@@
expression s;
@@
-s->is_if
+nir_src_is_if(s)
@@
expression s;
@@
-s.is_if
+nir_src_is_if(&s)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671>
These will become nontrivial later in the series. For now these have no smarts
in them, in order to make the conversion completely mechanical.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671>
It is invalid to read parent_instr for an if-use (or parent_if for a
non-if-use). Make sure we read the right one when handling if-uses.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671>
This re-introduces "radv: fix alignment of DGC command buffers" and
"radv/amdgpu: fix alignment of command buffers" which were valid
changes.
IBs need to be aligned to the IB size requirement, not the number of
padded NOPs.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25588>
This enables some extensions and a bunch of formats for ycbcr
support.
dEQP-VK.api.info.format_properties.g8_b8_r8_3plane_420_unorm,Fail
dEQP-VK.api.info.format_properties.g8_b8r8_2plane_420_unorm,Fail
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25609>
this just introduces a single plane and refactors code to use it.
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25609>
This should make no functional difference, except cleanup the code.
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25609>
This just makes things a bit cleaner, and reuses the common code.
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25609>
Since we can't sample from these directly, just don't create the
functions.
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25609>
dEQP-VK.api.copy_and_blit.copy_commands2.blit_image.simple_tests.array.all_remaining_layers
Fixes: 35c02f79c9 ("lavapipe: add some asserts for blit region extents")
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Roland Scheidegger <sroland@vmware.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25609>
Next part don't know whether p_end_with_regs args are loaded from
memory ops or not, need to wait it's done here.
Other memory load needs to be waited too like:
a = load_mem()
b = ...
if (...) {
wait_mem(a)
store_mem(a)
}
p_end_with_regs(b)
"a" still needs to be waited, otherwise next shader part regs may
be overwritten by unfinished memory loads.
Memory stores are waited too. When >=gfx10 and last VGT has no
parameter export, we need to wait all memeory stores done before
pos export (see ac_nir_export_position). So when merged shader
(ES+GS or VS+GS) is partially built, first stage needs to wait
all memory stores done, otherwise second stage don't know if
any memory stores pending before.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Signe-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24973>
