fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2026-05-18 13:48:06 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
mesa/src/compiler/nir/nir_control_flow.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

975 lines
29 KiB
C
Raw Normal View History

nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
/*
* Copyright © 2014 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*
* Authors:
* Connor Abbott (cwabbott0@gmail.com)
*
*/
#include "nir_control_flow_private.h"
/**
* \name Control flow modification
*
* These functions modify the control flow tree while keeping the control flow
* graph up-to-date. The invariants respected are:
* 1. Each then statement, else statement, or loop body must have at least one
* control flow node.
* 2. Each if-statement and loop must have one basic block before it and one
* after.
* 3. Two basic blocks cannot be directly next to each other.
* 4. If a basic block has a jump instruction, there must be only one and it
* must be at the end of the block.
*
* The purpose of the second one is so that we have places to insert code during
* GCM, as well as eliminating the possibility of critical edges.
*/
/*@{*/
static inline void
block_add_pred(nir_block *block, nir_block *pred)
{
_mesa_set_add(block->predecessors, pred);
}
nir/cf: Make a helper function for removing a predecessor. I need to do this in a second place, and I'd rather make a helper function than cut and paste the code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-09-03 00:31:19 -07:00
static inline void
block_remove_pred(nir_block *block, nir_block *pred)
{
struct set_entry *entry = _mesa_set_search(block->predecessors, pred);
assert(entry);
_mesa_set_remove(block->predecessors, entry);
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
static void
link_blocks(nir_block *pred, nir_block *succ1, nir_block *succ2)
{
pred->successors[0] = succ1;
nir/cf: fix link_blocks() when there are no successors When we insert a single basic block A into another basic block B, we will split B into C and D, insert A in the middle, and then splice together C, A, and D. When we splice together C and A, we need to move the successors of A into C -- except A has no successors, since it hasn't been inserted yet. So in move_successors(), we need to handle the case where the block whose successors are to be moved doesn't have any successors. Fixing link_blocks() here prevents a segfault and makes it work correctly. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:29 -07:00
if (succ1 != NULL)
block_add_pred(succ1, pred);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
pred->successors[1] = succ2;
if (succ2 != NULL)
block_add_pred(succ2, pred);
}
static void
unlink_blocks(nir_block *pred, nir_block *succ)
{
if (pred->successors[0] == succ) {
pred->successors[0] = pred->successors[1];
pred->successors[1] = NULL;
} else {
assert(pred->successors[1] == succ);
pred->successors[1] = NULL;
}
nir/cf: Make a helper function for removing a predecessor. I need to do this in a second place, and I'd rather make a helper function than cut and paste the code. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-09-03 00:31:19 -07:00
block_remove_pred(succ, pred);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
}
static void
unlink_block_successors(nir_block *block)
{
if (block->successors[1] != NULL)
unlink_blocks(block, block->successors[1]);
nir/cf: Fix unlink_block_successors to actually unlink the second one. Calling unlink_blocks(block, block->successors[0]) will successfully unlink the first successor, but then will shift block->successors[1] down to block->successor[0]. So the successors[1] != NULL check will always fail. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-09-01 22:56:29 -07:00
if (block->successors[0] != NULL)
unlink_blocks(block, block->successors[0]);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
}
static void
link_non_block_to_block(nir_cf_node *node, nir_block *block)
{
if (node->type == nir_cf_node_if) {
/*
* We're trying to link an if to a block after it; this just means linking
* the last block of the then and else branches.
*/
nir_if *if_stmt = nir_cf_node_as_if(node);
nir: Make nir_foo_first/last_cf_node return a block instead One of NIR's invariants is that control flow lists always start and end with blocks. There's no good reason why we should return a cf_node from these functions since we know that it's always a block. Making it a block lets us remove a bunch of code. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2016-10-05 19:08:57 -07:00
nir_block *last_then_block = nir_if_last_then_block(if_stmt);
nir_block *last_else_block = nir_if_last_else_block(if_stmt);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
nir: Pull block_ends_in_jump into nir.h We had two different implementations in different files. May as well have one and put it in nir.h. Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
2018-08-24 09:34:05 -05:00
if (!nir_block_ends_in_jump(last_then_block)) {
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
unlink_block_successors(last_then_block);
link_blocks(last_then_block, block, NULL);
}
nir: Pull block_ends_in_jump into nir.h We had two different implementations in different files. May as well have one and put it in nir.h. Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
2018-08-24 09:34:05 -05:00
if (!nir_block_ends_in_jump(last_else_block)) {
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
unlink_block_successors(last_else_block);
link_blocks(last_else_block, block, NULL);
}
} else {
assert(node->type == nir_cf_node_loop);
}
}
static void
link_block_to_non_block(nir_block *block, nir_cf_node *node)
{
if (node->type == nir_cf_node_if) {
/*
* We're trying to link a block to an if after it; this just means linking
* the block to the first block of the then and else branches.
*/
nir_if *if_stmt = nir_cf_node_as_if(node);
nir: Make nir_foo_first/last_cf_node return a block instead One of NIR's invariants is that control flow lists always start and end with blocks. There's no good reason why we should return a cf_node from these functions since we know that it's always a block. Making it a block lets us remove a bunch of code. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2016-10-05 19:08:57 -07:00
nir_block *first_then_block = nir_if_first_then_block(if_stmt);
nir_block *first_else_block = nir_if_first_else_block(if_stmt);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
unlink_block_successors(block);
link_blocks(block, first_then_block, first_else_block);
nir: Add goto_if jump instruction Signed-off-by: Karol Herbst <kherbst@redhat.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2401>
2020-07-02 14:32:04 +02:00
} else if (node->type == nir_cf_node_loop) {
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
/*
* For similar reasons as the corresponding case in
* link_non_block_to_block(), don't worry about if the loop header has
* any predecessors that need to be unlinked.
*/
nir_loop *loop = nir_cf_node_as_loop(node);
nir: Make nir_foo_first/last_cf_node return a block instead One of NIR's invariants is that control flow lists always start and end with blocks. There's no good reason why we should return a cf_node from these functions since we know that it's always a block. Making it a block lets us remove a bunch of code. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2016-10-05 19:08:57 -07:00
nir_block *loop_header_block = nir_loop_first_block(loop);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
unlink_block_successors(block);
link_blocks(block, loop_header_block, NULL);
}
}
nir/cf: Don't break outer-block successors in split_block_beginning(). Consider the following NIR: block block_0; /* succs: block_1 block_2 */ if (...) { block block_1; ... } else { block block_2; } Calling split_block_beginning() on block_1 would break block_0's successors: link_block() sets both successors of a block, so calling link_block(block_0, new_block, NULL) would throw away the second successor, leaving only /* succ: new_block */. This is invalid: the block before an if statement must have two successors. Changing the call to link_block(pred, new_block, pred->successors[0]) would correctly leave both successors in place, but because unlink_block may shift successor[1] to successor[0], it may not preserve the original order. NIR maintains a convention that successor[0] must point to the "then" block, while successor[1] points to the "else" block, so we need to take care to preserve this ordering. This patch creates a new function that swaps out one successor for another, preserving the ordering. It then uses this to fix the issue. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-09-03 00:33:50 -07:00
/**
* Replace a block's successor with a different one.
*/
static void
replace_successor(nir_block *block, nir_block *old_succ, nir_block *new_succ)
{
if (block->successors[0] == old_succ) {
block->successors[0] = new_succ;
} else {
assert(block->successors[1] == old_succ);
block->successors[1] = new_succ;
}
block_remove_pred(old_succ, block);
block_add_pred(new_succ, block);
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
/**
* Takes a basic block and inserts a new empty basic block before it, making its
* predecessors point to the new block. This essentially splits the block into
* an empty header and a body so that another non-block CF node can be inserted
* between the two. Note that this does *not* link the two basic blocks, so
* some kind of cleanup *must* be performed after this call.
*/
static nir_block *
split_block_beginning(nir_block *block)
{
nir_block *new_block = nir_block_create(ralloc_parent(block));
new_block->cf_node.parent = block->cf_node.parent;
exec_node_insert_node_before(&block->cf_node.node, &new_block->cf_node.node);
set_foreach(block->predecessors, entry) {
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
nir_block *pred = (nir_block *)entry->key;
nir/cf: Don't break outer-block successors in split_block_beginning(). Consider the following NIR: block block_0; /* succs: block_1 block_2 */ if (...) { block block_1; ... } else { block block_2; } Calling split_block_beginning() on block_1 would break block_0's successors: link_block() sets both successors of a block, so calling link_block(block_0, new_block, NULL) would throw away the second successor, leaving only /* succ: new_block */. This is invalid: the block before an if statement must have two successors. Changing the call to link_block(pred, new_block, pred->successors[0]) would correctly leave both successors in place, but because unlink_block may shift successor[1] to successor[0], it may not preserve the original order. NIR maintains a convention that successor[0] must point to the "then" block, while successor[1] points to the "else" block, so we need to take care to preserve this ordering. This patch creates a new function that swaps out one successor for another, preserving the ordering. It then uses this to fix the issue. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-09-03 00:33:50 -07:00
replace_successor(pred, block, new_block);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
}
nir/cf: handle phi nodes better in split_block_beginning() Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:23 -07:00
/* Any phi nodes must stay part of the new block, or else their
nir: keep the phi order when splitting blocks All things being equal is better to keep the original order. Since the new block is empty, push the phis in order to tail. Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Daniel Schürmann <daniel.schuermann@campus.tu-berlin.de>
2019-01-26 01:05:14 -08:00
* sources will be messed up.
nir/cf: handle phi nodes better in split_block_beginning() Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:23 -07:00
*/
nir: Use nir_foreach_phi(_safe) The pattern shows up all the time open-coded. Use the macro instead. Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com> Reviewed-by: Jesse Natalie <jenatali@microsoft.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22967>
2023-05-11 13:20:43 -04:00
nir_foreach_phi_safe(phi, block) {
exec_node_remove(&phi->instr.node);
phi->instr.block = new_block;
exec_list_push_tail(&new_block->instr_list, &phi->instr.node);
nir/cf: handle phi nodes better in split_block_beginning() Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:23 -07:00
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
return new_block;
}
static void
rewrite_phi_preds(nir_block *block, nir_block *old_pred, nir_block *new_pred)
{
nir: Use nir_foreach_phi(_safe) The pattern shows up all the time open-coded. Use the macro instead. Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com> Reviewed-by: Jesse Natalie <jenatali@microsoft.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22967>
2023-05-11 13:20:43 -04:00
nir_foreach_phi_safe(phi, block) {
nir: Switch the arguments to nir_foreach_phi_src This matches the "foreach x in container" pattern found in many other programming languages. Generated by the following regular expression: s/nir_foreach_phi_src(\([^,]*\),\s*\([^,]*\))/nir_foreach_phi_src(\2, \1)/ and a similar expression for nir_foreach_phi_src_safe. Reviewed-by: Eduardo Lima Mitev <elima@igalia.com>
2016-04-26 20:16:21 -07:00
nir_foreach_phi_src(src, phi) {
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
if (src->pred == old_pred) {
src->pred = new_pred;
break;
}
}
}
}
nir/lower_returns: Append missing phis' sources after "break" insertion After we lowered `return` into `break` - the control flow is changed and the block with this change has a new successor, which means that in this new successor phis should have additional source. Since the instructions that use phis in the successor are predicated - it's ok for a new phi source to be undef. If `return` is lowered in a nested loop, `break` is inserted in the outer loops, so all new blocks with break require the same changes to phis described above. Examples of NIR before lowering: block block_0: loop { block block_1: if ssa_2 { block block_2: return // succs: block_6 } else { block block_2: break; // succs: block_5 } block block_4: } block block_5: // preds: block_3 vec1 32 ssa_4 = phi block_3: ssa_1 // succs: block_6 block block_6: Here converting return to break should add block_2 to the phis of block_5. block block_0: loop { block block_1: loop { block block_2: if ssa_2 { block block_3: return // succs: block_8 } else { block block_4: break; // succs: block_6 } block block_5: } block block_6: break; // succs: block_7 } block block_7: // preds: block_6 vec1 32 ssa_4 = phi block_6: ssa_1 // succs: block_8 block block_8: Here converting return to break will insert conditional break in the outer loop, changing block_6 predcessors. Cc: <mesa-stable@lists.freedesktop.org> Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3322 Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3498 Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6186>
2020-08-04 18:12:47 +03:00
void
nir_insert_phi_undef(nir_block *block, nir_block *pred)
nir/cf: add insert_phi_undef() helper Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:19 -07:00
{
nir_function_impl *impl = nir_cf_node_get_function(&block->cf_node);
nir: Use nir_foreach_phi(_safe) The pattern shows up all the time open-coded. Use the macro instead. Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com> Reviewed-by: Jesse Natalie <jenatali@microsoft.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22967>
2023-05-11 13:20:43 -04:00
nir_foreach_phi(phi, block) {
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_undef_instr *undef =
nir_undef_instr_create(impl->function->shader,
nir: Drop nir_dest Instead, we replace every use of it with nir_def. Most of this commit was generated by sed: sed -i -e 's/dest.ssa/def/g' src/**/*.h src/**/*.c src/**/*.cpp A few manual fixups were required in lima and the nir_legacy code. Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 11:56:00 -05:00
phi->def.num_components,
phi->def.bit_size);
nir/cf: add insert_phi_undef() helper Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:19 -07:00
nir_instr_insert_before_cf_list(&impl->body, &undef->instr);
nir: Take a nir_def * in nir_phi_instr_add_src() Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24729>
2023-08-17 15:26:32 -05:00
nir_phi_src *src = nir_phi_instr_add_src(phi, pred, &undef->def);
nir/cf: add insert_phi_undef() helper Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:19 -07:00
list_addtail(&src->src.use_link, &undef->def.uses);
}
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
/**
* Moves the successors of source to the successors of dest, leaving both
* successors of source NULL.
*/
static void
move_successors(nir_block *source, nir_block *dest)
{
nir_block *succ1 = source->successors[0];
nir_block *succ2 = source->successors[1];
if (succ1) {
unlink_blocks(source, succ1);
rewrite_phi_preds(succ1, source, dest);
}
if (succ2) {
unlink_blocks(source, succ2);
rewrite_phi_preds(succ2, source, dest);
}
unlink_block_successors(dest);
link_blocks(dest, succ1, succ2);
}
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
/* Given a basic block with no successors that has been inserted into the
* control flow tree, gives it the successors it would normally have assuming
* it doesn't end in a jump instruction. Also inserts phi sources with undefs
* if necessary.
*/
static void
block_add_normal_succs(nir_block *block)
{
if (exec_node_is_tail_sentinel(block->cf_node.node.next)) {
nir_cf_node *parent = block->cf_node.parent;
if (parent->type == nir_cf_node_if) {
nir_cf_node *next = nir_cf_node_next(parent);
nir_block *next_block = nir_cf_node_as_block(next);
link_blocks(block, next_block, NULL);
nir: Add undef phi srcs when adding successors Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22913>
2023-05-08 17:48:20 -07:00
nir_insert_phi_undef(next_block, block);
nir/cf: Handle relinking top-level blocks This can happen if a function ends in a return instruction and you remove the return. Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2015-12-18 11:27:00 -08:00
} else if (parent->type == nir_cf_node_loop) {
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
nir_loop *loop = nir_cf_node_as_loop(parent);
nir: add Continue Construct to nir_loop The added continue_list corresponds to the SPIR-V Continue Construct and serves as a converged control-flow construct and is executed after each continue statement and before the next iteration of the loop body. Also adds validation rules for loops with Continue Construct Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13962>
2021-12-01 17:34:48 +01:00
nir_block *cont_block;
if (block == nir_loop_last_block(loop)) {
cont_block = nir_loop_continue_target(loop);
} else {
assert(block == nir_loop_last_continue_block(loop));
cont_block = nir_loop_first_block(loop);
}
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
nir: add Continue Construct to nir_loop The added continue_list corresponds to the SPIR-V Continue Construct and serves as a converged control-flow construct and is executed after each continue statement and before the next iteration of the loop body. Also adds validation rules for loops with Continue Construct Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13962>
2021-12-01 17:34:48 +01:00
link_blocks(block, cont_block, NULL);
nir_insert_phi_undef(cont_block, block);
nir/cf: Handle relinking top-level blocks This can happen if a function ends in a return instruction and you remove the return. Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2015-12-18 11:27:00 -08:00
} else {
nir_function_impl *impl = nir_cf_node_as_function(parent);
link_blocks(block, impl->end_block, NULL);
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
}
} else {
nir_cf_node *next = nir_cf_node_next(&block->cf_node);
if (next->type == nir_cf_node_if) {
nir_if *next_if = nir_cf_node_as_if(next);
nir: Make nir_foo_first/last_cf_node return a block instead One of NIR's invariants is that control flow lists always start and end with blocks. There's no good reason why we should return a cf_node from these functions since we know that it's always a block. Making it a block lets us remove a bunch of code. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2016-10-05 19:08:57 -07:00
nir_block *first_then_block = nir_if_first_then_block(next_if);
nir_block *first_else_block = nir_if_first_else_block(next_if);
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
link_blocks(block, first_then_block, first_else_block);
nir: Add undef phi srcs when adding successors Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22913>
2023-05-08 17:48:20 -07:00
nir_insert_phi_undef(first_then_block, block);
nir_insert_phi_undef(first_else_block, block);
nir: Add goto_if jump instruction Signed-off-by: Karol Herbst <kherbst@redhat.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2401>
2020-07-02 14:32:04 +02:00
} else if (next->type == nir_cf_node_loop) {
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
nir_loop *next_loop = nir_cf_node_as_loop(next);
nir: Make nir_foo_first/last_cf_node return a block instead One of NIR's invariants is that control flow lists always start and end with blocks. There's no good reason why we should return a cf_node from these functions since we know that it's always a block. Making it a block lets us remove a bunch of code. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2016-10-05 19:08:57 -07:00
nir_block *first_block = nir_loop_first_block(next_loop);
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
link_blocks(block, first_block, NULL);
nir/lower_returns: Append missing phis' sources after "break" insertion After we lowered `return` into `break` - the control flow is changed and the block with this change has a new successor, which means that in this new successor phis should have additional source. Since the instructions that use phis in the successor are predicated - it's ok for a new phi source to be undef. If `return` is lowered in a nested loop, `break` is inserted in the outer loops, so all new blocks with break require the same changes to phis described above. Examples of NIR before lowering: block block_0: loop { block block_1: if ssa_2 { block block_2: return // succs: block_6 } else { block block_2: break; // succs: block_5 } block block_4: } block block_5: // preds: block_3 vec1 32 ssa_4 = phi block_3: ssa_1 // succs: block_6 block block_6: Here converting return to break should add block_2 to the phis of block_5. block block_0: loop { block block_1: loop { block block_2: if ssa_2 { block block_3: return // succs: block_8 } else { block block_4: break; // succs: block_6 } block block_5: } block block_6: break; // succs: block_7 } block block_7: // preds: block_6 vec1 32 ssa_4 = phi block_6: ssa_1 // succs: block_8 block block_8: Here converting return to break will insert conditional break in the outer loop, changing block_6 predcessors. Cc: <mesa-stable@lists.freedesktop.org> Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3322 Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3498 Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6186>
2020-08-04 18:12:47 +03:00
nir_insert_phi_undef(first_block, block);
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
}
}
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
static nir_block *
split_block_end(nir_block *block)
{
nir_block *new_block = nir_block_create(ralloc_parent(block));
new_block->cf_node.parent = block->cf_node.parent;
exec_node_insert_after(&block->cf_node.node, &new_block->cf_node.node);
nir: Pull block_ends_in_jump into nir.h We had two different implementations in different files. May as well have one and put it in nir.h. Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
2018-08-24 09:34:05 -05:00
if (nir_block_ends_in_jump(block)) {
nir/cf: handle jumps in split_block_end() Before, we would only split a block with a jump at the end if we were inserting something after a block with a jump, which never happened in practice. But now, we want to use this to extract control flow lists which may end in a jump, in which case we really need to do the correct patching up. As a side effect, when removing jumps we now correctly insert undef phi sources in some corner cases, which can't hurt. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:25 -07:00
/* Figure out what successor block would've had if it didn't have a jump
* instruction, and make new_block have that successor.
*/
block_add_normal_succs(new_block);
} else {
move_successors(block, new_block);
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
return new_block;
}
nir/cf: add split_block_before_instr() Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:31 -07:00
static nir_block *
split_block_before_instr(nir_instr *instr)
{
assert(instr->type != nir_instr_type_phi);
nir_block *new_block = split_block_beginning(instr->block);
nir: Switch the arguments to nir_foreach_instr This matches the "foreach x in container" pattern found in many other programming languages. Generated by the following regular expression: s/nir_foreach_instr(\([^,]*\),\s*\([^,]*\))/nir_foreach_instr(\2, \1)/ and similar expressions for nir_foreach_instr_safe etc. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2016-04-26 18:34:19 -07:00
nir_foreach_instr_safe(cur_instr, instr->block) {
nir/cf: add split_block_before_instr() Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:31 -07:00
if (cur_instr == instr)
break;
exec_node_remove(&cur_instr->node);
cur_instr->block = new_block;
exec_list_push_tail(&new_block->instr_list, &cur_instr->node);
}
return new_block;
}
nir/cf: add split_block_cursor() This is a helper that will be shared between the new control flow insertion and modification code. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:32 -07:00
/* Splits a basic block at the point specified by the cursor. The "before" and
* "after" arguments are filled out with the blocks resulting from the split
* if non-NULL. Note that the "beginning" of the block is actually interpreted
* as before the first non-phi instruction, and it's illegal to split a block
* before a phi instruction.
*/
static void
split_block_cursor(nir_cursor cursor,
nir_block **_before, nir_block **_after)
{
nir_block *before, *after;
switch (cursor.option) {
case nir_cursor_before_block:
after = cursor.block;
before = split_block_beginning(cursor.block);
break;
case nir_cursor_after_block:
before = cursor.block;
after = split_block_end(cursor.block);
break;
case nir_cursor_before_instr:
after = cursor.instr->block;
before = split_block_before_instr(cursor.instr);
break;
case nir_cursor_after_instr:
/* We lower this to split_block_before_instr() so that we can keep the
* after-a-jump-instr case contained to split_block_end().
*/
if (nir_instr_is_last(cursor.instr)) {
before = cursor.instr->block;
after = split_block_end(cursor.instr->block);
} else {
after = cursor.instr->block;
before = split_block_before_instr(nir_instr_next(cursor.instr));
}
break;
nir: Silence GCC maybe-uninitialized warnings. nir/nir_control_flow.c: In function ‘split_block_cursor.isra.11’: nir/nir_control_flow.c:460:15: warning: ‘after’ may be used uninitialized in this function [-Wmaybe-uninitialized] *_after = after; ^ nir/nir_control_flow.c:458:16: warning: ‘before’ may be used uninitialized in this function [-Wmaybe-uninitialized] *_before = before; ^ Signed-off-by: Vinson Lee <vlee@freedesktop.org> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2015-11-02 01:23:59 -08:00
default:
build: avoid redefining unreachable() which is standard in C23 In the C23 standard unreachable() is now a predefined function-like macro in <stddef.h> See https://android.googlesource.com/platform/bionic/+/HEAD/docs/c23.md#is-now-a-predefined-function_like-macro-in And this causes build errors when building for C23: ----------------------------------------------------------------------- In file included from ../src/util/log.h:30, from ../src/util/log.c:30: ../src/util/macros.h:123:9: warning: "unreachable" redefined 123 | #define unreachable(str) \ | ^~~~~~~~~~~ In file included from ../src/util/macros.h:31: /usr/lib/gcc/x86_64-linux-gnu/14/include/stddef.h:456:9: note: this is the location of the previous definition 456 | #define unreachable() (__builtin_unreachable ()) | ^~~~~~~~~~~ ----------------------------------------------------------------------- So don't redefine it with the same name, but use the name UNREACHABLE() to also signify it's a macro. Using a different name also makes sense because the behavior of the macro was extending the one of __builtin_unreachable() anyway, and it also had a different signature, accepting one argument, compared to the standard unreachable() with no arguments. This change improves the chances of building mesa with the C23 standard, which for instance is the default in recent AOSP versions. All the instances of the macro, including the definition, were updated with the following command line: git grep -l '[^_]unreachable(' -- "src/**" | sort | uniq | \ while read file; \ do \ sed -e 's/\([^_]\)unreachable(/\1UNREACHABLE(/g' -i "$file"; \ done && \ sed -e 's/#undef unreachable/#undef UNREACHABLE/g' -i src/intel/isl/isl_aux_info.c Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36437>
2025-07-23 09:17:35 +02:00
UNREACHABLE("not reached");
nir/cf: add split_block_cursor() This is a helper that will be shared between the new control flow insertion and modification code. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:32 -07:00
}
if (_before)
*_before = before;
if (_after)
*_after = after;
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
/**
* Inserts a non-basic block between two basic blocks and links them together.
*/
static void
insert_non_block(nir_block *before, nir_cf_node *node, nir_block *after)
{
node->parent = before->cf_node.parent;
exec_node_insert_after(&before->cf_node.node, &node->node);
nir/cf: fix insertion of loops/ifs after jumps Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10284>
2021-04-20 09:57:04 +01:00
if (!nir_block_ends_in_jump(before))
link_block_to_non_block(before, node);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
link_non_block_to_block(node, after);
}
/* walk up the control flow tree to find the innermost enclosed loop */
static nir_loop *
nearest_loop(nir_cf_node *node)
{
while (node->type != nir_cf_node_loop) {
node = node->parent;
}
return nir_cf_node_as_loop(node);
}
nir: create nir_push_continue() and related helpers nir_control_flow.h: void nir_loop_add_continue_construct(nir_loop *loop); void nir_loop_remove_continue_construct(nir_loop *loop); Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13962>
2021-12-02 12:26:02 +01:00
void
nir_loop_add_continue_construct(nir_loop *loop)
{
assert(!nir_loop_has_continue_construct(loop));
nir_block *cont = nir_block_create(ralloc_parent(loop));
exec_list_push_tail(&loop->continue_list, &cont->cf_node.node);
cont->cf_node.parent = &loop->cf_node;
/* change predecessors and successors */
nir_block *header = nir_loop_first_block(loop);
nir_block *preheader = nir_block_cf_tree_prev(header);
set_foreach(header->predecessors, entry) {
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
nir_block *pred = (nir_block *)entry->key;
nir: create nir_push_continue() and related helpers nir_control_flow.h: void nir_loop_add_continue_construct(nir_loop *loop); void nir_loop_remove_continue_construct(nir_loop *loop); Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13962>
2021-12-02 12:26:02 +01:00
if (pred != preheader)
replace_successor(pred, header, cont);
}
link_blocks(cont, header, NULL);
}
void
nir_loop_remove_continue_construct(nir_loop *loop)
{
assert(nir_cf_list_is_empty_block(&loop->continue_list));
/* change predecessors and successors */
nir_block *header = nir_loop_first_block(loop);
nir_block *cont = nir_loop_first_continue_block(loop);
set_foreach(cont->predecessors, entry) {
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
nir_block *pred = (nir_block *)entry->key;
nir: create nir_push_continue() and related helpers nir_control_flow.h: void nir_loop_add_continue_construct(nir_loop *loop); void nir_loop_remove_continue_construct(nir_loop *loop); Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13962>
2021-12-02 12:26:02 +01:00
replace_successor(pred, cont, header);
}
block_remove_pred(header, cont);
exec_node_remove(&cont->cf_node.node);
}
nir/cf: Remove phi sources if needed in nir_handle_add_jump If the block in which the jump is inserted is the predecessor of a phi then we need to remove phi sources otherwise the phi may end up with things improperly connected. This fixes the following CTS test when dEQP is run with SPIR-V optimization recipe 1: dEQP-VK.glsl.functions.control_flow.return_in_nested_loop_vertex Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-09-21 09:27:48 -05:00
static void
remove_phi_src(nir_block *block, nir_block *pred)
{
nir: Use nir_foreach_phi(_safe) The pattern shows up all the time open-coded. Use the macro instead. Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com> Reviewed-by: Jesse Natalie <jenatali@microsoft.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22967>
2023-05-11 13:20:43 -04:00
nir_foreach_phi(phi, block) {
nir/cf: Remove phi sources if needed in nir_handle_add_jump If the block in which the jump is inserted is the predecessor of a phi then we need to remove phi sources otherwise the phi may end up with things improperly connected. This fixes the following CTS test when dEQP is run with SPIR-V optimization recipe 1: dEQP-VK.glsl.functions.control_flow.return_in_nested_loop_vertex Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-09-21 09:27:48 -05:00
nir_foreach_phi_src_safe(src, phi) {
if (src->pred == pred) {
list_del(&src->src.use_link);
exec_node_remove(&src->node);
nir: use a GC context for instructions Gives an roughly -15% change in compile-time for RADV/ACO. Memory usage increase seems to be 5-6%. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5034 Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12910>
2021-09-08 15:24:10 +01:00
gc_free(src);
nir/cf: Remove phi sources if needed in nir_handle_add_jump If the block in which the jump is inserted is the predecessor of a phi then we need to remove phi sources otherwise the phi may end up with things improperly connected. This fixes the following CTS test when dEQP is run with SPIR-V optimization recipe 1: dEQP-VK.glsl.functions.control_flow.return_in_nested_loop_vertex Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-09-21 09:27:48 -05:00
}
}
}
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
/*
* update the CFG after a jump instruction has been added to the end of a block
*/
void
nir_handle_add_jump(nir_block *block)
{
nir_instr *instr = nir_block_last_instr(block);
nir_jump_instr *jump_instr = nir_instr_as_jump(instr);
nir/cf: Remove phi sources if needed in nir_handle_add_jump If the block in which the jump is inserted is the predecessor of a phi then we need to remove phi sources otherwise the phi may end up with things improperly connected. This fixes the following CTS test when dEQP is run with SPIR-V optimization recipe 1: dEQP-VK.glsl.functions.control_flow.return_in_nested_loop_vertex Cc: mesa-stable@lists.freedesktop.org Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
2018-09-21 09:27:48 -05:00
if (block->successors[0])
remove_phi_src(block->successors[0], block);
if (block->successors[1])
remove_phi_src(block->successors[1], block);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
unlink_block_successors(block);
nir_function_impl *impl = nir_cf_node_get_function(&block->cf_node);
treewide: Switch to nir_progress Via the Coccinelle patch at the end of the commit message, followed by sed -ie 's/progress = progress | /progress |=/g' $(git grep -l 'progress = prog') ninja -C ~/mesa/build clang-format cd ~/mesa/src/compiler/nir && clang-format -i *.c agxfmt @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} -return prog; +return nir_progress(prog, impl, metadata); @@ expression prog_expr, impl, metadata; @@ -if (prog_expr) { -nir_metadata_preserve(impl, metadata); -return true; -} else { -nir_metadata_preserve(impl, nir_metadata_all); -return false; -} +bool progress = prog_expr; +return nir_progress(progress, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -nir_metadata_preserve(impl, prog ? (metadata) : nir_metadata_all); -return prog; +return nir_progress(prog, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -nir_metadata_preserve(impl, prog ? (metadata) : nir_metadata_all); +nir_progress(prog, impl, metadata); @@ expression impl, metadata; @@ -nir_metadata_preserve(impl, metadata); -return true; +return nir_progress(true, impl, metadata); @@ expression impl; @@ -nir_metadata_preserve(impl, nir_metadata_all); -return false; +return nir_no_progress(impl); @@ identifier other_prog, prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} -other_prog |= prog; +other_prog = other_prog | nir_progress(prog, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +nir_progress(prog, impl, metadata); @@ identifier other_prog, prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -other_prog = true; -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +other_prog = other_prog | nir_progress(prog, impl, metadata); @@ expression prog_expr, impl, metadata; identifier prog; @@ -if (prog_expr) { -nir_metadata_preserve(impl, metadata); -prog = true; -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +bool impl_progress = prog_expr; +prog = prog | nir_progress(impl_progress, impl, metadata); @@ identifier other_prog, prog; expression impl, metadata; @@ -if (prog) { -other_prog = true; -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +other_prog = other_prog | nir_progress(prog, impl, metadata); @@ expression prog_expr, impl, metadata; identifier prog; @@ -if (prog_expr) { -prog = true; -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +bool impl_progress = prog_expr; +prog = prog | nir_progress(impl_progress, impl, metadata); @@ expression prog_expr, impl, metadata; @@ -if (prog_expr) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +bool impl_progress = prog_expr; +nir_progress(impl_progress, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -nir_metadata_preserve(impl, metadata); -prog = true; +prog = nir_progress(true, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} -return prog; +return nir_progress(prog, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} +nir_progress(prog, impl, metadata); @@ expression impl; @@ -nir_metadata_preserve(impl, nir_metadata_all); +nir_no_progress(impl); @@ expression impl, metadata; @@ -nir_metadata_preserve(impl, metadata); +nir_progress(true, impl, metadata); squashme! sed -ie 's/progress = progress | /progress |=/g' $(git grep -l 'progress = prog') Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33722>
2025-02-24 15:10:33 -05:00
nir_progress(true, impl, nir_metadata_none);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
nir: Use a switch statement in nir_handle_add_jump Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Reviewed-by: Karol Herbst <kherbst@redhat.com> Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5101>
2020-05-15 15:31:50 -05:00
switch (jump_instr->type) {
case nir_jump_return:
nir: Add a halt instruction type Halt is like a return for the entire shader or exit() if you prefer to think of it that way. Once an invocation hits a halt, it's 100% dead. Any writes to output variables which happened before the halt do, however, still apply. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>
2020-05-15 15:46:08 -05:00
case nir_jump_halt:
nir: Use a switch statement in nir_handle_add_jump Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Reviewed-by: Karol Herbst <kherbst@redhat.com> Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5101>
2020-05-15 15:31:50 -05:00
link_blocks(block, impl->end_block, NULL);
break;
case nir_jump_break: {
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
nir_loop *loop = nearest_loop(&block->cf_node);
nir: Use a switch statement in nir_handle_add_jump Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Reviewed-by: Karol Herbst <kherbst@redhat.com> Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5101>
2020-05-15 15:31:50 -05:00
nir_cf_node *after = nir_cf_node_next(&loop->cf_node);
nir_block *after_block = nir_cf_node_as_block(after);
link_blocks(block, after_block, NULL);
break;
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
nir: Use a switch statement in nir_handle_add_jump Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Reviewed-by: Karol Herbst <kherbst@redhat.com> Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5101>
2020-05-15 15:31:50 -05:00
case nir_jump_continue: {
nir_loop *loop = nearest_loop(&block->cf_node);
nir: add Continue Construct to nir_loop The added continue_list corresponds to the SPIR-V Continue Construct and serves as a converged control-flow construct and is executed after each continue statement and before the next iteration of the loop body. Also adds validation rules for loops with Continue Construct Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13962>
2021-12-01 17:34:48 +01:00
nir_block *cont_block = nir_loop_continue_target(loop);
link_blocks(block, cont_block, NULL);
nir: Use a switch statement in nir_handle_add_jump Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Reviewed-by: Karol Herbst <kherbst@redhat.com> Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5101>
2020-05-15 15:31:50 -05:00
break;
}
nir: Add goto_if jump instruction Signed-off-by: Karol Herbst <kherbst@redhat.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2401>
2020-07-02 14:32:04 +02:00
case nir_jump_goto:
link_blocks(block, jump_instr->target, NULL);
break;
case nir_jump_goto_if:
link_blocks(block, jump_instr->else_target, jump_instr->target);
break;
nir: Use a switch statement in nir_handle_add_jump Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Reviewed-by: Karol Herbst <kherbst@redhat.com> Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5101>
2020-05-15 15:31:50 -05:00
default:
build: avoid redefining unreachable() which is standard in C23 In the C23 standard unreachable() is now a predefined function-like macro in <stddef.h> See https://android.googlesource.com/platform/bionic/+/HEAD/docs/c23.md#is-now-a-predefined-function_like-macro-in And this causes build errors when building for C23: ----------------------------------------------------------------------- In file included from ../src/util/log.h:30, from ../src/util/log.c:30: ../src/util/macros.h:123:9: warning: "unreachable" redefined 123 | #define unreachable(str) \ | ^~~~~~~~~~~ In file included from ../src/util/macros.h:31: /usr/lib/gcc/x86_64-linux-gnu/14/include/stddef.h:456:9: note: this is the location of the previous definition 456 | #define unreachable() (__builtin_unreachable ()) | ^~~~~~~~~~~ ----------------------------------------------------------------------- So don't redefine it with the same name, but use the name UNREACHABLE() to also signify it's a macro. Using a different name also makes sense because the behavior of the macro was extending the one of __builtin_unreachable() anyway, and it also had a different signature, accepting one argument, compared to the standard unreachable() with no arguments. This change improves the chances of building mesa with the C23 standard, which for instance is the default in recent AOSP versions. All the instances of the macro, including the definition, were updated with the following command line: git grep -l '[^_]unreachable(' -- "src/**" | sort | uniq | \ while read file; \ do \ sed -e 's/\([^_]\)unreachable(/\1UNREACHABLE(/g' -i "$file"; \ done && \ sed -e 's/#undef unreachable/#undef UNREACHABLE/g' -i src/intel/isl/isl_aux_info.c Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36437>
2025-07-23 09:17:35 +02:00
UNREACHABLE("Invalid jump type");
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
}
}
nir: Remove fake edges in the CF handling code When NIR was first introduced, Connor added this fake-edge hack to work around issues related to unreachable blocks. Thanks to GLSL IR's jump lowering code, the only unreachable code you can have is a block after an infinite loop. With SPIR-V, we didn't have the jump lowering code so we could also end up with the "if (...) { break; } else { continue; }" case which generates an unreachable block after the if. Because of this, most of NIR had to be fixed up for handling unreachable blocks. The only remaining case of not handling unreachable blocks was specifically the block-after-infinite-loop case in dead_cf which was fixed by the previous commit. We can now delete the fake edge hack. Signed-off-by: Jason Ekstrand <jason@jlekstrand.net> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2016-08-31 14:45:08 -07:00
/* Removes the successor of a block with a jump. Note that the jump to be
* eliminated may be free-floating.
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
*/
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
nir/cf: Conditionally do block_add_normal_succs() in unlink_jump(); There is a bug where we mess up predecessors/successors due to the ordering of unlinking/recreating edges/adding fake edges. In order to fix that, I need everything in one routine. However, calling block_add_normal_succs() isn't safe from cleanup_cf_node() - it would crash trying to insert phi undefs. So unfortunately I need to add a parameter. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-09-22 18:04:14 -07:00
static void
unlink_jump(nir_block *block, nir_jump_type type, bool add_normal_successors)
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
{
if (block->successors[0])
remove_phi_src(block->successors[0], block);
if (block->successors[1])
remove_phi_src(block->successors[1], block);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
nir/cf: Alter block successors before adding a fake link. Consider the case of "while (...) { break }". Or in NIR: block block_0 (0x7ab640): ... /* succs: block_1 */ loop { block block_1: /* preds: block_0 */ break /* succs: block_2 */ } block block_2: Calling nir_handle_remove_jump(block_1, nir_jump_break) will remove the break. Unfortunately, it would mangle the predecessors and successors. Here, block_2->predecessors->entries == 1, so we would create a fake link, setting block_1->successors[1] = block_2, and adding block_1 to block_2's predecessor set. This is illegal: a block cannot specify the same successor twice. In particular, adding the predecessor would have no effect, as it was already present in the set. We'd then call unlink_block_successors(), which would delete the fake link and remove block_1 from block_2's predecessor set. It would then delete successors[0], and attempt to remove block_1 from block_2's predecessor set a second time...except that it wouldn't be present, triggering an assertion failure. The fix appears to be simple: simply unlink the block's successors and recreate them to point at the correct blocks first. Then, add the fake link. In the above example, removing the break would cause block_1 to have itself as a successor (as it becomes an infinite loop), so adding the fake link won't cause a duplicate successor. v2: Add comments (requested by Connor Abbott) and fix commit message. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-09-18 13:11:56 -07:00
unlink_block_successors(block);
if (add_normal_successors)
block_add_normal_succs(block);
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
}
void
nir_handle_remove_jump(nir_block *block, nir_jump_type type)
{
nir/cf: Conditionally do block_add_normal_succs() in unlink_jump(); There is a bug where we mess up predecessors/successors due to the ordering of unlinking/recreating edges/adding fake edges. In order to fix that, I need everything in one routine. However, calling block_add_normal_succs() isn't safe from cleanup_cf_node() - it would crash trying to insert phi undefs. So unfortunately I need to add a parameter. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-09-22 18:04:14 -07:00
unlink_jump(block, type, true);
nir/cf: split up and improve nir_handle_remove_jumps() Before, the process of removing a jump and wiring up the remaining block correctly was atomic, but with the new control flow modification it's split into two parts: first, we extract the jump, which creates a new block with re-wired successors as well as a free-floating jump, and then we delete the control flow containing the jump, which removes the entry in the predecessors and any phi node sources. Split up nir_handle_remove_jumps() to accomodate this, and add the missing support for removing phi node sources. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:22 -07:00
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
nir_function_impl *impl = nir_cf_node_get_function(&block->cf_node);
treewide: Switch to nir_progress Via the Coccinelle patch at the end of the commit message, followed by sed -ie 's/progress = progress | /progress |=/g' $(git grep -l 'progress = prog') ninja -C ~/mesa/build clang-format cd ~/mesa/src/compiler/nir && clang-format -i *.c agxfmt @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} -return prog; +return nir_progress(prog, impl, metadata); @@ expression prog_expr, impl, metadata; @@ -if (prog_expr) { -nir_metadata_preserve(impl, metadata); -return true; -} else { -nir_metadata_preserve(impl, nir_metadata_all); -return false; -} +bool progress = prog_expr; +return nir_progress(progress, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -nir_metadata_preserve(impl, prog ? (metadata) : nir_metadata_all); -return prog; +return nir_progress(prog, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -nir_metadata_preserve(impl, prog ? (metadata) : nir_metadata_all); +nir_progress(prog, impl, metadata); @@ expression impl, metadata; @@ -nir_metadata_preserve(impl, metadata); -return true; +return nir_progress(true, impl, metadata); @@ expression impl; @@ -nir_metadata_preserve(impl, nir_metadata_all); -return false; +return nir_no_progress(impl); @@ identifier other_prog, prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} -other_prog |= prog; +other_prog = other_prog | nir_progress(prog, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +nir_progress(prog, impl, metadata); @@ identifier other_prog, prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -other_prog = true; -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +other_prog = other_prog | nir_progress(prog, impl, metadata); @@ expression prog_expr, impl, metadata; identifier prog; @@ -if (prog_expr) { -nir_metadata_preserve(impl, metadata); -prog = true; -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +bool impl_progress = prog_expr; +prog = prog | nir_progress(impl_progress, impl, metadata); @@ identifier other_prog, prog; expression impl, metadata; @@ -if (prog) { -other_prog = true; -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +other_prog = other_prog | nir_progress(prog, impl, metadata); @@ expression prog_expr, impl, metadata; identifier prog; @@ -if (prog_expr) { -prog = true; -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +bool impl_progress = prog_expr; +prog = prog | nir_progress(impl_progress, impl, metadata); @@ expression prog_expr, impl, metadata; @@ -if (prog_expr) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +bool impl_progress = prog_expr; +nir_progress(impl_progress, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -nir_metadata_preserve(impl, metadata); -prog = true; +prog = nir_progress(true, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} -return prog; +return nir_progress(prog, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} +nir_progress(prog, impl, metadata); @@ expression impl; @@ -nir_metadata_preserve(impl, nir_metadata_all); +nir_no_progress(impl); @@ expression impl, metadata; @@ -nir_metadata_preserve(impl, metadata); +nir_progress(true, impl, metadata); squashme! sed -ie 's/progress = progress | /progress |=/g' $(git grep -l 'progress = prog') Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33722>
2025-02-24 15:10:33 -05:00
nir_progress(true, impl, nir_metadata_none);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
}
static void
update_if_uses(nir_cf_node *node)
{
if (node->type != nir_cf_node_if)
return;
nir_if *if_stmt = nir_cf_node_as_if(node);
nir: Combine if_uses with instruction uses Every nir_ssa_def is part of a chain of uses, implemented with doubly linked lists. That means each requires 2 * 64-bit = 16 bytes per def, which is memory intensive. Together they require 32 bytes per def. Not cool. To cut that memory use in half, we can combine the two linked lists into a single use list that contains both regular instruction uses and if-uses. To do this, we augment the nir_src with a boolean "is_if", and reimplement the abstract if-uses operations on top of that list. That boolean should fit into the padding already in nir_src so should not actually affect memory use, and in the future we sneak it into the bottom bit of a pointer. However, this creates a new inefficiency: now iterating over regular uses separate from if-uses is (nominally) more expensive. It turns out virtually every caller of nir_foreach_if_use(_safe) also calls nir_foreach_use(_safe) immediately before, so we rewrite most of the callers to instead call a new single `nir_foreach_use_including_if(_safe)` which predicates the logic based on `src->is_if`. This should mitigate the performance difference. There's a bit of churn, but this is largely a mechanical set of changes. Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com> Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22343>
2023-04-06 13:19:31 -04:00
nir_src_set_parent_if(&if_stmt->condition, if_stmt);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
nir: Collapse more SSA checks Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com> Reviewed-by: Marek Olšák <marek.olsak@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24432>
2023-08-01 12:33:53 -04:00
list_addtail(&if_stmt->condition.use_link,
&if_stmt->condition.ssa->uses);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
}
/**
* Stitch two basic blocks together into one. The aggregate must have the same
* predecessors as the first and the same successors as the second.
nir/cf: return cursor after insertion of cf_list This will be useful to cut code from one location and paste it at another place and later keep pasting after the previous insertions. v2: update comment (Jason) deal with stiching 2 empty blocks (Jason) Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
2022-04-19 15:05:04 +03:00
*
* Returns a cursor pointing at the end of the before block (i.e.m between the
* two blocks) once stiched together.
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
*/
nir/cf: return cursor after insertion of cf_list This will be useful to cut code from one location and paste it at another place and later keep pasting after the previous insertions. v2: update comment (Jason) deal with stiching 2 empty blocks (Jason) Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
2022-04-19 15:05:04 +03:00
static nir_cursor
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
stitch_blocks(nir_block *before, nir_block *after)
{
/*
* We move after into before, so we have to deal with up to 2 successors vs.
* possibly a large number of predecessors.
*
* TODO: special case when before is empty and after isn't?
*/
nir: Pull block_ends_in_jump into nir.h We had two different implementations in different files. May as well have one and put it in nir.h. Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
2018-08-24 09:34:05 -05:00
if (nir_block_ends_in_jump(before)) {
nir/cf: handle jumps better in stitch_blocks() In particular, handle the case where the earlier block ends in a jump and the later block is empty. In that case, we want to preserve the jump and remove any traces of the later block. Before, we would only hit this case when removing a control flow node after a jump, which wasn't a common occurance, but we'll need it to handle inserting a control flow list which ends in a jump, which should be more common/useful. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:26 -07:00
assert(exec_list_is_empty(&after->instr_list));
if (after->successors[0])
remove_phi_src(after->successors[0], after);
if (after->successors[1])
remove_phi_src(after->successors[1], after);
unlink_block_successors(after);
exec_node_remove(&after->cf_node.node);
nir/cf: return cursor after insertion of cf_list This will be useful to cut code from one location and paste it at another place and later keep pasting after the previous insertions. v2: update comment (Jason) deal with stiching 2 empty blocks (Jason) Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
2022-04-19 15:05:04 +03:00
return nir_after_block(before);
nir/cf: handle jumps better in stitch_blocks() In particular, handle the case where the earlier block ends in a jump and the later block is empty. In that case, we want to preserve the jump and remove any traces of the later block. Before, we would only hit this case when removing a control flow node after a jump, which wasn't a common occurance, but we'll need it to handle inserting a control flow list which ends in a jump, which should be more common/useful. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:26 -07:00
} else {
nir/cf: return cursor after insertion of cf_list This will be useful to cut code from one location and paste it at another place and later keep pasting after the previous insertions. v2: update comment (Jason) deal with stiching 2 empty blocks (Jason) Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
2022-04-19 15:05:04 +03:00
nir_instr *last_before_instr = nir_block_last_instr(before);
nir/cf: handle jumps better in stitch_blocks() In particular, handle the case where the earlier block ends in a jump and the later block is empty. In that case, we want to preserve the jump and remove any traces of the later block. Before, we would only hit this case when removing a control flow node after a jump, which wasn't a common occurance, but we'll need it to handle inserting a control flow list which ends in a jump, which should be more common/useful. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:26 -07:00
move_successors(after, before);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
nir/cf: handle jumps better in stitch_blocks() In particular, handle the case where the earlier block ends in a jump and the later block is empty. In that case, we want to preserve the jump and remove any traces of the later block. Before, we would only hit this case when removing a control flow node after a jump, which wasn't a common occurance, but we'll need it to handle inserting a control flow list which ends in a jump, which should be more common/useful. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:26 -07:00
foreach_list_typed(nir_instr, instr, node, &after->instr_list) {
instr->block = before;
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
nir/cf: handle jumps better in stitch_blocks() In particular, handle the case where the earlier block ends in a jump and the later block is empty. In that case, we want to preserve the jump and remove any traces of the later block. Before, we would only hit this case when removing a control flow node after a jump, which wasn't a common occurance, but we'll need it to handle inserting a control flow list which ends in a jump, which should be more common/useful. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:26 -07:00
exec_list_append(&before->instr_list, &after->instr_list);
exec_node_remove(&after->cf_node.node);
nir/cf: return cursor after insertion of cf_list This will be useful to cut code from one location and paste it at another place and later keep pasting after the previous insertions. v2: update comment (Jason) deal with stiching 2 empty blocks (Jason) Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
2022-04-19 15:05:04 +03:00
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
return last_before_instr ? nir_after_instr(last_before_instr) : nir_before_block(before);
nir/cf: handle jumps better in stitch_blocks() In particular, handle the case where the earlier block ends in a jump and the later block is empty. In that case, we want to preserve the jump and remove any traces of the later block. Before, we would only hit this case when removing a control flow node after a jump, which wasn't a common occurance, but we'll need it to handle inserting a control flow list which ends in a jump, which should be more common/useful. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:26 -07:00
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
}
nir/cf: use a cursor for inserting control flow Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:33 -07:00
void
nir_cf_node_insert(nir_cursor cursor, nir_cf_node *node)
{
nir_block *before, *after;
split_block_cursor(cursor, &before, &after);
if (node->type == nir_cf_node_block) {
nir_block *block = nir_cf_node_as_block(node);
exec_node_insert_after(&before->cf_node.node, &block->cf_node.node);
block->cf_node.parent = before->cf_node.parent;
/* stitch_blocks() assumes that any block that ends with a jump has
* already been setup with the correct successors, so we need to set
* up jumps here as the block is being inserted.
*/
nir: Pull block_ends_in_jump into nir.h We had two different implementations in different files. May as well have one and put it in nir.h. Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
2018-08-24 09:34:05 -05:00
if (nir_block_ends_in_jump(block))
nir/cf: use a cursor for inserting control flow Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:33 -07:00
nir_handle_add_jump(block);
stitch_blocks(block, after);
stitch_blocks(before, block);
} else {
update_if_uses(node);
insert_non_block(before, node, after);
}
}
nir/cf: remove uses of SSA definitions that are being deleted Unlike calling nir_instr_remove(), calling nir_cf_node_remove() (and later in the series, the nir_cf_list_delete()) implies that you're removing instructions that may still have uses, except those instructions are never executed so any uses will be undefined. When cleaning up a CF node for deletion, we must clean up any uses of the deleted instructions by making them point to undef instructions instead. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:27 -07:00
static bool
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
replace_ssa_def_uses(nir_def *def, void *void_impl)
nir/cf: remove uses of SSA definitions that are being deleted Unlike calling nir_instr_remove(), calling nir_cf_node_remove() (and later in the series, the nir_cf_list_delete()) implies that you're removing instructions that may still have uses, except those instructions are never executed so any uses will be undefined. When cleaning up a CF node for deletion, we must clean up any uses of the deleted instructions by making them point to undef instructions instead. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:27 -07:00
{
nir_function_impl *impl = void_impl;
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_undef_instr *undef =
nir_undef_instr_create(impl->function->shader,
def->num_components,
def->bit_size);
nir/cf: remove uses of SSA definitions that are being deleted Unlike calling nir_instr_remove(), calling nir_cf_node_remove() (and later in the series, the nir_cf_list_delete()) implies that you're removing instructions that may still have uses, except those instructions are never executed so any uses will be undefined. When cleaning up a CF node for deletion, we must clean up any uses of the deleted instructions by making them point to undef instructions instead. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:27 -07:00
nir_instr_insert_before_cf_list(&impl->body, &undef->instr);
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_def_rewrite_uses(def, &undef->def);
nir/cf: remove uses of SSA definitions that are being deleted Unlike calling nir_instr_remove(), calling nir_cf_node_remove() (and later in the series, the nir_cf_list_delete()) implies that you're removing instructions that may still have uses, except those instructions are never executed so any uses will be undefined. When cleaning up a CF node for deletion, we must clean up any uses of the deleted instructions by making them point to undef instructions instead. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:27 -07:00
return true;
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
static void
nir/cf: remove uses of SSA definitions that are being deleted Unlike calling nir_instr_remove(), calling nir_cf_node_remove() (and later in the series, the nir_cf_list_delete()) implies that you're removing instructions that may still have uses, except those instructions are never executed so any uses will be undefined. When cleaning up a CF node for deletion, we must clean up any uses of the deleted instructions by making them point to undef instructions instead. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:27 -07:00
cleanup_cf_node(nir_cf_node *node, nir_function_impl *impl)
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
{
switch (node->type) {
case nir_cf_node_block: {
nir_block *block = nir_cf_node_as_block(node);
/* We need to walk the instructions and clean up defs/uses */
nir: Switch the arguments to nir_foreach_instr This matches the "foreach x in container" pattern found in many other programming languages. Generated by the following regular expression: s/nir_foreach_instr(\([^,]*\),\s*\([^,]*\))/nir_foreach_instr(\2, \1)/ and similar expressions for nir_foreach_instr_safe etc. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
2016-04-26 18:34:19 -07:00
nir_foreach_instr_safe(instr, block) {
nir/cf: clean up jumps when cleaning up CF nodes We may delete a control flow node which contains structured jumps to other parts of the program. We need to remove the jump as a predecessor, as well as remove any phi node sources which reference it. Right now, the same problem exists for blocks that don't end in a jump instruction, but with the new API it shouldn't be an issue, since blocks that don't end in a jump must either point to another block in the same extracted CF list or not point to anything at all. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:28 -07:00
if (instr->type == nir_instr_type_jump) {
nir: Add goto_if jump instruction Signed-off-by: Karol Herbst <kherbst@redhat.com> Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2401>
2020-07-02 14:32:04 +02:00
nir_jump_instr *jump = nir_instr_as_jump(instr);
unlink_jump(block, jump->type, false);
if (jump->type == nir_jump_goto_if)
nir: Add a nir_instr_clear_src() helper and use it Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24729>
2023-08-16 11:16:00 -05:00
nir_instr_clear_src(instr, &jump->condition);
nir/cf: clean up jumps when cleaning up CF nodes We may delete a control flow node which contains structured jumps to other parts of the program. We need to remove the jump as a predecessor, as well as remove any phi node sources which reference it. Right now, the same problem exists for blocks that don't end in a jump instruction, but with the new API it shouldn't be an issue, since blocks that don't end in a jump must either point to another block in the same extracted CF list or not point to anything at all. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:28 -07:00
} else {
nir: nir_foreach_ssa_def() -> nir_foreach_def() s/nir_foreach_ssa_def/nir_foreach_def/g followed by ninja -C _build clang-format and a little hand clean-up in nir.c. Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24658>
2023-08-12 19:10:24 -05:00
nir_foreach_def(instr, replace_ssa_def_uses, impl);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
nir_instr_remove(instr);
nir/cf: remove uses of SSA definitions that are being deleted Unlike calling nir_instr_remove(), calling nir_cf_node_remove() (and later in the series, the nir_cf_list_delete()) implies that you're removing instructions that may still have uses, except those instructions are never executed so any uses will be undefined. When cleaning up a CF node for deletion, we must clean up any uses of the deleted instructions by making them point to undef instructions instead. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:27 -07:00
}
}
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
break;
}
case nir_cf_node_if: {
nir_if *if_stmt = nir_cf_node_as_if(node);
foreach_list_typed(nir_cf_node, child, node, &if_stmt->then_list)
nir/cf: remove uses of SSA definitions that are being deleted Unlike calling nir_instr_remove(), calling nir_cf_node_remove() (and later in the series, the nir_cf_list_delete()) implies that you're removing instructions that may still have uses, except those instructions are never executed so any uses will be undefined. When cleaning up a CF node for deletion, we must clean up any uses of the deleted instructions by making them point to undef instructions instead. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:27 -07:00
cleanup_cf_node(child, impl);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
foreach_list_typed(nir_cf_node, child, node, &if_stmt->else_list)
nir/cf: remove uses of SSA definitions that are being deleted Unlike calling nir_instr_remove(), calling nir_cf_node_remove() (and later in the series, the nir_cf_list_delete()) implies that you're removing instructions that may still have uses, except those instructions are never executed so any uses will be undefined. When cleaning up a CF node for deletion, we must clean up any uses of the deleted instructions by making them point to undef instructions instead. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:27 -07:00
cleanup_cf_node(child, impl);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
list_del(&if_stmt->condition.use_link);
break;
}
case nir_cf_node_loop: {
nir_loop *loop = nir_cf_node_as_loop(node);
foreach_list_typed(nir_cf_node, child, node, &loop->body)
nir/cf: remove uses of SSA definitions that are being deleted Unlike calling nir_instr_remove(), calling nir_cf_node_remove() (and later in the series, the nir_cf_list_delete()) implies that you're removing instructions that may still have uses, except those instructions are never executed so any uses will be undefined. When cleaning up a CF node for deletion, we must clean up any uses of the deleted instructions by making them point to undef instructions instead. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:27 -07:00
cleanup_cf_node(child, impl);
nir: add Continue Construct to nir_loop The added continue_list corresponds to the SPIR-V Continue Construct and serves as a converged control-flow construct and is executed after each continue statement and before the next iteration of the loop body. Also adds validation rules for loops with Continue Construct Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13962>
2021-12-01 17:34:48 +01:00
foreach_list_typed(nir_cf_node, child, node, &loop->continue_list)
cleanup_cf_node(child, impl);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
break;
}
case nir_cf_node_function: {
nir_function_impl *impl = nir_cf_node_as_function(node);
foreach_list_typed(nir_cf_node, child, node, &impl->body)
nir/cf: remove uses of SSA definitions that are being deleted Unlike calling nir_instr_remove(), calling nir_cf_node_remove() (and later in the series, the nir_cf_list_delete()) implies that you're removing instructions that may still have uses, except those instructions are never executed so any uses will be undefined. When cleaning up a CF node for deletion, we must clean up any uses of the deleted instructions by making them point to undef instructions instead. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:27 -07:00
cleanup_cf_node(child, impl);
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
break;
}
default:
build: avoid redefining unreachable() which is standard in C23 In the C23 standard unreachable() is now a predefined function-like macro in <stddef.h> See https://android.googlesource.com/platform/bionic/+/HEAD/docs/c23.md#is-now-a-predefined-function_like-macro-in And this causes build errors when building for C23: ----------------------------------------------------------------------- In file included from ../src/util/log.h:30, from ../src/util/log.c:30: ../src/util/macros.h:123:9: warning: "unreachable" redefined 123 | #define unreachable(str) \ | ^~~~~~~~~~~ In file included from ../src/util/macros.h:31: /usr/lib/gcc/x86_64-linux-gnu/14/include/stddef.h:456:9: note: this is the location of the previous definition 456 | #define unreachable() (__builtin_unreachable ()) | ^~~~~~~~~~~ ----------------------------------------------------------------------- So don't redefine it with the same name, but use the name UNREACHABLE() to also signify it's a macro. Using a different name also makes sense because the behavior of the macro was extending the one of __builtin_unreachable() anyway, and it also had a different signature, accepting one argument, compared to the standard unreachable() with no arguments. This change improves the chances of building mesa with the C23 standard, which for instance is the default in recent AOSP versions. All the instances of the macro, including the definition, were updated with the following command line: git grep -l '[^_]unreachable(' -- "src/**" | sort | uniq | \ while read file; \ do \ sed -e 's/\([^_]\)unreachable(/\1UNREACHABLE(/g' -i "$file"; \ done && \ sed -e 's/#undef unreachable/#undef UNREACHABLE/g' -i src/intel/isl/isl_aux_info.c Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36437>
2025-07-23 09:17:35 +02:00
UNREACHABLE("Invalid CF node type");
nir: move control flow modification to its own file We want to start reworking and expanding this code, but it'll be a lot easier to do once we disentangle it from the rest of the stuff in nir.c. Unfortunately, there are a few unavoidable dependencies in nir.c on methods we'd rather not expose publicly, since if not used in very specific situations they can cause Bad Things (tm) to happen. Namely, we need to do some magical control flow munging when adding/removing jumps. In the future, we may disallow adding/removing jumps in nir_instr_insert_*() and nir_instr_remove(), and use separate functions that are part of the control flow modification code, but for now we expose them and put them in a separate, private header. Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:18 -07:00
}
}
nir/cf: Return a cursor from nir_cf_extract as well Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
2022-04-26 08:53:31 -05:00
/**
* Extracts everything between two cursors. Returns the cursor which is
* equivalent to the old begin/end curosors.
*/
nir_cursor
nir/cf: add new control modification API's These will help us do a number of things, including: - Early return elimination. - Dead control flow elimination. - Various optimizations, such as replacing: if (foo) { ... } if (!foo) { ... } with: if (foo) { ... } else { ... } Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:34 -07:00
nir_cf_extract(nir_cf_list *extracted, nir_cursor begin, nir_cursor end)
{
nir_block *block_begin, *block_end, *block_before, *block_after;
nir/cf: Make extracting or re-inserting nothing a no-op Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2015-12-23 18:10:08 -08:00
if (nir_cursors_equal(begin, end)) {
exec_list_make_empty(&extracted->list);
extracted->impl = NULL; /* we shouldn't need this */
nir/cf: Return a cursor from nir_cf_extract as well Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
2022-04-26 08:53:31 -05:00
return begin;
nir/cf: Make extracting or re-inserting nothing a no-op Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2015-12-23 18:10:08 -08:00
}
nir/cf: add new control modification API's These will help us do a number of things, including: - Early return elimination. - Dead control flow elimination. - Various optimizations, such as replacing: if (foo) { ... } if (!foo) { ... } with: if (foo) { ... } else { ... } Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:34 -07:00
split_block_cursor(begin, &block_before, &block_begin);
nir/cf: Better handle intra-block splits In the case where end was a instruction-based cursor, we would mix up our blocks and end up with block_begin pointing after the second split. This causes a segfault as the cf_node list walk at the end of the function never terminates properly. There's also a possibility of mix-up if begin is an instruction-based cursor which was found by inspection. Fixes: fc7f2d2364a9 "nir/cf: add new control modification API's" Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Acked-by: Matt Turner <mattst88@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-24 23:50:24 -05:00
/* Splitting a block twice with two cursors created before either split is
* tricky and there are a couple of places it can go wrong if both cursors
* point to the same block. One is if the second cursor is an block-based
* cursor and, thanks to the split above, it ends up pointing to the wrong
* block. If it's a before_block cursor and it's in the same block as
* begin, then begin must also be a before_block cursor and it should be
* caught by the nir_cursors_equal check above and we won't get here. If
* it's an after_block cursor, we need to re-adjust to ensure that it
* points to the second one of the split blocks, regardless of which it is.
*/
if (end.option == nir_cursor_after_block && end.block == block_before)
end.block = block_begin;
nir/cf: add new control modification API's These will help us do a number of things, including: - Early return elimination. - Dead control flow elimination. - Various optimizations, such as replacing: if (foo) { ... } if (!foo) { ... } with: if (foo) { ... } else { ... } Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:34 -07:00
split_block_cursor(end, &block_end, &block_after);
nir/cf: Better handle intra-block splits In the case where end was a instruction-based cursor, we would mix up our blocks and end up with block_begin pointing after the second split. This causes a segfault as the cf_node list walk at the end of the function never terminates properly. There's also a possibility of mix-up if begin is an instruction-based cursor which was found by inspection. Fixes: fc7f2d2364a9 "nir/cf: add new control modification API's" Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Acked-by: Matt Turner <mattst88@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6866>
2020-09-24 23:50:24 -05:00
/* The second place this can all go wrong is that it could be that the
* second split places the original block after the new block in which case
* the block_begin pointer that we saved off above is pointing to the block
* at the end rather than the block in the middle like it's supposed to be.
* In this case, we have to re-adjust begin_block to point to the middle
* one.
*/
if (block_begin == block_after)
block_begin = block_end;
nir/cf: add new control modification API's These will help us do a number of things, including: - Early return elimination. - Dead control flow elimination. - Various optimizations, such as replacing: if (foo) { ... } if (!foo) { ... } with: if (foo) { ... } else { ... } Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:34 -07:00
extracted->impl = nir_cf_node_get_function(&block_begin->cf_node);
exec_list_make_empty(&extracted->list);
nir/cf: Fix dominance metadata in the dead control flow pass. The NIR control flow modification API churns the block structure, splitting blocks, stitching them back together, and so on. Preserving information about block dominance is hard (and probably not worthwhile). This patch makes nir_cf_extract() throw away all metadata, like we do when adding/removing jumps. We then make the dead control flow pass compute dominance information right before it uses it. This is necessary because earlier work by the pass may have invalidated it. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-09-19 04:40:07 -07:00
/* Dominance and other block-related information is toast. */
treewide: Switch to nir_progress Via the Coccinelle patch at the end of the commit message, followed by sed -ie 's/progress = progress | /progress |=/g' $(git grep -l 'progress = prog') ninja -C ~/mesa/build clang-format cd ~/mesa/src/compiler/nir && clang-format -i *.c agxfmt @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} -return prog; +return nir_progress(prog, impl, metadata); @@ expression prog_expr, impl, metadata; @@ -if (prog_expr) { -nir_metadata_preserve(impl, metadata); -return true; -} else { -nir_metadata_preserve(impl, nir_metadata_all); -return false; -} +bool progress = prog_expr; +return nir_progress(progress, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -nir_metadata_preserve(impl, prog ? (metadata) : nir_metadata_all); -return prog; +return nir_progress(prog, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -nir_metadata_preserve(impl, prog ? (metadata) : nir_metadata_all); +nir_progress(prog, impl, metadata); @@ expression impl, metadata; @@ -nir_metadata_preserve(impl, metadata); -return true; +return nir_progress(true, impl, metadata); @@ expression impl; @@ -nir_metadata_preserve(impl, nir_metadata_all); -return false; +return nir_no_progress(impl); @@ identifier other_prog, prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} -other_prog |= prog; +other_prog = other_prog | nir_progress(prog, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +nir_progress(prog, impl, metadata); @@ identifier other_prog, prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -other_prog = true; -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +other_prog = other_prog | nir_progress(prog, impl, metadata); @@ expression prog_expr, impl, metadata; identifier prog; @@ -if (prog_expr) { -nir_metadata_preserve(impl, metadata); -prog = true; -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +bool impl_progress = prog_expr; +prog = prog | nir_progress(impl_progress, impl, metadata); @@ identifier other_prog, prog; expression impl, metadata; @@ -if (prog) { -other_prog = true; -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +other_prog = other_prog | nir_progress(prog, impl, metadata); @@ expression prog_expr, impl, metadata; identifier prog; @@ -if (prog_expr) { -prog = true; -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +bool impl_progress = prog_expr; +prog = prog | nir_progress(impl_progress, impl, metadata); @@ expression prog_expr, impl, metadata; @@ -if (prog_expr) { -nir_metadata_preserve(impl, metadata); -} else { -nir_metadata_preserve(impl, nir_metadata_all); -} +bool impl_progress = prog_expr; +nir_progress(impl_progress, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -nir_metadata_preserve(impl, metadata); -prog = true; +prog = nir_progress(true, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} -return prog; +return nir_progress(prog, impl, metadata); @@ identifier prog; expression impl, metadata; @@ -if (prog) { -nir_metadata_preserve(impl, metadata); -} +nir_progress(prog, impl, metadata); @@ expression impl; @@ -nir_metadata_preserve(impl, nir_metadata_all); +nir_no_progress(impl); @@ expression impl, metadata; @@ -nir_metadata_preserve(impl, metadata); +nir_progress(true, impl, metadata); squashme! sed -ie 's/progress = progress | /progress |=/g' $(git grep -l 'progress = prog') Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33722>
2025-02-24 15:10:33 -05:00
nir_progress(true, extracted->impl, nir_metadata_none);
nir/cf: Fix dominance metadata in the dead control flow pass. The NIR control flow modification API churns the block structure, splitting blocks, stitching them back together, and so on. Preserving information about block dominance is hard (and probably not worthwhile). This patch makes nir_cf_extract() throw away all metadata, like we do when adding/removing jumps. We then make the dead control flow pass compute dominance information right before it uses it. This is necessary because earlier work by the pass may have invalidated it. Signed-off-by: Kenneth Graunke <kenneth@whitecape.org> Reviewed-by: Connor Abbott <cwabbott0@gmail.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@intel.com>
2015-09-19 04:40:07 -07:00
nir/cf: add new control modification API's These will help us do a number of things, including: - Early return elimination. - Dead control flow elimination. - Various optimizations, such as replacing: if (foo) { ... } if (!foo) { ... } with: if (foo) { ... } else { ... } Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:34 -07:00
nir_cf_node *cf_node = &block_begin->cf_node;
nir_cf_node *cf_node_end = &block_end->cf_node;
while (true) {
nir_cf_node *next = nir_cf_node_next(cf_node);
exec_node_remove(&cf_node->node);
cf_node->parent = NULL;
exec_list_push_tail(&extracted->list, &cf_node->node);
if (cf_node == cf_node_end)
break;
cf_node = next;
}
nir/cf: Return a cursor from nir_cf_extract as well Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
2022-04-26 08:53:31 -05:00
return stitch_blocks(block_before, block_after);
nir/cf: add new control modification API's These will help us do a number of things, including: - Early return elimination. - Dead control flow elimination. - Various optimizations, such as replacing: if (foo) { ... } if (!foo) { ... } with: if (foo) { ... } else { ... } Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:34 -07:00
}
nir: Add a halt instruction type Halt is like a return for the entire shader or exit() if you prefer to think of it that way. Once an invocation hits a halt, it's 100% dead. Any writes to output variables which happened before the halt do, however, still apply. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>
2020-05-15 15:46:08 -05:00
static void
relink_jump_halt_cf_node(nir_cf_node *node, nir_block *end_block)
{
switch (node->type) {
case nir_cf_node_block: {
nir_block *block = nir_cf_node_as_block(node);
nir_instr *last_instr = nir_block_last_instr(block);
if (last_instr == NULL || last_instr->type != nir_instr_type_jump)
break;
nir_jump_instr *jump = nir_instr_as_jump(last_instr);
/* We can't move a CF list from one function to another while we still
* have returns.
*/
assert(jump->type != nir_jump_return);
if (jump->type == nir_jump_halt) {
unlink_block_successors(block);
link_blocks(block, end_block, NULL);
}
break;
}
case nir_cf_node_if: {
nir_if *if_stmt = nir_cf_node_as_if(node);
foreach_list_typed(nir_cf_node, child, node, &if_stmt->then_list)
relink_jump_halt_cf_node(child, end_block);
foreach_list_typed(nir_cf_node, child, node, &if_stmt->else_list)
relink_jump_halt_cf_node(child, end_block);
break;
}
case nir_cf_node_loop: {
nir_loop *loop = nir_cf_node_as_loop(node);
foreach_list_typed(nir_cf_node, child, node, &loop->body)
relink_jump_halt_cf_node(child, end_block);
nir: add Continue Construct to nir_loop The added continue_list corresponds to the SPIR-V Continue Construct and serves as a converged control-flow construct and is executed after each continue statement and before the next iteration of the loop body. Also adds validation rules for loops with Continue Construct Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13962>
2021-12-01 17:34:48 +01:00
foreach_list_typed(nir_cf_node, child, node, &loop->continue_list)
relink_jump_halt_cf_node(child, end_block);
nir: Add a halt instruction type Halt is like a return for the entire shader or exit() if you prefer to think of it that way. Once an invocation hits a halt, it's 100% dead. Any writes to output variables which happened before the halt do, however, still apply. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>
2020-05-15 15:46:08 -05:00
break;
}
case nir_cf_node_function:
build: avoid redefining unreachable() which is standard in C23 In the C23 standard unreachable() is now a predefined function-like macro in <stddef.h> See https://android.googlesource.com/platform/bionic/+/HEAD/docs/c23.md#is-now-a-predefined-function_like-macro-in And this causes build errors when building for C23: ----------------------------------------------------------------------- In file included from ../src/util/log.h:30, from ../src/util/log.c:30: ../src/util/macros.h:123:9: warning: "unreachable" redefined 123 | #define unreachable(str) \ | ^~~~~~~~~~~ In file included from ../src/util/macros.h:31: /usr/lib/gcc/x86_64-linux-gnu/14/include/stddef.h:456:9: note: this is the location of the previous definition 456 | #define unreachable() (__builtin_unreachable ()) | ^~~~~~~~~~~ ----------------------------------------------------------------------- So don't redefine it with the same name, but use the name UNREACHABLE() to also signify it's a macro. Using a different name also makes sense because the behavior of the macro was extending the one of __builtin_unreachable() anyway, and it also had a different signature, accepting one argument, compared to the standard unreachable() with no arguments. This change improves the chances of building mesa with the C23 standard, which for instance is the default in recent AOSP versions. All the instances of the macro, including the definition, were updated with the following command line: git grep -l '[^_]unreachable(' -- "src/**" | sort | uniq | \ while read file; \ do \ sed -e 's/\([^_]\)unreachable(/\1UNREACHABLE(/g' -i "$file"; \ done && \ sed -e 's/#undef unreachable/#undef UNREACHABLE/g' -i src/intel/isl/isl_aux_info.c Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36437>
2025-07-23 09:17:35 +02:00
UNREACHABLE("Cannot insert a function in a function");
nir: Add a halt instruction type Halt is like a return for the entire shader or exit() if you prefer to think of it that way. Once an invocation hits a halt, it's 100% dead. Any writes to output variables which happened before the halt do, however, still apply. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>
2020-05-15 15:46:08 -05:00
default:
build: avoid redefining unreachable() which is standard in C23 In the C23 standard unreachable() is now a predefined function-like macro in <stddef.h> See https://android.googlesource.com/platform/bionic/+/HEAD/docs/c23.md#is-now-a-predefined-function_like-macro-in And this causes build errors when building for C23: ----------------------------------------------------------------------- In file included from ../src/util/log.h:30, from ../src/util/log.c:30: ../src/util/macros.h:123:9: warning: "unreachable" redefined 123 | #define unreachable(str) \ | ^~~~~~~~~~~ In file included from ../src/util/macros.h:31: /usr/lib/gcc/x86_64-linux-gnu/14/include/stddef.h:456:9: note: this is the location of the previous definition 456 | #define unreachable() (__builtin_unreachable ()) | ^~~~~~~~~~~ ----------------------------------------------------------------------- So don't redefine it with the same name, but use the name UNREACHABLE() to also signify it's a macro. Using a different name also makes sense because the behavior of the macro was extending the one of __builtin_unreachable() anyway, and it also had a different signature, accepting one argument, compared to the standard unreachable() with no arguments. This change improves the chances of building mesa with the C23 standard, which for instance is the default in recent AOSP versions. All the instances of the macro, including the definition, were updated with the following command line: git grep -l '[^_]unreachable(' -- "src/**" | sort | uniq | \ while read file; \ do \ sed -e 's/\([^_]\)unreachable(/\1UNREACHABLE(/g' -i "$file"; \ done && \ sed -e 's/#undef unreachable/#undef UNREACHABLE/g' -i src/intel/isl/isl_aux_info.c Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36437>
2025-07-23 09:17:35 +02:00
UNREACHABLE("Invalid CF node type");
nir: Add a halt instruction type Halt is like a return for the entire shader or exit() if you prefer to think of it that way. Once an invocation hits a halt, it's 100% dead. Any writes to output variables which happened before the halt do, however, still apply. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>
2020-05-15 15:46:08 -05:00
}
}
nir/cf: return cursor after insertion of cf_list This will be useful to cut code from one location and paste it at another place and later keep pasting after the previous insertions. v2: update comment (Jason) deal with stiching 2 empty blocks (Jason) Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
2022-04-19 15:05:04 +03:00
/**
* Inserts a list at a given cursor. Returns the cursor at the end of the
* insertion (i.e., at the end of the instructions contained in cf_list).
*/
nir_cursor
nir/cf: add new control modification API's These will help us do a number of things, including: - Early return elimination. - Dead control flow elimination. - Various optimizations, such as replacing: if (foo) { ... } if (!foo) { ... } with: if (foo) { ... } else { ... } Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:34 -07:00
nir_cf_reinsert(nir_cf_list *cf_list, nir_cursor cursor)
{
nir_block *before, *after;
nir/cf: Make extracting or re-inserting nothing a no-op Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2015-12-23 18:10:08 -08:00
if (exec_list_is_empty(&cf_list->list))
nir/cf: return cursor after insertion of cf_list This will be useful to cut code from one location and paste it at another place and later keep pasting after the previous insertions. v2: update comment (Jason) deal with stiching 2 empty blocks (Jason) Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
2022-04-19 15:05:04 +03:00
return cursor;
nir/cf: Make extracting or re-inserting nothing a no-op Reviewed-by: Jordan Justen <jordan.l.justen@intel.com> Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
2015-12-23 18:10:08 -08:00
nir: Add a halt instruction type Halt is like a return for the entire shader or exit() if you prefer to think of it that way. Once an invocation hits a halt, it's 100% dead. Any writes to output variables which happened before the halt do, however, still apply. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>
2020-05-15 15:46:08 -05:00
nir_function_impl *cursor_impl =
nir_cf_node_get_function(&nir_cursor_current_block(cursor)->cf_node);
if (cf_list->impl != cursor_impl) {
foreach_list_typed(nir_cf_node, node, node, &cf_list->list)
relink_jump_halt_cf_node(node, cursor_impl->end_block);
}
nir/cf: add new control modification API's These will help us do a number of things, including: - Early return elimination. - Dead control flow elimination. - Various optimizations, such as replacing: if (foo) { ... } if (!foo) { ... } with: if (foo) { ... } else { ... } Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:34 -07:00
split_block_cursor(cursor, &before, &after);
foreach_list_typed_safe(nir_cf_node, node, node, &cf_list->list) {
exec_node_remove(&node->node);
node->parent = before->cf_node.parent;
exec_node_insert_node_before(&after->cf_node.node, &node->node);
}
stitch_blocks(before,
nir_cf_node_as_block(nir_cf_node_next(&before->cf_node)));
nir/cf: return cursor after insertion of cf_list This will be useful to cut code from one location and paste it at another place and later keep pasting after the previous insertions. v2: update comment (Jason) deal with stiching 2 empty blocks (Jason) Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com> Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16036>
2022-04-19 15:05:04 +03:00
return stitch_blocks(nir_cf_node_as_block(nir_cf_node_prev(&after->cf_node)),
after);
nir/cf: add new control modification API's These will help us do a number of things, including: - Early return elimination. - Dead control flow elimination. - Various optimizations, such as replacing: if (foo) { ... } if (!foo) { ... } with: if (foo) { ... } else { ... } Signed-off-by: Connor Abbott <connor.w.abbott@intel.com> Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
2015-07-21 19:54:34 -07:00
}
void
nir_cf_delete(nir_cf_list *cf_list)
{
foreach_list_typed(nir_cf_node, node, node, &cf_list->list) {
cleanup_cf_node(node, cf_list->impl);
}
}
nir: Add a sort_unstructured_blocks() helper Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28300>
2024-03-19 12:45:06 -05:00
nir/cf: have nir_remove_after_cf_node remove phis at the start too Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Backport-to: 25.1 Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35975>
2025-07-07 14:47:22 +01:00
void
nir_remove_after_cf_node(nir_cf_node *node)
{
nir_cf_node *end = node;
while (!nir_cf_node_is_last(end))
end = nir_cf_node_next(end);
nir_cursor begin = nir_after_cf_node(node);
if (begin.option == nir_cursor_before_block) {
/* nir_cf_extract() would ignore these phis */
nir_function_impl *impl = nir_cf_node_get_function(node);
nir_foreach_phi_safe(phi, begin.block) {
replace_ssa_def_uses(&phi->def, impl);
nir_instr_remove_v(&phi->instr);
}
}
nir_cf_list list;
nir_cf_extract(&list, begin, nir_after_cf_node(end));
nir_cf_delete(&list);
}
nir: Add a sort_unstructured_blocks() helper Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28300>
2024-03-19 12:45:06 -05:00
struct block_index {
nir_block *block;
uint32_t index;
};
static void
calc_cfg_post_dfs_indices(nir_function_impl *impl,
nir_block *block,
struct block_index *blocks,
uint32_t *count)
{
if (block == impl->end_block)
return;
assert(block->index < impl->num_blocks);
if (blocks[block->index].block != NULL) {
assert(blocks[block->index].block == block);
return;
}
blocks[block->index].block = block;
for (uint32_t i = 0; i < ARRAY_SIZE(block->successors); i++) {
if (block->successors[i] != NULL)
calc_cfg_post_dfs_indices(impl, block->successors[i], blocks, count);
}
/* Pre-increment so that unreachable blocks have an index of 0 */
blocks[block->index].index = ++(*count);
}
static int
rev_cmp_block_index(const void *_a, const void *_b)
{
const struct block_index *a = _a, *b = _b;
return b->index - a->index;
}
void
nir_sort_unstructured_blocks(nir_function_impl *impl)
{
/* Re-index the blocks.
*
* We hand-roll it here instead of calling the helper because we also want
* to assert that there are no structured control-flow constructs.
*/
impl->num_blocks = 0;
foreach_list_typed(nir_cf_node, node, node, &impl->body) {
nir_block *block = nir_cf_node_as_block(node);
block->index = impl->num_blocks++;
}
struct block_index *blocks =
rzalloc_array(NULL, struct block_index, impl->num_blocks);
uint32_t count = 0;
calc_cfg_post_dfs_indices(impl, nir_start_block(impl), blocks, &count);
assert(count <= impl->num_blocks);
qsort(blocks, impl->num_blocks, sizeof(*blocks), rev_cmp_block_index);
struct exec_list dead_blocks;
exec_list_move_nodes_to(&impl->body, &dead_blocks);
for (uint32_t i = 0; i < count; i++) {
nir_block *block = blocks[i].block;
exec_node_remove(&block->cf_node.node);
block->index = i;
exec_list_push_tail(&impl->body, &block->cf_node.node);
}
impl->end_block->index = count;
for (uint32_t i = count; i < impl->num_blocks; i++) {
assert(blocks[i].index == 0);
assert(blocks[i].block == NULL);
}
nir: Update num_blocks in sort_unstructured_blocks Cc: mesa-stable Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33219>
2025-01-25 12:27:08 -05:00
impl->num_blocks = count;
nir: Add a sort_unstructured_blocks() helper Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28300>
2024-03-19 12:45:06 -05:00
foreach_list_typed_safe(nir_cf_node, node, node, &dead_blocks)
cleanup_cf_node(node, impl);
ralloc_free(blocks);
/* Dominance is toast but we indexed blocks as part of this pass. */
impl->valid_metadata &= nir_metadata_dominance;
impl->valid_metadata |= nir_metadata_block_index;
}
Reference in a new issue Copy permalink