nir/opt_if: Both parts of logic-joined conditions can be evaluated

For cases like 'if (X && Y)', both X and Y must be true in the then
branch. Their values are unknown in the else branch. Similarly, 'if (X
|| Y)' must have both X and Y false in the else branch.

The shader-db results are pretty bad, especially on Skylake. Ouch. The
fossil-db results are good enough that they make up for it.

v2: s/alu/alu_src/ in nir_src_parent_instr(use_src) !=
&alu_src->instr. Noticed by Rhys.

shader-db:

Lunar Lake
total instructions in shared programs: 17203905 -> 17196251 (-0.04%)
instructions in affected programs: 668828 -> 661174 (-1.14%)
helped: 352 / HURT: 2

total cycles in shared programs: 879896264 -> 888462774 (0.97%)
cycles in affected programs: 330523984 -> 339090494 (2.59%)
helped: 187 / HURT: 167

total spills in shared programs: 3318 -> 3329 (0.33%)
spills in affected programs: 4 -> 15 (275.00%)
helped: 0 / HURT: 4

total fills in shared programs: 1903 -> 1917 (0.74%)
fills in affected programs: 7 -> 21 (200.00%)
helped: 0 / HURT: 4

Meteor Lake and DG2 had similar results. (Meteor Lake shown)
total instructions in shared programs: 19969129 -> 19961439 (-0.04%)
instructions in affected programs: 665860 -> 658170 (-1.15%)
helped: 354 / HURT: 0

total cycles in shared programs: 884509249 -> 887353784 (0.32%)
cycles in affected programs: 323242817 -> 326087352 (0.88%)
helped: 208 / HURT: 146

total spills in shared programs: 4801 -> 4808 (0.15%)
spills in affected programs: 14 -> 21 (50.00%)
helped: 0 / HURT: 6

total fills in shared programs: 4454 -> 4467 (0.29%)
fills in affected programs: 17 -> 30 (76.47%)
helped: 0 / HURT: 6

Tiger Lake and Ice Lake had similar results. (Tiger Lake shown)
total instructions in shared programs: 19913774 -> 19906147 (-0.04%)
instructions in affected programs: 667348 -> 659721 (-1.14%)
helped: 351 / HURT: 3

total cycles in shared programs: 861253468 -> 864535803 (0.38%)
cycles in affected programs: 325577148 -> 328859483 (1.01%)
helped: 180 / HURT: 174

total spills in shared programs: 3440 -> 3455 (0.44%)
spills in affected programs: 18 -> 33 (83.33%)
helped: 0 / HURT: 8

total fills in shared programs: 1946 -> 1961 (0.77%)
fills in affected programs: 18 -> 33 (83.33%)
helped: 0 / HURT: 8

Skylake
total instructions in shared programs: 19031768 -> 19023604 (-0.04%)
instructions in affected programs: 671633 -> 663469 (-1.22%)
helped: 347 / HURT: 7

total cycles in shared programs: 868474831 -> 868132073 (-0.04%)
cycles in affected programs: 320499758 -> 320157000 (-0.11%)
helped: 246 / HURT: 108

total spills in shared programs: 4024 -> 4063 (0.97%)
spills in affected programs: 28 -> 67 (139.29%)
helped: 0 / HURT: 18

total fills in shared programs: 3722 -> 3746 (0.64%)
fills in affected programs: 34 -> 58 (70.59%)
helped: 0 / HURT: 18

fossil-db:

Lunar Lake
Totals:
Instrs: 928574038 -> 928568364 (-0.00%); split: -0.00%, +0.00%
Subgroup size: 40916656 -> 40916672 (+0.00%)
Send messages: 41467974 -> 41467909 (-0.00%); split: -0.00%, +0.00%
Loop count: 970202 -> 970191 (-0.00%)
Cycle count: 106297789925 -> 106301305901 (+0.00%); split: -0.00%, +0.01%
Spill count: 3424464 -> 3424452 (-0.00%); split: -0.00%, +0.00%
Fill count: 6525458 -> 6525119 (-0.01%); split: -0.01%, +0.00%
Max live registers: 193525368 -> 193524886 (-0.00%); split: -0.00%, +0.00%
Non SSA regs after NIR: 232027347 -> 232026610 (-0.00%); split: -0.00%, +0.00%

Totals from 1130 (0.06% of 2018793) affected shaders:
Instrs: 2662692 -> 2657018 (-0.21%); split: -0.27%, +0.06%
Subgroup size: 16 -> 32 (+100.00%)
Send messages: 112689 -> 112624 (-0.06%); split: -0.07%, +0.01%
Loop count: 5723 -> 5712 (-0.19%)
Cycle count: 1176696438 -> 1180212414 (+0.30%); split: -0.33%, +0.63%
Spill count: 9895 -> 9883 (-0.12%); split: -0.13%, +0.01%
Fill count: 26892 -> 26553 (-1.26%); split: -1.26%, +0.00%
Max live registers: 215462 -> 214980 (-0.22%); split: -0.30%, +0.08%
Non SSA regs after NIR: 398940 -> 398203 (-0.18%); split: -0.21%, +0.03%

Meteor Lake, DG2, Tiger Lake, Ice Lake, and Skylake had similar results. (Meteor Lake shown)
Totals:
Instrs: 1000318839 -> 1000314218 (-0.00%); split: -0.00%, +0.00%
Send messages: 45548952 -> 45548887 (-0.00%); split: -0.00%, +0.00%
Loop count: 1026441 -> 1026430 (-0.00%)
Cycle count: 92411461807 -> 92395024225 (-0.02%); split: -0.02%, +0.00%
Spill count: 3665265 -> 3665221 (-0.00%); split: -0.00%, +0.00%
Fill count: 6504830 -> 6504801 (-0.00%); split: -0.00%, +0.00%
Max live registers: 121790079 -> 121789811 (-0.00%); split: -0.00%, +0.00%
Max dispatch width: 38062488 -> 38062648 (+0.00%)
Non SSA regs after NIR: 256900770 -> 256900038 (-0.00%); split: -0.00%, +0.00%

Totals from 1124 (0.05% of 2284852) affected shaders:
Instrs: 2724110 -> 2719489 (-0.17%); split: -0.24%, +0.07%
Send messages: 112096 -> 112031 (-0.06%); split: -0.07%, +0.01%
Loop count: 5697 -> 5686 (-0.19%)
Cycle count: 960659254 -> 944221672 (-1.71%); split: -1.91%, +0.20%
Spill count: 13791 -> 13747 (-0.32%); split: -0.40%, +0.08%
Fill count: 43216 -> 43187 (-0.07%); split: -0.14%, +0.08%
Max live registers: 114877 -> 114609 (-0.23%); split: -0.31%, +0.07%
Max dispatch width: 12768 -> 12928 (+1.25%)
Non SSA regs after NIR: 412320 -> 411588 (-0.18%); split: -0.20%, +0.03%

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38321>

src/compiler/nir/nir_opt_if.c

@@ -1000,7 +1000,9 @@ evaluate_condition_use(nir_builder *b, nir_if *nif, nir_src *use_src,
       progress = true;
    }
 
-   if (!nir_src_is_if(use_src) && can_propagate_through_alu(use_src), false) {
+   const bool ignore_bcsel = branch != either_branch;
+   if (!nir_src_is_if(use_src) && can_propagate_through_alu(use_src,
+                                                            ignore_bcsel)) {
       nir_alu_instr *alu = nir_instr_as_alu(nir_src_parent_instr(use_src));
 
       nir_foreach_use_including_if_safe(alu_use, &alu->def) {
@@ -1023,6 +1025,7 @@ opt_if_evaluate_condition_use(nir_builder *b, nir_if *nif)
          progress |= evaluate_condition_use(b, nif, use_src, false, either_branch);
    }
 
+   bool invert = false;
    nir_alu_instr *alu = nir_src_as_alu_instr(nif->condition);
    if (alu != NULL && alu->op == nir_op_inot &&
        nir_src_num_components(alu->src[0].src) == 1) {
@@ -1042,6 +1045,47 @@ opt_if_evaluate_condition_use(nir_builder *b, nir_if *nif)
          if (nir_src_is_if(use_src) || nir_src_parent_instr(use_src) != &alu->instr)
             progress |= evaluate_condition_use(b, nif, use_src, true, either_branch);
       }
+
+      invert = true;
+      alu = nir_src_as_alu_instr(alu->src[0].src);
+   }
+
+   if (alu != NULL) {
+      /* For cases like 'if (X && Y)', both X and Y must be true in the then
+       * branch. Their values are unknown in the else branch. Similarly, 'if
+       * (X || Y)' must have both X and Y false in the else branch.
+       */
+      if (alu->op == nir_op_iand || alu->op == nir_op_ior) {
+         enum branch_to_eval branch =
+            invert != (alu->op == nir_op_ior) ? else_branch : then_branch;
+
+         for (unsigned i = 0; i < 2; i++) {
+            nir_def *def = alu->src[i].src.ssa;
+
+            if (def->num_components != 1)
+               continue;
+
+            nir_foreach_use_including_if_safe(use_src, def) {
+               progress |= evaluate_condition_use(b, nif, use_src,
+                                                  invert, branch);
+            }
+
+            /* Just like above, if the op is inot, peel the inot off and try
+             * some more.
+             */
+            nir_alu_instr *alu_src = nir_src_as_alu_instr(alu->src[i].src);
+            if (alu_src != NULL && alu_src->op == nir_op_inot &&
+                nir_src_num_components(alu_src->src[0].src) == 1) {
+               nir_foreach_use_including_if_safe(use_src, alu_src->src[0].src.ssa) {
+                  if (nir_src_is_if(use_src) ||
+                      nir_src_parent_instr(use_src) != &alu_src->instr) {
+                     progress |= evaluate_condition_use(b, nif, use_src,
+                                                        !invert, branch);
+                  }
+               }
+            }
+         }
+      }
    }
 
    return progress;