fdo-mirrors/mesa
1
0
Fork 0
mirror of https://gitlab.freedesktop.org/mesa/mesa.git synced 2026-05-18 20:18:06 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions 2
mesa/src/compiler/nir/nir_range_analysis.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2535 lines
80 KiB
C
Raw Normal View History

nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
/*
* Copyright © 2018 Intel Corporation
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
* to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense,
* and/or sell copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice (including the next
* paragraph) shall be included in all copies or substantial portions of the
* Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
* IN THE SOFTWARE.
*/
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
#include "nir.h"
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
#include "nir_range_analysis.h"
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
#include <float.h>
#include <math.h>
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
#include "util/hash_table.h"
nir/range_analysis: add helpers for limiting stack usage Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:42:22 +00:00
#include "util/u_dynarray.h"
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
#include "util/u_math.h"
nir: Fix use of alloca() without #include c99_alloca.h Reviewed-by: Jesse Natalie <jenatali@microsoft.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22150>
2023-03-27 16:29:32 -04:00
#include "c99_alloca.h"
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
/**
* Analyzes a sequence of operations to determine some aspects of the range of
* the result.
*/
nir/range_analysis: add helpers for limiting stack usage Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:42:22 +00:00
struct analysis_query {
uint32_t pushed_queries;
uint32_t result_index;
};
struct analysis_state {
nir_shader *shader;
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
void *range_ht;
nir/range_analysis: add helpers for limiting stack usage Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:42:22 +00:00
struct util_dynarray query_stack;
struct util_dynarray result_stack;
size_t query_size;
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
uint32_t (*get_key)(struct analysis_query *q);
bool (*lookup)(void *table, uint32_t key, uint32_t *value);
void (*insert)(void *table, uint32_t key, uint32_t value);
nir/range_analysis: add helpers for limiting stack usage Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:42:22 +00:00
void (*process_query)(struct analysis_state *state, struct analysis_query *q,
uint32_t *result, const uint32_t *src);
};
static void *
push_analysis_query(struct analysis_state *state, size_t size)
{
struct analysis_query *q = util_dynarray_grow_bytes(&state->query_stack, 1, size);
q->pushed_queries = 0;
q->result_index = util_dynarray_num_elements(&state->result_stack, uint32_t);
util/dynarray: infer type in append Most of the time, we can infer the type to append in util_dynarray_append using __typeof__, which is standardized in C23 and support in Jesse's MSMSVCV. This patch drops the type argument most of the time, making util_dynarray a little more ergonomic to use. This is done in four steps. First, rename util_dynarray_append -> util_dynarray_append_typed bash -c "find . -type f -exec sed -i -e 's/util_dynarray_append(/util_dynarray_append_typed(/g' \{} \;" Then, add a new append that infers the type. This is much more ergonomic for what you want most of the time. Next, use type-inferred append as much as possible, via Coccinelle patch (plus manual fixup): @@ expression dynarray, element; type type; @@ -util_dynarray_append_typed(dynarray, type, element); +util_dynarray_append(dynarray, element); Finally, hand fixup cases that Coccinelle missed or incorrectly translated, of which there were several because we can't used the untyped append with a literal (since the sizeof won't do what you want). All four steps are squashed to produce a single patch changing every util_dynarray_append call site in tree to either drop a type parameter (if possible) or insert a _typed suffix (if we can't infer). As such, the final patch is best reviewed by hand even though it was tool-assisted. No Long Linguine Meals were involved in the making of this patch. Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38038>
2025-10-23 15:36:13 -04:00
util_dynarray_append_typed(&state->result_stack, uint32_t, 0);
nir/range_analysis: add helpers for limiting stack usage Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:42:22 +00:00
return q;
}
/* Helper for performing range analysis without recursion. */
static uint32_t
perform_analysis(struct analysis_state *state)
{
while (state->query_stack.size) {
struct analysis_query *cur =
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
(struct analysis_query *)((char *)util_dynarray_end(&state->query_stack) - state->query_size);
nir/range_analysis: add helpers for limiting stack usage Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:42:22 +00:00
uint32_t *result = util_dynarray_element(&state->result_stack, uint32_t, cur->result_index);
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
uint32_t key = state->get_key(cur);
nir/range_analysis: add helpers for limiting stack usage Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:42:22 +00:00
/* There might be a cycle-resolving entry for loop header phis. Ignore this when finishing
* them by testing pushed_queries.
*/
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
if (cur->pushed_queries == 0 && key != UINT32_MAX &&
state->lookup(state->range_ht, key, result)) {
nir/range_analysis: add helpers for limiting stack usage Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:42:22 +00:00
state->query_stack.size -= state->query_size;
continue;
}
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
uint32_t *src = (uint32_t *)util_dynarray_end(&state->result_stack) - cur->pushed_queries;
nir/range_analysis: add helpers for limiting stack usage Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:42:22 +00:00
state->result_stack.size -= sizeof(uint32_t) * cur->pushed_queries;
uint32_t prev_num_queries = state->query_stack.size;
state->process_query(state, cur, result, src);
uint32_t num_queries = state->query_stack.size;
if (num_queries > prev_num_queries) {
cur = (struct analysis_query *)util_dynarray_element(&state->query_stack, char,
prev_num_queries - state->query_size);
cur->pushed_queries = (num_queries - prev_num_queries) / state->query_size;
continue;
}
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
if (key != UINT32_MAX)
state->insert(state->range_ht, key, *result);
nir/range_analysis: add helpers for limiting stack usage Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:42:22 +00:00
state->query_stack.size -= state->query_size;
}
assert(state->result_stack.size == sizeof(uint32_t));
uint32_t res = util_dynarray_top(&state->result_stack, uint32_t);
util_dynarray_fini(&state->query_stack);
util_dynarray_fini(&state->result_stack);
return res;
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
static fp_class_mask
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
analyze_fp_constant(const nir_load_const_instr *const load)
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
{
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
fp_class_mask result = 0;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
for (unsigned i = 0; i < load->def.num_components; ++i) {
const double v = nir_const_value_as_float(load->value[i],
nir: remove non float nir_analyse_range support This was always unused/unfinished. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:15:42 +01:00
load->def.bit_size);
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (!isnan(v) && floor(v) != v)
result |= FP_CLASS_NON_INTEGRAL;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: remove non float nir_analyse_range support This was always unused/unfinished. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:15:42 +01:00
if (isnan(v))
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
result |= FP_CLASS_NAN;
else if (v == -INFINITY)
result |= FP_CLASS_NEG_INF;
else if (v < -1.0)
result |= FP_CLASS_LT_NEG_ONE;
else if (v == -1.0)
result |= FP_CLASS_NEG_ONE;
else if (v < 0.0)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE;
else if (dui(v) == 0)
result |= FP_CLASS_POS_ZERO;
else if (v == 0.0)
result |= FP_CLASS_NEG_ZERO;
else if (v < 1.0)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE;
else if (v == 1.0)
result |= FP_CLASS_POS_ONE;
else if (v < INFINITY)
result |= FP_CLASS_GT_POS_ONE;
else
result |= FP_CLASS_POS_INF;
if (v != 0) {
/* handle potential denorm flushing. */
bool is_denorm = false;
switch (load->def.bit_size) {
case 64:
is_denorm = fabs(v) < DBL_MIN;
break;
case 32:
is_denorm = fabs(v) < FLT_MIN;
break;
case 16:
is_denorm = fabs(v) < ldexp(1.0, -14);
break;
default:
UNREACHABLE("unsupported float size");
}
if (is_denorm)
result |= v < 0.0 ? FP_CLASS_NEG_ZERO : FP_CLASS_POS_ZERO;
}
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
return result;
nir/range-analysis: Add a lot more assertions about the contents of tables v2: Update several of the comments. Drop some redundant uses of ASSERT_UNION_OF_OTHERS_MATCHES_UNKNOWN_*_SOURCE source. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Suggested-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-08-12 15:40:20 -07:00
}
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
struct fp_query {
struct analysis_query head;
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
const nir_def *def;
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
};
static void
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
push_fp_query(struct analysis_state *state, const nir_def *def)
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
{
struct fp_query *pushed_q = push_analysis_query(state, sizeof(struct fp_query));
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
pushed_q->def = def;
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
}
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
static uint32_t
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
get_fp_key(struct analysis_query *q)
{
struct fp_query *fp_q = (struct fp_query *)q;
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
if (!nir_def_is_alu(fp_q->def))
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
return UINT32_MAX;
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
nir/range_analysis: use sparse array for float analysis This seems to be faster. ministat (nir_analyze_fp_range): Difference at 95.0% confidence -592900 +/- 2302.24 -27.6432% +/- 0.0998961% (Student's t, pooled s = 2719.05) ministat (overall): Difference at 95.0% confidence -76.8333 +/- 27.2345 -0.632558% +/- 0.223407% (Student's t, pooled s = 46.867) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:44:38 +00:00
return fp_q->def->index;
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
}
nir/range_analysis: use sparse array for float analysis This seems to be faster. ministat (nir_analyze_fp_range): Difference at 95.0% confidence -592900 +/- 2302.24 -27.6432% +/- 0.0998961% (Student's t, pooled s = 2719.05) ministat (overall): Difference at 95.0% confidence -76.8333 +/- 27.2345 -0.632558% +/- 0.223407% (Student's t, pooled s = 46.867) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:44:38 +00:00
static bool
fp_lookup(void *table, uint32_t key, uint32_t *value)
{
nir_fp_analysis_state *state = table;
if (BITSET_TEST(state->bitset, key)) {
nir/range_analysis: use uint16_t for sparse array elements ministat (nir_analyze_fp_class): Difference at 95.0% confidence -4484.55 +/- 1288.68 -0.205419% +/- 0.0589514% (Student's t, pooled s = 1521.99) This should also use less memory. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40346>
2026-03-09 09:42:03 +00:00
*value = *(uint16_t *)util_sparse_array_get(&state->arr, key);
nir/range_analysis: use sparse array for float analysis This seems to be faster. ministat (nir_analyze_fp_range): Difference at 95.0% confidence -592900 +/- 2302.24 -27.6432% +/- 0.0998961% (Student's t, pooled s = 2719.05) ministat (overall): Difference at 95.0% confidence -76.8333 +/- 27.2345 -0.632558% +/- 0.223407% (Student's t, pooled s = 46.867) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:44:38 +00:00
return true;
} else {
return false;
}
}
static void
fp_insert(void *table, uint32_t key, uint32_t value)
{
nir_fp_analysis_state *state = table;
BITSET_SET(state->bitset, key);
state->max = MAX2(state->max, (int)key);
nir/range_analysis: use uint16_t for sparse array elements ministat (nir_analyze_fp_class): Difference at 95.0% confidence -4484.55 +/- 1288.68 -0.205419% +/- 0.0589514% (Student's t, pooled s = 1521.99) This should also use less memory. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40346>
2026-03-09 09:42:03 +00:00
*(uint16_t *)util_sparse_array_get(&state->arr, key) = value;
nir/range_analysis: use sparse array for float analysis This seems to be faster. ministat (nir_analyze_fp_range): Difference at 95.0% confidence -592900 +/- 2302.24 -27.6432% +/- 0.0998961% (Student's t, pooled s = 2719.05) ministat (overall): Difference at 95.0% confidence -76.8333 +/- 27.2345 -0.632558% +/- 0.223407% (Student's t, pooled s = 46.867) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:44:38 +00:00
}
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
static fp_class_mask
fneg_fp_class(fp_class_mask src)
nir: add is_a_number analysis for ffma Foz-DB Navi21: Totals from 508 (0.64% of 79789) affected shaders: Instrs: 796183 -> 795838 (-0.04%) CodeSize: 4303420 -> 4303384 (-0.00%); split: -0.00%, +0.00% Latency: 7806095 -> 7805458 (-0.01%); split: -0.01%, +0.00% InvThroughput: 1377028 -> 1376824 (-0.01%); split: -0.01%, +0.00% Copies: 63297 -> 63299 (+0.00%); split: -0.00%, +0.00% PreVGPRs: 29818 -> 29819 (+0.00%) VALU: 562067 -> 561885 (-0.03%); split: -0.03%, +0.00% SALU: 89896 -> 89733 (-0.18%) Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34125>
2025-03-22 15:58:52 +01:00
{
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
fp_class_mask result = src & (FP_CLASS_NAN | FP_CLASS_NON_INTEGRAL);
#define NEG_BIT(neg, pos) \
if (src & FP_CLASS_##pos) \
result |= FP_CLASS_##neg; \
if (src & FP_CLASS_##neg) \
result |= FP_CLASS_##pos;
NEG_BIT(NEG_INF, POS_INF);
NEG_BIT(LT_NEG_ONE, GT_POS_ONE);
NEG_BIT(NEG_ONE, POS_ONE);
NEG_BIT(LT_ZERO_GT_NEG_ONE, GT_ZERO_LT_POS_ONE);
NEG_BIT(NEG_ZERO, POS_ZERO);
#undef NEG_BIT
return result;
}
static fp_class_mask
fmul_fp_class(fp_class_mask left, fp_class_mask right, bool mulz, bool src_eq, bool src_neg_eq)
{
/* For runtime performance, shortcut the common completely unknown case. */
if (left == FP_CLASS_UNKNOWN && right == FP_CLASS_UNKNOWN && !src_eq && !src_neg_eq)
return FP_CLASS_UNKNOWN;
fp_class_mask result = 0;
if (left & FP_CLASS_NAN) {
if (right & FP_CLASS_ANY_ZERO)
result |= mulz ? FP_CLASS_POS_ZERO : FP_CLASS_NAN;
if (right & (FP_CLASS_ANY_NEG | FP_CLASS_ANY_POS | FP_CLASS_NAN))
result |= FP_CLASS_NAN;
}
if (left & FP_CLASS_NEG_INF) {
if (right & FP_CLASS_ANY_ZERO)
result |= mulz ? FP_CLASS_POS_ZERO : FP_CLASS_NAN;
if (right & FP_CLASS_ANY_NEG)
result |= FP_CLASS_POS_INF;
if (right & FP_CLASS_ANY_POS)
result |= FP_CLASS_NEG_INF;
if (right & FP_CLASS_NAN)
result |= FP_CLASS_NAN;
}
if (left & FP_CLASS_POS_INF) {
if (right & FP_CLASS_ANY_ZERO)
result |= mulz ? FP_CLASS_POS_ZERO : FP_CLASS_NAN;
if (right & FP_CLASS_ANY_POS)
result |= FP_CLASS_POS_INF;
if (right & FP_CLASS_ANY_NEG)
result |= FP_CLASS_NEG_INF;
if (right & FP_CLASS_NAN)
result |= FP_CLASS_NAN;
}
if (left & FP_CLASS_ANY_NEG_FINITE) {
if (right & FP_CLASS_POS_ZERO)
result |= mulz ? FP_CLASS_POS_ZERO : FP_CLASS_NEG_ZERO;
result |= fneg_fp_class(right & (FP_CLASS_NEG_ZERO | FP_CLASS_POS_INF | FP_CLASS_NEG_INF | FP_CLASS_NAN));
}
if (left & FP_CLASS_ANY_POS_FINITE) {
if (right & FP_CLASS_NEG_ZERO)
result |= mulz ? FP_CLASS_POS_ZERO : FP_CLASS_NEG_ZERO;
result |= right & (FP_CLASS_POS_ZERO | FP_CLASS_POS_INF | FP_CLASS_NEG_INF | FP_CLASS_NAN);
}
if (left & FP_CLASS_LT_NEG_ONE) {
if (right & FP_CLASS_LT_NEG_ONE)
result |= FP_CLASS_POS_INF | FP_CLASS_GT_POS_ONE;
if (right & FP_CLASS_NEG_ONE)
result |= FP_CLASS_GT_POS_ONE;
if (right & FP_CLASS_LT_ZERO_GT_NEG_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_GT_POS_ONE;
if (right & FP_CLASS_GT_POS_ONE)
result |= FP_CLASS_NEG_INF | FP_CLASS_LT_NEG_ONE;
if (right & FP_CLASS_POS_ONE)
result |= FP_CLASS_LT_NEG_ONE;
if (right & FP_CLASS_GT_ZERO_LT_POS_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_NEG_ONE | FP_CLASS_LT_NEG_ONE;
}
if (left & FP_CLASS_GT_POS_ONE) {
if (right & FP_CLASS_GT_POS_ONE)
result |= FP_CLASS_POS_INF | FP_CLASS_GT_POS_ONE;
if (right & FP_CLASS_POS_ONE)
result |= FP_CLASS_GT_POS_ONE;
if (right & FP_CLASS_GT_ZERO_LT_POS_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_GT_POS_ONE;
if (right & FP_CLASS_LT_NEG_ONE)
result |= FP_CLASS_NEG_INF | FP_CLASS_LT_NEG_ONE;
if (right & FP_CLASS_NEG_ONE)
result |= FP_CLASS_LT_NEG_ONE;
if (right & FP_CLASS_LT_ZERO_GT_NEG_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_NEG_ONE | FP_CLASS_LT_NEG_ONE;
}
if (left & FP_CLASS_NEG_ONE)
result |= fneg_fp_class(right & ~FP_CLASS_ANY_ZERO);
if (left & FP_CLASS_POS_ONE)
result |= right & ~FP_CLASS_ANY_ZERO;
if (left & FP_CLASS_LT_ZERO_GT_NEG_ONE) {
if (right & FP_CLASS_LT_NEG_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_GT_POS_ONE;
if (right & FP_CLASS_NEG_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE;
if (right & FP_CLASS_LT_ZERO_GT_NEG_ONE)
result |= FP_CLASS_POS_ZERO | FP_CLASS_GT_ZERO_LT_POS_ONE;
if (right & FP_CLASS_GT_POS_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_NEG_ONE | FP_CLASS_LT_NEG_ONE;
if (right & FP_CLASS_POS_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE;
if (right & FP_CLASS_GT_ZERO_LT_POS_ONE)
result |= FP_CLASS_NEG_ZERO | FP_CLASS_LT_ZERO_GT_NEG_ONE;
}
if (left & FP_CLASS_GT_ZERO_LT_POS_ONE) {
if (right & FP_CLASS_GT_POS_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_GT_POS_ONE;
if (right & FP_CLASS_POS_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE;
if (right & FP_CLASS_GT_ZERO_LT_POS_ONE)
result |= FP_CLASS_POS_ZERO | FP_CLASS_GT_ZERO_LT_POS_ONE;
if (right & FP_CLASS_LT_NEG_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_NEG_ONE | FP_CLASS_LT_NEG_ONE;
if (right & FP_CLASS_NEG_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE;
if (right & FP_CLASS_LT_ZERO_GT_NEG_ONE)
result |= FP_CLASS_NEG_ZERO | FP_CLASS_LT_ZERO_GT_NEG_ONE;
}
if (left & FP_CLASS_NEG_ZERO) {
if (mulz) {
result |= FP_CLASS_POS_ZERO;
} else {
if (right & (FP_CLASS_ANY_INF | FP_CLASS_NAN))
result |= FP_CLASS_NAN;
if (right & (FP_CLASS_ANY_NEG_FINITE | FP_CLASS_NEG_ZERO))
result |= FP_CLASS_POS_ZERO;
if (right & (FP_CLASS_ANY_POS_FINITE | FP_CLASS_POS_ZERO))
result |= FP_CLASS_NEG_ZERO;
}
}
if (left & FP_CLASS_POS_ZERO) {
if (mulz) {
result |= FP_CLASS_POS_ZERO;
} else {
if (right & (FP_CLASS_ANY_INF | FP_CLASS_NAN))
result |= FP_CLASS_NAN;
if (right & (FP_CLASS_ANY_POS_FINITE | FP_CLASS_POS_ZERO))
result |= FP_CLASS_POS_ZERO;
if (right & (FP_CLASS_ANY_NEG_FINITE | FP_CLASS_NEG_ZERO))
result |= FP_CLASS_NEG_ZERO;
}
}
if (src_eq || src_neg_eq) {
/* This case can't create new ones. */
if (!(left & (FP_CLASS_POS_ONE | FP_CLASS_NEG_ONE)))
result &= ~(FP_CLASS_POS_ONE | FP_CLASS_NEG_ONE);
if (src_eq)
result &= ~(FP_CLASS_ANY_NEG | FP_CLASS_NEG_ZERO);
else if (src_neg_eq && mulz)
result &= ~FP_CLASS_ANY_POS;
else if (src_neg_eq)
result &= ~(FP_CLASS_ANY_POS | FP_CLASS_POS_ZERO);
}
if ((left | right) & FP_CLASS_NON_INTEGRAL) {
if (result & (FP_CLASS_LT_NEG_ONE | FP_CLASS_LT_ZERO_GT_NEG_ONE |
FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_GT_POS_ONE))
result |= FP_CLASS_NON_INTEGRAL;
nir: add is_a_number analysis for ffma Foz-DB Navi21: Totals from 508 (0.64% of 79789) affected shaders: Instrs: 796183 -> 795838 (-0.04%) CodeSize: 4303420 -> 4303384 (-0.00%); split: -0.00%, +0.00% Latency: 7806095 -> 7805458 (-0.01%); split: -0.01%, +0.00% InvThroughput: 1377028 -> 1376824 (-0.01%); split: -0.01%, +0.00% Copies: 63297 -> 63299 (+0.00%); split: -0.00%, +0.00% PreVGPRs: 29818 -> 29819 (+0.00%) VALU: 562067 -> 561885 (-0.03%); split: -0.03%, +0.00% SALU: 89896 -> 89733 (-0.18%) Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34125>
2025-03-22 15:58:52 +01:00
} else {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
result &= ~(FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_GT_ZERO_LT_POS_ONE);
nir: add is_a_number analysis for ffma Foz-DB Navi21: Totals from 508 (0.64% of 79789) affected shaders: Instrs: 796183 -> 795838 (-0.04%) CodeSize: 4303420 -> 4303384 (-0.00%); split: -0.00%, +0.00% Latency: 7806095 -> 7805458 (-0.01%); split: -0.01%, +0.00% InvThroughput: 1377028 -> 1376824 (-0.01%); split: -0.01%, +0.00% Copies: 63297 -> 63299 (+0.00%); split: -0.00%, +0.00% PreVGPRs: 29818 -> 29819 (+0.00%) VALU: 562067 -> 561885 (-0.03%); split: -0.03%, +0.00% SALU: 89896 -> 89733 (-0.18%) Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34125>
2025-03-22 15:58:52 +01:00
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
return result;
nir: add is_a_number analysis for ffma Foz-DB Navi21: Totals from 508 (0.64% of 79789) affected shaders: Instrs: 796183 -> 795838 (-0.04%) CodeSize: 4303420 -> 4303384 (-0.00%); split: -0.00%, +0.00% Latency: 7806095 -> 7805458 (-0.01%); split: -0.01%, +0.00% InvThroughput: 1377028 -> 1376824 (-0.01%); split: -0.01%, +0.00% Copies: 63297 -> 63299 (+0.00%); split: -0.00%, +0.00% PreVGPRs: 29818 -> 29819 (+0.00%) VALU: 562067 -> 561885 (-0.03%); split: -0.03%, +0.00% SALU: 89896 -> 89733 (-0.18%) Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34125>
2025-03-22 15:58:52 +01:00
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
static fp_class_mask
fadd_fp_class(fp_class_mask left, fp_class_mask right)
nir: add is_a_number analysis for ffma Foz-DB Navi21: Totals from 508 (0.64% of 79789) affected shaders: Instrs: 796183 -> 795838 (-0.04%) CodeSize: 4303420 -> 4303384 (-0.00%); split: -0.00%, +0.00% Latency: 7806095 -> 7805458 (-0.01%); split: -0.01%, +0.00% InvThroughput: 1377028 -> 1376824 (-0.01%); split: -0.01%, +0.00% Copies: 63297 -> 63299 (+0.00%); split: -0.00%, +0.00% PreVGPRs: 29818 -> 29819 (+0.00%) VALU: 562067 -> 561885 (-0.03%); split: -0.03%, +0.00% SALU: 89896 -> 89733 (-0.18%) Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34125>
2025-03-22 15:58:52 +01:00
{
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
/* For runtime performance, shortcut the common completely unknown case. */
if (left == FP_CLASS_UNKNOWN && right == FP_CLASS_UNKNOWN)
return FP_CLASS_UNKNOWN;
fp_class_mask result = (left | right) & FP_CLASS_NAN;
nir: add is_a_number analysis for ffma Foz-DB Navi21: Totals from 508 (0.64% of 79789) affected shaders: Instrs: 796183 -> 795838 (-0.04%) CodeSize: 4303420 -> 4303384 (-0.00%); split: -0.00%, +0.00% Latency: 7806095 -> 7805458 (-0.01%); split: -0.01%, +0.00% InvThroughput: 1377028 -> 1376824 (-0.01%); split: -0.01%, +0.00% Copies: 63297 -> 63299 (+0.00%); split: -0.00%, +0.00% PreVGPRs: 29818 -> 29819 (+0.00%) VALU: 562067 -> 561885 (-0.03%); split: -0.03%, +0.00% SALU: 89896 -> 89733 (-0.18%) Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34125>
2025-03-22 15:58:52 +01:00
/* X + Y is NaN if either operand is NaN or if one operand is +Inf and
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
* the other is -Inf.
nir: add is_a_number analysis for ffma Foz-DB Navi21: Totals from 508 (0.64% of 79789) affected shaders: Instrs: 796183 -> 795838 (-0.04%) CodeSize: 4303420 -> 4303384 (-0.00%); split: -0.00%, +0.00% Latency: 7806095 -> 7805458 (-0.01%); split: -0.01%, +0.00% InvThroughput: 1377028 -> 1376824 (-0.01%); split: -0.01%, +0.00% Copies: 63297 -> 63299 (+0.00%); split: -0.00%, +0.00% PreVGPRs: 29818 -> 29819 (+0.00%) VALU: 562067 -> 561885 (-0.03%); split: -0.03%, +0.00% SALU: 89896 -> 89733 (-0.18%) Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34125>
2025-03-22 15:58:52 +01:00
*/
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (left & FP_CLASS_NEG_INF) {
if (right & FP_CLASS_POS_INF)
result |= FP_CLASS_NAN;
if (right & (FP_CLASS_ANY_FINITE | FP_CLASS_NEG_INF))
result |= FP_CLASS_NEG_INF;
}
if (left & FP_CLASS_POS_INF) {
if (right & FP_CLASS_NEG_INF)
result |= FP_CLASS_NAN;
if (right & (FP_CLASS_ANY_FINITE | FP_CLASS_POS_INF))
result |= FP_CLASS_POS_INF;
}
if (left & FP_CLASS_LT_NEG_ONE) {
result |= (right & FP_CLASS_ANY_INF);
if (right & FP_CLASS_LT_NEG_ONE)
result |= FP_CLASS_NEG_INF | FP_CLASS_LT_NEG_ONE;
if (right & (FP_CLASS_NEG_ONE | FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_ANY_ZERO))
result |= FP_CLASS_LT_NEG_ONE;
if (right & (FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ONE))
result |= FP_CLASS_LT_NEG_ONE | FP_CLASS_NEG_ONE | FP_CLASS_LT_ZERO_GT_NEG_ONE;
if (right & FP_CLASS_GT_POS_ONE)
result |= FP_CLASS_ANY_FINITE;
}
if (left & FP_CLASS_GT_POS_ONE) {
result |= (right & FP_CLASS_ANY_INF);
if (right & FP_CLASS_GT_POS_ONE)
result |= FP_CLASS_POS_INF | FP_CLASS_GT_POS_ONE;
if (right & (FP_CLASS_POS_ONE | FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_ANY_ZERO))
result |= FP_CLASS_GT_POS_ONE;
if (right & (FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_NEG_ONE))
result |= FP_CLASS_GT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_GT_ZERO_LT_POS_ONE;
if (right & FP_CLASS_LT_NEG_ONE)
result |= FP_CLASS_ANY_FINITE;
}
if (left & FP_CLASS_NEG_ONE) {
result |= (right & FP_CLASS_ANY_INF);
if (right & (FP_CLASS_LT_NEG_ONE | FP_CLASS_NEG_ONE | FP_CLASS_LT_ZERO_GT_NEG_ONE))
result |= FP_CLASS_LT_NEG_ONE;
if (right & (FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_ANY_ZERO | FP_CLASS_GT_ZERO_LT_POS_ONE))
result |= FP_CLASS_NEG_ONE;
if (right & FP_CLASS_GT_ZERO_LT_POS_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE;
if (right & FP_CLASS_POS_ONE)
result |= FP_CLASS_POS_ZERO;
if (right & FP_CLASS_GT_POS_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_GT_POS_ONE;
}
if (left & FP_CLASS_POS_ONE) {
result |= (right & FP_CLASS_ANY_INF);
if (right & (FP_CLASS_GT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_GT_ZERO_LT_POS_ONE))
result |= FP_CLASS_GT_POS_ONE;
if (right & (FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_ANY_ZERO | FP_CLASS_LT_ZERO_GT_NEG_ONE))
result |= FP_CLASS_POS_ONE;
if (right & FP_CLASS_LT_ZERO_GT_NEG_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE;
if (right & FP_CLASS_NEG_ONE)
result |= FP_CLASS_POS_ZERO;
if (right & FP_CLASS_LT_NEG_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_NEG_ONE | FP_CLASS_LT_NEG_ONE;
}
if (left & FP_CLASS_LT_ZERO_GT_NEG_ONE) {
result |= (right & FP_CLASS_ANY_INF);
if (right & FP_CLASS_LT_NEG_ONE)
result |= FP_CLASS_LT_NEG_ONE;
if (right & FP_CLASS_NEG_ONE)
result |= FP_CLASS_LT_NEG_ONE | FP_CLASS_NEG_ONE;
if (right & FP_CLASS_LT_ZERO_GT_NEG_ONE)
result |= FP_CLASS_LT_NEG_ONE | FP_CLASS_NEG_ONE | FP_CLASS_LT_ZERO_GT_NEG_ONE;
if (right & FP_CLASS_ANY_ZERO)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE;
if (right & FP_CLASS_GT_ZERO_LT_POS_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_ANY_ZERO | FP_CLASS_GT_ZERO_LT_POS_ONE;
if (right & FP_CLASS_POS_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ONE;
if (right & FP_CLASS_GT_POS_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_GT_POS_ONE;
}
if (left & FP_CLASS_GT_ZERO_LT_POS_ONE) {
result |= (right & FP_CLASS_ANY_INF);
if (right & FP_CLASS_GT_POS_ONE)
result |= FP_CLASS_GT_POS_ONE;
if (right & FP_CLASS_POS_ONE)
result |= FP_CLASS_GT_POS_ONE | FP_CLASS_POS_ONE;
if (right & FP_CLASS_GT_ZERO_LT_POS_ONE)
result |= FP_CLASS_GT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_GT_ZERO_LT_POS_ONE;
if (right & FP_CLASS_ANY_ZERO)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE;
if (right & FP_CLASS_LT_ZERO_GT_NEG_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_ANY_ZERO | FP_CLASS_LT_ZERO_GT_NEG_ONE;
if (right & FP_CLASS_NEG_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_NEG_ONE;
if (right & FP_CLASS_LT_NEG_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_NEG_ONE | FP_CLASS_LT_NEG_ONE;
}
if (left & FP_CLASS_NEG_ZERO) {
result |= right;
}
if (left & FP_CLASS_POS_ZERO) {
result |= right & ~FP_CLASS_NEG_ZERO;
if (right & FP_CLASS_NEG_ZERO)
result |= FP_CLASS_POS_ZERO;
}
if ((left | right) & FP_CLASS_NON_INTEGRAL) {
if (result & (FP_CLASS_LT_NEG_ONE | FP_CLASS_LT_ZERO_GT_NEG_ONE |
FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_GT_POS_ONE))
result |= FP_CLASS_NON_INTEGRAL;
} else {
result &= ~(FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_GT_ZERO_LT_POS_ONE);
}
return result;
}
static fp_class_mask
frcp_fp_class(fp_class_mask src)
{
fp_class_mask result = src & FP_CLASS_NAN;
/* Inf/Zero result in Zero/Inf.*/
if (src & FP_CLASS_NEG_INF)
result |= FP_CLASS_NEG_ZERO;
if (src & FP_CLASS_POS_INF)
result |= FP_CLASS_POS_ZERO;
if (src & FP_CLASS_NEG_ZERO)
result |= FP_CLASS_NEG_INF;
if (src & FP_CLASS_POS_ZERO)
result |= FP_CLASS_POS_INF;
/* One results in one. */
if (src & FP_CLASS_NEG_ONE)
result |= FP_CLASS_NEG_ONE;
if (src & FP_CLASS_POS_ONE)
result |= FP_CLASS_POS_ONE;
if (src & FP_CLASS_LT_NEG_ONE)
result |= FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_NEG_ZERO | FP_CLASS_NON_INTEGRAL;
if (src & FP_CLASS_GT_POS_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ZERO | FP_CLASS_NON_INTEGRAL;
if (src & FP_CLASS_LT_ZERO_GT_NEG_ONE)
result |= FP_CLASS_LT_NEG_ONE | FP_CLASS_NEG_INF | FP_CLASS_NON_INTEGRAL;
if (src & FP_CLASS_GT_ZERO_LT_POS_ONE)
result |= FP_CLASS_GT_POS_ONE | FP_CLASS_POS_INF | FP_CLASS_NON_INTEGRAL;
return result;
}
static fp_class_mask
fsqrt_fp_class(fp_class_mask src)
{
fp_class_mask result = src & (FP_CLASS_NAN | FP_CLASS_ANY_ZERO);
if (src & FP_CLASS_ANY_NEG)
result |= FP_CLASS_NAN;
if (src & FP_CLASS_GT_ZERO_LT_POS_ONE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_NON_INTEGRAL;
if (src & FP_CLASS_POS_ONE)
result |= FP_CLASS_POS_ONE;
if (src & FP_CLASS_GT_POS_ONE)
result |= FP_CLASS_GT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_NON_INTEGRAL;
if (src & FP_CLASS_POS_INF)
result |= FP_CLASS_POS_INF;
return result;
}
static fp_class_mask
fmin_part(fp_class_mask upper_bound, fp_class_mask value)
{
/* Find the highest value in upper_bound, and return all
* smaller or equal values from value.
*/
upper_bound &= FP_CLASS_ANY_NEG | FP_CLASS_ANY_ZERO | FP_CLASS_ANY_POS;
value &= FP_CLASS_ANY_NEG | FP_CLASS_ANY_ZERO | FP_CLASS_ANY_POS;
/* This works even in the case where upper_bound is 0 */
return value & BITFIELD_MASK(util_last_bit(upper_bound));
}
static fp_class_mask
fmin_fp_class(fp_class_mask left, fp_class_mask right)
{
fp_class_mask result = 0;
/* If one source is NaN, we have to include the whole range of the other source. */
if (left & FP_CLASS_NAN)
result |= right;
if (right & FP_CLASS_NAN)
result |= left;
result |= fmin_part(left, right);
result |= fmin_part(right, left);
/* Could probably do better, but meh. */
if ((left | right) & FP_CLASS_NON_INTEGRAL) {
if (result & (FP_CLASS_LT_NEG_ONE | FP_CLASS_LT_ZERO_GT_NEG_ONE |
FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_GT_POS_ONE))
result |= FP_CLASS_NON_INTEGRAL;
}
return result;
}
static fp_class_mask
handle_sz(const nir_alu_instr *alu, fp_class_mask src)
{
if (nir_alu_instr_is_signed_zero_preserve(alu) || !(src & FP_CLASS_ANY_ZERO))
return src;
return src | FP_CLASS_ANY_ZERO;
nir: add is_a_number analysis for ffma Foz-DB Navi21: Totals from 508 (0.64% of 79789) affected shaders: Instrs: 796183 -> 795838 (-0.04%) CodeSize: 4303420 -> 4303384 (-0.00%); split: -0.00%, +0.00% Latency: 7806095 -> 7805458 (-0.01%); split: -0.01%, +0.00% InvThroughput: 1377028 -> 1376824 (-0.01%); split: -0.01%, +0.00% Copies: 63297 -> 63299 (+0.00%); split: -0.00%, +0.00% PreVGPRs: 29818 -> 29819 (+0.00%) VALU: 562067 -> 561885 (-0.03%); split: -0.03%, +0.00% SALU: 89896 -> 89733 (-0.18%) Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34125>
2025-03-22 15:58:52 +01:00
}
nir: add fp class analysis for some intrinsics I also tried ddx/ddy, but that was not worth it. Foz-DB Navi48: Totals from 1019 (1.23% of 82636) affected shaders: Instrs: 516459 -> 515700 (-0.15%); split: -0.17%, +0.02% CodeSize: 2712428 -> 2707008 (-0.20%); split: -0.21%, +0.01% VGPRs: 70152 -> 70140 (-0.02%) Latency: 1799198 -> 1795926 (-0.18%); split: -0.19%, +0.00% InvThroughput: 233497 -> 232628 (-0.37%); split: -0.37%, +0.00% VClause: 15315 -> 15346 (+0.20%); split: -0.11%, +0.31% Copies: 30009 -> 30035 (+0.09%); split: -0.06%, +0.14% VALU: 305519 -> 304727 (-0.26%); split: -0.27%, +0.01% SALU: 45855 -> 45854 (-0.00%) Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-12 10:17:46 +01:00
static fp_class_mask
intrinsic_fp_class(const nir_intrinsic_instr *intrin)
{
switch (intrin->intrinsic) {
case nir_intrinsic_load_typed_buffer_amd: {
const enum pipe_format format = nir_intrinsic_format(intrin);
if (format == PIPE_FORMAT_NONE)
return FP_CLASS_UNKNOWN;
const struct util_format_description *desc = util_format_description(format);
int i = util_format_get_first_non_void_channel(format);
if (i == -1)
return FP_CLASS_UNKNOWN;
bool is_signed = desc->channel[i].type == UTIL_FORMAT_TYPE_SIGNED;
bool is_unsigned = desc->channel[i].type == UTIL_FORMAT_TYPE_UNSIGNED;
bool normalized = desc->channel[i].normalized;
if ((!is_signed && !is_unsigned) || desc->channel[i].pure_integer)
return FP_CLASS_UNKNOWN;
fp_class_mask result = FP_CLASS_POS_ZERO | FP_CLASS_POS_ONE;
result |= is_signed ? FP_CLASS_NEG_ONE : 0;
if (normalized) {
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_NON_INTEGRAL;
result |= is_signed ? FP_CLASS_LT_ZERO_GT_NEG_ONE : 0;
} else {
result |= FP_CLASS_GT_POS_ONE;
result |= is_signed ? FP_CLASS_LT_NEG_ONE : 0;
}
return result;
}
case nir_intrinsic_load_front_face_fsign:
return FP_CLASS_POS_ONE | FP_CLASS_NEG_ONE;
default:
return FP_CLASS_UNKNOWN;
}
}
nir: add fp class analysis for shadow compare Foz-DB Navi48: Totals from 145 (0.18% of 82636) affected shaders: Instrs: 280871 -> 280729 (-0.05%) CodeSize: 1545724 -> 1545488 (-0.02%); split: -0.02%, +0.00% Latency: 10840265 -> 10840216 (-0.00%); split: -0.00%, +0.00% InvThroughput: 2093707 -> 2093646 (-0.00%) SClause: 4483 -> 4481 (-0.04%) VALU: 188142 -> 188039 (-0.05%) SALU: 22238 -> 22236 (-0.01%) Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-13 13:37:40 +01:00
static fp_class_mask
tex_fp_class(nir_tex_instr *tex)
{
/* Not much to analyze, except shadow compare. */
if (!tex->is_shadow)
return FP_CLASS_UNKNOWN;
fp_class_mask result = FP_CLASS_POS_ZERO | FP_CLASS_POS_ONE;
/* Gather returns 0 or 1, other ops can interpolate.
* Cube corners are special even for gathers.
*/
if (tex->op != nir_texop_tg4 || tex->sampler_dim == GLSL_SAMPLER_DIM_CUBE)
result |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_NON_INTEGRAL;
return result;
}
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
/**
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
* Analyze an expression to determine the possible fp classes of its result
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
*/
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
static void
process_fp_query(struct analysis_state *state, struct analysis_query *aq, uint32_t *result,
const uint32_t *src_res)
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
{
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
struct fp_query q = *(struct fp_query *)aq;
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
const nir_def *def = q.def;
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
if (nir_def_is_const(def)) {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
*result = analyze_fp_constant(nir_def_as_load_const(def));
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
return;
nir: add fp class analysis for some intrinsics I also tried ddx/ddy, but that was not worth it. Foz-DB Navi48: Totals from 1019 (1.23% of 82636) affected shaders: Instrs: 516459 -> 515700 (-0.15%); split: -0.17%, +0.02% CodeSize: 2712428 -> 2707008 (-0.20%); split: -0.21%, +0.01% VGPRs: 70152 -> 70140 (-0.02%) Latency: 1799198 -> 1795926 (-0.18%); split: -0.19%, +0.00% InvThroughput: 233497 -> 232628 (-0.37%); split: -0.37%, +0.00% VClause: 15315 -> 15346 (+0.20%); split: -0.11%, +0.31% Copies: 30009 -> 30035 (+0.09%); split: -0.06%, +0.14% VALU: 305519 -> 304727 (-0.26%); split: -0.27%, +0.01% SALU: 45855 -> 45854 (-0.00%) Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-12 10:17:46 +01:00
} else if (nir_def_is_intrinsic(def)) {
*result = intrinsic_fp_class(nir_def_as_intrinsic(def));
return;
nir: add fp class analysis for shadow compare Foz-DB Navi48: Totals from 145 (0.18% of 82636) affected shaders: Instrs: 280871 -> 280729 (-0.05%) CodeSize: 1545724 -> 1545488 (-0.02%); split: -0.02%, +0.00% Latency: 10840265 -> 10840216 (-0.00%); split: -0.00%, +0.00% InvThroughput: 2093707 -> 2093646 (-0.00%) SClause: 4483 -> 4481 (-0.04%) VALU: 188142 -> 188039 (-0.05%) SALU: 22238 -> 22236 (-0.01%) Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-13 13:37:40 +01:00
} else if (nir_def_is_tex(def)) {
*result = tex_fp_class(nir_def_as_tex(def));
return;
nir: add fp class analysis for some intrinsics I also tried ddx/ddy, but that was not worth it. Foz-DB Navi48: Totals from 1019 (1.23% of 82636) affected shaders: Instrs: 516459 -> 515700 (-0.15%); split: -0.17%, +0.02% CodeSize: 2712428 -> 2707008 (-0.20%); split: -0.21%, +0.01% VGPRs: 70152 -> 70140 (-0.02%) Latency: 1799198 -> 1795926 (-0.18%); split: -0.19%, +0.00% InvThroughput: 233497 -> 232628 (-0.37%); split: -0.37%, +0.00% VClause: 15315 -> 15346 (+0.20%); split: -0.11%, +0.31% Copies: 30009 -> 30035 (+0.09%); split: -0.06%, +0.14% VALU: 305519 -> 304727 (-0.26%); split: -0.27%, +0.01% SALU: 45855 -> 45854 (-0.00%) Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-12 10:17:46 +01:00
} else if (!nir_def_is_alu(def)) {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
*result = FP_CLASS_UNKNOWN;
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
return;
}
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
const nir_alu_instr *const alu = nir_def_as_alu(def);
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
if (!aq->pushed_queries) {
switch (alu->op) {
case nir_op_bcsel:
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
push_fp_query(state, alu->src[1].src.ssa);
push_fp_query(state, alu->src[2].src.ssa);
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
return;
case nir_op_mov:
case nir_op_fabs:
case nir_op_fexp2:
nir: add fp class analysis for flog2 Foz-DB Navi48: Totals from 230 (0.28% of 82636) affected shaders: Instrs: 599005 -> 598615 (-0.07%); split: -0.09%, +0.02% CodeSize: 3110528 -> 3103136 (-0.24%); split: -0.24%, +0.00% Latency: 3661526 -> 3663241 (+0.05%); split: -0.01%, +0.05% InvThroughput: 526561 -> 526487 (-0.01%); split: -0.01%, +0.00% Copies: 33735 -> 33820 (+0.25%); split: -0.06%, +0.31% VALU: 378034 -> 377904 (-0.03%); split: -0.03%, +0.00% SALU: 65156 -> 65045 (-0.17%); split: -0.19%, +0.02% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-11 19:58:41 +01:00
case nir_op_flog2:
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
case nir_op_frcp:
nir: improve fsqrt range analysis Foz-DB Navi21: Totals from 3 (0.00% of 79377) affected shaders: MaxWaves: 88 -> 96 (+9.09%) Instrs: 1058 -> 951 (-10.11%) CodeSize: 5964 -> 5368 (-9.99%) VGPRs: 104 -> 96 (-7.69%) Latency: 15283 -> 14099 (-7.75%); split: -8.37%, +0.62% InvThroughput: 4951 -> 4238 (-14.40%) Copies: 81 -> 76 (-6.17%) PreVGPRs: 93 -> 84 (-9.68%) VALU: 820 -> 737 (-10.12%) SALU: 115 -> 91 (-20.87%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 20:13:58 +01:00
case nir_op_fsqrt:
nir: fix frsq range analysis Foz-DB Navi21: Totals from 98 (0.12% of 79377) affected shaders: Instrs: 157311 -> 157675 (+0.23%); split: -0.03%, +0.26% CodeSize: 844296 -> 846648 (+0.28%); split: -0.00%, +0.28% Latency: 1275467 -> 1276259 (+0.06%); split: -0.00%, +0.06% InvThroughput: 266980 -> 267098 (+0.04%); split: -0.03%, +0.07% Copies: 11094 -> 11093 (-0.01%) PreVGPRs: 5945 -> 5977 (+0.54%) VALU: 110585 -> 110953 (+0.33%); split: -0.04%, +0.38% SALU: 18481 -> 18476 (-0.03%) Cc: mesa-stable Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 20:05:54 +01:00
case nir_op_frsq:
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
case nir_op_fneg:
case nir_op_fsat:
case nir_op_fsign:
case nir_op_ffloor:
case nir_op_fceil:
case nir_op_ftrunc:
nir: add fp class analysis for fround_even Foz-DB Navi48: Totals from 383 (0.33% of 114655) affected shaders: MaxWaves: 9806 -> 9808 (+0.02%) Instrs: 502508 -> 501762 (-0.15%); split: -0.16%, +0.01% CodeSize: 2711404 -> 2707604 (-0.14%); split: -0.15%, +0.01% VGPRs: 24360 -> 24348 (-0.05%) Latency: 2068105 -> 2066817 (-0.06%); split: -0.07%, +0.01% InvThroughput: 370962 -> 370081 (-0.24%) VClause: 7045 -> 7041 (-0.06%) SClause: 10551 -> 10559 (+0.08%); split: -0.08%, +0.15% Copies: 29135 -> 29117 (-0.06%); split: -0.12%, +0.05% Branches: 17333 -> 17328 (-0.03%) PreSGPRs: 21511 -> 21510 (-0.00%) PreVGPRs: 18555 -> 18545 (-0.05%) VALU: 274445 -> 273874 (-0.21%); split: -0.21%, +0.00% SALU: 78819 -> 78779 (-0.05%); split: -0.07%, +0.02% VMEM: 10918 -> 10913 (-0.05%) SMEM: 17662 -> 17656 (-0.03%) Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-03-02 17:58:23 +01:00
case nir_op_fround_even:
nir: range analysis for ffract Foz-DB Navi21: Totals from 75 (0.09% of 79377) affected shaders: Instrs: 69239 -> 68383 (-1.24%) CodeSize: 385088 -> 379532 (-1.44%) Latency: 427188 -> 421729 (-1.28%); split: -1.28%, +0.00% InvThroughput: 103086 -> 101926 (-1.13%) VClause: 785 -> 753 (-4.08%) SClause: 1624 -> 1598 (-1.60%) Copies: 5679 -> 5671 (-0.14%); split: -0.72%, +0.58% PreSGPRs: 3961 -> 3937 (-0.61%) VALU: 51107 -> 50457 (-1.27%) SALU: 9034 -> 8950 (-0.93%) VMEM: 1123 -> 1091 (-2.85%) SMEM: 2862 -> 2830 (-1.12%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 18:54:09 +01:00
case nir_op_ffract:
nir: add fp class analysis for sin/cos Foz-DB Navi48: Totals from 264 (0.32% of 82636) affected shaders: CodeSize: 1688676 -> 1688672 (-0.00%) Latency: 510773 -> 510772 (-0.00%) InvThroughput: 138569 -> 138568 (-0.00%) Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-11 19:53:46 +01:00
case nir_op_fsin:
case nir_op_fcos:
case nir_op_fsin_amd:
case nir_op_fcos_amd:
nir/range_analysis: look through f2f Foz-DB Navi31: Totals from 93 (0.12% of 80273) affected shaders: Instrs: 123927 -> 121073 (-2.30%); split: -2.30%, +0.00% CodeSize: 670832 -> 653332 (-2.61%); split: -2.61%, +0.00% Latency: 337678 -> 322803 (-4.41%); split: -4.41%, +0.00% InvThroughput: 63277 -> 61083 (-3.47%) VClause: 460 -> 373 (-18.91%) SClause: 2178 -> 2100 (-3.58%) Copies: 7637 -> 7744 (+1.40%) PreSGPRs: 4414 -> 4287 (-2.88%) PreVGPRs: 4229 -> 4230 (+0.02%) VALU: 77375 -> 75693 (-2.17%) SALU: 16497 -> 16383 (-0.69%); split: -0.73%, +0.04% VMEM: 561 -> 477 (-14.97%) SMEM: 3197 -> 3113 (-2.63%) Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36468>
2025-07-30 15:04:06 +02:00
case nir_op_f2f16:
case nir_op_f2f16_rtz:
case nir_op_f2f16_rtne:
case nir_op_f2f32:
case nir_op_f2f64:
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
case nir_op_fdot2:
case nir_op_fdot3:
case nir_op_fdot4:
case nir_op_fdot8:
case nir_op_fdot16:
case nir_op_fdot2_replicated:
case nir_op_fdot3_replicated:
case nir_op_fdot4_replicated:
case nir_op_fdot8_replicated:
case nir_op_fdot16_replicated:
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
push_fp_query(state, alu->src[0].src.ssa);
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
return;
case nir_op_fadd:
nir: add fp class analysis for fsub Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-16 18:13:10 +01:00
case nir_op_fsub:
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
case nir_op_fmax:
case nir_op_fmin:
case nir_op_fmul:
case nir_op_fmulz:
case nir_op_fpow:
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
case nir_op_vec2:
push_fp_query(state, alu->src[0].src.ssa);
push_fp_query(state, alu->src[1].src.ssa);
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
return;
case nir_op_ffma:
nir: add range analysis for ffmaz Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34125>
2025-03-24 08:20:46 +01:00
case nir_op_ffmaz:
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
case nir_op_flrp:
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
push_fp_query(state, alu->src[0].src.ssa);
push_fp_query(state, alu->src[1].src.ssa);
push_fp_query(state, alu->src[2].src.ssa);
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
return;
default:
break;
}
}
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
fp_class_mask r = FP_CLASS_UNKNOWN;
nir/range-analysis: Add a lot more assertions about the contents of tables v2: Update several of the comments. Drop some redundant uses of ASSERT_UNION_OF_OTHERS_MATCHES_UNKNOWN_*_SOURCE source. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Suggested-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-08-12 15:40:20 -07:00
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
switch (alu->op) {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_b2i16:
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
case nir_op_b2i32:
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_b2i64:
/* b2i32 will generate either 0x00000000 or 0x00000001. When those bit
nir/range_analysis: Add "is finite" range analysis tracking The obvious changes to nir_search_helpers.h are in a separate commit to limit the scope of this change. These additions are really only needed to support the next commit "nir/range_analysis: Add "is a number" range analysis tracking". This reduction in scope is intended to increase the suitability for stable branches. No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_finite. v3: Split nir_search_helpers.h changes into a separate commit. v4: Remove assertion intended for the next commit. Update is_finite comment for fsign. Both noticed by Rhys. Fix is_finite handling for load_const vectors. If any element is not finite, set the flag to false. This is the same way is_integral is already handled. v5: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-08-17 15:56:24 -07:00
* patterns are interpreted as floating point, they are 0.0 and
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
* 1.401298464324817e-45. The latter is subnormal.
nir/range_analysis: Add "is finite" range analysis tracking The obvious changes to nir_search_helpers.h are in a separate commit to limit the scope of this change. These additions are really only needed to support the next commit "nir/range_analysis: Add "is a number" range analysis tracking". This reduction in scope is intended to increase the suitability for stable branches. No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_finite. v3: Split nir_search_helpers.h changes into a separate commit. v4: Remove assertion intended for the next commit. Update is_finite comment for fsign. Both noticed by Rhys. Fix is_finite handling for load_const vectors. If any element is not finite, set the flag to false. This is the same way is_integral is already handled. v5: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-08-17 15:56:24 -07:00
*/
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = FP_CLASS_POS_ZERO | FP_CLASS_GT_ZERO_LT_POS_ONE;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_b2f16:
case nir_op_b2f32:
case nir_op_b2f64:
r = FP_CLASS_POS_ZERO | FP_CLASS_POS_ONE;
break;
nir/range-analysis: Range tracking for bcsel This could be squashed with the previous commit. I kept it separate to ease review. v2: Add some missing cases. Use nir_src_is_const helper. Both suggested by Caio. Use a table for mapping source ranges to a result range. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-24 20:23:15 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_vec2:
case nir_op_bcsel:
r = src_res[0] | src_res[1];
nir/range-analysis: Range tracking for bcsel This could be squashed with the previous commit. I kept it separate to ease review. v2: Add some missing cases. Use nir_src_is_const helper. Both suggested by Caio. Use a table for mapping source ranges to a result range. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-24 20:23:15 +08:00
break;
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_i2f16:
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
case nir_op_i2f32:
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_i2f64:
r &= ~FP_CLASS_NAN;
r &= ~FP_CLASS_NON_INTEGRAL;
r &= ~FP_CLASS_GT_ZERO_LT_POS_ONE;
r &= ~FP_CLASS_LT_ZERO_GT_NEG_ONE;
r &= ~FP_CLASS_NEG_ZERO;
if (alu->def.bit_size > 16 || alu->src[0].src.ssa->bit_size <= 16)
r &= ~FP_CLASS_ANY_INF;
break;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_u2f16:
case nir_op_u2f32:
case nir_op_u2f64:
r &= ~FP_CLASS_NAN;
r &= ~FP_CLASS_NON_INTEGRAL;
r &= ~FP_CLASS_GT_ZERO_LT_POS_ONE;
r &= ~FP_CLASS_NEG_ZERO;
r &= ~FP_CLASS_ANY_NEG;
if (alu->def.bit_size > 16 || alu->src[0].src.ssa->bit_size < 16)
r &= ~FP_CLASS_ANY_INF;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
nir/range_analysis: look through f2f Foz-DB Navi31: Totals from 93 (0.12% of 80273) affected shaders: Instrs: 123927 -> 121073 (-2.30%); split: -2.30%, +0.00% CodeSize: 670832 -> 653332 (-2.61%); split: -2.61%, +0.00% Latency: 337678 -> 322803 (-4.41%); split: -4.41%, +0.00% InvThroughput: 63277 -> 61083 (-3.47%) VClause: 460 -> 373 (-18.91%) SClause: 2178 -> 2100 (-3.58%) Copies: 7637 -> 7744 (+1.40%) PreSGPRs: 4414 -> 4287 (-2.88%) PreVGPRs: 4229 -> 4230 (+0.02%) VALU: 77375 -> 75693 (-2.17%) SALU: 16497 -> 16383 (-0.69%); split: -0.73%, +0.04% VMEM: 561 -> 477 (-14.97%) SMEM: 3197 -> 3113 (-2.63%) Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36468>
2025-07-30 15:04:06 +02:00
case nir_op_f2f16:
case nir_op_f2f16_rtz:
case nir_op_f2f16_rtne:
case nir_op_f2f32:
case nir_op_f2f64: {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = handle_sz(alu, src_res[0]);
nir/range_analysis: look through f2f Foz-DB Navi31: Totals from 93 (0.12% of 80273) affected shaders: Instrs: 123927 -> 121073 (-2.30%); split: -2.30%, +0.00% CodeSize: 670832 -> 653332 (-2.61%); split: -2.61%, +0.00% Latency: 337678 -> 322803 (-4.41%); split: -4.41%, +0.00% InvThroughput: 63277 -> 61083 (-3.47%) VClause: 460 -> 373 (-18.91%) SClause: 2178 -> 2100 (-3.58%) Copies: 7637 -> 7744 (+1.40%) PreSGPRs: 4414 -> 4287 (-2.88%) PreVGPRs: 4229 -> 4230 (+0.02%) VALU: 77375 -> 75693 (-2.17%) SALU: 16497 -> 16383 (-0.69%); split: -0.73%, +0.04% VMEM: 561 -> 477 (-14.97%) SMEM: 3197 -> 3113 (-2.63%) Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36468>
2025-07-30 15:04:06 +02:00
if (alu->src[0].src.ssa->bit_size > alu->def.bit_size) {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
bool rtz = alu->op == nir_op_f2f16_rtz;
if (alu->op != nir_op_f2f16_rtne && alu->op != nir_op_f2f16_rtz) {
nir_shader *shader = nir_cf_node_get_function(&alu->instr.block->cf_node)->function->shader;
unsigned execution_mode = shader->info.float_controls_execution_mode;
rtz = nir_is_rounding_mode_rtz(execution_mode, alu->def.bit_size);
}
nir/range_analysis: look through f2f Foz-DB Navi31: Totals from 93 (0.12% of 80273) affected shaders: Instrs: 123927 -> 121073 (-2.30%); split: -2.30%, +0.00% CodeSize: 670832 -> 653332 (-2.61%); split: -2.61%, +0.00% Latency: 337678 -> 322803 (-4.41%); split: -4.41%, +0.00% InvThroughput: 63277 -> 61083 (-3.47%) VClause: 460 -> 373 (-18.91%) SClause: 2178 -> 2100 (-3.58%) Copies: 7637 -> 7744 (+1.40%) PreSGPRs: 4414 -> 4287 (-2.88%) PreVGPRs: 4229 -> 4230 (+0.02%) VALU: 77375 -> 75693 (-2.17%) SALU: 16497 -> 16383 (-0.69%); split: -0.73%, +0.04% VMEM: 561 -> 477 (-14.97%) SMEM: 3197 -> 3113 (-2.63%) Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36468>
2025-07-30 15:04:06 +02:00
/* Unless we are rounding towards zero, large values can create Inf. */
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (r & FP_CLASS_LT_NEG_ONE) {
if (!rtz)
r |= FP_CLASS_NEG_INF;
r |= FP_CLASS_NEG_ONE;
}
if (r & FP_CLASS_GT_POS_ONE) {
if (!rtz)
r |= FP_CLASS_POS_INF;
r |= FP_CLASS_POS_ONE;
}
nir/range_analysis: look through f2f Foz-DB Navi31: Totals from 93 (0.12% of 80273) affected shaders: Instrs: 123927 -> 121073 (-2.30%); split: -2.30%, +0.00% CodeSize: 670832 -> 653332 (-2.61%); split: -2.61%, +0.00% Latency: 337678 -> 322803 (-4.41%); split: -4.41%, +0.00% InvThroughput: 63277 -> 61083 (-3.47%) VClause: 460 -> 373 (-18.91%) SClause: 2178 -> 2100 (-3.58%) Copies: 7637 -> 7744 (+1.40%) PreSGPRs: 4414 -> 4287 (-2.88%) PreVGPRs: 4229 -> 4230 (+0.02%) VALU: 77375 -> 75693 (-2.17%) SALU: 16497 -> 16383 (-0.69%); split: -0.73%, +0.04% VMEM: 561 -> 477 (-14.97%) SMEM: 3197 -> 3113 (-2.63%) Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36468>
2025-07-30 15:04:06 +02:00
/* Underflow can create new zeros. */
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (r & FP_CLASS_LT_ZERO_GT_NEG_ONE) {
if (!rtz)
r |= FP_CLASS_NEG_ONE;
r |= FP_CLASS_NEG_ZERO;
}
if (r & FP_CLASS_GT_ZERO_LT_POS_ONE) {
if (!rtz)
r |= FP_CLASS_POS_ONE;
r |= FP_CLASS_POS_ZERO;
}
nir/range_analysis: look through f2f Foz-DB Navi31: Totals from 93 (0.12% of 80273) affected shaders: Instrs: 123927 -> 121073 (-2.30%); split: -2.30%, +0.00% CodeSize: 670832 -> 653332 (-2.61%); split: -2.61%, +0.00% Latency: 337678 -> 322803 (-4.41%); split: -4.41%, +0.00% InvThroughput: 63277 -> 61083 (-3.47%) VClause: 460 -> 373 (-18.91%) SClause: 2178 -> 2100 (-3.58%) Copies: 7637 -> 7744 (+1.40%) PreSGPRs: 4414 -> 4287 (-2.88%) PreVGPRs: 4229 -> 4230 (+0.02%) VALU: 77375 -> 75693 (-2.17%) SALU: 16497 -> 16383 (-0.69%); split: -0.73%, +0.04% VMEM: 561 -> 477 (-14.97%) SMEM: 3197 -> 3113 (-2.63%) Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36468>
2025-07-30 15:04:06 +02:00
}
break;
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_fneg:
r = fneg_fp_class(src_res[0]);
break;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_fabs:
r = src_res[0];
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r |= fneg_fp_class(r & (FP_CLASS_ANY_NEG | FP_CLASS_NEG_ZERO));
r &= ~(FP_CLASS_ANY_NEG | FP_CLASS_NEG_ZERO);
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
case nir_op_fadd: {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = fadd_fp_class(src_res[0], src_res[1]);
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
}
nir: add fp class analysis for fsub Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-16 18:13:10 +01:00
case nir_op_fsub: {
r = fadd_fp_class(src_res[0], fneg_fp_class(src_res[1]));
break;
}
nir/range-analysis: Adjust result range of exp2 to account for flush-to-zero Fixes piglit tests (new in piglit!110): - fs-underflow-exp2-compare-zero.shader_test Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=111308 Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Most of the shaders affected are, unsurprisingly, in Unigine Heaven. All Gen6+ platforms had similar results. (Ice Lake shown) total instructions in shared programs: 16278207 -> 16278465 (<.01%) instructions in affected programs: 11374 -> 11632 (2.27%) helped: 0 HURT: 58 HURT stats (abs) min: 2 max: 13 x̄: 4.45 x̃: 4 HURT stats (rel) min: 0.54% max: 4.11% x̄: 2.42% x̃: 2.82% 95% mean confidence interval for instructions value: 3.77 5.13 95% mean confidence interval for instructions %-change: 2.19% 2.64% Instructions are HURT. total cycles in shared programs: 367134284 -> 367135159 (<.01%) cycles in affected programs: 81207 -> 82082 (1.08%) helped: 17 HURT: 36 helped stats (abs) min: 6 max: 356 x̄: 90.35 x̃: 6 helped stats (rel) min: 0.69% max: 21.45% x̄: 5.71% x̃: 0.78% HURT stats (abs) min: 4 max: 235 x̄: 66.97 x̃: 16 HURT stats (rel) min: 0.35% max: 27.58% x̄: 5.34% x̃: 1.09% 95% mean confidence interval for cycles value: -20.36 53.38 95% mean confidence interval for cycles %-change: -1.08% 4.67% Inconclusive result (value mean confidence interval includes 0). No changes on any earlier platforms.
2019-08-07 08:56:22 -07:00
case nir_op_fexp2: {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
fp_class_mask src = src_res[0];
r = 0;
nir/range-analysis: Adjust result range of exp2 to account for flush-to-zero Fixes piglit tests (new in piglit!110): - fs-underflow-exp2-compare-zero.shader_test Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=111308 Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Most of the shaders affected are, unsurprisingly, in Unigine Heaven. All Gen6+ platforms had similar results. (Ice Lake shown) total instructions in shared programs: 16278207 -> 16278465 (<.01%) instructions in affected programs: 11374 -> 11632 (2.27%) helped: 0 HURT: 58 HURT stats (abs) min: 2 max: 13 x̄: 4.45 x̃: 4 HURT stats (rel) min: 0.54% max: 4.11% x̄: 2.42% x̃: 2.82% 95% mean confidence interval for instructions value: 3.77 5.13 95% mean confidence interval for instructions %-change: 2.19% 2.64% Instructions are HURT. total cycles in shared programs: 367134284 -> 367135159 (<.01%) cycles in affected programs: 81207 -> 82082 (1.08%) helped: 17 HURT: 36 helped stats (abs) min: 6 max: 356 x̄: 90.35 x̃: 6 helped stats (rel) min: 0.69% max: 21.45% x̄: 5.71% x̃: 0.78% HURT stats (abs) min: 4 max: 235 x̄: 66.97 x̃: 16 HURT stats (rel) min: 0.35% max: 27.58% x̄: 5.34% x̃: 1.09% 95% mean confidence interval for cycles value: -20.36 53.38 95% mean confidence interval for cycles %-change: -1.08% 4.67% Inconclusive result (value mean confidence interval includes 0). No changes on any earlier platforms.
2019-08-07 08:56:22 -07:00
/* If the parameter might be less than zero, the mathematically result
* will be on (0, 1). For sufficiently large magnitude negative
* parameters, the result will flush to zero.
*/
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src & FP_CLASS_NEG_INF)
r |= FP_CLASS_POS_ZERO;
nir/range-analysis: Adjust result range of exp2 to account for flush-to-zero Fixes piglit tests (new in piglit!110): - fs-underflow-exp2-compare-zero.shader_test Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=111308 Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Most of the shaders affected are, unsurprisingly, in Unigine Heaven. All Gen6+ platforms had similar results. (Ice Lake shown) total instructions in shared programs: 16278207 -> 16278465 (<.01%) instructions in affected programs: 11374 -> 11632 (2.27%) helped: 0 HURT: 58 HURT stats (abs) min: 2 max: 13 x̄: 4.45 x̃: 4 HURT stats (rel) min: 0.54% max: 4.11% x̄: 2.42% x̃: 2.82% 95% mean confidence interval for instructions value: 3.77 5.13 95% mean confidence interval for instructions %-change: 2.19% 2.64% Instructions are HURT. total cycles in shared programs: 367134284 -> 367135159 (<.01%) cycles in affected programs: 81207 -> 82082 (1.08%) helped: 17 HURT: 36 helped stats (abs) min: 6 max: 356 x̄: 90.35 x̃: 6 helped stats (rel) min: 0.69% max: 21.45% x̄: 5.71% x̃: 0.78% HURT stats (abs) min: 4 max: 235 x̄: 66.97 x̃: 16 HURT stats (rel) min: 0.35% max: 27.58% x̄: 5.34% x̃: 1.09% 95% mean confidence interval for cycles value: -20.36 53.38 95% mean confidence interval for cycles %-change: -1.08% 4.67% Inconclusive result (value mean confidence interval includes 0). No changes on any earlier platforms.
2019-08-07 08:56:22 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src & FP_CLASS_LT_NEG_ONE)
r |= FP_CLASS_POS_ZERO | FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_NON_INTEGRAL;
nir/range-analysis: Adjust result range of exp2 to account for flush-to-zero Fixes piglit tests (new in piglit!110): - fs-underflow-exp2-compare-zero.shader_test Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=111308 Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Most of the shaders affected are, unsurprisingly, in Unigine Heaven. All Gen6+ platforms had similar results. (Ice Lake shown) total instructions in shared programs: 16278207 -> 16278465 (<.01%) instructions in affected programs: 11374 -> 11632 (2.27%) helped: 0 HURT: 58 HURT stats (abs) min: 2 max: 13 x̄: 4.45 x̃: 4 HURT stats (rel) min: 0.54% max: 4.11% x̄: 2.42% x̃: 2.82% 95% mean confidence interval for instructions value: 3.77 5.13 95% mean confidence interval for instructions %-change: 2.19% 2.64% Instructions are HURT. total cycles in shared programs: 367134284 -> 367135159 (<.01%) cycles in affected programs: 81207 -> 82082 (1.08%) helped: 17 HURT: 36 helped stats (abs) min: 6 max: 356 x̄: 90.35 x̃: 6 helped stats (rel) min: 0.69% max: 21.45% x̄: 5.71% x̃: 0.78% HURT stats (abs) min: 4 max: 235 x̄: 66.97 x̃: 16 HURT stats (rel) min: 0.35% max: 27.58% x̄: 5.34% x̃: 1.09% 95% mean confidence interval for cycles value: -20.36 53.38 95% mean confidence interval for cycles %-change: -1.08% 4.67% Inconclusive result (value mean confidence interval includes 0). No changes on any earlier platforms.
2019-08-07 08:56:22 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src & (FP_CLASS_NEG_ONE | FP_CLASS_LT_ZERO_GT_NEG_ONE))
r |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_NON_INTEGRAL;
nir/range-analysis: Add a lot more assertions about the contents of tables v2: Update several of the comments. Drop some redundant uses of ASSERT_UNION_OF_OTHERS_MATCHES_UNKNOWN_*_SOURCE source. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> Suggested-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-08-12 15:40:20 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src & (FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_ANY_ZERO | FP_CLASS_GT_ZERO_LT_POS_ONE))
r |= FP_CLASS_POS_ONE;
nir/range_analysis: Add "is finite" range analysis tracking The obvious changes to nir_search_helpers.h are in a separate commit to limit the scope of this change. These additions are really only needed to support the next commit "nir/range_analysis: Add "is a number" range analysis tracking". This reduction in scope is intended to increase the suitability for stable branches. No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_finite. v3: Split nir_search_helpers.h changes into a separate commit. v4: Remove assertion intended for the next commit. Update is_finite comment for fsign. Both noticed by Rhys. Fix is_finite handling for load_const vectors. If any element is not finite, set the flag to false. This is the same way is_integral is already handled. v5: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-08-17 15:56:24 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src & (FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ONE))
r |= FP_CLASS_GT_POS_ONE;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src & (FP_CLASS_GT_POS_ONE))
r |= FP_CLASS_GT_POS_ONE | FP_CLASS_POS_INF;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src & FP_CLASS_POS_INF)
r |= FP_CLASS_POS_INF;
nir/range_analysis: Add "is finite" range analysis tracking The obvious changes to nir_search_helpers.h are in a separate commit to limit the scope of this change. These additions are really only needed to support the next commit "nir/range_analysis: Add "is a number" range analysis tracking". This reduction in scope is intended to increase the suitability for stable branches. No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_finite. v3: Split nir_search_helpers.h changes into a separate commit. v4: Remove assertion intended for the next commit. Update is_finite comment for fsign. Both noticed by Rhys. Fix is_finite handling for load_const vectors. If any element is not finite, set the flag to false. This is the same way is_integral is already handled. v5: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-08-17 15:56:24 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src & FP_CLASS_NON_INTEGRAL)
r |= FP_CLASS_NON_INTEGRAL;
nir/range_analysis: Fix analysis of fmin, fmax, or fsat with NaN source Recall that when either value is NaN, fmax will pick the other value. This means the result range of the fmax will either be the "ideal" result range (calculated above) or the range of the non-NaN value. Previously, something like fmax({gt_zero}, {lt_zero, is_a_number}) would return a range of gt_zero. However, if the "gt_zero" parameter is NaN, the actual result will be the "lt_zero" parameter. This analysis depends on the is_a_number analysis also added in this MR. Assuming this doesn't cause any unforeseen problems, I believe we should wait a bit, then nominate a subset of the series for the stable branches. This fixes the piglit tests tests/spec/glsl-1.30/execution/range_analysis_fmax_of_nan.shader_test tests/spec/glsl-1.30/execution/range_analysis_fmin_of_nan.shader_test from https://gitlab.freedesktop.org/mesa/piglit/-/merge_requests/463. Even with the added fsat fixes, range_analysis_fsat_of_nan.shader_test still fails. There are some other issues there that will be addressed in later commits (in another MR). v2: Add fsat fixes. Suggested by Rhys. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Shader-db results: All Intel platforms had similar results. (Tiger Lake shown) total instructions in shared programs: 21049290 -> 21049314 (<.01%) instructions in affected programs: 3175 -> 3199 (0.76%) helped: 0 HURT: 17 HURT stats (abs) min: 1 max: 3 x̄: 1.41 x̃: 1 HURT stats (rel) min: 0.20% max: 1.89% x̄: 0.97% x̃: 0.92% 95% mean confidence interval for instructions value: 1.09 1.73 95% mean confidence interval for instructions %-change: 0.75% 1.19% Instructions are HURT. total cycles in shared programs: 855136176 -> 855136406 (<.01%) cycles in affected programs: 37579 -> 37809 (0.61%) helped: 0 HURT: 17 HURT stats (abs) min: 12 max: 20 x̄: 13.53 x̃: 14 HURT stats (rel) min: 0.17% max: 1.13% x̄: 0.79% x̃: 0.91% 95% mean confidence interval for cycles value: 12.53 14.53 95% mean confidence interval for cycles %-change: 0.63% 0.94% Cycles are HURT. Fossil-db results: Tiger Lake Instructions in all programs: 160901033 -> 160902591 (+0.0%) SENDs in all programs: 6812270 -> 6812270 (+0.0%) Loops in all programs: 38225 -> 38225 (+0.0%) Cycles in all programs: 7430016795 -> 7429003266 (-0.0%) Spills in all programs: 192582 -> 192582 (+0.0%) Fills in all programs: 304539 -> 304539 (+0.0%) Ice Lake Instructions in all programs: 145299102 -> 145301634 (+0.0%) SENDs in all programs: 6863890 -> 6863890 (+0.0%) Loops in all programs: 38219 -> 38219 (+0.0%) Cycles in all programs: 8798390846 -> 8798589772 (+0.0%) Spills in all programs: 216880 -> 216880 (+0.0%) Fills in all programs: 334250 -> 334250 (+0.0%) Skylake Instructions in all programs: 135889478 -> 135892010 (+0.0%) SENDs in all programs: 6802916 -> 6802916 (+0.0%) Loops in all programs: 38216 -> 38216 (+0.0%) Cycles in all programs: 8442624166 -> 8442597324 (-0.0%) Spills in all programs: 194839 -> 194839 (+0.0%) Fills in all programs: 301116 -> 301116 (+0.0%) Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2021-01-27 19:42:44 -08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src & FP_CLASS_NAN)
r |= FP_CLASS_NAN;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
}
nir: add fp class analysis for flog2 Foz-DB Navi48: Totals from 230 (0.28% of 82636) affected shaders: Instrs: 599005 -> 598615 (-0.07%); split: -0.09%, +0.02% CodeSize: 3110528 -> 3103136 (-0.24%); split: -0.24%, +0.00% Latency: 3661526 -> 3663241 (+0.05%); split: -0.01%, +0.05% InvThroughput: 526561 -> 526487 (-0.01%); split: -0.01%, +0.00% Copies: 33735 -> 33820 (+0.25%); split: -0.06%, +0.31% VALU: 378034 -> 377904 (-0.03%); split: -0.03%, +0.00% SALU: 65156 -> 65045 (-0.17%); split: -0.19%, +0.02% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-11 19:58:41 +01:00
case nir_op_flog2: {
r = 0;
if (src_res[0] & (FP_CLASS_ANY_NEG | FP_CLASS_NAN))
r |= FP_CLASS_NAN;
if (src_res[0] & FP_CLASS_ANY_ZERO)
r |= FP_CLASS_NEG_INF;
if (src_res[0] & FP_CLASS_GT_ZERO_LT_POS_ONE)
r |= FP_CLASS_ANY_NEG | FP_CLASS_NON_INTEGRAL;
if (src_res[0] & FP_CLASS_POS_ONE)
r |= FP_CLASS_POS_ZERO;
if (src_res[0] & FP_CLASS_GT_POS_ONE)
r |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_GT_POS_ONE | FP_CLASS_NON_INTEGRAL;
if (src_res[0] & FP_CLASS_POS_INF)
r |= FP_CLASS_POS_INF;
break;
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_fmax: {
fp_class_mask left = fneg_fp_class(src_res[0]);
fp_class_mask right = fneg_fp_class(src_res[1]);
r = fneg_fp_class(fmin_fp_class(left, right));
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_fmin:
r = fmin_fp_class(src_res[0], src_res[1]);
break;
nir/algebraic: optimize expressions using fmulz/ffmaz Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13436>
2021-09-14 18:02:01 +01:00
case nir_op_fmul:
case nir_op_fmulz: {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
bool mulz = alu->op == nir_op_fmulz;
bool src_eq = nir_alu_srcs_equal(alu, alu, 0, 1);
bool src_neg_eq = !nir_src_is_const(alu->src[0].src) && nir_alu_srcs_negative_equal(alu, alu, 0, 1);
r = fmul_fp_class(src_res[0], src_res[1], mulz, src_eq, src_neg_eq);
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_frcp:
r = frcp_fp_class(handle_sz(alu, src_res[0]));
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
nir/range-analysis: Use types to provide better ranges from bcsel and mov Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> All Gen7+ platforms had similar results. (Ice Lake shown) total instructions in shared programs: 16328255 -> 16315391 (-0.08%) instructions in affected programs: 218318 -> 205454 (-5.89%) helped: 988 HURT: 0 helped stats (abs) min: 1 max: 72 x̄: 13.02 x̃: 10 helped stats (rel) min: 0.33% max: 16.04% x̄: 6.27% x̃: 4.88% 95% mean confidence interval for instructions value: -13.69 -12.35 95% mean confidence interval for instructions %-change: -6.55% -5.99% Instructions are helped. total cycles in shared programs: 363683977 -> 363615417 (-0.02%) cycles in affected programs: 1475193 -> 1406633 (-4.65%) helped: 923 HURT: 36 helped stats (abs) min: 1 max: 624 x̄: 75.78 x̃: 48 helped stats (rel) min: 0.08% max: 13.89% x̄: 5.20% x̃: 5.08% HURT stats (abs) min: 1 max: 179 x̄: 38.58 x̃: 4 HURT stats (rel) min: 0.06% max: 16.56% x̄: 3.33% x̃: 0.29% 95% mean confidence interval for cycles value: -75.88 -67.10 95% mean confidence interval for cycles %-change: -5.10% -4.66% Cycles are helped. Sandy Bridge total instructions in shared programs: 10785779 -> 10785654 (<.01%) instructions in affected programs: 13855 -> 13730 (-0.90%) helped: 67 HURT: 0 helped stats (abs) min: 1 max: 15 x̄: 1.87 x̃: 1 helped stats (rel) min: 0.20% max: 3.45% x̄: 0.97% x̃: 0.78% 95% mean confidence interval for instructions value: -2.47 -1.26 95% mean confidence interval for instructions %-change: -1.13% -0.81% Instructions are helped. total cycles in shared programs: 153704799 -> 153704481 (<.01%) cycles in affected programs: 101509 -> 101191 (-0.31%) helped: 38 HURT: 13 helped stats (abs) min: 1 max: 38 x̄: 12.53 x̃: 16 helped stats (rel) min: 0.07% max: 2.69% x̄: 0.87% x̃: 0.53% HURT stats (abs) min: 1 max: 36 x̄: 12.15 x̃: 7 HURT stats (rel) min: 0.06% max: 2.53% x̄: 0.73% x̃: 0.44% 95% mean confidence interval for cycles value: -10.24 -2.24 95% mean confidence interval for cycles %-change: -0.75% -0.17% Cycles are helped. LOST: 2 GAINED: 0 No shader-db change on Iron Lake or GM45.
2019-08-12 17:28:35 -07:00
case nir_op_mov:
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = src_res[0];
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
nir/range_analysis: Refactor fsat handling This will greatly simplify a later commit. The assert(r.is_integral) in the eq_zero case is dropped because I don't think it's useful anymore. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2021-03-08 14:30:00 -08:00
case nir_op_fsat: {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = src_res[0];
nir/range-analysis: Tighten the range of fsat based on the range of its source This could be squashed with the previous commit. I kept it separate to ease review. v2: Use a switch statement and add more comments. Both suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-06-01 18:54:49 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
/* max(+0.0, x) */
if (r & (FP_CLASS_ANY_NEG | FP_CLASS_NEG_ZERO | FP_CLASS_NAN)) {
r &= ~(FP_CLASS_ANY_NEG | FP_CLASS_NEG_ZERO | FP_CLASS_NAN);
r |= FP_CLASS_POS_ZERO;
}
nir/range-analysis: Tighten the range of fsat based on the range of its source This could be squashed with the previous commit. I kept it separate to ease review. v2: Use a switch statement and add more comments. Both suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-06-01 18:54:49 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
/* min(+1.0, x) */
if (r & (FP_CLASS_GT_POS_ONE | FP_CLASS_POS_INF)) {
r &= ~(FP_CLASS_GT_POS_ONE | FP_CLASS_POS_INF);
r |= FP_CLASS_POS_ONE;
nir/range-analysis: Tighten the range of fsat based on the range of its source This could be squashed with the previous commit. I kept it separate to ease review. v2: Use a switch statement and add more comments. Both suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-06-01 18:54:49 -07:00
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (!(r & FP_CLASS_GT_ZERO_LT_POS_ONE))
r &= ~FP_CLASS_NON_INTEGRAL;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
nir/range_analysis: Refactor fsat handling This will greatly simplify a later commit. The assert(r.is_integral) in the eq_zero case is dropped because I don't think it's useful anymore. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2021-03-08 14:30:00 -08:00
}
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
case nir_op_fsign:
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = 0;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src_res[0] & FP_CLASS_ANY_NEG)
r |= FP_CLASS_NEG_ONE;
nir: improve fsqrt range analysis Foz-DB Navi21: Totals from 3 (0.00% of 79377) affected shaders: MaxWaves: 88 -> 96 (+9.09%) Instrs: 1058 -> 951 (-10.11%) CodeSize: 5964 -> 5368 (-9.99%) VGPRs: 104 -> 96 (-7.69%) Latency: 15283 -> 14099 (-7.75%); split: -8.37%, +0.62% InvThroughput: 4951 -> 4238 (-14.40%) Copies: 81 -> 76 (-6.17%) PreVGPRs: 93 -> 84 (-9.68%) VALU: 820 -> 737 (-10.12%) SALU: 115 -> 91 (-20.87%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 20:13:58 +01:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src_res[0] & FP_CLASS_ANY_ZERO)
r |= FP_CLASS_ANY_ZERO;
nir: improve fsqrt range analysis Foz-DB Navi21: Totals from 3 (0.00% of 79377) affected shaders: MaxWaves: 88 -> 96 (+9.09%) Instrs: 1058 -> 951 (-10.11%) CodeSize: 5964 -> 5368 (-9.99%) VGPRs: 104 -> 96 (-7.69%) Latency: 15283 -> 14099 (-7.75%); split: -8.37%, +0.62% InvThroughput: 4951 -> 4238 (-14.40%) Copies: 81 -> 76 (-6.17%) PreVGPRs: 93 -> 84 (-9.68%) VALU: 820 -> 737 (-10.12%) SALU: 115 -> 91 (-20.87%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 20:13:58 +01:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src_res[0] & FP_CLASS_ANY_POS)
r |= FP_CLASS_POS_ONE;
nir: improve fsqrt range analysis Foz-DB Navi21: Totals from 3 (0.00% of 79377) affected shaders: MaxWaves: 88 -> 96 (+9.09%) Instrs: 1058 -> 951 (-10.11%) CodeSize: 5964 -> 5368 (-9.99%) VGPRs: 104 -> 96 (-7.69%) Latency: 15283 -> 14099 (-7.75%); split: -8.37%, +0.62% InvThroughput: 4951 -> 4238 (-14.40%) Copies: 81 -> 76 (-6.17%) PreVGPRs: 93 -> 84 (-9.68%) VALU: 820 -> 737 (-10.12%) SALU: 115 -> 91 (-20.87%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 20:13:58 +01:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
/* fsign is -1, 0, or 1, even for NaN */
if (src_res[0] & FP_CLASS_NAN)
r |= FP_CLASS_NEG_ONE | FP_CLASS_ANY_ZERO | FP_CLASS_POS_ONE;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
nir: fix frsq range analysis Foz-DB Navi21: Totals from 98 (0.12% of 79377) affected shaders: Instrs: 157311 -> 157675 (+0.23%); split: -0.03%, +0.26% CodeSize: 844296 -> 846648 (+0.28%); split: -0.00%, +0.28% Latency: 1275467 -> 1276259 (+0.06%); split: -0.00%, +0.06% InvThroughput: 266980 -> 267098 (+0.04%); split: -0.03%, +0.07% Copies: 11094 -> 11093 (-0.01%) PreVGPRs: 5945 -> 5977 (+0.54%) VALU: 110585 -> 110953 (+0.33%); split: -0.04%, +0.38% SALU: 18481 -> 18476 (-0.03%) Cc: mesa-stable Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 20:05:54 +01:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_fsqrt:
r = fsqrt_fp_class(src_res[0]);
break;
nir: fix frsq range analysis Foz-DB Navi21: Totals from 98 (0.12% of 79377) affected shaders: Instrs: 157311 -> 157675 (+0.23%); split: -0.03%, +0.26% CodeSize: 844296 -> 846648 (+0.28%); split: -0.00%, +0.28% Latency: 1275467 -> 1276259 (+0.06%); split: -0.00%, +0.06% InvThroughput: 266980 -> 267098 (+0.04%); split: -0.03%, +0.07% Copies: 11094 -> 11093 (-0.01%) PreVGPRs: 5945 -> 5977 (+0.54%) VALU: 110585 -> 110953 (+0.33%); split: -0.04%, +0.38% SALU: 18481 -> 18476 (-0.03%) Cc: mesa-stable Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 20:05:54 +01:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
case nir_op_frsq:
r = frcp_fp_class(fsqrt_fp_class(handle_sz(alu, src_res[0])));
nir: fix frsq range analysis Foz-DB Navi21: Totals from 98 (0.12% of 79377) affected shaders: Instrs: 157311 -> 157675 (+0.23%); split: -0.03%, +0.26% CodeSize: 844296 -> 846648 (+0.28%); split: -0.00%, +0.28% Latency: 1275467 -> 1276259 (+0.06%); split: -0.00%, +0.06% InvThroughput: 266980 -> 267098 (+0.04%); split: -0.03%, +0.07% Copies: 11094 -> 11093 (-0.01%) PreVGPRs: 5945 -> 5977 (+0.54%) VALU: 110585 -> 110953 (+0.33%); split: -0.04%, +0.38% SALU: 18481 -> 18476 (-0.03%) Cc: mesa-stable Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 20:05:54 +01:00
break;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
case nir_op_ffloor: {
nir/range_analysis: Add "is finite" range analysis tracking The obvious changes to nir_search_helpers.h are in a separate commit to limit the scope of this change. These additions are really only needed to support the next commit "nir/range_analysis: Add "is a number" range analysis tracking". This reduction in scope is intended to increase the suitability for stable branches. No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_finite. v3: Split nir_search_helpers.h changes into a separate commit. v4: Remove assertion intended for the next commit. Update is_finite comment for fsign. Both noticed by Rhys. Fix is_finite handling for load_const vectors. If any element is not finite, set the flag to false. This is the same way is_integral is already handled. v5: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-08-17 15:56:24 -07:00
/* In IEEE 754, floor(NaN) is NaN, and floor(±Inf) is ±Inf. See
* https://pubs.opengroup.org/onlinepubs/9699919799.2016edition/functions/floor.html
*/
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = src_res[0];
nir/range_analysis: Add "is finite" range analysis tracking The obvious changes to nir_search_helpers.h are in a separate commit to limit the scope of this change. These additions are really only needed to support the next commit "nir/range_analysis: Add "is a number" range analysis tracking". This reduction in scope is intended to increase the suitability for stable branches. No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_finite. v3: Split nir_search_helpers.h changes into a separate commit. v4: Remove assertion intended for the next commit. Update is_finite comment for fsign. Both noticed by Rhys. Fix is_finite handling for load_const vectors. If any element is not finite, set the flag to false. This is the same way is_integral is already handled. v5: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-08-17 15:56:24 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (r & FP_CLASS_NON_INTEGRAL) {
if (r & FP_CLASS_LT_ZERO_GT_NEG_ONE)
r |= FP_CLASS_NEG_ONE;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (r & FP_CLASS_GT_ZERO_LT_POS_ONE)
r |= FP_CLASS_POS_ZERO;
if (r & FP_CLASS_GT_POS_ONE)
r |= FP_CLASS_POS_ONE;
r &= ~(FP_CLASS_NON_INTEGRAL | FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_GT_ZERO_LT_POS_ONE);
}
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
}
case nir_op_fceil: {
nir/range_analysis: Add "is finite" range analysis tracking The obvious changes to nir_search_helpers.h are in a separate commit to limit the scope of this change. These additions are really only needed to support the next commit "nir/range_analysis: Add "is a number" range analysis tracking". This reduction in scope is intended to increase the suitability for stable branches. No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_finite. v3: Split nir_search_helpers.h changes into a separate commit. v4: Remove assertion intended for the next commit. Update is_finite comment for fsign. Both noticed by Rhys. Fix is_finite handling for load_const vectors. If any element is not finite, set the flag to false. This is the same way is_integral is already handled. v5: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-08-17 15:56:24 -07:00
/* In IEEE 754, ceil(NaN) is NaN, and ceil(±Inf) is ±Inf. See
* https://pubs.opengroup.org/onlinepubs/9699919799.2016edition/functions/ceil.html
*/
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = src_res[0];
if (r & FP_CLASS_NON_INTEGRAL) {
if (r & FP_CLASS_LT_NEG_ONE)
r |= FP_CLASS_NEG_ONE;
nir/range_analysis: Add "is finite" range analysis tracking The obvious changes to nir_search_helpers.h are in a separate commit to limit the scope of this change. These additions are really only needed to support the next commit "nir/range_analysis: Add "is a number" range analysis tracking". This reduction in scope is intended to increase the suitability for stable branches. No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_finite. v3: Split nir_search_helpers.h changes into a separate commit. v4: Remove assertion intended for the next commit. Update is_finite comment for fsign. Both noticed by Rhys. Fix is_finite handling for load_const vectors. If any element is not finite, set the flag to false. This is the same way is_integral is already handled. v5: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-08-17 15:56:24 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (r & FP_CLASS_LT_ZERO_GT_NEG_ONE)
r |= FP_CLASS_NEG_ZERO;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (r & FP_CLASS_GT_ZERO_LT_POS_ONE)
r |= FP_CLASS_POS_ONE;
r &= ~(FP_CLASS_NON_INTEGRAL | FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_GT_ZERO_LT_POS_ONE);
}
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
}
case nir_op_ftrunc: {
nir/range_analysis: Add "is finite" range analysis tracking The obvious changes to nir_search_helpers.h are in a separate commit to limit the scope of this change. These additions are really only needed to support the next commit "nir/range_analysis: Add "is a number" range analysis tracking". This reduction in scope is intended to increase the suitability for stable branches. No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_finite. v3: Split nir_search_helpers.h changes into a separate commit. v4: Remove assertion intended for the next commit. Update is_finite comment for fsign. Both noticed by Rhys. Fix is_finite handling for load_const vectors. If any element is not finite, set the flag to false. This is the same way is_integral is already handled. v5: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-08-17 15:56:24 -07:00
/* In IEEE 754, trunc(NaN) is NaN, and trunc(±Inf) is ±Inf. See
* https://pubs.opengroup.org/onlinepubs/9699919799.2016edition/functions/trunc.html
*/
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = src_res[0];
if (r & FP_CLASS_NON_INTEGRAL) {
if (r & FP_CLASS_LT_NEG_ONE)
r |= FP_CLASS_NEG_ONE;
if (r & FP_CLASS_LT_ZERO_GT_NEG_ONE)
r |= FP_CLASS_NEG_ZERO;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (r & FP_CLASS_GT_ZERO_LT_POS_ONE)
r |= FP_CLASS_POS_ZERO;
if (r & FP_CLASS_GT_POS_ONE)
r |= FP_CLASS_POS_ONE;
r &= ~(FP_CLASS_NON_INTEGRAL | FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_GT_ZERO_LT_POS_ONE);
}
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
}
nir: add fp class analysis for fround_even Foz-DB Navi48: Totals from 383 (0.33% of 114655) affected shaders: MaxWaves: 9806 -> 9808 (+0.02%) Instrs: 502508 -> 501762 (-0.15%); split: -0.16%, +0.01% CodeSize: 2711404 -> 2707604 (-0.14%); split: -0.15%, +0.01% VGPRs: 24360 -> 24348 (-0.05%) Latency: 2068105 -> 2066817 (-0.06%); split: -0.07%, +0.01% InvThroughput: 370962 -> 370081 (-0.24%) VClause: 7045 -> 7041 (-0.06%) SClause: 10551 -> 10559 (+0.08%); split: -0.08%, +0.15% Copies: 29135 -> 29117 (-0.06%); split: -0.12%, +0.05% Branches: 17333 -> 17328 (-0.03%) PreSGPRs: 21511 -> 21510 (-0.00%) PreVGPRs: 18555 -> 18545 (-0.05%) VALU: 274445 -> 273874 (-0.21%); split: -0.21%, +0.00% SALU: 78819 -> 78779 (-0.05%); split: -0.07%, +0.02% VMEM: 10918 -> 10913 (-0.05%) SMEM: 17662 -> 17656 (-0.03%) Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-03-02 17:58:23 +01:00
case nir_op_fround_even: {
r = src_res[0];
if (r & FP_CLASS_NON_INTEGRAL) {
if (r & FP_CLASS_LT_NEG_ONE)
r |= FP_CLASS_NEG_ONE;
if (r & FP_CLASS_LT_ZERO_GT_NEG_ONE)
r |= FP_CLASS_NEG_ZERO | FP_CLASS_NEG_ONE;
if (r & FP_CLASS_GT_ZERO_LT_POS_ONE)
r |= FP_CLASS_POS_ZERO | FP_CLASS_POS_ONE;
if (r & FP_CLASS_GT_POS_ONE)
r |= FP_CLASS_POS_ONE;
r &= ~(FP_CLASS_NON_INTEGRAL | FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_GT_ZERO_LT_POS_ONE);
}
break;
}
nir: range analysis for ffract Foz-DB Navi21: Totals from 75 (0.09% of 79377) affected shaders: Instrs: 69239 -> 68383 (-1.24%) CodeSize: 385088 -> 379532 (-1.44%) Latency: 427188 -> 421729 (-1.28%); split: -1.28%, +0.00% InvThroughput: 103086 -> 101926 (-1.13%) VClause: 785 -> 753 (-4.08%) SClause: 1624 -> 1598 (-1.60%) Copies: 5679 -> 5671 (-0.14%); split: -0.72%, +0.58% PreSGPRs: 3961 -> 3937 (-0.61%) VALU: 51107 -> 50457 (-1.27%) SALU: 9034 -> 8950 (-0.93%) VMEM: 1123 -> 1091 (-2.85%) SMEM: 2862 -> 2830 (-1.12%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 18:54:09 +01:00
case nir_op_ffract: {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = 0;
nir: range analysis for ffract Foz-DB Navi21: Totals from 75 (0.09% of 79377) affected shaders: Instrs: 69239 -> 68383 (-1.24%) CodeSize: 385088 -> 379532 (-1.44%) Latency: 427188 -> 421729 (-1.28%); split: -1.28%, +0.00% InvThroughput: 103086 -> 101926 (-1.13%) VClause: 785 -> 753 (-4.08%) SClause: 1624 -> 1598 (-1.60%) Copies: 5679 -> 5671 (-0.14%); split: -0.72%, +0.58% PreSGPRs: 3961 -> 3937 (-0.61%) VALU: 51107 -> 50457 (-1.27%) SALU: 9034 -> 8950 (-0.93%) VMEM: 1123 -> 1091 (-2.85%) SMEM: 2862 -> 2830 (-1.12%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 18:54:09 +01:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
/* fract(±Inf) is NaN. */
if (src_res[0] & (FP_CLASS_ANY_INF | FP_CLASS_NAN))
r |= FP_CLASS_NAN;
nir: range analysis for ffract Foz-DB Navi21: Totals from 75 (0.09% of 79377) affected shaders: Instrs: 69239 -> 68383 (-1.24%) CodeSize: 385088 -> 379532 (-1.44%) Latency: 427188 -> 421729 (-1.28%); split: -1.28%, +0.00% InvThroughput: 103086 -> 101926 (-1.13%) VClause: 785 -> 753 (-4.08%) SClause: 1624 -> 1598 (-1.60%) Copies: 5679 -> 5671 (-0.14%); split: -0.72%, +0.58% PreSGPRs: 3961 -> 3937 (-0.61%) VALU: 51107 -> 50457 (-1.27%) SALU: 9034 -> 8950 (-0.93%) VMEM: 1123 -> 1091 (-2.85%) SMEM: 2862 -> 2830 (-1.12%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 18:54:09 +01:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
/* fract(non_integral) is in (0, 1). */
if (src_res[0] & FP_CLASS_NON_INTEGRAL)
r |= FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_NON_INTEGRAL;
/* fract(small, negative) can be 1.0. */
if (src_res[0] & FP_CLASS_LT_ZERO_GT_NEG_ONE)
r |= FP_CLASS_POS_ONE;
/* fract(integral) is +0.0. */
if (src_res[0] & (FP_CLASS_LT_NEG_ONE | FP_CLASS_NEG_ONE | FP_CLASS_ANY_ZERO | FP_CLASS_POS_ONE | FP_CLASS_GT_POS_ONE))
r |= FP_CLASS_POS_ZERO;
nir: range analysis for ffract Foz-DB Navi21: Totals from 75 (0.09% of 79377) affected shaders: Instrs: 69239 -> 68383 (-1.24%) CodeSize: 385088 -> 379532 (-1.44%) Latency: 427188 -> 421729 (-1.28%); split: -1.28%, +0.00% InvThroughput: 103086 -> 101926 (-1.13%) VClause: 785 -> 753 (-4.08%) SClause: 1624 -> 1598 (-1.60%) Copies: 5679 -> 5671 (-0.14%); split: -0.72%, +0.58% PreSGPRs: 3961 -> 3937 (-0.61%) VALU: 51107 -> 50457 (-1.27%) SALU: 9034 -> 8950 (-0.93%) VMEM: 1123 -> 1091 (-2.85%) SMEM: 2862 -> 2830 (-1.12%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33557>
2025-02-14 18:54:09 +01:00
break;
}
nir: add fp class analysis for sin/cos Foz-DB Navi48: Totals from 264 (0.32% of 82636) affected shaders: CodeSize: 1688676 -> 1688672 (-0.00%) Latency: 510773 -> 510772 (-0.00%) InvThroughput: 138569 -> 138568 (-0.00%) Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-11 19:53:46 +01:00
case nir_op_fsin:
case nir_op_fcos:
case nir_op_fsin_amd:
case nir_op_fcos_amd: {
/* [-1, +1], and sin/cos(Inf) is NaN */
r = FP_CLASS_NEG_ONE | FP_CLASS_LT_ZERO_GT_NEG_ONE | FP_CLASS_ANY_ZERO |
FP_CLASS_GT_ZERO_LT_POS_ONE | FP_CLASS_POS_ONE | FP_CLASS_NON_INTEGRAL;
if (src_res[0] & (FP_CLASS_NAN | FP_CLASS_ANY_INF))
r |= FP_CLASS_NAN;
break;
}
nir/range_analysis: Teach range analysis about fdot opcodes This really, really helps on platforms where fabs() isn't free. A great many shaders use a * frsq(fabs(fdot(a, a))) to normalize a vector. Since the result of the fdot must be non-negative, the fabs can be eliminated by an existing algebraic rule. shader-db results: r300 (run on R420 - X800XL) total instructions in shared programs: 1369807 -> 1368550 (-0.09%) instructions in affected programs: 59986 -> 58729 (-2.10%) helped: 609 HURT: 0 total vinst in shared programs: 512899 -> 512861 (<.01%) vinst in affected programs: 1522 -> 1484 (-2.50%) helped: 36 HURT: 0 total sinst in shared programs: 260690 -> 260570 (-0.05%) sinst in affected programs: 1419 -> 1299 (-8.46%) helped: 120 HURT: 0 total consts in shared programs: 957295 -> 957230 (<.01%) consts in affected programs: 849 -> 784 (-7.66%) helped: 65 HURT: 0 LOST: 0 GAINED: 3 The 3 gained shaders are all vertex shaders from XCom: Enemy Unknown. I'm guessing that game is never going to run on my X800XL. :) i915 total instructions in shared programs: 791121 -> 780843 (-1.30%) instructions in affected programs: 220170 -> 209892 (-4.67%) helped: 2085 HURT: 0 total temps in shared programs: 47765 -> 47766 (<.01%) temps in affected programs: 9 -> 10 (11.11%) helped: 0 HURT: 1 total const in shared programs: 93048 -> 92983 (-0.07%) const in affected programs: 784 -> 719 (-8.29%) helped: 65 HURT: 0 LOST: 0 GAINED: 36 Haswell, Ivy Bridge, and Sandy Bridge had similar results. (Haswell shown) total instructions in shared programs: 16702250 -> 16697908 (-0.03%) instructions in affected programs: 119277 -> 114935 (-3.64%) helped: 1065 HURT: 0 helped stats (abs) min: 1 max: 20 x̄: 4.08 x̃: 4 helped stats (rel) min: 0.48% max: 10.17% x̄: 3.66% x̃: 3.94% 95% mean confidence interval for instructions value: -4.26 -3.89 95% mean confidence interval for instructions %-change: -3.76% -3.56% Instructions are helped. total cycles in shared programs: 880772068 -> 880734134 (<.01%) cycles in affected programs: 2134456 -> 2096522 (-1.78%) helped: 941 HURT: 324 helped stats (abs) min: 2 max: 2180 x̄: 123.06 x̃: 44 helped stats (rel) min: 0.04% max: 49.96% x̄: 7.08% x̃: 3.81% HURT stats (abs) min: 2 max: 2098 x̄: 240.33 x̃: 35 HURT stats (rel) min: 0.04% max: 77.07% x̄: 12.34% x̃: 3.00% 95% mean confidence interval for cycles value: -47.93 -12.04 95% mean confidence interval for cycles %-change: -2.87% -1.34% Cycles are helped. No shader-db changes on any other Intel platform. Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17181>
2022-06-21 16:47:31 -07:00
case nir_op_fdot2:
case nir_op_fdot3:
case nir_op_fdot4:
case nir_op_fdot8:
case nir_op_fdot16:
case nir_op_fdot2_replicated:
case nir_op_fdot3_replicated:
case nir_op_fdot4_replicated:
case nir_op_fdot8_replicated:
case nir_op_fdot16_replicated: {
/* If the two sources are the same SSA value, then the result is either
* NaN or some number >= 0. If one source is the negation of the other,
* the result is either NaN or some number <= 0.
*
* In either of these two cases, if one source is a number, then the
* other must also be a number. Since it should not be possible to get
* Inf-Inf in the dot-product, the result must also be a number.
*/
if (nir_alu_srcs_equal(alu, alu, 0, 1)) {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = FP_CLASS_ANY_POS | FP_CLASS_POS_ZERO;
nir/range_analysis: Teach range analysis about fdot opcodes This really, really helps on platforms where fabs() isn't free. A great many shaders use a * frsq(fabs(fdot(a, a))) to normalize a vector. Since the result of the fdot must be non-negative, the fabs can be eliminated by an existing algebraic rule. shader-db results: r300 (run on R420 - X800XL) total instructions in shared programs: 1369807 -> 1368550 (-0.09%) instructions in affected programs: 59986 -> 58729 (-2.10%) helped: 609 HURT: 0 total vinst in shared programs: 512899 -> 512861 (<.01%) vinst in affected programs: 1522 -> 1484 (-2.50%) helped: 36 HURT: 0 total sinst in shared programs: 260690 -> 260570 (-0.05%) sinst in affected programs: 1419 -> 1299 (-8.46%) helped: 120 HURT: 0 total consts in shared programs: 957295 -> 957230 (<.01%) consts in affected programs: 849 -> 784 (-7.66%) helped: 65 HURT: 0 LOST: 0 GAINED: 3 The 3 gained shaders are all vertex shaders from XCom: Enemy Unknown. I'm guessing that game is never going to run on my X800XL. :) i915 total instructions in shared programs: 791121 -> 780843 (-1.30%) instructions in affected programs: 220170 -> 209892 (-4.67%) helped: 2085 HURT: 0 total temps in shared programs: 47765 -> 47766 (<.01%) temps in affected programs: 9 -> 10 (11.11%) helped: 0 HURT: 1 total const in shared programs: 93048 -> 92983 (-0.07%) const in affected programs: 784 -> 719 (-8.29%) helped: 65 HURT: 0 LOST: 0 GAINED: 36 Haswell, Ivy Bridge, and Sandy Bridge had similar results. (Haswell shown) total instructions in shared programs: 16702250 -> 16697908 (-0.03%) instructions in affected programs: 119277 -> 114935 (-3.64%) helped: 1065 HURT: 0 helped stats (abs) min: 1 max: 20 x̄: 4.08 x̃: 4 helped stats (rel) min: 0.48% max: 10.17% x̄: 3.66% x̃: 3.94% 95% mean confidence interval for instructions value: -4.26 -3.89 95% mean confidence interval for instructions %-change: -3.76% -3.56% Instructions are helped. total cycles in shared programs: 880772068 -> 880734134 (<.01%) cycles in affected programs: 2134456 -> 2096522 (-1.78%) helped: 941 HURT: 324 helped stats (abs) min: 2 max: 2180 x̄: 123.06 x̃: 44 helped stats (rel) min: 0.04% max: 49.96% x̄: 7.08% x̃: 3.81% HURT stats (abs) min: 2 max: 2098 x̄: 240.33 x̃: 35 HURT stats (rel) min: 0.04% max: 77.07% x̄: 12.34% x̃: 3.00% 95% mean confidence interval for cycles value: -47.93 -12.04 95% mean confidence interval for cycles %-change: -2.87% -1.34% Cycles are helped. No shader-db changes on any other Intel platform. Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17181>
2022-06-21 16:47:31 -07:00
} else if (nir_alu_srcs_negative_equal(alu, alu, 0, 1)) {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = FP_CLASS_ANY_NEG | FP_CLASS_NEG_ZERO;
nir/range_analysis: Teach range analysis about fdot opcodes This really, really helps on platforms where fabs() isn't free. A great many shaders use a * frsq(fabs(fdot(a, a))) to normalize a vector. Since the result of the fdot must be non-negative, the fabs can be eliminated by an existing algebraic rule. shader-db results: r300 (run on R420 - X800XL) total instructions in shared programs: 1369807 -> 1368550 (-0.09%) instructions in affected programs: 59986 -> 58729 (-2.10%) helped: 609 HURT: 0 total vinst in shared programs: 512899 -> 512861 (<.01%) vinst in affected programs: 1522 -> 1484 (-2.50%) helped: 36 HURT: 0 total sinst in shared programs: 260690 -> 260570 (-0.05%) sinst in affected programs: 1419 -> 1299 (-8.46%) helped: 120 HURT: 0 total consts in shared programs: 957295 -> 957230 (<.01%) consts in affected programs: 849 -> 784 (-7.66%) helped: 65 HURT: 0 LOST: 0 GAINED: 3 The 3 gained shaders are all vertex shaders from XCom: Enemy Unknown. I'm guessing that game is never going to run on my X800XL. :) i915 total instructions in shared programs: 791121 -> 780843 (-1.30%) instructions in affected programs: 220170 -> 209892 (-4.67%) helped: 2085 HURT: 0 total temps in shared programs: 47765 -> 47766 (<.01%) temps in affected programs: 9 -> 10 (11.11%) helped: 0 HURT: 1 total const in shared programs: 93048 -> 92983 (-0.07%) const in affected programs: 784 -> 719 (-8.29%) helped: 65 HURT: 0 LOST: 0 GAINED: 36 Haswell, Ivy Bridge, and Sandy Bridge had similar results. (Haswell shown) total instructions in shared programs: 16702250 -> 16697908 (-0.03%) instructions in affected programs: 119277 -> 114935 (-3.64%) helped: 1065 HURT: 0 helped stats (abs) min: 1 max: 20 x̄: 4.08 x̃: 4 helped stats (rel) min: 0.48% max: 10.17% x̄: 3.66% x̃: 3.94% 95% mean confidence interval for instructions value: -4.26 -3.89 95% mean confidence interval for instructions %-change: -3.76% -3.56% Instructions are helped. total cycles in shared programs: 880772068 -> 880734134 (<.01%) cycles in affected programs: 2134456 -> 2096522 (-1.78%) helped: 941 HURT: 324 helped stats (abs) min: 2 max: 2180 x̄: 123.06 x̃: 44 helped stats (rel) min: 0.04% max: 49.96% x̄: 7.08% x̃: 3.81% HURT stats (abs) min: 2 max: 2098 x̄: 240.33 x̃: 35 HURT stats (rel) min: 0.04% max: 77.07% x̄: 12.34% x̃: 3.00% 95% mean confidence interval for cycles value: -47.93 -12.04 95% mean confidence interval for cycles %-change: -2.87% -1.34% Cycles are helped. No shader-db changes on any other Intel platform. Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17181>
2022-06-21 16:47:31 -07:00
} else {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = FP_CLASS_UNKNOWN;
nir/range_analysis: Teach range analysis about fdot opcodes This really, really helps on platforms where fabs() isn't free. A great many shaders use a * frsq(fabs(fdot(a, a))) to normalize a vector. Since the result of the fdot must be non-negative, the fabs can be eliminated by an existing algebraic rule. shader-db results: r300 (run on R420 - X800XL) total instructions in shared programs: 1369807 -> 1368550 (-0.09%) instructions in affected programs: 59986 -> 58729 (-2.10%) helped: 609 HURT: 0 total vinst in shared programs: 512899 -> 512861 (<.01%) vinst in affected programs: 1522 -> 1484 (-2.50%) helped: 36 HURT: 0 total sinst in shared programs: 260690 -> 260570 (-0.05%) sinst in affected programs: 1419 -> 1299 (-8.46%) helped: 120 HURT: 0 total consts in shared programs: 957295 -> 957230 (<.01%) consts in affected programs: 849 -> 784 (-7.66%) helped: 65 HURT: 0 LOST: 0 GAINED: 3 The 3 gained shaders are all vertex shaders from XCom: Enemy Unknown. I'm guessing that game is never going to run on my X800XL. :) i915 total instructions in shared programs: 791121 -> 780843 (-1.30%) instructions in affected programs: 220170 -> 209892 (-4.67%) helped: 2085 HURT: 0 total temps in shared programs: 47765 -> 47766 (<.01%) temps in affected programs: 9 -> 10 (11.11%) helped: 0 HURT: 1 total const in shared programs: 93048 -> 92983 (-0.07%) const in affected programs: 784 -> 719 (-8.29%) helped: 65 HURT: 0 LOST: 0 GAINED: 36 Haswell, Ivy Bridge, and Sandy Bridge had similar results. (Haswell shown) total instructions in shared programs: 16702250 -> 16697908 (-0.03%) instructions in affected programs: 119277 -> 114935 (-3.64%) helped: 1065 HURT: 0 helped stats (abs) min: 1 max: 20 x̄: 4.08 x̃: 4 helped stats (rel) min: 0.48% max: 10.17% x̄: 3.66% x̃: 3.94% 95% mean confidence interval for instructions value: -4.26 -3.89 95% mean confidence interval for instructions %-change: -3.76% -3.56% Instructions are helped. total cycles in shared programs: 880772068 -> 880734134 (<.01%) cycles in affected programs: 2134456 -> 2096522 (-1.78%) helped: 941 HURT: 324 helped stats (abs) min: 2 max: 2180 x̄: 123.06 x̃: 44 helped stats (rel) min: 0.04% max: 49.96% x̄: 7.08% x̃: 3.81% HURT stats (abs) min: 2 max: 2098 x̄: 240.33 x̃: 35 HURT stats (rel) min: 0.04% max: 77.07% x̄: 12.34% x̃: 3.00% 95% mean confidence interval for cycles value: -47.93 -12.04 95% mean confidence interval for cycles %-change: -2.87% -1.34% Cycles are helped. No shader-db changes on any other Intel platform. Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17181>
2022-06-21 16:47:31 -07:00
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (src_res[0] & FP_CLASS_NAN)
r |= FP_CLASS_NAN;
if (src_res[0] & FP_CLASS_NON_INTEGRAL)
r |= FP_CLASS_NON_INTEGRAL;
nir/range_analysis: Teach range analysis about fdot opcodes This really, really helps on platforms where fabs() isn't free. A great many shaders use a * frsq(fabs(fdot(a, a))) to normalize a vector. Since the result of the fdot must be non-negative, the fabs can be eliminated by an existing algebraic rule. shader-db results: r300 (run on R420 - X800XL) total instructions in shared programs: 1369807 -> 1368550 (-0.09%) instructions in affected programs: 59986 -> 58729 (-2.10%) helped: 609 HURT: 0 total vinst in shared programs: 512899 -> 512861 (<.01%) vinst in affected programs: 1522 -> 1484 (-2.50%) helped: 36 HURT: 0 total sinst in shared programs: 260690 -> 260570 (-0.05%) sinst in affected programs: 1419 -> 1299 (-8.46%) helped: 120 HURT: 0 total consts in shared programs: 957295 -> 957230 (<.01%) consts in affected programs: 849 -> 784 (-7.66%) helped: 65 HURT: 0 LOST: 0 GAINED: 3 The 3 gained shaders are all vertex shaders from XCom: Enemy Unknown. I'm guessing that game is never going to run on my X800XL. :) i915 total instructions in shared programs: 791121 -> 780843 (-1.30%) instructions in affected programs: 220170 -> 209892 (-4.67%) helped: 2085 HURT: 0 total temps in shared programs: 47765 -> 47766 (<.01%) temps in affected programs: 9 -> 10 (11.11%) helped: 0 HURT: 1 total const in shared programs: 93048 -> 92983 (-0.07%) const in affected programs: 784 -> 719 (-8.29%) helped: 65 HURT: 0 LOST: 0 GAINED: 36 Haswell, Ivy Bridge, and Sandy Bridge had similar results. (Haswell shown) total instructions in shared programs: 16702250 -> 16697908 (-0.03%) instructions in affected programs: 119277 -> 114935 (-3.64%) helped: 1065 HURT: 0 helped stats (abs) min: 1 max: 20 x̄: 4.08 x̃: 4 helped stats (rel) min: 0.48% max: 10.17% x̄: 3.66% x̃: 3.94% 95% mean confidence interval for instructions value: -4.26 -3.89 95% mean confidence interval for instructions %-change: -3.76% -3.56% Instructions are helped. total cycles in shared programs: 880772068 -> 880734134 (<.01%) cycles in affected programs: 2134456 -> 2096522 (-1.78%) helped: 941 HURT: 324 helped stats (abs) min: 2 max: 2180 x̄: 123.06 x̃: 44 helped stats (rel) min: 0.04% max: 49.96% x̄: 7.08% x̃: 3.81% HURT stats (abs) min: 2 max: 2098 x̄: 240.33 x̃: 35 HURT stats (rel) min: 0.04% max: 77.07% x̄: 12.34% x̃: 3.00% 95% mean confidence interval for cycles value: -47.93 -12.04 95% mean confidence interval for cycles %-change: -2.87% -1.34% Cycles are helped. No shader-db changes on any other Intel platform. Reviewed-by: Jason Ekstrand <jason.ekstrand@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17181>
2022-06-21 16:47:31 -07:00
break;
}
nir/range-analysis: Range tracking for fpow One shader from Metro Last Light and the rest from Rochard. In the Rochard cases, something like: min(1.0, max(pow(saturate(x), y), z)) was transformed to saturate(max(pow(saturate(x), y), z)) because the result of the pow must be >= 0. The Metro Last Light case was similar. An instance of min(pow(abs(x), y), 1.0) became saturate(pow(abs(x), y)) v2: Fix some comments. Suggested by Caio. v3: Fix setting is_intgral when the exponent might be negative. See also Mesa MR !1778. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> All Intel platforms had similar results. (Ice Lake shown) total instructions in shared programs: 16280670 -> 16280659 (<.01%) instructions in affected programs: 1130 -> 1119 (-0.97%) helped: 11 HURT: 0 helped stats (abs) min: 1 max: 1 x̄: 1.00 x̃: 1 helped stats (rel) min: 0.72% max: 1.43% x̄: 1.03% x̃: 0.97% 95% mean confidence interval for instructions value: -1.00 -1.00 95% mean confidence interval for instructions %-change: -1.19% -0.86% Instructions are helped. total cycles in shared programs: 367168430 -> 367168270 (<.01%) cycles in affected programs: 10281 -> 10121 (-1.56%) helped: 10 HURT: 1 helped stats (abs) min: 16 max: 18 x̄: 17.00 x̃: 17 helped stats (rel) min: 1.31% max: 2.43% x̄: 1.79% x̃: 1.70% HURT stats (abs) min: 10 max: 10 x̄: 10.00 x̃: 10 HURT stats (rel) min: 3.10% max: 3.10% x̄: 3.10% x̃: 3.10% 95% mean confidence interval for cycles value: -20.06 -9.04 95% mean confidence interval for cycles %-change: -2.36% -0.32% Cycles are helped.
2019-08-09 12:48:27 -07:00
case nir_op_fpow: {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
/* This is a basic port of the old range analysis, the opcode is very
* underdefined. But improvements are likely possible.
* Due to flush-to-zero semanatics of floating-point numbers with very
nir/range-analysis: Range tracking for fpow One shader from Metro Last Light and the rest from Rochard. In the Rochard cases, something like: min(1.0, max(pow(saturate(x), y), z)) was transformed to saturate(max(pow(saturate(x), y), z)) because the result of the pow must be >= 0. The Metro Last Light case was similar. An instance of min(pow(abs(x), y), 1.0) became saturate(pow(abs(x), y)) v2: Fix some comments. Suggested by Caio. v3: Fix setting is_intgral when the exponent might be negative. See also Mesa MR !1778. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> All Intel platforms had similar results. (Ice Lake shown) total instructions in shared programs: 16280670 -> 16280659 (<.01%) instructions in affected programs: 1130 -> 1119 (-0.97%) helped: 11 HURT: 0 helped stats (abs) min: 1 max: 1 x̄: 1.00 x̃: 1 helped stats (rel) min: 0.72% max: 1.43% x̄: 1.03% x̃: 0.97% 95% mean confidence interval for instructions value: -1.00 -1.00 95% mean confidence interval for instructions %-change: -1.19% -0.86% Instructions are helped. total cycles in shared programs: 367168430 -> 367168270 (<.01%) cycles in affected programs: 10281 -> 10121 (-1.56%) helped: 10 HURT: 1 helped stats (abs) min: 16 max: 18 x̄: 17.00 x̃: 17 helped stats (rel) min: 1.31% max: 2.43% x̄: 1.79% x̃: 1.70% HURT stats (abs) min: 10 max: 10 x̄: 10.00 x̃: 10 HURT stats (rel) min: 3.10% max: 3.10% x̄: 3.10% x̃: 3.10% 95% mean confidence interval for cycles value: -20.06 -9.04 95% mean confidence interval for cycles %-change: -2.36% -0.32% Cycles are helped.
2019-08-09 12:48:27 -07:00
* small mangnitudes, we can never really be sure a result will be
* non-zero.
*
* NIR uses pow() and powf() to constant evaluate nir_op_fpow. The man
* page for that function says:
*
* If y is 0, the result is 1.0 (even if x is a NaN).
*
* gt_zero: pow(*, eq_zero)
* | pow(eq_zero, lt_zero) # 0^-y = +inf
* | pow(eq_zero, le_zero) # 0^-y = +inf or 0^0 = 1.0
* ;
*
* eq_zero: pow(eq_zero, gt_zero)
* ;
*
* ge_zero: pow(gt_zero, gt_zero)
* | pow(gt_zero, ge_zero)
* | pow(gt_zero, lt_zero)
* | pow(gt_zero, le_zero)
* | pow(gt_zero, ne_zero)
* | pow(gt_zero, unknown)
* | pow(ge_zero, gt_zero)
* | pow(ge_zero, ge_zero)
* | pow(ge_zero, lt_zero)
* | pow(ge_zero, le_zero)
* | pow(ge_zero, ne_zero)
* | pow(ge_zero, unknown)
* | pow(eq_zero, ge_zero) # 0^0 = 1.0 or 0^+y = 0.0
* | pow(eq_zero, ne_zero) # 0^-y = +inf or 0^+y = 0.0
* | pow(eq_zero, unknown) # union of all other y cases
* ;
*
* All other cases are unknown.
*
* We could do better if the right operand is a constant, integral
* value.
*/
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
fp_class_mask left = src_res[0];
fp_class_mask right = src_res[1];
if (!(right & (FP_CLASS_ANY_NEG | FP_CLASS_ANY_POS))) {
r = FP_CLASS_ANY_POS;
} else if (left & (FP_CLASS_ANY_NEG | FP_CLASS_NEG_ZERO)) {
r = FP_CLASS_UNKNOWN;
} else {
r = FP_CLASS_ANY_POS | FP_CLASS_ANY_ZERO;
if ((right & (FP_CLASS_ANY_NEG | FP_CLASS_NON_INTEGRAL)) || (left & FP_CLASS_NON_INTEGRAL))
r |= FP_CLASS_NON_INTEGRAL;
}
nir/range_analysis: Add "is a number" range analysis tracking This commit is necessary to support "nir/range_analysis: Fix analysis of fmin and fmax with NaN". No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_a_number. v3: Don't set is_a_number of integer constants. The bit pattern might be NaN. v4: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-07-02 18:18:09 -07:00
/* Various cases can result in NaN, so assume the worst. */
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r |= FP_CLASS_NAN;
nir/range_analysis: Add "is a number" range analysis tracking This commit is necessary to support "nir/range_analysis: Fix analysis of fmin and fmax with NaN". No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_a_number. v3: Don't set is_a_number of integer constants. The bit pattern might be NaN. v4: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-07-02 18:18:09 -07:00
nir/range-analysis: Range tracking for fpow One shader from Metro Last Light and the rest from Rochard. In the Rochard cases, something like: min(1.0, max(pow(saturate(x), y), z)) was transformed to saturate(max(pow(saturate(x), y), z)) because the result of the pow must be >= 0. The Metro Last Light case was similar. An instance of min(pow(abs(x), y), 1.0) became saturate(pow(abs(x), y)) v2: Fix some comments. Suggested by Caio. v3: Fix setting is_intgral when the exponent might be negative. See also Mesa MR !1778. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com> All Intel platforms had similar results. (Ice Lake shown) total instructions in shared programs: 16280670 -> 16280659 (<.01%) instructions in affected programs: 1130 -> 1119 (-0.97%) helped: 11 HURT: 0 helped stats (abs) min: 1 max: 1 x̄: 1.00 x̃: 1 helped stats (rel) min: 0.72% max: 1.43% x̄: 1.03% x̃: 0.97% 95% mean confidence interval for instructions value: -1.00 -1.00 95% mean confidence interval for instructions %-change: -1.19% -0.86% Instructions are helped. total cycles in shared programs: 367168430 -> 367168270 (<.01%) cycles in affected programs: 10281 -> 10121 (-1.56%) helped: 10 HURT: 1 helped stats (abs) min: 16 max: 18 x̄: 17.00 x̃: 17 helped stats (rel) min: 1.31% max: 2.43% x̄: 1.79% x̃: 1.70% HURT stats (abs) min: 10 max: 10 x̄: 10.00 x̃: 10 HURT stats (rel) min: 3.10% max: 3.10% x̄: 3.10% x̃: 3.10% 95% mean confidence interval for cycles value: -20.06 -9.04 95% mean confidence interval for cycles %-change: -2.36% -0.32% Cycles are helped.
2019-08-09 12:48:27 -07:00
break;
}
nir: add range analysis for ffmaz Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34125>
2025-03-24 08:20:46 +01:00
case nir_op_ffma:
case nir_op_ffmaz: {
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
bool mulz = alu->op == nir_op_ffmaz;
bool src_eq = nir_alu_srcs_equal(alu, alu, 0, 1);
bool src_neg_eq = !nir_src_is_const(alu->src[0].src) && nir_alu_srcs_negative_equal(alu, alu, 0, 1);
fp_class_mask r_mul = fmul_fp_class(src_res[0], src_res[1], mulz, src_eq, src_neg_eq);
r = fadd_fp_class(r_mul, src_res[2]);
/* fma(a, b, +0.0) can be -0.0 if a * b underflows.
* When fused, the underflow is not flushed before the addition.
*/
bool mul_underflow = (((src_res[0] & FP_CLASS_LT_ZERO_GT_NEG_ONE) && (src_res[1] & FP_CLASS_GT_ZERO_LT_POS_ONE)) ||
((src_res[1] & FP_CLASS_LT_ZERO_GT_NEG_ONE) && (src_res[0] & FP_CLASS_GT_ZERO_LT_POS_ONE)));
if (!src_eq && mul_underflow && (src_res[2] & FP_CLASS_POS_ZERO))
r |= FP_CLASS_NEG_ZERO;
nir/range-analysis: Range tracking for ffma and flrp A similar technique could be used for fmin3, fmax3, and fmid3. This could be squashed with the previous commit. I kept it separate to ease review. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-08-01 11:51:36 -07:00
break;
}
case nir_op_flrp: {
/* Decompose the flrp to first + third * (second + -first) */
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
fp_class_mask inner_fadd_class =
fadd_fp_class(src_res[1], fneg_fp_class(src_res[0]));
nir/range-analysis: Range tracking for ffma and flrp A similar technique could be used for fmin3, fmax3, and fmid3. This could be squashed with the previous commit. I kept it separate to ease review. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-08-01 11:51:36 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
fp_class_mask fmul_class =
fmul_fp_class(src_res[2], inner_fadd_class, false, false, false);
nir/range-analysis: Range tracking for ffma and flrp A similar technique could be used for fmin3, fmax3, and fmid3. This could be squashed with the previous commit. I kept it separate to ease review. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-08-01 11:51:36 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = fadd_fp_class(src_res[0], fmul_class);
/* Various cases can result in NaN, so assume the worst. */
r |= FP_CLASS_NAN;
nir/range-analysis: Range tracking for ffma and flrp A similar technique could be used for fmin3, fmax3, and fmid3. This could be squashed with the previous commit. I kept it separate to ease review. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2019-08-01 11:51:36 -07:00
break;
}
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
default:
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
r = FP_CLASS_UNKNOWN;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
break;
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
if (nir_alu_type_get_base_type(nir_op_infos[alu->op].output_type) == nir_type_float)
r = handle_sz(alu, r);
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
assert((r & FP_CLASS_UNKNOWN) == r);
assert((r & ~FP_CLASS_NON_INTEGRAL) != 0);
assert(!(r & FP_CLASS_NON_INTEGRAL) || (r & (FP_CLASS_LT_NEG_ONE | FP_CLASS_LT_ZERO_GT_NEG_ONE |
FP_CLASS_GT_POS_ONE | FP_CLASS_GT_ZERO_LT_POS_ONE)));
nir/range_analysis: Add "is a number" range analysis tracking This commit is necessary to support "nir/range_analysis: Fix analysis of fmin and fmax with NaN". No shader-db or fossil-db changes on any Intel platform. v2: Pack and unpack is_a_number. v3: Don't set is_a_number of integer constants. The bit pattern might be NaN. v4: Update handling of b2i32. intBitsToFloat(int(true)) is 1.401298464324817e-45. Return a value consistent with that. Fixes: 405de7ccb6c ("nir/range-analysis: Rudimentary value range analysis pass") Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9108>
2020-07-02 18:18:09 -07:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
*result = r;
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
}
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
fp_class_mask
nir_analyze_fp_class(nir_fp_analysis_state *fp_state, const nir_def *def)
nir/range-analysis: Rudimentary value range analysis pass Most integer operations are omitted because dealing with integer overflow is hard. There are a few things that could be smarter if there was a small amount more tracking of ranges of integer types (i.e., operands are Boolean, operand values fit in 16 bits, etc.). The changes to nir_search_helpers.h are included in this patch to simplify reordering the changes to nir_opt_algebraic.py. v2: Memoize range analysis results. Without this, some shaders appear to get stuck in infinite loops. v3: Rebase on many months of Mesa changes, including 1-bit Boolean changes. v4: Rebase on "nir: Drop imov/fmov in favor of one mov instruction". v5: Use nir_alu_srcs_equal for detecting (a*a). Previously just the SSA value was compared, and this incorrectly matched (a.x*a.y). v6: Many code improvements including (but not limited to) better names, more comments, and better use of helper functions. All suggested by Caio. Rework the handling of several opcodes to use a table for mapping source ranges to a result range. This change fixed a bug that caused fmax(gt_zero, ge_zero) to be incorrectly recognized as ge_zero. Slightly tighten the range of fmul by recognizing that x*x is gt_zero if x is gt_zero. Add similar handling for -x*x. v7: Use _______ in the tables as an alias for unknown. Suggested by Caio. Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
2018-01-23 09:48:43 +08:00
{
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
struct fp_query query_alloc[64];
uint32_t result_alloc[64];
struct analysis_state state;
nir/range_analysis: use sparse array for float analysis This seems to be faster. ministat (nir_analyze_fp_range): Difference at 95.0% confidence -592900 +/- 2302.24 -27.6432% +/- 0.0998961% (Student's t, pooled s = 2719.05) ministat (overall): Difference at 95.0% confidence -76.8333 +/- 27.2345 -0.632558% +/- 0.223407% (Student's t, pooled s = 46.867) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:44:38 +00:00
state.range_ht = fp_state;
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
util_dynarray_init_from_stack(&state.query_stack, query_alloc, sizeof(query_alloc));
util_dynarray_init_from_stack(&state.result_stack, result_alloc, sizeof(result_alloc));
state.query_size = sizeof(struct fp_query);
state.get_key = &get_fp_key;
nir/range_analysis: use sparse array for float analysis This seems to be faster. ministat (nir_analyze_fp_range): Difference at 95.0% confidence -592900 +/- 2302.24 -27.6432% +/- 0.0998961% (Student's t, pooled s = 2719.05) ministat (overall): Difference at 95.0% confidence -76.8333 +/- 27.2345 -0.632558% +/- 0.223407% (Student's t, pooled s = 46.867) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:44:38 +00:00
state.lookup = &fp_lookup;
state.insert = &fp_insert;
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
state.process_query = &process_fp_query;
nir: let nir_analyze_fp_range take a nir_def This is midly worse for vector constants, but so much simpler. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39756>
2026-02-07 22:36:25 +01:00
push_fp_query(&state, def);
nir/range_analysis: use perform_analysis() in nir_analyze_range() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-14 21:38:41 +00:00
nir: rewrite fp range analysis as a fp class analysis Knowing if a value is not larger than one helps proving finite results of fmul/fadd and will allow skipping/creating more fsat. Knowing that a value is larger than one helps proving non zero results of fmul. Separating positive and negative zero also has advantages when signed zero correctness is required. Foz-DB Navi48: Totals from 1344 (1.63% of 82636) affected shaders: Instrs: 5319389 -> 5312280 (-0.13%); split: -0.14%, +0.01% CodeSize: 29702516 -> 29665684 (-0.12%); split: -0.13%, +0.01% Latency: 40694344 -> 40694545 (+0.00%); split: -0.01%, +0.02% InvThroughput: 7481192 -> 7480403 (-0.01%); split: -0.02%, +0.01% VClause: 121947 -> 121946 (-0.00%); split: -0.00%, +0.00% SClause: 104972 -> 104923 (-0.05%); split: -0.05%, +0.00% Copies: 371098 -> 371092 (-0.00%); split: -0.02%, +0.02% Branches: 122929 -> 122919 (-0.01%); split: -0.01%, +0.00% PreSGPRs: 82506 -> 82510 (+0.00%); split: -0.00%, +0.01% PreVGPRs: 79175 -> 79168 (-0.01%) VALU: 2906718 -> 2904777 (-0.07%); split: -0.07%, +0.00% SALU: 726256 -> 723454 (-0.39%); split: -0.39%, +0.00% VMEM: 205021 -> 205016 (-0.00%) SMEM: 163972 -> 163916 (-0.03%) VOPD: 303354 -> 303298 (-0.02%); split: +0.02%, -0.04% Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39987>
2026-02-08 23:03:16 +01:00
return perform_analysis(&state);
}
nir: add nir_fp_analysis_state Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:32:41 +00:00
nir_fp_analysis_state
nir_create_fp_analysis_state(nir_function_impl *impl)
{
nir/range_analysis: use sparse array for float analysis This seems to be faster. ministat (nir_analyze_fp_range): Difference at 95.0% confidence -592900 +/- 2302.24 -27.6432% +/- 0.0998961% (Student's t, pooled s = 2719.05) ministat (overall): Difference at 95.0% confidence -76.8333 +/- 27.2345 -0.632558% +/- 0.223407% (Student's t, pooled s = 46.867) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:44:38 +00:00
nir_fp_analysis_state state;
state.impl = impl;
/* Over-allocate the bitset, so that we can keep using the allocated table memory
* even when new SSA values are added. */
state.size = BITSET_BYTES(impl->ssa_alloc + impl->ssa_alloc / 4u);
state.max = -1;
state.bitset = calloc(state.size, 1);
nir/range_analysis: use uint16_t for sparse array elements ministat (nir_analyze_fp_class): Difference at 95.0% confidence -4484.55 +/- 1288.68 -0.205419% +/- 0.0589514% (Student's t, pooled s = 1521.99) This should also use less memory. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40346>
2026-03-09 09:42:03 +00:00
util_sparse_array_init(&state.arr, 2, 256);
nir/range_analysis: use sparse array for float analysis This seems to be faster. ministat (nir_analyze_fp_range): Difference at 95.0% confidence -592900 +/- 2302.24 -27.6432% +/- 0.0998961% (Student's t, pooled s = 2719.05) ministat (overall): Difference at 95.0% confidence -76.8333 +/- 27.2345 -0.632558% +/- 0.223407% (Student's t, pooled s = 46.867) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:44:38 +00:00
return state;
nir: add nir_fp_analysis_state Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:32:41 +00:00
}
void
nir_invalidate_fp_analysis_state(nir_fp_analysis_state *state)
{
nir/range_analysis: use sparse array for float analysis This seems to be faster. ministat (nir_analyze_fp_range): Difference at 95.0% confidence -592900 +/- 2302.24 -27.6432% +/- 0.0998961% (Student's t, pooled s = 2719.05) ministat (overall): Difference at 95.0% confidence -76.8333 +/- 27.2345 -0.632558% +/- 0.223407% (Student's t, pooled s = 46.867) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:44:38 +00:00
if (BITSET_BYTES(state->impl->ssa_alloc) > state->size) {
state->size = BITSET_BYTES(state->impl->ssa_alloc + state->impl->ssa_alloc / 4u);
free(state->bitset);
state->bitset = calloc(state->size, 1);
} else if (state->max >= 0) {
memset(state->bitset, 0, BITSET_BYTES(state->max + 1));
}
state->max = -1;
nir: add nir_fp_analysis_state Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:32:41 +00:00
}
void
nir_free_fp_analysis_state(nir_fp_analysis_state *state)
{
nir/range_analysis: use sparse array for float analysis This seems to be faster. ministat (nir_analyze_fp_range): Difference at 95.0% confidence -592900 +/- 2302.24 -27.6432% +/- 0.0998961% (Student's t, pooled s = 2719.05) ministat (overall): Difference at 95.0% confidence -76.8333 +/- 27.2345 -0.632558% +/- 0.223407% (Student's t, pooled s = 46.867) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:44:38 +00:00
util_sparse_array_finish(&state->arr);
free(state->bitset);
nir: add nir_fp_analysis_state Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 15:32:41 +00:00
}
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
static uint32_t
bitmask(uint32_t size)
{
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
return size >= 32 ? 0xffffffffu : ((uint32_t)1 << size) - 1u;
}
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
static uint64_t
mul_clamp(uint32_t a, uint32_t b)
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
{
if (a != 0 && (a * b) / a != b)
return (uint64_t)UINT32_MAX + 1;
else
return a * b;
}
nir/unsigned_upper_bound: fix buffer overflow in search_phi_bcsel It should only recurse if there's enough space to add the phi sources. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Fixes: 72ac3f60261 ("nir: add nir_unsigned_upper_bound and nir_addition_might_overflow") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7748>
2020-11-24 10:52:56 +00:00
/* recursively gather at most "buf_size" phi/bcsel sources */
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
static unsigned
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
search_phi_bcsel(nir_scalar scalar, nir_scalar *buf, unsigned buf_size, struct set *visited)
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
{
if (_mesa_set_search(visited, scalar.def))
return 0;
_mesa_set_add(visited, scalar.def);
treewide: add & use parent instr helpers We add a bunch of new helpers to avoid the need to touch >parent_instr, including the full set of: * nir_def_is_* * nir_def_as_*_or_null * nir_def_as_* [assumes the right instr type] * nir_src_is_* * nir_src_as_* * nir_scalar_is_* * nir_scalar_as_* Plus nir_def_instr() where there's no more suitable helper. Also an existing helper is renamed to unify all the names, while we're churning the tree: * nir_src_as_alu_instr -> nir_src_as_alu ..and then we port the tree to use the helpers as much as possible, using nir_def_instr() where that does not work. Acked-by: Marek Olšák <maraeo@gmail.com> --- To eliminate nir_def::parent_instr we need to churn the tree anyway, so I'm taking this opportunity to clean up a lot of NIR patterns. Co-authored-by: Konstantin Seurer <konstantin.seurer@gmail.com> Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38313>
2025-11-07 21:38:36 +08:00
if (nir_def_instr_type(scalar.def) == nir_instr_type_phi) {
treewide: use nir_def_as_* Via Coccinelle patch: @@ expression definition; @@ -nir_instr_as_alu(definition->parent_instr) +nir_def_as_alu(definition) @@ expression definition; @@ -nir_instr_as_intrinsic(definition->parent_instr) +nir_def_as_intrinsic(definition) @@ expression definition; @@ -nir_instr_as_phi(definition->parent_instr) +nir_def_as_phi(definition) @@ expression definition; @@ -nir_instr_as_load_const(definition->parent_instr) +nir_def_as_load_const(definition) @@ expression definition; @@ -nir_instr_as_deref(definition->parent_instr) +nir_def_as_deref(definition) @@ expression definition; @@ -nir_instr_as_tex(definition->parent_instr) +nir_def_as_tex(definition) Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Emma Anholt <emma@anholt.net> Reviewed-by: Marek Olšák <maraeo@gmail.com> Acked-by: Karol Herbst <kherbst@redhat.com> Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36489>
2025-07-31 09:49:36 -04:00
nir_phi_instr *phi = nir_def_as_phi(scalar.def);
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
unsigned num_sources_left = exec_list_length(&phi->srcs);
nir/unsigned_upper_bound: fix buffer overflow in search_phi_bcsel It should only recurse if there's enough space to add the phi sources. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Fixes: 72ac3f60261 ("nir: add nir_unsigned_upper_bound and nir_addition_might_overflow") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7748>
2020-11-24 10:52:56 +00:00
if (buf_size >= num_sources_left) {
unsigned total_added = 0;
nir_foreach_phi_src(src, phi) {
nir/unsigned_upper_bound: decrement num_sources_left before recursing Otherwise, search_phi_bcsel() will be called with a buf_size that is slightly lower than it has to be. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7748>
2020-11-24 14:53:04 +00:00
num_sources_left--;
nir s/nir_get_ssa_scalar/nir_get_scalar/ Generated with sed: sed -i -e 's/nir_get_ssa_scalar/nir_get_scalar/g' src/**/*.h src/**/*.c src/**/*.cpp Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24703>
2023-08-15 10:07:24 -05:00
unsigned added = search_phi_bcsel(nir_get_scalar(src->src.ssa, scalar.comp),
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
buf + total_added, buf_size - num_sources_left, visited);
nir/unsigned_upper_bound: decrement num_sources_left before recursing Otherwise, search_phi_bcsel() will be called with a buf_size that is slightly lower than it has to be. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7748>
2020-11-24 14:53:04 +00:00
assert(added <= buf_size);
nir/unsigned_upper_bound: fix buffer overflow in search_phi_bcsel It should only recurse if there's enough space to add the phi sources. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Fixes: 72ac3f60261 ("nir: add nir_unsigned_upper_bound and nir_addition_might_overflow") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7748>
2020-11-24 10:52:56 +00:00
buf_size -= added;
total_added += added;
}
return total_added;
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
}
}
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
if (nir_scalar_is_alu(scalar)) {
nir_op op = nir_scalar_alu_op(scalar);
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
if ((op == nir_op_bcsel || op == nir_op_b32csel) && buf_size >= 2) {
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_scalar src1 = nir_scalar_chase_alu_src(scalar, 1);
nir_scalar src2 = nir_scalar_chase_alu_src(scalar, 2);
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
nir/unsigned_upper_bound: fix phi(bcsel) This was looking at the wrong sources. src0 is the condition. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com> Fixes: 72ac3f60261 ("nir: add nir_unsigned_upper_bound and nir_addition_might_overflow") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23990>
2023-07-03 18:36:28 +01:00
unsigned added = search_phi_bcsel(src1, buf, buf_size - 1, visited);
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
buf_size -= added;
nir/unsigned_upper_bound: fix phi(bcsel) This was looking at the wrong sources. src0 is the condition. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com> Fixes: 72ac3f60261 ("nir: add nir_unsigned_upper_bound and nir_addition_might_overflow") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23990>
2023-07-03 18:36:28 +01:00
added += search_phi_bcsel(src2, buf + added, buf_size, visited);
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
return added;
}
}
buf[0] = scalar;
return 1;
}
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
static uint32_t
get_max_workgroup_invocations(nir_shader *nir)
{
if (!nir->options || !nir->options->max_workgroup_invocations)
return UINT16_MAX;
nir/range_analysis: Set higher default maximum for max_workgroup_count Fixes: c2a81ebe19f ("nir: Add default unsigned upper bound configuration.") Closes: #7676 Reviewed-by: Karol Herbst <kherbst@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19835>
2022-11-17 12:47:59 -08:00
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
return nir->options->max_workgroup_invocations;
}
static uint32_t
get_max_workgroup_count(nir_shader *nir, unsigned dim)
{
nir/range_analysis: Set higher default maximum for max_workgroup_count Fixes: c2a81ebe19f ("nir: Add default unsigned upper bound configuration.") Closes: #7676 Reviewed-by: Karol Herbst <kherbst@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/19835>
2022-11-17 12:47:59 -08:00
/* max_workgroup_count represents the maximum compute shader / kernel
* dispatchable work size. On most hardware, this is essentially
* unbounded. On some hardware max_workgroup_count[1] and
* max_workgroup_count[2] may be smaller.
*/
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
if (!nir->options || !nir->options->max_workgroup_count[dim])
return UINT32_MAX;
return nir->options->max_workgroup_count[dim];
}
nir: Add default unsigned upper bound configuration. Signed-off-by: Timur Kristóf <timur.kristof@gmail.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-11 12:43:56 +01:00
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
struct scalar_query {
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
struct analysis_query head;
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_scalar scalar;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
};
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
static void
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
push_scalar_query(struct analysis_state *state, nir_scalar scalar)
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
{
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
struct scalar_query *pushed_q = push_analysis_query(state, sizeof(struct scalar_query));
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
pushed_q->scalar = scalar;
}
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
static uint32_t
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
get_scalar_key(struct analysis_query *q)
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
{
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
nir_scalar scalar = ((struct scalar_query *)q)->scalar;
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
/* keys can't be 0, so we have to add 1 to the index */
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
unsigned shift_amount = ffs(NIR_MAX_VEC_COMPONENTS) - 1;
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
return nir_scalar_is_const(scalar)
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
? UINT32_MAX
: ((scalar.def->index + 1) << shift_amount) | scalar.comp;
}
static bool
scalar_lookup(void *table, uint32_t key, uint32_t *value)
{
struct hash_table *ht = table;
struct hash_entry *he = _mesa_hash_table_search(ht, (void *)(uintptr_t)key);
if (he)
*value = (uintptr_t)he->data;
return he != NULL;
}
static void
scalar_insert(void *table, uint32_t key, uint32_t value)
{
struct hash_table *ht = table;
_mesa_hash_table_insert(ht, (void *)(uintptr_t)key, (void *)(uintptr_t)value);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
}
amd: Add extra source to the mbcnt_amd NIR intrinsic. The v_mbcnt instructions can take an extra source that they add to the result. This is not exposed in SPIR-V but we now expose it in NIR. Signed-off-by: Timur Kristóf <timur.kristof@gmail.com> Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11072>
2021-06-09 11:00:22 +02:00
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
static void
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
get_intrinsic_uub(struct analysis_state *state, struct scalar_query q, uint32_t *result,
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
const uint32_t *src)
{
nir_shader *shader = state->shader;
treewide: use nir_def_as_* Via Coccinelle patch: @@ expression definition; @@ -nir_instr_as_alu(definition->parent_instr) +nir_def_as_alu(definition) @@ expression definition; @@ -nir_instr_as_intrinsic(definition->parent_instr) +nir_def_as_intrinsic(definition) @@ expression definition; @@ -nir_instr_as_phi(definition->parent_instr) +nir_def_as_phi(definition) @@ expression definition; @@ -nir_instr_as_load_const(definition->parent_instr) +nir_def_as_load_const(definition) @@ expression definition; @@ -nir_instr_as_deref(definition->parent_instr) +nir_def_as_deref(definition) @@ expression definition; @@ -nir_instr_as_tex(definition->parent_instr) +nir_def_as_tex(definition) Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Emma Anholt <emma@anholt.net> Reviewed-by: Marek Olšák <maraeo@gmail.com> Acked-by: Karol Herbst <kherbst@redhat.com> Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36489>
2025-07-31 09:49:36 -04:00
nir_intrinsic_instr *intrin = nir_def_as_intrinsic(q.scalar.def);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
switch (intrin->intrinsic) {
case nir_intrinsic_load_local_invocation_index:
/* The local invocation index is used under the hood by RADV for
* some non-compute-like shaders (eg. LS and NGG). These technically
* run in workgroups on the HW, even though this fact is not exposed
* by the API.
* They can safely use the same code path here as variable sized
* compute-like shader stages.
*/
all: rename gl_shader_stage_uses_workgroup to mesa_shader_stage_uses_workgroup Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com> Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Yonggang Luo <luoyonggang@gmail.com> Acked-by: Marek Olšák <marek.olsak@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36569>
2025-08-05 16:50:43 +08:00
if (!mesa_shader_stage_uses_workgroup(shader->info.stage) ||
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
shader->info.workgroup_size_variable) {
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
*result = get_max_workgroup_invocations(shader) - 1;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
} else {
*result = (shader->info.workgroup_size[0] *
shader->info.workgroup_size[1] *
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
shader->info.workgroup_size[2]) -
1u;
amd: Add extra source to the mbcnt_amd NIR intrinsic. The v_mbcnt instructions can take an extra source that they add to the result. This is not exposed in SPIR-V but we now expose it in NIR. Signed-off-by: Timur Kristóf <timur.kristof@gmail.com> Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11072>
2021-06-09 11:00:22 +02:00
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
case nir_intrinsic_load_local_invocation_id:
if (shader->info.workgroup_size_variable)
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
*result = get_max_workgroup_invocations(shader) - 1u;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
else
*result = shader->info.workgroup_size[q.scalar.comp] - 1u;
break;
case nir_intrinsic_load_workgroup_id:
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
*result = get_max_workgroup_count(shader, q.scalar.comp) - 1u;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
case nir_intrinsic_load_num_workgroups:
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
*result = get_max_workgroup_count(shader, q.scalar.comp);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
case nir_intrinsic_load_global_invocation_id:
if (shader->info.workgroup_size_variable) {
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
*result = mul_clamp(get_max_workgroup_invocations(shader),
get_max_workgroup_count(shader, q.scalar.comp)) -
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
1u;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
} else {
*result = (shader->info.workgroup_size[q.scalar.comp] *
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
get_max_workgroup_count(shader, q.scalar.comp)) -
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
1u;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
}
break;
case nir_intrinsic_load_invocation_id:
if (shader->info.stage == MESA_SHADER_TESS_CTRL)
*result = shader->info.tess.tcs_vertices_out
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
? (shader->info.tess.tcs_vertices_out - 1)
: 511; /* Generous maximum output patch size of 512 */
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
case nir_intrinsic_load_subgroup_invocation:
case nir_intrinsic_first_invocation:
nir/uub: use shader_info subgroup size Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:09:45 +02:00
*result = shader->info.max_subgroup_size - 1;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
case nir_intrinsic_mbcnt_amd: {
if (!q.head.pushed_queries) {
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
push_scalar_query(state, nir_get_scalar(intrin->src[1].ssa, 0));
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
return;
} else {
nir/uub: use shader_info subgroup size Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:09:45 +02:00
uint32_t src0 = shader->info.max_subgroup_size - 1;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
uint32_t src1 = src[0];
if (src0 + src1 >= src0) /* check overflow */
*result = src0 + src1;
}
break;
}
case nir_intrinsic_load_subgroup_size:
nir/uub: use shader_info subgroup size Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:09:45 +02:00
if (shader->info.api_subgroup_size)
*result = shader->info.api_subgroup_size;
else
*result = shader->info.max_subgroup_size;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
case nir_intrinsic_load_subgroup_id:
case nir_intrinsic_load_num_subgroups: {
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
uint32_t workgroup_size = get_max_workgroup_invocations(shader);
all: rename gl_shader_stage_uses_workgroup to mesa_shader_stage_uses_workgroup Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com> Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Yonggang Luo <luoyonggang@gmail.com> Acked-by: Marek Olšák <marek.olsak@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36569>
2025-08-05 16:50:43 +08:00
if (mesa_shader_stage_uses_workgroup(shader->info.stage) &&
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
!shader->info.workgroup_size_variable) {
workgroup_size = shader->info.workgroup_size[0] *
shader->info.workgroup_size[1] *
shader->info.workgroup_size[2];
}
nir/uub: use shader_info subgroup size Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:09:45 +02:00
*result = DIV_ROUND_UP(workgroup_size, shader->info.min_subgroup_size);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
if (intrin->intrinsic == nir_intrinsic_load_subgroup_id)
(*result)--;
break;
}
case nir_intrinsic_reduce:
case nir_intrinsic_inclusive_scan:
case nir_intrinsic_exclusive_scan: {
nir_op op = nir_intrinsic_reduction_op(intrin);
nir/uub: fix exclusive scans Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Cc: mesa-stable Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36874>
2025-08-20 12:29:03 +02:00
nir/uub: handle more reduction ops Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36874>
2025-08-20 12:43:07 +02:00
switch (op) {
case nir_op_umin:
case nir_op_umax:
case nir_op_imax:
case nir_op_imin:
case nir_op_iand:
case nir_op_ior:
case nir_op_ixor:
case nir_op_iadd:
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
if (!q.head.pushed_queries) {
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
push_scalar_query(state, nir_get_scalar(intrin->src[0].ssa, q.scalar.comp));
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
return;
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
}
nir/uub: handle more reduction ops Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36874>
2025-08-20 12:43:07 +02:00
break;
default:
return;
}
unsigned bit_size = q.scalar.def->bit_size;
bool exclusive = intrin->intrinsic == nir_intrinsic_exclusive_scan;
switch (op) {
case nir_op_umin:
case nir_op_umax:
case nir_op_imax:
case nir_op_imin:
case nir_op_iand:
*result = src[0];
break;
case nir_op_ior:
case nir_op_ixor:
*result = bitmask(util_last_bit64(src[0]));
break;
case nir_op_iadd:
nir/uub: use shader_info subgroup size Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:09:45 +02:00
*result = MIN2(*result, (uint64_t)src[0] * (shader->info.max_subgroup_size - exclusive));
nir/uub: handle more reduction ops Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36874>
2025-08-20 12:43:07 +02:00
break;
default:
UNREACHABLE("unhandled op");
}
if (exclusive) {
uint32_t identity = nir_const_value_as_uint(nir_alu_binop_identity(op, bit_size), bit_size);
*result = MAX2(*result, identity);
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
}
nir/uub: handle more reduction ops Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36874>
2025-08-20 12:43:07 +02:00
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
}
case nir_intrinsic_read_first_invocation:
case nir_intrinsic_read_invocation:
case nir_intrinsic_shuffle:
case nir_intrinsic_shuffle_xor:
case nir_intrinsic_shuffle_up:
case nir_intrinsic_shuffle_down:
case nir_intrinsic_quad_broadcast:
case nir_intrinsic_quad_swap_horizontal:
case nir_intrinsic_quad_swap_vertical:
case nir_intrinsic_quad_swap_diagonal:
case nir_intrinsic_quad_swizzle_amd:
case nir_intrinsic_masked_swizzle_amd:
if (!q.head.pushed_queries) {
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
push_scalar_query(state, nir_get_scalar(intrin->src[0].ssa, q.scalar.comp));
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
return;
} else {
*result = src[0];
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
case nir_intrinsic_write_invocation_amd:
if (!q.head.pushed_queries) {
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
push_scalar_query(state, nir_get_scalar(intrin->src[0].ssa, q.scalar.comp));
push_scalar_query(state, nir_get_scalar(intrin->src[1].ssa, q.scalar.comp));
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
return;
} else {
*result = MAX2(src[0], src[1]);
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
case nir_intrinsic_load_tess_rel_patch_id_amd:
case nir_intrinsic_load_tcs_num_patches_amd:
/* Very generous maximum: TCS/TES executed by largest possible workgroup */
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
*result = get_max_workgroup_invocations(shader) / MAX2(shader->info.tess.tcs_vertices_out, 1u);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
case nir_intrinsic_load_typed_buffer_amd: {
const enum pipe_format format = nir_intrinsic_format(intrin);
if (format == PIPE_FORMAT_NONE)
nir: Add tessellation related AMD-specific intrinsics. Signed-off-by: Timur Kristóf <timur.kristof@gmail.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-02-15 22:01:02 +01:00
break;
nir: Add load_typed_buffer_amd intrinsic. This new intrinsic maps to the MTBUF instruction format on AMD GPUs and represents a typed buffer load in NIR. Also add an unsigned upper bound for the new intrinsic. Code for that ported from aco_instruction_selection_setup. Signed-off-by: Timur Kristóf <timur.kristof@gmail.com> Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com> Reviewed-by: Marek Olšák <marek.olsak@amd.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16805>
2023-02-02 10:47:58 +01:00
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
const struct util_format_description *desc = util_format_description(format);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
if (desc->channel[q.scalar.comp].type != UTIL_FORMAT_TYPE_UNSIGNED)
nir: Add upper bound for AMD shader arg intrinsics. Signed-off-by: Timur Kristóf <timur.kristof@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13155>
2022-03-23 18:44:57 +01:00
break;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
if (desc->channel[q.scalar.comp].normalized) {
*result = fui(1.0);
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
break;
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
const uint32_t chan_max = u_uintN_max(desc->channel[q.scalar.comp].size);
*result = desc->channel[q.scalar.comp].pure_integer ? chan_max : fui(chan_max);
break;
}
ac: lower load_workgroup_ids for ACO in NIR Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39638>
2026-01-31 14:54:30 -05:00
case nir_intrinsic_load_ttmp_register_amd:
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
case nir_intrinsic_load_scalar_arg_amd:
case nir_intrinsic_load_vector_arg_amd: {
uint32_t upper_bound = nir_intrinsic_arg_upper_bound_u32_amd(intrin);
if (upper_bound)
*result = upper_bound;
break;
}
nir/uub: Use an optional max_samples from drivers for sample counts. This triggers some unrolling in Fallout 4, GTAV, and Rocky Planet in my shader-db. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38585>
2025-11-24 16:28:44 -08:00
case nir_intrinsic_image_samples:
if (state->shader->options->max_samples > 0)
*result = state->shader->options->max_samples;
break;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
default:
break;
}
}
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
static void
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
get_alu_uub(struct analysis_state *state, struct scalar_query q, uint32_t *result, const uint32_t *src)
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
{
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_op op = nir_scalar_alu_op(q.scalar);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
/* Early exit for unsupported ALU opcodes. */
switch (op) {
case nir_op_umin:
case nir_op_imin:
case nir_op_imax:
case nir_op_umax:
case nir_op_iand:
case nir_op_ior:
case nir_op_ixor:
case nir_op_ishl:
case nir_op_imul:
case nir_op_ushr:
case nir_op_ishr:
case nir_op_iadd:
case nir_op_umod:
case nir_op_udiv:
case nir_op_bcsel:
case nir_op_b32csel:
case nir_op_ubfe:
nir/range_analysis: Handle bfi and bitfield_select in get_alu_uub I noticed some things related to this while implementing support for bitfield_select / BFN in BRW. shader-db: Lunar Lake total instructions in shared programs: 17183140 -> 17183128 (<.01%) instructions in affected programs: 3830 -> 3818 (-0.31%) helped: 6 / HURT: 0 total cycles in shared programs: 889936934 -> 889936056 (<.01%) cycles in affected programs: 253758 -> 252880 (-0.35%) helped: 4 / HURT: 2 No shader-db changes on any other Intel platform. fossil-db: Lunar Lake Totals: Instrs: 233285343 -> 233284796 (-0.00%); split: -0.00%, +0.00% Cycle count: 32756777978 -> 32756399804 (-0.00%); split: -0.00%, +0.00% Max live registers: 71738646 -> 71738626 (-0.00%) Non SSA regs after NIR: 67837900 -> 67837902 (+0.00%) Totals from 177 (0.02% of 790723) affected shaders: Instrs: 389849 -> 389302 (-0.14%); split: -0.14%, +0.00% Cycle count: 356341872 -> 355963698 (-0.11%); split: -0.11%, +0.01% Max live registers: 39364 -> 39344 (-0.05%) Non SSA regs after NIR: 70453 -> 70455 (+0.00%) Meteor Lake, DG2, and Ice Lake had similar results. (Meteor Lake shown) Totals: Instrs: 264095611 -> 264095358 (-0.00%) Cycle count: 26555705299 -> 26554303407 (-0.01%); split: -0.01%, +0.00% Fill count: 613233 -> 613231 (-0.00%) Totals from 123 (0.01% of 905547) affected shaders: Instrs: 334830 -> 334577 (-0.08%) Cycle count: 326531667 -> 325129775 (-0.43%); split: -0.65%, +0.22% Fill count: 4145 -> 4143 (-0.05%) Tiger Lake and Skylake had similar results. (Tiger Lake shown) Totals: Instrs: 269733849 -> 269733590 (-0.00%) Cycle count: 25240548036 -> 25241435039 (+0.00%); split: -0.00%, +0.01% Totals from 123 (0.01% of 903812) affected shaders: Instrs: 338617 -> 338358 (-0.08%) Cycle count: 326605644 -> 327492647 (+0.27%); split: -0.13%, +0.40% Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37186>
2025-06-16 11:38:53 -07:00
case nir_op_bfi:
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
case nir_op_bfm:
nir/range_analysis: Handle bfi and bitfield_select in get_alu_uub I noticed some things related to this while implementing support for bitfield_select / BFN in BRW. shader-db: Lunar Lake total instructions in shared programs: 17183140 -> 17183128 (<.01%) instructions in affected programs: 3830 -> 3818 (-0.31%) helped: 6 / HURT: 0 total cycles in shared programs: 889936934 -> 889936056 (<.01%) cycles in affected programs: 253758 -> 252880 (-0.35%) helped: 4 / HURT: 2 No shader-db changes on any other Intel platform. fossil-db: Lunar Lake Totals: Instrs: 233285343 -> 233284796 (-0.00%); split: -0.00%, +0.00% Cycle count: 32756777978 -> 32756399804 (-0.00%); split: -0.00%, +0.00% Max live registers: 71738646 -> 71738626 (-0.00%) Non SSA regs after NIR: 67837900 -> 67837902 (+0.00%) Totals from 177 (0.02% of 790723) affected shaders: Instrs: 389849 -> 389302 (-0.14%); split: -0.14%, +0.00% Cycle count: 356341872 -> 355963698 (-0.11%); split: -0.11%, +0.01% Max live registers: 39364 -> 39344 (-0.05%) Non SSA regs after NIR: 70453 -> 70455 (+0.00%) Meteor Lake, DG2, and Ice Lake had similar results. (Meteor Lake shown) Totals: Instrs: 264095611 -> 264095358 (-0.00%) Cycle count: 26555705299 -> 26554303407 (-0.01%); split: -0.01%, +0.00% Fill count: 613233 -> 613231 (-0.00%) Totals from 123 (0.01% of 905547) affected shaders: Instrs: 334830 -> 334577 (-0.08%) Cycle count: 326531667 -> 325129775 (-0.43%); split: -0.65%, +0.22% Fill count: 4145 -> 4143 (-0.05%) Tiger Lake and Skylake had similar results. (Tiger Lake shown) Totals: Instrs: 269733849 -> 269733590 (-0.00%) Cycle count: 25240548036 -> 25241435039 (+0.00%); split: -0.00%, +0.01% Totals from 123 (0.01% of 903812) affected shaders: Instrs: 338617 -> 338358 (-0.08%) Cycle count: 326605644 -> 327492647 (+0.27%); split: -0.13%, +0.40% Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37186>
2025-06-16 11:38:53 -07:00
case nir_op_bitfield_select:
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
case nir_op_extract_u8:
case nir_op_extract_i8:
case nir_op_extract_u16:
case nir_op_extract_i16:
case nir_op_b2i8:
case nir_op_b2i16:
case nir_op_b2i32:
break;
case nir_op_u2u1:
case nir_op_u2u8:
case nir_op_u2u16:
case nir_op_u2u32:
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
if (nir_scalar_chase_alu_src(q.scalar, 0).def->bit_size > 32) {
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
/* If src is >32 bits, return max */
return;
}
break;
nir: add unsigned upper bound support for fsat Foz-DB Navi21: Totals from 89 (0.11% of 79395) affected shaders: Instrs: 97018 -> 96995 (-0.02%) CodeSize: 492996 -> 492488 (-0.10%) Latency: 504649 -> 504555 (-0.02%) InvThroughput: 121968 -> 121875 (-0.08%) VALU: 67427 -> 67404 (-0.03%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32565>
2024-12-09 22:47:51 +01:00
case nir_op_fsat:
nir/uub: properly limit float support to 32bit Cc: mesa-stable Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32565>
2024-12-09 22:57:58 +01:00
case nir_op_fmul:
case nir_op_fmulz:
case nir_op_f2u32:
nir: add unsigned upper bound support for f2i32 Foz-DB Navi21: Totals from 649 (0.82% of 79395) affected shaders: CodeSize: 2330592 -> 2314112 (-0.71%) Latency: 2068161 -> 2053370 (-0.72%) InvThroughput: 346583 -> 329425 (-4.95%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32565>
2024-12-09 22:28:09 +01:00
case nir_op_f2i32:
nir/uub: properly limit float support to 32bit Cc: mesa-stable Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32565>
2024-12-09 22:57:58 +01:00
if (nir_scalar_chase_alu_src(q.scalar, 0).def->bit_size != 32) {
/* Only 32bit floats support for now, return max */
return;
}
break;
nir/uub: handle bit_count Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36874>
2025-08-20 12:49:57 +02:00
case nir_op_bit_count:
if (nir_scalar_chase_alu_src(q.scalar, 0).def->bit_size > 32) {
*result = nir_scalar_chase_alu_src(q.scalar, 0).def->bit_size;
return;
}
break;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
default:
return;
}
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
if (!q.head.pushed_queries) {
for (unsigned i = 0; i < nir_op_infos[op].num_inputs; i++)
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
push_scalar_query(state, nir_scalar_chase_alu_src(q.scalar, i));
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
return;
}
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
uint32_t max = bitmask(q.scalar.def->bit_size);
switch (op) {
case nir_op_umin:
*result = src[0] < src[1] ? src[0] : src[1];
break;
case nir_op_imin:
case nir_op_imax:
case nir_op_umax:
*result = src[0] > src[1] ? src[0] : src[1];
break;
nir/uub: improve iand with constant sources fossil-db (navi21): Totals from 9 (0.01% of 79653) affected shaders: Instrs: 11878 -> 11868 (-0.08%) CodeSize: 61572 -> 61508 (-0.10%) Latency: 44585 -> 44581 (-0.01%); split: -0.02%, +0.01% InvThroughput: 9697 -> 9660 (-0.38%) VALU: 8889 -> 8876 (-0.15%) SALU: 1339 -> 1342 (+0.22%) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35514>
2025-06-13 11:08:47 +01:00
case nir_op_iand: {
nir_scalar src0_scalar = nir_scalar_chase_alu_src(q.scalar, 0);
nir_scalar src1_scalar = nir_scalar_chase_alu_src(q.scalar, 1);
if (nir_scalar_is_const(src0_scalar))
*result = bitmask(util_last_bit64(src[1])) & nir_scalar_as_uint(src0_scalar);
else if (nir_scalar_is_const(src1_scalar))
*result = bitmask(util_last_bit64(src[0])) & nir_scalar_as_uint(src1_scalar);
else
*result = bitmask(util_last_bit64(src[0])) & bitmask(util_last_bit64(src[1]));
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
nir/uub: improve iand with constant sources fossil-db (navi21): Totals from 9 (0.01% of 79653) affected shaders: Instrs: 11878 -> 11868 (-0.08%) CodeSize: 61572 -> 61508 (-0.10%) Latency: 44585 -> 44581 (-0.01%); split: -0.02%, +0.01% InvThroughput: 9697 -> 9660 (-0.38%) VALU: 8889 -> 8876 (-0.15%) SALU: 1339 -> 1342 (+0.22%) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35514>
2025-06-13 11:08:47 +01:00
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
case nir_op_ior:
nir/uub: improve ior/ixor with constant sources No fossil-db changes. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35514>
2025-06-13 11:13:00 +01:00
case nir_op_ixor: {
nir_scalar src0_scalar = nir_scalar_chase_alu_src(q.scalar, 0);
nir_scalar src1_scalar = nir_scalar_chase_alu_src(q.scalar, 1);
if (nir_scalar_is_const(src0_scalar))
*result = bitmask(util_last_bit64(src[1])) | nir_scalar_as_uint(src0_scalar);
else if (nir_scalar_is_const(src1_scalar))
*result = bitmask(util_last_bit64(src[0])) | nir_scalar_as_uint(src1_scalar);
else
*result = bitmask(util_last_bit64(src[0])) | bitmask(util_last_bit64(src[1]));
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
nir/uub: improve ior/ixor with constant sources No fossil-db changes. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35514>
2025-06-13 11:13:00 +01:00
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
case nir_op_ishl: {
uint32_t src1 = MIN2(src[1], q.scalar.def->bit_size - 1u);
if (util_last_bit64(src[0]) + src1 <= q.scalar.def->bit_size) /* check overflow */
*result = src[0] << src1;
nir/uub: fix 8/16-bit overflow Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Backport-to: 25.1 Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13552 Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13553 Tested-by: @LingMan Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36372>
2025-07-25 10:28:55 +01:00
*result = MIN2(*result, max);
nir/uub: improve ishl/imul with constant sources fossil-db (navi21): Totals from 1 (0.00% of 79653) affected shaders: Instrs: 1339 -> 1338 (-0.07%) CodeSize: 7244 -> 7240 (-0.06%) Latency: 19827 -> 19822 (-0.03%) InvThroughput: 9913 -> 9911 (-0.02%) SALU: 419 -> 418 (-0.24%) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35514>
2025-06-13 11:06:49 +01:00
nir_scalar src1_scalar = nir_scalar_chase_alu_src(q.scalar, 1);
if (nir_scalar_is_const(src1_scalar)) {
uint32_t const_val = 1u << (nir_scalar_as_uint(src1_scalar) & (q.scalar.def->bit_size - 1u));
*result = MIN2(*result, max / const_val * const_val);
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
}
nir/uub: improve ishl/imul with constant sources fossil-db (navi21): Totals from 1 (0.00% of 79653) affected shaders: Instrs: 1339 -> 1338 (-0.07%) CodeSize: 7244 -> 7240 (-0.06%) Latency: 19827 -> 19822 (-0.03%) InvThroughput: 9913 -> 9911 (-0.02%) SALU: 419 -> 418 (-0.24%) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35514>
2025-06-13 11:06:49 +01:00
case nir_op_imul: {
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
if (src[0] == 0 || (src[0] * src[1]) / src[0] == src[1]) /* check overflow */
*result = src[0] * src[1];
nir/uub: fix 8/16-bit overflow Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Backport-to: 25.1 Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13552 Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13553 Tested-by: @LingMan Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36372>
2025-07-25 10:28:55 +01:00
*result = MIN2(*result, max);
nir/uub: improve ishl/imul with constant sources fossil-db (navi21): Totals from 1 (0.00% of 79653) affected shaders: Instrs: 1339 -> 1338 (-0.07%) CodeSize: 7244 -> 7240 (-0.06%) Latency: 19827 -> 19822 (-0.03%) InvThroughput: 9913 -> 9911 (-0.02%) SALU: 419 -> 418 (-0.24%) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35514>
2025-06-13 11:06:49 +01:00
nir_scalar src0_scalar = nir_scalar_chase_alu_src(q.scalar, 0);
nir_scalar src1_scalar = nir_scalar_chase_alu_src(q.scalar, 1);
if (nir_scalar_is_const(src0_scalar)) {
uint32_t const_val = nir_scalar_as_uint(src0_scalar);
nir/uub: guard against division by 0 Fixes: 8ee54400733 ("nir/uub: improve ishl/imul with constant sources") Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com> Reviewed-by: Karol Herbst <kherbst@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36805>
2025-08-17 21:53:26 +02:00
*result = const_val ? MIN2(*result, max / const_val * const_val) : 0;
nir/uub: improve ishl/imul with constant sources fossil-db (navi21): Totals from 1 (0.00% of 79653) affected shaders: Instrs: 1339 -> 1338 (-0.07%) CodeSize: 7244 -> 7240 (-0.06%) Latency: 19827 -> 19822 (-0.03%) InvThroughput: 9913 -> 9911 (-0.02%) SALU: 419 -> 418 (-0.24%) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35514>
2025-06-13 11:06:49 +01:00
} else if (nir_scalar_is_const(src1_scalar)) {
uint32_t const_val = nir_scalar_as_uint(src1_scalar);
nir/uub: guard against division by 0 Fixes: 8ee54400733 ("nir/uub: improve ishl/imul with constant sources") Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com> Reviewed-by: Karol Herbst <kherbst@redhat.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36805>
2025-08-17 21:53:26 +02:00
*result = const_val ? MIN2(*result, max / const_val * const_val) : 0;
nir/uub: improve ishl/imul with constant sources fossil-db (navi21): Totals from 1 (0.00% of 79653) affected shaders: Instrs: 1339 -> 1338 (-0.07%) CodeSize: 7244 -> 7240 (-0.06%) Latency: 19827 -> 19822 (-0.03%) InvThroughput: 9913 -> 9911 (-0.02%) SALU: 419 -> 418 (-0.24%) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35514>
2025-06-13 11:06:49 +01:00
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
nir/uub: improve ishl/imul with constant sources fossil-db (navi21): Totals from 1 (0.00% of 79653) affected shaders: Instrs: 1339 -> 1338 (-0.07%) CodeSize: 7244 -> 7240 (-0.06%) Latency: 19827 -> 19822 (-0.03%) InvThroughput: 9913 -> 9911 (-0.02%) SALU: 419 -> 418 (-0.24%) Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35514>
2025-06-13 11:06:49 +01:00
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
case nir_op_ushr: {
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_scalar src1_scalar = nir_scalar_chase_alu_src(q.scalar, 1);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
uint32_t mask = q.scalar.def->bit_size - 1u;
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
if (nir_scalar_is_const(src1_scalar))
*result = src[0] >> (nir_scalar_as_uint(src1_scalar) & mask);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
else
*result = src[0];
break;
}
case nir_op_ishr: {
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_scalar src1_scalar = nir_scalar_chase_alu_src(q.scalar, 1);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
uint32_t mask = q.scalar.def->bit_size - 1u;
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
if (src[0] <= 2147483647 && nir_scalar_is_const(src1_scalar))
*result = src[0] >> (nir_scalar_as_uint(src1_scalar) & mask);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
else
*result = src[0];
break;
}
case nir_op_iadd:
if (src[0] + src[1] >= src[0]) /* check overflow */
*result = src[0] + src[1];
nir/uub: fix 8/16-bit overflow Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Backport-to: 25.1 Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13552 Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13553 Tested-by: @LingMan Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36372>
2025-07-25 10:28:55 +01:00
*result = MIN2(*result, max);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
case nir_op_umod:
*result = src[1] ? src[1] - 1 : 0;
break;
case nir_op_udiv: {
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_scalar src1_scalar = nir_scalar_chase_alu_src(q.scalar, 1);
if (nir_scalar_is_const(src1_scalar))
*result = nir_scalar_as_uint(src1_scalar)
? src[0] / nir_scalar_as_uint(src1_scalar)
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
: 0;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
else
*result = src[0];
break;
}
case nir_op_bcsel:
case nir_op_b32csel:
*result = src[1] > src[2] ? src[1] : src[2];
break;
case nir_op_ubfe:
*result = bitmask(MIN2(src[2], q.scalar.def->bit_size));
break;
case nir_op_bfm: {
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_scalar src1_scalar = nir_scalar_chase_alu_src(q.scalar, 1);
if (nir_scalar_is_const(src1_scalar)) {
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
uint32_t src0 = MIN2(src[0], 31);
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
uint32_t src1 = nir_scalar_as_uint(src1_scalar) & 0x1fu;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
*result = bitmask(src0) << src1;
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
} else {
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
uint32_t src0 = MIN2(src[0], 31);
uint32_t src1 = MIN2(src[1], 31);
*result = bitmask(MIN2(src0 + src1, 32));
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
}
nir/range_analysis: Handle bfi and bitfield_select in get_alu_uub I noticed some things related to this while implementing support for bitfield_select / BFN in BRW. shader-db: Lunar Lake total instructions in shared programs: 17183140 -> 17183128 (<.01%) instructions in affected programs: 3830 -> 3818 (-0.31%) helped: 6 / HURT: 0 total cycles in shared programs: 889936934 -> 889936056 (<.01%) cycles in affected programs: 253758 -> 252880 (-0.35%) helped: 4 / HURT: 2 No shader-db changes on any other Intel platform. fossil-db: Lunar Lake Totals: Instrs: 233285343 -> 233284796 (-0.00%); split: -0.00%, +0.00% Cycle count: 32756777978 -> 32756399804 (-0.00%); split: -0.00%, +0.00% Max live registers: 71738646 -> 71738626 (-0.00%) Non SSA regs after NIR: 67837900 -> 67837902 (+0.00%) Totals from 177 (0.02% of 790723) affected shaders: Instrs: 389849 -> 389302 (-0.14%); split: -0.14%, +0.00% Cycle count: 356341872 -> 355963698 (-0.11%); split: -0.11%, +0.01% Max live registers: 39364 -> 39344 (-0.05%) Non SSA regs after NIR: 70453 -> 70455 (+0.00%) Meteor Lake, DG2, and Ice Lake had similar results. (Meteor Lake shown) Totals: Instrs: 264095611 -> 264095358 (-0.00%) Cycle count: 26555705299 -> 26554303407 (-0.01%); split: -0.01%, +0.00% Fill count: 613233 -> 613231 (-0.00%) Totals from 123 (0.01% of 905547) affected shaders: Instrs: 334830 -> 334577 (-0.08%) Cycle count: 326531667 -> 325129775 (-0.43%); split: -0.65%, +0.22% Fill count: 4145 -> 4143 (-0.05%) Tiger Lake and Skylake had similar results. (Tiger Lake shown) Totals: Instrs: 269733849 -> 269733590 (-0.00%) Cycle count: 25240548036 -> 25241435039 (+0.00%); split: -0.00%, +0.01% Totals from 123 (0.01% of 903812) affected shaders: Instrs: 338617 -> 338358 (-0.08%) Cycle count: 326605644 -> 327492647 (+0.27%); split: -0.13%, +0.40% Reviewed-by: Matt Turner <mattst88@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37186>
2025-06-16 11:38:53 -07:00
case nir_op_bfi: {
nir_scalar src0_scalar = nir_scalar_chase_alu_src(q.scalar, 0);
const uint64_t s1 = bitmask(util_last_bit64(src[1]));
const uint64_t s2 = bitmask(util_last_bit64(src[2]));
if (nir_scalar_is_const(src0_scalar)) {
const uint64_t s0 = nir_scalar_as_uint(src0_scalar);
/* This case should be eliminated by opt_algebraic. */
if (s0 == 0) {
*result = s2;
} else {
const int x = ffsll(s0) - 1;
*result = (s0 & (s1 << x)) | (~s0 & s2);
}
} else {
const uint64_t s0 = bitmask(util_last_bit64(src[0]));
/* Due to the unpredictable shift, the true maximum value of (s0 &
* (s1 << x)) cannot be known. However, it cannot be larger than
* s0.
*
* inot doesn't work in get_alu_uub. It is known that (~s0 & s2)
* cannot be larger than s2, so just use s2 as a loose upper bound.
*/
*result = s0 | s2;
}
break;
}
case nir_op_bitfield_select: {
nir_scalar src0_scalar = nir_scalar_chase_alu_src(q.scalar, 0);
const uint64_t s1 = bitmask(util_last_bit64(src[1]));
const uint64_t s2 = bitmask(util_last_bit64(src[2]));
if (nir_scalar_is_const(src0_scalar)) {
const uint64_t s0 = nir_scalar_as_uint(src0_scalar);
*result = (s0 & s1) | (~s0 & s2);
} else {
const uint64_t s0 = bitmask(util_last_bit64(src[0]));
/* inot doesn't work in get_alu_uub. It is known that (~s0 & s2)
* cannot be larger than s2, so just use s2 as a loose upper bound.
*/
*result = (s0 & s1) | s2;
}
break;
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
/* limited floating-point support for f2u32(fmul(load_input(), <constant>)) */
nir: add unsigned upper bound support for f2i32 Foz-DB Navi21: Totals from 649 (0.82% of 79395) affected shaders: CodeSize: 2330592 -> 2314112 (-0.71%) Latency: 2068161 -> 2053370 (-0.72%) InvThroughput: 346583 -> 329425 (-4.95%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32565>
2024-12-09 22:28:09 +01:00
case nir_op_f2i32:
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
case nir_op_f2u32:
/* infinity/NaN starts at 0x7f800000u, negative numbers at 0x80000000 */
if (src[0] < 0x7f800000u) {
float val;
memcpy(&val, &src[0], 4);
*result = (uint32_t)val;
}
break;
case nir_op_fmul:
case nir_op_fmulz:
/* infinity/NaN starts at 0x7f800000u, negative numbers at 0x80000000 */
if (src[0] < 0x7f800000u && src[1] < 0x7f800000u) {
float src0_f, src1_f;
memcpy(&src0_f, &src[0], 4);
memcpy(&src1_f, &src[1], 4);
/* not a proper rounding-up multiplication, but should be good enough */
float max_f = ceilf(src0_f) * ceilf(src1_f);
memcpy(result, &max_f, 4);
}
break;
nir: add unsigned upper bound support for fsat Foz-DB Navi21: Totals from 89 (0.11% of 79395) affected shaders: Instrs: 97018 -> 96995 (-0.02%) CodeSize: 492996 -> 492488 (-0.10%) Latency: 504649 -> 504555 (-0.02%) InvThroughput: 121968 -> 121875 (-0.08%) VALU: 67427 -> 67404 (-0.03%) Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32565>
2024-12-09 22:47:51 +01:00
case nir_op_fsat:
*result = 0x3f800000u;
break;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
case nir_op_u2u1:
case nir_op_u2u8:
case nir_op_u2u16:
case nir_op_u2u32:
*result = MIN2(src[0], max);
break;
case nir_op_b2i8:
case nir_op_b2i16:
case nir_op_b2i32:
*result = 1;
break;
nir: add msad_4x8 Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26907>
2023-11-17 11:21:19 +00:00
case nir_op_msad_4x8:
*result = MIN2((uint64_t)src[2] + 4 * 255, UINT32_MAX);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
break;
case nir_op_extract_u8:
*result = MIN2(src[0], UINT8_MAX);
break;
case nir_op_extract_i8:
*result = (src[0] >= 0x80) ? max : MIN2(src[0], INT8_MAX);
break;
case nir_op_extract_u16:
*result = MIN2(src[0], UINT16_MAX);
break;
case nir_op_extract_i16:
*result = (src[0] >= 0x8000) ? max : MIN2(src[0], INT16_MAX);
break;
nir/uub: handle bit_count Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36874>
2025-08-20 12:49:57 +02:00
case nir_op_bit_count:
*result = util_last_bit64(src[0]);
break;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
default:
break;
}
}
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
nir/uub: Use an optional max_samples from drivers for sample counts. This triggers some unrolling in Fallout 4, GTAV, and Rocky Planet in my shader-db. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38585>
2025-11-24 16:28:44 -08:00
static void
get_tex_uub(struct analysis_state *state, struct scalar_query q, uint32_t *result, const uint32_t *src)
{
nir_tex_instr *tex = nir_scalar_as_tex(q.scalar);
if (tex->op == nir_texop_texture_samples && state->shader->options->max_samples > 0)
*result = state->shader->options->max_samples;
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
static void
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
get_phi_uub(struct analysis_state *state, struct scalar_query q, uint32_t *result, const uint32_t *src)
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
{
treewide: use nir_def_as_* Via Coccinelle patch: @@ expression definition; @@ -nir_instr_as_alu(definition->parent_instr) +nir_def_as_alu(definition) @@ expression definition; @@ -nir_instr_as_intrinsic(definition->parent_instr) +nir_def_as_intrinsic(definition) @@ expression definition; @@ -nir_instr_as_phi(definition->parent_instr) +nir_def_as_phi(definition) @@ expression definition; @@ -nir_instr_as_load_const(definition->parent_instr) +nir_def_as_load_const(definition) @@ expression definition; @@ -nir_instr_as_deref(definition->parent_instr) +nir_def_as_deref(definition) @@ expression definition; @@ -nir_instr_as_tex(definition->parent_instr) +nir_def_as_tex(definition) Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Emma Anholt <emma@anholt.net> Reviewed-by: Marek Olšák <maraeo@gmail.com> Acked-by: Karol Herbst <kherbst@redhat.com> Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36489>
2025-07-31 09:49:36 -04:00
nir_phi_instr *phi = nir_def_as_phi(q.scalar.def);
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
if (exec_list_is_empty(&phi->srcs))
return;
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
if (q.head.pushed_queries) {
*result = src[0];
for (unsigned i = 1; i < q.head.pushed_queries; i++)
*result = MAX2(*result, src[i]);
return;
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
}
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
nir_cf_node *prev = nir_cf_node_prev(&phi->instr.block->cf_node);
if (!prev || prev->type == nir_cf_node_block) {
/* Resolve cycles by inserting max into range_ht. */
uint32_t max = bitmask(q.scalar.def->bit_size);
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
scalar_insert(state->range_ht, get_scalar_key(&q.head), max);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
struct set *visited = _mesa_pointer_set_create(NULL);
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_scalar *defs = alloca(sizeof(nir_scalar) * 64);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
unsigned def_count = search_phi_bcsel(q.scalar, defs, 64, visited);
_mesa_set_destroy(visited, NULL);
for (unsigned i = 0; i < def_count; i++)
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
push_scalar_query(state, defs[i]);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
} else {
nir_foreach_phi_src(src, phi)
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
push_scalar_query(state, nir_get_scalar(src->src.ssa, q.scalar.comp));
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
}
}
static void
process_uub_query(struct analysis_state *state, struct analysis_query *aq, uint32_t *result,
const uint32_t *src)
{
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
struct scalar_query q = *(struct scalar_query *)aq;
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
*result = bitmask(q.scalar.def->bit_size);
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
if (nir_scalar_is_const(q.scalar))
*result = nir_scalar_as_uint(q.scalar);
nir: add nir_scalar intrinsic helpers Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24656>
2023-08-13 00:03:03 +02:00
else if (nir_scalar_is_intrinsic(q.scalar))
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
get_intrinsic_uub(state, q, result, src);
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
else if (nir_scalar_is_alu(q.scalar))
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
get_alu_uub(state, q, result, src);
nir/uub: Use an optional max_samples from drivers for sample counts. This triggers some unrolling in Fallout 4, GTAV, and Rocky Planet in my shader-db. Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38585>
2025-11-24 16:28:44 -08:00
else if (nir_def_instr_type(q.scalar.def) == nir_instr_type_tex)
get_tex_uub(state, q, result, src);
treewide: add & use parent instr helpers We add a bunch of new helpers to avoid the need to touch >parent_instr, including the full set of: * nir_def_is_* * nir_def_as_*_or_null * nir_def_as_* [assumes the right instr type] * nir_src_is_* * nir_src_as_* * nir_scalar_is_* * nir_scalar_as_* Plus nir_def_instr() where there's no more suitable helper. Also an existing helper is renamed to unify all the names, while we're churning the tree: * nir_src_as_alu_instr -> nir_src_as_alu ..and then we port the tree to use the helpers as much as possible, using nir_def_instr() where that does not work. Acked-by: Marek Olšák <maraeo@gmail.com> --- To eliminate nir_def::parent_instr we need to churn the tree anyway, so I'm taking this opportunity to clean up a lot of NIR patterns. Co-authored-by: Konstantin Seurer <konstantin.seurer@gmail.com> Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38313>
2025-11-07 21:38:36 +08:00
else if (nir_def_instr_type(q.scalar.def) == nir_instr_type_phi)
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
get_phi_uub(state, q, result, src);
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
}
nir: Apply a maximum stack depth to avoid stack overflows. A stackless (or at least using allocated memory for stack) version might be nice but for now this works around some games compiling large shaders and hitting stack overflows. CC: mesa-stable Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21231>
2023-01-10 16:12:34 +01:00
uint32_t
nir_unsigned_upper_bound(nir_shader *shader, struct hash_table *range_ht,
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
nir_scalar scalar)
nir: clang-format src/compiler/nir/*.[ch] Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24382>
2023-08-08 12:00:35 -05:00
{
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
struct scalar_query query_alloc[16];
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
uint32_t result_alloc[16];
struct analysis_state state;
state.shader = shader;
state.range_ht = range_ht;
util_dynarray_init_from_stack(&state.query_stack, query_alloc, sizeof(query_alloc));
util_dynarray_init_from_stack(&state.result_stack, result_alloc, sizeof(result_alloc));
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
state.query_size = sizeof(struct scalar_query);
state.get_key = &get_scalar_key;
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
state.lookup = &scalar_lookup,
state.insert = &scalar_insert,
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
state.process_query = &process_uub_query;
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
push_scalar_query(&state, scalar);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
nir/range_analysis: set deleted key If (uintptr_t)&deleted_key is small enough, inserting entries into the hash table might not work correctly. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Backport-to: 26.0 Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-03 13:44:01 +00:00
_mesa_hash_table_set_deleted_key(range_ht, (void *)(uintptr_t)UINT32_MAX);
nir/range_analysis: use perform_analysis() in nir_unsigned_upper_bound() Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21381>
2023-02-10 16:24:39 +00:00
return perform_analysis(&state);
nir: Apply a maximum stack depth to avoid stack overflows. A stackless (or at least using allocated memory for stack) version might be nice but for now this works around some games compiling large shaders and hitting stack overflows. CC: mesa-stable Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21231>
2023-01-10 16:12:34 +01:00
}
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
bool
nir_addition_might_overflow(nir_shader *shader, struct hash_table *range_ht,
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
nir_scalar ssa, unsigned const_val)
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
{
nir: remove unsigned upper bound config All config information is now either in nir->info or nir->options. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37361>
2025-09-14 17:43:57 +02:00
uint32_t ub = nir_unsigned_upper_bound(shader, range_ht, ssa);
nir: add nir_unsigned_upper_bound and nir_addition_might_overflow This adds a nir_unsigned_upper_bound() helper which does something similar to nir_analyze_range() except it tries to obtain the largest possible value instead of it's relation to zero. It also adds nir_addition_might_overflow(), which uses this helper to try to prove that an unsigned addition does not wrap around. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/2720>
2019-11-12 17:51:19 +00:00
return const_val + ub < const_val;
}
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
static uint64_t
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
ssa_def_bits_used(const nir_def *def, int recur)
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
{
uint64_t bits_used = 0;
uint64_t all_bits = BITFIELD64_MASK(def->bit_size);
nir/range_analysis: Handle vectors better in ssa_def_bits_used If a query is made of a vector ssa_def (possibly from an intermediate result), return all_bits. If a constant source is a vector, swizzle the correct component. Unit tests were added for the constant vector cases. I don't see a great way to make unit tests for the other cases. v2: Add a FINIHSME comment about u16vec2 hardware. Fixes: 96303a59eae ("nir: Add some range analysis for used bits") Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9123>
2021-02-17 19:40:07 -08:00
/* Querying the bits used from a vector is too hard of a question to
* answer. Return the conservative answer that all bits are used. To
* handle this, the function would need to be extended to be a query of a
* single component of the vector. That would also necessary to fully
* handle the 'num_components > 1' inside the loop below.
*
* FINISHME: This restriction will eventually need to be restricted to be
* useful for hardware that uses u16vec2 as the native 16-bit integer type.
*/
if (def->num_components > 1)
return all_bits;
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
/* Limit recursion */
if (recur-- <= 0)
return all_bits;
nir_foreach_use(src, def) {
nir: Use getters for nir_src::parent_* First, we need to give the parent_instr field a unique name to be able to replace with a helper. We have parent_instr fields for both nir_src and nir_def, so let's rename nir_src::parent_instr in preparation for rework. This was done with a combination of sed and manual fix-ups. Then we use semantic patches plus manual fixups: @@ expression s; @@ -s->renamed_parent_instr +nir_src_parent_instr(s) @@ expression s; @@ -s.renamed_parent_instr +nir_src_parent_instr(&s) @@ expression s; @@ -s->parent_if +nir_src_parent_if(s) @@ expression s; @@ -s.renamed_parent_if +nir_src_parent_if(&s) @@ expression s; @@ -s->is_if +nir_src_is_if(s) @@ expression s; @@ -s.is_if +nir_src_is_if(&s) Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671>
2023-08-14 09:58:47 -04:00
switch (nir_src_parent_instr(src)->type) {
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
case nir_instr_type_alu: {
nir: Use getters for nir_src::parent_* First, we need to give the parent_instr field a unique name to be able to replace with a helper. We have parent_instr fields for both nir_src and nir_def, so let's rename nir_src::parent_instr in preparation for rework. This was done with a combination of sed and manual fix-ups. Then we use semantic patches plus manual fixups: @@ expression s; @@ -s->renamed_parent_instr +nir_src_parent_instr(s) @@ expression s; @@ -s.renamed_parent_instr +nir_src_parent_instr(&s) @@ expression s; @@ -s->parent_if +nir_src_parent_if(s) @@ expression s; @@ -s.renamed_parent_if +nir_src_parent_if(&s) @@ expression s; @@ -s->is_if +nir_src_is_if(s) @@ expression s; @@ -s.is_if +nir_src_is_if(&s) Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671>
2023-08-14 09:58:47 -04:00
nir_alu_instr *use_alu = nir_instr_as_alu(nir_src_parent_instr(src));
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
unsigned src_idx = container_of(src, nir_alu_src, src) - use_alu->src;
nir/range_analysis: Handle vectors better in ssa_def_bits_used If a query is made of a vector ssa_def (possibly from an intermediate result), return all_bits. If a constant source is a vector, swizzle the correct component. Unit tests were added for the constant vector cases. I don't see a great way to make unit tests for the other cases. v2: Add a FINIHSME comment about u16vec2 hardware. Fixes: 96303a59eae ("nir: Add some range analysis for used bits") Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9123>
2021-02-17 19:40:07 -08:00
/* If a user of the value produces a vector result, return the
* conservative answer that all bits are used. It is possible to
* answer this query by looping over the components used. For example,
*
* vec4 32 ssa_5 = load_const(0x0000f000, 0x00000f00, 0x000000f0, 0x0000000f)
* ...
* vec4 32 ssa_8 = iand ssa_7.xxxx, ssa_5
*
* could conceivably return 0x0000ffff when queyring the bits used of
* ssa_7. This is unlikely to be worth the effort because the
* question can eventually answered after the shader has been
* scalarized.
*/
nir: Drop nir_alu_dest Instead, we replace it directly with nir_def. We could replace it with nir_dest but the next commit gets rid of that so this avoids unnecessary churn. Most of this commit was generated by sed: sed -i -e 's/dest.dest.ssa/def/g' src/**/*.h src/**/*.c src/**/*.cpp There were a few manual fixups required in the nir_legacy.c and nir_from_ssa.c as nir_legacy_reg and nir_parallel_copy_entry both have a similar pattern. Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 11:43:35 -05:00
if (use_alu->def.num_components > 1)
nir/range_analysis: Handle vectors better in ssa_def_bits_used If a query is made of a vector ssa_def (possibly from an intermediate result), return all_bits. If a constant source is a vector, swizzle the correct component. Unit tests were added for the constant vector cases. I don't see a great way to make unit tests for the other cases. v2: Add a FINIHSME comment about u16vec2 hardware. Fixes: 96303a59eae ("nir: Add some range analysis for used bits") Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9123>
2021-02-17 19:40:07 -08:00
return all_bits;
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
switch (use_alu->op) {
case nir_op_u2u8:
case nir_op_i2i8:
case nir_op_u2u16:
case nir_op_i2i16:
case nir_op_u2u32:
case nir_op_i2i32:
nir: handle u2u/i2i recursively in nir_def_bits_used to get the number of bits actually used by the uses. Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34489>
2025-04-13 19:47:56 -04:00
case nir_op_u2u64:
case nir_op_i2i64: {
uint64_t def_bits_used = ssa_def_bits_used(&use_alu->def, recur);
/* If one of the sign-extended bits is used, set the last src bit
* as used.
*/
if ((use_alu->op == nir_op_i2i8 || use_alu->op == nir_op_i2i16 ||
use_alu->op == nir_op_i2i32 || use_alu->op == nir_op_i2i64) &&
def_bits_used & ~all_bits)
def_bits_used |= BITFIELD64_BIT(def->bit_size - 1);
bits_used |= def_bits_used & all_bits;
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
break;
nir: handle u2u/i2i recursively in nir_def_bits_used to get the number of bits actually used by the uses. Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34489>
2025-04-13 19:47:56 -04:00
}
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
case nir_op_extract_u8:
case nir_op_extract_i8:
case nir_op_extract_u16:
case nir_op_extract_i16:
if (src_idx == 0 && nir_src_is_const(use_alu->src[1].src)) {
nir: handle extract opcodes recursively in nir_def_bits_used Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34489>
2025-04-13 20:25:01 -04:00
unsigned chunk = nir_alu_src_as_uint(use_alu->src[1]);
uint64_t defs_bits_used = ssa_def_bits_used(&use_alu->def, recur);
unsigned field_bits = use_alu->op == nir_op_extract_u8 ||
use_alu->op == nir_op_extract_i8 ? 8 : 16;
uint64_t field_bitmask = BITFIELD64_MASK(field_bits);
/* If one of the sign-extended bits is used, set the last src bit
* as used.
*/
if ((use_alu->op == nir_op_extract_i8 ||
use_alu->op == nir_op_extract_i16) &&
defs_bits_used & ~field_bitmask)
defs_bits_used |= BITFIELD64_BIT(field_bits - 1);
bits_used |= (field_bitmask & defs_bits_used) <<
(chunk * field_bits);
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
break;
} else {
return all_bits;
}
case nir_op_ishl:
case nir_op_ishr:
case nir_op_ushr:
nir: handle bit shifts by constants in nir_def_bits_used useful for open-coded bitfield extracts that are not using ubfe/ibfe Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34489>
2025-04-10 20:49:50 -04:00
if (src_idx == 0 && nir_src_is_const(use_alu->src[1].src)) {
unsigned bit_size = def->bit_size;
unsigned shift = nir_alu_src_as_uint(use_alu->src[1]) & (bit_size - 1);
uint64_t def_bits_used = ssa_def_bits_used(&use_alu->def, recur);
/* If one of the sign-extended bits is used, set the "last src
* bit before shifting" as used.
*/
if (use_alu->op == nir_op_ishr &&
def_bits_used & ~(all_bits >> shift))
def_bits_used |= BITFIELD64_BIT(bit_size - 1 - shift);
/* Reverse the shift to get the bits before shifting. */
if (use_alu->op == nir_op_ushr || use_alu->op == nir_op_ishr)
bits_used |= (def_bits_used << shift) & all_bits;
else
bits_used |= def_bits_used >> shift;
break;
} else if (src_idx == 1) {
bits_used |= use_alu->def.bit_size - 1;
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
break;
} else {
return all_bits;
}
case nir_op_iand:
case nir_op_ior:
assert(src_idx < 2);
if (nir_src_is_const(use_alu->src[1 - src_idx].src)) {
nir: handle iand/ior opcodes recursively in nir_def_bits_used Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34489>
2025-04-13 20:25:46 -04:00
uint64_t other_src = nir_alu_src_as_uint(use_alu->src[1 - src_idx]);
if (use_alu->op == nir_op_iand)
bits_used |= ssa_def_bits_used(&use_alu->def, recur) & other_src;
else
bits_used |= ssa_def_bits_used(&use_alu->def, recur) & ~other_src;
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
break;
} else {
return all_bits;
}
nir: handle ibfe/ubfe in nir_def_bits_used it will be used by radeonsi Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34489>
2025-04-10 20:47:23 -04:00
case nir_op_ibfe:
case nir_op_ubfe:
if (src_idx == 0 && nir_src_is_const(use_alu->src[1].src)) {
uint64_t def_bits_used = ssa_def_bits_used(&use_alu->def, recur);
unsigned bit_size = use_alu->def.bit_size;
unsigned offset = nir_alu_src_as_uint(use_alu->src[1]) & (bit_size - 1);
unsigned bits = nir_src_is_const(use_alu->src[2].src) ?
nir_alu_src_as_uint(use_alu->src[2]) & (bit_size - 1) :
/* Worst case if bits is not constant. */
(bit_size - offset);
uint64_t field_bitmask = BITFIELD64_MASK(bits);
/* If one of the sign-extended bits is used, set the last src
* bit as used.
* If bits is not constant, all bits can be the last one.
*/
if (use_alu->op == nir_op_ibfe &&
(def_bits_used >> offset) & ~field_bitmask) {
if (nir_alu_src_as_uint(use_alu->src[2]))
def_bits_used |= BITFIELD64_BIT(bits - 1);
else
def_bits_used |= field_bitmask;
}
bits_used |= (field_bitmask & def_bits_used) << offset;
break;
} else if (src_idx == 1 || src_idx == 2) {
bits_used |= use_alu->src[0].src.ssa->bit_size - 1;
break;
} else {
return all_bits;
}
nir: handle mul24 in nir_def_bits_used Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34489>
2025-04-10 20:49:50 -04:00
case nir_op_imul24:
case nir_op_umul24:
bits_used |= all_bits & 0xffffff;
break;
nir: handle mov and bcsel in nir_def_bits_used Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34489>
2025-05-12 20:43:51 -04:00
case nir_op_mov:
bits_used |= ssa_def_bits_used(&use_alu->def, recur);
break;
case nir_op_bcsel:
if (src_idx == 0)
bits_used |= 0x1;
else
bits_used |= ssa_def_bits_used(&use_alu->def, recur);
break;
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
default:
/* We don't know what this op does */
return all_bits;
}
break;
}
case nir_instr_type_intrinsic: {
nir_intrinsic_instr *use_intrin =
nir: Use getters for nir_src::parent_* First, we need to give the parent_instr field a unique name to be able to replace with a helper. We have parent_instr fields for both nir_src and nir_def, so let's rename nir_src::parent_instr in preparation for rework. This was done with a combination of sed and manual fix-ups. Then we use semantic patches plus manual fixups: @@ expression s; @@ -s->renamed_parent_instr +nir_src_parent_instr(s) @@ expression s; @@ -s.renamed_parent_instr +nir_src_parent_instr(&s) @@ expression s; @@ -s->parent_if +nir_src_parent_if(s) @@ expression s; @@ -s.renamed_parent_if +nir_src_parent_if(&s) @@ expression s; @@ -s->is_if +nir_src_is_if(s) @@ expression s; @@ -s.is_if +nir_src_is_if(&s) Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671>
2023-08-14 09:58:47 -04:00
nir_instr_as_intrinsic(nir_src_parent_instr(src));
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
unsigned src_idx = src - use_intrin->src;
switch (use_intrin->intrinsic) {
case nir_intrinsic_read_invocation:
case nir_intrinsic_shuffle:
case nir_intrinsic_shuffle_up:
case nir_intrinsic_shuffle_down:
case nir_intrinsic_shuffle_xor:
case nir_intrinsic_quad_broadcast:
case nir_intrinsic_quad_swap_horizontal:
case nir_intrinsic_quad_swap_vertical:
case nir_intrinsic_quad_swap_diagonal:
if (src_idx == 0) {
nir: Drop nir_dest Instead, we replace every use of it with nir_def. Most of this commit was generated by sed: sed -i -e 's/dest.ssa/def/g' src/**/*.h src/**/*.c src/**/*.cpp A few manual fixups were required in lima and the nir_legacy code. Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 11:56:00 -05:00
bits_used |= ssa_def_bits_used(&use_intrin->def, recur);
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
} else {
if (use_intrin->intrinsic == nir_intrinsic_quad_broadcast) {
bits_used |= 3;
} else {
/* Subgroups larger than 128 are not a thing */
bits_used |= 127;
}
}
break;
case nir_intrinsic_reduce:
case nir_intrinsic_inclusive_scan:
case nir_intrinsic_exclusive_scan:
assert(src_idx == 0);
switch (nir_intrinsic_reduction_op(use_intrin)) {
case nir_op_iadd:
case nir_op_imul:
case nir_op_ior:
case nir_op_iand:
case nir_op_ixor:
nir: Drop nir_dest Instead, we replace every use of it with nir_def. Most of this commit was generated by sed: sed -i -e 's/dest.ssa/def/g' src/**/*.h src/**/*.c src/**/*.cpp A few manual fixups were required in lima and the nir_legacy code. Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 11:56:00 -05:00
bits_used |= ssa_def_bits_used(&use_intrin->def, recur);
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
break;
default:
return all_bits;
}
break;
default:
/* We don't know what this op does */
return all_bits;
}
break;
}
case nir_instr_type_phi: {
nir: Use getters for nir_src::parent_* First, we need to give the parent_instr field a unique name to be able to replace with a helper. We have parent_instr fields for both nir_src and nir_def, so let's rename nir_src::parent_instr in preparation for rework. This was done with a combination of sed and manual fix-ups. Then we use semantic patches plus manual fixups: @@ expression s; @@ -s->renamed_parent_instr +nir_src_parent_instr(s) @@ expression s; @@ -s.renamed_parent_instr +nir_src_parent_instr(&s) @@ expression s; @@ -s->parent_if +nir_src_parent_if(s) @@ expression s; @@ -s.renamed_parent_if +nir_src_parent_if(&s) @@ expression s; @@ -s->is_if +nir_src_is_if(s) @@ expression s; @@ -s.is_if +nir_src_is_if(&s) Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24671>
2023-08-14 09:58:47 -04:00
nir_phi_instr *use_phi = nir_instr_as_phi(nir_src_parent_instr(src));
nir: Drop nir_dest Instead, we replace every use of it with nir_def. Most of this commit was generated by sed: sed -i -e 's/dest.ssa/def/g' src/**/*.h src/**/*.c src/**/*.cpp A few manual fixups were required in lima and the nir_legacy code. Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24674>
2023-08-14 11:56:00 -05:00
bits_used |= ssa_def_bits_used(&use_phi->def, recur);
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
break;
}
default:
return all_bits;
}
/* If we've somehow shown that all our bits are used, we're done */
assert((bits_used & ~all_bits) == 0);
if (bits_used == all_bits)
return all_bits;
}
return bits_used;
}
uint64_t
nir: Drop "SSA" from NIR language Everything is SSA now. sed -e 's/nir_ssa_def/nir_def/g' \ -e 's/nir_ssa_undef/nir_undef/g' \ -e 's/nir_ssa_scalar/nir_scalar/g' \ -e 's/nir_src_rewrite_ssa/nir_src_rewrite/g' \ -e 's/nir_gather_ssa_types/nir_gather_types/g' \ -i $(git grep -l nir | grep -v relnotes) git mv src/compiler/nir/nir_gather_ssa_types.c \ src/compiler/nir/nir_gather_types.c ninja -C build/ clang-format cd src/compiler/nir && find *.c *.h -type f -exec clang-format -i \{} \; Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io> Acked-by: Faith Ekstrand <faith.ekstrand@collabora.com> Acked-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24585>
2023-08-12 16:17:15 -04:00
nir_def_bits_used(const nir_def *def)
nir: Add some range analysis for used bits This isn't 100% accurate, of course, but it should be good enough for what we're about to do with it. Reviewed-by: Ian Romanick <ian.d.romanick@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8872>
2021-02-04 11:38:40 -06:00
{
return ssa_def_bits_used(def, 2);
}
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
static void
get_alu_num_lsb(struct analysis_state *state, struct scalar_query q, uint32_t *result, const uint32_t *src)
{
nir_op op = nir_scalar_alu_op(q.scalar);
switch (op) {
case nir_op_ior:
case nir_op_ixor:
case nir_op_iadd:
case nir_op_iand:
case nir_op_imul:
if (!q.head.pushed_queries) {
push_scalar_query(state, nir_scalar_chase_alu_src(q.scalar, 0));
push_scalar_query(state, nir_scalar_chase_alu_src(q.scalar, 1));
return;
}
break;
case nir_op_ishl:
if (!q.head.pushed_queries) {
push_scalar_query(state, nir_scalar_chase_alu_src(q.scalar, 0));
return;
}
break;
case nir_op_ishr:
case nir_op_ushr:
if (!q.head.pushed_queries) {
if (nir_scalar_is_const(nir_scalar_chase_alu_src(q.scalar, 1)))
push_scalar_query(state, nir_scalar_chase_alu_src(q.scalar, 0));
return;
}
break;
case nir_op_bcsel:
if (!q.head.pushed_queries) {
push_scalar_query(state, nir_scalar_chase_alu_src(q.scalar, 1));
push_scalar_query(state, nir_scalar_chase_alu_src(q.scalar, 2));
return;
}
break;
default:
return;
}
switch (op) {
case nir_op_ior:
case nir_op_ixor:
case nir_op_iadd: {
*result = MIN2(src[0], src[1]);
break;
}
case nir_op_iand: {
*result = MAX2(src[0], src[1]);
break;
}
case nir_op_imul: {
*result = MIN2(src[0] + src[1], q.scalar.def->bit_size);
break;
}
case nir_op_ishl: {
nir_scalar src1 = nir_scalar_chase_alu_src(q.scalar, 1);
uint32_t mask = q.scalar.def->bit_size - 1;
unsigned amount = nir_scalar_is_const(src1) ? nir_scalar_as_uint(src1) & mask : 0;
*result = MIN2(src[0] + amount, q.scalar.def->bit_size);
break;
}
case nir_op_ishr:
case nir_op_ushr: {
nir_scalar src1 = nir_scalar_chase_alu_src(q.scalar, 1);
unsigned amount = nir_scalar_as_uint(src1) & (q.scalar.def->bit_size - 1);
*result = amount > src[0] ? 0 : src[0] - amount;
break;
}
case nir_op_bcsel: {
*result = MIN2(src[0], src[1]);
break;
}
default:
UNREACHABLE("Unknown opcode");
}
}
static void
process_num_lsb_query(struct analysis_state *state, struct analysis_query *aq, uint32_t *result,
const uint32_t *src)
{
struct scalar_query q = *(struct scalar_query *)aq;
*result = 0;
if (nir_scalar_is_const(q.scalar)) {
uint64_t val = nir_scalar_as_uint(q.scalar);
*result = val ? ffsll(val) - 1 : q.scalar.def->bit_size;
} else if (nir_scalar_is_alu(q.scalar)) {
get_alu_num_lsb(state, q, result, src);
}
}
unsigned
nir_def_num_lsb_zero(struct hash_table *numlsb_ht, nir_scalar def)
{
struct scalar_query query_alloc[16];
uint32_t result_alloc[16];
struct analysis_state state;
state.shader = NULL;
state.range_ht = numlsb_ht;
util_dynarray_init_from_stack(&state.query_stack, query_alloc, sizeof(query_alloc));
util_dynarray_init_from_stack(&state.result_stack, result_alloc, sizeof(result_alloc));
state.query_size = sizeof(struct scalar_query);
state.get_key = &get_scalar_key;
nir/range_analysis: use function pointers for lookup Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-02 16:20:26 +00:00
state.lookup = &scalar_lookup,
state.insert = &scalar_insert,
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
state.process_query = &process_num_lsb_query;
push_scalar_query(&state, def);
nir/range_analysis: set deleted key If (uintptr_t)&deleted_key is small enough, inserting entries into the hash table might not work correctly. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Backport-to: 26.0 Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Emma Anholt <emma@anholt.net> Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40190>
2026-03-03 13:44:01 +00:00
_mesa_hash_table_set_deleted_key(numlsb_ht, (void *)(uintptr_t)UINT32_MAX);
nir: add nir_def_num_lsb_zero Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36760>
2025-08-13 10:56:51 +01:00
return perform_analysis(&state);
}
Reference in a new issue Copy permalink