mesa/src/compiler/nir
Rhys Perry ce9205c03b nir: add a load/store vectorization pass
This pass combines intersecting, adjacent and identical loads/stores into
potentially larger ones and will be used by ACO to greatly reduce the
number of memory operations.

v2: handle nir_deref_type_ptr_as_array
v3: assume explicitly laid out types for derefs
v4: create less deref casts
v4: fix shared boolean vectorization
v4: fix copy+paste error in resources_different
v4: fix extract_subvector() to pass
    nir_load_store_vectorize_test.ssbo_load_intersecting_32_32_64
v4: rebase
v5: subtract from deref/offset instead of scheduling offset calculations
v5: various non-functional changes/cleanups
v5: require less metadata and preserve more
v5: rebase
v6: cleanup and improve dependency handling
v6: emit less deref casts
v6: pass undef to components not set in the write_mask for new stores
v7: fix 8-bit extract_vector() with 64-bit input
v7: cleanup creation of store write data
v7: update align correctly for when the bit size of load/store increases
v7: rename extract_vector to extract_component and update comment
v8: prevent combining of row-major matrix column acceses
v9: rework process_block() to be able to vectorize more
v9: rework the callback function
v9: update alignment on all loads/stores, even if they're not vectorized
v9: remove entry::store_value, since it will not be updated if it's was
    from a vectorized load
v9: fix bug in subtract_deref(), causing artifacts in Dishonored 2
v9: handle nir_intrinsic_scoped_memory_barrier
v10: use nir_ssa_scalar
v10: handle non-32-bit offsets
v10: use signed offsets for comparison
v10: improve create_entry_key_from_offset()
v10: support load_shared/store_shared
v10: remove strip_deref_casts()
v10: don't ever pass NULL to memcmp
v10: remove recursion in gcd()
v10: fix outdated comment
v11: use the new nir_extract_bits()
v12: remove use of nir_src_as_const_value in resources_different
v13: make entry key hash function deterministic
v13: simplify mask_sign_extend()
v14: add comment in hash_entry_key() about hashing pointers

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com> (v9)
2019-11-25 13:59:11 +00:00
..
tests nir: Add tests for nir_extract_bits 2019-11-11 17:17:02 +00:00
meson.build nir: add a load/store vectorization pass 2019-11-25 13:59:11 +00:00
nir.c nir: add nir_num_variable_modes and nir_var_mem_push_const 2019-11-25 13:59:11 +00:00
nir.h nir: add a load/store vectorization pass 2019-11-25 13:59:11 +00:00
nir_algebraic.py nir/algebraic: Add the ability to mark a replacement as exact 2019-11-04 14:05:49 -08:00
nir_builder.h nir: fix a couple signed/unsigned comparison warnings in nir_builder.h 2019-11-12 11:44:02 -07:00
nir_builder_opcodes_h.py nir: add bit_size parameter to system values with multiple allowed bit sizes 2019-01-21 00:17:18 +01:00
nir_builtin_builder.c nir/builder: Move nir_atan and nir_atan2 from SPIR-V translator 2019-10-12 09:43:17 +02:00
nir_builtin_builder.h vtn/opencl: add clz support 2019-11-22 04:37:41 +10:00
nir_clone.c util: rename list_empty() to list_is_empty() 2019-10-28 11:24:38 +00:00
nir_constant_expressions.h nir: add support for flushing to zero denorm constants 2019-09-17 23:39:18 +03:00
nir_constant_expressions.py nir: fix denorms in unpack_half_1x16() 2019-09-17 23:39:18 +03:00
nir_control_flow.c nir: keep the phi order when splitting blocks 2019-02-04 20:41:13 -08:00
nir_control_flow.h nir: add a new nir_cf_list_clone_and_reinsert() helper 2018-12-10 13:59:50 +11:00
nir_control_flow_private.h
nir_deref.c nir: fix deref offset builder 2019-11-22 04:37:41 +10:00
nir_deref.h nir/derefs: Add a nir_derefs_do_not_alias enum value 2018-11-28 14:29:25 -06:00
nir_divergence_analysis.c nir/divergence: improve DA of shuffle 2019-11-12 17:21:38 +00:00
nir_dominance.c nir: Add a block_is_unreachable helper 2019-09-06 23:39:01 +00:00
nir_format_convert.h nir: make nir_const_value scalar 2019-04-14 22:25:56 +02:00
nir_from_ssa.c util: rename list_empty() to list_is_empty() 2019-10-28 11:24:38 +00:00
nir_gather_info.c nir: add shader_info::last_msaa_image 2019-10-09 17:12:19 -04:00
nir_gather_ssa_types.c nir: fix gather_ssa_types 2019-05-31 21:35:26 +00:00
nir_gather_xfb_info.c nir: pack nir_variable::data::xfb_* 2019-11-04 18:17:34 -05:00
nir_gs_count_vertices.c util: use C99 declaration in the for-loop set_foreach() macro 2018-10-25 12:43:18 +01:00
nir_inline_functions.c nir/inline_functions: Break inlining into a builder helper 2019-03-06 17:24:57 +00:00
nir_instr_set.c nir/instr_set: Expose nir_instrs_equal() 2019-07-10 00:20:59 +00:00
nir_instr_set.h
nir_intrinsics.py nir: Add load_sampler_lod_paramaters_pan intrinsic 2019-11-22 05:07:19 +00:00
nir_intrinsics_c.py nir: add legal bit_sizes to intrinsics 2019-01-21 00:16:51 +01:00
nir_intrinsics_h.py python: Specify the template output encoding 2018-08-07 13:28:35 -07:00
nir_linking_helpers.c nir: Fix overlapping vars in nir_assign_io_var_locations() 2019-09-25 15:53:50 +02:00
nir_liveness.c nir: Remove a bunch of large stack arrays 2019-07-22 16:17:18 -05:00
nir_loop_analyze.c nir: add support for flushing to zero denorm constants 2019-09-17 23:39:18 +03:00
nir_loop_analyze.h nir: move ends_in_break() helper to nir_loop_analyze.h 2018-06-07 11:33:04 +10:00
nir_lower_alpha_test.c Revert "nir: drop support for using load_alpha_ref_float" 2019-10-23 13:03:52 +02:00
nir_lower_alu.c nir: Add lowering for nir_op_bit_count. 2018-06-06 13:44:28 -07:00
nir_lower_alu_to_scalar.c nir/lower_alu_to_scalar: Support lowering 8- and 16-bit reduce ops 2019-11-20 14:09:43 +01:00
nir_lower_amul.c nir: add nir_lower_amul pass 2019-10-18 15:08:54 -07:00
nir_lower_array_deref_of_vec.c nir: Add a new pass to lower array dereferences on vectors 2019-03-15 23:10:27 -05:00
nir_lower_atomics_to_ssbo.c nir: Fix num_ssbos when lowering atomic counters 2019-09-03 15:54:54 +02:00
nir_lower_bit_size.c compiler/nir: add a lowering pass to convert the bit size of ALU operations 2018-05-03 11:40:25 +02:00
nir_lower_bitmap.c nir: Use sampler derefs in drawpixels and bitmap lowering. 2019-02-11 21:34:44 -08:00
nir_lower_bool_to_float.c nir: fix lower_{int,bool}_to_float for new mov opcode 2019-05-31 21:35:26 +00:00
nir_lower_bool_to_int32.c nir: Drop imov/fmov in favor of one mov instruction 2019-05-24 08:38:11 -05:00
nir_lower_clamp_color_outputs.c nir: add tess support to nir_lower_clamp_color_outputs() 2019-06-26 00:36:48 +00:00
nir_lower_clip.c nir: patch up deref-vars when lowering clip-planes 2019-11-12 09:13:22 +01:00
nir_lower_clip_cull_distance_arrays.c nir/lower_clip_cull: Fix an incorrect assert 2019-02-21 12:05:12 -06:00
nir_lower_constant_initializers.c nir: Make nir_constant a vector rather than a matrix 2019-06-19 21:05:54 +00:00
nir_lower_double_ops.c nir: fix denorm flush-to-zero in sqrt's lowering at nir_lower_double_ops 2019-09-17 23:39:18 +03:00
nir_lower_drawpixels.c nir/drawpixels: fix what appears to be a copy-paste bug in get_texcoord_const 2019-10-10 15:49:18 -04:00
nir_lower_fb_read.c nir: add pass to lower fb reads 2019-05-02 11:19:22 -07:00
nir_lower_flatshade.c nir: add a pass to lower flat shading. 2019-10-17 10:41:36 +02:00
nir_lower_flrp.c nir/compiler: keep same bit size when lowering with flrp 2019-07-12 16:15:20 +00:00
nir_lower_fragcoord_wtrans.c nir: add rcp(w) lowering for gl_FragCoord 2019-04-29 02:46:44 +00:00
nir_lower_frexp.c spirv,nir: lower frexp_exp/frexp_sig inside a new NIR pass 2019-03-22 19:41:46 +01:00
nir_lower_global_vars_to_local.c nir: Fix non-determinism in lower_global_vars_to_local 2019-11-14 13:10:58 +00:00
nir_lower_gs_intrinsics.c util: use C99 declaration in the for-loop set_foreach() macro 2018-10-25 12:43:18 +01:00
nir_lower_idiv.c nir/lower_idiv: add new llvm-based path 2019-10-21 18:49:46 +00:00
nir_lower_indirect_derefs.c nir: fix bit_size in lower indirect derefs. 2019-04-26 12:59:43 +10:00
nir_lower_input_attachments.c nir/lower_input_attachments: pass on non-uniform access flag 2019-10-11 14:26:58 +00:00
nir_lower_int64.c nir: add 64-bit ufind_msb lowering support. (v2) 2019-11-22 04:37:37 +10:00
nir_lower_int_to_float.c tree-wide: replace MAYBE_UNUSED with ASSERTED 2019-07-31 09:41:05 +01:00
nir_lower_interpolation.c nir: add pass to lower load_interpolated_input 2019-07-02 16:15:25 +00:00
nir_lower_io.c nir: fix unused variable warning in nir_lower_vars_to_explicit_types 2019-11-07 11:32:55 +11:00
nir_lower_io_arrays_to_elements.c nir: fix couple of compile warnings 2019-10-31 00:21:44 +00:00
nir_lower_io_to_scalar.c nir: copy intrinsic type when lowering load input/uniform and store output 2019-06-03 12:46:14 +00:00
nir_lower_io_to_temporaries.c nir/lower_io_to_temporaries: Fix hash table leak 2019-07-09 10:39:37 +02:00
nir_lower_io_to_vector.c nir: add amul instruction 2019-10-18 15:08:54 -07:00
nir_lower_load_const_to_scalar.c nir/load_const_to_scalar: Get rid of a bit size switch statement 2019-04-14 22:25:56 +02:00
nir_lower_locals_to_regs.c nir: Drop imov/fmov in favor of one mov instruction 2019-05-24 08:38:11 -05:00
nir_lower_non_uniform_access.c nir/lower_non_uniform: lower image/texture instructions taking derefs 2019-10-11 14:26:58 +00:00
nir_lower_packing.c compiler/lower_64bit_packing: rename the pass to be more generic 2018-05-03 11:40:26 +02:00
nir_lower_passthrough_edgeflags.c st/mesa: assign driver locations for VS inputs for NIR before caching 2019-10-23 21:12:52 -04:00
nir_lower_patch_vertices.c i965: Combine both gl_PatchVerticesIn lowering passes. 2018-07-26 21:51:36 -07:00
nir_lower_phis_to_scalar.c nir: Drop imov/fmov in favor of one mov instruction 2019-05-24 08:38:11 -05:00
nir_lower_point_size.c nir/lower_point_size: assume scalar PSIZ 2019-09-12 06:40:04 +00:00
nir_lower_point_size_mov.c nir: add lowering-pass for point-size mov 2019-10-17 10:41:36 +02:00
nir_lower_regs_to_ssa.c util: rename list_empty() to list_is_empty() 2019-10-28 11:24:38 +00:00
nir_lower_returns.c nir: Unset metadata debug bit if no progress made 2019-01-09 16:42:40 -08:00
nir_lower_samplers.c nir: Carve out nir_lower_samplers from GLSL code. 2019-09-06 12:20:20 +03:00
nir_lower_scratch.c nir: use common deref has indirect code in scratch lowering. 2019-08-01 08:32:12 +10:00
nir_lower_subgroups.c nir/subgroups: Lower clustered reductions with cluster_size >= subgroup_size into reductions 2019-08-20 17:40:10 +02:00
nir_lower_system_values.c nir/lower_system_values: Support lowering more intrinsics 2019-07-16 16:05:16 +00:00
nir_lower_tex.c nir/lower_tex: Add an assert() in nir_lower_txs_lod() 2019-06-20 09:15:53 -07:00
nir_lower_to_source_mods.c util: rename list_empty() to list_is_empty() 2019-10-28 11:24:38 +00:00
nir_lower_two_sided_color.c nir: support derefs in two-sided lighting lowering 2019-10-17 10:41:36 +02:00
nir_lower_uniforms_to_ubo.c nir: allow nir_lower_uniforms_to_ubo to be run repeatedly 2019-10-22 14:41:23 -04:00
nir_lower_var_copies.c nir: Allow qualifiers on copy_deref and image instructions 2019-06-19 14:08:27 +02:00
nir_lower_vars_to_ssa.c nir: Remove a bunch of large stack arrays 2019-07-22 16:17:18 -05:00
nir_lower_vec_to_movs.c util: rename list_empty() to list_is_empty() 2019-10-28 11:24:38 +00:00
nir_lower_viewport_transform.c nir/lower_viewport: Check variable mode first 2019-07-18 14:21:41 +02:00
nir_lower_wpos_center.c intel: Use a system value for gl_FragCoord 2019-07-29 23:30:26 +00:00
nir_lower_wpos_ytransform.c nir: fix lower_wpos_ytransform in load_frag_coord case 2019-05-02 11:19:22 -07:00
nir_metadata.c
nir_move_vec_src_uses_to_dest.c
nir_normalize_cubemap_coords.c
nir_opcodes.py nir: Add a 8-bit bool type 2019-11-20 14:09:43 +01:00
nir_opcodes_c.py nir: Drop imov/fmov in favor of one mov instruction 2019-05-24 08:38:11 -05:00
nir_opcodes_h.py python: Better iterate over dictionaries 2018-07-24 11:07:04 -07:00
nir_opt_access.c nir: move data.image.access to data.access 2019-11-19 18:20:05 -05:00
nir_opt_algebraic.py nir/algebraic: Mark other comparison exact when removing a == a 2019-11-04 14:05:49 -08:00
nir_opt_combine_stores.c util: rename list_empty() to list_is_empty() 2019-10-28 11:24:38 +00:00
nir_opt_comparison_pre.c nir: Keep the range analysis HT around intra-pass until we make a change. 2019-10-04 19:15:01 +00:00
nir_opt_conditional_discard.c nir,intel: lower if (cond) demote() to new intrinsic demote_if(cond) 2019-07-24 13:02:18 -05:00
nir_opt_constant_folding.c nir/constant_folding: fold load_constant intrinsics 2019-10-07 19:49:53 +01:00
nir_opt_copy_prop_vars.c nir: Add scoped_memory_barrier intrinsic 2019-10-24 11:39:55 -07:00
nir_opt_copy_propagate.c nir: Drop imov/fmov in favor of one mov instruction 2019-05-24 08:38:11 -05:00
nir_opt_cse.c nir: clone instruction set rather than removing individual entries 2019-02-22 08:36:36 +11:00
nir_opt_dce.c nir: Unset metadata debug bit if no progress made 2019-01-09 16:42:40 -08:00
nir_opt_dead_cf.c nir/dead_cf: Remove dead control flow after infinite loops. 2019-10-11 17:24:26 +02:00
nir_opt_dead_write_vars.c nir: Add scoped_memory_barrier intrinsic 2019-10-24 11:39:55 -07:00
nir_opt_find_array_copies.c nir/find_array_copies: Reject copies with mismatched lengths 2019-08-14 18:11:31 +00:00
nir_opt_gcm.c nir: Add a nir_tex_instr_has_implicit_derivatives helper 2019-07-23 13:40:41 -05:00
nir_opt_idiv_const.c nir: Fix nir_opt_idiv_const when negatives are involved 2019-05-16 10:55:03 -07:00
nir_opt_if.c nir/opt_if: Fix undef handling in opt_split_alu_of_phi() 2019-09-18 17:18:34 -04:00
nir_opt_intrinsics.c nir: optimize gl_SampleMaskIn to gl_HelperInvocation for radeonsi when possible 2019-04-16 10:24:19 -04:00
nir_opt_large_constants.c nir/large_constants: use nir_index_vars and nir_variable::index 2019-11-20 15:05:42 +00:00
nir_opt_load_store_vectorize.c nir: add a load/store vectorization pass 2019-11-25 13:59:11 +00:00
nir_opt_loop_unroll.c util: rename list_empty() to list_is_empty() 2019-10-28 11:24:38 +00:00
nir_opt_move.c nir: merge and extend nir_opt_move_comparisons and nir_opt_move_load_ubo 2019-08-12 22:01:30 +00:00
nir_opt_peephole_select.c util: rename list_empty() to list_is_empty() 2019-10-28 11:24:38 +00:00
nir_opt_rematerialize_compares.c nir: Only rematerialize comparisons with all SSA sources 2019-07-19 19:45:36 +00:00
nir_opt_remove_phis.c nir/opt_remove_phis: handle phis with no sources 2019-09-25 00:58:30 +00:00
nir_opt_shrink_load.c
nir_opt_sink.c nir/sink: Don't sink load_ubo to outside of its defining loop 2019-10-09 17:55:25 +00:00
nir_opt_trivial_continues.c
nir_opt_undef.c nir: Drop imov/fmov in favor of one mov instruction 2019-05-24 08:38:11 -05:00
nir_opt_vectorize.c util: rename list_empty() to list_is_empty() 2019-10-28 11:24:38 +00:00
nir_phi_builder.c nir: Remove a bunch of large stack arrays 2019-07-22 16:17:18 -05:00
nir_phi_builder.h
nir_print.c nir/print: only print image.format for image variables 2019-11-19 18:20:07 -05:00
nir_propagate_invariant.c nir/propagate_invariant: Don't add NULL vars to the hash table 2019-06-06 00:27:53 +00:00
nir_range_analysis.c nir: no-op C99 _Pragma() with MSVC 2019-11-23 10:34:24 -07:00
nir_range_analysis.h nir: Keep the range analysis HT around intra-pass until we make a change. 2019-10-04 19:15:01 +00:00
nir_remove_dead_variables.c nir: rename nir_var_shared to nir_var_mem_shared 2019-01-19 20:01:41 +01:00
nir_repair_ssa.c nir/repair_ssa: Replace the unreachable check with the phi builder 2019-09-23 16:19:24 +00:00
nir_search.c nir/algebraic: Add the ability to mark a replacement as exact 2019-11-04 14:05:49 -08:00
nir_search.h nir/algebraic: Add the ability to mark a replacement as exact 2019-11-04 14:05:49 -08:00
nir_search_helpers.h util: rename list_empty() to list_is_empty() 2019-10-28 11:24:38 +00:00
nir_serialize.c nir/serialize: support any num_components for remaining instructions 2019-11-23 00:02:10 -05:00
nir_serialize.h nir: add a strip parameter to nir_serialize 2019-10-10 15:47:07 -04:00
nir_split_per_member_structs.c nir: free dead_ctx in case of no progress 2019-03-07 07:40:19 +02:00
nir_split_var_copies.c nir: Allow qualifiers on copy_deref and image instructions 2019-06-19 14:08:27 +02:00
nir_split_vars.c nir: Handle complex derefs in nir_split_array_vars 2019-09-06 23:39:01 +00:00
nir_sweep.c nir: Get rid of global registers 2019-04-09 00:29:36 -05:00
nir_to_lcssa.c nir/lcssa: allow to create LCSSA phis for loop-invariant booleans 2019-08-20 17:40:05 +02:00
nir_validate.c nir/validate: validate num_components on registers and intrinsics 2019-11-21 01:10:24 +01:00
nir_vla.h
nir_worklist.c
nir_worklist.h util: rename list_empty() to list_is_empty() 2019-10-28 11:24:38 +00:00
nir_xfb_info.h nir/xfb: move varyings info out of nir_xfb_info 2019-03-15 11:59:32 +01:00
README

New IR, or NIR, is an IR for Mesa intended to sit below GLSL IR and Mesa IR.
Its design inherits from the various IRs that Mesa has used in the past, as
well as Direct3D assembly, and it includes a few new ideas as well. It is a
flat (in terms of using instructions instead of expressions), typeless IR,
similar to TGSI and Mesa IR.  It also supports SSA (although it doesn't require
it).

Variables
=========

NIR includes support for source-level GLSL variables through a structure mostly
copied from GLSL IR. These will be used for linking and conversion from GLSL IR
(and later, from an AST), but for the most part, they will be lowered to
registers (see below) and loads/stores.

Registers
=========

Registers are light-weight; they consist of a structure that only contains its
size, its index for liveness analysis, and an optional name for debugging. In
addition, registers can be local to a function or global to the entire shader;
the latter will be used in ARB_shader_subroutine for passing parameters and
getting return values from subroutines. Registers can also be an array, in which
case they can be accessed indirectly. Each ALU instruction (add, subtract, etc.)
works directly with registers or SSA values (see below).

SSA
========

Everywhere a register can be loaded/stored, an SSA value can be used instead.
The only exception is that arrays/indirect addressing are not supported with
SSA; although research has been done on extensions of SSA to arrays before, it's
usually for the purpose of parallelization (which we're not interested in), and
adds some overhead in the form of adding copies or extra arrays (which is much
more expensive than introducing copies between non-array registers). SSA uses
point directly to their corresponding definition, which in turn points to the
instruction it is part of. This creates an implicit use-def chain and avoids the
need for an external structure for each SSA register.

Functions
=========

Support for function calls is mostly similar to GLSL IR. Each shader contains a
list of functions, and each function has a list of overloads. Each overload
contains a list of parameters, and may contain an implementation which specifies
the variables that correspond to the parameters and return value. Inlining a
function, assuming it has a single return point, is as simple as copying its
instructions, registers, and local variables into the target function and then
inserting copies to and from the new parameters as appropriate. After functions
are inlined and any non-subroutine functions are deleted, parameters and return
variables will be converted to global variables and then global registers. We
don't do this lowering earlier (i.e. the fortranizer idea) for a few reasons:

- If we want to do optimizations before link time, we need to have the function
signature available during link-time.

- If we do any inlining before link time, then we might wind up with the
inlined function and the non-inlined function using the same global
variables/registers which would preclude optimization.

Intrinsics
=========

Any operation (other than function calls and textures) which touches a variable
or is not referentially transparent is represented by an intrinsic. Intrinsics
are similar to the idea of a "builtin function," i.e. a function declaration
whose implementation is provided by the backend, except they are more powerful
in the following ways:

- They can also load and store registers when appropriate, which limits the
number of variables needed in later stages of the IR while obviating the need
for a separate load/store variable instruction.

- Intrinsics can be marked as side-effect free, which permits them to be
treated like any other instruction when it comes to optimizations. This allows
load intrinsics to be represented as intrinsics while still being optimized
away by dead code elimination, common subexpression elimination, etc.

Intrinsics are used for:

- Atomic operations
- Memory barriers
- Subroutine calls
- Geometry shader emitVertex and endPrimitive
- Loading and storing variables (before lowering)
- Loading and storing uniforms, shader inputs and outputs, etc (after lowering)
- Copying variables (cases where in GLSL the destination is a structure or
array)
- The kitchen sink
- ...

Textures
=========

Unfortunately, there are far too many texture operations to represent each one
of them with an intrinsic, so there's a special texture instruction similar to
the GLSL IR one. The biggest difference is that, while the texture instruction
has a sampler dereference field used just like in GLSL IR, this gets lowered to
a texture unit index (with a possible indirect offset) while the type
information of the original sampler is kept around for backends. Also, all the
non-constant sources are stored in a single array to make it easier for
optimization passes to iterate over all the sources.

Control Flow
=========

Like in GLSL IR, control flow consists of a tree of "control flow nodes", which
include if statements and loops, and jump instructions (break, continue, and
return). Unlike GLSL IR, though, the leaves of the tree aren't statements but
basic blocks. Each basic block also keeps track of its successors and
predecessors, and function implementations keep track of the beginning basic
block (the first basic block of the function) and the ending basic block (a fake
basic block that every return statement points to). Together, these elements
make up the control flow graph, in this case a redundant piece of information on
top of the control flow tree that will be used by almost all the optimizations.
There are helper functions to add and remove control flow nodes that also update
the control flow graph, and so usually it doesn't need to be touched by passes
that modify control flow nodes.