mesa/src/intel/compiler/brw
Lionel Landwerlin fa523aedd0
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
brw: fence SLM writes between workgroups
On LSC platforms the SLM writes are unfenced between workgroups. This
means a workgroup W1 finishing might have uncompleted SLM writes.
Another workgroup W2 dispatched after W1 which gets allocated an
overlapping SLM location might have writes that race with the previous
W1 operations.

The solution to this is fence all write operations (store & atomics)
of a workgroup before ending the threads. We do this by emitting a
single SLM fence either at the end of the shader or if there is only a
single unfenced right, at the end of that block.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13924
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40430>
2026-03-26 22:38:55 +00:00
..
tests brw: Properly set 'desc as register' for SEND in assembler 2025-12-10 19:46:52 +00:00
brw_analysis.cpp
brw_analysis.h brw: Don't mark_invalid in update_for_reads for non-VGRF destination 2026-03-05 00:02:50 +00:00
brw_analysis_def.cpp brw: Also check for ADDRESS file in update_for_reads 2026-03-05 00:02:51 +00:00
brw_analysis_liveness.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_analysis_performance.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_asm.c brw: Remove global variables from brw_asm parser 2026-01-23 23:13:31 +00:00
brw_asm.h
brw_asm_internal.h brw: Remove global variables from brw_asm parser 2026-01-23 23:13:31 +00:00
brw_asm_tool.c intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_builder.cpp intel: Rename wm_prog_data to fs_prog_data 2026-02-06 20:51:59 -08:00
brw_builder.h brw: Make brw_builder::uniform() ignore previous group 2026-02-20 16:50:41 +00:00
brw_cfg.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_cfg.h intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_compile_bs.cpp brw: Include backend NIR passes in mda files 2026-01-28 19:52:02 +00:00
brw_compile_cs.cpp brw: Include backend NIR passes in mda files 2026-01-28 19:52:02 +00:00
brw_compile_fs.cpp brw: do not omit RT writes if dual_src_blend is on 2026-03-19 23:38:40 +00:00
brw_compile_gs.cpp brw: Combine brw_assign_*_urb_setup() into one function 2026-03-12 21:40:37 +00:00
brw_compile_mesh.cpp brw: Include backend NIR passes in mda files 2026-01-28 19:52:02 +00:00
brw_compile_tcs.cpp brw: Lower TCS single patch invocation ID calculations in NIR 2026-03-12 21:40:37 +00:00
brw_compile_tes.cpp brw: Combine brw_assign_*_urb_setup() into one function 2026-03-12 21:40:37 +00:00
brw_compile_vs.cpp brw: Combine brw_assign_*_urb_setup() into one function 2026-03-12 21:40:37 +00:00
brw_compiler.c Rename *_sha1 names to *_blake3 2026-03-23 07:03:28 +00:00
brw_compiler.h Rename *_sha1 names to *_blake3 2026-03-23 07:03:28 +00:00
brw_disasm.c brw: add missing base offset decoding 2026-01-20 21:25:52 +00:00
brw_disasm.h
brw_disasm_info.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_disasm_info.h intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_disasm_tool.c intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_eu.c intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_eu.h brw: move brw_can_coherent_fb_fetch to a C header 2026-03-02 12:44:42 +00:00
brw_eu_compact.c intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_eu_defines.h brw: make PULL_CONSTANT opcodes more like MEMORY opcodes 2026-02-12 16:45:22 +00:00
brw_eu_emit.c brw: Fix encoding of accumulator sources of 3-source instructions 2026-03-09 19:21:39 +00:00
brw_eu_inst.h intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_eu_validate.c brw/validate: Eliminate duplicate integer multiply validation 2026-03-09 19:21:38 +00:00
brw_from_nir.cpp brw: Handle scalars and swizzles correctly in is_const_zero 2026-03-24 01:31:25 +00:00
brw_generator.cpp intel/compiler: move validation assert after brw_shader_debug_log 2026-03-24 04:54:31 +00:00
brw_generator.h brw: Rework UIP and JIP setting code 2026-01-08 22:01:45 +00:00
brw_gram.y brw/asm: Don't drop accumulator number in the assembler 2026-03-09 19:21:39 +00:00
brw_inst.cpp intel/brw: Nuke brw_inst::is_volatile() 2026-01-05 14:11:47 +00:00
brw_inst.h intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_isa_info.h intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_lex.l intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_load_reg.cpp
brw_lower.cpp intel: Rename wm_prog_data to fs_prog_data 2026-02-06 20:51:59 -08:00
brw_lower_dpas.cpp
brw_lower_fill_spill.cpp anv, brw: Consolidate ex_bso bits to a static devinfo inline 2026-02-16 21:33:47 +00:00
brw_lower_integer_multiplication.cpp
brw_lower_logical_sends.cpp intel/brw: Add and call brw_lsc_supports_base_offset() in places that checks for support of this feature 2026-02-19 16:53:03 +00:00
brw_lower_pack.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_lower_regioning.cpp brw/lower_regioning: Allow integer conversions in SEL 2026-03-24 01:31:26 +00:00
brw_lower_scoreboard.cpp brw/scoreboard: Don't track dependencies for UNDEFs 2026-02-26 06:54:48 +00:00
brw_lower_simd_width.cpp brw: handle lowering of a couple of opcodes 2026-01-12 12:19:21 +00:00
brw_lower_subgroup_ops.cpp
brw_nir.c brw: fence SLM writes between workgroups 2026-03-26 22:38:55 +00:00
brw_nir.h brw: fence SLM writes between workgroups 2026-03-26 22:38:55 +00:00
brw_nir_fence_shared_stores.c brw: fence SLM writes between workgroups 2026-03-26 22:38:55 +00:00
brw_nir_lower_alpha_to_coverage.c intel: Rename intel_msaa_flags to intel_fs_config 2026-02-06 20:51:43 -08:00
brw_nir_lower_cooperative_matrix.c brw: preserve fp_math_ctrl when lowering cmat alu 2026-03-26 13:15:50 +00:00
brw_nir_lower_cs_intrinsics.c brw: fix local_invocation_index with quad derivaties on mesh/task shaders 2026-01-27 22:28:19 +00:00
brw_nir_lower_fs_barycentrics.c intel: Rename intel_msaa_flags to intel_fs_config 2026-02-06 20:51:43 -08:00
brw_nir_lower_fs_load_output.c intel: Rename wm_prog_key to fs_prog_key 2026-02-06 20:52:01 -08:00
brw_nir_lower_fsign.py brw/nir_lower_fsign: try to fix NaN correctness 2026-02-10 18:42:03 +00:00
brw_nir_lower_immediate_offsets.c brw/nir: enable constant offsets for global_constant_uniform_block_intel 2026-03-06 06:34:43 +00:00
brw_nir_lower_intersection_shader.c intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_nir_lower_ray_queries.c intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_nir_lower_rt_intrinsics.c anv/brw: remove push constant load emulation from the backend compiler 2026-03-06 06:34:43 +00:00
brw_nir_lower_rt_intrinsics_pre_trace.c
brw_nir_lower_sample_index_in_coord.c brw/nir_lower_sample_index_in_coord: use helpers 2025-10-09 09:50:20 -04:00
brw_nir_lower_shader_calls.c intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_nir_lower_storage_image.c brw/lower_storage_image: use f2f32 instead of unpack_half 2026-02-06 06:12:36 +00:00
brw_nir_lower_texel_address.c brw: fixup 64bit atomics emulation on 2D array images 2025-11-14 00:01:50 +00:00
brw_nir_lower_texture.c intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_nir_opt_fsat.c nir: only set fp_math_ctrl if meaningful 2026-02-18 14:04:22 +00:00
brw_nir_rt.c brw/nir: handle inline_data_intel more like push_data_intel 2026-02-25 10:44:09 +00:00
brw_nir_rt.h intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_nir_rt_builder.h intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_nir_wa_18019110168.c intel: Rename intel_msaa_flags to intel_fs_config 2026-02-06 20:51:43 -08:00
brw_nir_workarounds.py intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_opt.cpp intel/compiler/brw: fix potential unsigned overflow 2025-10-24 07:52:09 -07:00
brw_opt_address_reg_load.cpp
brw_opt_algebraic.cpp brw/algebraic: Allow mixed types in saturate constant folding 2026-03-24 01:31:25 +00:00
brw_opt_bank_conflicts.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_opt_cmod_propagation.cpp brw/cmod: Don't propagate from CMP to ADD if there is a write between 2026-02-19 21:28:54 +00:00
brw_opt_cmp_flag_destination.cpp brw: Change the flags written by some CMP 2026-03-24 01:31:26 +00:00
brw_opt_combine_constants.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_opt_copy_propagation.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_opt_cse.cpp brw/cse: use copies in operands_match instead of in-place modification 2026-02-11 18:43:02 +00:00
brw_opt_dead_code_eliminate.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_opt_fill_spill.cpp brw: Eliminate duplicate fills 2025-11-26 17:20:13 +00:00
brw_opt_register_coalesce.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_opt_saturate_propagation.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_opt_txf_combiner.cpp brw: switch to new sampler payload description scheme 2025-10-16 12:08:15 +00:00
brw_opt_virtual_grfs.cpp intel/brw: Add BRW_DEPENDENCY_INSTRUCTIONS invalidation when instructions are added or removed in brw_opt_split_virtual_grfs() 2026-02-02 14:46:50 +00:00
brw_packed_float.c intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_print.cpp brw/print: Don't print extra space at the end 2026-02-06 01:00:31 +00:00
brw_private.h brw: Include backend NIR passes in mda files 2026-01-28 19:52:02 +00:00
brw_reg.cpp brw/algebraic: Allow mixed types in saturate constant folding 2026-03-24 01:31:25 +00:00
brw_reg.h brw/algebraic: Allow mixed types in saturate constant folding 2026-03-24 01:31:25 +00:00
brw_reg_allocate.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_reg_type.c intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_reg_type.h intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_rt.h anv: Fix Wa_14021821874, Wa_14018813551, Wa_14026600921 2026-03-10 22:41:54 +00:00
brw_sampler.c brw: handling dynamic programmable offsets pre-Xe2 2025-10-21 06:13:10 +00:00
brw_sampler.h brw: new Xe2 sampler opcodes 2025-10-16 12:08:16 +00:00
brw_schedule_instructions.cpp brw: use BITSET_LINEAR_ZALLOC 2026-01-26 20:15:43 +00:00
brw_shader.cpp brw: Change the flags written by some CMP 2026-03-24 01:31:26 +00:00
brw_shader.h brw: Change the flags written by some CMP 2026-03-24 01:31:26 +00:00
brw_simd_selection.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
brw_thread_payload.cpp brw: Purge source_depth_to_render_target 2026-03-12 21:40:37 +00:00
brw_thread_payload.h brw: Purge source_depth_to_render_target 2026-03-12 21:40:37 +00:00
brw_validate.cpp brw/validate: Perform more 3-src validation in brw_validate instead of brw_eu_emit 2026-03-09 19:21:39 +00:00
brw_vue_map.c brw: Use memset for initializing varying/slot maps 2026-02-16 15:15:38 -08:00
brw_workaround.cpp brw: Remove foreach_block_safe / reverse_safe 2026-01-24 01:15:52 +00:00
meson.build brw: fence SLM writes between workgroups 2026-03-26 22:38:55 +00:00
test_def_analysis.cpp brw: Also check for ADDRESS file in update_for_reads 2026-03-05 00:02:51 +00:00
test_eu_compact.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
test_eu_validate.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
test_helpers.cpp
test_helpers.h
test_insert_load_reg.cpp
test_lower_regioning.cpp brw/lower_regioning: Allow integer conversions in SEL 2026-03-24 01:31:26 +00:00
test_lower_scoreboard.cpp brw/scoreboard: Support local implicit out-of-order dependencies 2026-01-21 22:29:28 +00:00
test_opt_algebraic.cpp brw/algebraic: Allow mixed types in saturate constant folding 2026-03-24 01:31:25 +00:00
test_opt_cmod_propagation.cpp brw/cmod: Don't propagate from CMP to ADD if there is a write between 2026-02-19 21:28:54 +00:00
test_opt_combine_constants.cpp
test_opt_copy_propagation.cpp
test_opt_cse.cpp
test_opt_register_coalesce.cpp
test_opt_saturate_propagation.cpp brw: switch to new sampler payload description scheme 2025-10-16 12:08:15 +00:00
test_simd_selection.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00
test_vf_float_conversions.cpp intel/compiler: Use SPDX annotations 2026-01-24 20:37:31 +00:00