Commit graph

136 commits

Author SHA1 Message Date
Qiang Yu
196569b1a4 all: rename gl_shader_stage to mesa_shader_stage
It's not only for GL, change to a generic name.

Use command:
  find . -type f -not -path '*/.git/*' -exec sed -i 's/\bgl_shader_stage\b/mesa_shader_stage/g' {} +

Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36569>
2025-08-06 10:28:40 +08:00
Alyssa Rosenzweig
3719983edf brw: replace lower_fs_msaa with nir_inline_sysval
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36516>
2025-08-03 21:27:47 +00:00
Alyssa Rosenzweig
cc6e3b84cb treewide: use nir_def_as_*
Via Coccinelle patch:

    @@
    expression definition;
    @@

    -nir_instr_as_alu(definition->parent_instr)
    +nir_def_as_alu(definition)

    @@
    expression definition;
    @@

    -nir_instr_as_intrinsic(definition->parent_instr)
    +nir_def_as_intrinsic(definition)

    @@
    expression definition;
    @@

    -nir_instr_as_phi(definition->parent_instr)
    +nir_def_as_phi(definition)

    @@
    expression definition;
    @@

    -nir_instr_as_load_const(definition->parent_instr)
    +nir_def_as_load_const(definition)

    @@
    expression definition;
    @@

    -nir_instr_as_deref(definition->parent_instr)
    +nir_def_as_deref(definition)

    @@
    expression definition;
    @@

    -nir_instr_as_tex(definition->parent_instr)
    +nir_def_as_tex(definition)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Emma Anholt <emma@anholt.net>
Reviewed-by: Marek Olšák <maraeo@gmail.com>
Acked-by: Karol Herbst <kherbst@redhat.com>
Acked-by: Konstantin Seurer <konstantin.seurer@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36489>
2025-08-01 15:34:24 +00:00
Lionel Landwerlin
fcf4401824 brw: handle wa_18019110168 with independent shader compilation
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:35 +00:00
Lionel Landwerlin
bc8d18aee2 brw: make a helper for vertex attribute offset computation
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:34 +00:00
Lionel Landwerlin
5cc66e2c8d anv/brw: move Wa_18019110168 handling to backend
We simplify the implementation by assuming the worse case, copying
entire per-vertex regions if necessary.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35103>
2025-06-28 05:55:32 +00:00
Rohan Garg
e103afe7be brw: run the nir_opt_offsets pass and set the maximum offset size
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Perf A/B testing on DG2: no changes
Perf A/B testing on BMG: +2.1% Blackops3, +1.5% Cyberpunk

DG2 stats (mostly insignificant):
Assassins Creed Valhalla:
 Totals from 1169 (55.67% of 2100) affected shaders:
 Instrs: 509237 -> 509215 (-0.00%)
 Cycle count: 30614325 -> 30607419 (-0.02%); split: -0.03%, +0.00%
 Non SSA regs after NIR: 83434 -> 85909 (+2.97%)

Blackops 3:
 Totals from 1045 (64.63% of 1617) affected shaders:
 Instrs: 527312 -> 527310 (-0.00%)
 Cycle count: 496912222 -> 496902846 (-0.00%); split: -0.00%, +0.00%
 Non SSA regs after NIR: 106883 -> 109095 (+2.07%)

Cyberpunk:
 Totals from 706 (56.03% of 1260) affected shaders:
 Instrs: 345976 -> 345974 (-0.00%); split: -0.00%, +0.00%
 Cycle count: 9775138 -> 9775472 (+0.00%); split: -0.00%, +0.00%
 Max live registers: 40295 -> 40297 (+0.00%)
 Non SSA regs after NIR: 93245 -> 94718 (+1.58%)

Fortnite:
 Totals from 4210 (55.98% of 7521) affected shaders:
 Instrs: 2205471 -> 2205469 (-0.00%)
 Cycle count: 91451040 -> 91450956 (-0.00%); split: -0.00%, +0.00%
 Non SSA regs after NIR: 952354 -> 961664 (+0.98%)

LNL stats (notable changes):
Assassins Creed Valhalla:
 Totals from 1684 (83.57% of 2015) affected shaders:
 Instrs: 774305 -> 764501 (-1.27%); split: -1.27%, +0.01%
 Cycle count: 58845842 -> 58699250 (-0.25%); split: -0.98%, +0.73%
 Spill count: 625 -> 638 (+2.08%)
 Fill count: 1490 -> 1503 (+0.87%)
 Scratch Memory Size: 41984 -> 44032 (+4.88%)
 Max live registers: 196424 -> 197561 (+0.58%); split: -0.10%, +0.68%

Blackops 3:
 Totals from 1125 (76.53% of 1470) affected shaders:
 Instrs: 781749 -> 773275 (-1.08%); split: -1.08%, +0.00%
 Subgroup size: 22896 -> 22912 (+0.07%)
 Cycle count: 659864454 -> 654641032 (-0.79%); split: -1.10%, +0.31%
 Max live registers: 116772 -> 116854 (+0.07%); split: -0.01%, +0.08%
 Non SSA regs after NIR: 172648 -> 168260 (-2.54%); split: -2.55%, +0.01%

Control:
 Totals from 378 (51.50% of 734) affected shaders:
 Instrs: 148184 -> 147544 (-0.43%)
 Cycle count: 6905200 -> 6913366 (+0.12%); split: -0.30%, +0.42%
 Max live registers: 41271 -> 41281 (+0.02%)
 Non SSA regs after NIR: 44964 -> 43868 (-2.44%); split: -2.45%, +0.01%

Cyberpunk:
 Totals from 1141 (92.46% of 1234) affected shaders:
 Instrs: 636744 -> 629333 (-1.16%)
 Subgroup size: 24256 -> 24272 (+0.07%)
 Cycle count: 24952258 -> 24801298 (-0.60%); split: -1.39%, +0.78%
 Max live registers: 125848 -> 126855 (+0.80%); split: -0.00%, +0.80%
 Non SSA regs after NIR: 127399 -> 119837 (-5.94%); split: -5.95%, +0.02%

Fortnite:
 Totals from 5497 (83.52% of 6582) affected shaders:
 Instrs: 4072831 -> 4041852 (-0.76%); split: -0.77%, +0.01%
 Subgroup size: 103296 -> 103312 (+0.02%)
 Cycle count: 133046874 -> 132789242 (-0.19%); split: -0.67%, +0.48%
 Spill count: 7218 -> 7254 (+0.50%); split: -0.33%, +0.83%
 Fill count: 11724 -> 11749 (+0.21%); split: -0.34%, +0.55%
 Scratch Memory Size: 591872 -> 599040 (+1.21%)
 Max live registers: 816530 -> 818522 (+0.24%); split: -0.01%, +0.26%
 Non SSA regs after NIR: 1610296 -> 1560284 (-3.11%); split: -3.11%, +0.00%

Hitman3:
 Totals from 4713 (92.39% of 5101) affected shaders:
 Instrs: 2731598 -> 2698224 (-1.22%)
 Cycle count: 186422098 -> 185472640 (-0.51%); split: -1.12%, +0.61%
 Spill count: 3244 -> 3242 (-0.06%)
 Fill count: 9937 -> 9933 (-0.04%)
 Max live registers: 585035 -> 589801 (+0.81%); split: -0.00%, +0.82%
 Non SSA regs after NIR: 347681 -> 324314 (-6.72%); split: -6.73%, +0.01%

Hogwarts Legacy:
 Totals from 930 (59.81% of 1555) affected shaders:
 Instrs: 464146 -> 459526 (-1.00%); split: -1.00%, +0.01%
 Subgroup size: 19104 -> 19120 (+0.08%)
 Cycle count: 24062460 -> 24078964 (+0.07%); split: -0.49%, +0.56%
 Spill count: 2068 -> 1964 (-5.03%); split: -5.22%, +0.19%
 Fill count: 2342 -> 2205 (-5.85%); split: -6.40%, +0.56%
 Scratch Memory Size: 147456 -> 141312 (-4.17%)
 Max live registers: 112384 -> 112787 (+0.36%); split: -0.08%, +0.44%
 Non SSA regs after NIR: 80293 -> 79161 (-1.41%); split: -1.72%, +0.32%

Metro Exodus:
 Totals from 29755 (78.62% of 37846) affected shaders:
 Instrs: 11495578 -> 11492951 (-0.02%); split: -0.02%, +0.00%
 Subgroup size: 644688 -> 644704 (+0.00%)
 Cycle count: 301572068 -> 301548054 (-0.01%); split: -0.03%, +0.02%
 Max live registers: 3369504 -> 3370454 (+0.03%); split: -0.00%, +0.03%
 Non SSA regs after NIR: 2476561 -> 2396090 (-3.25%); split: -3.27%, +0.02%

Red Dead Redemption 2:
 Totals from 4161 (78.61% of 5293) affected shaders:
 Instrs: 2428782 -> 2409032 (-0.81%); split: -0.82%, +0.00%
 Subgroup size: 85344 -> 85360 (+0.02%)
 Cycle count: 8514984142 -> 8533415324 (+0.22%); split: -0.02%, +0.23%
 Spill count: 4659 -> 4674 (+0.32%); split: -0.02%, +0.34%
 Fill count: 11236 -> 11231 (-0.04%); split: -0.19%, +0.14%
 Scratch Memory Size: 398336 -> 397312 (-0.26%)
 Max live registers: 473946 -> 475798 (+0.39%); split: -0.08%, +0.47%
 Non SSA regs after NIR: 616820 -> 567706 (-7.96%); split: -8.09%, +0.12%

Rise Of The Tomb Raider:
 Totals from 68 (46.58% of 146) affected shaders:
 Instrs: 28209 -> 27801 (-1.45%)
 Subgroup size: 1584 -> 1600 (+1.01%)
 Cycle count: 16182992 -> 16249364 (+0.41%); split: -0.97%, +1.38%
 Max live registers: 7320 -> 7296 (-0.33%); split: -0.38%, +0.05%
 Non SSA regs after NIR: 8438 -> 8207 (-2.74%); split: -2.82%, +0.08%

Spiderman Remastered:
 Totals from 6403 (93.87% of 6821) affected shaders:
 Instrs: 5662713 -> 5597949 (-1.14%); split: -1.28%, +0.14%
 Cycle count: 282861519016 -> 279806958122 (-1.08%); split: -1.26%, +0.18%
 Spill count: 61150 -> 60754 (-0.65%); split: -1.13%, +0.48%
 Fill count: 162597 -> 163190 (+0.36%); split: -0.84%, +1.21%
 Scratch Memory Size: 5834752 -> 5804032 (-0.53%); split: -0.70%, +0.18%
 Max live registers: 901926 -> 903820 (+0.21%); split: -0.01%, +0.22%
 Non SSA regs after NIR: 555053 -> 521016 (-6.13%); split: -6.14%, +0.01%

Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35252>
2025-06-22 10:55:24 +00:00
Lionel Landwerlin
52f73db5b7 brw: implement read without format lowering
Load the format enum and then just go through a series of :

   if format == R16G16B16A16_UNORM
      color = lower_r32g32_uint_tor_r16g16b16a16_unorm(color)
   else if format == R16G16B16A16_SNORM
      ...

For Gfx12.5, there is no in-shader conversion.

For Gfx12/11, the in-shader conversion covers the following formats :
    - ISL_FORMAT_R10G10B10A2_UNORM
    - ISL_FORMAT_R10G10B10A2_UINT
    - ISL_FORMAT_R11G11B10_FLOAT

For Gfx9, the following formats :
    - ISL_FORMAT_R16G16B16A16_UNORM
    - ISL_FORMAT_R16G16B16A16_SNORM
    - ISL_FORMAT_R10G10B10A2_UNORM
    - ISL_FORMAT_R10G10B10A2_UINT
    - ISL_FORMAT_R8G8B8A8_UNORM
    - ISL_FORMAT_R8G8B8A8_SNORM
    - ISL_FORMAT_R16G16_UNORM
    - ISL_FORMAT_R16G16_SNORM
    - ISL_FORMAT_R11G11B10_FLOAT
    - ISL_FORMAT_R8G8_UNORM
    - ISL_FORMAT_R8G8_SNORM
    - ISL_FORMAT_R16_UNORM
    - ISL_FORMAT_R16_SNORM
    - ISL_FORMAT_R8_UNORM
    - ISL_FORMAT_R8_SNORM

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22524>
2025-06-06 12:28:42 +00:00
Lionel Landwerlin
79498a0849 brw: fix brw_nir_fs_needs_null_rt helper
In 9b42215e0d ("iris: ensure null render target for specific cases") I
wrongly assumed that writing gl_SampleMask would only happen in
multisampled cases.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 9b42215e0d ("iris: ensure null render target for specific cases")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13292
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35313>
2025-06-04 10:10:38 +00:00
Iván Briano
27a2f6d1ff brw: add lowering passes for FS barycentric inputs
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34445>
2025-05-20 20:57:59 +00:00
Iván Briano
acdd30a9da brw: check if the FS needs vertex_attributes_bypass to be set
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34445>
2025-05-20 20:57:58 +00:00
Lionel Landwerlin
c467444670 brw/nir: use a new intrinsic for fs_msaa_flag
Avoid NIR code doing offset computations.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34109>
2025-05-08 06:48:34 +00:00
Lionel Landwerlin
b64f237dc4 brw: move helper to brw_nir.c
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34109>
2025-05-08 06:48:34 +00:00
Lionel Landwerlin
63f633557f intel: fix null render target setup logic
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Or current render target cache setting is to key on the binding table
index, meaning the HW associates a number in the range [0, 7] to a
RENDER_SURFACE_STATE description. If you want change the render target
0 between 2 draw calls, you need to insert a PIPE_CONTROL in between
the 2 draw calls with pb-stall + rt-flush in order to flush an writes
to a previous RENDER_SURFACE_STATE that has now becomed disassociated
with the [0, 7] number.

This PIPE_CONTROL taking care of the flush is dealt with in
cmd_buffer_maybe_flush_rt_writes(). This function diffs the current
BTI setup for render targets (first 0 to 7 BTIs) with what the next
fragment shader wants.

The issue here is we might have a render pass with 0 color attachments
and yet in 98cdb9349a we added one pointing to the render target 0,
but in the emit_binding_table() when we finally program the BTI, we
check the render pass color count and program a null surface state
instead of an actual surface state. And this leads to hangs because
the render target cache will end up with inconsistent state data.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: 98cdb9349a ("anv: ensure null-rt bit in compiler isn't used when there is ds attachment")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/12955
Reviewed-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34603>
2025-05-01 11:25:18 +00:00
Caio Oliveira
a38960e8f3 brw, nir: Use glsl_base_type instead of nir_alu_type for @dpas_intel
This will allow including types that don't have a nir_alu_type
equivalent, like bfloat16.

Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34105>
2025-04-29 16:29:37 +00:00
Lionel Landwerlin
4346210ae6 brw: move texture offset packing to NIR
That way we can deal with upcoming non constant values for
VK_KHR_maintenance8.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33138>
2025-03-29 02:15:18 +00:00
Lionel Landwerlin
67ae49dede intel: move lower_texture to brw
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33138>
2025-03-29 02:15:18 +00:00
Sagar Ghuge
191d1e7345 intel/compiler: Don't lower 64bit data memory access on LSC
Some checks are pending
macOS-CI / macOS-CI (dri) (push) Waiting to run
macOS-CI / macOS-CI (xlib) (push) Waiting to run
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34189>
2025-03-28 03:07:56 +00:00
Sagar Ghuge
1bfe2571f5 intel/compiler: Lower sample index into coord for MSRT messages
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32690>
2025-03-07 23:06:14 +00:00
Lionel Landwerlin
ce7208c3ee brw: add support for texel address lowering
The expectations are :
  - no MSAA images
  - a single tiling mode is used when not linear

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32676>
2025-02-23 15:16:50 +00:00
Lionel Landwerlin
b25e050ec7 brw: add support for 64bit storage images load/store
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32676>
2025-02-23 15:16:50 +00:00
Lionel Landwerlin
6768eb31e5 intel: rework CL pre-compile
Stolen from asahi_clc :)

We drop the nasty LLVM17+ workaround code (Thanks Alyssa!)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Tested-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Dylan Baker <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33014>
2025-01-25 03:28:07 +00:00
Lionel Landwerlin
bf8a1e1e71 brw/elk: move internal kernel parsing out of intel_clc
So it can be called internally.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Tested-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Dylan Baker <None>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33014>
2025-01-25 03:28:07 +00:00
Marek Olšák
c21bc65ba7 nir/opt_load_store_vectorize: make hole_size signed to indicate overlapping loads
A negative hole size means the loads overlap. This will be used by drivers
to handle overlapping loads in the callback easily.

Reviewed-by: Mel Henning <drawoc@darkrefraction.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32699>
2025-01-01 00:03:55 +00:00
Caio Oliveira
056b14b882 intel/brw: Move two NIR passes to brw_nir.c
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32799>
2024-12-30 20:18:23 +00:00
Caio Oliveira
abe41b1d2c intel/compiler: Use #pragma once instead of header guards
Acked-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32534>
2024-12-11 19:47:44 +00:00
Lionel Landwerlin
97b17aa0b1 brw/nir: rework inline_data_intel to work with compute
This intrinsic was initially dedicated to mesh/task shaders, but the
mechanism it exposes also exists in the compute shaders on Gfx12.5+.

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31508>
2024-10-17 19:35:59 +00:00
Lionel Landwerlin
be3f62af15 brw: remove unused prototype
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31508>
2024-10-17 19:35:59 +00:00
Marek Olšák
02923e237d nir: add hole_size parameter into the vectorize callback
It will be used to allow merging loads with a hole between them.

Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29398>
2024-10-15 05:50:24 +00:00
Lionel Landwerlin
9b42215e0d iris: ensure null render target for specific cases
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31196>
2024-09-23 15:56:02 +00:00
Kenneth Graunke
8a6903e50d intel/brw: Rename lsc_aop_for_nir_intrinsic to "op" instead of "aop"
This is going to handle more than atomics shortly.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30828>
2024-09-12 20:54:36 +00:00
Ian Romanick
119801e647 intel/brw: Move fsat instructions closer to the source
Intel GPUs have a saturate destination modifier, and
brw_fs_opt_saturate_propagation tries to replace explicit saturate
operations with this destination modifier. That pass is limited in
several ways. If the source of the explicit saturate is in a different
block or if the source of the explicit saturate is live after the
explicit saturate, brw_fs_opt_saturate_propagation will be unable to
make progress.

This optimization exists to help brw_fs_opt_saturate_propagation make
more progress. It tries to move NIR fsat instructions to the same block
that contains the definition of its source. It does this only in cases
where it will not create additional live values. It also attempts to do
this only in cases where the explicit saturate will ultimiately be
converted to a destination modifier.

v2: Fix metadata_preserve when theres no progress and use
nir_metadata_control_flow when there is progress. All suggested by
Alyssa.

v3: Fix a typo in the file header comment. Noticed by Ken.  Don't
require nir_metadata_instr_index. Use nir_def_rewrite_uses_after instead
of open-coding something slightly more specific. Both suggested by Ken.

shader-db:

All Intel platforms had similar results. (Meteor Lake shown)
total instructions in shared programs: 19733645 -> 19733028 (<.01%)
instructions in affected programs: 193300 -> 192683 (-0.32%)
helped: 246
HURT: 1
helped stats (abs) min: 2 max: 48 x̄: 2.51 x̃: 2
helped stats (rel) min: 0.18% max: 0.39% x̄: 0.33% x̃: 0.34%
HURT stats (abs)   min: 1 max: 1 x̄: 1.00 x̃: 1
HURT stats (rel)   min: 0.31% max: 0.31% x̄: 0.31% x̃: 0.31%
95% mean confidence interval for instructions value: -2.87 -2.13
95% mean confidence interval for instructions %-change: -0.34% -0.32%
Instructions are helped.

total cycles in shared programs: 916180971 -> 916264656 (<.01%)
cycles in affected programs: 30197180 -> 30280865 (0.28%)
helped: 194
HURT: 142
helped stats (abs) min: 1 max: 21251 x̄: 872.75 x̃: 19
helped stats (rel) min: <.01% max: 23.17% x̄: 2.59% x̃: 0.23%
HURT stats (abs)   min: 1 max: 28058 x̄: 1781.68 x̃: 399
HURT stats (rel)   min: <.01% max: 37.21% x̄: 4.85% x̃: 1.63%
95% mean confidence interval for cycles value: -196.84 694.97
95% mean confidence interval for cycles %-change: -0.17% 1.27%
Inconclusive result (value mean confidence interval includes 0).

fossil-db:

Meteor Lake, DG2, and Tiger Lake had similar results. (Meteor Lake shown)
Totals:
Instrs: 151512021 -> 151511351 (-0.00%); split: -0.00%, +0.00%
Cycle count: 17209013596 -> 17209840995 (+0.00%); split: -0.02%, +0.02%
Max live registers: 32013312 -> 32013549 (+0.00%)
Max dispatch width: 5512304 -> 5512136 (-0.00%)

Totals from 774 (0.12% of 630172) affected shaders:
Instrs: 1559285 -> 1558615 (-0.04%); split: -0.05%, +0.01%
Cycle count: 1312656268 -> 1313483667 (+0.06%); split: -0.24%, +0.30%
Max live registers: 82195 -> 82432 (+0.29%)
Max dispatch width: 6664 -> 6496 (-2.52%)

Ice Lake
Totals:
Instrs: 151416791 -> 151416137 (-0.00%); split: -0.00%, +0.00%
Cycle count: 15162468885 -> 15163298824 (+0.01%); split: -0.00%, +0.01%
Max live registers: 32471367 -> 32471603 (+0.00%)
Max dispatch width: 5623752 -> 5623712 (-0.00%)

Totals from 733 (0.12% of 635598) affected shaders:
Instrs: 877965 -> 877311 (-0.07%); split: -0.09%, +0.01%
Cycle count: 190763628 -> 191593567 (+0.44%); split: -0.21%, +0.64%
Max live registers: 72067 -> 72303 (+0.33%)
Max dispatch width: 6216 -> 6176 (-0.64%)

Skylake
Totals:
Instrs: 140794845 -> 140794075 (-0.00%); split: -0.00%, +0.00%
Cycle count: 14665159301 -> 14665320514 (+0.00%); split: -0.00%, +0.01%
Max live registers: 31783341 -> 31783662 (+0.00%); split: -0.00%, +0.00%

Totals from 659 (0.11% of 625670) affected shaders:
Instrs: 829061 -> 828291 (-0.09%); split: -0.09%, +0.00%
Cycle count: 185478478 -> 185639691 (+0.09%); split: -0.33%, +0.41%
Max live registers: 67491 -> 67812 (+0.48%); split: -0.01%, +0.48%

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29774>
2024-08-09 14:26:10 -07:00
Lionel Landwerlin
9a36278475 intel/nir: add printf lowering
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25814>
2024-05-15 13:13:38 +00:00
Lionel Landwerlin
dde91d18c2 intel/nir: remove unused prototypes
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25814>
2024-05-15 13:13:37 +00:00
Ian Romanick
3f151c03af intel/brw: Handle fsign optimization in a NIR algebraic pass
This is a lot less code, and it makes it easier to experiment with other
pattern-based optimizations in the future.

The results here are nearly identical to the results I got from Ken's
"intel/brw: Make fsign (for 16/32-bit) in SSA form"... which are not
particularly good.

In this commit and in Ken's, all of the shader-db shaders hurt for
spills and fills are from Deus Ex Mankind Divided. Each shader has a
bunch of texture instructions with a single fsign between the
blocks. With the dependency on the flag removed, the scheduler puts all
of the texture instructions at the start... and there are a LOT of them.

shader-db:

All Intel platforms had similar results. (Meteor Lake shown)
total instructions in shared programs: 19647060 -> 19650207 (0.02%)
instructions in affected programs: 734718 -> 737865 (0.43%)
helped: 382 / HURT: 1984

total cycles in shared programs: 823238442 -> 822785913 (-0.05%)
cycles in affected programs: 426901157 -> 426448628 (-0.11%)
helped: 3408 / HURT: 3671

total spills in shared programs: 3887 -> 3891 (0.10%)
spills in affected programs: 256 -> 260 (1.56%)
helped: 0 / HURT: 4

total fills in shared programs: 3236 -> 3306 (2.16%)
fills in affected programs: 882 -> 952 (7.94%)
helped: 0 / HURT: 12

LOST:   37
GAINED: 34

fossil-db:

DG2 and Meteor Lake had similar results. (Meteor Lake shown)
Totals:
Instrs: 154005469 -> 154008294 (+0.00%); split: -0.00%, +0.00%
Cycle count: 17551859277 -> 17554293955 (+0.01%); split: -0.02%, +0.04%
Spill count: 142078 -> 142090 (+0.01%)
Fill count: 266761 -> 266729 (-0.01%); split: -0.02%, +0.01%
Max live registers: 32593578 -> 32593858 (+0.00%)
Max dispatch width: 5535944 -> 5536816 (+0.02%); split: +0.02%, -0.01%

Totals from 5867 (0.93% of 631350) affected shaders:
Instrs: 5475544 -> 5478369 (+0.05%); split: -0.04%, +0.09%
Cycle count: 1649032029 -> 1651466707 (+0.15%); split: -0.24%, +0.39%
Spill count: 26411 -> 26423 (+0.05%)
Fill count: 57364 -> 57332 (-0.06%); split: -0.10%, +0.04%
Max live registers: 431561 -> 431841 (+0.06%)
Max dispatch width: 49784 -> 50656 (+1.75%); split: +2.38%, -0.63%

Tiger Lake
Totals:
Instrs: 149530671 -> 149533588 (+0.00%); split: -0.00%, +0.00%
Cycle count: 15261418953 -> 15264764921 (+0.02%); split: -0.00%, +0.03%
Spill count: 60317 -> 60316 (-0.00%); split: -0.02%, +0.01%
Max live registers: 32249201 -> 32249464 (+0.00%)
Max dispatch width: 5540608 -> 5540584 (-0.00%)

Totals from 5862 (0.93% of 630309) affected shaders:
Instrs: 4740800 -> 4743717 (+0.06%); split: -0.04%, +0.10%
Cycle count: 566531248 -> 569877216 (+0.59%); split: -0.13%, +0.72%
Spill count: 11709 -> 11708 (-0.01%); split: -0.09%, +0.08%
Max live registers: 424560 -> 424823 (+0.06%)
Max dispatch width: 50304 -> 50280 (-0.05%)

Ice Lake
Totals:
Instrs: 150499705 -> 150502608 (+0.00%); split: -0.00%, +0.00%
Cycle count: 15105629116 -> 15105425880 (-0.00%); split: -0.00%, +0.00%
Spill count: 60087 -> 60090 (+0.00%)
Fill count: 100542 -> 100541 (-0.00%); split: -0.00%, +0.00%
Max live registers: 32605215 -> 32605495 (+0.00%)
Max dispatch width: 5617752 -> 5617792 (+0.00%); split: +0.00%, -0.00%

Totals from 5882 (0.93% of 634934) affected shaders:
Instrs: 4737206 -> 4740109 (+0.06%); split: -0.04%, +0.10%
Cycle count: 598882104 -> 598678868 (-0.03%); split: -0.08%, +0.05%
Spill count: 10278 -> 10281 (+0.03%)
Fill count: 22504 -> 22503 (-0.00%); split: -0.01%, +0.01%
Max live registers: 424184 -> 424464 (+0.07%)
Max dispatch width: 50216 -> 50256 (+0.08%); split: +0.25%, -0.18%

Skylake
Totals:
Instrs: 139092612 -> 139095257 (+0.00%); split: -0.00%, +0.00%
Cycle count: 14533550285 -> 14533544716 (-0.00%); split: -0.00%, +0.00%
Spill count: 58176 -> 58172 (-0.01%)
Fill count: 95877 -> 95796 (-0.08%)
Max live registers: 31924594 -> 31924874 (+0.00%)
Max dispatch width: 5484568 -> 5484552 (-0.00%); split: +0.00%, -0.00%

Totals from 5789 (0.93% of 625512) affected shaders:
Instrs: 4481987 -> 4484632 (+0.06%); split: -0.04%, +0.10%
Cycle count: 578310124 -> 578304555 (-0.00%); split: -0.05%, +0.05%
Spill count: 9248 -> 9244 (-0.04%)
Fill count: 19677 -> 19596 (-0.41%)
Max live registers: 415340 -> 415620 (+0.07%)
Max dispatch width: 49720 -> 49704 (-0.03%); split: +0.10%, -0.13%

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29095>
2024-05-14 01:28:20 +00:00
Caio Oliveira
865ef36609 intel/brw: Remove brw_shader.h
Find a better home for its existing content.  Some functions are
now just static functions at the usage sites.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27861>
2024-02-29 19:28:06 +00:00
Caio Oliveira
63a4a4400a intel/brw: Remove edgeflag_is_last VS parameter
Suggested by Ken.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27691>
2024-02-28 05:45:39 +00:00
Caio Oliveira
5a3f65e678 intel/brw: Remove unused attrib workarounds
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27691>
2024-02-28 05:45:39 +00:00
Caio Oliveira
a1e694a890 intel/brw: Remove Gfx8- code from NIR passes
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27691>
2024-02-28 05:45:38 +00:00
Caio Oliveira
1ee29f82d2 intel/brw: Remove Gfx8- code from lower storage image pass
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27691>
2024-02-28 05:45:38 +00:00
Caio Oliveira
7c23b90537 intel/brw: Always use scalar shaders
Remove scalar_stage[] array, since now it is always scalar.  This
removes any usage of vec4 shaders in brw.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27691>
2024-02-28 05:45:37 +00:00
Lionel Landwerlin
cf193af762 anv: fixup push descriptor shader analysis
There are a couple mistakes here :

   - using a bitfield as an index to generate a bitfield...

   - in anv_nir_push_desc_ubo_fully_promoted(), confusing binding
     table access of the descriptor buffer with actual descriptors

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Fixes: ff91c5ca42 ("anv: add analysis for push descriptor uses and store it in shader cache")
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27504>
2024-02-19 11:10:29 +00:00
Caio Oliveira
dc76cfc781 intel/compiler: Collect NIR-only passes in intel_nir.h
Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27644>
2024-02-16 22:35:05 +00:00
Caio Oliveira
c5b80de583 intel/compiler: Rename brw_vue_map to intel_vue_map
And move to the intel_shader_enums.h file.

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27475>
2024-02-14 22:31:23 -08:00
Lionel Landwerlin
2a1ff08376 intel/compiler: make default NIR compiler options visible
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26797>
2024-02-13 00:06:45 +00:00
Sagar Ghuge
73a3257968 intel/compiler: Add texture operation lowering pass
This pass combines the LOD or LOD bias and array index into a single
32-bit value since Xe2+ sampler messages requires us to do that.

v2: (Alyssa)
- Use nir_iand_imm instead of nir_iand and nir_imm_int
- Use nir_trim_vector instead of nir_swizzle

Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27458>
2024-02-12 21:25:48 +00:00
Kenneth Graunke
10ed4f1cab intel/nir: Pass devinfo and prog_data to brw_nir_lower_cs_intrinsics
We'll want to check for Alchemist and set various prog_data fields
in the next patch, in order to enable some optimizations.  Passing
NULL for prog_data will remain valid and continue working as before.

Reviewed-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27167>
2024-01-25 08:43:04 +00:00
Caio Oliveira
dba6451ce8 intel/cmat: Add pass to lower cooperative matrix to subgroup operations
This is just the skeleton of the implementation. Future commits will
fill it all in.

v2: Move to src/intel/compiler

v3 (idr): Use vecN instead of array[N] for slice type.

v4 (idr): Refactor lower_cooperative_matrix_load and
lower_cooperative_matrix_store into a single function.

v5 (idr): Remove old, verbose debug logging. Assert that entry is not
NULL in get_coop_type_for_slice. Use nir_component_mask(...) instead of
0xffff. s/cooperative_matrix/cmat/. All suggested by Caio.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>

I put both R-b on this because, at this point, we've each done equal
parts authoring and reviewing.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25994>
2023-12-29 20:24:15 -08:00
Caio Oliveira
f4601d82c1 intel/compiler: Remove unused parameter from brw_nir_analyze_ubo_ranges()
This parameter was used by i965 driver that is now gone.

Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25986>
2023-11-08 18:10:31 +00:00
Caio Oliveira
d2125dac85 intel/compiler: Take more precise params in brw_nir_optimize()
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25986>
2023-11-08 18:10:31 +00:00