nir_build_frag_coord generates the correct sysval loads based on NIR
options. nir_load_frag_coord shouldn't be used directly because drivers
don't have to support it.
v2: RADV can't use it because nir->options isn't set, so use load_pixel_coord.
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/41227>
First, the surface dimensions are used to determine the range of valid
pages that the data in the buffer overlaps, then rows are removed from
the surface until it does not overfetch into any neighboring pages. If
any rows were removed, an extra BTI is set up with a texel buffer that
views the contents of all the rows that were removed, and the shader is
compiled with a branch to sample the last rows through the texel buffer
instead of the main surface.
Using the texel buffer allows it to access the last rows without dealing
with overfetch or weird alignment hacks, and restricting texel buffer
usage to just the part of the surface that can't be accessed safely
ensures that we don't significantly impact performance for any buffer to
image copy that is unlucky enough to be close to a page boundry.
Co-authored-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40149>
Since we now have a ISL_SURF_USAGE_NO_OVERFETCH_PADDING_BIT flag to turn extra
padding calculations on and off, we can align the row pitch of linear surfaces
that are accessed through the sampler to minimize the number of L3 cachelines
that each sampler cacheline overlaps for added efficiency.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40149>
Bspec 58779 describes various cases where additional padding is required on the
bottom and right sides of a sampling engine surface to avoid page faults.
Since we don't want to mess up the other drivers that also use ISL, there's now
a requires_padding boolean in isl_dev that can be used to enable/disable the
extra padding calculations per device and driver.
The extra padding can also be disabled per-surface by adding the usage flag
ISL_SURF_USAGE_NO_OVERFETCH_PADDING_BIT, like when a specific row pitch is
needed.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/40149>
When copying data between two surfaces, independently increase the size
of each surface's format (bits-per-pixel) as alignment constraints
allow. Adjust the other surface parameters and blorp_copy() parameters
accordingly.
This fixes copies between the 16bpp YCRCB formats and 32bpp formats:
dEQP-VK.ycbcr.single_plane_copy.linear.linear.r8g8b8a8_to_g8b8g8r8_422
This new test failure was reported by Iván Briano.
More generally, this increases the efficiency of our copies. As shown in
the configuration pages of the PRMs, our sampler is able to fetch texels
at a fixed rate of texels / clock regardless of the texel size
(presumably our rendering hardware has similar behavior). By using the
largest texel size possible, we can transfer more bits / clock.
Improves the performance of a number of traces in the performance CI for
BMG:
* TotalWarWarhammer3 +2.24%
* Payday3 +1.87%
* BaldursGate3 +1.34%
* Control +1.25%
* TotalWarPharaoh +1.22%
Four additional traces are helped between +0.44% and +0.96%.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39974>
Don't use it without ISL_AUX_USAGE_STC_CCS. With a future patch, this
will allow blorp_copy() calls to increase the size of the surface format
for CPB.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39974>
blorp_copy() will soon start changing the format in a way which drivers
cannot rely on to do things like manage the texture cache (see iris).
Narrow down the scope of blorp_copy_get_formats() and
blorp_copy_get_color_format() such that the returned value can only be
trusted if compression would be enabled on each image.
By doing this (and adapting iris to reflect this), we'll get the
required flushes on the platforms which need
WaSamplerCacheFlushBetweenRedescribedSurfaceReads:
* On the platforms which need the workaround for all formats,
blorp_copy() will stick with the queried format on compressed
surfaces.
* On the platforms which need the workaround when switching from ASTC
and non-ASTC formats, blorp_copy() may actually change the queried
format on compressed surfaces. This is not a problem, because
surfaces which may be read with ASTC formats are not compressible.
Prevents gfx9 from failing tests under:
* KHR-GL46.copy_image.functional_src_target_texture_2d_array_src_format_r3_g3_b2*
* KHR-GL46.copy_image.functional_src_target_texture_2d_array_src_format_rgb5*
* KHR-GL46.copy_image.functional_src_target_texture_2d_array_src_format_rgba2*
* KHR-GL46.copy_image.functional_src_target_texture_2d_array_src_format_rgba4*
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39974>
We're going to start changing the surface format during blorp_copy().
Changing the surface format could lead to incorrect image alignment
parameters, so return a fixed halign and valign for images with a single
subresource. That's all that will be needed for the upcoming
blorp_copy() changes.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39974>
It's pretty much the same mechanism, except it's a different register
location.
With this change we gain indirect loading support.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39405>
This is the program data for the fragment shader. Nobody even knows
what the windowizer/masker unit is or does anymore. Even on Gen4-6,
"fs" is still clearer. This makes the codebase easier to read.
This is only about 15 years overdue.
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39748>
Tested on PTL, fixes various copy_and_blit tests that utilize compute
after ab9d3528dc that exposed this to them.
Fixes: ab9d3528dc ("anv: fix queue check in anv_blorp_execute_on_companion on xe3")
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39548>
This gives us the infrastructure that allows us to slowly migrate
pieces of blorp shaders from NIR to OpenCL, which, IMHO, are much
easier to read. We can't fully migrate everything due to all the
conditional building we do with these shaders, but I'm sure we'll find
opportunities to replace some NIR with OpenCL eventually.
The conversion of blorp_check_in_bounds() serves as the first example.
I also plan to have the shaders from the new indirect copy extension
be OpenCL shaders (mixed with some NIR as well), so having this patch
merged now will reduce the diff for the extension later.
Thanks to Alyssa Rosenzweig for her help here.
v2:
- Use SPDX (Alyssa).
- Use nir_trim_vector() (Alyssa).
- Adjust CL variable declaration (Alyssa).
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39046>
We have two distinct code paths sharing blorp_params->wm_inputs for
different purposes: the code from blorp_blit.c and the code from
blorp_clear.c. While blorp_blit.c uses most of the parameters (all
except clear_color), blorp_clear.c only uses clear_color and
bounds_rect. Split the parameters in two structs: one for blits and
the other for clears.
This not only helps save some space in the shader inputs, but it also
organizes things so it's more clear which parameters are used by what.
In addition, my plan is to later add struct blorp_wm_inputs_indirect,
which won't share anything that the others use, and would otherwise
grow the struct even more.
This change would reduce the size of struct blorp_wm_inputs from 96 to
80, but we have to add padding due to the assertion that compares it
to cs_prog_data->push.cross_thread.size. Still good, though.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39046>
I'm sorry, but I have OCD and the rest of the file is nicely aligned.
Reviewed-by: José Roberto de Souza <jose.souza@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39046>
We are passing number of layers as inline parameter register, so figure
out z_pos and write to 2D MSAA array images in compute
shader. We already get component X, Y and sample index, all we needed
was the number of layers.
Ken:
- Use load/store var instead of derefs
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33905>
Only 3D shader gets dispatched per sample not the compute shader.
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33905>
This is done by grep ALIGN( to align(
docs,*.xml,blake3 is excluded
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Alyssa Rosenzweig <alyssa.rosenzweig@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38365>
One advantage here of moving a bunch of stuff to NIR is that we can
now have consistent payload types straight from the NIR conversion to
BRW.
This massively simplifies the BRW lowering code and avoids type errors
that are quite common to make in the backend.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37527>
It's not only for GL, change to a generic name.
Use command:
find . -type f -not -path '*/.git/*' -exec sed -i 's/\bgl_shader_stage\b/mesa_shader_stage/g' {} +
Acked-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36569>
In the C23 standard unreachable() is now a predefined function-like
macro in <stddef.h>
See https://android.googlesource.com/platform/bionic/+/HEAD/docs/c23.md#is-now-a-predefined-function_like-macro-in
And this causes build errors when building for C23:
-----------------------------------------------------------------------
In file included from ../src/util/log.h:30,
from ../src/util/log.c:30:
../src/util/macros.h:123:9: warning: "unreachable" redefined
123 | #define unreachable(str) \
| ^~~~~~~~~~~
In file included from ../src/util/macros.h:31:
/usr/lib/gcc/x86_64-linux-gnu/14/include/stddef.h:456:9: note: this is the location of the previous definition
456 | #define unreachable() (__builtin_unreachable ())
| ^~~~~~~~~~~
-----------------------------------------------------------------------
So don't redefine it with the same name, but use the name UNREACHABLE()
to also signify it's a macro.
Using a different name also makes sense because the behavior of the
macro was extending the one of __builtin_unreachable() anyway, and it
also had a different signature, accepting one argument, compared to the
standard unreachable() with no arguments.
This change improves the chances of building mesa with the C23 standard,
which for instance is the default in recent AOSP versions.
All the instances of the macro, including the definition, were updated
with the following command line:
git grep -l '[^_]unreachable(' -- "src/**" | sort | uniq | \
while read file; \
do \
sed -e 's/\([^_]\)unreachable(/\1UNREACHABLE(/g' -i "$file"; \
done && \
sed -e 's/#undef unreachable/#undef UNREACHABLE/g' -i src/intel/isl/isl_aux_info.c
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36437>
Allow CCS for non-display linear surfaces in isl_surf_supports_ccs().
We're going to rely more on the helper to determine CCS-enabling for Xe2
on iris.
Reviewed-by: Jianxun Zhang <jianxun.zhang@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32120>
Use get_copy_format_for_bpb() instead of
get_ccs_compatible_uint_format() when performing blorp_copy(). This
matches the code path taken on gfx20 and increases the testing of cases
which would impact gfx12.0 in isl_get_sampler_clear_field_offset().
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35329>
Through testing, I've found that the sampler will fetch the clear color
pixel from the converted clear color field in more cases. So, stop
reporting the raw dword offset for them:
* On gfx12.5, for 32-bpc color images.
* On gfx11-12.0, for 64-bpp color images.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35329>
Unify code which creates surfaces from buffers. The behavior is slightly
changed to use array layers to enable arrayed buffer clears (as needed).
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33776>
Compressed CPS surfaces operations such as copies and clears need to be
handled through the depth stencil hw to ensure that the aux data is
handled correctly.
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20741>
Currently blorp assumes that copies of depth/stencil is restricted
to/from depth/stencil formats. We want to allow color<->depth copies.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Acked-by: Ivan Briano <ivan.briano@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31983>
Request a fixed subgroup size for pixel shaders that require it due to
the hardware restrictions of fast clears and repeated data clears.
This requires plumbing the "is_fast_clear" boolean across several
callers since blorp_compile_fs_brw() currently has no information
regarding whether the kernel is intended for a fast clear operation.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32664>
In iris, this should avoid some partial resolves when copying between
images. In anv, this will reduce restrictions on dmabufs which have
clear color support in the next patch.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31136>
blorp_copy_get_formats() tries to make the source and destination view
formats match as much as possible. This avoids some casting in the copy
shader, but it makes determining the format that will be used for a
surface impossible without having the ISL surface for both that surface
and a source or destination.
We'd like to enable the Vulkan driver to know as early as possible what
format an image may be reinterpreted as for correctness. So, determine
the copy formats more independently and expose a helper which does so
for drivers.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31136>
Previously, we left it NULL until later in the compile. However, some
builder helpers are starting to check the options and they blow up when
options == NULL.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27617>
Allocate the prog_data instead of expecting one from the user, also
explicitly return both kernel and prog_data size, so that the
plumbing code isn't required to use the exact prog_data type.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27581>
Remove the brw_ prefix from some blorp functions and structs to avoid
confusion with what's related to brw and what's not. Some of those will
become independent of brw in upcoming patches.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27581>
Xe2+ changed the msaa mapping for 2D/3D Tile64 surfaces, introduce a
Xe2+ specific enum to handle this change.
Signed-off-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27113>
XY_BLOCK_COPY_BLT and XY_FAST_COLOR_BLT instruction supports AUX_CCS
surface mode. Stencil buffer compression is stencil data with CCS so we
can easily support that on blitter engine.
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26167>
The data in Ys/Yf/Tile64 tiled images is arranged differently for 2D &
3D images. Therefore we cannot assume that we will use 2D images for
blits/copies.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Nanley Chery <nanley.g.chery@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/23620>
