Thanks to Felix Degrood for discovering that we missed enabling this
additional caching on Tigerlake! Felix also benchmarked the changes.
We now use MOCS 48 (HDC:L1 + L3 + LLC) for render targets, textures,
and pull constant buffers. We leave storage buffers & images, as well
as stateless messages, using the previous MOCS 2 value. We can't use
HDC:L1 with atomics, and we don't know a priori whether storage buffers
will be used with atomics or not. Similarly, the Vulkan buffer device
address feature allows atomics to be performed on buffers via stateless
messages, and we only can control MOCS at the base address level, so
we can't do much there.
This is closer to what the Windows Vulkan and OpenGL drivers do,
though it isn't quite the same - they also disable LLC in some cases,
but we observed this to have noticable performance regressions when
we tried (though a couple titles benefited). We may try experiment
with that in the future.
Improves performance in a number of titles:
- Unreal Engine 4 Shooter Demo [VK]: 11.8%
- Witcher 3 [DXVK]: 3.9%
- Rise of the Tomb Raider [VK]: 1.5%
- Shadow of the Tomb Raider [VK]: 1.0%
- Grand Theft Auto V [DXVK]: 0.8%
We did not observe any performance regressions.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7104>
On Gen12+, we can enable additional caches in certain usage situations.
This routes that decision making to a central place in ISL, based on
surface usage flags, and updates both drivers to use it. (i965 doesn't
need to change because it doesn't support Gen12.)
We continue handling the "external" decision via an anv_mocs() wrapper
for now, since we store that flag in anv_bo, which isl doesn't know
about. (We could introduce an ISL_SURF_USAGE_EXTERNAL, but I'm not
actually sure that would be cleaner.)
This patch should not have any functional nor performance effects, as
we continue selecting the exact same MOCS values for now.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7104>
Most uses of this function deal with destination buffers, but for
copy_buffer_to_image, the buffer is the source, and isn't rendered
to. We should avoid setting ISL_SURF_USAGE_RENDER_TARGET_BIT.
Also, we should avoid setting ISL_SURF_USAGE_TEXTURE_BIT for the
destination, which isn't sampled from.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7104>
It's a bit hard to exactly map our implementation to the limits
described by Vulkan. The bindless surface handle in the extended
message descriptors is 20 bits and it's an index into the table of
RENDER_SURFACE_STATE structs that starts at bindless surface base
address. This means that we can have at must 1M surface states
allocated at any given time. Since most image views take two
descriptors, this means we have a limit of about 500K image views.
However, since we allocate surface states at vkCreateImageView time,
this means our limit is actually something on the order of 500K image
views allocated at any time. The actual limit describe by Vulkan, on
the other hand, is a limit of how many you can have in a descriptor set.
Assuming anyone using 1M descriptors will be using the same image view
twice a bunch of times (or a bunch of null descriptors), we can safely
advertise a larger limit. 1M is what's required by D3D12, so let's
advertise that.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3335
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7180>
According to the Vulkan specification, the
VK_COMMAND_BUFFER_USAGE_RENDER_PASS_CONTINUE_BIT flag will be ignored if
included in a VkCommandBufferBeginInfo for a primary command buffer.
This also implies pBeginInfo->pInheritanceInfo should not be read even
if the flag is present, and makes it legal to include the flag knowing
it will be ignored.
Signed-off-by: Ricardo Garcia <rgarcia@igalia.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7128>
For terminate operation, jump the invocation without predicating on
the rest of the quad being disabled -- which is what is done for
demote and discard.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7150>
The enables removal of gen_device_info::is_cannonlake.
v2: Remove GEN10_FEATURES and GEN10_HW_INFO macros. Suggested by
Lionel.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6899>
v2: Re-wrap lines in meson.build. Suggested by Jason.
v3: Also update Makefile.sources and Android build files. Noticed by
Lionel.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net> [v2]
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6899>
I considered a couple other options (including adding #if / #endif
around UNUSED and adding an UNUSED_ON_SOME_GEN), but this seemed the
best. There was also at least one other case of having UNUSED on a
paramter that is sometimes unused (params in
blorp_emit_color_calc_state).
This header gets included in a lot of places (esp. in files that get
built per-Gen), so the warnings are repeated a lot.
In file included from src/mesa/drivers/dri/i965/genX_blorp_exec.c:33:
src/intel/blorp/blorp_genX_exec.h: In function ‘emit_urb_config’:
src/intel/blorp/blorp_genX_exec.h:193:48: warning: unused parameter ‘deref_block_size’ [-Wunused-parameter]
193 | enum gen_urb_deref_block_size *deref_block_size)
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
src/intel/blorp/blorp_genX_exec.h: In function ‘blorp_fill_vertex_buffer_state’:
src/intel/blorp/blorp_genX_exec.h:350:52: warning: unused parameter ‘batch’ [-Wunused-parameter]
350 | blorp_fill_vertex_buffer_state(struct blorp_batch *batch,
| ~~~~~~~~~~~~~~~~~~~~^~~~~
src/intel/blorp/blorp_genX_exec.h: In function ‘blorp_emit_surface_state’:
src/intel/blorp/blorp_genX_exec.h:1403:42: warning: unused parameter ‘aux_op’ [-Wunused-parameter]
1403 | enum isl_aux_op aux_op,
| ~~~~~~~~~~~~~~~~^~~~~~
src/intel/blorp/blorp_genX_exec.h: In function ‘blorp_update_clear_color’:
src/intel/blorp/blorp_genX_exec.h:1867:46: warning: unused parameter ‘batch’ [-Wunused-parameter]
1867 | blorp_update_clear_color(struct blorp_batch *batch,
| ~~~~~~~~~~~~~~~~~~~~^~~~~
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6899>
cs_prog_data->slm_size is basically redundant with
prog_data->total_shared, which is the field that we actually use for
controlling the shared local memory size in all drivers. We were
still using it in one place for VK_EXT_pipeline_executable_properties,
but we should just fix that and delete the field.
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7152>
Without this, we end up with indirect sampler messages all the time
because we don't propagate the texture/image BTI. This makes debugging
shaders with imageSize or textureSamples in them a pain.
Shader-db results on Ice Lake:
total instructions in shared programs: 19720612 -> 19720564 (<.01%)
instructions in affected programs: 4998 -> 4950 (-0.96%)
helped: 12
HURT: 0
All affected shaders were compute shaders in Deus Ex: Mankind Divided.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6794>
I can't find a restriction for enabling CCS on these surfaces in recent
versions of the Bspec. Since I didn't cite my source, I'm not even sure
such a restriction existed in the first place.
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7085>
The current scratch mechanism uses an MRF hack where we reserve a few
GRF registers to treat like the MRF and we collect the data into that
MRF region before doing a scratch write. We also use that region for
the header for scratch reads.
This commit changes things and gets rid of the MRF hack. Instead, we
reserve a single register (which RA is free to pick) for the scratch
header and uses split sends for scratch writes to avoid having to do
the copy. This should provide RA with more freedom in the presence of
spilling as well as avoid some unnecessary data moves. In future, the
new GEN9_SCRATCH_HEADER opcode gives us a place where we can do our own
per-thread scratch base address calculations rather than depending on
the scratch base address that gets pushed into g0. Having an opcode for
this lets us do it once at the top of the shader rather than repeating
it at every read/write.
One other noticeable difference is the use of SHADER_OPCODE_SEND. We
can get away with this thanks to the fact that we're now using a set to
track which instructions are generated by spills and don't rely on the
opcodes to find spill/fill instructions. This allows us to avoid adding
more virtual opcodes and let the normal code paths handle things like
scoreboard dependencies between header setup and the SEND. It also
means that post-RA scheduling may be able to space out the header setup
MOV and the SEND for better latency hiding.
Shader-db results on Skylake:
total spills in shared programs: 12137 -> 10604 (-12.63%)
spills in affected programs: 6685 -> 5152 (-22.93%)
helped: 274
HURT: 2
total fills in shared programs: 13065 -> 11515 (-11.86%)
fills in affected programs: 9007 -> 7457 (-17.21%)
helped: 275
HURT: 1
Shader-db results on Ice Lake:
total spills in shared programs: 12482 -> 10953 (-12.25%)
spills in affected programs: 6586 -> 5057 (-23.22%)
helped: 275
HURT: 0
total fills in shared programs: 12819 -> 11234 (-12.36%)
fills in affected programs: 7867 -> 6282 (-20.15%)
helped: 274
HURT: 0
Shader-db results on Tigerlake:
total spills in shared programs: 11689 -> 10233 (-12.46%)
spills in affected programs: 4740 -> 3284 (-30.72%)
helped: 259
HURT: 0
total fills in shared programs: 10840 -> 9443 (-12.89%)
fills in affected programs: 6244 -> 4847 (-22.37%)
helped: 259
HURT: 0
Fossil-db results on Ice Lake:
Spills in all programs: 245249 -> 201633 (-17.8%)
Fills in all programs: 366066 -> 314368 (-14.1%)
More practically, this seems to give about a 0.5-1% perf boost in
Witcher 3 (DXVK) and Shadow of the Tomb Raider (Vulkan native).
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7084>
Starting with e99081e76d, we don't re-construct liveness information
every time we spill a register. Instead, we're very careful to track
which instructions are spill instructions and not contribute those to
the IP count so that we can continue to use the old liveness information
even though instructions have been added. This commit adds an assert
that sanity-checks that we count the same number of instructions as our
liveness information is based on.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7084>
This opcode is responsible for setting up the buffer base address and
per-thread scratch space fields of a scratch message header. For the
most part, it's a copy of g0 but some messages need us to zero out g0.2
and the bottom bits of g0.5.
This may actually fix a bug when nir_load/store_scratch is used. The
docs say that the DWORD scattered messages respect the per-thread
scratch size specified in gN.3[3:0] in the message header but we've been
leaving it zero. This may mean that we've been ignoring any scratch
reads/writes from a load/store_scratch intrinsic above the 1KB mark.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7084>
In theory, this fixes a bug where we were dropping the PTSS bound on the
floor. The hardware docs claim that the A32 DWORD and BYTE scattered
read/write messages do a PTSS bounds check. However, in practice, it
seems that the hardware ignores the bounds check so this doesn't
actually matter. I verified this with the following couple of piglit
tests:
https://gitlab.freedesktop.org/mesa/piglit/-/merge_requests/399
In practice, this prevents the next commit from making a subtle
behavioral change.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7084>
Because we hard-coded the default to vec4, any platform where it doesn't
have a "Dispatch Mode" field gets vec4 by default. This includes Gen11+
where vec4 is no longer a thing. Change the default so it works on
newer hardware.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7084>
Add an option to nir_lower_gs_intrinsics which tells it to track
the number of emitted primitives, not just vertices. Additionally,
also make it per-stream.
Also rename the set_vertex_count intrinsic to
set_vertex_and_primitive_count.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6964>
This reverts commit bcfec61d1e.
The previous patch fixed the underlying issue that the above commit was
actually working around. It turns out that the previously observed
performance regression was due to invalid aux-map entries for
multi-layer HiZ+CCS buffers.
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7046>
Fixes rendering corruption in the shadowmappingcascade Sascha Willems
Vulkan demo. To see the corruption, I adjusted the demo options as
follows:
1. Enable "Display depth map"
2. Set "Split lambda" to 0.100
3. Make "Cascade" non-zero.
Fixes: 80ffbe915f ("anv: Add support for HiZ+CCS")
Reviewed-by: Sagar Ghuge <sagar.ghuge@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7046>
On Gen7, the data cache is pretty terrible so we'd rather avoid it
there. On Gen8+, it should be fine and is less likely to conflict with
texturing so we should get less cache thrashing there.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/3932>
It's identical to nir_intrinsic_load_global except that it works on data
that's guaranteed to be constant throughout the shader invocation.
Fixes: ff2f44d865 "intel/fs: Implement nir_intrinsic_load_global_constant"
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6872>
In 53bfcdeecf, we added load/store_scratch instructions which deviate
a little bit from most memory load/store instructions in that we can't
use the normal untyped read/write instructions which can read and write
up to a vec4 at a time. Instead, we have to use the DWORD scattered
read/write instructions which are scalar. To handle this, we added code
to brw_nir_lower_mem_access_bit_sizes to cause them to be scalarized.
However, one case was missing: the load-as-larger-vector case. In this
case, we take small bit-sized constant-offset loads replace it with a
32-bit load and shuffle the result around as needed.
For scratch, this case is much trickier to get right because it often
emits vec2 or wider which we would then have to lower again. We did
this for other load and store ops because, for lower bit-sizes we have
to scalarize thanks to the byte scattered read/write instructions being
scalar. However, for scratch we're not losing as much because we can't
vectorize 32-bit loads and stores either. It's easier to just disallow
it whenever we have to scalarize.
Fixes: 53bfcdeecf "intel/fs: Implement the new load/store_scratch..."
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6872>
