Make the "block after DO" more stable so that adding instructions after
a DO doesn't require repairing the CFG. Use a new SHADER_OPCODE_FLOW
instruction that is a placeholder representing "go to the next block"
and disappears at code generation.
For some context, there are a few facts about how CFG currently works
- Blocks are assumed to not be empty;
- DO is always by itself in a block, i.e. starts and ends a block;
- There are no empty blocks;
- Predicated WHILE and CONTINUE will link to the "block after DO";
- When nesting loops, it is possible that the "block after DO" is
another "DO".
Reasons and further explanations for those are in the brw_cfg.c comments.
What makes this new change useful is that a pass might want to add
instructions between two DO instructions. When that happens, a new
block must be created and any predicated WHILE and CONTINUE must be
repaired.
So, instead of requiring a repair (which has proven to be tricky in
the past), this change adds a block that can be "virtually" empty but
allow instructions to be added without further changes.
One alternative design would be allowing empty blocks, that would be
a deeper change since the blocks are currently assumed to be not empty
in various places. We'll save that for when other changes are made to
the CFG.
The problem described happens in brw_opt_combine_constants, and a
different patch will clean that up.
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33536>
Shorter and a preparation to add some functionality to DO().
Had to make it const since that's the convention for builder, so
just made all the sibling helpers const too.
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33536>
The expectations are :
- no MSAA images
- a single tiling mode is used when not linear
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32676>
This is a debug feature that we kind of manage in the driver atm. It's
better that we move this completely to the compiler and can load it
from the cache.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Michael Cheng <michael.cheng@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33643>
I dumped assembly generated by our driver with INTEL_DEBUG=shaders,
copied and pasted it into a lua file, tried to run it with
src/intel/executor, but the disassembler started telling me some
instructions were invalid.
This happened because we print the "compacted" flag in our assembly
text, so when brw_gram.y parses our assembly flag, it sees the
"compacted" flag and sets it to the instruction by calling
add_instruction_option(). But the executor tool never sets the
BRW_ASSEMBLE_COMPACT flag when it calls brw_assemble(), so when
brw_assemble() calls dump_assembly(), which calls brw_disassbemble(),
the disassembler gets confused and prints misinterpreted instructions
and calls them invalid.
It is not the job of brw_gram.y (our text assembly parser) to mark
instructions as compacted. Whatever is later assembling the
instruction is the entity that should decide if the instructions are
compacted or not. So in this patch we just ignore this flag.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33614>
For the instructions we parse with brw_gram.y, don't unconditionally
call brw_eu_inst_set_cond_modifier(). Do it like we do in
brw_generator::generate_code() and only call it if we have a
cond_modifier to set.
Why? Because for ONE_SRC instructions, CondCtrl (bits 95:92) only
exists if Src.IsImm is false. If Src.Imm is true, then bits 95:64 are
actually Src0.ImmValue[63:32]. If we unconditionally call
brw_eu_inst_set_cond_modifier(), we'll end up zeroing bits 95:92 for
ONE_SRC instructions with 64bit immediates. See BSpec page
Structure_EU_INSTRUCTION_BASIC_ONE_SRC (56880).
This issue can be reproduced with src/intel/executor if you try to
have the following instruction:
mov(16) g10<1>Q 0xfedcba9876543210:Q { align1 WE_all 1H };
our parser will end up zeroing the top bits, so the value of the
immediate will be 0x0edcba9876543210.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33559>
Since Xe2, the registers are bigger and even the instruction
structures got updated to have 6 bits.
The way I detected this issue was when I tried to use
src/intel/executor to add the following instruction:
add(8) g6.8<1>UD g4<8,8,1>UD 0x00000008UD { align1 WE_all 1Q I@1 };
Executor would read this and end up emitting an add with dst being
g6<1>UD instead of what we wanted. It turns out that inside
brw_gram.y, at dstoperand and dstoperandex we do:
$$.subnr = $$.subnr * brw_type_size_bytes($4);
which would overflow subnr back to 0.
The overflow doesn't seem to be a problem with code we emit directly
(unlike the code we parse, like above) due to the fact that we seem to
treat Xe2 registers as smaller all the way until we call phys_nr() and
phys_subnr() during code generation. The phys_subnr() function can
generate a value that would overflow reg.subnr, but this value is
never written back to reg.subnr, it's just returned as an unsigned
int.
Fixes: e9f63df2f2 ("intel/dev: Enable LNL PCI IDs without INTEL_FORCE_PROBE")
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Signed-off-by: Paulo Zanoni <paulo.r.zanoni@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33539>
Normally the driver & compiler work together to use as few
3DSTATE_VERTEX_ELEMENTS/VERTEX_BUFFER_ELEMENT data as possible.
The compiler ignores unused bits and driver avoids emitting the
corresponding elements in 3DSTATE_VERTEX_ELEMENTS.
For device generated commands, we want an 3DSTATE_VERTEX_ELEMENTS
programming that is independent from the shader so that we can
implement indirect pipeline binding without complicating the
generation shader as well as emitting fewer generated commands.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32418>
It's not used outside of the compiler.
We add a new nr_attribute_regs which now seems useless but will be
useful in a later change.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32418>
Looks like we are not using the primitive leaf desc loading code part at
all. Let's just drop it.
Signed-off-by: Sagar Ghuge <sagar.ghuge@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33497>
This is just a small refactor.
Originally there was an extra commit on top of this. That commit didn't
help generated code quality, so it was dropped.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33049>
The offset is measured in bytes. Some of the code here acted as though
it were measured in src.type units. Also modify the assertion to check
that all extracted bits come from data in the immediate value.
Fixes: 580e1c592d ("intel/brw: Introduce a new SSA-based copy propagation pass")
Fixes: da395e6985 ("intel/brw: Fix extract_imm for subregion reads of 64-bit immediates")
Yes, I missed this error *twice* in code review.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33049>
The logical send lowering code sets these, and is the code which
-should- set these.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
FS_OPCODE_INTERPOLATE_AT_{SAMPLE,SHARED_OFFSET} never have a mlen set.
They are lowered to SHADER_OPCODE_SEND in logical send lowering, at
which point they acquire an mlen, but cease to be those opcodes.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
We teach lower_logical_sends to lower these to SHADER_OPCODE_SEND
and drop all the corresponding generator and eu_emit code.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
We're about to start lowering these in the IR, at which point the
scheduler will see SEND instructions with fence messages. Previously,
we handled those in the generator, and didn't handle the virtual opcodes
here, letting them fall through to the default case of 14 cycles.
These new numbers are completely fabricated, matching the times we have
for atomic operations. This is basically what we did for LSC atomics.
While it may not be accurate, it's at least better than 14 cycles.
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
Memory fences do not refer to an element of a binding table. Rather,
the reason we had "BTI" in these opcodes was to distinguish what in
modern terms are called UGM (untyped memory data cache) vs. SLM
(cross-thread shared local memory) fences.
Icelake and older platforms used the "data cache" SFID for both
purposes, distinguishing them by having a special binding table
index, 254, meaning "this is actually SLM access". This is where
the notion that fences had BTIs came in. (In fact, prior to Icelake,
separate SLM fences were not a thing, so BTI wasn't used there either.)
To avoid confusion about BTI being involved, we choose a simpler lie: we
have Icelake SLM fences target GFX12_SFID_SLM (like modern platforms
would), even though it didn't really exist back then. Later lowering
code sets it back to the correct Data Cache SFID with magic SLM binding
table index. This eliminates BTI everywhere and an unnecessary source.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
For some reason, we were using UW type for the destination of memory
fences at the generator level, while in the IR we selected UD.
There are some comments in the documentation for the message about it
writing the notification register to the destination, which is 32-bit.
Prior to Xe2, bits 31:16 were Reserved/MBZ. But on Xe2, all 32 bits
are populated with actual data.
I don't know whether this will fix anything in practice, but it seems
like a better plan to use UD. Often we used UW types to avoid having
the destination region of sends span too many registers, but we're in
SIMD1 here, so it shouldn't matter.
Acked-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
brw_memory_fence() overrides the instructions generated by the
MEMORY_FENCE or INTERLOCK opcodes to be force_writemask_all with
exec_size == 1. But the IR was emitting it in SIMD8 (regardless
of dispatch width). Instead, just emit the IR as SIMD1/NoMask so
the IR matches what we actually generate. Have size_written indicate
that the entire destination is written, however, as it is ultimately
going to be a SEND that writes a whole register.
We were also using a UD register for the source of
FS_OPCODE_SCHEDULING_FENCE when the generator overrides it to UW,
so just specify UW in the IR as well so that they line up.
Also add validation for MEMORY_FENCE/INTERLOCK that we've done the
exec_size and masking right in the IR.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
We can just specify this as a source to the logical FB read/write
opcodes. Notably FB reads had no sources before; now they have one.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
Rather than using a bit in the generic fs_inst data structure, we can
simply set a source on our logical FB write messages. (We already do
so for many other cases.)
In the repclear shader, setting this wasn't actually having an effect,
as we were setting it on a SHADER_OPCODE_SEND message which ignored it.
(We had already correctly set the bit in the message descriptor.)
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
This was used for legacy depth passthrough on older hardware. Gfx9+
doesn't actually have dst depth as part of the message, which is the
only hardware brw supports these days.
It sure looks like we were setting it though...
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
We already have logical pixel interpolator messages that get lowered
to send messages. We can just add an extra boolean source to those
opcodes rather than sticking a opcode-specific boolean in the generic
fs_inst data structure.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
brw_lower_logical_sends can just check for the TEX_LOGICAL_SRC_SHADOW_C
source; we don't need a generic instruction bit for this. We used to
have one because this was handled in the generator for older hardware
before the advent of logical opcode lowering.
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33297>
This was originally turned into a separate struct for reuse between vec4
and fs backends, that's not needed anymore.
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/33334>
