Commit graph

2313 commits

Author SHA1 Message Date
Alyssa Rosenzweig
623bd2127f pan/bi: Add dead branch elimination pass
Ported from Midgard due to the same quirk of our code generation.
Additional validation, though.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
d0902aa2d4 pan/bi: Pass through wait_{6, 7} flags
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
19b195d3bf pan/bi: Move bi_next_clause to bir.c
Not really packing specific anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
2492074d3d pan/bi: Pull out bi_count_read_registers helper
I want to transition away from the ad hoc masks anyway.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
a69c73988b pan/bi: Fix M1/M2 decoding in disassembler
C's definition of the % operator has a footgun around sign conversion.
Avoid it and just use bitwise arithemtic instead like the hardware
would, fixing the disassembly and making buggy assembly more obvious.

Fixes: 08a9e5e3e8 ("pan/bi: Decode M values in disasm")
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
6f28a4449a pan/bi: Fix dependency wait calculation
Unconditional branches have a successor in the first slot only.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
6cb0a0ad63 pan/bi: Fix staging register packing
Writes are from the previous tuple, not the current one, otherwise we
incorrectly write to "two" places at once and raise an INSTR_INVALID_ENC
fault. While we're at it, fix the weird spacing.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
bca242c785 pan/bi: Fix IDLE register mode packing
Was incorrectly returning zero. Special case like IDLE_1.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
f1d551ea9f pan/bi: Print disasm/stats with DEBUG=internal
Arguably more important than the IR prints.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
d3c92d32ba pan/bi: Lint for infinite loops
I would make this unconditional, but conditionally branching to the same
clause in a tight loop is (disturbingly) legal, as far as I know.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
63c2ee2c4c pan/bi: Refactor PC-relative printing
Let's get the offset in a named variable for validation.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
24991d787f pan/bi: Print FAU index in verbose mode
Even if we're not loading a uniform, this is useful information. The
uniform pretty-printing didn't correspond well to the hardware anyway so
this is a net win, although if somebody really wanted pretty-printing
could be added in here.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
15d03ed783 pan/bi: Validate format 12 tuple count in disasm
We were throwing away this information. Let's just use a lookup table
and add an assertion. Would have caught a bug in this series resulting
in INSTR_INVALID_ENC faults.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
61af9cb76b pan/bi: Add internal debug flag
Since 3186401751 ("pan/bi: Suppress disassembly for internal shaders"),
we haven't had a good way to debug blit shaders. I keep rewriting this
patch manually, let's just a debug flag for it.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
b1ebe7a19b panfrost: Allow waiting on slots 6/7 during preload
I don't understand the underlying uarch details but ATEST needs to wait
on slot 6 and BLEND needs to wait on both, so these bits are used if
ATEST/BLEND are in the first clause, which happens if e.g. a constant
colour is written, or if the input is preloaded.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Alyssa Rosenzweig
a5780cdb9b panfrost: Fix TLS sizing if cores are missing
I have no idea if there are any implementations we care about that have
missing shader cores (a mask of 1101 or something like that), but if one
crops up, this would be a royal pain to debug so let's just get it
right...

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8723>
2021-01-29 16:55:43 +00:00
Italo Nicola
7584a25919 pan/mdg: implement nir_intrinsic_image_size
To implement it in midgard, we just need to read the image_size sysval.

Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8066>
2021-01-27 12:39:41 +00:00
Italo Nicola
8edccb7e8d pan/mdg: implement shader image instructions
Implements load store opreations as midgard_op_ld_image_* and
midgard_op_st_image_*.

Implements midgard_op_lea_tex, which takes an image coordinate and
returns it's memory address.

Image atomics are implemented as a combination of midgard_op_lea_tex and
the usual midgard atomic opcodes.

Currently we don't support multisampled shader images.

Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8066>
2021-01-27 12:39:41 +00:00
Italo Nicola
a14f825126 pan/mdg: rename st_image opcodes and add float16 versions
Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8066>
2021-01-27 12:39:41 +00:00
Italo Nicola
45095c42ac pan/mdg: add ld_image opcodes
Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8066>
2021-01-27 12:39:41 +00:00
Italo Nicola
8cfedd36ba pan/mdg: enable image bitsize lowering pass
Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8066>
2021-01-27 12:39:41 +00:00
Italo Nicola
db725e1d52 pan/mdg: create nir pass to lower image coord bitsize
Image coordinates are 16-bit long in midgard, so we must lower them
before emitting image opcodes.

Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8066>
2021-01-27 12:39:41 +00:00
Italo Nicola
0efe7a6eb9 panfrost: implement image_size sysval
Since there's no hardware support for it, we use a sysval to implement
nir_intrinsic_image_size.

Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8066>
2021-01-27 12:39:41 +00:00
Italo Nicola
c7c4fbc823 panfrost: decode 3d attribute continuation
Adds 3D attribute buffer continuations to pandecode and a message to
keep track of which buffer continuations we aren't decoding yet.

Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8066>
2021-01-27 12:39:41 +00:00
Italo Nicola
2638f72d2d panfrost: add 3d attribute buffer continuation to XML
Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8066>
2021-01-27 12:39:41 +00:00
Italo Nicola
a36c95cd77 panfrost: fix attribute continuation decoding
Because attribute continuations aren't considered whole attributes, we
need to increase the total number of attributes for each continuation
when decoding, otherwise we will end up missing a buffer for every
continuation we have.

Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8066>
2021-01-27 12:39:41 +00:00
Connor Abbott
5c92b3c460 panfrost: Assume that nir_tex_instr::dest_type is sized
Get rid of some now-redundant code, and cleanup the is-float check in
the bifrost compiler.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>
2021-01-25 11:21:59 +01:00
Connor Abbott
f0cb161e78 panfrost/blit: Use sized types for nir_tex_instr::dest_type
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7989>
2021-01-25 11:21:43 +01:00
Icecream95
0d0536c1a7 pan/decode: Free mapped memory objects on BO unreference
Cc: mesa-stable
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8583>
2021-01-21 00:32:28 +00:00
Icecream95
2e97d7c835 panfrost: Transaction elimination support
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7086>
2021-01-20 23:43:39 +00:00
Icecream95
7707ccf286 panfrost: Add a debug flag to disable checksumming
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7086>
2021-01-20 23:43:39 +00:00
Icecream95
5eb39dd95f pan/bi: Iterate from zero when setting RA interference
It is now valid for a node to have an index of zero.

Fixes: 39aa8c4a5a ("pan/bi: Switch to new IR")
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Alyssa Rosenzweig
a63960b7f3 pan/bi: Fix printing of node 0
Fixes: 1893a3805e ("pan/bi: Generate instruction printer")
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Alyssa Rosenzweig
3a16ab84e2 pan/bi: Fix RA of node 0
Fixes: 39aa8c4a5a ("pan/bi: Switch to new IR")
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Alyssa Rosenzweig
92461a1133 pan/bi: Fix 64-bit SSBO addresses
Fixes: 9c7efc4510 ("pan/bi: Add intrinsic emits for builder")
Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
a8c91f15f0 pan/mdg: Fix spilling when scratch memory is used
Add the tls_size from NIR before spilling so that it doesn't alias
with spill slots.

Fixes: 152bc5d15e ("pan/mdg: Support loads and stores to scratch memory")
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
6f0eff548c pan/bi: Implement packing ops between 32-bit vec1 and 16-bit vec2
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
c9c637a707 pan/bi: Implement ihadd/irhadd operations
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
591ccbcf47 pan/bi: Implement saturated add/sub operations
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
3f7e7495fc pan/bi: Lower umul_high
Also lower uadd_carry, which the mul_high lowering generates.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
2e372d0c3b pan/bi: Add w0 to the 'h01' swizzle bucket
A number of instructions, such as LOAD.i8, use this swizzle in the XML.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
2ec0897b11 pan/mdg: Allow 64-bit src_bitsize for comparison operations
Fixes Piglit test attributes.cl.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
b5d6e5049f pan/mdg: Don't reorder loads/stores past each other
Fixes Piglit test local-memory.cl.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
24fcc032ef pan/mdg: Limit int64 vectorization
Previously, nir_opt_vectorize was sometimes vectorizing 64-bit
load_const instructions to vec4.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
f0109e9ac0 panfrost: Assert on sysval overflow
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
24867386ee panfrost: Add a sysval for local_work_dim
Fixes Piglit test get-work-dim.cl.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
f5a35918db panfrost: Add a sysval for local_group_size
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
76fa57d195 pan/bi: Use pan_nir_lower_64bit_intrin
The intrinsics covered by the pass are implemented by reading 32-bit
registers, so there is no reason to keep them 64-bit.

Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
c82ab9b94a pan/bi: Improve unknown intrinsic error
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00
Icecream95
c71d4d931e pan/bi: Implement load_kernel_input
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8358>
2021-01-18 20:49:45 +00:00