We go to initialize the disk cache before we've compiled any shaders so
agx_compiler_debug is 0 at this point. Don't try to read it, instead go through
sa safe getter that will do the right thing.
Fixes: 5e9538c12e ("agx: isolate compiler debug flags")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
(cherry picked from commit e9b471d1b3)
I keep re-implementing this every time I look at resource-related
issues. Let's just make it official so we can turn it on with a flag
instead of having to add printfs every time ^^
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>
These operations can fail for complex reasons through no fault of mesa,
so we should have proper runtime checks for them even in release builds.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>
Rather than implicitly packing them with the sample_mask. Again, this is just
changing where they're emitted, no functional changes yet. Bug for bug
compatibility with the old behaviour.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>
This is the name applegpu is currently using, to capture the semantics of a
pixel fence. I'm not sure what Apple calls this but wait_pix is closer than
writeout for sure.
This commit just does the rename. It doesn't fix the broken semantics we've had,
this is to ease review and bisection.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>
There's a second instruction here, and a second source in the first instruction.
applegpu has known about the encodings for a while but I never updated the
packing code. We will need to stop hardcoding this for multisampling support, as
preparation tease apart the magic pieces.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>
Khronos APIs require that we support mipmapping even for 1D textures. However,
it isn't clear if this is supported in the hardware, and how it would work even
if it is. But 1D textures are pretty useless, so we just lower 1D textures to 2D
textures instead of worrying about that.
Fixes piles of Piglits relating to 1D textures.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>
Use the same silly workaround that Metal does, to fill in texture descriptors
when there's nothing bound in the interest of robust behaviour.
Fixes null pointer dereference in
arb_shading_language_420pack-active-sampler-conflict.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>
Try harder to coalesce collects, by trying to allocate collects only to regions
of the register file where we actually have a full vector worth of registers
free. If we already know that the vector will be blocked later, it's not a good
base register to pick since we'd be force to shuffle later. So, this tweak to
the collect coalescing heuristic lets us eliminate a pile of pointless copying.
shader-db results are excellent. Note that, although we use more registers,
none of the shaders tested had their thread count affected, likely because the
max HURT isn't too high and most of the scary % here is from using a few more
registers when the register pressure is already low. In the near future, that
property will become guaranteed thanks to live range splitting, too.
total instructions in shared programs: 1507337 -> 1500562 (-0.45%)
instructions in affected programs: 428137 -> 421362 (-1.58%)
helped: 2658
HURT: 167
helped stats (abs) min: 1.0 max: 34.0 x̄: 2.63 x̃: 2
helped stats (rel) min: 0.10% max: 25.00% x̄: 3.04% x̃: 2.14%
HURT stats (abs) min: 1.0 max: 10.0 x̄: 1.24 x̃: 1
HURT stats (rel) min: 0.20% max: 23.81% x̄: 3.90% x̃: 3.57%
95% mean confidence interval for instructions value: -2.49 -2.31
95% mean confidence interval for instructions %-change: -2.76% -2.51%
Instructions are helped.
total bytes in shared programs: 10333670 -> 10293172 (-0.39%)
bytes in affected programs: 2996682 -> 2956184 (-1.35%)
helped: 2660
HURT: 175
helped stats (abs) min: 2.0 max: 204.0 x̄: 15.70 x̃: 12
helped stats (rel) min: 0.08% max: 23.08% x̄: 2.64% x̃: 1.83%
HURT stats (abs) min: 2.0 max: 60.0 x̄: 7.26 x̃: 6
HURT stats (rel) min: 0.12% max: 22.39% x̄: 3.19% x̃: 2.78%
95% mean confidence interval for bytes value: -14.81 -13.76
95% mean confidence interval for bytes %-change: -2.39% -2.18%
Bytes are helped.
total halfregs in shared programs: 417284 -> 427363 (2.42%)
halfregs in affected programs: 49814 -> 59893 (20.23%)
helped: 95
HURT: 3018
helped stats (abs) min: 1.0 max: 8.0 x̄: 2.29 x̃: 2
helped stats (rel) min: 2.44% max: 28.57% x̄: 9.20% x̃: 6.06%
HURT stats (abs) min: 1.0 max: 14.0 x̄: 3.41 x̃: 4
HURT stats (rel) min: 2.08% max: 150.00% x̄: 36.54% x̃: 27.27%
95% mean confidence interval for halfregs value: 3.17 3.31
95% mean confidence interval for halfregs %-change: 34.05% 36.23%
Halfregs are HURT.
total threads in shared programs: 16465280 -> 16465280 (0.00%)
threads in affected programs: 0 -> 0
helped: 0
HURT: 0
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>
Add information about the relationship between program register usage and
program occupancy (the maximum number of threads that may execute concurrently
on a single shader core). This table is derived from studying the
maxTotalThreadsPerThreadgroup property in Metal while varying the register
usage, something I blogged about a few years back. It's probably not 100%
accurate and it hasn't been tested against hardware, but it matters "only" for
performance (not correctness) so I'm not super stressed about the details.
In the (near) future, RA will be able to make use of this information to know
exactly when it can use more registers without hurting performance. In the
present, it's just used for better shader-db statistics.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>
To simplify live range splitting, RA will soon assume that DCE has run (removing
extraneous vectors). So run DCE even when otherwise disabling backend
optimizations. AGX_MESA_DEBUG=noopt is still useful for disabling instruction
combining, which is the more-likely-to-be-buggy pass anyway.
This also fixes IR not being printed with noopt.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>
Fixes KHR-GLES3.copy_tex_image_conversions.forbidden.*
Arguably working around a mesa/st issue but more format support is good for
compatibility and performance anyway.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Janne Grunau <j@jannau.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22353>
Needed for Vulkan, and for dropping GLSL frontend lowering for the deqp
coverage override case.
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22083>
Also drop my email address in the copyright lines and fix some "Copyright 208
Alyssa Rosenzweig" lines, I'm not *that* old. Together this drops a lot of
boilerplate without losing any meaningful licensing information. SPDX is already
in use for the MIT-licensed code in turnip, venus, and a few other scattered
parts of the tree, so this should be ok from a Mesa licensing standpoint.
This reduces friction to create new files, by parsing the copy/paste boilerplate
and being short enough you can easily type it out if you want. It makes new
files seem less daunting: 20 lines of header for 30 lines of code is
discouraging, but 2 lines of header for 30 lines of code is reasonable for a
simple compiler pass. This has technical effects, as lowering the barrier to
making new files should encourage people to split code into more modular files
with (hopefully positive) effects on project compile time.
This helps with consistency between files. Across the tree we have at least a
half dozen variants of the MIT license text (probably more), plus code that uses
SPDX headers instead. I've already been using SPDX headers in Asahi manually, so
you can tell old vs new code based on the headers.
Finally, it means less for reviewers to scroll through adding files. Minimal
actual cognitive burden for reviewers thanks to banner blindness, but the big
headers still bloat diffs that add/delete files.
I originally proposed this in December (for much more of the tree) but someone
requested I wait until January to discuss. I've been trying to get in touch with
them since then. It is now almost April and, with still no response, I'd like to
press forward with this. So with a joint sign-off from the major authors of the
code in question, let's do this.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Acked-by: Emma Anholt <emma@anholt.net>
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Acked-by: Rose Hudson <rose@krx.sh>
Acked-by: Lyude Paul [over IRC: "yes I'm fine with that"]
Meh'd-by: Rob Clark <robdclark@chromium.org>
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22062>
Now that we have stub sync support in the submission API, we can
implement the batch tracking changes required to support an explicit
sync world. This excludes the UAPI-specific bits (command decoding and
status parsing).
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21620>
When a BO is exported, implicit sync convention requires that writers
signal a fence on the object when complete. We already do this for BOs
that are *already* exported, but it is possible for a BO to be written
to, then exported for the first time.
Add a field to agx_bo to keep track of the current writer syncobj
handle. On first export, we use this to import it into the DMA-BUF.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21620>
Currently, the generated tests consist of some boilerplate, generated
test cases, and at the very end the actual test. This is bad for readability,
because the actual code is all the way at the bottom. It's also bad for
clang-format linting: even though the test cases are /* clang-format off */,
they still take an exceptionally long time to parse when linting. I suspect this
is a clang-format bug, but it's easy enough to workaround.
To solve these issues, restructure so that the test cases are in separate files
(containing the actual data), but the manually written test functions are
consolidated into a new family of generated layout tests. This is probably
cleaner.
Parallel clang-format linting is now 10x faster on the M1, which means it's
now practical to lint in my "publish branch" hook.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21854>
Rather than ingesting separate control and memory barriers, ingest only the
combined and optimized scoped_barrier intrinsic. For barriers originating from
GLSL, this makes it easier to ensure correctness. For barriers originating from
SPIR-V, this is required for translation at all, as spirv_to_nir knows only
scoped barriers. So this gets us closer to Vulkan and OpenCL.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21752>
Lowers away 64-bit loads, which we'll create in the sysval lowering for
dynamically indexed UBOs/VBOs. The lowering generates pack_64_2x32 instructions,
so lower those too.
No shader-db changes.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21674>
Instead of lowering to bitwise ops. Yet another way of subdividing in NIR.
Probably insignificant but makes it easy to check that the pass ordering from the
previous pass is right. It does let us get much better codegen for
unpacksnorm2x16, whatever that's worth.
No shader-db changes.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21674>
As intended. We can't CSE with partial null destinations in the way, so we
shouldn't eliminate dead destinations until after CSE has run. But we should
still eliminate dead instructions to ensure CSE doesn't move things around
needlessly, hurting register pressure.
Noticed while debugging live range splitting.
No GLES3.0 shader-db changes.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21674>
Our dead code elimination pass does two things:
1. delete instructions that are entirely unnecessary
2. delete unnecessary destinations of necessary instructions
To deal with pass ordering issues, we sometimes want to do #1 without #2.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21674>
We should handle nir_op_unpack_32_2x16_split_* natively, since we can generate
better code with agx_subdivide (coalescing the ops away) than the bitshift
lowering.
That said, we do need some extra instructions for the floating point
conversions.
No shader-db changes (which makes sense because we're targetting the GLES3.0
shader-db, which doesn't have the packing GLSL functions).
The real motivation of this change isn't optimizing some GLSL pack functions,
though, it's avoiding a code regression from using NIR's memory bit size
lowering in a future MR. That lowering will turn things like "load i16vec4" into
"load i32vec2 + unpack_32_2x16", so we need to be able to coalesce that unpack.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21674>
Implement buffer textures in full generality. There are a few issues here:
* OpenGL requires buffer textures support a minimum size of 65536 elements,
however 1D textures in AGX are (at most) 8192 elements.
* OpenGL 4.0 (and OpenGL ES) require buffer textures to support the "RGB32"
texture formats. These are 3 packed channels of 32-bits each. In general,
non-power-of-two texel sizes are problematic. AGX does not support any such
formats and we rely on the GL frontend to lower to a padded format (RGBX) if
necessary. Such a lowering cannot work for buffer textures, however, so we
need to find a way to implement RGB32 buffer textures.
We solve these issues in the follow way:
* Use 2D texture descriptors for buffer textures, with a large fixed
power-of-two size along one axis. Then large texel indices may be accessed at
a small vec2 texel coordinate, and since the fixed dimension is a
power-of-two, that vector may be recovered by simply shifting and masking.
This effectively avoids size restriction. We do need to clamp texel indices to
the buffer size to avoid faulting on OOB reads, since we may read past the end
of the buffer (if the app binds a non-page-aligned offset into the buffer).
* Use a general purpose memory load for RGB32 buffer textures. Lower the texture
load instruction to a memory load from the buffer and some address arithmetic.
There's no format conversion needed for RGB32, other than maybe filling in a
format-appropriate alpha, so this is straightforward. Again, we need to clamp
the texel index for robustness with OOB reads.
Each of these solutions brings its own problem.
* Using 2D textures instead of 1D requires physically rounding up the buffer
size when packing the descriptor, so we can no longer implement textureSize()
by reading off the texture descriptor like normal.
* We don't know at compile-time whether a given texture load will read from an
RGB32 buffer texture or not, so we need to emit code for both. In Vulkan, we
can't key the shader to this property, either, since it's descriptor set state
and not pipeline state.
And each of these problems in turn brings its own solution:
* The texture descriptor is linear, so the "compression buffer address" field is
ignored by the hardware. We stash the real buffer size there so that
textureSize becomes a load from the texture descriptor like usual, without
requiring a sideband (which would complicate bindless textures).
* If we determine a texture descriptor contains RGB32 data, then it will never
be interpreted by the hardware and hence does not need to be a valid texture
descriptor. So, we extend the hardware's format enum to contain a
software-defined RGB32 format enum. Then, when lowering texture buffer loads,
we either read it as a typed RGB32 memory load or as a texture load depending
on the value of the format field in the texture descriptor.
All of this is accomplished with a big NIR pass generating a pile of strange
looking code. But it should be good enough in practice for this silly feature.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21672>
In NIR, texelFetch (txf) does not use a sampler, but in AGX, it does -- even
though the contents of the sampler are semantically irrelevant. Rather than
requiring the state tracker to bind a sampler anyway (indicated for texture
buffers with PIPE_CAP_TEXTURE_BUFFER_SAMPLER), just add a dummy sampler
ourselves if txf is used and there are otherwise no samplers. This is helpful
because PIPE_CAP_TEXTURE_BUFFER_SAMPLER isn't honoured by Rusticl or seemingly
mesa/st's PBO code, and after implementing this dummy sampler workaround in
Panfrost for Rusticl, I realized this CAP is silly and shouldn't exist in the
first place. (And I regret pushing for its reinclusion.)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21672>
Get the texture/sampler index from the texture/sampler_offset source (which
is an offset from 0 thanks to the lower_index_to_offset lowering) and feed it in
as corresponding 16-bit texture instruction sources.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21704>
For indirect indexing into the binding table. Note this does not handle packing
the bindless forms, since that's a bit more involved.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21704>
The gallium disk cache is about to depend on these, and I don't want to
create a dependency on agx_opcodes.h.py for that. So, make a new header
for them that doesn't have build dependencies.
Rename them to agx_compiler_* too, to avoid collisions with the other
driver debug flags.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21776>
We can't add a 64-bit immediate with the hardware iadd, that won't work. What we
can do is add a 32-bit immediate, derived as the low 32-bits of a 64-bit
nir_ssa_def.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21643>
If we manage to fold in a left shift that's bigger than the hardware can do, we
should at least avoid generating a useless right shift to feed the hardware
rather bailing completely.
For motivation, this form of address arithmetic is encountered when indexing
into arrays with large power-of-two element sizes (array-of-structs).
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21643>
Optimize address arithmetic of the form
base + u2u64((index << shift) + const)
into hardware operands
base, index << (shift - format_shift) + const'
which (if format_shift = shift) can be simply
base, index + const'
rather than the current naive translation
base, ((index << shift) + const) >> format_shift
This saves at least one pointless shift. We can't do this optimization with
nir_opt_algebraic, because explicitly optimizing "(a << #b) >> #b" to "a" isn't
sound due to overflow. But there's no overflow issue here, which is what this
whole pass is designed around.
For motivation, this address arithmetic implements "dynamically indexing into an
array inside of a C structure", where the const is the offset of the array
relative to the structure.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21643>
Once we've matched a summand, commit to it. This avoids needlessly checking the
second source if the first matched, and removes some indentation/funny control
flow.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21643>
