We need to keep RADV and RadeonSI on the same page about this due to modifiers.
Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13153>
We don't have to use the special DCC settings for lower resolutions.
This will cause corruption if X and an windowed app use different Mesa
versions. The fix is to restart the X server. I expect to get false bug
reports due to this.
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13013>
With NGG culling, the shaders are split into two parts:
the top part that computes just the position output,
and the bottom part which produces the other outputs.
To reduce redundancy between the two, I added some code
to reuse uniform variables between them. However, there is
an edge case I didn't think about: because of vertex repacking,
it is possible for the bottom part to process a different vertex.
Therefore it can take a different divergent code path
(though it must still take the same uniform code path).
Due to this, when a uniform value comes from divergent control
flow, this may be undefined in the bottom part.
This commit stops reusing uniform variables from
divergent control flow, to fix issues that arise from this.
Fossil DB stats on Sienna Cichlid with NGGC on:
Totals from 1723 (1.34% of 128647) affected shaders:
VGPRs: 89312 -> 89184 (-0.14%); split: -0.15%, +0.01%
SpillSGPRs: 4575 -> 120 (-97.38%)
CodeSize: 10846424 -> 10873836 (+0.25%); split: -0.68%, +0.93%
MaxWaves: 34582 -> 34602 (+0.06%); split: +0.06%, -0.01%
Instrs: 2124471 -> 2128835 (+0.21%); split: -0.51%, +0.72%
Latency: 7274569 -> 7293899 (+0.27%); split: -0.22%, +0.48%
InvThroughput: 1637130 -> 1635490 (-0.10%); split: -0.17%, +0.07%
VClause: 25141 -> 25414 (+1.09%); split: -0.02%, +1.10%
SClause: 56367 -> 59503 (+5.56%); split: -1.36%, +6.93%
Copies: 230704 -> 219313 (-4.94%); split: -5.49%, +0.55%
Branches: 72781 -> 72681 (-0.14%); split: -0.21%, +0.07%
PreSGPRs: 118766 -> 100176 (-15.65%); split: -15.70%, +0.05%
PreVGPRs: 76876 -> 76833 (-0.06%)
Fixes: 0bb543bb60
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13001>
match_mask checks the intrinsic type and decides whether it's
per-patch or not. VS don't have per-patch outputs,
so this causes wrong behaviour there.
Found using the GCC undefined behavior sanitizer.
Fixes the following error:
runtime error:
shift exponent 18446744073709551584 is too large
for 64-bit type 'long unsigned int'
Closes: #5319
Fixes: bf966d1c1d
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12719>
The byte-permute instruction v_perm_b32 is not exposed by older
LLVM releases (only available on LLVM 13 and later), therefore a new
sequence is needed which we can use with these LLVM versions too.
The prefix sum is replaced by two alternatives:
1. For GPUs that support v_dot, we shift 0x01 to the wanted byte
positions and then use v_dot to sum the results.
2. For older GPUs (Navi 10), we simply shift out the unwanted bytes
and use v_sad_u8 to produce the sum.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Acked-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12786>
Helper function to check if a modifier supports DCC image stores.
Signed-off-by: Joshua Ashton <joshua@froggi.es>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Bas Nieuwenhhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12862>
Fills the "Wave mode" in "Pipelines" for GPUs that supports Wave32.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12896>
Fills the "Scatch Mem" with "Yes/No" in "Pipelines", this requires
instruction timing to be enabled.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12896>
This is the first part of a refactor to make vertex compaction optional.
Additionally, it may yield a very small benefit to allocate the PC
space sligtly sooner.
Fossil DB stats on Sienna Cichlid with NGGC on:
Totals from 58239 (45.27% of 128647) affected shaders:
CodeSize: 160502348 -> 160502340 (-0.00%)
Instrs: 30722664 -> 30722662 (-0.00%)
Latency: 137627419 -> 137782218 (+0.11%); split: -0.00%, +0.11%
InvThroughput: 21698587 -> 21699068 (+0.00%); split: -0.00%, +0.00%
Copies: 3288263 -> 3288261 (-0.00%)
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12246>
Previously, indices 0, 2, 4 were used.
This worked, but it was somewhat unintuitive.
This commit changes it to use indices 0, 1, 2 instead, which
makes the code easier to understand.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12511>
These are not needed anymore, because the EXEC mask doesn't depend on them.
Fossil DB results on Sienna Cichlid (with NGGC on):
Totals from 58239 (45.27% of 128647) affected shaders:
Latency: 138113669 -> 138285372 (+0.12%)
InvThroughput: 22404840 -> 22405245 (+0.00%)
No Fossil DB changes with NGGC off.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11908>
This prevents us from recalculating the EXEC mask later
in the shader, and removes the requirement for
counting the number of primitives.
The stats are better than expected because they also
show that some code that is still there is now DCE'd by ACO.
Fossil DB results on Sienna Cichlid (with NGGC on):
Totals from 58239 (45.27% of 128647) affected shaders:
SpillSGPRs: 330 -> 340 (+3.03%)
CodeSize: 166356072 -> 162805724 (-2.13%)
Instrs: 31920041 -> 31089256 (-2.60%)
Latency: 138815742 -> 138113669 (-0.51%); split: -0.54%, +0.03%
InvThroughput: 22459553 -> 22404840 (-0.24%); split: -0.26%, +0.02%
SClause: 753746 -> 753765 (+0.00%); split: -0.00%, +0.01%
Copies: 3226647 -> 3268973 (+1.31%); split: -0.45%, +1.76%
Branches: 1223441 -> 1223440 (-0.00%); split: -0.00%, +0.00%
PreSGPRs: 2025339 -> 2091013 (+3.24%)
No Fossil DB changes with NGGC off.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11908>
This avoids re-calculating the exec mask for ES vertices,
and makes it unnecessary to count the number of vertices left.
Fossil DB results on Sienna Cichlid (with NGGC on):
Totals from 58239 (45.27% of 128647) affected shaders:
CodeSize: 166521108 -> 166356072 (-0.10%); split: -0.10%, +0.00%
Instrs: 31961308 -> 31920041 (-0.13%); split: -0.13%, +0.00%
Latency: 138820463 -> 138815742 (-0.00%); split: -0.04%, +0.04%
InvThroughput: 22460177 -> 22459553 (-0.00%); split: -0.00%, +0.00%
SClause: 753744 -> 753746 (+0.00%)
Copies: 3093140 -> 3226647 (+4.32%); split: -0.03%, +4.34%
No Fossil DB changes with NGGC off.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11908>
Instead of v_bfe + v_lshl_or for each vertex, get all 3 edge flags
at once of every vertex. This takes fewer VALU instructions than
previously.
Fossil DB results on Sienna Cichlid (with NGGC on):
Totals from 56917 (44.24% of 128647) affected shaders:
CodeSize: 161028288 -> 158751628 (-1.41%)
Instrs: 30917985 -> 30519571 (-1.29%)
Latency: 130617204 -> 129975532 (-0.49%); split: -0.50%, +0.01%
InvThroughput: 21280238 -> 20927401 (-1.66%)
Copies: 3011120 -> 3011125 (+0.00%); split: -0.00%, +0.00%
No Fossil DB changed with NGGC off.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11908>
The result is about +5-ish fps in Doom Eternal.
It turns out that the location of position exports matters more
than we thought, and it's actually better to keep them at the bottom
for culling shaders rather than schedule it up to the top.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>
Culling is traditionally done by the rasterizer, but that
can be a bottleneck when an app creates a large number
of primitives. Eg. a lot of tiny triangles reduce the
rasterziation efficiency.
NGG makes it possible for the shader to check primitives
and delete those that it can prove are not needed.
After this is done, we have to repack the surviving invocations
so they remain compact. This also saves bandwidth, because
some memory loads are only executed by those vertices that
survived the culling.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10525>
