Commit graph

2231 commits

Author SHA1 Message Date
Samuel Pitoiset
0d4fa8e5c6 ac/spm: introduce ac_spm_trace and ac_spm_get_trace()
For more code isolation.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22707>
2023-04-27 07:24:54 +00:00
Dave Airlie
e895c7ba56 radeonsi/ac: move some vcn defines to common
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21980>
2023-04-27 02:00:03 +00:00
Samuel Pitoiset
9c33c4dcbe ac/sqtt: add a helper to get cmdbuf IDs per queue
These will be used by RADV to implement queue event timings.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22670>
2023-04-26 13:57:39 +00:00
Samuel Pitoiset
a7b4573c11 ac/sqtt: add rgp_sqtt_marker_cb_id definition
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22670>
2023-04-26 13:57:39 +00:00
Qiang Yu
df74919bc2 ac/nir/ps: remove used nir_variable if created
RADV won't do this, so remove them at last.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22199>
2023-04-26 03:27:26 +00:00
Qiang Yu
471418077a ac/nir/ps: add no_color_export option
For radv which always do ps lower but may use epilog or not.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22199>
2023-04-26 03:27:26 +00:00
Qiang Yu
c877d26454 ac/nir/ps: use nir_export_dual_src_blend_amd when aco
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22199>
2023-04-26 03:27:26 +00:00
Rhys Perry
01f4addc18 ac/nir/ps: fix null export write mask miss set to 0xf
Fixes: c182154456 ("ac/nir: add ac_nir_lower_ps")
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22199>
2023-04-26 03:27:26 +00:00
Pierre-Eric Pelloux-Prayer
65b40d0b7e radeonsi: implement fw based mcbp
Some chips support firmware based mcbp. If supported this means
radeonsi needs to allocate 3 buffers and pass them to the firmware.

From there, the firmware will handle mcbp and register shadowing
on its own so we don't need to insert LOAD packet in the preamble.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21986>
2023-04-25 06:47:11 +00:00
Pierre-Eric Pelloux-Prayer
8fe39e9997 amd: determine info->has_fw_based_shadowing
The shadow_size value will be 0 if unsupported.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21986>
2023-04-25 06:47:11 +00:00
Pierre-Eric Pelloux-Prayer
dc5a9e176c amd: update amdgpu_drm.h
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21986>
2023-04-25 06:47:11 +00:00
Marek Olšák
e6e406b483 nir: add next_stage parameter to nir_remove_varying
so that e.g. the POS output is removed if the next stage is not FS.

Reviewed-by: Qiang Yu <yuq825@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21861>
2023-04-19 21:42:11 +00:00
Qiang Yu
3ff9153a3b ac,radv: move ps arg compation to common place
To be shared with radeonsi when aco is used.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22525>
2023-04-19 08:39:46 +00:00
Qiang Yu
35d5c7c251 ac/nir/ps: lower sample mask input when needed
Ported from si_llvm_build_ps_prolog().

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21683>
2023-04-17 02:11:56 +00:00
Qiang Yu
1103d4ed74 ac/nir/ps: add force lower barycentric load options
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21683>
2023-04-17 02:11:55 +00:00
Qiang Yu
0e4ac0c1d6 ac/nir/ps: lower barycentric load when bc_optimize
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21683>
2023-04-17 02:11:55 +00:00
Timur Kristóf
948a122f30 amd: Rename INDIRECT_BUFFER_CIK to just INDIRECT_BUFFER.
This packet is supported on GFX6 too, its name should relect that.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22406>
2023-04-13 23:58:48 +00:00
Harri Nieminen
f85f511a38 amd: fix typos in code
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22432>
2023-04-13 23:08:22 +00:00
Harri Nieminen
aea48a4ff1 amd: fix typos
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22432>
2023-04-13 23:08:22 +00:00
Qiang Yu
45826e42c5 ac,aco: move gfx10 ngg prim count zero workaround to nir
To simplify both llvm and aco backend and remove unnecessary
workaround code where prim count is known to be not zero.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22381>
2023-04-13 08:12:03 +00:00
Timur Kristóf
7036d1a155 ac/nir/ngg: Don't store primitive IDs from culled primitives.
Primitive export used the gs_accepted variable after culling,
so we overwrote this variable after vertex compaction to make
sure not to hang the GPU.

This had an unintended side effect when storing the primitive ID
to LDS on GS threads: the LDS store was done even on threads whose
triangle was culled; potentially causing issues.

As a fix, create a separate boolean variable that remembers
which invocations need to export a primitive; and don't store
the primitive ID to LDS when gs_accepted is false.

Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8805
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22424>
2023-04-12 23:55:37 +00:00
Timur Kristóf
4ae46840cd ac, aco, radv: Clarify LDS size on GFX6, and NGG shaders.
This information was wrong in some places, let's fix it now.

GFX6:
The GPU has 64KB LDS, but only 32KB is usable by a workgroup.

NGG:
There was some misinformation about NGG only being able to
address 32 KB LDS, it turns out this is actually not true
and it can address the full 64K.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21935>
2023-04-12 17:30:02 +00:00
Timur Kristóf
daeb409559 ac: Add maximum number of submitted IBs.
The number of IBs per submit isn't infinite, it depends on the IP type
(ie. some initial setup needed for a submit) and the packet size.
It can be calculated according to the kernel source code as:
(ring->max_dw - emit_frame_size) / emit_ib_size

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Qiang Yu
a2cecbbc44 ac/nir/ngg: fix store shared alignment
For stream!=0, this align_mul=4 is not true. Not observe any
problem yet, just for correctness.

Fixes: 60ac5dda82 ("ac: Add NIR lowering for NGG GS.")
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22304>
2023-04-07 03:42:25 +00:00
Qiang Yu
c082cdacae ac/nir/ngg: fix gs culling vertex liveness check for odd vertices
If vertex does not complete a primitive, it should not set the odd
flag which miss lead liveness check when culling is enabled.

For example, if odd flag is set regardless of complete flag, when
culling is enabled, 3 vertices of a triangle's init prim flag:
  [0x00 0x04 0x01]

then after culling, this triangle has been culled, their prim flag:
  [0x00 0x04 0x00]

the second vertex is miss treat as live because its odd flag (code
check prim_flag!=0 for liveness).

Fixes: 1bdeb961bd ("ac/nir/ngg: add gs culling")
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8725
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22304>
2023-04-07 03:42:25 +00:00
Sonny Jiang
5639b23644 amd/common: Add gfx940 codec query support
Add support for GFX940 VCN query

Signed-off-by: Sonny Jiang <sonny.jiang@amd.com>
Reviewed-by: Ruijing Dong <ruijing.dong@amd.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22158>
2023-04-06 15:00:53 +00:00
Sathishkumar S
d0477cbd07 radeonsi/vcn: add support for picture crop on JPEG 4.0.3
set the crop region and enable the feature if requested

Signed-off-by: Sathishkumar S <sathishkumar.sundararaju@amd.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22158>
2023-04-06 15:00:53 +00:00
Sathishkumar S
74572084d8 radeonsi/vcn: use register versions for jpeg
update the register version and select appropriate registers during decoder create

Signed-off-by: Sathishkumar S <sathishkumar.sundararaju@amd.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22158>
2023-04-06 15:00:53 +00:00
Sathishkumar S
9422627074 radeonsi/vcn: add register definitions for JPEG 4.0.3
Signed-off-by: Sathishkumar S <sathishkumar.sundararaju@amd.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Reviewed-by: Leo Liu <leo.liu@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22158>
2023-04-06 15:00:53 +00:00
Ganesh Belgur Ramachandra
d75f0d6c26 ac/nir: fix CDNA image lowering for array textures
The x,y coordinates were not added.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22158>
2023-04-06 15:00:53 +00:00
Marek Olšák
eaf98b1422 ac/nir: implement image opcode emulation for CDNA, enable it in radeonsi
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22158>
2023-04-06 15:00:53 +00:00
Marek Olšák
96913bbf3e ac/surface: force linear image layout for chips not supporting image opcodes
Image opcodes will be emulated.

Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22158>
2023-04-06 15:00:53 +00:00
Marek Olšák
d3b03fedd8 amd: add initial code for gfx940
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22158>
2023-04-06 15:00:53 +00:00
Qiang Yu
003d84b660 ac/nir: add ac_nir_load_arg_at_offset
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22045>
2023-04-03 01:35:06 +00:00
Timur Kristóf
4de9a4b2b8 ac/nir: When task->mesh dispatch Y or Z are 0, also set X to 0.
AMD recommends doing this to speed up the CP when it processes
the draw ring entries. LLPC also does this.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22211>
2023-04-01 14:46:50 +00:00
Timur Kristóf
4683b21399 ac/nir: Store only lowest 8 bits for task draw ring DWORD3.
When writing the draw ready bit, don't write the high 24 bits
of DWORD3, because that is used by the HW for something else
according to LLPC.

Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22211>
2023-04-01 14:46:50 +00:00
Timur Kristóf
115958b6f0 ac/nir/ngg: Slightly improve attribute ring offset calculation.
Inspired by Nicolai Hähnle's commit in LLPC.
Instead of using a SALU instruction to add to the scalar
offset, rely on the buffer swizzling and use constant offset.

Fossil DB stats on GFX1100:

Totals from 47910 (35.51% of 134913) affected shaders:
CodeSize: 87927612 -> 86968136 (-1.09%)
Instrs: 17584007 -> 17440094 (-0.82%)
Latency: 97232173 -> 97126311 (-0.11%)
InvThroughput: 9904586 -> 9905288 (+0.01%); split: -0.02%, +0.02%
VClause: 544430 -> 542566 (-0.34%)

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22227>
2023-03-31 17:02:17 +00:00
Timur Kristóf
93e4382438 radv, ac/nir: Move sin/cos lowering to a common pass.
Also ran clang-format on the affected code.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21971>
2023-03-30 05:59:13 +00:00
Marek Olšák
0b6a7cba0b amd: rename GFX1036 -> RAPHAEL_MENDOCINO
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22084>
2023-03-29 20:36:09 +00:00
Marek Olšák
52b6886992 amd: update addrlib
Reviewed-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22084>
2023-03-29 20:36:09 +00:00
Timur Kristóf
b0cae2fafe ac/nir: Remove ac_nir_apply_first_task_to_task_shader.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22139>
2023-03-29 15:08:55 +00:00
Timur Kristóf
3093622f35 ac/nir/ngg: Remove NV_mesh_shader support.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22139>
2023-03-29 15:08:55 +00:00
Qiang Yu
c182154456 ac/nir: add ac_nir_lower_ps
Lower ps output to nir_export_amd.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Signed-off-by: Qiang Yu <yuq825@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21552>
2023-03-28 19:57:11 +00:00
Pierre-Eric Pelloux-Prayer
7f94b80001 ac/surface: introduce umd metadata v2
Update the metadata format. For gfx8- chips nothing change.

For gfx9 chips:
* for textures without a valid modifier a dw is added at index=10
  containing the stride
* for textures with a valid modifier the modifier is stored at
  index 10 and 11. Then the number of planes is stored at 12.
  Then for each plane the offset and the stride are stored.

The goal here is to be able to create textures from dmabuf from
umr - without these changes this is impossible because these
values can't be guessed.

The new layout is compatible with version=1 so old/new UMD can
be used together without issues and isn't used by default.
For radeonsi, it will be possible to use it with a AMD_DEBUG=...
option.

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21984>
2023-03-28 15:17:28 +00:00
Pierre-Eric Pelloux-Prayer
4abb911bd2 amd/surface: rename metadata functions
Use more specific verbs to avoid confusion:
   set -> apply
   get -> compute

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21984>
2023-03-28 15:17:28 +00:00
Bas Nieuwenhuizen
0f045d43d6 ac/surface,radv: Opt out of stencil adjust.
We never implemented it, and having broken mipmaps works out better
for applications and CTS. Actually implementing stencil adjust is
going to be a major pain due to stuff like the GENERAL layout.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21869>
2023-03-25 18:15:08 +00:00
Bas Nieuwenhuizen
3bfa0d44fb ac/surface: Only allow stencil pitch adjustment for mipmaps.
Otherwise we can just have addrlib do the depth pitch adjustment
and have no copies.

v2: update CI expectations (olv)

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21869>
2023-03-25 18:15:08 +00:00
Bas Nieuwenhuizen
71a50bdaad ac/surface,radv: Avoid pitch weirdness if image not used for rendertarget.
Literally no point to it.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21869>
2023-03-25 18:15:08 +00:00
Timur Kristóf
59a4fb6af7 ac/nir/ngg: Store special MS outputs in attribute ring for PS to read.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21409>
2023-03-23 11:49:35 +00:00
Timur Kristóf
b2a71c56d9 ac/nir/ngg: Fix mesh shader layer on GFX11.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21409>
2023-03-23 11:49:35 +00:00