Find a file
Gurchetan Singh 7b50e62179 gfxstream: mega-change to support guest Linux WSI with gfxstream
This is a mega-change to support Linux guest WSI with gfxstream.
We tried to do a branch where every commit was buildable and
runnable, but that quickly proved unworkable. So we squashed
the branch into a mega-change.

Zink provides the GL implementation for Linux guests, so we just
needed to implement the proper Vulkan Wayland/X11 WSI
entrypoints.

The overall strategy to support this is to use Mesa's WSI
functions.  The Vulkan WSI layer was also considered:

https://gitlab.freedesktop.org/mesa/vulkan-wsi-layer

But it was less maintained compared to Mesa.  The way Mesa common
layers communicate with drivers is the through base objects
embedded in driver and a common dispatch layer:

https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/docs/vulkan/dispatch.rst
https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/docs/vulkan/base-objs.rst

Our objects are defined in gfxstream_vk_private.h.  Currently,
Mesa-derived Vulkan objects just serve as shim to gfxstream
Vulkan’s internal handle mapping. Long-term, we can use
Mesa-derived objects inside gfxstream guest Vulkan exclusively.

The flow is typically inside a Vulkan entrypoint is:

- VK_FROM_HANDLE(vk-object) to convert to a gfxstream_vk_obj
  object
- Call ResourceTracker::func(gfxstream_vk_obj->internal) or
  VkEncoder::func(gfxstream_vk_obj>internal)
- Return result

A good follow-up cleanup would be to delete older gfxstream
objects.  For example, we now have struct gfxstream_vk_device
and info_VkDevice in ResourceTracker.

Most of this logic was auto-generated and included in
func_table.cpp. Some vulkan functions were too difficult to
auto-generate or required special logic, and these are included
in gfxstream_vk_device.cpp.  For example, anything that needs to
setup the HostConnection requires special handling.

Android Blueprint support is added to the parts of Mesa needed
to build the Vulkan runtime.  One thing to call out it's
required to build the guest/vulkan_enc and guest/vulkan files
in the same shared library now, when previously have
libvulkan_enc.so and libvulkan_ranchu.so was sufficient
[otherwise, some weak pointer logic wouldn't work].

A side effect of this is libOpenglSystem must also be a static
lib, and so should libandroid_aemu too.  That conceptually makes
sense and the Meson build had been doing this all a long.  We
can probably transition everything besides libGLESv1_emulation.so,
libGLESv2_emulation.so and libvulkan_ranchu.so to be static.

This requires changes in the end2end tests, because since each
HostConnection is separate and internal to it's constituent
library. Lifetimes need to be managed separately: for example
the HostConnection instance created by the end2end tests would
not be visible inside libvulkan_ranchu.so anymore. Probably the
best solution would be improve the testing facade so a
HostConnection represents one virtio-gpu context, while some
other entity represents a virtio-gpu device (client-server
would work).

vk.xml was modified, but change sent to Khronos:

https://gitlab.khronos.org/vulkan/vulkan/-/merge_requests/6325

Fuchsia builds still need to be migrated, but they already have
Fuchsia Mesa with all the build rules so that shouldn't be too
bad. Just need to copy them over the gfxstream/Mesa hybrid.

The new command for building Linux guests is:

meson amd64-build/ -Dvulkan-drivers="gfxstream" -Dgallium-drivers="" -Dvk-no-nir=true -Dopengl=false

Big shout-out to Aaron Ruby, who did most of the gnarly codegen
needed to get the function table logic to work.

     * Run Weston/vkcube on Linux and automotive platform
     * launch_cvd --gpu_mode=gfxstream vkcube
     * launch_cvd --gpu_mod=gfxstream_guest_angle
     * vkcube + 3D Mark Slingshot extreme work with guest ANGLE and
       GL-VK interop
     * GfxstreamEnd2EndTests
     * Some select dEQP tests

Aaron Ruby (46):
  gfxstream: function table: remove entry points that are hand-written.
  gfxstream: function table: more changes
  gfxstream: function table: scope internal_arrays to encoder
  gfxstream: function table: autogenerate compoundType params
  gfxstream: add handwritten EnumeratePhysicalDeviceGroup entrypoint.
  gfxstream: function table: handle nested handle arrays
  gfxstream: function table: adding some handwritten implementations
  gfxstream: revert some unnecessary changes
  gfxstream: use vk_object_zalloc/free instead of vk_zalloc/free.
  gfxstream: revert most gfxstream objects to use vk_object_base
  gfxstream: function table: handwritten commmand-buffers/pools
  gfxstream: codegen functionality to handle special param
  gfxstream: function table: random fixes
  gfxstream: add vk_command_buffer_ops handlers
  gfxstream: func_table.py: Codegen support for nested compound type
  gfxstream: remove handwritten/add autogen entry points
  gfxstream: add gfxstream_vk_device.cpp
  gfxstream: query device and instance extensions early
  gfxstream: func_table: explicit allocation for nested arrays/compound
    types
  gfxstream: goldfish_vulkan: fix commandBuffer allocation.
  gfxstream: meson: Raise api_version in ICD config to 1.1.
  gfxstream: function table: add more handwritten entries
  gfxstream: goldfish_vulkan: update VkDescriptorSetAllocateInfo logic
  gfxstream: function table: NULL check on internal_object dereference
  gfxstream: function table: Remove POSTPROCESSES handling from
    functable
  gfxstream: mesa: Add 'gfxstream' as a -Dvulkan-drivers
  gfxstream: ResourceTracker: add some allowedExtensions
  gfxstream: gfxstream_vk_device: add wsi_common_entrypoints
  gfxstream: Move instance handling into gfxstream_vk_device.cpp
  gfxstream: ResourceTracker: Enable Linux WSI-related extensions
  gfxstream: wsi: add wsi_device initialization
  gfxstream: gfxstream_vk_device: use Mesa common physical device
    management
  gfxstream: ResourceTracker: translate mesa objects in user buffer
  gfxstream: exclude VkSampler and VkDescriptorSet objects from
    translation
  gfxstream: Add guest-side external memory support with colorBuffers.
  gfxstream: function table: Modify semaphoreList inputs to no-op
    semaphores
  gfxstream: function table: Allow VK_NULL_HANDLE for free/destroy APIs.
  gfxstream: cereal: Add VK_EXT_depth_clip_enable as supported feature.
  gfxstream: vulkan_enc: un-namespace vk_util.h and vk_struct_id.h
  gfxstream: gfxstream_vk_device.cpp: Support VK_KHR_surface and
    VK_*_surface
  gfxstream: vulkan_enc: Add support for Mesa-only extensions.
  gfxstream: ResourceTracker: Use DEVICE_TYPE_VIRTUAL_GPU always
  gfxstream: platform: add dma-buf export support with dedicatedBuffer.
  gfxstream: ResourceTracker: add VK_EXT_depth_clip_enable allowed
    extension
  gfxstream: ResourceTracker: external memory via QNX_screen_buffer
    extension
  gfxstream: Add VK_QNX_external_memory_screen_buffer to VulkanDispatch

Gurchetan Singh (18):
  gfxstream: mesa: write Android.bp files
  gfxstream: generate gfxstream_vk_entrypoints.{c, h}
  gfxstream: vulkan_enc: add gfxstream_vk_private.h (objects)
  gfxstream: function table: modify function table to use gfxstream_vk_*
  gfxstream: compiles
  gfxstream: build system improvements
  gfxstream: ResourceTracker: don't crash without
    VkBindImageMemorySwapchainInfoKHR
  gfxstream: vk.xml: make some vkAcquireImageANDROID params optional
  gfxstream_vk_device: filter out swapchain maintenance guest side
  gfxstream: end2end: fixes for End2End tests
  gfxstream: func_table: custom vkEnumerateInstanceLayerProperties
  gfxstream: add VK_EXT_DEBUG_UTILS_EXTENSION_NAME into Mesa list
  gfxstream: clang-format guest code
  gfxstream: libandroid AEMU static
  gfxstream: vkEnumerateInstanceVersion
  gfxstream: vkCreateComputePipeLines
  gfxstream: make end2end tests happy
  gfxstream: delete prior vk.xml, vk_icd_gen.py

Reviewed-by: Aaron Ruby <aruby@blackberry.com>
Acked-by: Yonggang Luo <luoyonggang@gmail.com>
Acked-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27246>
2024-09-19 20:05:59 +00:00
.ci-farms ci/lima: Temporarily disable 2024-09-12 18:14:00 +00:00
.ci-farms-disabled ci/lima: Temporarily disable 2024-09-12 18:14:00 +00:00
.github/workflows add zink to macos ci 2022-11-22 19:04:13 +00:00
.gitlab gitlab: Reference hang debugging documenttion 2024-05-16 09:47:53 +00:00
.gitlab-ci Uprev Piglit to e9ab30aeaed97b69868cf4d6d6a3f70f3b53c362 2024-09-19 15:41:32 +00:00
android android: Extract version from llvm-project instead of hardcoding it 2024-08-17 21:11:08 +00:00
bin ci: Add Python requirements include 2024-09-12 18:17:52 +00:00
build-support meson: move tsan-blacklist.txt to build-support with the other build support files 2024-05-01 07:05:12 +00:00
docs docs: add sha sum for 24.2.3 2024-09-18 22:25:17 +00:00
include vulkan: Update XML and headers to 1.3.295 2024-09-10 18:22:42 +00:00
src gfxstream: mega-change to support guest Linux WSI with gfxstream 2024-09-19 20:05:59 +00:00
subprojects ci: Bump DirectX-Headers and Agility SDK dependencies to 1.614.1 2024-09-12 15:01:18 +00:00
.clang-format meson: enable the clang-format target 2023-05-29 11:57:08 +00:00
.clang-format-ignore ci: enforce formatting for RADV & ACO 2023-06-16 19:59:52 +00:00
.clang-format-include teflon: Initial commit 2024-01-24 10:02:10 +00:00
.dir-locals.el
.editorconfig glsl: Fixes ident issue in glsl_parser.yy and update editorconfig for it 2022-12-16 19:02:17 +00:00
.git-blame-ignore-revs freedreno: Add reformatting commits to .git-blame-ignore-revs 2023-09-22 02:07:36 +00:00
.gitattributes gitlab: Highlight .cl as C 2023-11-02 11:37:46 +00:00
.gitignore .gitignore: add .cache folder 2024-05-13 14:32:12 +00:00
.gitlab-ci.yml ci: merge build-x86_64 & build-misc 2024-08-29 14:33:15 +00:00
.graphqlrc.yml
.mailmap mailmap: update rohan's primary email address 2024-06-19 09:06:15 +00:00
.mr-label-maker.yml mr-label-maker: Add panvk 2024-09-09 15:59:52 +00:00
CODEOWNERS CODEOWNERS: update Imagination maintainers 2024-01-19 10:26:15 +00:00
meson.build meson: fix linking gallium with xcb-xfixes when using x11 2024-09-19 09:51:02 +00:00
meson_options.txt meson: remove dri-search-path 2024-09-19 05:06:50 +00:00
README.rst docs: Match README installation guide with meson.rst 2024-07-27 11:53:53 +00:00
VERSION VERSION: bump to 24.3 2024-07-19 16:37:23 +00:00

`Mesa <https://mesa3d.org>`_ - The 3D Graphics Library
======================================================


Source
------

This repository lives at https://gitlab.freedesktop.org/mesa/mesa.
Other repositories are likely forks, and code found there is not supported.


Build & install
---------------

You can find more information in our documentation (`docs/install.rst
<https://docs.mesa3d.org/install.html>`_), but the recommended way is to use
Meson (`docs/meson.rst <https://docs.mesa3d.org/meson.html>`_):

.. code-block:: sh

  $ meson setup build
  $ ninja -C build/
  $ sudo ninja -C build/ install

Support
-------

Many Mesa devs hang on IRC; if you're not sure which channel is
appropriate, you should ask your question on `OFTC's #dri-devel
<irc://irc.oftc.net/dri-devel>`_, someone will redirect you if
necessary.
Remember that not everyone is in the same timezone as you, so it might
take a while before someone qualified sees your question.
To figure out who you're talking to, or which nick to ping for your
question, check out `Who's Who on IRC
<https://dri.freedesktop.org/wiki/WhosWho/>`_.

The next best option is to ask your question in an email to the
mailing lists: `mesa-dev\@lists.freedesktop.org
<https://lists.freedesktop.org/mailman/listinfo/mesa-dev>`_


Bug reports
-----------

If you think something isn't working properly, please file a bug report
(`docs/bugs.rst <https://docs.mesa3d.org/bugs.html>`_).


Contributing
------------

Contributions are welcome, and step-by-step instructions can be found in our
documentation (`docs/submittingpatches.rst
<https://docs.mesa3d.org/submittingpatches.html>`_).

Note that Mesa uses gitlab for patches submission, review and discussions.