init-stage2.sh and setup-test-env.sh are already downloaded on the DUT as
part of the mesa-build overlay, which downloads the Mesa artifacts from
S3.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35051>
Instead of the current allowlist in export-gitlab-job-env-for-dut.sh,
filter out unwanted environment variables and forward the rest to
bare-metal and LAVA DUTs, as well as crosvm.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35051>
With the rest of the Qualcomm devices moving to LAVA, we can remove the
original (!) bare-metal infrastructure, leaving only the Igalia RPi
devices still using bare-metal. When those are converted to b2c, we can
remove the rest of bare-metal.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35148>
Firmware packages continue to grow in size, so stop installing them in
the test-base image.
The necessary firmware is now collected and uploaded per vendor in an
external repository.
LAVA devices can opt into optional firmware by specifying the name of the
archive via LAVA_FIRMWARE.
For bare-metal, Qualcomm firmware required for DUTs in the Google lab is
included in the baremetal image.
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/13051
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34861>
We can have jobs with a lower job timeout values, given by
CI_JOB_TIMEOUT environment variable, such as the pytest ones.
The previously hardcoded burst length of 1000 messages at a simulated
rate of 1 msg/sec caused tests to exceed these timeouts and fail
unexpectedly on specific job timeouts.
Reported-by: Eric Engestrom <eric@igalia.com>
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34907>
Some jobs, like those using pytest, have lower `CI_JOB_TIMEOUT` values.
This change ensures that the estimated LAVA overhead (in minutes) is
compatible with the actual job timeout, failing early with an assertion
and also avoiding mismatches and unintended timeouts.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34907>
The x86_64 shared runners often have long queues.
Move the rustfmt, python-artifacts, and yaml-toml-shell-py-test jobs to
use the aarch64 runners instead.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35000>
Every job must define the right timeouts anyway, so it doesn't make
sense to set these here.
Additionally, the 90/100 minutes overall timeout was unreachable since
the per-boot-cycle timeout was 45 minutes and there was no
BOOT_CYCLE__RETRIES set, so no retry was possible.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34899>
The EXTERNAL_KERNEL_TAG variable is no longer needed.
For LAVA and bare-metal, we can override the KERNEL_TAG variable to fetch
both the kernel image and modules from a different tag than the default
mainline gfx-ci/linux kernel.
For LAVA, this also avoids the issue where jobs using EXTERNAL_KERNEL_TAG
would still have mainline kernel modules downloaded by the LAVA overlay.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34873>
The previous check used a directory path without the filename, causing
`curl` to return 404 and always fall back to the fork's URL. Add the
rootfs filename to ensure the check works as intended.
Fixes: 13db56320b ("ci/baremetal: Use container rootfs")
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34873>
We named these jobs like this despite b2c being an implementation
detail because we did not have a name for this bare-metal infra.
Now that we do (CI-Tron), let's rename the jobs to remove the
confusion.
Co-authored-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34795>
This makes it easier to debug inheritance issues like the one fixed
earlier in this series, and makes it easier to reproduce a job without
needing more than looking at the job log.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34795>
Up until now, every project using CI-Tron had to write their own job
submission flow because CI-Tron itself was not providing any official
way of interacting with it via GitLab.
This however changed, and the solution is vastly superior to what we
have been using in Mesa:
* Ability to pass all the environment variables of the job to the DUT,
so no need to remember to add variables in
`export-gitlab-job-env-for-dut.sh` anymore
* No dependency on Mesa code, which means no need to wait on
python-artifacts and the ability to replicate a run by just copying
the job description outputted by the job \o/
* Ability to have as many initrd, HTTP, and TFTP artifacts as wanted
* Ability to expose a variable through a TFTP/HTTP endpoint or as an
initrd
* Ability to overwrite the platform environment (machine-specific FW)
* Ability to have as many kernel cmdline variables, all merged when
generating the final cmdline. This makes it easy to share some
snippets of cmdline between jobs
Transitioning from the custom to the generic template is however pretty
involved. This commit does the minimum changes needed to switch to the
new model, often simply replacing the B2C_ prefix with CI_TRON_.
Further renaming of "b2c" prefixes into "ci-tron" is left for future
commits.
Co-authored-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34795>
etnaviv is the important exception, as it needs to set two fields for
the setup tags, which need to be handled individually as runner tags.
The other exception is the nightly vangogh job that has a custom tag
added, which I'll hopefully get rid of Soon™.
One farm's jobs were also missing farm:$RUNNER_FARM_LOCATION; this
change ensures that it's always set, avoiding the risk of accidentally
picking an equivalent device in another farm.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34795>
Introduce per-component CI_BUILD_COMPONENTS_* variables to allow
more flexible composition of CI_BUILD_COMPONENTS, and add these to
every container job by listing them in .container.
Also rework the -arm64 and -x86_64 definitions, which were still
based on the old kernel+rootfs layout. Split them into -gl and -vk,
but keep -arm32, which builds fewer components.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34813>
Replace HWCI_KVM="true" with HWCI_ENABLE_X86_KVM=1 for consistency with
other integer-based toggles (e.g. HWCI_START_WESTON,
PARALLELISE_VIA_LP_THREADS).
The new name also makes the variable’s purpose clearer by explicitly
referencing X86.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34813>
Including the kernel image in test-base requires rebuilding every
x86_64 container image on each kernel uprev, even though only the
crosvm jobs use this kernel.
Move the download to runtime and update the distribution tags to avoid
triggering container rebuilds on kernel changes.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34813>
Structured tagging captures a checksum of the component we think we're
building, and verifies this through the chain.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34813>
Bumping DEBIAN_BASE_TAG already triggers a rebuild of the test-gl and
test-vk containers, so their tags don’t need to be updated manually.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34813>
history:
- 8d83ffe531 ("radv/ci: Add coverage for ETC2 emulation.")
var added to the jobs as `radv_require_etc2`, but not passed to duts
- 567c32b55c ("radv, drirc: rename radv_require_{etc2,astc}")
var renamed to `vk_require_etc2`, except in the jobs
- 5f177018f7 ("radv/ci: re-enable ET2C emulation testing on non-native GPUs")
var renamed in the jobs, but still not passed to duts
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34796>
No corresponding warnings left. Tighten the screws to prevent new
warnings from creeping in.
v2:
* The debian-clang-release job still needs
-Wno-error=sometimes-uninitialized.
v3:
* Drop more from debian-arm64-asan & debian-testing-msan jobs.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34799>
The S3_PATH_FLUSTER variable already included the archive filename.
Appending `vectors.tar.zst` again during upload caused the file to be
stored at the wrong location, which broke retrieval in subsequent builds.
Fix this behavior and rename the variable to S3_FLUSTER_TAR to make its
purpose clearer and avoid future confusion.
Fixes: a3fb667b1d ("ci: Include Fluster vectors in the rootfs")
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34787>
The strucutered tag replaces the need to have a separate
FLUSTER_VECTORS_VERSION variable just to check if the vectors changed.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34787>
Structured tagging captures a checksum of the component we think we're
building, and verifies this through the chain.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34787>
