The polling period of the monitoring loop is a constant hardcoded and defined
to an arbitrary default number. This is related to the output reporting to the
user the progress in the pipeline running, so it shows this tool is alive when
it can be generating a huge number of output lines. Transforming it to a
definable variable allows the possibility to experiment with different values,
because of the balance between when a job can be triggered and the verbose
output produced.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39105>
The behavior of crnm when the target resolves to be one single job is special;
the job traces of this single job are dumped as the output of this monitoring
tool. There may be user cases where the wished behavior of this tool would be
equivalent to when the target resolves on multiple jobs.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39105>
After the migration to use the rich library, the execution within a GitLab job
gets dirty: coloring disappeared, the number of columns defaults to 80, and
job name padding fails with a broken link in the lines.
This is only a specialization when the tool detects it is running within a
job, without changing the behavior in the usual usage of this tool.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39105>
Instead of an exception that completely stops the run'n'monitor process, when
a job with a specific status can't be enabled, just log it with a warning.
Don't attempt to enable it again unless it changes the status.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39049>
The counter about how many times a job has been tried to enable, can host the
information about the job status when these attempts happen. This groups the
attempts to separate unrelated differences between job statuses.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/39049>
With the default non-colored job status, all the listed non-colored job
statuses can be absorbed by the default behavior.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38919>
When GitLab adds new job statuses, we need to upgrade the coloring dictionary.
By having a default value for non-colored output, we handle future unknown
status, and avoid crashes.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38919>
Update python-gitlab from 4.x to 5.x to fix AttributeError with
__annotations__ when running on Python 3.14. The 4.x series is
incompatible with Python 3.14 due to changes in how class annotations
are handled.
Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38123>
Remove the finally block with a return statement in GitlabGQL.query()
method. This pattern causes a SyntaxWarning in Python 3.14 as the return
in finally will override any return value or exception from the try/except
blocks.
Move the return statement to the end of the except block where it belongs,
maintaining the same error recovery behavior while fixing the warning.
Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/38123>
This script didn't have tests defined. Now, the objects on this tool have
tests to verify its functionality, as well as the core method called to do
all the processes to provide a sorted list of the marge queue.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37395>
As proposed in !37454, we can benefit from `rich` python module to simplify
output formatting. Using it here it's only the link print in console what
needs to be adjusted. There is a side effect, that look ok, with some
coloring in the timestamps and MR ids listed.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37395>
The output line with the MR link only works in some consoles; it can be
interesting for some developers to have visibility of the MR id.
It can be useful, too, to have some sort of a header showing the fields
printed from each merge request in the queue.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37395>
For testing and an eventual use of tenacity, it is practical to encapsulate
calls to the GitLab module in methods.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37395>
Use the parameter retry_transient_errors on the GitLab object creation to
protect the script from transient errors that can be well handled.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37395>
Rebuild the information gathering about the marge queue and how the
information is later prompted to the user.
The queue provided to the user is sorted, so the user knows what will be
merged first (when the corresponding merge request pipeline succeeds).
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37395>
Enhancement of the module with two structures that can encapsulate
functionalities and establish links between data collected.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37395>
Instead of printing an exception on the screen when the process is interrupted
from the keyboard, handle it and print a more friendly message.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37395>
Encapsulate the procedure in a method that can be imported from another tool
or even a python console.
Also include a typehint fix.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37395>
With the migration concluded on !37510 of the lava-job-submitter (it is now an
independent project under the gfx-ci namespace), a lava requirements file was
not necessary anymore, but there was still one reference.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37713>
The links in the console are broken depending on the console type; for example,
when it runs within a GitLab job. This can be improved using rich. But as we
have a dependency on colorama too, we can migrate all the coloring to use this
other library too.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/37454>
Bump `fire` from 0.5.0 to 0.7.0. The newer version removes the dependency
on the deprecated `pipes` module, which was removed in Python 3.13.
This fixes the LAVA tests in the `yaml-toml-shell-py-test` job when
running on Debian 13.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35853>
Since 655cf2f553 ("ci: Simplify filter_env_vars using indirect expansion"),
PIGLIT_REPLAY_DEVICE_NAME is declared using `declare -x`, not `export`.
Update the regex in update_traces_checksum.py accordingly. This is safe
for now, as all performance trace jobs currently run in LAVA and use this
declaration style.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/36070>
Check the arguments to have a consistent definition of metavars to control the
string shown, and have them all lowercase. Also define the types expected on
those arguments.
Signed-off-by: Sergi Blanch-Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35828>
The --rev argument mentions that it accepts a git revision. I think it would
be practical to describe that it accepts a commit id, as well as a tag or
branch name.
Signed-off-by: Sergi Blanch-Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35828>
ci-run_n_monitor tool can work with more projects than Mesa. It would be split
to its own repo with transparent access from the scripts in Mesa. But
meanwhile, this simple change allows using more GitLab instances than the
default gl.fd.o.
Signed-off-by: Sergi Blanch-Torne <sergi.blanch.torne@collabora.com>
Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Acked-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35826>
8f557b84f6 was done because `text` is sometimes just an int,
but the fix was only applies to the padding calculation.
Unfortunately, the padding direction is also different between strings
and integers, which means the behaviour is now incoherent.
Let's convert `text` to a string before we start doing anything so that
everything afterwards is coherent.
Fixes: 8f557b84f6 ("ci: crnm: fix hyperlink format")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35587>
Judging by comments in the chat, it seems the usage of the tokens in crnm is
only natural when you've been using it for a while. New users would appreciate
reading it in the documentation, beyond the help in the tool.
Also, mentioning how to create a token and what's the minimal scope of the
token to be used with the tool can help new users.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34860>
The number of columns should never be less than 1, otherwise we can
break the script such as:
```
Traceback (most recent call last):
File "/var/home/guilherme/projects/mesa/bin/ci/ci_run_n_monitor.py", line 734, in <module>
main()
File "/var/home/guilherme/projects/mesa/bin/ci/ci_run_n_monitor.py", line 713, in main
target_job_id, ret, exec_t = monitor_pipeline(
^^^^^^^^^^^^^^^^^
File "/var/home/guilherme/projects/mesa/bin/ci/ci_run_n_monitor.py", line 221, in monitor_pipeline
cancel_jobs(project, to_cancel)
File "/var/home/guilherme/projects/mesa/bin/ci/ci_run_n_monitor.py", line 400, in cancel_jobs
print_formatted_list(cancelled_jobs, indentation=8)
File "/var/home/guilherme/projects/mesa/bin/ci/gitlab_gql.py", line 373, in print_formatted_list
step = (len(elements) // n_columns) + 1
~~~~~~~~~~~~~~^^~~~~~~~~~~
ZeroDivisionError: integer division or modulo by zero
```
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/35250>
Compared to the existing Debian-based x86_64_pyutils container, this
Alpine-based variant reduces the image size by approximately 83%.
Include all the necessary python artifacts, including lava_job_submitter
in the container to avoid having to download them at the start of each
test job.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34980>
Move LAVA-related Python packages into a separate requirements file to
simplify installing only the necessary dependencies.
Signed-off-by: Valentine Burley <valentine.burley@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34980>
Some text paddings are not enough during the execution of the script. Having
an adaptative way for this variable can help the structure and visibility of
the output.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34856>
It's another example of a long list of job names, when the jobs waiting for a
status update is printed. The print formatted list can help to present it.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34856>
Rewrite how the jobs are cancelled within a pool of threads to simplify how
it can then use the print formatted list to for a better presentation of the
results.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34856>
When we are printing a long list, and it needs more than one line, it gets
hard to review the content. This is an idea to group the elements to be
printed in columns to make it easier to identify individual elements in
the output.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/34856>
