Skip to contentSkip to conversation

GitLab Custom Runners and Jobs (was: QEMU Gating CI)

PATCH Superseded Cleber Rosa 3 participants 18 messages Oct 14, 2020 – Oct 19, 2020

v3 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v3 cover GitLab Custom Runners and Jobs (was: QEMU Gating CI)

TL;DR: this should allow the QEMU maintainer to push to the staging branch, and have custom jobs running on the project's aarch64 and s390x machines. Simple usage looks like:

   git push remote staging
   ./scripts/ci/gitlab-pipeline-status --verbose --wait

Long version:

The idea about a public facing Gating CI for QEMU was summarized in an RFC[1]. Since then, it was decided that a simpler version should be attempted first.

At this point, there are two specific runners (an aarch64 and an s390) registered with GitLab, at https://gitlab.com/qemu-project, currently setup to the "qemu" repository.

Changes from v2:

- The overall idea of "Gating CI" has been re-worded "custom runners", given that the other jobs running on shared runners are also considered gating (Daniel)

- Fixed wording and typos on the documentation, including:
* update -> up to date (Erik)
* a different set of CI jobs -> different CI jobs (Erik)
* Pull requests will only be merged -> code will only be merged (Stefan)
* Setup -> set up (Stefan)
* them -> they (Stefan)
* the -> where the (Stefan)
* dropped "in the near future" (Stefan)

- Changed comment on "build-environment.yml" regarding the origin of the package list (Stefan)

- Removed inclusion of "vars.yml" from "build-environment.yml", given that no external variable is used there

- Updated package list in "build-environment.yml" from current dockerfiles

- Tested "build-environment" on Fedora 31 and 32 (in addition to Fedora 30), and noted that it's possible to use it on those distros

- Moved CI documentation from "testing.rst" to its own file (Phillipe)

- Split "GitLab Gating CI: initial set of jobs, documentation and scripts" into (Phillipe):
1) Basic documentation and configuration (gitlab-ci.yml) placeholder
2) Playbooks for setting up a build environment
3) Playbooks for setting up gitlab-runner
4) Actual GitLab CI jobs configuration

- Set custom jobs to be on the "build" stage, given that they combine build and test.

- Set custom jobs to not depend on any other job, so they can start right away.

- Set rules for starting jobs so that all pushing to any branch that start with name "staging". This allows the project maintainer to use the "push to staging" workflow, while also allowing others to generate similar jobs. If this project has configured custom runners, the jobs will run, if not, the pipeline will be marked as "stuck".

- Changed "scripts" on custom jobs to follow the now common pattern (on other jobs) of creating a "build" directory.

Changes from v1:

- Added jobs that require specific GitLab runners already available (Ubuntu 20.04 on aarch64, and Ubuntu 18.04 on s390x)
- Removed jobs that require specific GitLab runners not yet available (Fedora 30, FreeBSD 12.1)
- Updated documentation
- Added copyright and license to new scripts
- Moved script to from "contrib" to "scripts/ci/"
- Moved setup playbooks form "contrib" to "scripts/ci/setup"
- Moved "gating.yml" to ".gitlab-ci.d" directory
- Removed "staging" only branch restriction on jobs defined in ".gitlab-ci.yml", assumes that the additional jobs on the staging branch running on the freely available gitlab shared runner are positive
- Dropped patches 1-3 (already merged)
- Simplified amount of version specifity on Ubuntu, from 18.04.3 to simply 18.04 (assumes no diverse minor levels will be used or specific runners)

Changes from the RFC patches[2] accompanying the RFC document:

- Moved gating job definitions to .gitlab-ci-gating.yml
- Added info on "--disable-libssh" build option requirement
(https://bugs.launchpad.net/qemu/+bug/1838763) to Ubuntu 18.04 jobs
- Added info on "--disable-glusterfs" build option requirement (there's no static version of those libs in distro supplied packages) to one
- Dropped ubuntu-18.04.3-x86_64-notools job definition, because it doesn't fall into the general scope of gating job described by PMM (and it did not run any test)
- Added w32 and w64 cross builds based on Fedora 30
- Added a FreeBSD based job that builds all targets and runs make check
- Added "-jnproc" and "-jsysctl -n hw.ncpu" options to make as a simple but effective way of speeding up the builds and tests by using a number of make jobs matching the number of CPUs
- Because the Ansible playbooks reference the content on Dockerfiles, some fixes to some Dockerfiles caught in the process were included
- New patch with script to check or wait on a pipeline execution

[1] - https://lists.gnu.org/archive/html/qemu-devel/2019-12/msg00231.html
[2] - https://lists.gnu.org/archive/html/qemu-devel/2020-02/msg00154.html

Cleber Rosa (4):
Jobs based on custom runners: documentation and configuration
placeholder
Jobs based on custom runners: build environment docs and playbook Jobs based on custom runners: docs and gitlab-runner setup playbook Jobs based on custom runners: add job definitions for QEMU's machines

 .gitlab-ci.d/custom-runners.yml        | 206 +++++++++++++++++++++++
 .gitlab-ci.yml                         |   1 +
 docs/devel/ci.rst                      | 149 +++++++++++++++++
 docs/devel/index.rst                   |   1 +
 scripts/ci/setup/.gitignore            |   1 +
 scripts/ci/setup/build-environment.yml | 220 +++++++++++++++++++++++++
 scripts/ci/setup/gitlab-runner.yml     |  72 ++++++++
 scripts/ci/setup/inventory             |   2 +
 scripts/ci/setup/vars.yml.template     |  13 ++
 9 files changed, 665 insertions(+)
 create mode 100644 .gitlab-ci.d/custom-runners.yml
 create mode 100644 docs/devel/ci.rst
 create mode 100644 scripts/ci/setup/.gitignore
 create mode 100644 scripts/ci/setup/build-environment.yml
 create mode 100644 scripts/ci/setup/gitlab-runner.yml
 create mode 100644 scripts/ci/setup/inventory
 create mode 100644 scripts/ci/setup/vars.yml.template
Signature · 1 line
2.25.4
#1 lore raw

PATCH v3 1/4 Jobs based on custom runners: documentation and configuration placeholder

As described in the included documentation, the "custom runner" jobs extend the GitLab CI jobs already in place.

Those jobs are intended to run on hardware and/or Operating Systems not provided by GitLab's shared runners.

---
 .gitlab-ci.d/custom-runners.yml | 14 +++++++++
 .gitlab-ci.yml                  |  1 +
 docs/devel/ci.rst               | 54 +++++++++++++++++++++++++++++++++
 docs/devel/index.rst            |  1 +
 4 files changed, 70 insertions(+)
 create mode 100644 .gitlab-ci.d/custom-runners.yml
 create mode 100644 docs/devel/ci.rst
.gitlab-ci.d/custom-runners.yml new file +14 -0
new file mode 100644
index 0000000000..3004da2bda
@@ -0,0 +1,14 @@
+# The CI jobs defined here require GitLab runners installed and
+# registered on machines that match their operating system names,
+# versions and architectures.  This is in contrast to the other CI
+# jobs that are intended to run on GitLab's "shared" runners.
+
+# Different than the default approach on "shared" runners, based on
+# containers, the custom runners have no such *requirement*, as those
+# jobs should be capable of running on operating systems with no
+# compatible container implementation, or no support from
+# gitlab-runner.  To avoid problems that gitlab-runner can cause while
+# reusing the GIT repository, let's enable the recursive submodule
+# strategy.
+variables:
+  GIT_SUBMODULE_STRATEGY: recursive
.gitlab-ci.yml +1 -0
index 8ffd415ca5..b33c433fd7 100644
@@ -18,6 +18,7 @@ include:
   - local: '/.gitlab-ci.d/opensbi.yml'
   - local: '/.gitlab-ci.d/containers.yml'
   - local: '/.gitlab-ci.d/crossbuilds.yml'
+  - local: '/.gitlab-ci.d/custom-runners.yml'
 
 .native_build_job_template: &native_build_job_definition
   stage: build
docs/devel/ci.rst new file +54 -0

54 lines not on this page · raw message

docs/devel/index.rst +1 -0
index 77baae5c77..2fdd36e751 100644
@@ -21,6 +21,7 @@ Contents:
    atomics
    stable-process
    testing
+   ci
    qtest
    decodetree
    secure-coding-practices
Signature · 1 line
2.25.4
Reviewed-by on v3 1/4
#2 lore raw

Re: [PATCH v3 1/4] Jobs based on custom runners: documentation and configuration placeholder

Quoted 7 lines · Cleber Rosa

As described in the included documentation, the "custom runner" jobs extend the GitLab CI jobs already in place.

Those jobs are intended to run on hardware and/or Operating Systems not provided by GitLab's shared runners.

Quoted 8 lines
---
 .gitlab-ci.d/custom-runners.yml | 14 +++++++++
 .gitlab-ci.yml                  |  1 +
 docs/devel/ci.rst               | 54 +++++++++++++++++++++++++++++++++
 docs/devel/index.rst            |  1 +
 4 files changed, 70 insertions(+)
 create mode 100644 .gitlab-ci.d/custom-runners.yml
 create mode 100644 docs/devel/ci.rst

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#3 lore raw

PATCH v3 2/4 Jobs based on custom runners: build environment docs and playbook

In reply to Cleber Rosa

To run basic jobs on custom runners, the environment needs to be properly set up. The most common requirement is having the right packages installed.

The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.

---
 docs/devel/ci.rst                      |  32 ++++
 scripts/ci/setup/build-environment.yml | 220 +++++++++++++++++++++++++
 scripts/ci/setup/inventory             |   2 +
 3 files changed, 254 insertions(+)
 create mode 100644 scripts/ci/setup/build-environment.yml
 create mode 100644 scripts/ci/setup/inventory
docs/devel/ci.rst +32 -0
index 41a4bbddad..208b5e399b 100644
@@ -52,3 +52,35 @@ As a general rule, those newly added contributed jobs should run as
 The precise minimum requirements and exact rules for machine
 configuration documentation/scripts, and the success rate of jobs are
 still to be defined.
+
+Machine Setup Howto
+-------------------
+
+For all Linux based systems, the setup can be mostly automated by the
+execution of two Ansible playbooks.  Start by adding your machines to
+the ``inventory`` file under ``scripts/ci/setup``, such as this::
+
+  [local]
+  fully.qualified.domain
+  other.machine.hostname
+
+You may need to set some variables in the inventory file itself.  One
+very common need is to tell Ansible to use a Python 3 interpreter on
+those hosts.  This would look like::
+
+  [local]
+  fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
+  other.machine.hostname ansible_python_interpreter=/usr/bin/python3
+
+Build environment
+~~~~~~~~~~~~~~~~~
+
+The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
+set up machines with the environment needed to perform builds and run
+QEMU tests.  It covers a number of different Linux distributions and
+FreeBSD.
+
+To run the playbook, execute::
+
+  cd scripts/ci/setup
+  ansible-playbook -i inventory build-environment.yml
scripts/ci/setup/build-environment.yml new file +220 -0

220 lines not on this page · raw message

scripts/ci/setup/inventory new file +2 -0
new file mode 100644
index 0000000000..8bb7ba6b33
@@ -0,0 +1,2 @@
+[local]
+localhost
Signature · 1 line
2.25.4
#4 lore raw

Re: [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook

Quoted 110 lines · Cleber Rosa

To run basic jobs on custom runners, the environment needs to be properly set up. The most common requirement is having the right packages installed.

The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.

---
 docs/devel/ci.rst                      |  32 ++++
 scripts/ci/setup/build-environment.yml | 220 +++++++++++++++++++++++++
 scripts/ci/setup/inventory             |   2 +
 3 files changed, 254 insertions(+)
 create mode 100644 scripts/ci/setup/build-environment.yml
 create mode 100644 scripts/ci/setup/inventory
docs/devel/ci.rst +32 -0
index 41a4bbddad..208b5e399b 100644
@@ -52,3 +52,35 @@ As a general rule, those newly added contributed jobs should run as
 The precise minimum requirements and exact rules for machine
 configuration documentation/scripts, and the success rate of jobs are
 still to be defined.
+
+Machine Setup Howto
+-------------------
+
+For all Linux based systems, the setup can be mostly automated by the
+execution of two Ansible playbooks.  Start by adding your machines to
+the ``inventory`` file under ``scripts/ci/setup``, such as this::
+
+  [local]
+  fully.qualified.domain
+  other.machine.hostname
+
+You may need to set some variables in the inventory file itself.  One
+very common need is to tell Ansible to use a Python 3 interpreter on
+those hosts.  This would look like::
+
+  [local]
+  fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
+  other.machine.hostname ansible_python_interpreter=/usr/bin/python3
+
+Build environment
+~~~~~~~~~~~~~~~~~
+
+The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
+set up machines with the environment needed to perform builds and run
+QEMU tests.  It covers a number of different Linux distributions and
+FreeBSD.
+
+To run the playbook, execute::
+
+  cd scripts/ci/setup
+  ansible-playbook -i inventory build-environment.yml
scripts/ci/setup/build-environment.yml new file +47 -0
new file mode 100644
index 0000000000..0e8894bca9
@@ -0,0 +1,220 @@
+---
+- name: Installation of basic packages to build QEMU
+  hosts: all
+  tasks:
+    - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
+      apt:
+        update_cache: yes
+        # Originally from tests/docker/dockerfiles/ubuntu1804.docker
+        pkg:
+          - ccache
+          - clang
+          - gcc
+          - gettext
+          - git
+          - glusterfs-common
+          - libaio-dev
+          - libattr1-dev
+          - libbrlapi-dev
+          - libbz2-dev
+          - libcacard-dev
+          - libcap-ng-dev
+          - libcurl4-gnutls-dev
+          - libdrm-dev
+          - libepoxy-dev
+          - libfdt-dev
+          - libgbm-dev
+          - libgtk-3-dev
+          - libibverbs-dev
+          - libiscsi-dev
+          - libjemalloc-dev
+          - libjpeg-turbo8-dev
+          - liblzo2-dev
+          - libncurses5-dev
+          - libncursesw5-dev
+          - libnfs-dev
+          - libnss3-dev
+          - libnuma-dev
+          - libpixman-1-dev
+          - librados-dev
+          - librbd-dev
+          - librdmacm-dev
+          - libsasl2-dev
+          - libsdl2-dev
+          - libseccomp-dev
+          - libsnappy-dev
+          - libspice-protocol-dev
+          - libssh-dev

scripts/ci/setup/build-environment.yml line 57

Quoted 10 lines
scripts/ci/setup/build-environment.yml new file +10 -0
+          - libusb-1.0-0-dev
+          - libusbredirhost-dev
+          - libvdeplug-dev
+          - libvte-2.91-dev
+          - libzstd-dev
+          - make
+          - python3-yaml
+          - python3-sphinx
+          - sparse
+          - xfslibs-dev

This needs updating to add meson, and with Paolo's series today you might as well go ahead and add ninja-build immediately too

https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg04025.html

Same for all the other distro package lists.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#5 lore raw

Re: [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook

Quoted 125 lines · Daniel P. Berrangé
Quoted 110 lines · Cleber Rosa

To run basic jobs on custom runners, the environment needs to be properly set up. The most common requirement is having the right packages installed.

The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.

---
 docs/devel/ci.rst                      |  32 ++++
 scripts/ci/setup/build-environment.yml | 220 +++++++++++++++++++++++++
 scripts/ci/setup/inventory             |   2 +
 3 files changed, 254 insertions(+)
 create mode 100644 scripts/ci/setup/build-environment.yml
 create mode 100644 scripts/ci/setup/inventory
docs/devel/ci.rst +32 -0
index 41a4bbddad..208b5e399b 100644
@@ -52,3 +52,35 @@ As a general rule, those newly added contributed jobs should run as
 The precise minimum requirements and exact rules for machine
 configuration documentation/scripts, and the success rate of jobs are
 still to be defined.
+
+Machine Setup Howto
+-------------------
+
+For all Linux based systems, the setup can be mostly automated by the
+execution of two Ansible playbooks.  Start by adding your machines to
+the ``inventory`` file under ``scripts/ci/setup``, such as this::
+
+  [local]
+  fully.qualified.domain
+  other.machine.hostname
+
+You may need to set some variables in the inventory file itself.  One
+very common need is to tell Ansible to use a Python 3 interpreter on
+those hosts.  This would look like::
+
+  [local]
+  fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
+  other.machine.hostname ansible_python_interpreter=/usr/bin/python3
+
+Build environment
+~~~~~~~~~~~~~~~~~
+
+The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
+set up machines with the environment needed to perform builds and run
+QEMU tests.  It covers a number of different Linux distributions and
+FreeBSD.
+
+To run the playbook, execute::
+
+  cd scripts/ci/setup
+  ansible-playbook -i inventory build-environment.yml
scripts/ci/setup/build-environment.yml new file +47 -0
new file mode 100644
index 0000000000..0e8894bca9
@@ -0,0 +1,220 @@
+---
+- name: Installation of basic packages to build QEMU
+  hosts: all
+  tasks:
+    - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
+      apt:
+        update_cache: yes
+        # Originally from tests/docker/dockerfiles/ubuntu1804.docker
+        pkg:
+          - ccache
+          - clang
+          - gcc
+          - gettext
+          - git
+          - glusterfs-common
+          - libaio-dev
+          - libattr1-dev
+          - libbrlapi-dev
+          - libbz2-dev
+          - libcacard-dev
+          - libcap-ng-dev
+          - libcurl4-gnutls-dev
+          - libdrm-dev
+          - libepoxy-dev
+          - libfdt-dev
+          - libgbm-dev
+          - libgtk-3-dev
+          - libibverbs-dev
+          - libiscsi-dev
+          - libjemalloc-dev
+          - libjpeg-turbo8-dev
+          - liblzo2-dev
+          - libncurses5-dev
+          - libncursesw5-dev
+          - libnfs-dev
+          - libnss3-dev
+          - libnuma-dev
+          - libpixman-1-dev
+          - librados-dev
+          - librbd-dev
+          - librdmacm-dev
+          - libsasl2-dev
+          - libsdl2-dev
+          - libseccomp-dev
+          - libsnappy-dev
+          - libspice-protocol-dev
+          - libssh-dev

scripts/ci/setup/build-environment.yml line 57

Quoted 10 lines
scripts/ci/setup/build-environment.yml new file +10 -0
+          - libusb-1.0-0-dev
+          - libusbredirhost-dev
+          - libvdeplug-dev
+          - libvte-2.91-dev
+          - libzstd-dev
+          - make
+          - python3-yaml
+          - python3-sphinx
+          - sparse
+          - xfslibs-dev

This needs updating to add meson, and with Paolo's series today you might as well go ahead and add ninja-build immediately too

Quoted 5 lines

https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg04025.html

Same for all the other distro package lists.

Good point, will do.

Thanks!
- Cleber.

Quoted 8 lines

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#6 lore raw

Re: [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook

Quoted 5 lines · Cleber Rosa
Quoted 4 lines · Daniel P. Berrangé

This needs updating to add meson, and with Paolo's series today you might as well go ahead and add ninja-build immediately too

I replied too quickly, but allow me to get this right: meson is *not* included in the dockerfiles (and other similar configurations), and all setups I found rely on the submodule. Are suggesting to add meson and diverge from the dockerfiles?

Quoted 2 lines
Quoted 2 lines

https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg04025.html

^ I'll add meson according to this, of course.

Thanks,
- Cleber.

Quoted 3 lines
Quoted 3 lines

Same for all the other distro package lists.

#7 lore raw

Re: [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook

Quoted 11 lines · Cleber Rosa
Quoted 5 lines · Cleber Rosa
Quoted 4 lines · Daniel P. Berrangé

This needs updating to add meson, and with Paolo's series today you might as well go ahead and add ninja-build immediately too

I replied too quickly, but allow me to get this right: meson is *not* included in the dockerfiles (and other similar configurations), and all setups I found rely on the submodule. Are suggesting to add meson and diverge from the dockerfiles?

Doh, right, I forgot that we use the submodule for now, since we need such a new meson. So ignore this...

Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg04025.html

^ I'll add meson according to this, of course.

Just ninja is needed

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#8 lore raw

Re: [PATCH v3 2/4] Jobs based on custom runners: build environment docs and playbook

Quoted 16 lines · Daniel P. Berrangé
Quoted 11 lines · Cleber Rosa
Quoted 5 lines · Cleber Rosa
Quoted 4 lines · Daniel P. Berrangé

This needs updating to add meson, and with Paolo's series today you might as well go ahead and add ninja-build immediately too

I replied too quickly, but allow me to get this right: meson is *not* included in the dockerfiles (and other similar configurations), and all setups I found rely on the submodule. Are suggesting to add meson and diverge from the dockerfiles?

Doh, right, I forgot that we use the submodule for now, since we need such a new meson. So ignore this...

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines

https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg04025.html

^ I'll add meson according to this, of course.

Just ninja is needed

Right, I meant ninja there!

Thanks,
- Cleber.

#9 lore raw

PATCH v3 3/4 Jobs based on custom runners: docs and gitlab-runner setup playbook

In reply to Cleber Rosa

To have the jobs dispatched to custom runners, gitlab-runner must be installed, active as a service and properly configured. The variables file and playbook introduced here should help with those steps.

The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.

---
 docs/devel/ci.rst                  | 63 ++++++++++++++++++++++++++
 scripts/ci/setup/.gitignore        |  1 +
 scripts/ci/setup/gitlab-runner.yml | 72 ++++++++++++++++++++++++++++++
 scripts/ci/setup/vars.yml.template | 13 ++++++
 4 files changed, 149 insertions(+)
 create mode 100644 scripts/ci/setup/.gitignore
 create mode 100644 scripts/ci/setup/gitlab-runner.yml
 create mode 100644 scripts/ci/setup/vars.yml.template
docs/devel/ci.rst +63 -0
index 208b5e399b..a234a5e24c 100644
@@ -84,3 +84,66 @@ To run the playbook, execute::
 
   cd scripts/ci/setup
   ansible-playbook -i inventory build-environment.yml
+
+gitlab-runner setup and registration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The gitlab-runner agent needs to be installed on each machine that
+will run jobs.  The association between a machine and a GitLab project
+happens with a registration token.  To find the registration token for
+your repository/project, navigate on GitLab's web UI to:
+
+ * Settings (the gears like icon), then
+ * CI/CD, then
+ * Runners, and click on the "Expand" button, then
+ * Under "Set up a specific Runner manually", look for the value under
+   "Use the following registration token during setup"
+
+Copy the ``scripts/ci/setup/vars.yml.template`` file to
+``scripts/ci/setup/vars.yml``.  Then, set the
+``gitlab_runner_registration_token`` variable to the value obtained
+earlier.
+
+.. note:: gitlab-runner is not available from the standard location
+          for all OS and architectures combinations.  For some systems,
+          a custom build may be necessary.  Some builds are avaiable
+          at https://cleber.fedorapeople.org/gitlab-runner/ and this
+          URI may be used as a value on ``vars.yml``
+
+To run the playbook, execute::
+
+  cd scripts/ci/setup
+  ansible-playbook -i inventory gitlab-runner.yml
+
+.. note:: there are currently limitations to gitlab-runner itself when
+          setting up a service under FreeBSD systems.  You will need to
+          perform steps 4 to 10 manually, as described at
+          https://docs.gitlab.com/runner/install/freebsd.html
+
+Following the registration, it's necessary to configure the runner tags,
+and optionally other configurations on the GitLab UI.  Navigate to:
+
+ * Settings (the gears like icon), then
+ * CI/CD, then
+ * Runners, and click on the "Expand" button, then
+ * "Runners activated for this project", then
+ * Click on the "Edit" icon (next to the "Lock" Icon)
+
+Under tags, add values matching the jobs a runner should run.  For a
+FreeBSD 12.1 x86_64 system, the tags should be set as::
+
+  freebsd12.1,x86_64
+
+Because the job definition at ``.gitlab-ci.d/custom-runners.yml``
+would contain::
+
+  freebsd-12.1-x86_64-all:
+   tags:
+   - freebsd_12.1
+   - x86_64
+
+It's also recommended to:
+
+ * increase the "Maximum job timeout" to something like ``2h``
+ * uncheck the "Run untagged jobs" check box
+ * give it a better Description
scripts/ci/setup/.gitignore new file +1 -0
new file mode 100644
index 0000000000..f112d05dd0
@@ -0,0 +1 @@
+vars.yml
\ No newline at end of file
scripts/ci/setup/gitlab-runner.yml new file +72 -0

72 lines not on this page · raw message

scripts/ci/setup/vars.yml.template new file +13 -0
new file mode 100644
index 0000000000..621435d030
@@ -0,0 +1,13 @@
+# The version of the gitlab-runner to use
+gitlab_runner_version: 13.1.1
+# The base location of gitlab-runner binaries, this will be suffixed by $OS-$ARCH
+gitlab_runner_base_url: https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-
+# The URL of the gitlab server to use, usually https://gitlab.com unless you're
+# using a private GitLab instance
+gitlab_runner_server_url: https://gitlab.com
+# Defaults to linux, checks can be used to change this
+gitlab_runner_os: linux
+# Defaults to amd64 (x86_64), checks can be used to change this
+gitlab_runner_arch: amd64
+# A unique token made available by GitLab to your project for registering runners
+gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN
Signature · 1 line
2.25.4
Reviewed-by on v3 3/4
#10 lore raw

Re: [PATCH v3 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook

Quoted 10 lines · Cleber Rosa

To have the jobs dispatched to custom runners, gitlab-runner must be installed, active as a service and properly configured. The variables file and playbook introduced here should help with those steps.

The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.

Quoted 9 lines
---
 docs/devel/ci.rst                  | 63 ++++++++++++++++++++++++++
 scripts/ci/setup/.gitignore        |  1 +
 scripts/ci/setup/gitlab-runner.yml | 72 ++++++++++++++++++++++++++++++
 scripts/ci/setup/vars.yml.template | 13 ++++++
 4 files changed, 149 insertions(+)
 create mode 100644 scripts/ci/setup/.gitignore
 create mode 100644 scripts/ci/setup/gitlab-runner.yml
 create mode 100644 scripts/ci/setup/vars.yml.template

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#11 lore raw

PATCH v3 4/4 Jobs based on custom runners: add job definitions for QEMU's machines

In reply to Cleber Rosa

The QEMU project has two machines (aarch64 and s390) that can be used for jobs that do build and run tests. This introduces those jobs, which are a mapping of custom scripts used for the same purpose.

---
 .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
 1 file changed, 192 insertions(+)
.gitlab-ci.d/custom-runners.yml +192 -0
index 3004da2bda..5b51d1b336 100644
@@ -12,3 +12,195 @@
 # strategy.
 variables:
   GIT_SUBMODULE_STRATEGY: recursive
+
+# All ubuntu-18.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-18.04-s390x-all-linux-static:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-18.04-s390x-all:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-alldbg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --disable-libssh
+ - make clean
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-clang:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-18.04-s390x-tci:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --enable-tcg-interpreter
+ - make --output-sync -j`nproc`
+
+ubuntu-18.04-s390x-notcg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --disable-tcg
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+# All ubuntu-20.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/qemu/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-20.04-aarch64-all-linux-static:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
+ # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1
+
+ubuntu-20.04-aarch64-all:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-alldbg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --disable-libssh
+ - make clean
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-clang:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+
+ubuntu-20.04-aarch64-tci:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --enable-tcg-interpreter
+ - make --output-sync -j`nproc`
+
+ubuntu-20.04-aarch64-notcg:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_20.04
+ - aarch64
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ script:
+ - mkdir build
+ - cd build
+ - ../configure --disable-libssh --disable-tcg
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
Signature · 1 line
2.25.4
#12 lore raw

Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

Quoted 19 lines · Cleber Rosa

The QEMU project has two machines (aarch64 and s390) that can be used for jobs that do build and run tests. This introduces those jobs, which are a mapping of custom scripts used for the same purpose.

---
 .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
 1 file changed, 192 insertions(+)
.gitlab-ci.d/custom-runners.yml +2 -0
index 3004da2bda..5b51d1b336 100644
@@ -12,3 +12,195 @@
 # strategy.
 variables:
   GIT_SUBMODULE_STRATEGY: recursive
+
+# All ubuntu-18.04 jobs should run successfully in an environment

.gitlab-ci.d/custom-runners.yml line 26

Quoted 10 lines
.gitlab-ci.d/custom-runners.yml +10 -0
+# setup by the scripts/ci/setup/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-18.04-s390x-all-linux-static:
+ needs: []
+ stage: build
+ tags:
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'

IIRC, in the previous v2 (or was it v1) we discussed changing this so that users who provide their own runners, don't have to always use the "staging" branch name.

IIUC, the key thing is that we don't want the job running on the "master" or "stable-*" branches in the primary QEMU git. So could check

$CI_PROJECT_NAMESPACE == 'qemu-project'
&&
$CI_COMMIT_BRANCH !~ '^master$'
&&
$CI_COMMIT_BRANCH !~ '^stable-$'

which would let it work on users forks no matter what branch names they use

What happens to the job if the user doesn't have runners ? Is it simply skipped, or does the pipeline stall and get marked as failed ?

If the jobs aren't auto-skiped, we would need to add an env variable

(
$CI_PROJECT_NAMESPACE == 'qemu-project'
&&
$CI_COMMIT_BRANCH !~ '^master$'
&&
$CI_COMMIT_BRANCH !~ '^stable-$'
)
||
$QEMU_ENABLE_CUSTOM_RUNNERS == 'yes'

and require the user to set the QEMU_ENABLE_CUSTOM_RUNNERS variable in the web UI for their fork

That all said, I don't mind if you postpone this rules change to a followup patch.

.gitlab-ci.d/custom-runners.yml line 28

Quoted 2 lines
.gitlab-ci.d/custom-runners.yml +2 -0
+ script:
+ # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763

This bug links to

https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514

which is marked as fixed. So I'm thinking we can drop the --disable-libssh arg from all these jobs

.gitlab-ci.d/custom-runners.yml line 35

Quoted 7 lines
.gitlab-ci.d/custom-runners.yml +7 -0
+ # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#13 lore raw

Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

Quoted 26 lines · Daniel P. Berrangé
Quoted 25 lines · Cleber Rosa

The QEMU project has two machines (aarch64 and s390) that can be used for jobs that do build and run tests. This introduces those jobs, which are a mapping of custom scripts used for the same purpose.

---
 .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
 1 file changed, 192 insertions(+)
.gitlab-ci.d/custom-runners.yml +8 -0
index 3004da2bda..5b51d1b336 100644
@@ -12,3 +12,195 @@
 # strategy.
 variables:
   GIT_SUBMODULE_STRATEGY: recursive
+
+# All ubuntu-18.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-18.04-s390x-all-linux-static:
+ needs: []
+ stage: build
+ tags:
Quoted 9 lines

.gitlab-ci.d/custom-runners.yml line 26

Quoted 4 lines
.gitlab-ci.d/custom-runners.yml +4 -0
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'

IIRC, in the previous v2 (or was it v1) we discussed changing this so that users who provide their own runners, don't have to always use the "staging" branch name.

Right, and what I got from that is that users can use a *prefix* as a flag to indicate that they want the extra set of jobs.

Quoted 10 lines

IIUC, the key thing is that we don't want the job running on the "master" or "stable-*" branches in the primary QEMU git. So could check

   $CI_PROJECT_NAMESPACE == 'qemu-project'
   &&
   $CI_COMMIT_BRANCH !~ '^master$'
   &&
   $CI_COMMIT_BRANCH !~ '^stable-$'
Quoted 6 lines

which would let it work on users forks no matter what branch names they use

What happens to the job if the user doesn't have runners ? Is it simply skipped, or does the pipeline stall and get marked as failed ?

The pipeline gets "stuck" (literaly, that's the status name it gets). That's the main issue that made me believe that opting *in* (by using a common branch name prefix) was the simpler solution.

Quoted 12 lines

If the jobs aren't auto-skiped, we would need to add an env variable

   (
   $CI_PROJECT_NAMESPACE == 'qemu-project'
   &&
   $CI_COMMIT_BRANCH !~ '^master$'
   &&
   $CI_COMMIT_BRANCH !~ '^stable-$'
   )
   ||
   $QEMU_ENABLE_CUSTOM_RUNNERS == 'yes'
Quoted 3 lines

and require the user to set the QEMU_ENABLE_CUSTOM_RUNNERS variable in the web UI for their fork

We can do that, but I think it's more than we need. The odds that a user will have all of the same runners, and will be able to run all the extra jobs, are very very low IMO. Right from the start, very few people have an s390 machine running Ubuntu 18.04.

So, I believe that whenever a user pushes to a branch such as "staging-topic-foo", he will have to deal with some of the extra jobs (such as canceling the ones that will never run) anyway. Having to deal with those on every single push, or alternatively having to turn on/off $QEMU_ENABLE_CUSTOM_RUNNERS doesn't the best experience to me.

The "staging" prefix convention (with a better prefix name now or in the future?) seems to result in the best experience to me.

Quoted 3 lines

That all said, I don't mind if you postpone this rules change to a followup patch.

OK, let me know if you agree with my explanation above, or if you really want to see a followup patch.

Quoted 10 lines

.gitlab-ci.d/custom-runners.yml line 28

Quoted 2 lines
.gitlab-ci.d/custom-runners.yml +2 -0
+ script:
+ # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763

This bug links to

https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514

which is marked as fixed. So I'm thinking we can drop the --disable-libssh arg from all these jobs

OK, I'll double check that and, if the fix is comfirmed beyond the bug tracker, I'll update it.

Thanks!
- Cleber.

Quoted 16 lines

.gitlab-ci.d/custom-runners.yml line 35

Quoted 7 lines
.gitlab-ci.d/custom-runners.yml +7 -0
+ # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
+ - mkdir build
+ - cd build
+ - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh
+ - make --output-sync -j`nproc`
+ - make --output-sync -j`nproc` check V=1
+ - make --output-sync -j`nproc` check-tcg V=1

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#14 lore raw

Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

Quoted 4 lines · Cleber Rosa
Quoted 3 lines · Daniel P. Berrangé

This bug links to

Quoted 9 lines
Quoted 5 lines

https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514

which is marked as fixed. So I'm thinking we can drop the --disable-libssh arg from all these jobs

OK, I'll double check that and, if the fix is comfirmed beyond the bug tracker, I'll update it.

I was still able to reproduce the build failure with the latest packages.

The last update on the bug, is very misleading, but there's a hint of its outcome:

---
Martin Wimpress (flexiondotorg) on 2019-12-17
tags: added: rls-bb-notfixing
tags: removed: bionic
no longer affects: libssh (Ubuntu Bionic)
---

So, *notfixing* means it no longer affects that package on that distro? Right...

Anyway, keeping this AS IS.

Regards,
- Cleber.

#15 lore raw

Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

Quoted 27 lines · Cleber Rosa
Quoted 4 lines · Cleber Rosa
Quoted 3 lines · Daniel P. Berrangé

This bug links to

Quoted 9 lines
Quoted 5 lines

https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514

which is marked as fixed. So I'm thinking we can drop the --disable-libssh arg from all these jobs

OK, I'll double check that and, if the fix is comfirmed beyond the bug tracker, I'll update it.

I was still able to reproduce the build failure with the latest packages.

The last update on the bug, is very misleading, but there's a hint of its outcome:

---
Martin Wimpress (flexiondotorg) on 2019-12-17
tags: added: rls-bb-notfixing
tags: removed: bionic
no longer affects: libssh (Ubuntu Bionic)
---

Quoted 4 lines

So, *notfixing* means it no longer affects that package on that distro? Right...

Anyway, keeping this AS IS.

I can confirm that the libssh bug is still there in Ubuntu 18.04 ... when I was updating our .travis.yml some weeks ago to stop using Xenial, I also hit the bug again on Bionic.

Maybe rather a question for Peter, but what about updating the runners to 20.04 (Focal) instead? The libssh bug is gone there.

Thomas

#16 lore raw

Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

In reply to Cleber Rosa
Quoted 27 lines · Cleber Rosa
Quoted 4 lines · Cleber Rosa
Quoted 3 lines · Daniel P. Berrangé

This bug links to

Quoted 9 lines
Quoted 5 lines

https://bugs.launchpad.net/ubuntu/+source/libssh/+bug/1847514

which is marked as fixed. So I'm thinking we can drop the --disable-libssh arg from all these jobs

OK, I'll double check that and, if the fix is comfirmed beyond the bug tracker, I'll update it.

I was still able to reproduce the build failure with the latest packages.

The last update on the bug, is very misleading, but there's a hint of its outcome:

---
Martin Wimpress (flexiondotorg) on 2019-12-17
tags: added: rls-bb-notfixing
tags: removed: bionic
no longer affects: libssh (Ubuntu Bionic)
---

Quoted 2 lines

So, *notfixing* means it no longer affects that package on that distro? Right...

Urgh, launchpad is such a confusing bug tracker :-(

Quoted 1 line

Anyway, keeping this AS IS.

Yep

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#17 lore raw

Re: [PATCH v3 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

In reply to Cleber Rosa
Quoted 53 lines · Cleber Rosa
Quoted 26 lines · Daniel P. Berrangé
Quoted 25 lines · Cleber Rosa

The QEMU project has two machines (aarch64 and s390) that can be used for jobs that do build and run tests. This introduces those jobs, which are a mapping of custom scripts used for the same purpose.

---
 .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++
 1 file changed, 192 insertions(+)
.gitlab-ci.d/custom-runners.yml +8 -0
index 3004da2bda..5b51d1b336 100644
@@ -12,3 +12,195 @@
 # strategy.
 variables:
   GIT_SUBMODULE_STRATEGY: recursive
+
+# All ubuntu-18.04 jobs should run successfully in an environment
+# setup by the scripts/ci/setup/build-environment.yml task
+# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
+ubuntu-18.04-s390x-all-linux-static:
+ needs: []
+ stage: build
+ tags:
Quoted 9 lines

.gitlab-ci.d/custom-runners.yml line 26

Quoted 4 lines
.gitlab-ci.d/custom-runners.yml +4 -0
+ - ubuntu_18.04
+ - s390x
+ rules:
+ - if: '$CI_COMMIT_BRANCH =~ /^staging/'

IIRC, in the previous v2 (or was it v1) we discussed changing this so that users who provide their own runners, don't have to always use the "staging" branch name.

Right, and what I got from that is that users can use a *prefix* as a flag to indicate that they want the extra set of jobs.

Quoted 13 lines

IIUC, the key thing is that we don't want the job running on the "master" or "stable-*" branches in the primary QEMU git. So could check

   $CI_PROJECT_NAMESPACE == 'qemu-project'
   &&
   $CI_COMMIT_BRANCH !~ '^master$'
   &&
   $CI_COMMIT_BRANCH !~ '^stable-$'

which would let it work on users forks no matter what branch names they use

Quoted 7 lines
Quoted 3 lines

What happens to the job if the user doesn't have runners ? Is it simply skipped, or does the pipeline stall and get marked as failed ?

The pipeline gets "stuck" (literaly, that's the status name it gets). That's the main issue that made me believe that opting *in* (by using a common branch name prefix) was the simpler solution.

Hmm, that's very annoying behaviour.

Quoted 21 lines
Quoted 12 lines

If the jobs aren't auto-skiped, we would need to add an env variable

   (
   $CI_PROJECT_NAMESPACE == 'qemu-project'
   &&
   $CI_COMMIT_BRANCH !~ '^master$'
   &&
   $CI_COMMIT_BRANCH !~ '^stable-$'
   )
   ||
   $QEMU_ENABLE_CUSTOM_RUNNERS == 'yes'
Quoted 3 lines

and require the user to set the QEMU_ENABLE_CUSTOM_RUNNERS variable in the web UI for their fork

We can do that, but I think it's more than we need. The odds that a user will have all of the same runners, and will be able to run all the extra jobs, are very very low IMO. Right from the start, very few people have an s390 machine running Ubuntu 18.04.

Quoted 8 lines

So, I believe that whenever a user pushes to a branch such as "staging-topic-foo", he will have to deal with some of the extra jobs (such as canceling the ones that will never run) anyway. Having to deal with those on every single push, or alternatively having to turn on/off $QEMU_ENABLE_CUSTOM_RUNNERS doesn't the best experience to me.

The "staging" prefix convention (with a better prefix name now or in the future?) seems to result in the best experience to me.

Well "staging" prefix wasn';t appealing to me since none of the branches I work on have such a name prefix.

Quoted 6 lines
Quoted 3 lines

That all said, I don't mind if you postpone this rules change to a followup patch.

OK, let me know if you agree with my explanation above, or if you really want to see a followup patch.

Just ignore it for now. I'll do more thinking to see if I can figure out a more attractive solution.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|