GitLab Custom Runners and Jobs (was: QEMU Gating CI)
v4 cover GitLab Custom Runners and Jobs (was: QEMU Gating CI)
TL;DR: this should allow the QEMU maintainer to push to the staging branch, and have custom jobs running on the project's aarch64 and s390x machines. Simple usage looks like:
git push remote staging ./scripts/ci/gitlab-pipeline-status --verbose --wait
Long version:
The idea about a public facing Gating CI for QEMU was summarized in an RFC[1]. Since then, it was decided that a simpler version should be attempted first.
At this point, there are two specific runners (an aarch64 and an s390) registered with GitLab, at https://gitlab.com/qemu-project, currently setup to the "qemu" repository.
Changes from v3:
- Applied changes to match <[email protected]>, that is, added ninja-build to "build-environment.yml" list of packages and enabled PowerTools repository on CentOS 8.
Changes from v2:
- The overall idea of "Gating CI" has been re-worded "custom runners", given that the other jobs running on shared runners are also considered gating (Daniel)
- Fixed wording and typos on the documentation, including:
* update -> up to date (Erik)
* a different set of CI jobs -> different CI jobs (Erik)
* Pull requests will only be merged -> code will only be merged (Stefan)
* Setup -> set up (Stefan)
* them -> they (Stefan)
* the -> where the (Stefan)
* dropped "in the near future" (Stefan)
- Changed comment on "build-environment.yml" regarding the origin of the package list (Stefan)
- Removed inclusion of "vars.yml" from "build-environment.yml", given that no external variable is used there
- Updated package list in "build-environment.yml" from current dockerfiles
- Tested "build-environment" on Fedora 31 and 32 (in addition to Fedora 30), and noted that it's possible to use it on those distros
- Moved CI documentation from "testing.rst" to its own file (Phillipe)
- Split "GitLab Gating CI: initial set of jobs, documentation and scripts" into (Phillipe):
1) Basic documentation and configuration (gitlab-ci.yml) placeholder
2) Playbooks for setting up a build environment
3) Playbooks for setting up gitlab-runner
4) Actual GitLab CI jobs configuration
- Set custom jobs to be on the "build" stage, given that they combine build and test.
- Set custom jobs to not depend on any other job, so they can start right away.
- Set rules for starting jobs so that all pushing to any branch that start with name "staging". This allows the project maintainer to use the "push to staging" workflow, while also allowing others to generate similar jobs. If this project has configured custom runners, the jobs will run, if not, the pipeline will be marked as "stuck".
- Changed "scripts" on custom jobs to follow the now common pattern (on other jobs) of creating a "build" directory.
Changes from v1:
- Added jobs that require specific GitLab runners already available (Ubuntu 20.04 on aarch64, and Ubuntu 18.04 on s390x)
- Removed jobs that require specific GitLab runners not yet available (Fedora 30, FreeBSD 12.1)
- Updated documentation
- Added copyright and license to new scripts
- Moved script to from "contrib" to "scripts/ci/"
- Moved setup playbooks form "contrib" to "scripts/ci/setup"
- Moved "gating.yml" to ".gitlab-ci.d" directory
- Removed "staging" only branch restriction on jobs defined in ".gitlab-ci.yml", assumes that the additional jobs on the staging branch running on the freely available gitlab shared runner are positive
- Dropped patches 1-3 (already merged)
- Simplified amount of version specifity on Ubuntu, from 18.04.3 to simply 18.04 (assumes no diverse minor levels will be used or specific runners)
Changes from the RFC patches[2] accompanying the RFC document:
- Moved gating job definitions to .gitlab-ci-gating.yml
- Added info on "--disable-libssh" build option requirement
(https://bugs.launchpad.net/qemu/+bug/1838763) to Ubuntu 18.04 jobs
- Added info on "--disable-glusterfs" build option requirement (there's no static version of those libs in distro supplied packages) to one
- Dropped ubuntu-18.04.3-x86_64-notools job definition, because it doesn't fall into the general scope of gating job described by PMM (and it did not run any test)
- Added w32 and w64 cross builds based on Fedora 30
- Added a FreeBSD based job that builds all targets and runs make check
- Added "-jnproc" and "-jsysctl -n hw.ncpu" options to make as a simple but effective way of speeding up the builds and tests by using a number of make jobs matching the number of CPUs
- Because the Ansible playbooks reference the content on Dockerfiles, some fixes to some Dockerfiles caught in the process were included
- New patch with script to check or wait on a pipeline execution
[1] - https://lists.gnu.org/archive/html/qemu-devel/2019-12/msg00231.html
[2] - https://lists.gnu.org/archive/html/qemu-devel/2020-02/msg00154.html
Cleber Rosa (4):
Jobs based on custom runners: documentation and configuration
placeholder
Jobs based on custom runners: build environment docs and playbook Jobs based on custom runners: docs and gitlab-runner setup playbook Jobs based on custom runners: add job definitions for QEMU's machines
.gitlab-ci.d/custom-runners.yml | 206 ++++++++++++++++++++++ .gitlab-ci.yml | 1 + docs/devel/ci.rst | 149 ++++++++++++++++ docs/devel/index.rst | 1 + scripts/ci/setup/.gitignore | 1 + scripts/ci/setup/build-environment.yml | 233 +++++++++++++++++++++++++ scripts/ci/setup/gitlab-runner.yml | 72 ++++++++ scripts/ci/setup/inventory | 2 + scripts/ci/setup/vars.yml.template | 13 ++ 9 files changed, 678 insertions(+) create mode 100644 .gitlab-ci.d/custom-runners.yml create mode 100644 docs/devel/ci.rst create mode 100644 scripts/ci/setup/.gitignore create mode 100644 scripts/ci/setup/build-environment.yml create mode 100644 scripts/ci/setup/gitlab-runner.yml create mode 100644 scripts/ci/setup/inventory create mode 100644 scripts/ci/setup/vars.yml.template
Signature · 1 line
2.25.4
PATCH v4 1/4 Jobs based on custom runners: documentation and configuration placeholder
As described in the included documentation, the "custom runner" jobs extend the GitLab CI jobs already in place.
Those jobs are intended to run on hardware and/or Operating Systems not provided by GitLab's shared runners.
--- .gitlab-ci.d/custom-runners.yml | 14 +++++++++ .gitlab-ci.yml | 1 + docs/devel/ci.rst | 54 +++++++++++++++++++++++++++++++++ docs/devel/index.rst | 1 + 4 files changed, 70 insertions(+) create mode 100644 .gitlab-ci.d/custom-runners.yml create mode 100644 docs/devel/ci.rst
.gitlab-ci.d/custom-runners.yml new file +14 -0
@@ -0,0 +1,14 @@ +# The CI jobs defined here require GitLab runners installed and +# registered on machines that match their operating system names, +# versions and architectures. This is in contrast to the other CI +# jobs that are intended to run on GitLab's "shared" runners. + +# Different than the default approach on "shared" runners, based on +# containers, the custom runners have no such *requirement*, as those +# jobs should be capable of running on operating systems with no +# compatible container implementation, or no support from +# gitlab-runner. To avoid problems that gitlab-runner can cause while +# reusing the GIT repository, let's enable the recursive submodule +# strategy. +variables: + GIT_SUBMODULE_STRATEGY: recursive
.gitlab-ci.yml +1 -0
@@ -18,6 +18,7 @@ include: - local: '/.gitlab-ci.d/opensbi.yml' - local: '/.gitlab-ci.d/containers.yml' - local: '/.gitlab-ci.d/crossbuilds.yml' + - local: '/.gitlab-ci.d/custom-runners.yml' .native_build_job_template: &native_build_job_definition stage: build
docs/devel/ci.rst new file +54 -0
54 lines not on this page · raw message
docs/devel/index.rst +1 -0
@@ -21,6 +21,7 @@ Contents: atomics stable-process testing + ci qtest decodetree secure-coding-practices
Signature · 1 line
2.25.4
Re: [PATCH v4 1/4] Jobs based on custom runners: documentation and configuration placeholder
Quoted 69 lines · Cleber Rosa
As described in the included documentation, the "custom runner" jobs extend the GitLab CI jobs already in place.
Those jobs are intended to run on hardware and/or Operating Systems not provided by GitLab's shared runners.
Signed-off-by Cleber Rosa <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/custom-runners.yml | 14 +++++++++ .gitlab-ci.yml | 1 + docs/devel/ci.rst | 54 +++++++++++++++++++++++++++++++++ docs/devel/index.rst | 1 + 4 files changed, 70 insertions(+) create mode 100644 .gitlab-ci.d/custom-runners.yml create mode 100644 docs/devel/ci.rst.gitlab-ci.d/custom-runners.yml new file +14
-0@@ -0,0 +1,14 @@ +# The CI jobs defined here require GitLab runners installed and +# registered on machines that match their operating system names, +# versions and architectures. This is in contrast to the other CI +# jobs that are intended to run on GitLab's "shared" runners. + +# Different than the default approach on "shared" runners, based on +# containers, the custom runners have no such *requirement*, as those +# jobs should be capable of running on operating systems with no +# compatible container implementation, or no support from +# gitlab-runner. To avoid problems that gitlab-runner can cause while +# reusing the GIT repository, let's enable the recursive submodule +# strategy. +variables: + GIT_SUBMODULE_STRATEGY: recursive.gitlab-ci.yml +1
-0@@ -18,6 +18,7 @@ include: - local: '/.gitlab-ci.d/opensbi.yml' - local: '/.gitlab-ci.d/containers.yml' - local: '/.gitlab-ci.d/crossbuilds.yml' + - local: '/.gitlab-ci.d/custom-runners.yml' .native_build_job_template: &native_build_job_definition stage: builddocs/devel/ci.rst new file +14
-0@@ -0,0 +1,54 @@ +== +CI +== + +QEMU has configurations enabled for a number of different CI services. +The most up to date information about them and their status can be +found at:: + + https://wiki.qemu.org/Testing/CI + +Jobs on Custom Runners +====================== + +Besides the jobs run under the various CI systems listed before, there
docs/devel/ci.rst line 24
Quoted 10 lines
docs/devel/ci.rst new file +10
-0+are a number additional jobs that will run before an actual merge. +These use the same GitLab CI's service/framework already used for all +other GitLab based CI jobs, but rely on additional systems, not the +ones provided by GitLab as "shared runners". + +The architecture of GitLab's CI service allows different machines to +be set up with GitLab's "agent", called gitlab-runner, which will take +care of running jobs created by events such as a push to a branch. +Here, the combination of a machine, properly configured with GitLab's +gitlab-runner, is called a "custom runner" here.
Nit: Remove one of the two "here" in the above sentence.
docs/devel/ci.rst line 37
Quoted 12 lines
docs/devel/ci.rst new file +12
-0+The GitLab CI jobs definition for the custom runners are located under:: + + .gitlab-ci.d/custom-runners.yml + +Current Jobs +------------ + +The current CI jobs based on custom runners have the primary goal of +catching and preventing regressions on a wider number of host systems +than the ones provided by GitLab's shared runners. + +Also, the mechanics of reliability, capacity and overall maintanance
s/maintanance/maintenance/
docs/devel/ci.rst line 39
Quoted 2 lines
docs/devel/ci.rst new file +2
-0+of the machines provided by the QEMU project itself for those jobs +will be evaluated.
I'm not sure what this sentence is really good for... of course new stuff has to prove its usefulness first, but that's always the case and does not need to be mentioned in the documentation, I think? ... maybe that sentence is better something for the patch description instead of (hopefully) long-lasting documentation here?
Quoted 20 lines
docs/devel/ci.rst new file +14
-0+Future Plans and Jobs +--------------------- + +Once the CI Jobs based on custom runners have been proved mature with +the initial set of jobs run on machines from the QEMU project, other +members in the community should be able provide their own machine +configuration documentation/scripts, and accompanying job definitions. + +As a general rule, those newly added contributed jobs should run as +"non-gating", until their reliability is verified. + +The precise minimum requirements and exact rules for machine +configuration documentation/scripts, and the success rate of jobs are +still to be defined.docs/devel/index.rst +0
-0
docs/devel/index.rst Contents: · line 27
Quoted 7 lines
docs/devel/index.rst +1
-0@@ -21,6 +21,7 @@ Contents: atomics stable-process testing + ci qtest decodetree secure-coding-practices
Thomas
Re: [PATCH v4 1/4] Jobs based on custom runners: documentation and configuration placeholder
Quoted 83 lines · Thomas Huth
Quoted 69 lines · Cleber Rosa
As described in the included documentation, the "custom runner" jobs extend the GitLab CI jobs already in place.
Those jobs are intended to run on hardware and/or Operating Systems not provided by GitLab's shared runners.
Signed-off-by Cleber Rosa <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/custom-runners.yml | 14 +++++++++ .gitlab-ci.yml | 1 + docs/devel/ci.rst | 54 +++++++++++++++++++++++++++++++++ docs/devel/index.rst | 1 + 4 files changed, 70 insertions(+) create mode 100644 .gitlab-ci.d/custom-runners.yml create mode 100644 docs/devel/ci.rst.gitlab-ci.d/custom-runners.yml new file +14
-0@@ -0,0 +1,14 @@ +# The CI jobs defined here require GitLab runners installed and +# registered on machines that match their operating system names, +# versions and architectures. This is in contrast to the other CI +# jobs that are intended to run on GitLab's "shared" runners. + +# Different than the default approach on "shared" runners, based on +# containers, the custom runners have no such *requirement*, as those +# jobs should be capable of running on operating systems with no +# compatible container implementation, or no support from +# gitlab-runner. To avoid problems that gitlab-runner can cause while +# reusing the GIT repository, let's enable the recursive submodule +# strategy. +variables: + GIT_SUBMODULE_STRATEGY: recursive.gitlab-ci.yml +1
-0@@ -18,6 +18,7 @@ include: - local: '/.gitlab-ci.d/opensbi.yml' - local: '/.gitlab-ci.d/containers.yml' - local: '/.gitlab-ci.d/crossbuilds.yml' + - local: '/.gitlab-ci.d/custom-runners.yml' .native_build_job_template: &native_build_job_definition stage: builddocs/devel/ci.rst new file +14
-0@@ -0,0 +1,54 @@ +== +CI +== + +QEMU has configurations enabled for a number of different CI services. +The most up to date information about them and their status can be +found at:: + + https://wiki.qemu.org/Testing/CI + +Jobs on Custom Runners +====================== + +Besides the jobs run under the various CI systems listed before, theredocs/devel/ci.rst line 24
Quoted 10 lines
docs/devel/ci.rst new file +10
-0+are a number additional jobs that will run before an actual merge. +These use the same GitLab CI's service/framework already used for all +other GitLab based CI jobs, but rely on additional systems, not the +ones provided by GitLab as "shared runners". + +The architecture of GitLab's CI service allows different machines to +be set up with GitLab's "agent", called gitlab-runner, which will take +care of running jobs created by events such as a push to a branch. +Here, the combination of a machine, properly configured with GitLab's +gitlab-runner, is called a "custom runner" here.Nit: Remove one of the two "here" in the above sentence.
Quoted 15 lines
docs/devel/ci.rst line 37
Quoted 12 lines
docs/devel/ci.rst new file +12
-0+The GitLab CI jobs definition for the custom runners are located under:: + + .gitlab-ci.d/custom-runners.yml + +Current Jobs +------------ + +The current CI jobs based on custom runners have the primary goal of +catching and preventing regressions on a wider number of host systems +than the ones provided by GitLab's shared runners. + +Also, the mechanics of reliability, capacity and overall maintanances/maintanance/maintenance/
Oopsie... thanks.
Quoted 9 lines
docs/devel/ci.rst line 39
Quoted 2 lines
docs/devel/ci.rst new file +2
-0+of the machines provided by the QEMU project itself for those jobs +will be evaluated.I'm not sure what this sentence is really good for... of course new stuff has to prove its usefulness first, but that's always the case and does not need to be mentioned in the documentation, I think? ... maybe that sentence is better something for the patch description instead of (hopefully) long-lasting documentation here?
I think the statement attempts to set the tone here, and answer a question that I've seen more than once... It certainly feels a bit out of place here, but I also think the commit message would be too transient and almost invisible. I expect a lot will change in the CI and in its docs, so I don't think this exact sentence will be too long lasting.
Let me know if that convinces you it deserves to be in the docs, if not I'll gladly moved it to the commit message.
Thanks!
- Cleber
PATCH v4 2/4 Jobs based on custom runners: build environment docs and playbook
In reply to Cleber RosaTo run basic jobs on custom runners, the environment needs to be properly set up. The most common requirement is having the right packages installed.
The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.
--- docs/devel/ci.rst | 32 ++++ scripts/ci/setup/build-environment.yml | 233 +++++++++++++++++++++++++ scripts/ci/setup/inventory | 2 + 3 files changed, 267 insertions(+) create mode 100644 scripts/ci/setup/build-environment.yml create mode 100644 scripts/ci/setup/inventory
docs/devel/ci.rst +32 -0
@@ -52,3 +52,35 @@ As a general rule, those newly added contributed jobs should run as The precise minimum requirements and exact rules for machine configuration documentation/scripts, and the success rate of jobs are still to be defined. + +Machine Setup Howto +------------------- + +For all Linux based systems, the setup can be mostly automated by the +execution of two Ansible playbooks. Start by adding your machines to +the ``inventory`` file under ``scripts/ci/setup``, such as this:: + + [local] + fully.qualified.domain + other.machine.hostname + +You may need to set some variables in the inventory file itself. One +very common need is to tell Ansible to use a Python 3 interpreter on +those hosts. This would look like:: + + [local] + fully.qualified.domain ansible_python_interpreter=/usr/bin/python3 + other.machine.hostname ansible_python_interpreter=/usr/bin/python3 + +Build environment +~~~~~~~~~~~~~~~~~ + +The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will +set up machines with the environment needed to perform builds and run +QEMU tests. It covers a number of different Linux distributions and +FreeBSD. + +To run the playbook, execute:: + + cd scripts/ci/setup + ansible-playbook -i inventory build-environment.yml
scripts/ci/setup/build-environment.yml new file +233 -0
233 lines not on this page · raw message
scripts/ci/setup/inventory new file +2 -0
@@ -0,0 +1,2 @@ +[local] +localhost
Signature · 1 line
2.25.4
Re: [PATCH v4 2/4] Jobs based on custom runners: build environment docs and playbook
...
scripts/ci/setup/inventory line 1
Quoted 7 lines
scripts/ci/setup/inventory new file +1
-0@@ -0,0 +1,2 @@ +[local]
Nit pick, is a group for localhost actually needed?
Regards,
Erik
Re: [PATCH v4 2/4] Jobs based on custom runners: build environment docs and playbook
Quoted 12 lines · Erik Skultety
...
scripts/ci/setup/inventory line 1
Quoted 7 lines
scripts/ci/setup/inventory new file +1
-0@@ -0,0 +1,2 @@ +[local]Nit pick, is a group for localhost actually needed?
You're right, it's not needed... I just thought it gave the "localhost" entry some "shelter" and "context". :)
And, I think a mostly "ini-like" file without a section triggers an OCD reaction in me. I can remove it if it does something similar to you! :)
Thanks!
- Cleber.
Quoted 2 lines
Regards,
Erik
Re: [PATCH v4 2/4] Jobs based on custom runners: build environment docs and playbook
Quoted 14 lines · Cleber Rosa
Quoted 12 lines · Erik Skultety
...
scripts/ci/setup/inventory line 1
Quoted 7 lines
scripts/ci/setup/inventory new file +1
-0@@ -0,0 +1,2 @@ +[local]Nit pick, is a group for localhost actually needed?
Quoted 8 lines
You're right, it's not needed... I just thought it gave the "localhost" entry some "shelter" and "context". :)
And, I think a mostly "ini-like" file without a section triggers an OCD reaction in me. I can remove it if it does something similar to you! :)
Thanks!
- Cleber.
I understand, but even though it may be an ini-like config, "sections" have a very distinct meaning in Ansible and that is grouping, so unless there are hosts to group, I'd keep the inventory in a form of a simple list of individual hosts.
Regards,
Erik
Re: [PATCH v4 2/4] Jobs based on custom runners: build environment docs and playbook
Quoted 21 lines · Erik Skultety
Quoted 20 lines · Cleber Rosa
Quoted 12 lines · Erik Skultety
...
scripts/ci/setup/inventory line 1
Quoted 7 lines
scripts/ci/setup/inventory new file +1
-0@@ -0,0 +1,2 @@ +[local]Nit pick, is a group for localhost actually needed?
You're right, it's not needed... I just thought it gave the "localhost" entry some "shelter" and "context". :)
And, I think a mostly "ini-like" file without a section triggers an OCD reaction in me. I can remove it if it does something similar to you! :)
Quoted 8 lines
Quoted 2 lines
Thanks!
- Cleber.I understand, but even though it may be an ini-like config, "sections" have a very distinct meaning in Ansible and that is grouping, so unless there are hosts to group, I'd keep the inventory in a form of a simple list of individual hosts.
Fair enough. Changing it for v5.
Thanks!
- Cleber.
Quoted 2 lines
Regards,
Erik
Re: [PATCH v4 2/4] Jobs based on custom runners: build environment docs and playbook
In reply to Cleber Rosascripts/ci/setup/build-environment.yml line 4
Quoted 6 lines · Cleber Rosa
+++
b/scripts/ci/setup/build-environment.ymlscripts/ci/setup/build-environment.yml new file +4
-0@@ -0,0 +1,233 @@ +--- +- name: Installation of basic packages to build QEMU + hosts: all + tasks:
My Ansible-fu is a bit rusty at the moment, so please double-check my claims and apologies in advance for the ones that I will get wrong O:-)
scripts/ci/setup/build-environment.yml line 11
Quoted 7 lines
scripts/ci/setup/build-environment.yml new file +7
-0+ - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04 + apt: + update_cache: yes + # Originally from tests/docker/dockerfiles/ubuntu1804.docker + pkg: + - ccache + - clang
Instead of using the 'apt' module here, and the equivalent module further down, you could just do
package:
name:
- pkg1
- pkg2
...
state: present
every single time and let Ansible take care of the differences for you.
scripts/ci/setup/build-environment.yml line 60
Quoted 1 line
scripts/ci/setup/build-environment.yml new file +1
-0+ when: "ansible_facts['distribution'] == 'Ubuntu'"
Quoting the entire condition is not necessary, you can just have
when: ansible_facts['distribution'] == 'Ubuntu'
or, my preference,
when:
- ansible_facts['distribution'] == 'Ubuntu'
which results in a nicer diff when you add/remove conditions.
scripts/ci/setup/build-environment.yml line 67
Quoted 6 lines
scripts/ci/setup/build-environment.yml new file +6
-0+ - name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s390x + apt: + update_cache: yes + pkg: + - libspice-server-dev + - libxen-dev
Indentation of list items is inconsistent here.
scripts/ci/setup/build-environment.yml line 76
Quoted 4 lines
scripts/ci/setup/build-environment.yml new file +4
-0+ - name: Install basic packages to build QEMU on FreeBSD 12.x + pkgng: + # Originally from packages on .cirrus.yml under the freebsd_12_task + name: bash,curl,cyrus-sasl,git,glib,gmake,gnutls,gsed,nettle,ninja,perl5,pixman,pkgconf,png,usbredir
See above for 'pkgng' vs 'package', but at the very least this should be
pkgng:
name:
- bash
- curl
...
or each time the list is touched the resulting diff is going to be unmanageable.
scripts/ci/setup/build-environment.yml line 198
Quoted 9 lines
scripts/ci/setup/build-environment.yml new file +9
-0+ - name: Enable PowerTools repo on CentOS 8 + ini_file: + path: /etc/yum.repos.d/CentOS-PowerTools.repo + section: PowerTools + option: enabled + value: "1" + when: + - "ansible_facts['distribution'] == 'CentOS'" + - "ansible_facts['distribution_major_version'] == '8'"
Another option would be to use
command: 'dnf config-manager --set-enabled Stream-PowerTools -y'
args:
warn: nobut I have to admit the way you're doing it is very clever ;)
Signature · 1 line
Andrea Bolognani / Red Hat / Virtualization
Re: [PATCH v4 2/4] Jobs based on custom runners: build environment docs and playbook
Quoted 29 lines · Andrea Bolognani
scripts/ci/setup/build-environment.yml line 4
Quoted 6 lines · Cleber Rosa
+++
b/scripts/ci/setup/build-environment.ymlscripts/ci/setup/build-environment.yml new file +4
-0@@ -0,0 +1,233 @@ +--- +- name: Installation of basic packages to build QEMU + hosts: all + tasks:My Ansible-fu is a bit rusty at the moment, so please double-check my claims and apologies in advance for the ones that I will get wrong O:-)
scripts/ci/setup/build-environment.yml line 11
Quoted 7 lines
scripts/ci/setup/build-environment.yml new file +7
-0+ - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04 + apt: + update_cache: yes + # Originally from tests/docker/dockerfiles/ubuntu1804.docker + pkg: + - ccache + - clangInstead of using the 'apt' module here, and the equivalent module further down, you could just do
package:
name:
- pkg1
- pkg2
...
state: present
Quoted 3 lines
every single time and let Ansible take care of the differences for you.
I'm almost sure that this was a conscious decision. I remeber it had to do with not being able to set update_cache, and failures on recently installed systems and containers that did not update the APT cache. There may be something else, but I'll have to give it another round of testing.
FIY, under the hood, package is not really a module, but an action plugin that forwards these very limited options to the set or detected package manager, so it brings uniformity in the playbook, but limits the control too. IMO, it's very low impact to leave it AS IS.
Quoted 11 lines
scripts/ci/setup/build-environment.yml line 60
Quoted 1 line
scripts/ci/setup/build-environment.yml new file +1
-0+ when: "ansible_facts['distribution'] == 'Ubuntu'"Quoting the entire condition is not necessary, you can just have
when: ansible_facts['distribution'] == 'Ubuntu'
or, my preference,
when:
- ansible_facts['distribution'] == 'Ubuntu'
Yep, I've used the explicit lists when there was more than one condition, but having a standard style is better indeed.
Quoted 11 lines
which results in a nicer diff when you add/remove conditions.
scripts/ci/setup/build-environment.yml line 67
Quoted 6 lines
scripts/ci/setup/build-environment.yml new file +6
-0+ - name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s390x + apt: + update_cache: yes + pkg: + - libspice-server-dev + - libxen-devIndentation of list items is inconsistent here.
True. Fixed, thanks!
Quoted 8 lines
scripts/ci/setup/build-environment.yml line 76
Quoted 4 lines
scripts/ci/setup/build-environment.yml new file +4
-0+ - name: Install basic packages to build QEMU on FreeBSD 12.x + pkgng: + # Originally from packages on .cirrus.yml under the freebsd_12_task + name: bash,curl,cyrus-sasl,git,glib,gmake,gnutls,gsed,nettle,ninja,perl5,pixman,pkgconf,png,usbredirSee above for 'pkgng' vs 'package', but at the very least this should be
Quoted 9 lines
pkgng:
name:
- bash
- curl
...or each time the list is touched the resulting diff is going to be unmanageable.
The documentation suggests a comma separated list of package names:
https://docs.ansible.com/ansible/2.8/modules/pkgng_module.html#pkgng-module
And the reason is that this module is not as smart as others, and will run one separate command for each individual package name value:
It's a tradeoff indeed, but I think we're aligned with the docs.
Quoted 10 lines
scripts/ci/setup/build-environment.yml line 198
Quoted 9 lines
scripts/ci/setup/build-environment.yml new file +9
-0+ - name: Enable PowerTools repo on CentOS 8 + ini_file: + path: /etc/yum.repos.d/CentOS-PowerTools.repo + section: PowerTools + option: enabled + value: "1" + when: + - "ansible_facts['distribution'] == 'CentOS'" + - "ansible_facts['distribution_major_version'] == '8'"
Quoted 8 lines
Another option would be to use
command: 'dnf config-manager --set-enabled Stream-PowerTools -y' args: warn: nobut I have to admit the way you're doing it is very clever ;)
Yeah, that would require another package to be installed, and then the command executed... So I think this is cheaper and eaiser indeed.
Quoted 3 lines
Signature · 1 line
Andrea Bolognani / Red Hat / Virtualization
Thanks for the review, I'll report on the additional points as soon as I test them. If appropriate, I'll put notes on the v5.
- Cleber.
Re: [PATCH v4 2/4] Jobs based on custom runners: build environment docs and playbook
Quoted 21 lines · Cleber Rosa
Quoted 9 lines · Andrea Bolognani
scripts/ci/setup/build-environment.yml line 76
Quoted 4 lines · Cleber Rosa
scripts/ci/setup/build-environment.yml new file +4
-0+ - name: Install basic packages to build QEMU on FreeBSD 12.x + pkgng: + # Originally from packages on .cirrus.yml under the freebsd_12_task + name: bash,curl,cyrus-sasl,git,glib,gmake,gnutls,gsed,nettle,ninja,perl5,pixman,pkgconf,png,usbredirSee above for 'pkgng' vs 'package', but at the very least this should be
Quoted 8 lines
pkgng:
name:
- bash
- curl
...or each time the list is touched the resulting diff is going to be unmanageable.
The documentation suggests a comma separated list of package names:
Quoted 8 lines
https://docs.ansible.com/ansible/2.8/modules/pkgng_module.html#pkgng-module
And the reason is that this module is not as smart as others, and will run one separate command for each individual package name value:
It's a tradeoff indeed, but I think we're aligned with the docs.
I would probably take the performance hit over having to deal with an unmaintainable blob. As a rule of thumb, reviewer time is more valuable than machine time.
If the performance hit turns out to be unacceptably big, you could at least do something along the lines of
- set_fact:
freebsd_packages:
- bash
- curl
...
- pkgng:
name: "{ freebsd_packages | join(',') }"to keep things reviewable.
Signature · 1 line
Andrea Bolognani / Red Hat / Virtualization
Re: [PATCH v4 2/4] Jobs based on custom runners: build environment docs and playbook
In reply to Cleber RosaQuoted 10 lines · Cleber Rosa
To run basic jobs on custom runners, the environment needs to be properly set up. The most common requirement is having the right packages installed.
The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.
Signed-off-by Cleber Rosa <[email protected]>---
[...]
Quoted 5 lines
scripts/ci/setup/build-environment.yml new file +5
-0+ - name: Enable PowerTools repo on CentOS 8 + ini_file: + path: /etc/yum.repos.d/CentOS-PowerTools.repo + section: PowerTools + option: enabled
scripts/ci/setup/build-environment.yml line 204
Quoted 10 lines
scripts/ci/setup/build-environment.yml new file +10
-0+ value: "1" + when: + - "ansible_facts['distribution'] == 'CentOS'" + - "ansible_facts['distribution_major_version'] == '8'" + + - name: Install basic packages to build QEMU on CentOS 8 + dnf: + # Originally from tests/docker/dockerfiles/centos8.docker + name: + - SDL-devel
We do not support SDL1 in QEMU anymore, so this should be SDL2-devel now. Yes, we've also got it wrong in the docker files ... I'll send a patch to fix it there.
Thomas
Re: [PATCH v4 2/4] Jobs based on custom runners: build environment docs and playbook
Quoted 28 lines · Thomas Huth
Quoted 10 lines · Cleber Rosa
To run basic jobs on custom runners, the environment needs to be properly set up. The most common requirement is having the right packages installed.
The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.
Signed-off-by Cleber Rosa <[email protected]>---
[...]
Quoted 5 lines
scripts/ci/setup/build-environment.yml new file +5
-0+ - name: Enable PowerTools repo on CentOS 8 + ini_file: + path: /etc/yum.repos.d/CentOS-PowerTools.repo + section: PowerTools + option: enabledscripts/ci/setup/build-environment.yml line 204
Quoted 10 lines
scripts/ci/setup/build-environment.yml new file +10
-0+ value: "1" + when: + - "ansible_facts['distribution'] == 'CentOS'" + - "ansible_facts['distribution_major_version'] == '8'" + + - name: Install basic packages to build QEMU on CentOS 8 + dnf: + # Originally from tests/docker/dockerfiles/centos8.docker + name: + - SDL-devel
Quoted 5 lines
We do not support SDL1 in QEMU anymore, so this should be SDL2-devel now. Yes, we've also got it wrong in the docker files ... I'll send a patch to fix it there.
Thomas
Nice catch. I see the dockerfile changes are already merged, so updating them here.
- Cleber.
PATCH v4 3/4 Jobs based on custom runners: docs and gitlab-runner setup playbook
In reply to Cleber RosaTo have the jobs dispatched to custom runners, gitlab-runner must be installed, active as a service and properly configured. The variables file and playbook introduced here should help with those steps.
The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.
--- docs/devel/ci.rst | 63 ++++++++++++++++++++++++++ scripts/ci/setup/.gitignore | 1 + scripts/ci/setup/gitlab-runner.yml | 72 ++++++++++++++++++++++++++++++ scripts/ci/setup/vars.yml.template | 13 ++++++ 4 files changed, 149 insertions(+) create mode 100644 scripts/ci/setup/.gitignore create mode 100644 scripts/ci/setup/gitlab-runner.yml create mode 100644 scripts/ci/setup/vars.yml.template
docs/devel/ci.rst +63 -0
@@ -84,3 +84,66 @@ To run the playbook, execute:: cd scripts/ci/setup ansible-playbook -i inventory build-environment.yml + +gitlab-runner setup and registration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The gitlab-runner agent needs to be installed on each machine that +will run jobs. The association between a machine and a GitLab project +happens with a registration token. To find the registration token for +your repository/project, navigate on GitLab's web UI to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * Under "Set up a specific Runner manually", look for the value under + "Use the following registration token during setup" + +Copy the ``scripts/ci/setup/vars.yml.template`` file to +``scripts/ci/setup/vars.yml``. Then, set the +``gitlab_runner_registration_token`` variable to the value obtained +earlier. + +.. note:: gitlab-runner is not available from the standard location + for all OS and architectures combinations. For some systems, + a custom build may be necessary. Some builds are avaiable + at https://cleber.fedorapeople.org/gitlab-runner/ and this + URI may be used as a value on ``vars.yml`` + +To run the playbook, execute:: + + cd scripts/ci/setup + ansible-playbook -i inventory gitlab-runner.yml + +.. note:: there are currently limitations to gitlab-runner itself when + setting up a service under FreeBSD systems. You will need to + perform steps 4 to 10 manually, as described at + https://docs.gitlab.com/runner/install/freebsd.html + +Following the registration, it's necessary to configure the runner tags, +and optionally other configurations on the GitLab UI. Navigate to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * "Runners activated for this project", then + * Click on the "Edit" icon (next to the "Lock" Icon) + +Under tags, add values matching the jobs a runner should run. For a +FreeBSD 12.1 x86_64 system, the tags should be set as:: + + freebsd12.1,x86_64 + +Because the job definition at ``.gitlab-ci.d/custom-runners.yml`` +would contain:: + + freebsd-12.1-x86_64-all: + tags: + - freebsd_12.1 + - x86_64 + +It's also recommended to: + + * increase the "Maximum job timeout" to something like ``2h`` + * uncheck the "Run untagged jobs" check box + * give it a better Description
scripts/ci/setup/.gitignore new file +1 -0
@@ -0,0 +1 @@ +vars.yml
scripts/ci/setup/gitlab-runner.yml new file +72 -0
72 lines not on this page · raw message
scripts/ci/setup/vars.yml.template new file +13 -0
@@ -0,0 +1,13 @@ +# The version of the gitlab-runner to use +gitlab_runner_version: 13.1.1 +# The base location of gitlab-runner binaries, this will be suffixed by $OS-$ARCH +gitlab_runner_base_url: https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner- +# The URL of the gitlab server to use, usually https://gitlab.com unless you're +# using a private GitLab instance +gitlab_runner_server_url: https://gitlab.com +# Defaults to linux, checks can be used to change this +gitlab_runner_os: linux +# Defaults to amd64 (x86_64), checks can be used to change this +gitlab_runner_arch: amd64 +# A unique token made available by GitLab to your project for registering runners +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN
Signature · 1 line
2.25.4
Re: [PATCH v4 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook
Quoted 5 lines · Cleber Rosa
To have the jobs dispatched to custom runners, gitlab-runner must be installed, active as a service and properly configured. The variables file and playbook introduced here should help with those steps.
Quoted 7 lines
The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.
Signed-off-by Cleber Rosa <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>---
In general, there's been put quite some effort into the playbooks - sorry I'm late to the game - is there a plan to introduce QEMU as a project to lcitool? We've taken care of most of the bits in the playbooks that are being introduced and for the remaining ones I think it would be that big of an overhaul to do the adjustments. One major re-factor though would IMO be to break the dependency lcitool has on the machine naming, kind of restricting it to a limited set of hosts and corresponding names (e.g. libvirt-fedora-32) which makes it inconvenient to prepare physical hosts.
More comments inline...
Quoted 32 lines
docs/devel/ci.rst | 63 ++++++++++++++++++++++++++ scripts/ci/setup/.gitignore | 1 + scripts/ci/setup/gitlab-runner.yml | 72 ++++++++++++++++++++++++++++++ scripts/ci/setup/vars.yml.template | 13 ++++++ 4 files changed, 149 insertions(+) create mode 100644 scripts/ci/setup/.gitignore create mode 100644 scripts/ci/setup/gitlab-runner.yml create mode 100644 scripts/ci/setup/vars.yml.templatedocs/devel/ci.rst +15
-0@@ -84,3 +84,66 @@ To run the playbook, execute:: cd scripts/ci/setup ansible-playbook -i inventory build-environment.yml + +gitlab-runner setup and registration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The gitlab-runner agent needs to be installed on each machine that +will run jobs. The association between a machine and a GitLab project +happens with a registration token. To find the registration token for +your repository/project, navigate on GitLab's web UI to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * Under "Set up a specific Runner manually", look for the value under + "Use the following registration token during setup" +
docs/devel/ci.rst To run the playbook, execute:: · line 111
Quoted 10 lines
docs/devel/ci.rst +10
-0+Copy the ``scripts/ci/setup/vars.yml.template`` file to +``scripts/ci/setup/vars.yml``. Then, set the +``gitlab_runner_registration_token`` variable to the value obtained +earlier. + +.. note:: gitlab-runner is not available from the standard location + for all OS and architectures combinations. For some systems, + a custom build may be necessary. Some builds are avaiable + at https://cleber.fedorapeople.org/gitlab-runner/ and this + URI may be used as a value on ``vars.yml``
Yes, this can be suboptimal...Would it make sense to fall back to build the binary of a given version from git as a fallback during this playbook if the necessary arch version isn't provided the official way? Just an idea, I'd like to avoid the need for you to become the maintainer of the binaries and keep up with the releases.
docs/devel/ci.rst To run the playbook, execute:: · line 120
Quoted 9 lines
docs/devel/ci.rst +9
-0+ +To run the playbook, execute:: + + cd scripts/ci/setup + ansible-playbook -i inventory gitlab-runner.yml + +.. note:: there are currently limitations to gitlab-runner itself when + setting up a service under FreeBSD systems. You will need to + perform steps 4 to 10 manually, as described at
Which one of them is considered an automation problem? In lcitool we made gitlab-runner completely automated on all distros, including FreeBSD:
4) log file permissions - you're creating the user, you can as well create the file with correct permissions
5) ensure /usr/local/etc/rc.d exists - once you execute build-environment.yml it will pull a bunch of dependencies which ensure the dir exists
6) gitlab_runner service script should IMO provided by this automation as a template and install to the correct location
7) ensure the service script is executable - template module can do that
8) register the runner - you're doing that
9) enable the service - Ansible's service module is generic and supports init/OpenRC
10) I don't see a step 10
IOW I think there should be as little manual intervention as possible and in this case I don't think any manual steps are needed by the user.
Regards,
Erik
Quoted 50 lines
docs/devel/ci.rst +29
-0+ https://docs.gitlab.com/runner/install/freebsd.html + +Following the registration, it's necessary to configure the runner tags, +and optionally other configurations on the GitLab UI. Navigate to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * "Runners activated for this project", then + * Click on the "Edit" icon (next to the "Lock" Icon) + +Under tags, add values matching the jobs a runner should run. For a +FreeBSD 12.1 x86_64 system, the tags should be set as:: + + freebsd12.1,x86_64 + +Because the job definition at ``.gitlab-ci.d/custom-runners.yml`` +would contain:: + + freebsd-12.1-x86_64-all: + tags: + - freebsd_12.1 + - x86_64 + +It's also recommended to: + + * increase the "Maximum job timeout" to something like ``2h`` + * uncheck the "Run untagged jobs" check box + * give it a better Descriptionscripts/ci/setup/.gitignore new file +1
-0@@ -0,0 +1 @@ +vars.ymlscripts/ci/setup/gitlab-runner.yml new file +7
-0@@ -0,0 +1,72 @@ +--- +- name: Installation of gitlab-runner + hosts: all + vars_files: + - vars.yml + tasks: + - debug:
scripts/ci/setup/gitlab-runner.yml line 17
Quoted 10 lines
scripts/ci/setup/gitlab-runner.yml new file +10
-0+ msg: 'Checking for a valid GitLab registration token' + failed_when: "gitlab_runner_registration_token == 'PLEASE_PROVIDE_A_VALID_TOKEN'" + + - name: Checks the availability of official gitlab-runner builds in the archive + uri: + url: https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-linux-386 + method: HEAD + status_code: + - 200 + - 403
Why is 403 an acceptable success status code?
Quoted 77 lines
scripts/ci/setup/gitlab-runner.yml new file +55
-0+ register: gitlab_runner_available_archive + + - name: Update base url + set_fact: + gitlab_runner_base_url: https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner- + when: gitlab_runner_available_archive.status == 200 + - debug: + msg: Base gitlab-runner url is {{ gitlab_runner_base_url }} + + - name: Set OS name (FreeBSD) + set_fact: + gitlab_runner_os: freebsd + when: "ansible_facts['system'] == 'FreeBSD'" + + - name: Create a group for the gitlab-runner service + group: + name: gitlab-runner + + - name: Create a user for the gitlab-runner service + user: + user: gitlab-runner + group: gitlab-runner + comment: GitLab Runner + home: /home/gitlab-runner + shell: /bin/bash + + - name: Remove the .bash_logout file when on Ubuntu systems + file: + path: /home/gitlab-runner/.bash_logout + state: absent + when: "ansible_facts['distribution'] == 'Ubuntu'" + + - name: Downloads the matching gitlab-runner + get_url: + dest: /usr/local/bin/gitlab-runner + url: "{{ gitlab_runner_base_url }}{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}" + owner: gitlab-runner + group: gitlab-runner + mode: u=rwx,g=rwx,o=rx + + - name: Register the gitlab-runner + command: "/usr/local/bin/gitlab-runner register --non-interactive --url {{ gitlab_runner_server_url }} --registration-token {{ gitlab_runner_registration_token }} --executor shell --description '{{ ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"] }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'" + + - name: Install the gitlab-runner service using its own functionality + command: /usr/local/bin/gitlab-runner install --user gitlab-runner --working-directory /home/gitlab-runner + register: gitlab_runner_install_service_result + failed_when: "gitlab_runner_install_service_result.rc != 0 and \"already exists\" not in gitlab_runner_install_service_result.stderr" + when: "ansible_facts['os_family'] != 'FreeBSD'" + + - name: Enable the gitlab-runner service + service: + name: gitlab-runner + state: started + enabled: yes + when: "ansible_facts['os_family'] != 'FreeBSD'"scripts/ci/setup/vars.yml.template new file +13
-0@@ -0,0 +1,13 @@ +# The version of the gitlab-runner to use +gitlab_runner_version: 13.1.1 +# The base location of gitlab-runner binaries, this will be suffixed by $OS-$ARCH +gitlab_runner_base_url: https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner- +# The URL of the gitlab server to use, usually https://gitlab.com unless you're +# using a private GitLab instance +gitlab_runner_server_url: https://gitlab.com +# Defaults to linux, checks can be used to change this +gitlab_runner_os: linux +# Defaults to amd64 (x86_64), checks can be used to change this +gitlab_runner_arch: amd64 +# A unique token made available by GitLab to your project for registering runners +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKENSignature · 1 line
2.25.4
Re: [PATCH v4 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook
Quoted 10 lines · Erik Skultety
Quoted 9 lines · Cleber Rosa
To have the jobs dispatched to custom runners, gitlab-runner must be installed, active as a service and properly configured. The variables file and playbook introduced here should help with those steps.
The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.
Quoted 6 lines
Quoted 3 lines
Signed-off-by Cleber Rosa <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>---
In general, there's been put quite some effort into the playbooks - sorry I'm late to the game - is there a plan to introduce QEMU as a project to lcitool?
I think it's becoming quite clear that having so much duplication (in the dockerfiles, tests/vm, this playbook, etc) is costly and error prone. I don't know if anyone has invested time in a PoC to consolidate those (with lcitool), but I can certainly see the upside to that. BTW, are you volunteering (wink wink)? :)
Quoted 7 lines
We've taken care of most of the bits in the playbooks that are being introduced and for the remaining ones I think it would be that big of an overhaul to do the adjustments. One major re-factor though would IMO be to break the dependency lcitool has on the machine naming, kind of restricting it to a limited set of hosts and corresponding names (e.g. libvirt-fedora-32) which makes it inconvenient to prepare physical hosts.
Right... I wasn't aware of that depedency. And, this may be a nice project to make sure that lcitool doesn't have any other libvirt specificities.
Quoted 42 lines
More comments inline...
Quoted 40 lines
docs/devel/ci.rst | 63 ++++++++++++++++++++++++++ scripts/ci/setup/.gitignore | 1 + scripts/ci/setup/gitlab-runner.yml | 72 ++++++++++++++++++++++++++++++ scripts/ci/setup/vars.yml.template | 13 ++++++ 4 files changed, 149 insertions(+) create mode 100644 scripts/ci/setup/.gitignore create mode 100644 scripts/ci/setup/gitlab-runner.yml create mode 100644 scripts/ci/setup/vars.yml.templatedocs/devel/ci.rst +23
-0@@ -84,3 +84,66 @@ To run the playbook, execute:: cd scripts/ci/setup ansible-playbook -i inventory build-environment.yml + +gitlab-runner setup and registration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The gitlab-runner agent needs to be installed on each machine that +will run jobs. The association between a machine and a GitLab project +happens with a registration token. To find the registration token for +your repository/project, navigate on GitLab's web UI to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * Under "Set up a specific Runner manually", look for the value under + "Use the following registration token during setup" + +Copy the ``scripts/ci/setup/vars.yml.template`` file to +``scripts/ci/setup/vars.yml``. Then, set the +``gitlab_runner_registration_token`` variable to the value obtained +earlier. + +.. note:: gitlab-runner is not available from the standard location + for all OS and architectures combinations. For some systems, + a custom build may be necessary. Some builds are avaiable
Quoted 9 lines
docs/devel/ci.rst To run the playbook, execute:: · line 111
Quoted 2 lines
docs/devel/ci.rst +2
-0+ at https://cleber.fedorapeople.org/gitlab-runner/ and this + URI may be used as a value on ``vars.yml``Yes, this can be suboptimal...Would it make sense to fall back to build the binary of a given version from git as a fallback during this playbook if the necessary arch version isn't provided the official way? Just an idea, I'd like to avoid the need for you to become the maintainer of the binaries and keep up with the releases.
Well, building them during the playbook would be a lot more complex... You can have your own repo with your own builds, and just tweak your vars.yml.
Quoted 4 lines
Quoted 4 lines
docs/devel/ci.rst +4
-0+ +To run the playbook, execute:: + + cd scripts/ci/setup
Quoted 9 lines
docs/devel/ci.rst To run the playbook, execute:: · line 120
Quoted 5 lines
docs/devel/ci.rst +5
-0+ ansible-playbook -i inventory gitlab-runner.yml + +.. note:: there are currently limitations to gitlab-runner itself when + setting up a service under FreeBSD systems. You will need to + perform steps 4 to 10 manually, as described atWhich one of them is considered an automation problem? In lcitool we made gitlab-runner completely automated on all distros, including FreeBSD:
It looks like lcitool went the more practical route. I was hoping to not have to treat gitlab-runner in such a special way in any "supported" OS. What I mean is, I'd rather write the code within gitlab-runner (or reespective libraries). Of course, I did not get to it, so that's why I just documented the steps here.
I'll take a look at lcitool's playbook to see if I can easily canibalize some of that. But, at this time, we don't runners for FreeBSD anyway, so I guess this is not *that* urgent.
Quoted 9 lines
4) log file permissions - you're creating the user, you can as well create the file with correct permissions
5) ensure
/usr/local/etc/rc.dexists - once you execute build-environment.yml it will pull a bunch of dependencies which ensure the dir exists6)
gitlab_runnerservice script should IMO provided by this automation as a template and install to the correct location
Quoted 9 lines
7) ensure the service script is executable - template module can do that
8) register the runner - you're doing that
9) enable the service - Ansible's service module is generic and supports init/OpenRC
10) I don't see a step 10
This was either a mistake or the installation steps got updated. I'll adjust that.
Quoted 3 lines
IOW I think there should be as little manual intervention as possible and in this case I don't think any manual steps are needed by the user.
Agreed. I was not super happy with the current state of this playbook wrt FreeBSD, but I decided to pick other battles to fight. Also, newer gitlab-runner versions *may* be using the updated service management lib, which may reduce/remove the need for custom handling.
Anyway, I'll look at what can be improved here, considering the cost.
Thanks!
- Cleber.
Re: [PATCH v4 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook
Quoted 15 lines · Cleber Rosa
Quoted 14 lines · Erik Skultety
Quoted 12 lines · Cleber Rosa
To have the jobs dispatched to custom runners, gitlab-runner must be installed, active as a service and properly configured. The variables file and playbook introduced here should help with those steps.
The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.
Signed-off-by Cleber Rosa <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>---
Quoted 8 lines
Quoted 2 lines
In general, there's been put quite some effort into the playbooks - sorry I'm late to the game - is there a plan to introduce QEMU as a project to lcitool?
I think it's becoming quite clear that having so much duplication (in the dockerfiles, tests/vm, this playbook, etc) is costly and error prone. I don't know if anyone has invested time in a PoC to consolidate those (with lcitool), but I can certainly see the upside to that. BTW, are you volunteering (wink wink)? :)
I don't think I was trying to :), but sure, I can dedicate some time to it. I'll need a bit of guidance in the QEMU world though for sure.
Quoted 9 lines
Quoted 8 lines
We've taken care of most of the bits in the playbooks that are being introduced and for the remaining ones I think it would be that big of an overhaul to do the adjustments. One major re-factor though would IMO be to break the dependency lcitool has on the machine naming, kind of restricting it to a limited set of hosts and corresponding names (e.g. libvirt-fedora-32) which makes it inconvenient to prepare physical hosts.
Quoted 3 lines
Right... I wasn't aware of that depedency. And, this may be a nice project to make sure that lcitool doesn't have any other libvirt specificities.
Yeah, that one will be a major change. Thinking about this some more, while having a QEMU project inside lcitool sounds definitely useful, it would probably be a wasted effort wrt task priority right now, because without expanding lcitool's range of applicable hosts I don't see how QEMU would utilize the tool without using quirks.
Quoted 53 lines
Quoted 43 lines
More comments inline...
Quoted 40 lines
docs/devel/ci.rst | 63 ++++++++++++++++++++++++++ scripts/ci/setup/.gitignore | 1 + scripts/ci/setup/gitlab-runner.yml | 72 ++++++++++++++++++++++++++++++ scripts/ci/setup/vars.yml.template | 13 ++++++ 4 files changed, 149 insertions(+) create mode 100644 scripts/ci/setup/.gitignore create mode 100644 scripts/ci/setup/gitlab-runner.yml create mode 100644 scripts/ci/setup/vars.yml.templatedocs/devel/ci.rst +23
-0@@ -84,3 +84,66 @@ To run the playbook, execute:: cd scripts/ci/setup ansible-playbook -i inventory build-environment.yml + +gitlab-runner setup and registration +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The gitlab-runner agent needs to be installed on each machine that +will run jobs. The association between a machine and a GitLab project +happens with a registration token. To find the registration token for +your repository/project, navigate on GitLab's web UI to: + + * Settings (the gears like icon), then + * CI/CD, then + * Runners, and click on the "Expand" button, then + * Under "Set up a specific Runner manually", look for the value under + "Use the following registration token during setup" + +Copy the ``scripts/ci/setup/vars.yml.template`` file to +``scripts/ci/setup/vars.yml``. Then, set the +``gitlab_runner_registration_token`` variable to the value obtained +earlier. + +.. note:: gitlab-runner is not available from the standard location + for all OS and architectures combinations. For some systems, + a custom build may be necessary. Some builds are avaiableQuoted 9 lines
docs/devel/ci.rst To run the playbook, execute:: · line 111
Quoted 2 lines
docs/devel/ci.rst +2
-0+ at https://cleber.fedorapeople.org/gitlab-runner/ and this + URI may be used as a value on ``vars.yml``Yes, this can be suboptimal...Would it make sense to fall back to build the binary of a given version from git as a fallback during this playbook if the necessary arch version isn't provided the official way? Just an idea, I'd like to avoid the need for you to become the maintainer of the binaries and keep up with the releases.
Quoted 3 lines
Well, building them during the playbook would be a lot more complex... You can have your own repo with your own builds, and just tweak your vars.yml.
I didn't want to suggest this as a replacement to vars.yml.
What if people wanting to run the playbook don't have their own repo and you (I mean you personally) are not providing a fresh build of gitlab-runner for the given architecture?
The automation machinery would surely be more complex, but at the same time, I can image a dedicated playbook that would handle gitlab-runner building and installation of the binary to the correct place, you'd then simply include that playbook if vars.yml don't specify an alternative repo and download from AWS failed.
Another thing to consider - you currently host the missing arch builds under your personal namespace on fedorapeople, what if you decide one day you're not interested in being involved in QEMU CI that much and suddenly you stop building the binaries, someone has to take over. On the other hand, if you automate this whole process, having exposed the builds would definitely not slow down machine provisions, but would also not cause them to fail if nobody exposes them.
Quoted 21 lines
Quoted 14 lines
docs/devel/ci.rst To run the playbook, execute:: · line 120
Quoted 10 lines
docs/devel/ci.rst +9
-0+ +To run the playbook, execute:: + + cd scripts/ci/setup + ansible-playbook -i inventory gitlab-runner.yml + +.. note:: there are currently limitations to gitlab-runner itself when + setting up a service under FreeBSD systems. You will need to + perform steps 4 to 10 manually, as described atWhich one of them is considered an automation problem? In lcitool we made gitlab-runner completely automated on all distros, including FreeBSD:
It looks like lcitool went the more practical route. I was hoping to not have to treat gitlab-runner in such a special way in any "supported" OS. What I mean is, I'd rather write the code within gitlab-runner (or reespective libraries). Of course, I did not get to it, so that's why I just documented the steps here.
Quoted 3 lines
I'll take a look at lcitool's playbook to see if I can easily canibalize some of that. But, at this time, we don't runners for FreeBSD anyway, so I guess this is not *that* urgent.
Okay, I was simply curious. If it's not something you don't need at the moment, then let's not focus on it.
Quoted 23 lines
Quoted 10 lines
4) log file permissions - you're creating the user, you can as well create the file with correct permissions
5) ensure
/usr/local/etc/rc.dexists - once you execute build-environment.yml it will pull a bunch of dependencies which ensure the dir exists6)
gitlab_runnerservice script should IMO provided by this automation as a template and install to the correct locationQuoted 9 lines
7) ensure the service script is executable - template module can do that
8) register the runner - you're doing that
9) enable the service - Ansible's service module is generic and supports init/OpenRC
10) I don't see a step 10
This was either a mistake or the installation steps got updated. I'll adjust that.
Quoted 8 lines
Quoted 3 lines
IOW I think there should be as little manual intervention as possible and in this case I don't think any manual steps are needed by the user.
Agreed. I was not super happy with the current state of this playbook wrt FreeBSD, but I decided to pick other battles to fight. Also, newer gitlab-runner versions *may* be using the updated service management lib, which may reduce/remove the need for custom handling.
What do you mean by "service management lib"?
Regards,
Erik
Re: [PATCH v4 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook
Quoted 19 lines · Erik Skultety
Quoted 18 lines · Cleber Rosa
Quoted 16 lines · Erik Skultety
Quoted 5 lines · Cleber Rosa
To have the jobs dispatched to custom runners, gitlab-runner must be installed, active as a service and properly configured. The variables file and playbook introduced here should help with those steps.
Quoted 7 lines
The playbook introduced here covers a number of different Linux distributions and FreeBSD, and are intended to provide a reproducible environment.
Signed-off-by Cleber Rosa <[email protected]>Reviewed-by Daniel P. Berrangé <[email protected]>---
In general, there's been put quite some effort into the playbooks - sorry I'm late to the game - is there a plan to introduce QEMU as a project to lcitool?
Quoted 8 lines
Quoted 5 lines
I think it's becoming quite clear that having so much duplication (in the dockerfiles, tests/vm, this playbook, etc) is costly and error prone. I don't know if anyone has invested time in a PoC to consolidate those (with lcitool), but I can certainly see the upside to that. BTW, are you volunteering (wink wink)? :)
I don't think I was trying to :), but sure, I can dedicate some time to it. I'll need a bit of guidance in the QEMU world though for sure.
I think the obvious and easy place is start using lcitool is for the tests/docker/dockerfiles/*. All that's required is to add mappings to lcitool for the various deps that QEMU has which libvirt does not already have. Then we should be able to start auto-generating the dockerfiles without too much difficulty. This will be a significant step forward because it will help us keep te package lists in sync across all the dockerfiles which is a major fail in QEMU right now.
Dealing with tests/vm replacement or these ansible recipes is likely to be a significantly more challenging proposition. Perhaps we can again start by just automating creation of the package lists that the tests/vm and ansibile recipes need, as again those are all inconsistent.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v4 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook
Quoted 7 lines · Daniel P. Berrangé
I think the obvious and easy place is start using lcitool is for the tests/docker/dockerfiles/*. All that's required is to add mappings to lcitool for the various deps that QEMU has which libvirt does not already have. Then we should be able to start auto-generating the dockerfiles without too much difficulty. This will be a significant step forward because it will help us keep te package lists in sync across all the dockerfiles which is a major fail in QEMU right now.
I took a quick look at the contents of those dockerfiles and while I agree that it's the obvious place to start, I'm not sure I would say that generating them using lcitool is going to be easy :)
Basically there seem to be a number of assumptions both in lcitool and in the QEMU dockerfiles, and getting the two to agree on those will require quite some work from what I can tell.
But again, I agree it's something that should be worked on.
Quoted 5 lines
Dealing with tests/vm replacement or these ansible recipes is likely to be a significantly more challenging proposition. Perhaps we can again start by just automating creation of the package lists that the tests/vm and ansibile recipes need, as again those are all inconsistent.
Perhaps 'lcitool variables' could be of use here.
Signature · 1 line
Andrea Bolognani / Red Hat / Virtualization
Re: [PATCH v4 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook
In reply to Erik SkultetyQuoted 10 lines · Erik Skultety
scripts/ci/setup/gitlab-runner.yml line 17
Quoted 8 lines · Cleber Rosa
scripts/ci/setup/gitlab-runner.yml new file +8
-0+ + - name: Checks the availability of official gitlab-runner builds in the archive + uri: + url: https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-linux-386 + method: HEAD + status_code: + - 200 + - 403
Quoted 2 lines
Why is 403 an acceptable success status code?
I missed this question in my last reply, sorry.
s3 will throw a 403 if the URI doesn't exist... and we don't want to fail the playbook because of that, given that we'll attempt the fallback repo defined in vars.yml{,.template}.
Quoted 32 lines
Quoted 32 lines
scripts/ci/setup/gitlab-runner.yml new file +32
-0+ register: gitlab_runner_available_archive + + - name: Update base url + set_fact: + gitlab_runner_base_url: https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner- + when: gitlab_runner_available_archive.status == 200 + - debug: + msg: Base gitlab-runner url is {{ gitlab_runner_base_url }} + + - name: Set OS name (FreeBSD) + set_fact: + gitlab_runner_os: freebsd + when: "ansible_facts['system'] == 'FreeBSD'" + + - name: Create a group for the gitlab-runner service + group: + name: gitlab-runner + + - name: Create a user for the gitlab-runner service + user: + user: gitlab-runner + group: gitlab-runner + comment: GitLab Runner + home: /home/gitlab-runner + shell: /bin/bash + + - name: Remove the .bash_logout file when on Ubuntu systems + file: + path: /home/gitlab-runner/.bash_logout + state: absent + when: "ansible_facts['distribution'] == 'Ubuntu'" +
Quoted 8 lines
scripts/ci/setup/gitlab-runner.yml line 57
Quoted 8 lines
scripts/ci/setup/gitlab-runner.yml new file +8
-0+ - name: Downloads the matching gitlab-runner + get_url: + dest: /usr/local/bin/gitlab-runner + url: "{{ gitlab_runner_base_url }}{{ gitlab_runner_os }}-{{ gitlab_runner_arch }}" + owner: gitlab-runner + group: gitlab-runner + mode: u=rwx,g=rwx,o=rx +
Here, the actual download is performed, and no 40x is considered OK. I hope that explains it.
Cheers,
- Cleber.
Re: [PATCH v4 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook
Quoted 11 lines · Cleber Rosa
Quoted 10 lines · Erik Skultety
Quoted 8 lines · Cleber Rosa
scripts/ci/setup/gitlab-runner.yml new file +8
-0+ + - name: Checks the availability of official gitlab-runner builds in the archive + uri: + url: https://s3.amazonaws.com/gitlab-runner-downloads/v{{ gitlab_runner_version }}/binaries/gitlab-runner-linux-386 + method: HEAD + status_code: + - 200 + - 403
Quoted 8 lines
Quoted 2 lines
Why is 403 an acceptable success status code?
I missed this question in my last reply, sorry.
s3 will throw a 403 if the URI doesn't exist... and we don't want to fail the playbook because of that, given that we'll attempt the fallback repo defined in vars.yml{,.template}.
Right, forgot about the repo in vars.yml fallback, thanks for explaining.
Regards,
Erik
PATCH v4 4/4 Jobs based on custom runners: add job definitions for QEMU's machines
In reply to Cleber RosaThe QEMU project has two machines (aarch64 and s390) that can be used for jobs that do build and run tests. This introduces those jobs, which are a mapping of custom scripts used for the same purpose.
--- .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++ 1 file changed, 192 insertions(+)
.gitlab-ci.d/custom-runners.yml +192 -0
@@ -12,3 +12,195 @@ # strategy. variables: GIT_SUBMODULE_STRATEGY: recursive + +# All ubuntu-18.04 jobs should run successfully in an environment +# setup by the scripts/ci/setup/build-environment.yml task +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04" +ubuntu-18.04-s390x-all-linux-static: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763 + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages + - mkdir build + - cd build + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + - make --output-sync -j`nproc` check-tcg V=1 + +ubuntu-18.04-s390x-all: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-18.04-s390x-alldbg: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --enable-debug --disable-libssh + - make clean + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-18.04-s390x-clang: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-18.04-s390x-tci: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --enable-tcg-interpreter + - make --output-sync -j`nproc` + +ubuntu-18.04-s390x-notcg: + needs: [] + stage: build + tags: + - ubuntu_18.04 + - s390x + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --disable-tcg + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +# All ubuntu-20.04 jobs should run successfully in an environment +# setup by the scripts/ci/setup/qemu/build-environment.yml task +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04" +ubuntu-20.04-aarch64-all-linux-static: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763 + # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages + - mkdir build + - cd build + - ../configure --enable-debug --static --disable-system --disable-glusterfs --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + - make --output-sync -j`nproc` check-tcg V=1 + +ubuntu-20.04-aarch64-all: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-20.04-aarch64-alldbg: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --enable-debug --disable-libssh + - make clean + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-20.04-aarch64-clang: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --cc=clang --cxx=clang++ --enable-sanitizers + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1 + +ubuntu-20.04-aarch64-tci: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --enable-tcg-interpreter + - make --output-sync -j`nproc` + +ubuntu-20.04-aarch64-notcg: + needs: [] + stage: build + tags: + - ubuntu_20.04 + - aarch64 + rules: + - if: '$CI_COMMIT_BRANCH =~ /^staging/' + script: + - mkdir build + - cd build + - ../configure --disable-libssh --disable-tcg + - make --output-sync -j`nproc` + - make --output-sync -j`nproc` check V=1
Signature · 1 line
2.25.4
Re: [PATCH v4 4/4] Jobs based on custom runners: add job definitions for QEMU's machines
Quoted 8 lines · Cleber Rosa
The QEMU project has two machines (aarch64 and s390) that can be used for jobs that do build and run tests. This introduces those jobs, which are a mapping of custom scripts used for the same purpose.
Signed-off-by Cleber Rosa <[email protected]>--- .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++ 1 file changed, 192 insertions(+)
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v4 4/4] Jobs based on custom runners: add job definitions for QEMU's machines
In reply to Cleber RosaQuoted 1 line · Cleber Rosa
The QEMU project has two machines (aarch64 and s390) that can be used
"s390x" =)
Quoted 7 lines
for jobs that do build and run tests. This introduces those jobs, which are a mapping of custom scripts used for the same purpose.
Signed-off-by Cleber Rosa <[email protected]>--- .gitlab-ci.d/custom-runners.yml | 192 ++++++++++++++++++++++++++++++++ 1 file changed, 192 insertions(+)
Re: [PATCH v4 4/4] Jobs based on custom runners: add job definitions for QEMU's machines
Quoted 5 lines · Philippe Mathieu-Daudé
Quoted 1 line · Cleber Rosa
The QEMU project has two machines (aarch64 and s390) that can be used
"s390x" =)
Sure!
Thanks!
- Cleber.
Quoted 6 lines · Cleber Rosa
TL;DR: this should allow the QEMU maintainer to push to the staging branch, and have custom jobs running on the project's aarch64 and s390x machines. Simple usage looks like:
git push remote staging
./scripts/ci/gitlab-pipeline-status--verbose --wait
Hi Cleber,
since the QEMU project is now using the gitlab repo as main repository, it should now be possible to finally enable these custom runners, too, I think.
Could you elaborate on what's left to do to get the s390x and aarch64 runners enabled, and rebase your patches if necessary?
Thanks,
Thomas
Quoted 5 lines · Thomas Huth
Quoted 4 lines · Cleber Rosa
TL;DR: this should allow the QEMU maintainer to push to the staging branch, and have custom jobs running on the project's aarch64 and s390x machines. Simple usage looks like:
Quoted 8 lines
Quoted 2 lines
git push remote staging
./scripts/ci/gitlab-pipeline-status--verbose --waitHi Cleber,
since the QEMU project is now using the gitlab repo as main repository, it should now be possible to finally enable these custom runners, too, I think.
I couldn't agree more!
Quoted 3 lines
Could you elaborate on what's left to do to get the s390x and aarch64 runners enabled, and rebase your patches if necessary?
Basically what's needed is to enable the jobs, meaning committing the relevant snippets to the "gitlab-ci.{yml,d/*}" files.
Both ansible playbooks have been run on both machines, so the gitlab-runner and tooling to build QEMU are in place. The gitlab-runners have also been registered on the "qemu-project/qemu" GitLab repo.
I'll send a rebased/updated version of the playbooks, gitlab-ci.yml configurations, etc. In it, I'll propose that we enable the jobs, but ignore errors for now (allow_failure: true), because it'll ease the review. Once the jobs prove mature, waiving of failures can be removed.
How does that sound?
Thanks,
- Cleber.
Quoted 2 lines
Thanks,
Thomas
Quoted 1 line · Cleber Rosa
On Thu, Jan 28, 2021 at 12:51:20PM +0100, Thomas Huth wrote:
[...]
Quoted 10 lines
Quoted 2 lines
Could you elaborate on what's left to do to get the s390x and aarch64 runners enabled, and rebase your patches if necessary
Basically what's needed is to enable the jobs, meaning committing the relevant snippets to the "gitlab-ci.{yml,d/*}" files.
Both ansible playbooks have been run on both machines, so the gitlab-runner and tooling to build QEMU are in place. The gitlab-runners have also been registered on the "qemu-project/qemu" GitLab repo.
Quoted 7 lines
I'll send a rebased/updated version of the playbooks, gitlab-ci.yml configurations, etc. In it, I'll propose that we enable the jobs, but ignore errors for now (
allow_failure: true), because it'll ease the review. Once the jobs prove mature, waiving of failures can be removed.How does that sound?
Sounds good, please go ahead!
Thanks,
Thomas