Skip to contentSkip to conversation

gitlab-ci.d/custom-runners: Improve rules for the staging branch

PATCH In review Thomas Huth 3 participants 4 messages Jul 28, 2021 – Jul 29, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 gitlab-ci.d/custom-runners: Improve rules for the staging branch

If maintainers are currently pushing to a branch called "staging" in their repository, they are ending up with some stuck jobs - unless they have a s390x CI runner machine available. That's ugly, we should make sure that the related jobs are really only started if such a runner is available. So let's only run these jobs if it's the "staging" branch of the main repository of the QEMU project (where we can be sure that the s390x runner is available), or if the user explicitly set a S390X_RUNNER_AVAILABLE variable in their CI configs to declare that they have such a runner available, too.

Fixes 4799c21023 ("Jobs based on custom runners: add job definitions ...")
---
 .gitlab-ci.d/custom-runners.yml | 40 +++++++++++++++++++++++----------
 1 file changed, 28 insertions(+), 12 deletions(-)
.gitlab-ci.d/custom-runners.yml +28 -12
index 061d3cdfed..564b94565d 100644
@@ -24,7 +24,8 @@ ubuntu-18.04-s390x-all-linux-static:
  - ubuntu_18.04
  - s390x
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+ - if: "$S390X_RUNNER_AVAILABLE"
  script:
  # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
  # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
@@ -43,7 +44,8 @@ ubuntu-18.04-s390x-all:
  - ubuntu_18.04
  - s390x
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+ - if: "$S390X_RUNNER_AVAILABLE"
  script:
  - mkdir build
  - cd build
@@ -59,7 +61,8 @@ ubuntu-18.04-s390x-alldbg:
  - ubuntu_18.04
  - s390x
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+ - if: "$S390X_RUNNER_AVAILABLE"
  script:
  - mkdir build
  - cd build
@@ -76,7 +79,9 @@ ubuntu-18.04-s390x-clang:
  - ubuntu_18.04
  - s390x
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+   when: manual
+ - if: "$S390X_RUNNER_AVAILABLE"
    when: manual
  script:
  - mkdir build
@@ -93,7 +98,8 @@ ubuntu-18.04-s390x-tci:
  - ubuntu_18.04
  - s390x
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+ - if: "$S390X_RUNNER_AVAILABLE"
  script:
  - mkdir build
  - cd build
@@ -108,7 +114,9 @@ ubuntu-18.04-s390x-notcg:
  - ubuntu_18.04
  - s390x
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+   when: manual
+ - if: "$S390X_RUNNER_AVAILABLE"
    when: manual
  script:
  - mkdir build
@@ -128,7 +136,8 @@ ubuntu-20.04-aarch64-all-linux-static:
  - ubuntu_20.04
  - aarch64
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+ - if: "$S390X_RUNNER_AVAILABLE"
  script:
  # --disable-libssh is needed because of https://bugs.launchpad.net/qemu/+bug/1838763
  # --disable-glusterfs is needed because there's no static version of those libs in distro supplied packages
@@ -147,7 +156,8 @@ ubuntu-20.04-aarch64-all:
  - ubuntu_20.04
  - aarch64
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+ - if: "$S390X_RUNNER_AVAILABLE"
  script:
  - mkdir build
  - cd build
@@ -163,7 +173,8 @@ ubuntu-20.04-aarch64-alldbg:
  - ubuntu_20.04
  - aarch64
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+ - if: "$S390X_RUNNER_AVAILABLE"
  script:
  - mkdir build
  - cd build
@@ -180,7 +191,9 @@ ubuntu-20.04-aarch64-clang:
  - ubuntu_20.04
  - aarch64
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+   when: manual
+ - if: "$S390X_RUNNER_AVAILABLE"
    when: manual
  script:
  - mkdir build
@@ -197,7 +210,8 @@ ubuntu-20.04-aarch64-tci:
  - ubuntu_20.04
  - aarch64
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+ - if: "$S390X_RUNNER_AVAILABLE"
  script:
  - mkdir build
  - cd build
@@ -212,7 +226,9 @@ ubuntu-20.04-aarch64-notcg:
  - ubuntu_20.04
  - aarch64
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+   when: manual
+ - if: "$S390X_RUNNER_AVAILABLE"
    when: manual
  script:
  - mkdir build
Signature · 1 line
2.27.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 21 lines · Thomas Huth

If maintainers are currently pushing to a branch called "staging" in their repository, they are ending up with some stuck jobs - unless they have a s390x CI runner machine available. That's ugly, we should make sure that the related jobs are really only started if such a runner is available. So let's only run these jobs if it's the "staging" branch of the main repository of the QEMU project (where we can be sure that the s390x runner is available), or if the user explicitly set a S390X_RUNNER_AVAILABLE variable in their CI configs to declare that they have such a runner available, too.

Fixes 4799c21023 ("Jobs based on custom runners: add job definitions ...")
---
 .gitlab-ci.d/custom-runners.yml | 40 +++++++++++++++++++++++----------
 1 file changed, 28 insertions(+), 12 deletions(-)
.gitlab-ci.d/custom-runners.yml +0 -0
index 061d3cdfed..564b94565d 100644

.gitlab-ci.d/custom-runners.yml ubuntu-18.04-s390x-all-linux-static: · line 28

Quoted 6 lines
.gitlab-ci.d/custom-runners.yml +2 -1
@@ -24,7 +24,8 @@ ubuntu-18.04-s390x-all-linux-static:
  - ubuntu_18.04
  - s390x
  rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+ - if: "$S390X_RUNNER_AVAILABLE"

If you base this patch on top of "docs: Document GitLab custom CI/CD variables" that you already queued, you can directly add a description for S390X_RUNNER_AVAILABLE in docs/devel/ci.rst, but this can be done later too.

Regardless of whether docs/devel/ci.rst is updated:

#2 lore raw
Quoted 17 lines · Philippe Mathieu-Daudé
Quoted 16 lines · Thomas Huth

If maintainers are currently pushing to a branch called "staging" in their repository, they are ending up with some stuck jobs - unless they have a s390x CI runner machine available. That's ugly, we should make sure that the related jobs are really only started if such a runner is available. So let's only run these jobs if it's the "staging" branch of the main repository of the QEMU project (where we can be sure that the s390x runner is available), or if the user explicitly set a S390X_RUNNER_AVAILABLE variable in their CI configs to declare that they have such a runner available, too.

Fixes 4799c21023 ("Jobs based on custom runners: add job definitions ...")
---
  .gitlab-ci.d/custom-runners.yml | 40 +++++++++++++++++++++++----------
  1 file changed, 28 insertions(+), 12 deletions(-)
Quoted 16 lines

.gitlab-ci.d/custom-runners.yml ubuntu-18.04-s390x-all-linux-static: · line 28

Quoted 11 lines
.gitlab-ci.d/custom-runners.yml +2 -1
index 061d3cdfed..564b94565d 100644
@@ -24,7 +24,8 @@ ubuntu-18.04-s390x-all-linux-static:
   - ubuntu_18.04
   - s390x
   rules:
- - if: '$CI_COMMIT_BRANCH =~ /^staging/'
+ - if: '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
+ - if: "$S390X_RUNNER_AVAILABLE"

If you base this patch on top of "docs: Document GitLab custom CI/CD variables" that you already queued, you can directly add a description for S390X_RUNNER_AVAILABLE in docs/devel/ci.rst, but this can be done later too.

Good idea! But I really want to get this out of the door to finally get a usable gitlab-CI again, so I'll rather send a patch for this later.

Thanks,
Thomas

Reviewed-by on v1 1/1
#3 lore raw
In reply to Thomas Huth
Quoted 11 lines · Thomas Huth

If maintainers are currently pushing to a branch called "staging" in their repository, they are ending up with some stuck jobs - unless they have a s390x CI runner machine available. That's ugly, we should make sure that the related jobs are really only started if such a runner is available. So let's only run these jobs if it's the "staging" branch of the main repository of the QEMU project (where we can be sure that the s390x runner is available), or if the user explicitly set a S390X_RUNNER_AVAILABLE variable in their CI configs to declare that they have such a runner available, too.

Quoted 6 lines
Fixes 4799c21023 ("Jobs based on custom runners: add job definitions ...")
---
 .gitlab-ci.d/custom-runners.yml | 40 +++++++++++++++++++++++----------
 1 file changed, 28 insertions(+), 12 deletions(-)