ci improvements
cover ci improvements
Primarily speeds up lava jobs by using our own container.
Jan
Jan Kiszka (3):
ci: Do not attach cve-checks to scheduled pipelines
ci: Add job to build own lavacli container for tests
ci: Switch test jobs to self-built lavacli container
.gitlab-ci.yml | 22 +++++++++++++++++++--- scripts/Dockerfile.lavacli | 17 +++++++++++++++++ scripts/submit_lava.sh | 2 -- 3 files changed, 36 insertions(+), 5 deletions(-) create mode 100644 scripts/Dockerfile.lavacli
Signature · 1 line
2.51.0
PATCH 1/3 ci: Do not attach cve-checks to scheduled pipelines
From: Jan Kiszka <[email protected]>
This one is only relevant for regular isar-cip-core image builds.
--- .gitlab-ci.yml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
.gitlab-ci.yml +4 -1
@@ -663,7 +663,10 @@ cve-checks: - large script: - scripts/run-cve-checks.sh - when: manual + rules: + - if: '$CI_PIPELINE_SOURCE == "schedule"' + when: never + - when: manual allow_failure: true artifacts: expire_in: 1 day
Signature · 1 line
2.51.0
PATCH 2/3 ci: Add job to build own lavacli container for tests
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
This shall accelerate the startup of our lava tests by using a base image which already contains all the required dependencies. That image will be re-generated every months by a scheduled pipeline and stored in the gitlab registry of the isar-cip-core repository by default.
--- .gitlab-ci.yml | 13 +++++++++++++ scripts/Dockerfile.lavacli | 17 +++++++++++++++++ 2 files changed, 30 insertions(+) create mode 100644 scripts/Dockerfile.lavacli
.gitlab-ci.yml +13 -0
@@ -674,3 +674,16 @@ cve-checks: - cve-reports include: '.reproducible-check-ci.yml' + +lavacli-container: + stage: build + image: quay.io/buildah/stable:v1.41.4 + tags: + - large + before_script: + - buildah login -u $CI_REGISTRY_USER --password $CI_REGISTRY_PASSWORD $CI_REGISTRY + script: + - buildah build -f scripts/Dockerfile.lavacli -t $CI_REGISTRY_IMAGE/lavacli . + - buildah push $CI_REGISTRY_IMAGE/lavacli + rules: + - if: '$CI_PIPELINE_SOURCE == "schedule" && $LAVACLI_CONTAINER'
scripts/Dockerfile.lavacli new file +17 -0
@@ -0,0 +1,17 @@ +# +# CIP Core, generic profile +# +# Copyright (c) Siemens, 2025 +# +# Authors: +# Jan Kiszka <[email protected]> +# +# SPDX-License-Identifier: MIT +# + +FROM debian:stable + +ARG DEBIAN_FRONTEND=noninteractive + +RUN apt-get update && \ + apt-get install -y --no-install-recommends gettext-base curl xmlstarlet lavacli
Signature · 1 line
2.51.0
RE: [cip-dev] [isar-cip-core][PATCH 2/3] ci: Add job to build own lavacli container for tests
Hello Jan,
Quoted 37 lines
From: [email protected] <[email protected]> On Behalf Of Jan Kiszka via lists.cip-project.org
Sent: 17 September 2025 09:01From: Jan Kiszka <[email protected]>
This shall accelerate the startup of our lava tests by using a base image which already contains all the required dependencies. That image will be re-generated every months by a scheduled pipeline and stored in the gitlab registry of the isar-cip-core repository by default.
Signed-off-by Jan Kiszka <[email protected]>--- .gitlab-ci.yml | 13 +++++++++++++ scripts/Dockerfile.lavacli | 17 +++++++++++++++++ 2 files changed, 30 insertions(+) create mode 100644 scripts/Dockerfile.lavacli.gitlab-ci.yml +8
-0@@ -674,3 +674,16 @@ cve-checks: - cve-reports include: '.reproducible-check-ci.yml' + +lavacli-container: + stage: build + image: quay.io/buildah/stable:v1.41.4 + tags: + - large + before_script: + - buildah login -u $CI_REGISTRY_USER --password $CI_REGISTRY_PASSWORD$CI_REGISTRY
.gitlab-ci.yml +2
-0+ script: + - buildah build -f scripts/Dockerfile.lavacli -t
Quoted 1 line
$CI_REGISTRY_IMAGE/lavacli .
Nice. I haven't seen buildah before.
Quoted 1 line
+ - buildah push $CI_REGISTRY_IMAGE/lavacli
Normally I'd worry about the "official" container being overwritten by development branches pushed to the same repo. However, I guess you're protected against that due to this job only running via a schedule.
.gitlab-ci.yml cve-checks: · line 689
Quoted 2 lines
.gitlab-ci.yml +2
-0+ rules: + - if: '$CI_PIPELINE_SOURCE == "schedule" && $LAVACLI_CONTAINER'
Maybe further protection could be added to only run it on the default branch, but probably overkill as you will already only run if $LAVACLI_CONTAINER is set.
Kind regards, Chris
Quoted 26 lines
scripts/Dockerfile.lavacli new file +17
-0@@ -0,0 +1,17 @@ +# +# CIP Core, generic profile +# +# Copyright (c) Siemens, 2025 +# +# Authors: +# Jan Kiszka <[email protected]> +# +# SPDX-License-Identifier: MIT +# + +FROM debian:stable + +ARG DEBIAN_FRONTEND=noninteractive + +RUN apt-get update && \ + apt-get install -y --no-install-recommends gettext-base curlxmlstarlet lavacli
--
2.51.0
Re: [cip-dev] [isar-cip-core][PATCH 2/3] ci: Add job to build own lavacli container for tests
Quoted 43 lines · Chris Paterson
Hello Jan,
Quoted 37 lines
From: [email protected] <[email protected]> On Behalf Of Jan Kiszka via lists.cip-project.org
Sent: 17 September 2025 09:01From: Jan Kiszka <[email protected]>
This shall accelerate the startup of our lava tests by using a base image which already contains all the required dependencies. That image will be re-generated every months by a scheduled pipeline and stored in the gitlab registry of the isar-cip-core repository by default.
Signed-off-by Jan Kiszka <[email protected]>--- .gitlab-ci.yml | 13 +++++++++++++ scripts/Dockerfile.lavacli | 17 +++++++++++++++++ 2 files changed, 30 insertions(+) create mode 100644 scripts/Dockerfile.lavacli.gitlab-ci.yml +8
-0@@ -674,3 +674,16 @@ cve-checks: - cve-reports include: '.reproducible-check-ci.yml' + +lavacli-container: + stage: build + image: quay.io/buildah/stable:v1.41.4 + tags: + - large + before_script: + - buildah login -u $CI_REGISTRY_USER --password $CI_REGISTRY_PASSWORD$CI_REGISTRY
.gitlab-ci.yml +2
-0+ script: + - buildah build -f scripts/Dockerfile.lavacli -tQuoted 1 line
$CI_REGISTRY_IMAGE/lavacli .
Nice. I haven't seen buildah before.
Quoted 5 lines
Quoted 1 line
+ - buildah push $CI_REGISTRY_IMAGE/lavacli
Normally I'd worry about the "official" container being overwritten by development branches pushed to the same repo. However, I guess you're protected against that due to this job only running via a schedule.
Yes, it should not collide with running jobs, but even if it would, you should either get the old image are already the new one once the latest tag is moved. We are using that pattern for a while in
https://gitlab.com/Xenomai/xenomai-images.
Quoted 6 lines
.gitlab-ci.yml cve-checks: · line 689
Quoted 3 lines
.gitlab-ci.yml +2
-0+ rules: + - if: '$CI_PIPELINE_SOURCE == "schedule" && $LAVACLI_CONTAINER'Maybe further protection could be added to only run it on the default branch, but probably overkill as you will already only run if $LAVACLI_CONTAINER is set.
Right, you need to pass LAVACLI_CONTAINER along the pipeline to get activated. Same pattern as we already use for the weekly reproducibility check.
Quoted 4 lines
Reviewed-by Chris Paterson <[email protected]>Kind regards, Chris
Thanks!
Jan
Signature · 2 lines
Siemens AG, Foundational Technologies Linux Expert Center
PATCH 3/3 ci: Switch test jobs to self-built lavacli container
In reply to Jan KiszkaFrom: Jan Kiszka <[email protected]>
This accelerates the startup of the test jobs.
--- .gitlab-ci.yml | 4 ++-- scripts/submit_lava.sh | 2 -- 2 files changed, 2 insertions(+), 4 deletions(-)
.gitlab-ci.yml +2 -2
@@ -12,8 +12,6 @@ # SPDX-License-Identifier: MIT # -image: ghcr.io/siemens/kas/kas-isar:4.8.1 - variables: GIT_STRATEGY: clone release: bookworm
@@ -47,6 +45,7 @@ default: .build_base: stage: build + image: ghcr.io/siemens/kas/kas-isar:4.8.1 rules: - if: $CI_COMMIT_TAG when: never
@@ -344,6 +343,7 @@ build:qemu-riscv64: .test-cip-core: stage: test + image: $CI_REGISTRY_IMAGE/lavacli rules: - if: $CI_COMMIT_TAG when: never
scripts/submit_lava.sh +0 -2
@@ -38,8 +38,6 @@ image_args[qemu-arm64]="-cpu cortex-a57 -machine virt -smp 4 -device virtio-seri image_args[qemu-arm]="-cpu cortex-a15 -machine virt -smp 2 -device virtio-serial-device -device virtconsole,chardev=con -chardev vc,id=con -device virtio-blk-device,drive=disk -device virtio-net-device,netdev=net -device tpm-tis-device,tpmdev=tpm0" set_up (){ - echo "Installing dependencies to run this script..." - sudo apt update && sudo apt install -y --no-install-recommends lavacli curl xmlstarlet job_dir="$(mktemp -d)" }
Signature · 1 line
2.51.0