Skip to contentSkip to conversation

ci improvements

PATCH In review Jan Kiszka 2 participants 6 messages Sep 17, 2025

lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

cover ci improvements

Primarily speeds up lava jobs by using our own container.

Jan

Jan Kiszka (3):
ci: Do not attach cve-checks to scheduled pipelines
ci: Add job to build own lavacli container for tests
ci: Switch test jobs to self-built lavacli container

 .gitlab-ci.yml             | 22 +++++++++++++++++++---
 scripts/Dockerfile.lavacli | 17 +++++++++++++++++
 scripts/submit_lava.sh     |  2 --
 3 files changed, 36 insertions(+), 5 deletions(-)
 create mode 100644 scripts/Dockerfile.lavacli
Signature · 1 line
2.51.0
author
#1 lore raw

PATCH 1/3 ci: Do not attach cve-checks to scheduled pipelines

From: Jan Kiszka <[email protected]>

This one is only relevant for regular isar-cip-core image builds.

---
 .gitlab-ci.yml | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
.gitlab-ci.yml +4 -1
index 7f0f4503..d951e160 100644
@@ -663,7 +663,10 @@ cve-checks:
     - large
   script:
     - scripts/run-cve-checks.sh
-  when: manual
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "schedule"'
+      when: never
+    - when: manual
   allow_failure: true
   artifacts:
     expire_in: 1 day
Signature · 1 line
2.51.0
author
#2 lore raw

PATCH 2/3 ci: Add job to build own lavacli container for tests

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

This shall accelerate the startup of our lava tests by using a base image which already contains all the required dependencies. That image will be re-generated every months by a scheduled pipeline and stored in the gitlab registry of the isar-cip-core repository by default.

---
 .gitlab-ci.yml             | 13 +++++++++++++
 scripts/Dockerfile.lavacli | 17 +++++++++++++++++
 2 files changed, 30 insertions(+)
 create mode 100644 scripts/Dockerfile.lavacli
.gitlab-ci.yml +13 -0
index d951e160..de082272 100644
@@ -674,3 +674,16 @@ cve-checks:
       - cve-reports
 
 include: '.reproducible-check-ci.yml'
+
+lavacli-container:
+  stage: build
+  image: quay.io/buildah/stable:v1.41.4
+  tags:
+    - large
+  before_script:
+    - buildah login -u $CI_REGISTRY_USER --password $CI_REGISTRY_PASSWORD $CI_REGISTRY
+  script:
+    - buildah build -f scripts/Dockerfile.lavacli -t $CI_REGISTRY_IMAGE/lavacli .
+    - buildah push $CI_REGISTRY_IMAGE/lavacli
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "schedule" && $LAVACLI_CONTAINER'
scripts/Dockerfile.lavacli new file +17 -0
new file mode 100644
index 00000000..763ed4dd
@@ -0,0 +1,17 @@
+#
+# CIP Core, generic profile
+#
+# Copyright (c) Siemens, 2025
+#
+# Authors:
+#  Jan Kiszka <[email protected]>
+#
+# SPDX-License-Identifier: MIT
+#
+
+FROM debian:stable
+
+ARG DEBIAN_FRONTEND=noninteractive
+
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends gettext-base curl xmlstarlet lavacli
Signature · 1 line
2.51.0
Reviewed-by on v1 2/3
#3 lore raw

RE: [cip-dev] [isar-cip-core][PATCH 2/3] ci: Add job to build own lavacli container for tests

Hello Jan,

Quoted 37 lines

From: [email protected] <[email protected]> On Behalf Of Jan Kiszka via lists.cip-project.org
Sent: 17 September 2025 09:01

From: Jan Kiszka <[email protected]>

This shall accelerate the startup of our lava tests by using a base image which already contains all the required dependencies. That image will be re-generated every months by a scheduled pipeline and stored in the gitlab registry of the isar-cip-core repository by default.

---
 .gitlab-ci.yml             | 13 +++++++++++++
 scripts/Dockerfile.lavacli | 17 +++++++++++++++++
 2 files changed, 30 insertions(+)
 create mode 100644 scripts/Dockerfile.lavacli
.gitlab-ci.yml +8 -0
index d951e160..de082272 100644
@@ -674,3 +674,16 @@ cve-checks:
       - cve-reports
 
 include: '.reproducible-check-ci.yml'
+
+lavacli-container:
+  stage: build
+  image: quay.io/buildah/stable:v1.41.4
+  tags:
+    - large
+  before_script:
+    - buildah login -u $CI_REGISTRY_USER --password $CI_REGISTRY_PASSWORD

$CI_REGISTRY

.gitlab-ci.yml +2 -0
+  script:
+    - buildah build -f scripts/Dockerfile.lavacli -t
Quoted 1 line

$CI_REGISTRY_IMAGE/lavacli .

Nice. I haven't seen buildah before.

Quoted 1 line

+ - buildah push $CI_REGISTRY_IMAGE/lavacli

Normally I'd worry about the "official" container being overwritten by development branches pushed to the same repo. However, I guess you're protected against that due to this job only running via a schedule.

.gitlab-ci.yml cve-checks: · line 689

Quoted 2 lines
.gitlab-ci.yml +2 -0
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "schedule" && $LAVACLI_CONTAINER'

Maybe further protection could be added to only run it on the default branch, but probably overkill as you will already only run if $LAVACLI_CONTAINER is set.

Kind regards, Chris

Quoted 26 lines
scripts/Dockerfile.lavacli new file +17 -0
new file mode 100644
index 00000000..763ed4dd
@@ -0,0 +1,17 @@
+#
+# CIP Core, generic profile
+#
+# Copyright (c) Siemens, 2025
+#
+# Authors:
+#  Jan Kiszka <[email protected]>
+#
+# SPDX-License-Identifier: MIT
+#
+
+FROM debian:stable
+
+ARG DEBIAN_FRONTEND=noninteractive
+
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends gettext-base curl

xmlstarlet lavacli
--
2.51.0

author
#4 lore raw

Re: [cip-dev] [isar-cip-core][PATCH 2/3] ci: Add job to build own lavacli container for tests

Quoted 43 lines · Chris Paterson

Hello Jan,

Quoted 37 lines

From: [email protected] <[email protected]> On Behalf Of Jan Kiszka via lists.cip-project.org
Sent: 17 September 2025 09:01

From: Jan Kiszka <[email protected]>

This shall accelerate the startup of our lava tests by using a base image which already contains all the required dependencies. That image will be re-generated every months by a scheduled pipeline and stored in the gitlab registry of the isar-cip-core repository by default.

---
 .gitlab-ci.yml             | 13 +++++++++++++
 scripts/Dockerfile.lavacli | 17 +++++++++++++++++
 2 files changed, 30 insertions(+)
 create mode 100644 scripts/Dockerfile.lavacli
.gitlab-ci.yml +8 -0
index d951e160..de082272 100644
@@ -674,3 +674,16 @@ cve-checks:
       - cve-reports
 
 include: '.reproducible-check-ci.yml'
+
+lavacli-container:
+  stage: build
+  image: quay.io/buildah/stable:v1.41.4
+  tags:
+    - large
+  before_script:
+    - buildah login -u $CI_REGISTRY_USER --password $CI_REGISTRY_PASSWORD

$CI_REGISTRY

.gitlab-ci.yml +2 -0
+  script:
+    - buildah build -f scripts/Dockerfile.lavacli -t
Quoted 1 line

$CI_REGISTRY_IMAGE/lavacli .

Nice. I haven't seen buildah before.

Quoted 5 lines
Quoted 1 line

+ - buildah push $CI_REGISTRY_IMAGE/lavacli

Normally I'd worry about the "official" container being overwritten by development branches pushed to the same repo. However, I guess you're protected against that due to this job only running via a schedule.

Yes, it should not collide with running jobs, but even if it would, you should either get the old image are already the new one once the latest tag is moved. We are using that pattern for a while in
https://gitlab.com/Xenomai/xenomai-images.

Quoted 6 lines

.gitlab-ci.yml cve-checks: · line 689

Quoted 3 lines
.gitlab-ci.yml +2 -0
+  rules:
+    - if: '$CI_PIPELINE_SOURCE == "schedule" && $LAVACLI_CONTAINER'

Maybe further protection could be added to only run it on the default branch, but probably overkill as you will already only run if $LAVACLI_CONTAINER is set.

Right, you need to pass LAVACLI_CONTAINER along the pipeline to get activated. Same pattern as we already use for the weekly reproducibility check.

Quoted 4 lines

Kind regards, Chris

Thanks!

Jan

Signature · 2 lines
Siemens AG, Foundational Technologies
Linux Expert Center
author
#5 lore raw

PATCH 3/3 ci: Switch test jobs to self-built lavacli container

In reply to Jan Kiszka

From: Jan Kiszka <[email protected]>

This accelerates the startup of the test jobs.

---
 .gitlab-ci.yml         | 4 ++--
 scripts/submit_lava.sh | 2 --
 2 files changed, 2 insertions(+), 4 deletions(-)
.gitlab-ci.yml +2 -2
index de082272..99c8013b 100644
@@ -12,8 +12,6 @@
 # SPDX-License-Identifier: MIT
 #
 
-image: ghcr.io/siemens/kas/kas-isar:4.8.1
-
 variables:
   GIT_STRATEGY: clone
   release: bookworm
@@ -47,6 +45,7 @@ default:
 
 .build_base:
   stage: build
+  image: ghcr.io/siemens/kas/kas-isar:4.8.1
   rules:
     - if: $CI_COMMIT_TAG
       when: never
@@ -344,6 +343,7 @@ build:qemu-riscv64:
 
 .test-cip-core:
   stage: test
+  image: $CI_REGISTRY_IMAGE/lavacli
   rules:
     - if: $CI_COMMIT_TAG
       when: never
scripts/submit_lava.sh +0 -2
index 5cb4430c..ea733748 100755
@@ -38,8 +38,6 @@ image_args[qemu-arm64]="-cpu cortex-a57 -machine virt -smp 4 -device virtio-seri
 image_args[qemu-arm]="-cpu cortex-a15 -machine virt -smp 2 -device virtio-serial-device -device virtconsole,chardev=con -chardev vc,id=con -device virtio-blk-device,drive=disk -device virtio-net-device,netdev=net -device tpm-tis-device,tpmdev=tpm0"
 
 set_up (){
-	echo "Installing dependencies to run this script..."
-	sudo apt update && sudo apt install -y --no-install-recommends lavacli curl xmlstarlet
 	job_dir="$(mktemp -d)"
 }
 
Signature · 1 line
2.51.0