Skip to contentSkip to conversation

automation: auto-build container, check generated file

PATCH In review Anthony PERARD 1 participant 5 messages Mar 1, 2022

lore
Share & tools
lore
#0 lore raw

cover automation: auto-build container, check generated file

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.gitlab-ci-build-container-v1

I wanted to automatically check that generated files that we have in our repo are up-to-date, those are autoconf and *.gen.go files generataed from libxl_types.idl. But this can only be check on debian stable. So new container, and I look at building the new container from the CI.

The second patch depends on some changes to our GitLab runner:
https://gitlab.com/xen-project/xen-gitlab-ci/-/merge_requests/15
I've already apply the changes to one runner, for testing.

The new debian:stable container is already published, but commiting the change should rebuild it.

Thanks.

Anthony PERARD (3):
automation: adding debian:stable container
automation: automatically build debian:stable container
automation: check for generated files

 .gitlab-ci.yml                            |  2 +
 automation/build/debian/stable.dockerfile | 53 ++++++++++++++++++++++
 automation/gitlab-ci/containers.yaml      | 22 +++++++++
 automation/gitlab-ci/test.yaml            | 10 +++++
 automation/scripts/check-generated-files  | 55 +++++++++++++++++++++++
 5 files changed, 142 insertions(+)
 create mode 100644 automation/build/debian/stable.dockerfile
 create mode 100644 automation/gitlab-ci/containers.yaml
 create mode 100755 automation/scripts/check-generated-files
Signature · 1 line
Anthony PERARD
#1 lore raw

PATCH 1/3 automation: adding debian:stable container

This container will be used to check that generated source file in the git repository are up to date.

---
 automation/build/debian/stable.dockerfile | 53 +++++++++++++++++++++++
 1 file changed, 53 insertions(+)
 create mode 100644 automation/build/debian/stable.dockerfile
automation/build/debian/stable.dockerfile new file +53 -0
new file mode 100644
index 0000000000..912c4a5345
@@ -0,0 +1,53 @@
+FROM debian:stable
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="[email protected]"
+
+RUN useradd -ms /bin/bash user
+ENV DEBIAN_FRONTEND=noninteractive
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes install \
+        build-essential \
+        autoconf \
+        zlib1g-dev \
+        libncurses5-dev \
+        libssl-dev \
+        python3-dev \
+        xorg-dev \
+        uuid-dev \
+        libyajl-dev \
+        libaio-dev \
+        libglib2.0-dev \
+        clang \
+        libpixman-1-dev \
+        pkg-config \
+        flex \
+        bison \
+        # gettext for Xen < 4.13
+        gettext \
+        acpica-tools \
+        bin86 \
+        bcc \
+        liblzma-dev \
+        # libc6-dev-i386 for Xen < 4.15
+        libc6-dev-i386 \
+        libnl-3-dev \
+        ocaml-nox \
+        libfindlib-ocaml-dev \
+        markdown \
+        transfig \
+        pandoc \
+        checkpolicy \
+        wget \
+        git \
+        nasm \
+        gnupg \
+        golang \
+        && \
+        apt-get autoremove -y && \
+        apt-get clean && \
+        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
+
+USER user
+WORKDIR /build
Signature · 1 line
Anthony PERARD
#2 lore raw

PATCH 2/3 automation: automatically build debian:stable container

In reply to Anthony PERARD

Only run this on the staging branch, whenever the dockerfile changes.

---
 .gitlab-ci.yml                       |  2 ++
 automation/gitlab-ci/containers.yaml | 22 ++++++++++++++++++++++
 2 files changed, 24 insertions(+)
 create mode 100644 automation/gitlab-ci/containers.yaml
.gitlab-ci.yml +2 -0
index c8bd7519d5..f934d3eff8 100644
@@ -1,7 +1,9 @@
 stages:
+  - containers
   - build
   - test
 
 include:
+  - 'automation/gitlab-ci/containers.yaml'
   - 'automation/gitlab-ci/build.yaml'
   - 'automation/gitlab-ci/test.yaml'
automation/gitlab-ci/containers.yaml new file +22 -0
new file mode 100644
index 0000000000..2daf257f11
@@ -0,0 +1,22 @@
+debian-stable-container:
+  stage: containers
+  image: docker:stable
+  variables:
+    BUILD_CONTAINER: debian/stable
+  tags:
+    - container-builder
+  only:
+    refs:
+      - staging
+    changes:
+      - automation/build/${BUILD_CONTAINER}.dockerfile
+  services:
+    - docker:dind
+  before_script:
+    - apk add make
+    - docker info
+    - docker login -u $CI_DEPLOY_USER -p $CI_DEPLOY_PASSWORD $CI_REGISTRY
+  script:
+    - make -C automation/build ${BUILD_CONTAINER} PUSH=1
+  after_script:
+    - docker logout
Signature · 1 line
Anthony PERARD
#3 lore raw

PATCH 3/3 automation: check for generated files

In reply to Anthony PERARD

Try to find out whether genereted files that are commited needs to be regenerated.

---
 automation/gitlab-ci/test.yaml           | 10 +++++
 automation/scripts/check-generated-files | 55 ++++++++++++++++++++++++
 2 files changed, 65 insertions(+)
 create mode 100755 automation/scripts/check-generated-files
automation/gitlab-ci/test.yaml +10 -0
index 56747fb335..a4b08c26ca 100644
@@ -25,6 +25,16 @@ build-each-commit-gcc:
   tags:
     - x86_64
 
+build-check-generated-files-tools:
+  extends: .test-jobs-common
+  variables:
+    CONTAINER: debian:stable
+  script:
+    - automation/scripts/check-generated-files
+  needs: []
+  tags:
+    - x86_64
+
 qemu-alpine-arm64-gcc:
   extends: .test-jobs-common
   variables:
automation/scripts/check-generated-files new file +55 -0

55 lines not on this page · raw message

Signature · 1 line
Anthony PERARD
#4 lore raw
In reply to Anthony PERARD
Quoted 8 lines · Anthony PERARD

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/xen-unstable.git br.gitlab-ci-build-container-v1

I wanted to automatically check that generated files that we have in our repo are up-to-date, those are autoconf and *.gen.go files generataed from libxl_types.idl. But this can only be check on debian stable. So new container, and I look at building the new container from the CI.

Quoted 6 lines

The second patch depends on some changes to our GitLab runner:
https://gitlab.com/xen-project/xen-gitlab-ci/-/merge_requests/15
I've already apply the changes to one runner, for testing.

The new debian:stable container is already published, but commiting the change should rebuild it.

Forgot to say that this series also depends on my previous one:
[XEN PATCH 0/4] Some improvement for gitlab CI jobs <[email protected]>
(for the last patch)

Signature · 1 line
Anthony PERARD