CI: part 3 (slimline, and PPC/RISCV fixes)
v2 cover CI: part 3 (slimline, and PPC/RISCV fixes)
Patch 1 fixes a bug in the containerize script
Patches 2-5 remove useless/obsolete testing
Patch 6 fixes a bug with the archlinux testing configuration
Patches 7-9 adjust the PPC64 testing
Patches 10-11 adjust the RISCV64 testing
Patch 12 refreshes the custom GCC-IBT container
All changes here follow best-guidance for dockerfiles (non-root, heredocs for improved legibility), and naming consistency improvements discussed previously on the Committers call. Breifly that's:
$DISTRO-$VERSION(numeric)-$ARCH-*
which sort more nicely and don't require e.g. people to remember which order buster/bullseye/bookworm are in terms of debain release.
For x86, this involves inserting an $ARCH of x86_64. Right now, x86_64 is implied by the absence, and all other variations have to state one.
Still to come:
* OpenSUSE container fixes (resolves the final rolling distro vs allowed_failure problem)
* Ubuntu/debian rename/trim
* Add new jobs for latest releases (Ubuntu, Alpine, Fedora)
Andrew Cooper (12):
CI: Fix CONTAINER_UID0=1 scripts/containerize
CI: Remove useless/misleading randconfig jobs
CI: Drop Debian Jessie dockerfiles
CI: Drop Debian Stretch testing
CI: Drop Ubuntu Trusty testing
CI: Mark Archlinux/x86 as allowing failures
CI: Introduce a debian:12-ppc64le container
CI: Use debian:12-ppc64le for both build and test
CI: Refresh bullseye-ppc64le as debian:11-ppc64le
CI: Introduce debian:11/12-riscv64 containers
CI: Swap to debian for riscv64 build and test
CI: Refresh and upgrade the GCC-IBT container
.../archlinux/current-riscv64.dockerfile | 22 --
automation/build/debian/11-ppc64le.dockerfile | 33 +++
automation/build/debian/11-riscv64.dockerfile | 33 +++
automation/build/debian/12-ppc64le.dockerfile | 36 +++
automation/build/debian/12-riscv64.dockerfile | 36 +++
...ockerfile => 12-x86_64-gcc-ibt.dockerfile} | 81 +++---
.../build/debian/bullseye-ppc64le.dockerfile | 32 ---
.../build/debian/jessie-i386.dockerfile | 55 -----
automation/build/debian/jessie.dockerfile | 52 ----
.../build/debian/stretch-i386.dockerfile | 57 -----
automation/build/debian/stretch.dockerfile | 60 -----
automation/build/ubuntu/trusty.dockerfile | 51 ----
automation/gitlab-ci/build.yaml | 233 +++++++-----------
automation/gitlab-ci/test.yaml | 13 +-
automation/scripts/containerize | 15 +-
automation/scripts/qemu-smoke-ppc64le.sh | 4 +-
.../qemu-system-ppc64/8.1.0-ppc64.dockerfile | 38 ---
17 files changed, 282 insertions(+), 569 deletions(-)
delete mode 100644 automation/build/archlinux/current-riscv64.dockerfile
create mode 100644 automation/build/debian/11-ppc64le.dockerfile
create mode 100644 automation/build/debian/11-riscv64.dockerfile
create mode 100644 automation/build/debian/12-ppc64le.dockerfile
create mode 100644 automation/build/debian/12-riscv64.dockerfile
rename automation/build/debian/{buster-gcc-ibt.dockerfile => 12-x86_64-gcc-ibt.dockerfile} (50%)
delete mode 100644 automation/build/debian/bullseye-ppc64le.dockerfile
delete mode 100644 automation/build/debian/jessie-i386.dockerfile
delete mode 100644 automation/build/debian/jessie.dockerfile
delete mode 100644 automation/build/debian/stretch-i386.dockerfile
delete mode 100644 automation/build/debian/stretch.dockerfile
delete mode 100644 automation/build/ubuntu/trusty.dockerfile
delete mode 100644 automation/tests-artifacts/qemu-system-ppc64/8.1.0-ppc64.dockerfileSignature · 1 line
2.39.2
PATCH v2 1/12 CI: Fix CONTAINER_UID0=1 scripts/containerize
Right now, most build containers use root. Archlinux, Fedora and Yocto set up a regular user called user.
For those containers, trying to containerize as root fails, because CONTAINER_UID0=1 does nothing, whereas CONTAINER_UID0=0 forces the user away from root.
To make CONTAINER_UID0=1 work reliably, force to root if requested.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
v2:
* Add fixes tag
--- automation/scripts/containerize | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
automation/scripts/containerize +1 -1
@@ -51,7 +51,7 @@ esac # Use this variable to control whether root should be used case "_${CONTAINER_UID0}" in - _1) userarg= ;; + _1) userarg="-u 0" ;; _0|_) userarg="-u $(id -u) $userns_podman" ;; esac
Signature · 1 line
2.39.2
Re: [PATCH 01/12] CI: Fix CONTAINER_UID0=1 scripts/containerize
Quoted 11 lines · Andrew Cooper
Right now, most build containers use root. Archlinux, Fedora and Yocto set up a regular user called
user.For those containers, trying to containerize as root fails, because CONTAINER_UID0=1 does nothing, whereas CONTAINER_UID0=0 forces the user away from root.
To make CONTAINER_UID0=1 work reliably, force to root if requested.
Fixes 17fbe6504dfd ("automation: introduce a new variable to control container user")Signed-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
PATCH v2 2/12 CI: Remove useless/misleading randconfig jobs
In reply to Andrew CooperRandconfig builds pick CONFIG_DEBUG with 50% probability. Therefore $foo{,-debug}-randconfig are two identical jobs with misleading names.
Furthermore, arm64 has a randconfig jobs for both the alpine and bookworm build environments. Both use GCC 12.2, so we have 4 identical jobs. Delete 3 of them.
This leaves us with 4 primary randconfig jobs:
gitlab-ci/build.yaml:381:debian-bookworm-gcc-arm32-randconfig:
gitlab-ci/build.yaml:429:alpine-3.18-gcc-arm64-randconfig:
gitlab-ci/build.yaml:495:archlinux-current-gcc-riscv64-randconfig:
gitlab-ci/build.yaml:640:debian-bookworm-gcc-randconfig:
as PPC64 doesn't want randconfig right now, and buster-gcc-ibt is a special job with a custom compiler.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
Here is an example debug-randconfig with a release builds:
https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7312848876
If we want to run multiple identical randconfig jobs, that's spelt
parallel: 5
in the configuration, and here is an example of what such a run looks like:
https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1368824041
Notice the randconfig jobs have a 5 in place of a retry button, and show a submenu when clicked on.
--- automation/gitlab-ci/build.yaml | 39 --------------------------------- 1 file changed, 39 deletions(-)
automation/gitlab-ci/build.yaml +0 -39
@@ -385,13 +385,6 @@ debian-bookworm-gcc-arm32-randconfig: HYPERVISOR_ONLY: y RANDCONFIG: y -debian-bookworm-gcc-arm32-debug-randconfig: - extends: .gcc-arm32-cross-build-debug - variables: - CONTAINER: debian:bookworm-arm64v8-arm32-gcc - HYPERVISOR_ONLY: y - RANDCONFIG: y - debian-bookworm-gcc-arm32-debug-staticmem: extends: .gcc-arm32-cross-build-debug variables:
@@ -423,18 +416,6 @@ debian-bookworm-gcc-debug-arm64: variables: CONTAINER: debian:bookworm-arm64v8 -debian-bookworm-gcc-arm64-randconfig: - extends: .gcc-arm64-build - variables: - CONTAINER: debian:bookworm-arm64v8 - RANDCONFIG: y - -debian-bookworm-gcc-debug-arm64-randconfig: - extends: .gcc-arm64-build-debug - variables: - CONTAINER: debian:bookworm-arm64v8 - RANDCONFIG: y - alpine-3.18-gcc-arm64: extends: .gcc-arm64-build variables:
@@ -451,12 +432,6 @@ alpine-3.18-gcc-arm64-randconfig: CONTAINER: alpine:3.18-arm64v8 RANDCONFIG: y -alpine-3.18-gcc-debug-arm64-randconfig: - extends: .gcc-arm64-build-debug - variables: - CONTAINER: alpine:3.18-arm64v8 - RANDCONFIG: y - alpine-3.18-gcc-debug-arm64-staticmem: extends: .gcc-arm64-build-debug variables:
@@ -525,14 +500,6 @@ archlinux-current-gcc-riscv64-randconfig: RANDCONFIG: y <<: *riscv-fixed-randconfig -archlinux-current-gcc-riscv64-debug-randconfig: - extends: .gcc-riscv64-cross-build-debug - variables: - CONTAINER: archlinux:current-riscv64 - KBUILD_DEFCONFIG: tiny64_defconfig - RANDCONFIG: y - <<: *riscv-fixed-randconfig - # Power cross-build debian-bullseye-gcc-ppc64le: extends: .gcc-ppc64le-cross-build
@@ -676,12 +643,6 @@ debian-bookworm-gcc-randconfig: CONTAINER: debian:bookworm RANDCONFIG: y -debian-bookworm-gcc-debug-randconfig: - extends: .gcc-x86-64-build-debug - variables: - CONTAINER: debian:bookworm - RANDCONFIG: y - debian-bookworm-32-clang-debug: extends: .clang-x86-32-build-debug variables:
Signature · 1 line
2.39.2
Re: [PATCH 02/12] CI: Remove useless/misleading randconfig jobs
Quoted 14 lines · Andrew Cooper
Randconfig builds pick
CONFIG_DEBUGwith 50% probability. Therefore $foo{,-debug}-randconfig are two identical jobs with misleading names.Furthermore, arm64 has a randconfig jobs for both the alpine and bookworm build environments. Both use GCC 12.2, so we have 4 identical jobs. Delete 3 of them.
This leaves us with 4 primary randconfig jobs:
gitlab-ci/build.yaml:381:debian-bookworm-gcc-arm32-randconfig:
gitlab-ci/build.yaml:429:alpine-3.18-gcc-arm64-randconfig:
gitlab-ci/build.yaml:495:archlinux-current-gcc-riscv64-randconfig:
gitlab-ci/build.yaml:640:debian-bookworm-gcc-randconfig:
Quoted 4 lines
as PPC64 doesn't want randconfig right now, and buster-gcc-ibt is a special job with a custom compiler.
Signed-off-by Andrew Cooper <[email protected]>
Thanks,
Quoted 9 lines
If we want to run multiple identical randconfig jobs, that's spelt
parallel: 5
in the configuration, and here is an example of what such a run looks like:
https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1368824041Notice the randconfig jobs have a 5 in place of a retry button, and show a submenu when clicked on.
We already have "parallel: 2" on the whole pipeline, give or take, but those are quick jobs so it might be worth it to run them a few more time on a pipeline.
Cheers,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 02/12] CI: Remove useless/misleading randconfig jobs
Quoted 15 lines · Anthony PERARD
Quoted 14 lines · Andrew Cooper
Randconfig builds pick
CONFIG_DEBUGwith 50% probability. Therefore $foo{,-debug}-randconfig are two identical jobs with misleading names.Furthermore, arm64 has a randconfig jobs for both the alpine and bookworm build environments. Both use GCC 12.2, so we have 4 identical jobs. Delete 3 of them.
This leaves us with 4 primary randconfig jobs:
gitlab-ci/build.yaml:381:debian-bookworm-gcc-arm32-randconfig:
gitlab-ci/build.yaml:429:alpine-3.18-gcc-arm64-randconfig:
gitlab-ci/build.yaml:495:archlinux-current-gcc-riscv64-randconfig:
gitlab-ci/build.yaml:640:debian-bookworm-gcc-randconfig:
Quoted 5 lines
Quoted 4 lines
as PPC64 doesn't want randconfig right now, and buster-gcc-ibt is a special job with a custom compiler.
Signed-off-by Andrew Cooper <[email protected]>Reviewed-by Anthony PERARD <[email protected]>
Thanks.
Quoted 7 lines
Thanks,
Quoted 4 lines
If we want to run multiple identical randconfig jobs, that's spelt
parallel: 5
Quoted 8 lines
Quoted 5 lines
in the configuration, and here is an example of what such a run looks like:
https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1368824041Notice the randconfig jobs have a 5 in place of a retry button, and show a submenu when clicked on.
We already have "parallel: 2" on the whole pipeline, give or take, but those are quick jobs so it might be worth it to run them a few more time on a pipeline.
I'm afraid I don't follow.
The runners themselves are set up to run concurrent jobs. I could well believe that's spelt parallel: in the runner config files.
Here, its really jobs:parallel which is the way of saying "I want 5 copies of this job to run in the pipeline".
Obviously, 5 is arbitrary. I was thinking of having a scheduled (once per day, at 3AM or whenever) running randconfig only with a parallel:20 or so, so we can make good use of otherwise-idle time.
~Andrew
Re: [PATCH 02/12] CI: Remove useless/misleading randconfig jobs
Quoted 6 lines · Andrew Cooper
Quoted 3 lines · Anthony PERARD
We already have "parallel: 2" on the whole pipeline, give or take, but those are quick jobs so it might be worth it to run them a few more time on a pipeline.
I'm afraid I don't follow.
Sorry, I just meant that the whole test suite is run twice, once in
https://gitlab.com/xen-project/hardware/xen/-/pipelines
and once in
https://gitlab.com/xen-project/xen/-/pipelines
(with just a few tests that are differents)
Quoted 3 lines
Obviously, 5 is arbitrary. I was thinking of having a scheduled (once per day, at 3AM or whenever) running randconfig only with a parallel:20 or so, so we can make good use of otherwise-idle time.
That sounds like a good idea.
Cheers,
--
Anthony Perard | Vates XCP-ng Developer
XCP-ng & Xen Orchestra - Vates solutions
web: https://vates.tech
Re: [PATCH 02/12] CI: Remove useless/misleading randconfig jobs
Quoted 11 lines · Anthony PERARD
Quoted 5 lines · Andrew Cooper
Quoted 3 lines · Anthony PERARD
We already have "parallel: 2" on the whole pipeline, give or take, but those are quick jobs so it might be worth it to run them a few more time on a pipeline.
I'm afraid I don't follow.
Sorry, I just meant that the whole test suite is run twice, once in
https://gitlab.com/xen-project/hardware/xen/-/pipelines
and once in
https://gitlab.com/xen-project/xen/-/pipelines
(with just a few tests that are differents)
And the rest. Every time I re-sync my tree, and probably the same for the other maintainers.
However, these have nothing on the overhead from patchew, which would be far more palatable if we didn't essentially throw away the result by not sending it anywhere useful.
~Andrew
PATCH v2 3/12 CI: Drop Debian Jessie dockerfiles
In reply to Andrew CooperThese were removed from testing in Xen 4.18.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
--- .../build/debian/jessie-i386.dockerfile | 55 ------------------- automation/build/debian/jessie.dockerfile | 52 ------------------ automation/scripts/containerize | 2 - 3 files changed, 109 deletions(-) delete mode 100644 automation/build/debian/jessie-i386.dockerfile delete mode 100644 automation/build/debian/jessie.dockerfile
automation/build/debian/jessie-i386.dockerfile deleted +0 -55
@@ -1,55 +0,0 @@ -# syntax=docker/dockerfile:1 -FROM --platform=linux/i386 debian/eol:jessie -LABEL maintainer.name="The Xen Project" \ - maintainer.email="[email protected]" - -ENV DEBIAN_FRONTEND=noninteractive -ENV USER root - -RUN mkdir /build -WORKDIR /build - -ENTRYPOINT ["linux32"] - -# replace repos in archive as release is EOL -COPY <<"END" /etc/apt/sources.list -deb http://archive.debian.org/debian/ jessie main contrib non-free -deb http://archive.debian.org/debian/ jessie-backports main contrib non-free -deb http://archive.debian.org/debian-security/ jessie/updates main contrib non-free -END - -# build depends -RUN apt-get update && \ - apt-get --quiet --yes install \ - build-essential \ - zlib1g-dev \ - libncurses5-dev \ - python-dev \ - uuid-dev \ - libyajl-dev \ - libaio-dev \ - libglib2.0-dev \ - clang \ - libpixman-1-dev \ - pkg-config \ - flex \ - bison \ - acpica-tools \ - bin86 \ - bcc \ - liblzma-dev \ - libc6-dev \ - libnl-3-dev \ - ocaml-nox \ - libfindlib-ocaml-dev \ - markdown \ - transfig \ - pandoc \ - checkpolicy \ - wget \ - git \ - nasm \ - && \ - apt-get autoremove -y && \ - apt-get clean && \ - rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
automation/build/debian/jessie.dockerfile deleted +0 -52
52 lines not on this page · raw message
automation/scripts/containerize +0 -2
@@ -31,8 +31,6 @@ case "_${CONTAINER}" in _centos7) CONTAINER="${BASE}/centos:7" ;; _fedora) CONTAINER="${BASE}/fedora:29";; _focal) CONTAINER="${BASE}/ubuntu:focal" ;; - _jessie) CONTAINER="${BASE}/debian:jessie" ;; - _jessie-i386) CONTAINER="${BASE}/debian:jessie-i386" ;; _bullseye-ppc64le) CONTAINER="${BASE}/debian:bullseye-ppc64le" ;; _stretch|_) CONTAINER="${BASE}/debian:stretch" ;; _stretch-i386) CONTAINER="${BASE}/debian:stretch-i386" ;;
Signature · 1 line
2.39.2
Re: [PATCH 03/12] CI: Drop Debian Jessie dockerfiles
Quoted 4 lines · Andrew Cooper
These were removed from testing in Xen 4.18.
Fixes 3817e3c1b4b8 ("automation: Remove testing on Debian Jessie")Signed-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
PATCH v2 4/12 CI: Drop Debian Stretch testing
In reply to Andrew CooperDebian stretch is also End of Life. Update a couple of test steps to use bookworm instead.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
--- .../build/debian/stretch-i386.dockerfile | 57 ------------------ automation/build/debian/stretch.dockerfile | 60 ------------------- automation/gitlab-ci/build.yaml | 34 +---------- automation/gitlab-ci/test.yaml | 4 +- automation/scripts/containerize | 2 - 5 files changed, 4 insertions(+), 153 deletions(-) delete mode 100644 automation/build/debian/stretch-i386.dockerfile delete mode 100644 automation/build/debian/stretch.dockerfile
automation/build/debian/stretch-i386.dockerfile deleted +0 -57
@@ -1,57 +0,0 @@ -# syntax=docker/dockerfile:1 -FROM --platform=linux/i386 debian:stretch -LABEL maintainer.name="The Xen Project" \ - maintainer.email="[email protected]" - -ENV DEBIAN_FRONTEND=noninteractive -ENV USER root - -RUN mkdir /build -WORKDIR /build - -ENTRYPOINT ["linux32"] - -# replace repos in archive as release is EOL -COPY <<"END" /etc/apt/sources.list -deb http://archive.debian.org/debian/ stretch main contrib non-free -deb http://archive.debian.org/debian/ stretch-backports main contrib non-free -deb http://archive.debian.org/debian-security/ stretch/updates main contrib non-free -END - -# build depends -RUN apt-get update && \ - apt-get --quiet --yes install \ - build-essential \ - zlib1g-dev \ - libncurses5-dev \ - python-dev \ - python3-dev \ - uuid-dev \ - libyajl-dev \ - libaio-dev \ - libglib2.0-dev \ - clang \ - libpixman-1-dev \ - pkg-config \ - flex \ - bison \ - acpica-tools \ - bin86 \ - bcc \ - liblzma-dev \ - libc6-dev \ - libnl-3-dev \ - ocaml-nox \ - libfindlib-ocaml-dev \ - markdown \ - transfig \ - pandoc \ - checkpolicy \ - wget \ - git \ - nasm \ - apt-transport-https \ - && \ - apt-get autoremove -y && \ - apt-get clean && \ - rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
automation/build/debian/stretch.dockerfile deleted +0 -60
60 lines not on this page · raw message
automation/gitlab-ci/build.yaml +2 -32
@@ -354,10 +354,10 @@ alpine-3.18-gcc-debug: variables: CONTAINER: alpine:3.18 -debian-stretch-gcc-debug: +debian-bookworm-gcc-debug: extends: .gcc-x86-64-build-debug variables: - CONTAINER: debian:stretch + CONTAINER: debian:bookworm debian-bookworm-clang-debug: extends: .clang-x86-64-build-debug
@@ -589,31 +589,6 @@ centos-7-gcc-debug: variables: CONTAINER: centos:7 -debian-stretch-gcc: - extends: .gcc-x86-64-build - variables: - CONTAINER: debian:stretch - -debian-stretch-clang: - extends: .clang-x86-64-build - variables: - CONTAINER: debian:stretch - -debian-stretch-clang-debug: - extends: .clang-x86-64-build-debug - variables: - CONTAINER: debian:stretch - -debian-stretch-32-clang-debug: - extends: .clang-x86-32-build-debug - variables: - CONTAINER: debian:stretch-i386 - -debian-stretch-32-gcc-debug: - extends: .gcc-x86-32-build-debug - variables: - CONTAINER: debian:stretch-i386 - debian-buster-gcc-ibt: extends: .gcc-x86-64-build variables:
@@ -632,11 +607,6 @@ debian-bookworm-gcc: variables: CONTAINER: debian:bookworm -debian-bookworm-gcc-debug: - extends: .gcc-x86-64-build-debug - variables: - CONTAINER: debian:bookworm - debian-bookworm-gcc-randconfig: extends: .gcc-x86-64-build variables:
automation/gitlab-ci/test.yaml +2 -2
@@ -412,7 +412,7 @@ qemu-smoke-x86-64-gcc: script: - ./automation/scripts/qemu-smoke-x86-64.sh pv 2>&1 | tee ${LOGFILE} needs: - - debian-stretch-gcc-debug + - debian-bookworm-gcc-debug qemu-smoke-x86-64-clang: extends: .qemu-x86-64
@@ -426,7 +426,7 @@ qemu-smoke-x86-64-gcc-pvh: script: - ./automation/scripts/qemu-smoke-x86-64.sh pvh 2>&1 | tee ${LOGFILE} needs: - - debian-stretch-gcc-debug + - debian-bookworm-gcc-debug qemu-smoke-x86-64-clang-pvh: extends: .qemu-x86-64
automation/scripts/containerize +0 -2
@@ -32,8 +32,6 @@ case "_${CONTAINER}" in _fedora) CONTAINER="${BASE}/fedora:29";; _focal) CONTAINER="${BASE}/ubuntu:focal" ;; _bullseye-ppc64le) CONTAINER="${BASE}/debian:bullseye-ppc64le" ;; - _stretch|_) CONTAINER="${BASE}/debian:stretch" ;; - _stretch-i386) CONTAINER="${BASE}/debian:stretch-i386" ;; _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;; _bookworm|_) CONTAINER="${BASE}/debian:bookworm" ;; _bookworm-i386) CONTAINER="${BASE}/debian:bookworm-i386" ;;
Signature · 1 line
2.39.2
Re: [PATCH 04/12] CI: Drop Debian Stretch testing
Quoted 4 lines · Andrew Cooper
Debian stretch is also End of Life. Update a couple of test steps to use bookworm instead.
Signed-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
PATCH v2 5/12 CI: Drop Ubuntu Trusty testing
In reply to Andrew CooperThis is also End of Life.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
--- automation/build/ubuntu/trusty.dockerfile | 51 ----------------------- automation/gitlab-ci/build.yaml | 12 ------ automation/scripts/containerize | 1 - 3 files changed, 64 deletions(-) delete mode 100644 automation/build/ubuntu/trusty.dockerfile
automation/build/ubuntu/trusty.dockerfile deleted +0 -51
@@ -1,51 +0,0 @@ -# syntax=docker/dockerfile:1 -FROM --platform=linux/amd64 ubuntu:14.04 -LABEL maintainer.name="The Xen Project" \ - maintainer.email="[email protected]" - -ENV DEBIAN_FRONTEND=noninteractive -ENV USER root - -RUN mkdir /build -WORKDIR /build - -# build depends -RUN apt-get update && \ - apt-get --quiet --yes install \ - build-essential \ - zlib1g-dev \ - libncurses5-dev \ - python-dev \ - python3-dev \ - uuid-dev \ - libyajl-dev \ - libaio-dev \ - libglib2.0-dev \ - clang \ - libpixman-1-dev \ - pkg-config \ - flex \ - bison \ - acpica-tools \ - bin86 \ - bcc \ - liblzma-dev \ - libnl-3-dev \ - ocaml-nox \ - libfindlib-ocaml-dev \ - markdown \ - transfig \ - pandoc \ - checkpolicy \ - wget \ - git \ - nasm \ - && \ - apt-get autoremove -y && \ - apt-get clean && \ - rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/* - -# Remove expired certificate that Let's Encrypt certificates used to relie on. -# (Not needed anymore) -RUN sed -i 's#mozilla/DST_Root_CA_X3\.crt#!\0#' /etc/ca-certificates.conf && \ - update-ca-certificates
automation/gitlab-ci/build.yaml +0 -12
@@ -633,18 +633,6 @@ fedora-gcc-debug: variables: CONTAINER: fedora:29 -# Ubuntu Trusty's Clang is 3.4 while Xen requires 3.5 - -ubuntu-trusty-gcc: - extends: .gcc-x86-64-build - variables: - CONTAINER: ubuntu:trusty - -ubuntu-trusty-gcc-debug: - extends: .gcc-x86-64-build-debug - variables: - CONTAINER: ubuntu:trusty - ubuntu-xenial-clang: extends: .clang-x86-64-build variables:
automation/scripts/containerize +0 -1
@@ -39,7 +39,6 @@ case "_${CONTAINER}" in _bookworm-arm64v8) CONTAINER="${BASE}/debian:bookworm-arm64v8" ;; _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;; _bionic) CONTAINER="${BASE}/ubuntu:bionic" ;; - _trusty) CONTAINER="${BASE}/ubuntu:trusty" ;; _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;; _opensuse-leap|_leap) CONTAINER="${BASE}/suse:opensuse-leap" ;; _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/suse:opensuse-tumbleweed" ;;
Signature · 1 line
2.39.2
Re: [PATCH 05/12] CI: Drop Ubuntu Trusty testing
Quoted 3 lines · Andrew Cooper
This is also End of Life.
Signed-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
PATCH v2 6/12 CI: Mark Archlinux/x86 as allowing failures
In reply to Andrew CooperArchlinux is a rolling distro, and is subject to arbitrary changes as the container is rebuilt.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
--- automation/gitlab-ci/build.yaml | 2 ++ 1 file changed, 2 insertions(+)
automation/gitlab-ci/build.yaml +2 -0
@@ -573,11 +573,13 @@ archlinux-gcc: extends: .gcc-x86-64-build variables: CONTAINER: archlinux:current + allow_failure: true archlinux-gcc-debug: extends: .gcc-x86-64-build-debug variables: CONTAINER: archlinux:current + allow_failure: true centos-7-gcc: extends: .gcc-x86-64-build
Signature · 1 line
2.39.2
Re: [PATCH 06/12] CI: Mark Archlinux/x86 as allowing failures
Quoted 2 lines · Andrew Cooper
Archlinux is a rolling distro, and is subject to arbitrary changes as the container is rebuilt.
I disagree about the changes been arbitrary, it is just new version of software. :-)
But I guess "arbirary changes" could be a description for package been removed from the main/official archive, and left for the users to build themselfs.
Quoted 1 line
Signed-off-by Andrew Cooper <[email protected]>
Anyway:
(Next, automatically rebuild the container every month, I could probably revive my old patch series doing that)
Cheers,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 06/12] CI: Mark Archlinux/x86 as allowing failures
Quoted 9 lines · Anthony PERARD
Quoted 2 lines · Andrew Cooper
Archlinux is a rolling distro, and is subject to arbitrary changes as the container is rebuilt.
I disagree about the changes been arbitrary, it is just new version of software. :-)
But I guess "arbirary changes" could be a description for package been removed from the main/official archive, and left for the users to build themselfs.
Specifically, rebuilding the container for staging has a side effect of jumping the toolchain version under all stable trees.
I'll see about tweaking the wording.
Quoted 4 lines
Quoted 2 lines
Signed-off-by Andrew Cooper <[email protected]>Anyway:
Reviewed-by Anthony PERARD <[email protected]>
Thanks.
Quoted 2 lines
(Next, automatically rebuild the container every month, I could probably revive my old patch series doing that)
That's also on the list, but not for 4.19. Qemu rebuild their containers as the first pipeline stage, and while I think that's overkill, I was going to see if I could turn it into a weekly(?) scheduled job.
~Andrew
PATCH v2 7/12 CI: Introduce a debian:12-ppc64le container
In reply to Andrew CooperMake it non-root by default, and set XEN_TARGET_ARCH=ppc64 too.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
--- automation/build/debian/12-ppc64le.dockerfile | 36 +++++++++++++++++++ automation/scripts/containerize | 1 + 2 files changed, 37 insertions(+) create mode 100644 automation/build/debian/12-ppc64le.dockerfile
automation/build/debian/12-ppc64le.dockerfile new file +36 -0
@@ -0,0 +1,36 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 debian:bookworm-slim +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV DEBIAN_FRONTEND=noninteractive +ENV CROSS_COMPILE=powerpc64le-linux-gnu- +ENV XEN_TARGET_ARCH=ppc64 + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy + flex + gcc-powerpc64le-linux-gnu + python3-minimal + + # Qemu for test phase + qemu-system-ppc + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF + +USER user +WORKDIR /build
automation/scripts/containerize +1 -0
@@ -32,6 +32,7 @@ case "_${CONTAINER}" in _fedora) CONTAINER="${BASE}/fedora:29";; _focal) CONTAINER="${BASE}/ubuntu:focal" ;; _bullseye-ppc64le) CONTAINER="${BASE}/debian:bullseye-ppc64le" ;; + _bookworm-ppc64le) CONTAINER="${BASE}/debian:12-ppc64le" ;; _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;; _bookworm|_) CONTAINER="${BASE}/debian:bookworm" ;; _bookworm-i386) CONTAINER="${BASE}/debian:bookworm-i386" ;;
Signature · 1 line
2.39.2
Re: [PATCH 07/12] CI: Introduce a debian:12-ppc64le container
Quoted 6 lines · Andrew Cooper
automation/build/debian/12-ppc64le.dockerfile new file +0
-0@@ -0,0 +1,36 @@
...
automation/build/debian/12-ppc64le.dockerfile line 8
Quoted 2 lines
automation/build/debian/12-ppc64le.dockerfile new file +2
-0+ENV CROSS_COMPILE=powerpc64le-linux-gnu- +ENV XEN_TARGET_ARCH=ppc64
I don't really like these two envvars. The second one is already set in the env by gitlab, the first one could in theory also come from gitlab-ci.
But I guess that comment comes from the fact that I don't really know why we would want a second bookworm containers with only a few packages.
Quoted 14 lines
automation/build/debian/12-ppc64le.dockerfile new file +14
-0+ +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy
automation/build/debian/12-ppc64le.dockerfile line 32
Quoted 10 lines
automation/build/debian/12-ppc64le.dockerfile new file +10
-0+ flex + gcc-powerpc64le-linux-gnu + python3-minimal + + # Qemu for test phase + qemu-system-ppc + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/*
Other containers do also apt-get clean is this not needed? Or maybe the original containers is setup to not save packages?
/me looking into /var/cache
Looks empty enough, so I guess apt-get clean is optional.
Changes looks good enough, so:
Cheers,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 07/12] CI: Introduce a debian:12-ppc64le container
Quoted 8 lines · Anthony PERARD
Quoted 6 lines · Andrew Cooper
automation/build/debian/12-ppc64le.dockerfile new file +0
-0@@ -0,0 +1,36 @@...
Quoted 5 lines
automation/build/debian/12-ppc64le.dockerfile line 8
Quoted 2 lines
automation/build/debian/12-ppc64le.dockerfile new file +2
-0+ENV CROSS_COMPILE=powerpc64le-linux-gnu- +ENV XEN_TARGET_ARCH=ppc64I don't really like these two envvars. The second one is already set in the env by gitlab, the first one could in theory also come from gitlab-ci.
The first one is pre-existing. (It shows as new because enough of the file changed for it not to be deemed a rename).
The second is something I added, for improved usability.
Quoted 2 lines
But I guess that comment comes from the fact that I don't really know why we would want a second bookworm containers with only a few packages.
Right now (for better or worse) - we have single purpose containers for a single arch.
For PPC and RISCV, where we're only doing a cross-build of the hypervisor, the dependencies are almost identical. But the moment we start doing userspace, it will diverge quickly.
Part of the work I'm doing is trimming the containers down. Container size is a relevant factor for how long the CI tests take. Also, with smaller containers, the "oops, out of disk space, delete everything" logic on various runners will trigger less frequently, so we get better cache utilisation.
We do not IMO want to have containers with a union of all deps of all architectures. They'd be giant.
So while we do have single-purpose containers, these ENV vars are ~mandatory for using the container for it's intended purpose, and it's far more usable for a human when you can just enter the container and type make.
Quoted 27 lines
Quoted 14 lines
automation/build/debian/12-ppc64le.dockerfile new file +14
-0+ +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicyautomation/build/debian/12-ppc64le.dockerfile line 32
Quoted 10 lines
automation/build/debian/12-ppc64le.dockerfile new file +10
-0+ flex + gcc-powerpc64le-linux-gnu + python3-minimal + + # Qemu for test phase + qemu-system-ppc + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/*Other containers do also
apt-get cleanis this not needed? Or maybe the original containers is setup to not save packages?
Quoted 3 lines
/me looking into
/var/cacheLooks empty enough, so I guess
apt-get cleanis optional.
https://docs.docker.com/build/building/best-practices/ notes that the official Debian and Ubuntu images automatically clean themselves, so you don't need to do it manually.
Quoted 2 lines
Changes looks good enough, so:
Reviewed-by Anthony PERARD <[email protected]>
Thanks.
~Andrew
PATCH v2 8/12 CI: Use debian:12-ppc64le for both build and test
In reply to Andrew Cooperqemu-system-ppc64/8.1.0-ppc64 was added because bullseye's Qemu didn't understand the powernv9 machine.
However, bookworm's Qemu does, and this is preforable to maintaining a random build of Qemu ourselves.
Add bookworm builds, and retain bullseye to keep our lower-bound toolchain testing.
Rename the jobs to follow to sort coherently ($DISTRO-$VERSION-$ARCH-*) and reposition the jobs to optimise starting the smoke test.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
--- automation/gitlab-ci/build.yaml | 59 ++++++++++--------- automation/gitlab-ci/test.yaml | 5 +- automation/scripts/qemu-smoke-ppc64le.sh | 4 +- .../qemu-system-ppc64/8.1.0-ppc64.dockerfile | 38 ------------ 4 files changed, 34 insertions(+), 72 deletions(-) delete mode 100644 automation/tests-artifacts/qemu-system-ppc64/8.1.0-ppc64.dockerfile
automation/gitlab-ci/build.yaml +30 -29
@@ -302,20 +302,6 @@ qemu-system-aarch64-6.0.0-arm32-export: tags: - arm64 -# ppc64 test artifacts - -qemu-system-ppc64-8.1.0-ppc64-export: - extends: .test-jobs-artifact-common - image: registry.gitlab.com/xen-project/xen/tests-artifacts/qemu-system-ppc64:8.1.0-ppc64 - script: - - mkdir binaries && cp /qemu-system-ppc64 /skiboot.lid binaries/ - artifacts: - paths: - - binaries/qemu-system-ppc64 - - binaries/skiboot.lid - tags: - - x86_64 - # x86_64 test artifacts alpine-3.18-rootfs-export:
@@ -364,6 +350,13 @@ debian-bookworm-clang-debug: variables: CONTAINER: debian:bookworm +debian-12-ppc64le-gcc-debug: + extends: .gcc-ppc64le-cross-build-debug + variables: + CONTAINER: debian:12-ppc64le + KBUILD_DEFCONFIG: ppc64_defconfig + HYPERVISOR_ONLY: y + # Arm32 cross-build debian-bookworm-gcc-arm32:
@@ -500,21 +493,6 @@ archlinux-current-gcc-riscv64-randconfig: RANDCONFIG: y <<: *riscv-fixed-randconfig -# Power cross-build -debian-bullseye-gcc-ppc64le: - extends: .gcc-ppc64le-cross-build - variables: - CONTAINER: debian:bullseye-ppc64le - KBUILD_DEFCONFIG: ppc64_defconfig - HYPERVISOR_ONLY: y - -debian-bullseye-gcc-ppc64le-debug: - extends: .gcc-ppc64le-cross-build-debug - variables: - CONTAINER: debian:bullseye-ppc64le - KBUILD_DEFCONFIG: ppc64_defconfig - HYPERVISOR_ONLY: y - # Yocto test jobs yocto-qemuarm64: extends: .yocto-test-arm64
@@ -738,3 +716,26 @@ opensuse-tumbleweed-gcc-debug: variables: CONTAINER: suse:opensuse-tumbleweed allow_failure: true + +# PowerPC builds (x86 cross) +debian-bullseye-gcc-ppc64le: + extends: .gcc-ppc64le-cross-build + variables: + CONTAINER: debian:bullseye-ppc64le + KBUILD_DEFCONFIG: ppc64_defconfig + HYPERVISOR_ONLY: y + +debian-bullseye-gcc-ppc64le-debug: + extends: .gcc-ppc64le-cross-build-debug + variables: + CONTAINER: debian:bullseye-ppc64le + KBUILD_DEFCONFIG: ppc64_defconfig + HYPERVISOR_ONLY: y + +debian-12-ppc64le-gcc: + extends: .gcc-ppc64le-cross-build + variables: + CONTAINER: debian:12-ppc64le + KBUILD_DEFCONFIG: ppc64_defconfig + HYPERVISOR_ONLY: y +
automation/gitlab-ci/test.yaml +2 -3
@@ -69,7 +69,7 @@ .qemu-ppc64le: extends: .test-jobs-common variables: - CONTAINER: debian:bullseye-ppc64le + CONTAINER: debian:12-ppc64le LOGFILE: qemu-smoke-ppc64le.log artifacts: paths:
@@ -447,5 +447,4 @@ qemu-smoke-ppc64le-powernv9-gcc: script: - ./automation/scripts/qemu-smoke-ppc64le.sh powernv9 2>&1 | tee ${LOGFILE} needs: - - qemu-system-ppc64-8.1.0-ppc64-export - - debian-bullseye-gcc-ppc64le-debug + - debian-12-ppc64le-gcc-debug
automation/scripts/qemu-smoke-ppc64le.sh +2 -2
@@ -12,8 +12,8 @@ rm -f ${serial_log} set +e timeout -k 1 20 \ -binaries/qemu-system-ppc64 \ - -bios binaries/skiboot.lid \ +qemu-system-ppc64 \ + -bios skiboot.lid \ -M $machine \ -m 2g \ -smp 1 \
automation/tests-artifacts/qemu-system-ppc64/8.1.0-ppc64.dockerfile deleted +0 -38
@@ -1,38 +0,0 @@ -# syntax=docker/dockerfile:1 -FROM --platform=linux/amd64 debian:bullseye-slim -LABEL maintainer.name="The Xen Project" \ - maintainer.email="[email protected]" - -ENV DEBIAN_FRONTEND=noninteractive -ENV QEMU_VERSION=8.1.0 -ENV USER root - -RUN mkdir /build -WORKDIR /build - -# build depends -RUN apt-get update && \ - apt-get --quiet --yes install \ - build-essential \ - curl \ - python3 \ - python3-pip \ - python3-elementpath \ - ninja-build \ - pkg-config \ - libglib2.0-dev \ - libpixman-1-dev \ - && \ - \ - curl -fsSLO https://download.qemu.org/qemu-"$QEMU_VERSION".tar.xz && \ - tar xvJf qemu-"$QEMU_VERSION".tar.xz && \ - cd qemu-"$QEMU_VERSION" && \ - ./configure --target-list=ppc64-softmmu && \ - make -j$(nproc) && \ - cp ./build/qemu-system-ppc64 / && \ - cp ./build/qemu-bundle/usr/local/share/qemu/skiboot.lid / && \ - cd /build && \ - rm -rf qemu-"$QEMU_VERSION"* && \ - apt-get autoremove -y && \ - apt-get clean && \ - rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
Signature · 1 line
2.39.2
Re: [PATCH 08/12] CI: Use debian:12-ppc64le for both build and test
Quoted 1 line · Andrew Cooper
qemu-system-ppc64/8.1.0-ppc64 was added because bullseye's Qemu didn't
QEMU is normally spelled "QEMU" unless we are in camel case context, or lower case context.
Quoted 3 lines
understand the powernv9 machine.
However, bookworm's Qemu does, and this is preforable to maintaining a random
"preforable", that's an intended mispell right? :-)
Quoted 6 lines
build of Qemu ourselves.
Add bookworm builds, and retain bullseye to keep our lower-bound toolchain testing.
Rename the jobs to follow to sort coherently ($DISTRO-$VERSION-$ARCH-*) and
There's no renamed jobs in this patch, but the new jobs does follow the new naming schema. Did you intend to rename both "debian-bullseye-gcc-ppc64le*" jobs?
Quoted 3 lines
reposition the jobs to optimise starting the smoke test.
Signed-off-by Andrew Cooper <[email protected]>
Patch is mostly ok, so:
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 08/12] CI: Use debian:12-ppc64le for both build and test
Quoted 9 lines · Anthony PERARD
Quoted 1 line · Andrew Cooper
qemu-system-ppc64/8.1.0-ppc64 was added because bullseye's Qemu didn't
QEMU is normally spelled "QEMU" unless we are in camel case context, or lower case context.
Quoted 3 lines
understand the powernv9 machine.
However, bookworm's Qemu does, and this is preforable to maintaining a random
"preforable", that's an intended mispell right? :-)
Yes. both fixed.
Quoted 9 lines
Quoted 7 lines
build of Qemu ourselves.
Add bookworm builds, and retain bullseye to keep our lower-bound toolchain testing.
Rename the jobs to follow to sort coherently ($DISTRO-$VERSION-$ARCH-*) and
There's no renamed jobs in this patch, but the new jobs does follow the new naming schema.
Oh - good point. I'll tweak the wording.
Quoted 2 lines
Did you intend to rename both
"debian-bullseye-gcc-ppc64le*" jobs?
No - that's done in the following patch.
Quoted 6 lines
Quoted 4 lines
reposition the jobs to optimise starting the smoke test.
Signed-off-by Andrew Cooper <[email protected]>Patch is mostly ok, so:
Reviewed-by Anthony PERARD <[email protected]>
Thanks.
~Andrew
PATCH v2 9/12 CI: Refresh bullseye-ppc64le as debian:11-ppc64le
In reply to Andrew Cooper... in the style of bookworm-ppc64le
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
--- automation/build/debian/11-ppc64le.dockerfile | 33 +++++++++++++++++++ .../build/debian/bullseye-ppc64le.dockerfile | 32 ------------------ automation/gitlab-ci/build.yaml | 8 ++--- automation/scripts/containerize | 2 +- 4 files changed, 38 insertions(+), 37 deletions(-) create mode 100644 automation/build/debian/11-ppc64le.dockerfile delete mode 100644 automation/build/debian/bullseye-ppc64le.dockerfile
automation/build/debian/11-ppc64le.dockerfile new file +33 -0
@@ -0,0 +1,33 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 debian:bullseye-slim +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV DEBIAN_FRONTEND=noninteractive +ENV CROSS_COMPILE=powerpc64le-linux-gnu- +ENV XEN_TARGET_ARCH=ppc64 + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy + flex + gcc-powerpc64le-linux-gnu + python3-minimal + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF + +USER user +WORKDIR /build
automation/build/debian/bullseye-ppc64le.dockerfile deleted +0 -32
@@ -1,32 +0,0 @@ -# syntax=docker/dockerfile:1 -FROM --platform=linux/amd64 debian:bullseye-slim -LABEL maintainer.name="The Xen Project" \ - maintainer.email="[email protected]" - -ENV DEBIAN_FRONTEND=noninteractive -ENV USER root - -# Add compiler path -ENV CROSS_COMPILE powerpc64le-linux-gnu- - -RUN mkdir /build -WORKDIR /build - -# build depends -RUN apt-get update && \ - apt-get --quiet --yes --no-install-recommends install \ - bison \ - build-essential \ - checkpolicy \ - flex \ - gawk \ - gcc-powerpc64le-linux-gnu \ - make \ - python3-minimal \ - # QEMU runtime dependencies for test phase - libglib2.0-0 \ - libpixman-1-0 \ - && \ - apt-get autoremove -y && \ - apt-get clean && \ - rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
automation/gitlab-ci/build.yaml +4 -4
@@ -718,17 +718,17 @@ opensuse-tumbleweed-gcc-debug: allow_failure: true # PowerPC builds (x86 cross) -debian-bullseye-gcc-ppc64le: +debian-11-ppc64le-gcc: extends: .gcc-ppc64le-cross-build variables: - CONTAINER: debian:bullseye-ppc64le + CONTAINER: debian:11-ppc64le KBUILD_DEFCONFIG: ppc64_defconfig HYPERVISOR_ONLY: y -debian-bullseye-gcc-ppc64le-debug: +debian-11-ppc64le-gcc-debug: extends: .gcc-ppc64le-cross-build-debug variables: - CONTAINER: debian:bullseye-ppc64le + CONTAINER: debian:11-ppc64le KBUILD_DEFCONFIG: ppc64_defconfig HYPERVISOR_ONLY: y
automation/scripts/containerize +1 -1
@@ -31,7 +31,7 @@ case "_${CONTAINER}" in _centos7) CONTAINER="${BASE}/centos:7" ;; _fedora) CONTAINER="${BASE}/fedora:29";; _focal) CONTAINER="${BASE}/ubuntu:focal" ;; - _bullseye-ppc64le) CONTAINER="${BASE}/debian:bullseye-ppc64le" ;; + _bullseye-ppc64le) CONTAINER="${BASE}/debian:11-ppc64le" ;; _bookworm-ppc64le) CONTAINER="${BASE}/debian:12-ppc64le" ;; _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;; _bookworm|_) CONTAINER="${BASE}/debian:bookworm" ;;
Signature · 1 line
2.39.2
Re: [PATCH 09/12] CI: Refresh bullseye-ppc64le as debian:11-ppc64le
Quoted 3 lines · Andrew Cooper
... in the style of bookworm-ppc64le
Signed-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
PATCH v2 10/12 CI: Introduce debian:11/12-riscv64 containers
In reply to Andrew CooperFor starters, they're slightly smaller:
$ docker image list <snip> registry.gitlab.com/xen-project/xen/debian 12-riscv64 772MB registry.gitlab.com/xen-project/xen/debian 11-riscv64 422MB registry.gitlab.com/xen-project/xen/archlinux current-riscv64 2.32GB
They also not rolling distros, so will be more predicatable testing for the stable trees in the future.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
--- automation/build/debian/11-riscv64.dockerfile | 33 +++++++++++++++++ automation/build/debian/12-riscv64.dockerfile | 36 +++++++++++++++++++ 2 files changed, 69 insertions(+) create mode 100644 automation/build/debian/11-riscv64.dockerfile create mode 100644 automation/build/debian/12-riscv64.dockerfile
automation/build/debian/11-riscv64.dockerfile new file +33 -0
@@ -0,0 +1,33 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 debian:bullseye-slim +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV DEBIAN_FRONTEND=noninteractive +ENV CROSS_COMPILE=riscv64-linux-gnu- +ENV XEN_TARGET_ARCH=riscv64 + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy + flex + gcc-riscv64-linux-gnu + python3-minimal + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF + +USER user +WORKDIR /build
automation/build/debian/12-riscv64.dockerfile new file +36 -0
@@ -0,0 +1,36 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 debian:bookworm-slim +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV DEBIAN_FRONTEND=noninteractive +ENV CROSS_COMPILE=riscv64-linux-gnu- +ENV XEN_TARGET_ARCH=riscv64 + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy + flex + gcc-riscv64-linux-gnu + python3-minimal + + # Qemu for test phase + qemu-system-riscv64 + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF + +USER user +WORKDIR /build
Signature · 1 line
2.39.2
Re: [PATCH 10/12] CI: Introduce debian:11/12-riscv64 containers
Quoted 11 lines · Andrew Cooper
For starters, they're slightly smaller:
$ docker image list <snip> registry.gitlab.com/xen-project/xen/debian 12-riscv64 772MB registry.gitlab.com/xen-project/xen/debian 11-riscv64 422MB registry.gitlab.com/xen-project/xen/archlinux current-riscv64 2.32GBThey also not rolling distros, so will be more predicatable testing for the stable trees in the future.
Signed-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 10/12] CI: Introduce debian:11/12-riscv64 containers
In reply to Andrew CooperQuoted 7 lines · Andrew Cooper
For starters, they're slightly smaller:
$ docker image list <snip> registry.gitlab.com/xen-project/xen/debian 12-riscv64 772MB registry.gitlab.com/xen-project/xen/debian 11-riscv64 422MB
Do we really need both 11-riscv64 and 12-riscv64?
Generally this patch LGTM: Reviewed-by: Oleksii Kurochko <[email protected]>
~ Oleksii
Quoted 108 lines
registry.gitlab.com/xen-project/xen/archlinux current-riscv64 2.32GB
They also not rolling distros, so will be more predicatable testing for the
stable trees in the future.Signed-off-by Andrew Cooper <[email protected]>---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
---
automation/build/debian/11-riscv64.dockerfile | 33 +++++++++++++++++ automation/build/debian/12-riscv64.dockerfile | 36
+++++++++++++++++++2 files changed, 69 insertions(+)create mode 100644 automation/build/debian/11-riscv64.dockerfile create mode 100644 automation/build/debian/12-riscv64.dockerfile
automation/build/debian/11-riscv64.dockerfile new file +33
-0@@ -0,0 +1,33 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 debian:bullseye-slim +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV DEBIAN_FRONTEND=noninteractive +ENV CROSS_COMPILE=riscv64-linux-gnu- +ENV XEN_TARGET_ARCH=riscv64 + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy + flex + gcc-riscv64-linux-gnu + python3-minimal + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF + +USER user +WORKDIR /buildautomation/build/debian/12-riscv64.dockerfile new file +36
-0@@ -0,0 +1,36 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 debian:bookworm-slim +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV DEBIAN_FRONTEND=noninteractive +ENV CROSS_COMPILE=riscv64-linux-gnu- +ENV XEN_TARGET_ARCH=riscv64 + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy + flex + gcc-riscv64-linux-gnu + python3-minimal + + # Qemu for test phase + qemu-system-riscv64 + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF + +USER user +WORKDIR /build
Re: [PATCH 10/12] CI: Introduce debian:11/12-riscv64 containers
Quoted 9 lines · Oleksii
Quoted 7 lines · Andrew Cooper
For starters, they're slightly smaller:
$ docker image list <snip> registry.gitlab.com/xen-project/xen/debian 12-riscv64 772MB registry.gitlab.com/xen-project/xen/debian 11-riscv64 422MBDo we really need both 11-riscv64 and 12-riscv64?
Need? No, not strictly.
Want? Yes, absolutely.
You always want at least 2 different toolchains worth of testing, or what you will find happens is that you end up accidentally depending on a quirk of the single compiler your using, and that you discover this at some point in the future, rather than now(ish) when CI says no.
At the moment, the RISC-V builds are very trivial and quick, so it makes a lot of sense to have a second toolchain. Before too much longer, you'll want to get Clang working too.
Quoted 2 lines
Generally this patch LGTM: Reviewed-by: Oleksii Kurochko <[email protected]>
Thanks.
~Andrew
Re: [PATCH 10/12] CI: Introduce debian:11/12-riscv64 containers
Quoted 24 lines · Andrew Cooper
Quoted 11 lines · Oleksii
Quoted 9 lines · Andrew Cooper
For starters, they're slightly smaller:
$ docker image list <snip> registry.gitlab.com/xen-project/xen/debian 12- riscv64 772MB registry.gitlab.com/xen-project/xen/debian 11- riscv64 422MBDo we really need both 11-riscv64 and 12-riscv64?
Need? No, not strictly.
Want? Yes, absolutely.
You always want at least 2 different toolchains worth of testing, or what you will find happens is that you end up accidentally depending on
a quirk of the single compiler your using, and that you discover this at
some point in the future, rather than now(ish) when CI says no.
Quoted 4 lines
At the moment, the RISC-V builds are very trivial and quick, so it makes
a lot of sense to have a second toolchain. Before too much longer, you'll want to get Clang working too.
Agree, I should consider to add Clang. I will do that in the one of my nearest patch series.
~ Oleksii
PATCH v2 11/12 CI: Swap to debian for riscv64 build and test
In reply to Andrew CooperThe containers are both much smaller, with stable toolchains over time, and this at least means we're not doing all testing with a single compiler.
Rename the jobs to follow to sort coherently ($DISTRO-$VERSION-$ARCH-*) and reposition the jobs to optimise starting the smoke test.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
--- .../archlinux/current-riscv64.dockerfile | 22 ----- automation/gitlab-ci/build.yaml | 83 +++++++++++-------- automation/gitlab-ci/test.yaml | 4 +- automation/scripts/containerize | 3 +- 4 files changed, 52 insertions(+), 60 deletions(-) delete mode 100644 automation/build/archlinux/current-riscv64.dockerfile
automation/build/archlinux/current-riscv64.dockerfile deleted +0 -22
@@ -1,22 +0,0 @@ -# syntax=docker/dockerfile:1 -FROM --platform=linux/amd64 archlinux -LABEL maintainer.name="The Xen Project" \ - maintainer.email="[email protected]" - -# Packages needed for the build -RUN pacman --noconfirm --needed -Syu \ - base-devel \ - git \ - inetutils \ - riscv64-linux-gnu-binutils \ - riscv64-linux-gnu-gcc \ - riscv64-linux-gnu-glibc \ - # For test phase - qemu-system-riscv - -# Add compiler path -ENV CROSS_COMPILE=riscv64-linux-gnu- - -RUN useradd --create-home user -USER user -WORKDIR /build
automation/gitlab-ci/build.yaml +48 -35
98 lines not on this page · raw message
automation/gitlab-ci/test.yaml +2 -2
@@ -56,7 +56,7 @@ .qemu-riscv64: extends: .test-jobs-common variables: - CONTAINER: archlinux:current-riscv64 + CONTAINER: debian:12 LOGFILE: qemu-smoke-riscv64.log artifacts: paths:
@@ -440,7 +440,7 @@ qemu-smoke-riscv64-gcc: script: - ./automation/scripts/qemu-smoke-riscv64.sh 2>&1 | tee ${LOGFILE} needs: - - archlinux-current-gcc-riscv64-debug + - debian-12-riscv64-gcc-debug qemu-smoke-ppc64le-powernv9-gcc: extends: .qemu-ppc64le
automation/scripts/containerize +2 -1
@@ -27,12 +27,13 @@ case "_${CONTAINER}" in _alpine) CONTAINER="${BASE}/alpine:3.18" ;; _alpine-arm64v8) CONTAINER="${BASE}/alpine:3.18-arm64v8" ;; _archlinux|_arch) CONTAINER="${BASE}/archlinux:current" ;; - _riscv64) CONTAINER="${BASE}/archlinux:current-riscv64" ;; _centos7) CONTAINER="${BASE}/centos:7" ;; _fedora) CONTAINER="${BASE}/fedora:29";; _focal) CONTAINER="${BASE}/ubuntu:focal" ;; _bullseye-ppc64le) CONTAINER="${BASE}/debian:11-ppc64le" ;; _bookworm-ppc64le) CONTAINER="${BASE}/debian:12-ppc64le" ;; + _bullseye-riscv64) CONTAINER="${BASE}/debian:11-riscv64" ;; + _bookworm-riscv64) CONTAINER="${BASE}/debian:12-riscv64" ;; _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;; _bookworm|_) CONTAINER="${BASE}/debian:bookworm" ;; _bookworm-i386) CONTAINER="${BASE}/debian:bookworm-i386" ;;
Signature · 1 line
2.39.2
Re: [PATCH 11/12] CI: Swap to debian for riscv64 build and test
Quoted 7 lines · Andrew Cooper
The containers are both much smaller, with stable toolchains over time, and this at least means we're not doing all testing with a single compiler.
Rename the jobs to follow to sort coherently ($DISTRO-$VERSION-$ARCH-*) and reposition the jobs to optimise starting the smoke test.
Signed-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 11/12] CI: Swap to debian for riscv64 build and test
In reply to Andrew CooperQuoted 10 lines · Andrew Cooper
The containers are both much smaller, with stable toolchains over time, and
this at least means we're not doing all testing with a single compiler.Rename the jobs to follow to sort coherently ($DISTRO-$VERSION-$ARCH- *) and
reposition the jobs to optimise starting the smoke test.Signed-off-by Andrew Cooper <[email protected]>
LGTM: Oleksii Kurochko <[email protected]>
~ Oleksii
Quoted 198 lines
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
---
.../archlinux/current-riscv64.dockerfile | 22 ----- automation/gitlab-ci/build.yaml | 83 +++++++++++------ --
automation/gitlab-ci/test.yaml | 4 +- automation/scripts/containerize | 3 +-4 files changed, 52 insertions(+), 60 deletions(-)delete mode 100644 automation/build/archlinux/current- riscv64.dockerfile
automation/build/archlinux/current-riscv64.dockerfile deleted +0
-22@@ -1,22 +0,0 @@ -# syntax=docker/dockerfile:1 -FROM --platform=linux/amd64 archlinux -LABEL maintainer.name="The Xen Project" \ - maintainer.email="[email protected]" - -# Packages needed for the build -RUN pacman --noconfirm --needed -Syu \ - base-devel \ - git \ - inetutils \ - riscv64-linux-gnu-binutils \ - riscv64-linux-gnu-gcc \ - riscv64-linux-gnu-glibc \ - # For test phase - qemu-system-riscv - -# Add compiler path -ENV CROSS_COMPILE=riscv64-linux-gnu- - -RUN useradd --create-home user -USER user -WORKDIR /buildautomation/gitlab- renamed from automation/gitlab-ci/build.yaml +0
-0
ci/build.yaml
indexe081664c4e95..4b9d80cc5632100644automation/gitlab-ci/build.yaml +0
-0@@ -357,6 +357,13 @@ debian-12-ppc64le-gcc-debug:
KBUILD_DEFCONFIG:ppc64_defconfig
HYPERVISOR_ONLY: yautomation/gitlab-ci/build.yaml +7
-0+debian-12-riscv64-gcc-debug: + extends: .gcc-riscv64-cross-build-debug + variables: + CONTAINER: debian:12-riscv64 + KBUILD_DEFCONFIG: tiny64_defconfig + HYPERVISOR_ONLY: y +# Arm32 cross-build
debian-bookworm-gcc-arm32:automation/gitlab-ci/build.yaml +0
-0@@ -458,41 +465,6 @@ alpine-3.18-gcc-debug-arm64-earlyprintk:
CONFIG_EARLY_UART_CHOICE_PL011=y
CONFIG_EARLY_UART_BASE_ADDRESS=0x9000000automation/gitlab-ci/build.yaml +0
-35-# RISC-V 64 cross-build -.riscv-fixed-randconfig: - variables: &riscv-fixed-randconfig - EXTRA_FIXED_RANDCONFIG: | - CONFIG_BOOT_TIME_CPUPOOLS=n - CONFIG_COVERAGE=n - CONFIG_EXPERT=y - CONFIG_GRANT_TABLE=n - CONFIG_MEM_ACCESS=n - CONFIG_PERF_COUNTERS=n - CONFIG_LIVEPATCH=n - CONFIG_XSM=n - -archlinux-current-gcc-riscv64: - extends: .gcc-riscv64-cross-build - variables: - CONTAINER: archlinux:current-riscv64 - KBUILD_DEFCONFIG: tiny64_defconfig - HYPERVISOR_ONLY: y - -archlinux-current-gcc-riscv64-debug: - extends: .gcc-riscv64-cross-build-debug - variables: - CONTAINER: archlinux:current-riscv64 - KBUILD_DEFCONFIG: tiny64_defconfig - HYPERVISOR_ONLY: y - -archlinux-current-gcc-riscv64-randconfig: - extends: .gcc-riscv64-cross-build - variables: - CONTAINER: archlinux:current-riscv64 - KBUILD_DEFCONFIG: tiny64_defconfig - RANDCONFIG: y - <<: *riscv-fixed-randconfig -# Yocto test jobs
yocto-qemuarm64:
extends: .yocto-test-arm64automation/gitlab-ci/build.yaml +0
-0@@ -739,3 +711,44 @@ debian-12-ppc64le-gcc:
KBUILD_DEFCONFIG:ppc64_defconfig
HYPERVISOR_ONLY: yautomation/gitlab-ci/build.yaml +41
-0+# RISC-V 64 cross-build +debian-11-riscv64-gcc: + extends: .gcc-riscv64-cross-build + variables: + CONTAINER: debian:11-riscv64 + KBUILD_DEFCONFIG: tiny64_defconfig + HYPERVISOR_ONLY: y + +debian-11-riscv64-gcc-debug: + extends: .gcc-riscv64-cross-build-debug + variables: + CONTAINER: debian:11-riscv64 + KBUILD_DEFCONFIG: tiny64_defconfig + HYPERVISOR_ONLY: y + +debian-12-riscv64-gcc: + extends: .gcc-riscv64-cross-build + variables: + CONTAINER: debian:12-riscv64 + KBUILD_DEFCONFIG: tiny64_defconfig + HYPERVISOR_ONLY: y + +.riscv-fixed-randconfig: + variables: &riscv-fixed-randconfig + EXTRA_FIXED_RANDCONFIG: | + CONFIG_BOOT_TIME_CPUPOOLS=n + CONFIG_COVERAGE=n + CONFIG_EXPERT=y + CONFIG_GRANT_TABLE=n + CONFIG_MEM_ACCESS=n + CONFIG_PERF_COUNTERS=n + CONFIG_LIVEPATCH=n + CONFIG_XSM=n + +debian-12-riscv64-gcc-randconfig: + extends: .gcc-riscv64-cross-build + variables: + CONTAINER: debian:12-riscv64 + KBUILD_DEFCONFIG: tiny64_defconfig + RANDCONFIG: y + <<: *riscv-fixed-randconfigautomation/gitlab- renamed from automation/gitlab-ci/test.yaml +0
-0
ci/test.yaml
index70c946f87799..f7e1753ead34100644automation/gitlab-ci/test.yaml +0
-0@@ -56,7 +56,7 @@.qemu-riscv64:
extends: .test-jobs-common
variables:
- CONTAINER: archlinux:current-riscv64
+ CONTAINER: debian:12
LOGFILE: qemu-smoke-riscv64.log
artifacts:
paths:automation/gitlab-ci/test.yaml +0
-0@@ -440,7 +440,7 @@ qemu-smoke-riscv64-gcc:script:
-./automation/scripts/qemu-smoke-riscv64.sh2>&1 | tee ${LOGFILE}
needs:
- - archlinux-current-gcc-riscv64-debug
+ - debian-12-riscv64-gcc-debug
qemu-smoke-ppc64le-powernv9-gcc:
extends: .qemu-ppc64leautomation/scripts/containerize +2
-1@@ -27,12 +27,13 @@ case "_${CONTAINER}" in _alpine) CONTAINER="${BASE}/alpine:3.18" ;; _alpine-arm64v8) CONTAINER="${BASE}/alpine:3.18-arm64v8" ;; _archlinux|_arch) CONTAINER="${BASE}/archlinux:current" ;; - _riscv64) CONTAINER="${BASE}/archlinux:current-riscv64" ;; _centos7) CONTAINER="${BASE}/centos:7" ;; _fedora) CONTAINER="${BASE}/fedora:29";; _focal) CONTAINER="${BASE}/ubuntu:focal" ;; _bullseye-ppc64le) CONTAINER="${BASE}/debian:11-ppc64le" ;; _bookworm-ppc64le) CONTAINER="${BASE}/debian:12-ppc64le" ;; + _bullseye-riscv64) CONTAINER="${BASE}/debian:11-riscv64" ;; + _bookworm-riscv64) CONTAINER="${BASE}/debian:12-riscv64" ;; _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;; _bookworm|_) CONTAINER="${BASE}/debian:bookworm" ;; _bookworm-i386) CONTAINER="${BASE}/debian:bookworm-i386" ;;
PATCH v2 12/12 CI: Refresh and upgrade the GCC-IBT container
In reply to Andrew CooperUpgrade from Debian buster to bookworm, GCC 11.3 to 11.4 and to be a non-root container.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
---
...ockerfile => 12-x86_64-gcc-ibt.dockerfile} | 81 +++++++++++--------
automation/gitlab-ci/build.yaml | 4 +-
automation/scripts/containerize | 2 +-
3 files changed, 49 insertions(+), 38 deletions(-)
rename automation/build/debian/{buster-gcc-ibt.dockerfile => 12-x86_64-gcc-ibt.dockerfile} (50%)automation/build/debian/12-x86_64-gcc-ibt.dockerfile renamed from automation/build/debian/buster-gcc-ibt.dockerfile +46 -35
@@ -1,28 +1,33 @@ # syntax=docker/dockerfile:1 -FROM --platform=linux/amd64 debian:buster-slim AS builder +FROM --platform=linux/amd64 debian:bookworm-slim AS builder ENV DEBIAN_FRONTEND=noninteractive -ENV USER root -RUN apt-get update && \ - apt-get --quiet --yes --no-install-recommends install \ - bison \ - build-essential \ - ca-certificates \ - flex \ - g++-multilib \ - libc6-dev-i386 \ - libgmp-dev \ - libisl-dev \ - libmpc-dev \ - libmpfr-dev \ - patch \ +RUN <<EOF +#!/bin/bash + set -e + apt-get -y update + + DEPS=( + bison + build-essential + ca-certificates + flex + g++-multilib + libc6-dev-i386 + libgmp-dev + libisl-dev + libmpc-dev + libmpfr-dev + patch wget + ) -RUN mkdir /build -WORKDIR /build + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF -RUN wget -q https://ftp.gnu.org/gnu/gcc/gcc-11.3.0/gcc-11.3.0.tar.xz -O - | tar xJ --strip=1 +RUN wget -q https://ftp.gnu.org/gnu/gcc/gcc-11.4.0/gcc-11.4.0.tar.xz -O - | tar xJ --strip=1 RUN wget -q https://xenbits.xen.org/people/andrewcoop/gcc-11.2-Add-fcf-check-attribute-yes-no.patch -O - | patch -p1 RUN ./configure \ --prefix=/opt/gcc-11-ibt \
@@ -42,29 +47,35 @@ RUN ./configure \ RUN make -j`nproc` && make -j`nproc` install -FROM --platform=linux/amd64 debian:buster-slim +FROM --platform=linux/amd64 debian:bookworm-slim COPY --from=builder /opt/gcc-11-ibt /opt/gcc-11-ibt LABEL maintainer.name="The Xen Project" \ maintainer.email="[email protected]" ENV DEBIAN_FRONTEND=noninteractive -ENV USER root ENV PATH="/opt/gcc-11-ibt/bin:${PATH}" -RUN mkdir /build -WORKDIR /build +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user -RUN apt-get update && \ - apt-get --quiet --yes --no-install-recommends install \ - bison \ - build-essential \ - checkpolicy \ - flex \ - gawk \ - make \ - python3-minimal \ - && \ - apt-get autoremove -y && \ - apt-get clean && \ - rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/* + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy + flex + python3-minimal + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF + +USER user +WORKDIR /build
automation/gitlab-ci/build.yaml +2 -2
@@ -541,10 +541,10 @@ centos-7-gcc-debug: variables: CONTAINER: centos:7 -debian-buster-gcc-ibt: +debian-12-x86_64-gcc-ibt: extends: .gcc-x86-64-build variables: - CONTAINER: debian:buster-gcc-ibt + CONTAINER: debian:12-x86_64-gcc-ibt RANDCONFIG: y EXTRA_FIXED_RANDCONFIG: | CONFIG_XEN_IBT=y
automation/scripts/containerize +1 -1
@@ -34,7 +34,7 @@ case "_${CONTAINER}" in _bookworm-ppc64le) CONTAINER="${BASE}/debian:12-ppc64le" ;; _bullseye-riscv64) CONTAINER="${BASE}/debian:11-riscv64" ;; _bookworm-riscv64) CONTAINER="${BASE}/debian:12-riscv64" ;; - _buster-gcc-ibt) CONTAINER="${BASE}/debian:buster-gcc-ibt" ;; + _bookworm-x86_64-gcc-ibt) CONTAINER="${BASE}/debian:12-x86_64-gcc-ibt" ;; _bookworm|_) CONTAINER="${BASE}/debian:bookworm" ;; _bookworm-i386) CONTAINER="${BASE}/debian:bookworm-i386" ;; _bookworm-arm64v8-arm32-gcc) CONTAINER="${BASE}/debian:bookworm-arm64v8-arm32-gcc" ;;
Signature · 1 line
2.39.2
Re: [PATCH 12/12] CI: Refresh and upgrade the GCC-IBT container
Quoted 4 lines · Andrew Cooper
Upgrade from Debian buster to bookworm, GCC 11.3 to 11.4 and to be a non-root container.
Signed-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
~ Oleksii
Quoted 93 lines · Andrew Cooper
Patch 1 fixes a bug in the containerize script
Patches 2-5 remove useless/obsolete testing
Patch 6 fixes a bug with the archlinux testing configuration
Patches 7-9 adjust the PPC64 testing
Patches 10-11 adjust the RISCV64 testing
Patch 12 refreshes the custom GCC-IBT container
All changes here follow best-guidance for dockerfiles (non-root, heredocs for
improved legibility), and naming consistency improvements discussed previously
on the Committers call. Breifly that's:$DISTRO-$VERSION(numeric)-$ARCH-*
which sort more nicely and don't require e.g. people to remember which order
buster/bullseye/bookworm are in terms of debain release.For x86, this involves inserting an $ARCH of x86_64. Right now,
x86_64is
implied by the absence, and all other variations have to state one.Still to come:
* OpenSUSE container fixes (resolves the final rolling distro vs
allowed_failureproblem)
* Ubuntu/debian rename/trim
* Add new jobs for latest releases (Ubuntu, Alpine, Fedora)Andrew Cooper (12):
CI: Fix CONTAINER_UID0=1scripts/containerizeCI: Remove useless/misleading randconfig jobs
CI: Drop Debian Jessie dockerfiles
CI: Drop Debian Stretch testing
CI: Drop Ubuntu Trusty testingCI: Mark Archlinux/x86 as allowing failures
CI: Introduce a debian:12-ppc64le container
CI: Use debian:12-ppc64le for both build and test
CI: Refresh bullseye-ppc64le as debian:11-ppc64leCI: Introduce debian:11/12-riscv64 containers
CI: Swap to debian for riscv64 build and testCI: Refresh and upgrade the GCC-IBT container
.../archlinux/current-riscv64.dockerfile | 22 --
automation/build/debian/11-ppc64le.dockerfile | 33 +++ automation/build/debian/11-riscv64.dockerfile | 33 +++ automation/build/debian/12-ppc64le.dockerfile | 36 +++ automation/build/debian/12-riscv64.dockerfile | 36 +++ ...ockerfile => 12-x86_64-gcc-ibt.dockerfile} | 81 +++---.../build/debian/bullseye-ppc64le.dockerfile | 32 --- .../build/debian/jessie-i386.dockerfile | 55 ----- automation/build/debian/jessie.dockerfile | 52 ---- .../build/debian/stretch-i386.dockerfile | 57 ----- automation/build/debian/stretch.dockerfile | 60 ----- automation/build/ubuntu/trusty.dockerfile | 51 ---- automation/gitlab-ci/build.yaml | 233 +++++++--------- --
automation/gitlab-ci/test.yaml | 13 +- automation/scripts/containerize | 15 +- automation/scripts/qemu-smoke-ppc64le.sh | 4 +- .../qemu-system-ppc64/8.1.0-ppc64.dockerfile | 38 ---17 files changed, 282 insertions(+), 569 deletions(-)delete mode 100644 automation/build/archlinux/current- riscv64.dockerfile
create mode 100644 automation/build/debian/11-ppc64le.dockerfile create mode 100644 automation/build/debian/11-riscv64.dockerfile create mode 100644 automation/build/debian/12-ppc64le.dockerfile create mode 100644 automation/build/debian/12-riscv64.dockerfile rename automation/build/debian/{buster-gcc-ibt.dockerfile => 12- x86_64-gcc-ibt.dockerfile} (50%)
delete mode 100644 automation/build/debian/bullseye- ppc64le.dockerfile
delete mode 100644 automation/build/debian/jessie-i386.dockerfile delete mode 100644 automation/build/debian/jessie.dockerfile delete mode 100644 automation/build/debian/stretch-i386.dockerfile delete mode 100644 automation/build/debian/stretch.dockerfile delete mode 100644 automation/build/ubuntu/trusty.dockerfile delete mode 100644 automation/tests-artifacts/qemu-system- ppc64/8.1.0-ppc64.dockerfile
[PATCH 13/12] CI: Refresh OpenSUSE Tumbleweed container
In reply to Andrew CooperExisting as suse:opensuse-tumbleweed is a historical quirk, and adjusted for consistency with all the other containers.
Make it non-root, use heredocs for legibility, and use the zypper long names for the benefit of those wondering what was being referenced or duplicated.
Trim the dependencies substantially. Testing docs isn't very interesting and saves a lot of space (pandoc pulls in all of Haskel). Other savings come from removing a huge pile of optional QEMU dependencies (QEMU just needs to build the Xen parts to be useful here, not have a full GUI environment).
Finally, there where some packages such as bc, libssh2-devel, libtasn1-devel and nasm that I'm not aware of any reason to have had, even historically.
This shaves 1G off the size of the container:
registry.gitlab.com/xen-project/xen/opensuse tumbleweed-x86_64 1.39GB registry.gitlab.com/xen-project/xen/suse opensuse-tumbleweed 2.35GB
Furthermore, identify which components of the build use which dependencies, which will help managing them in the future.
Thanks to Olaf Hering for dependency fixes that have been subsumed into this total overhaul.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
CC: Olaf Hering <[email protected]>
This includes both of Olaf's recent changes, python3-packaging (now required for a rebuilt container), and findutils (not required yet; still gets pulled in transitively), but added for good measure.
In terms of package delta, newly included are:
findutils # As suggested ocaml-findlib # Replaces ocaml-findlib-devel perl # Transitive previously python3 # Transitive previously python3-packaging # As suggested
while removed are:
bc # ? cmake # vtpm stubdom, optional discount # docs ghostscript # docs glibc-devel # Not really optional, given a toolchain hostname # Optional, not interesting libaio-devel # Xen < 4.13, perhaps Qemu still but optional libext2fs-devel # Libfsimage, off by default, optional libgnutls-devel # Qemu ? libjpeg62-devel # Qemu ? libnl3-devel # Libxl COLO(?), option libnuma-devel # ? libpng16-devel # Qemu ? libssh2-devel # ? libtasn1-devel # ? nasm # ? ocaml-findlib-devel # Swapped for non-devel ocaml-ocamlbuild # ? ocaml-ocamldoc # ? pandoc # docs 'pkgconfig(libpci)' # Qemu 'pkgconfig(sdl)' # Qemu 'pkgconfig(sdl2)' # Qemu systemd-devel # Xen < 4.19, optional transfig # docs valgrind-devel # really only for developers which # Xen < 4.17
I've tested this on staging-4.17 and staging, both of which build fine. staging-4.16 does not, because of SeaBIOS which we've decided not to fix, hence it's safe to drop which.
--- .../opensuse/tumbleweed-x86_64.dockerfile | 76 +++++++++++++++++++ .../build/suse/opensuse-tumbleweed.dockerfile | 74 ------------------ automation/gitlab-ci/build.yaml | 8 +- automation/scripts/containerize | 2 +- 4 files changed, 81 insertions(+), 79 deletions(-) create mode 100644 automation/build/opensuse/tumbleweed-x86_64.dockerfile delete mode 100644 automation/build/suse/opensuse-tumbleweed.dockerfile
automation/build/opensuse/tumbleweed-x86_64.dockerfile new file +76 -0
@@ -0,0 +1,76 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 opensuse/tumbleweed +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV XEN_TARGET_ARCH=x86_64 + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + zypper refresh + zypper dist-upgrade -y --no-recommends + + DEPS=( + # Xen + bison + checkpolicy + clang + diffutils + findutils + flex + gawk + gcc + make + python3 + + # Tools (general) + git + gzip + patch + perl + pkg-config + wget + # libxenguest dombuilder + libbz2-devel + libzstd-devel + lzo-devel + xz-devel + zlib-devel + # libacpi + acpica + # libxl + libuuid-devel + libyajl-devel + # Header Check + gcc-c++ + # xentop + ncurses-devel + # Python bindings + python3-devel + python3-setuptools + # Ocaml bindings/oxenstored + ocaml + ocaml-findlib + + # Stubdom (download/extract) + bzip2 + tar + + # Qemu build + glib2-devel + libpixman-1-0-devel + meson + ninja + python3-packaging + ) + + zypper install -y --no-recommends "${DEPS[@]}" + zypper clean -a +EOF + +USER user +WORKDIR /build
automation/build/suse/opensuse-tumbleweed.dockerfile deleted +0 -74
74 lines not on this page · raw message
automation/gitlab-ci/build.yaml +4 -4
@@ -668,25 +668,25 @@ opensuse-leap-gcc-debug: opensuse-tumbleweed-clang: extends: .clang-x86-64-build variables: - CONTAINER: suse:opensuse-tumbleweed + CONTAINER: opensuse:tumbleweed-x86_64 allow_failure: true opensuse-tumbleweed-clang-debug: extends: .clang-x86-64-build-debug variables: - CONTAINER: suse:opensuse-tumbleweed + CONTAINER: opensuse:tumbleweed-x86_64 allow_failure: true opensuse-tumbleweed-gcc: extends: .gcc-x86-64-build variables: - CONTAINER: suse:opensuse-tumbleweed + CONTAINER: opensuse:tumbleweed-x86_64 allow_failure: true opensuse-tumbleweed-gcc-debug: extends: .gcc-x86-64-build-debug variables: - CONTAINER: suse:opensuse-tumbleweed + CONTAINER: opensuse:tumbleweed-x86_64 allow_failure: true # PowerPC builds (x86 cross)
automation/scripts/containerize +1 -1
@@ -43,7 +43,7 @@ case "_${CONTAINER}" in _bionic) CONTAINER="${BASE}/ubuntu:bionic" ;; _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;; _opensuse-leap|_leap) CONTAINER="${BASE}/suse:opensuse-leap" ;; - _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/suse:opensuse-tumbleweed" ;; + _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;; esac # Use this variable to control whether root should be used
Signature · 1 line
2.39.2
Re: [PATCH 13/12] CI: Refresh OpenSUSE Tumbleweed container
Quoted 79 lines · Andrew Cooper
Existing as suse:opensuse-tumbleweed is a historical quirk, and adjusted for consistency with all the other containers.
Make it non-root, use heredocs for legibility, and use the zypper long names for the benefit of those wondering what was being referenced or duplicated.
Trim the dependencies substantially. Testing docs isn't very interesting and saves a lot of space (pandoc pulls in all of Haskel). Other savings come from removing a huge pile of optional QEMU dependencies (QEMU just needs to build the Xen parts to be useful here, not have a full GUI environment).
Finally, there where some packages such as bc, libssh2-devel, libtasn1-devel and nasm that I'm not aware of any reason to have had, even historically.
This shaves 1G off the size of the container:
registry.gitlab.com/xen-project/xen/opensuse tumbleweed-x86_64 1.39GB registry.gitlab.com/xen-project/xen/suse opensuse-tumbleweed 2.35GBFurthermore, identify which components of the build use which dependencies, which will help managing them in the future.
Thanks to Olaf Hering for dependency fixes that have been subsumed into this total overhaul.
Signed-off-by Andrew Cooper <[email protected]>---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
CC: Olaf Hering <[email protected]>This includes both of Olaf's recent changes, python3-packaging (now required for a rebuilt container), and findutils (not required yet; still gets pulled in transitively), but added for good measure.
In terms of package delta, newly included are:
findutils # As suggested ocaml-findlib # Replaces ocaml-findlib-devel perl # Transitive previously python3 # Transitive previously python3-packaging # As suggestedwhile removed are:
bc # ? cmake # vtpm stubdom, optional discount # docs ghostscript # docs glibc-devel # Not really optional, given a toolchain hostname # Optional, not interesting libaio-devel # Xen < 4.13, perhaps Qemu still but optional libext2fs-devel # Libfsimage, off by default, optional libgnutls-devel # Qemu ? libjpeg62-devel # Qemu ? libnl3-devel # Libxl COLO(?), option libnuma-devel # ? libpng16-devel # Qemu ? libssh2-devel # ? libtasn1-devel # ? nasm # ? ocaml-findlib-devel # Swapped for non-devel ocaml-ocamlbuild # ? ocaml-ocamldoc # ? pandoc # docs 'pkgconfig(libpci)' # Qemu 'pkgconfig(sdl)' # Qemu 'pkgconfig(sdl2)' # Qemu systemd-devel # Xen < 4.19, optional transfig # docs valgrind-devel # really only for developers which # Xen < 4.17
Quoted 3 lines
I've tested this on staging-4.17 and staging, both of which build fine. staging-4.16 does not, because of SeaBIOS which we've decided not to fix, hence it's safe to drop
which.
For good measure, runs with this container:
4.17: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7322567620
4.19: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7322493465
and the expected failure:
4.16: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7322499885
~Andrew
Re: [PATCH 13/12] CI: Refresh OpenSUSE Tumbleweed container
In reply to Andrew CooperThu, 11 Jul 2024 21:23:37 +0100 Andrew Cooper <[email protected]>:
Quoted 1 line
use the zypper long names for the benefit of those wondering what was being referenced or duplicated.
Names of rpm packages do change over time, configure already searches for "something" where it is available, instead of some library or header. If it is touched anyway, I would use more instead of less 'pkgconfig(something)' references:
libbz2-devel
pkgconfig(bzip2) = 1.0.8
libzstd-devel
pkgconfig(libzstd) = 1.5.6
lzo-devel
pkgconfig(lzo2) = 2.10
xz-devel
pkgconfig(liblzma) = 5.6.2
zlib-devel
pkgconfig(zlib) = 1.3.1
libuuid-devel
pkgconfig(uuid) = 2.40.1
libyajl-devel
pkgconfig(yajl) = 2.1.0
ncurses-devel
pkgconfig(form) = 6.5.20240629
pkgconfig(formw) = 6.5.20240629
pkgconfig(menu) = 6.5.20240629
pkgconfig(menuw) = 6.5.20240629
pkgconfig(ncurses) = 6.5.20240629
pkgconfig(ncurses++) = 6.5.20240629
pkgconfig(ncurses++w) = 6.5.20240629
pkgconfig(ncursesw) = 6.5.20240629
pkgconfig(panel) = 6.5.20240629
pkgconfig(panelw) = 6.5.20240629
pkgconfig(tic) = 6.5.20240629
pkgconfig(tinfo) = 6.5.20240629
glib2-devel
pkgconfig(gio-2.0) = 2.80.3
pkgconfig(gio-unix-2.0) = 2.80.3
pkgconfig(girepository-2.0) = 2.80.3
pkgconfig(glib-2.0) = 2.80.3
pkgconfig(gmodule-2.0) = 2.80.3
pkgconfig(gmodule-export-2.0) = 2.80.3
pkgconfig(gmodule-no-export-2.0) = 2.80.3
pkgconfig(gobject-2.0) = 2.80.3
pkgconfig(gthread-2.0) = 2.80.3
libpixman-1-0-devel
pkgconfig(pixman-1) = 0.43.4python3-devel is "special", can be used as is.
'git' could be just 'git-core'
Olaf
Re: [PATCH 13/12] CI: Refresh OpenSUSE Tumbleweed container
Quoted 4 lines · Olaf Hering
Thu, 11 Jul 2024 21:23:37 +0100 Andrew Cooper <[email protected]>:
Quoted 1 line
use the zypper long names for the benefit of those wondering what was being referenced or duplicated.
Names of rpm packages do change over time, configure already searches for "something" where it is available,
configure has to cope with distros taking different naming decisions, and while I appreciate that packages may change over time even within a distro ...
Quoted 34 lines
instead of some library or header.
If it is touched anyway, I would use more instead of less 'pkgconfig(something)' references:libbz2-devel
pkgconfig(bzip2) = 1.0.8 libzstd-devel pkgconfig(libzstd) = 1.5.6 lzo-devel pkgconfig(lzo2) = 2.10 xz-devel pkgconfig(liblzma) = 5.6.2 zlib-devel pkgconfig(zlib) = 1.3.1 libuuid-devel pkgconfig(uuid) = 2.40.1 libyajl-devel pkgconfig(yajl) = 2.1.0 ncurses-devel pkgconfig(form) = 6.5.20240629 pkgconfig(formw) = 6.5.20240629 pkgconfig(menu) = 6.5.20240629 pkgconfig(menuw) = 6.5.20240629 pkgconfig(ncurses) = 6.5.20240629 pkgconfig(ncurses++) = 6.5.20240629 pkgconfig(ncurses++w) = 6.5.20240629 pkgconfig(ncursesw) = 6.5.20240629 pkgconfig(panel) = 6.5.20240629 pkgconfig(panelw) = 6.5.20240629 pkgconfig(tic) = 6.5.20240629 pkgconfig(tinfo) = 6.5.20240629 glib2-devel pkgconfig(gio-2.0) = 2.80.3 pkgconfig(gio-unix-2.0) = 2.80.3 pkgconfig(girepository-2.0) = 2.80.3
Quoted 8 lines
pkgconfig(glib-2.0) = 2.80.3
pkgconfig(gmodule-2.0) = 2.80.3
pkgconfig(gmodule-export-2.0) = 2.80.3
pkgconfig(gmodule-no-export-2.0) = 2.80.3
pkgconfig(gobject-2.0) = 2.80.3
pkgconfig(gthread-2.0) = 2.80.3
libpixman-1-0-devel
pkgconfig(pixman-1) = 0.43.4... this is just a differently-inconsistent list of names, with some very dubious aliases such as form and menu.
What makes pkgconfig() any more stable than the main package name?
Why do we want to be treating libraries different to other packages?
Quoted 1 line
python3-devel is "special", can be used as is.
As an aside, I looked for a python3-minimal or equivalent, and couldn't find one. I take it there isn't one?
Quoted 1 line
'git' could be just 'git-core'
Thanks. Will fix locally, for Tumbleweed and Leap.
~Andrew
Re: [PATCH 13/12] CI: Refresh OpenSUSE Tumbleweed container
Thu, 11 Jul 2024 23:39:42 +0100 Andrew Cooper <[email protected]>:
Quoted 3 lines
configure has to cope with distros taking different naming decisions, and while I appreciate that packages may change over time even within a distro ...
configure does "pkgconfig --exists something" to probe for some API. I'm sure noone will rename the pkgconfig files, "something.pc" in this case.
Quoted 2 lines
... this is just a differently-inconsistent list of names, with some very dubious aliases such as
formandmenu.
I should have truncated that list of names, 'pkgconfig(ncurses)' and 'pkgconfig(glib-2.0)' are relevant.
Quoted 1 line
What makes
pkgconfig()any more stable than the main package name?
They are supposed to describe some API, and in some cases some functionality. It does not matter much which package provides the API.
Quoted 1 line
Why do we want to be treating libraries different to other packages?
Because there is no other namespace to describe functionality. It was implemented for pkg-config, it exists for some cmake and perl modules.
Quoted 3 lines
Quoted 1 line
python3-devel is "special", can be used as is.
As an aside, I looked for a python3-minimal or equivalent, and couldn't find one. I take it there isn't one?
I think there are libraries compiled. There is indeed a 'pkgconfig(python3)', my search just failed to find it. Otherwise, if there are really just .py files in the tree, 'python3-base' is the string provided by 'python3N-base'.
Olaf
[PATCH v2 13/12] CI: Refresh OpenSUSE Tumbleweed container
In reply to Andrew CooperExisting as suse:opensuse-tumbleweed is a historical quirk, and adjusted for consistency with all the other containers.
Make it non-root, use heredocs for legibility, and use the zypper long names for the benefit of those wondering what was being referenced or duplicated.
Trim the dependencies substantially. Testing docs isn't very interesting and saves a lot of space. Other savings come from removing a huge pile of optional QEMU dependencies (QEMU just needs to build the Xen parts to be useful here, not have a full GUI environment).
Finally, there where some packages such as bc, libssh2-devel, libtasn1-devel and nasm that I'm not aware of any reason to have had, even historically.
This shaves 1G off the size of the container:
registry.gitlab.com/xen-project/xen/opensuse tumbleweed-x86_64 1.39GB registry.gitlab.com/xen-project/xen/suse opensuse-tumbleweed 2.35GB
Furthermore, identify which components of the build use which dependencies, which will help managing them in the future.
Thanks to Olaf Hering for dependency fixes that have been subsumed into this total overhaul.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
CC: Olaf Hering <[email protected]>
This includes both of Olaf's recent changes, python3-packaging (now required for a rebuilt container), and findutils (not required yet; still gets pulled in transitively), but added for good measure.
In terms of package delta, newly included are:
findutils # As suggested ocaml-findlib # Replaces ocaml-findlib-devel perl # Transitive previously python3 # Transitive previously python3-packaging # As suggested
while removed are:
bc # ? cmake # vtpm stubdom, optional discount # docs ghostscript # docs glibc-devel # Not really optional, given a toolchain hostname # Optional, not interesting libaio-devel # Xen < 4.13, perhaps Qemu still but optional libext2fs-devel # Libfsimage, off by default, optional libgnutls-devel # Qemu ? libjpeg62-devel # Qemu ? libnl3-devel # Libxl COLO(?), option libnuma-devel # ? libpng16-devel # Qemu ? libssh2-devel # ? libtasn1-devel # ? nasm # ? ocaml-findlib-devel # Swapped for non-devel ocaml-ocamlbuild # ? ocaml-ocamldoc # ? pandoc # docs 'pkgconfig(libpci)' # Qemu 'pkgconfig(sdl)' # Qemu 'pkgconfig(sdl2)' # Qemu systemd-devel # Xen < 4.19, optional transfig # docs valgrind-devel # really only for developers which # Xen < 4.17
I've tested this on staging-4.17 and staging, both of which build fine. staging-4.16 does not, because of SeaBIOS which we've decided not to fix, hence it's safe to drop which.
v2:
* git -> git-core
* Use 'pkgconfig()' references
--- .../opensuse/tumbleweed-x86_64.dockerfile | 76 +++++++++++++++++++ .../build/suse/opensuse-tumbleweed.dockerfile | 74 ------------------ automation/gitlab-ci/build.yaml | 8 +- automation/scripts/containerize | 2 +- 4 files changed, 81 insertions(+), 79 deletions(-) create mode 100644 automation/build/opensuse/tumbleweed-x86_64.dockerfile delete mode 100644 automation/build/suse/opensuse-tumbleweed.dockerfile
automation/build/opensuse/tumbleweed-x86_64.dockerfile new file +76 -0
@@ -0,0 +1,76 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 opensuse/tumbleweed +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV XEN_TARGET_ARCH=x86_64 + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + zypper refresh + zypper dist-upgrade -y --no-recommends + + DEPS=( + # Xen + bison + checkpolicy + clang + diffutils + findutils + flex + gawk + gcc + make + python3 + + # Tools (general) + git-core + gzip + patch + perl + pkg-config + wget + # libxenguest dombuilder + 'pkgconfig(bzip2)' + 'pkgconfig(libzstd)' + 'pkgconfig(lzo2)' + 'pkgconfig(liblzma)' + 'pkgconfig(zlib)' + # libacpi + acpica + # libxl + 'pkgconfig(uuid)' + 'pkgconfig(yajl)' + # Header Check + gcc-c++ + # xentop + 'pkgconfig(ncurses)' + # Python bindings + python3-devel + python3-setuptools + # Ocaml bindings/oxenstored + ocaml + ocaml-findlib + + # Stubdom (download/extract) + bzip2 + tar + + # Qemu build + meson + ninja + 'pkgconfig(glib-2.0)' + 'pkgconfig(pixman-1)' + python3-packaging + ) + + zypper install -y --no-recommends "${DEPS[@]}" + zypper clean -a +EOF + +USER user +WORKDIR /build
automation/build/suse/opensuse-tumbleweed.dockerfile deleted +0 -74
74 lines not on this page · raw message
automation/gitlab-ci/build.yaml +4 -4
@@ -668,25 +668,25 @@ opensuse-leap-gcc-debug: opensuse-tumbleweed-clang: extends: .clang-x86-64-build variables: - CONTAINER: suse:opensuse-tumbleweed + CONTAINER: opensuse:tumbleweed-x86_64 allow_failure: true opensuse-tumbleweed-clang-debug: extends: .clang-x86-64-build-debug variables: - CONTAINER: suse:opensuse-tumbleweed + CONTAINER: opensuse:tumbleweed-x86_64 allow_failure: true opensuse-tumbleweed-gcc: extends: .gcc-x86-64-build variables: - CONTAINER: suse:opensuse-tumbleweed + CONTAINER: opensuse:tumbleweed-x86_64 allow_failure: true opensuse-tumbleweed-gcc-debug: extends: .gcc-x86-64-build-debug variables: - CONTAINER: suse:opensuse-tumbleweed + CONTAINER: opensuse:tumbleweed-x86_64 allow_failure: true # PowerPC builds (x86 cross)
automation/scripts/containerize +1 -1
@@ -43,7 +43,7 @@ case "_${CONTAINER}" in _bionic) CONTAINER="${BASE}/ubuntu:bionic" ;; _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;; _opensuse-leap|_leap) CONTAINER="${BASE}/suse:opensuse-leap" ;; - _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/suse:opensuse-tumbleweed" ;; + _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;; esac # Use this variable to control whether root should be used
Signature · 1 line
2.39.2
Anthony PERARD
Reviewed-byRe: [PATCH v2 13/12] CI: Refresh OpenSUSE Tumbleweed container
Quoted 3 lines · Andrew Cooper
while removed are:
bc # ?
I don't know why, but this one is also installed by osstest. The commit adding it doesn't say why it was needed.
Also, looks like many QEMU containers are installing bc, but I can't find a single use. Maybe historic.
Quoted 1 line
libnl3-devel # Libxl COLO(?), option
Yes.
Quoted 1 line
libnuma-devel # ?
There's some mention of it in QEMU.
Quoted 1 line
libpng16-devel # Qemu ?
It's added to a few QEMU containers, but I can't find any use.
Quoted 1 line
libssh2-devel # ?
It seems like it was used by QEMU at some point, but they switch to 'libssh'. Probably optional, like many QEMU deps.
Quoted 1 line
libtasn1-devel # ?
QEMU, and it seems to be only used in tests.
Quoted 1 line
nasm # ?
'nasm' is used to build OVMF.
If we don't want to build OVMF in our GitLab CI, which I don't think would be useful to do anyway (no dependency on xen.git), then:
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH v2 13/12] CI: Refresh OpenSUSE Tumbleweed container
Quoted 9 lines · Anthony PERARD
Quoted 3 lines · Andrew Cooper
while removed are:
bc # ?
I don't know why, but this one is also installed by osstest. The commit adding it doesn't say why it was needed.
Also, looks like many QEMU containers are installing
bc, but I can't find a single use. Maybe historic.
Part of the reason why our containers are so massive is because dependencies only ever accumulate...
But yes - this one is especially odd.
Quoted 6 lines
Quoted 1 line
nasm # ?
'nasm' is used to build OVMF.
If we don't want to build OVMF in our GitLab CI, which I don't think would be useful to do anyway (no dependency on xen.git)
Oh, interesting.
I'd assumed (by virtue of the jobs succeeding) that we were building OVMF by default, but we're clearly not.
./configure says:
--enable-ovmf Enable OVMF (default is DISABLED)
so it's really just a simple opt-in, which our build script does not.
In the longer term, we will want OVMF in Gitlab CI, but we'd want to pull it in as a separate artefact and use --with-system-ovmf=/path/to/artefact
So I agree we don't need to build OVMF inside this container.
Quoted 3 lines
, then:
Reviewed-by Anthony PERARD <[email protected]>
Thanks.
~Andrew
[PATCH 14/12] CI: Refresh OpenSUSE Leap container
In reply to Andrew CooperSee prior patch for most discussion.
Despite appearing to a fixed release (and therefore not marked as permitted failure), the dockerfile references the leap tag which is rolling in practice. Switch to 15.6 explicitly, for better test stability.
Vs tumbleweed, use zypper update rather than dist-upgrade, and retain the RomBIOS dependencies; bin86 and dev86.
In terms of size, this saves ~700M:
registry.gitlab.com/xen-project/xen/opensuse leap-15.6-x86_64 1.33GB registry.gitlab.com/xen-project/xen/suse opensuse-leap 2.05GB
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
CC: Olaf Hering <[email protected]>
Runs with this new container:
4.19: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7322493460
4.16: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7322499881
--- .../opensuse/leap-15.6-x86_64.dockerfile | 79 +++++++++++++++++++ .../build/suse/opensuse-leap.dockerfile | 75 ------------------ automation/gitlab-ci/build.yaml | 16 ++-- automation/scripts/containerize | 2 +- 4 files changed, 88 insertions(+), 84 deletions(-) create mode 100644 automation/build/opensuse/leap-15.6-x86_64.dockerfile delete mode 100644 automation/build/suse/opensuse-leap.dockerfile
automation/build/opensuse/leap-15.6-x86_64.dockerfile new file +79 -0
@@ -0,0 +1,79 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 opensuse/leap:15.6 +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV XEN_TARGET_ARCH=x86_64 + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + zypper refresh + zypper update -y --no-recommends + + DEPS=( + # Xen + bison + checkpolicy + clang + diffutils + findutils + flex + gawk + gcc + make + python3 + + # Tools (general) + git + gzip + patch + perl + pkg-config + wget + # libxenguest dombuilder + libbz2-devel + libzstd-devel + lzo-devel + xz-devel + zlib-devel + # libacpi + acpica + # libxl + libuuid-devel + libyajl-devel + # RomBIOS + bin86 + dev86 + # Header Check + gcc-c++ + # xentop + ncurses-devel + # Python bindings + python3-devel + python3-setuptools + # Ocaml bindings/oxenstored + ocaml + ocaml-findlib + + # Stubdom (download/extract) + bzip2 + tar + + # Qemu build + glib2-devel + libpixman-1-0-devel + meson + ninja + python3-packaging + ) + + zypper install -y --no-recommends "${DEPS[@]}" + zypper clean -a +EOF + +USER user +WORKDIR /build
automation/build/suse/opensuse-leap.dockerfile deleted +0 -75
75 lines not on this page · raw message
automation/gitlab-ci/build.yaml +8 -8
@@ -645,25 +645,25 @@ ubuntu-focal-clang-debug: variables: CONTAINER: ubuntu:focal -opensuse-leap-clang: +opensuse-leap-15.6-clang: extends: .clang-x86-64-build variables: - CONTAINER: suse:opensuse-leap + CONTAINER: opensuse:leap-15.6-x86_64 -opensuse-leap-clang-debug: +opensuse-leap-15.6-clang-debug: extends: .clang-x86-64-build-debug variables: - CONTAINER: suse:opensuse-leap + CONTAINER: opensuse:leap-15.6-x86_64 -opensuse-leap-gcc: +opensuse-leap-15.6-gcc: extends: .gcc-x86-64-build variables: - CONTAINER: suse:opensuse-leap + CONTAINER: opensuse:leap-15.6-x86_64 -opensuse-leap-gcc-debug: +opensuse-leap-15.6-gcc-debug: extends: .gcc-x86-64-build-debug variables: - CONTAINER: suse:opensuse-leap + CONTAINER: opensuse:leap-15.6-x86_64 opensuse-tumbleweed-clang: extends: .clang-x86-64-build
automation/scripts/containerize +1 -1
@@ -42,7 +42,7 @@ case "_${CONTAINER}" in _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;; _bionic) CONTAINER="${BASE}/ubuntu:bionic" ;; _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;; - _opensuse-leap|_leap) CONTAINER="${BASE}/suse:opensuse-leap" ;; + _opensuse-leap|_leap) CONTAINER="${BASE}/opensuse:leap-15.6-x86_64" ;; _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;; esac
Signature · 1 line
2.39.2
[PATCH v2 14/12] CI: Refresh OpenSUSE Leap container
See prior patch for most discussion.
Despite appearing to be a fixed release (and therefore not marked as permitted failure), the dockerfile references the leap tag which is rolling in practice. Switch to 15.6 explicitly, for better test stability.
Vs tumbleweed, use zypper update rather than dist-upgrade, and retain the RomBIOS dependencies; bin86 and dev86.
In terms of size, this saves ~700M:
registry.gitlab.com/xen-project/xen/opensuse leap-15.6-x86_64 1.33GB registry.gitlab.com/xen-project/xen/suse opensuse-leap 2.05GB
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
CC: Olaf Hering <[email protected]>
Runs with this new container:
4.19: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7322493460
4.16: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7322499881
v2:
* git -> git-core
* Use 'pkgconfig()' references
--- .../opensuse/leap-15.6-x86_64.dockerfile | 79 +++++++++++++++++++ .../build/suse/opensuse-leap.dockerfile | 75 ------------------ automation/gitlab-ci/build.yaml | 16 ++-- automation/scripts/containerize | 2 +- 4 files changed, 88 insertions(+), 84 deletions(-) create mode 100644 automation/build/opensuse/leap-15.6-x86_64.dockerfile delete mode 100644 automation/build/suse/opensuse-leap.dockerfile
automation/build/opensuse/leap-15.6-x86_64.dockerfile new file +79 -0
@@ -0,0 +1,79 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 opensuse/leap:15.6 +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV XEN_TARGET_ARCH=x86_64 + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + zypper refresh + zypper update -y --no-recommends + + DEPS=( + # Xen + bison + checkpolicy + clang + diffutils + findutils + flex + gawk + gcc + make + python3 + + # Tools (general) + git-core + gzip + patch + perl + pkg-config + wget + # libxenguest dombuilder + 'pkgconfig(bzip2)' + 'pkgconfig(libzstd)' + 'pkgconfig(lzo2)' + 'pkgconfig(liblzma)' + 'pkgconfig(zlib)' + # libacpi + acpica + # libxl + 'pkgconfig(uuid)' + 'pkgconfig(yajl)' + # RomBIOS + bin86 + dev86 + # Header Check + gcc-c++ + # xentop + 'pkgconfig(ncurses)' + # Python bindings + python3-devel + python3-setuptools + # Ocaml bindings/oxenstored + ocaml + ocaml-findlib + + # Stubdom (download/extract) + bzip2 + tar + + # Qemu build + meson + ninja + 'pkgconfig(glib-2.0)' + 'pkgconfig(pixman-1)' + python3-packaging + ) + + zypper install -y --no-recommends "${DEPS[@]}" + zypper clean -a +EOF + +USER user +WORKDIR /build
automation/build/suse/opensuse-leap.dockerfile deleted +0 -75
75 lines not on this page · raw message
automation/gitlab-ci/build.yaml +8 -8
@@ -645,25 +645,25 @@ ubuntu-focal-clang-debug: variables: CONTAINER: ubuntu:focal -opensuse-leap-clang: +opensuse-leap-15.6-clang: extends: .clang-x86-64-build variables: - CONTAINER: suse:opensuse-leap + CONTAINER: opensuse:leap-15.6-x86_64 -opensuse-leap-clang-debug: +opensuse-leap-15.6-clang-debug: extends: .clang-x86-64-build-debug variables: - CONTAINER: suse:opensuse-leap + CONTAINER: opensuse:leap-15.6-x86_64 -opensuse-leap-gcc: +opensuse-leap-15.6-gcc: extends: .gcc-x86-64-build variables: - CONTAINER: suse:opensuse-leap + CONTAINER: opensuse:leap-15.6-x86_64 -opensuse-leap-gcc-debug: +opensuse-leap-15.6-gcc-debug: extends: .gcc-x86-64-build-debug variables: - CONTAINER: suse:opensuse-leap + CONTAINER: opensuse:leap-15.6-x86_64 opensuse-tumbleweed-clang: extends: .clang-x86-64-build
automation/scripts/containerize +1 -1
@@ -42,7 +42,7 @@ case "_${CONTAINER}" in _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;; _bionic) CONTAINER="${BASE}/ubuntu:bionic" ;; _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;; - _opensuse-leap|_leap) CONTAINER="${BASE}/suse:opensuse-leap" ;; + _opensuse-leap|_leap) CONTAINER="${BASE}/opensuse:leap-15.6-x86_64" ;; _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;; esac
Signature · 1 line
2.39.2
Anthony PERARD
Reviewed-byRe: [PATCH v2 14/12] CI: Refresh OpenSUSE Leap container
Quoted 9 lines · Andrew Cooper
See prior patch for most discussion.
Despite appearing to be a fixed release (and therefore not marked as permitted failure), the dockerfile references the
leaptag which is rolling in practice. Switch to 15.6 explicitly, for better test stability.Vs tumbleweed, use
zypper updaterather than dist-upgrade, and retain the RomBIOS dependencies; bin86 and dev86.
Quoted 6 lines
In terms of size, this saves ~700M:
registry.gitlab.com/xen-project/xen/opensuse leap-15.6-x86_64 1.33GB registry.gitlab.com/xen-project/xen/suse opensuse-leap 2.05GBSigned-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
[PATCH 15/12] CI: Refresh Ubuntu Xenial container as 16.04-x86_64
In reply to Andrew CooperRework the container to be non-root, use heredocs for legibility, and use use apt-get --no-install-recommends to keep the size down.
Ubuntu Xenial has no ninja-build package, so can't build the QEMU referenced by Xen 4.16. Therefore, drop the dependencies too.
This saves ~500M:
registry.gitlab.com/xen-project/xen/ubuntu 16.04-x86_64 700MB registry.gitlab.com/xen-project/xen/ubuntu xenial 1.21GB
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
Runs with the new container:
4.16: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7323548805
4.19: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7323548805
--- .../build/ubuntu/16.04-x86_64.dockerfile | 65 +++++++++++++++++++ automation/build/ubuntu/xenial.dockerfile | 46 ------------- automation/gitlab-ci/build.yaml | 16 ++--- automation/scripts/containerize | 2 +- 4 files changed, 74 insertions(+), 55 deletions(-) create mode 100644 automation/build/ubuntu/16.04-x86_64.dockerfile delete mode 100644 automation/build/ubuntu/xenial.dockerfile
automation/build/ubuntu/16.04-x86_64.dockerfile new file +65 -0
@@ -0,0 +1,65 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 ubuntu:16.04 +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV DEBIAN_FRONTEND=noninteractive + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy + clang + flex + python3-minimal + + # Tools (general) + ca-certificates + git-core + gzip + patch + perl + pkg-config + wget + # libxenguest dombuilder + libbz2-dev + libzstd-dev + liblzo2-dev + liblzma-dev + zlib1g-dev + # libacpi + acpica-tools + # libxl + uuid-dev + libyajl-dev + # RomBIOS + bcc + bin86 + # xentop + libncurses5-dev + # Python bindings + python3-dev + python3-setuptools + # Ocaml bindings/oxenstored + ocaml-nox + ocaml-findlib + + # Stubdom download/extract + bzip2 + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF + +USER user +WORKDIR /build
automation/build/ubuntu/xenial.dockerfile deleted +0 -46
46 lines not on this page · raw message
automation/gitlab-ci/build.yaml +8 -8
@@ -588,25 +588,25 @@ fedora-gcc-debug: variables: CONTAINER: fedora:29 -ubuntu-xenial-clang: +ubuntu-16.04-x86_64-clang: extends: .clang-x86-64-build variables: - CONTAINER: ubuntu:xenial + CONTAINER: ubuntu:16.04-x86_64 -ubuntu-xenial-clang-debug: +ubuntu-16.04-x86_64-clang-debug: extends: .clang-x86-64-build-debug variables: - CONTAINER: ubuntu:xenial + CONTAINER: ubuntu:16.04-x86_64 -ubuntu-xenial-gcc: +ubuntu-16.04-x86_64-gcc: extends: .gcc-x86-64-build variables: - CONTAINER: ubuntu:xenial + CONTAINER: ubuntu:16.04-x86_64 -ubuntu-xenial-gcc-debug: +ubuntu-16.04-x86_64-gcc-debug: extends: .gcc-x86-64-build-debug variables: - CONTAINER: ubuntu:xenial + CONTAINER: ubuntu:16.04-x86_64 ubuntu-bionic-clang: extends: .clang-x86-64-build
automation/scripts/containerize +1 -1
@@ -52,9 +52,9 @@ case "_${CONTAINER}" in _bookworm-arm64v8) CONTAINER="${BASE}/debian:bookworm-arm64v8" ;; _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;; _bionic) CONTAINER="${BASE}/ubuntu:bionic" ;; - _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;; _opensuse-leap|_leap) CONTAINER="${BASE}/opensuse:leap-15.6-x86_64" ;; _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;; + _xenial) CONTAINER="${BASE}/ubuntu:16.04-x86_64" ;; *) guess_container ;; esac
Signature · 1 line
2.39.2
Anthony PERARD
Reviewed-byRe: [PATCH 15/12] CI: Refresh Ubuntu Xenial container as 16.04-x86_64
Quoted 6 lines · Andrew Cooper
Rework the container to be non-root, use heredocs for legibility, and use use apt-get --no-install-recommends to keep the size down.
Ubuntu Xenial has no ninja-build package, so can't build the QEMU referenced by Xen 4.16. Therefore, drop the dependencies too.
Quoted 6 lines
This saves ~500M:
registry.gitlab.com/xen-project/xen/ubuntu 16.04-x86_64 700MB registry.gitlab.com/xen-project/xen/ubuntu xenial 1.21GBSigned-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 15/12] CI: Refresh Ubuntu Xenial container as 16.04-x86_64
In reply to Andrew CooperQuoted 5 lines · Andrew Cooper
automation/scripts/containerize +0
-0
automation/scripts/containerize case "_${CONTAINER}" in · line 59
Quoted 9 lines
automation/scripts/containerize +1
-1@@ -52,9 +52,9 @@ case "_${CONTAINER}" in _bookworm-arm64v8) CONTAINER="${BASE}/debian:bookworm-arm64v8" ;; _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;; _bionic) CONTAINER="${BASE}/ubuntu:bionic" ;; - _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;; _opensuse-leap|_leap) CONTAINER="${BASE}/opensuse:leap-15.6-x86_64" ;; _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;; + _xenial) CONTAINER="${BASE}/ubuntu:16.04-x86_64" ;; *) guess_container ;;
I've been wondering why I couldn't apply the patch (and the next one), and there's these 2 lines of extra context ;-)
Cheers,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 15/12] CI: Refresh Ubuntu Xenial container as 16.04-x86_64
Quoted 14 lines · Anthony PERARD
Quoted 13 lines · Andrew Cooper
automation/scripts/containerize +1
-1@@ -52,9 +52,9 @@ case "_${CONTAINER}" in _bookworm-arm64v8) CONTAINER="${BASE}/debian:bookworm-arm64v8" ;; _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;; _bionic) CONTAINER="${BASE}/ubuntu:bionic" ;; - _xenial) CONTAINER="${BASE}/ubuntu:xenial" ;; _opensuse-leap|_leap) CONTAINER="${BASE}/opensuse:leap-15.6-x86_64" ;; _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;; + _xenial) CONTAINER="${BASE}/ubuntu:16.04-x86_64" ;;
Quoted 3 lines
automation/scripts/containerize case "_${CONTAINER}" in · line 59
Quoted 1 line
automation/scripts/containerize +0
-0*) guess_container ;;I've been wondering why I couldn't apply the patch (and the next one), and there's these 2 lines of extra context ;-)
Yeah, sorry. I found these when rebasing back into order.
https://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen.git;a=shortlog;h=refs/heads/auto-4.19
(and its simlarly-named variants) are the branches I'm doing all of this work on.
~Andrew
Re: [PATCH 15/12] CI: Refresh Ubuntu Xenial container as 16.04-x86_64
In reply to Andrew Cooperautomation/build/ubuntu/16.04-x86_64.dockerfile line 51
Quoted 3 lines · Andrew Cooper
automation/build/ubuntu/16.04-x86_64.dockerfile new file +1
-0+ python3-setuptools
Another thing,
I've just notice that "python3-setuptools" was originally only introduced to focal container on purpose, by:bbc72a7877d8 ("automation: Add python3's setuptools to some containers")
But now this patch series is introducing it to xenial and bionic. It might be worth mentioning, or removing the deps.
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 15/12] CI: Refresh Ubuntu Xenial container as 16.04-x86_64
Quoted 6 lines · Anthony PERARD
automation/build/ubuntu/16.04-x86_64.dockerfile line 51
Quoted 3 lines · Andrew Cooper
automation/build/ubuntu/16.04-x86_64.dockerfile new file +1
-0+ python3-setuptoolsAnother thing,
Quoted 7 lines
I've just notice that "python3-setuptools" was originally only introduced to focal container on purpose, by:
bbc72a7877d8("automation: Add python3's setuptools to some containers")But now this patch series is introducing it to xenial and bionic. It might be worth mentioning, or removing the deps.
Oh, good point. I'll take this back out of Xenial/Bionic.
Stretch/Trusty have already been removed.
Thanks,
~Andrew
[PATCH 16/12] CI: Refresh Ubuntu Bionic container as 18.04-x86_64
In reply to Andrew CooperAs with 16.04 (Focal), except that ninja-build is available so QEMU can be built.
This halves the size of the container:
registry.gitlab.com/xen-project/xen/ubuntu 18.04-x86_64 860MB registry.gitlab.com/xen-project/xen/ubuntu bionic 1.44GB
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
Runs with the new container:
4.16: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7323548809
4.19: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7323516970
--- .../build/ubuntu/18.04-x86_64.dockerfile | 72 +++++++++++++++++++ automation/build/ubuntu/bionic.dockerfile | 48 ------------- automation/gitlab-ci/build.yaml | 16 ++--- automation/scripts/containerize | 2 +- 4 files changed, 81 insertions(+), 57 deletions(-) create mode 100644 automation/build/ubuntu/18.04-x86_64.dockerfile delete mode 100644 automation/build/ubuntu/bionic.dockerfile
automation/build/ubuntu/18.04-x86_64.dockerfile new file +72 -0
@@ -0,0 +1,72 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 ubuntu:18.04 +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV DEBIAN_FRONTEND=noninteractive + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy + clang + flex + python3-minimal + + # Tools (general) + ca-certificates + git-core + gzip + patch + perl + pkg-config + wget + # libxenguest dombuilder + libbz2-dev + libzstd-dev + liblzo2-dev + liblzma-dev + zlib1g-dev + # libacpi + acpica-tools + # libxl + uuid-dev + libyajl-dev + # RomBIOS + bcc + bin86 + # xentop + libncurses5-dev + # Python bindings + python3-dev + python3-setuptools + # Ocaml bindings/oxenstored + ocaml-nox + ocaml-findlib + + # Stubdom download/extract + bzip2 + + # Qemu build + libglib2.0-dev + libpixman-1-dev + meson + ninja-build + python3-packaging + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF + +USER user +WORKDIR /build
automation/build/ubuntu/bionic.dockerfile deleted +0 -48
48 lines not on this page · raw message
automation/gitlab-ci/build.yaml +8 -8
@@ -608,25 +608,25 @@ ubuntu-16.04-x86_64-gcc-debug: variables: CONTAINER: ubuntu:16.04-x86_64 -ubuntu-bionic-clang: +ubuntu-18.04-x86_64-clang: extends: .clang-x86-64-build variables: - CONTAINER: ubuntu:bionic + CONTAINER: ubuntu:18.04-x86_64 -ubuntu-bionic-clang-debug: +ubuntu-18.04-x86_64-clang-debug: extends: .clang-x86-64-build-debug variables: - CONTAINER: ubuntu:bionic + CONTAINER: ubuntu:18.04-x86_64 -ubuntu-bionic-gcc: +ubuntu-18.04-x86_64-gcc: extends: .gcc-x86-64-build variables: - CONTAINER: ubuntu:bionic + CONTAINER: ubuntu:18.04-x86_64 -ubuntu-bionic-gcc-debug: +ubuntu-18.04-x86_64-gcc-debug: extends: .gcc-x86-64-build-debug variables: - CONTAINER: ubuntu:bionic + CONTAINER: ubuntu:18.04-x86_64 ubuntu-focal-gcc: extends: .gcc-x86-64-build
automation/scripts/containerize +1 -1
@@ -51,10 +51,10 @@ case "_${CONTAINER}" in _bookworm-arm64v8-arm32-gcc) CONTAINER="${BASE}/debian:bookworm-arm64v8-arm32-gcc" ;; _bookworm-arm64v8) CONTAINER="${BASE}/debian:bookworm-arm64v8" ;; _bookworm-cppcheck) CONTAINER="${BASE}/debian:bookworm-cppcheck" ;; - _bionic) CONTAINER="${BASE}/ubuntu:bionic" ;; _opensuse-leap|_leap) CONTAINER="${BASE}/opensuse:leap-15.6-x86_64" ;; _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;; _xenial) CONTAINER="${BASE}/ubuntu:16.04-x86_64" ;; + _bionic) CONTAINER="${BASE}/ubuntu:18.04-x86_64" ;; *) guess_container ;; esac
Signature · 1 line
2.39.2
Anthony PERARD
Reviewed-byRe: [PATCH 16/12] CI: Refresh Ubuntu Bionic container as 18.04-x86_64
Quoted 1 line · Andrew Cooper
As with 16.04 (Focal), except that ninja-build is available so QEMU can be
^ Xenial
These names all mixed up after a while :-)
Quoted 8 lines
built.
This halves the size of the container:
registry.gitlab.com/xen-project/xen/ubuntu 18.04-x86_64 860MB registry.gitlab.com/xen-project/xen/ubuntu bionic 1.44GBSigned-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 16/12] CI: Refresh Ubuntu Bionic container as 18.04-x86_64
Quoted 5 lines · Anthony PERARD
Quoted 1 line · Andrew Cooper
As with 16.04 (Focal), except that ninja-build is available so QEMU can be
^ Xenial
These names all mixed up after a while :-)
Bah, yes. This is one of the few cases where numbers are superior to names. Will fix.
Quoted 10 lines
Quoted 9 lines
built.
This halves the size of the container:
registry.gitlab.com/xen-project/xen/ubuntu 18.04-x86_64 860MB registry.gitlab.com/xen-project/xen/ubuntu bionic 1.44GBSigned-off-by Andrew Cooper <[email protected]>Reviewed-by Anthony PERARD <[email protected]>
Thanks,
~Andrew
[PATCH 17/12] CI: Refresh Ubuntu Focal container as 20.04-x86_64
In reply to Andrew CooperExactly as per 18.04 (Bionic). This saves ~500M:
registry.gitlab.com/xen-project/xen/ubuntu 20.04-x86_64 1.06GB registry.gitlab.com/xen-project/xen/ubuntu focal 1.57GB
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
Runs with the new container:
4.16: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7323548811
4.19: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7323517116
--- .../build/ubuntu/20.04-x86_64.dockerfile | 72 +++++++++++++++++++ automation/build/ubuntu/focal.dockerfile | 49 ------------- automation/gitlab-ci/build.yaml | 16 ++--- automation/scripts/containerize | 2 +- 4 files changed, 81 insertions(+), 58 deletions(-) create mode 100644 automation/build/ubuntu/20.04-x86_64.dockerfile delete mode 100644 automation/build/ubuntu/focal.dockerfile
automation/build/ubuntu/20.04-x86_64.dockerfile new file +72 -0
@@ -0,0 +1,72 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 ubuntu:20.04 +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV DEBIAN_FRONTEND=noninteractive + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy + clang + flex + python3-minimal + + # Tools (general) + ca-certificates + git-core + gzip + patch + perl + pkg-config + wget + # libxenguest dombuilder + libbz2-dev + libzstd-dev + liblzo2-dev + liblzma-dev + zlib1g-dev + # libacpi + acpica-tools + # libxl + uuid-dev + libyajl-dev + # RomBIOS + bcc + bin86 + # xentop + libncurses5-dev + # Python bindings + python3-dev + python3-setuptools + # Ocaml bindings/oxenstored + ocaml-nox + ocaml-findlib + + # Stubdom download/extract + bzip2 + + # Qemu build + libglib2.0-dev + libpixman-1-dev + meson + ninja-build + python3-packaging + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF + +USER user +WORKDIR /build
automation/build/ubuntu/focal.dockerfile deleted +0 -49
49 lines not on this page · raw message
automation/gitlab-ci/build.yaml +8 -8
@@ -628,25 +628,25 @@ ubuntu-18.04-x86_64-gcc-debug: variables: CONTAINER: ubuntu:18.04-x86_64 -ubuntu-focal-gcc: +ubuntu-20.04-x86_64-gcc: extends: .gcc-x86-64-build variables: - CONTAINER: ubuntu:focal + CONTAINER: ubuntu:20.04-x86_64 -ubuntu-focal-gcc-debug: +ubuntu-20.04-x86_64-gcc-debug: extends: .gcc-x86-64-build-debug variables: - CONTAINER: ubuntu:focal + CONTAINER: ubuntu:20.04-x86_64 -ubuntu-focal-clang: +ubuntu-20.04-x86_64-clang: extends: .clang-x86-64-build variables: - CONTAINER: ubuntu:focal + CONTAINER: ubuntu:20.04-x86_64 -ubuntu-focal-clang-debug: +ubuntu-20.04-x86_64-clang-debug: extends: .clang-x86-64-build-debug variables: - CONTAINER: ubuntu:focal + CONTAINER: ubuntu:20.04-x86_64 opensuse-leap-15.6-clang: extends: .clang-x86-64-build
automation/scripts/containerize +1 -1
@@ -40,7 +40,6 @@ case "_${CONTAINER}" in _archlinux|_arch) CONTAINER="${BASE}/archlinux:current" ;; _centos7) CONTAINER="${BASE}/centos:7" ;; _fedora) CONTAINER="${BASE}/fedora:29";; - _focal) CONTAINER="${BASE}/ubuntu:focal" ;; _bullseye-ppc64le) CONTAINER="${BASE}/debian:11-ppc64le" ;; _bookworm-ppc64le) CONTAINER="${BASE}/debian:12-ppc64le" ;; _bullseye-riscv64) CONTAINER="${BASE}/debian:11-riscv64" ;;
@@ -55,6 +54,7 @@ case "_${CONTAINER}" in _opensuse-tumbleweed|_tumbleweed) CONTAINER="${BASE}/opensuse:tumbleweed-x86_64" ;; _xenial) CONTAINER="${BASE}/ubuntu:16.04-x86_64" ;; _bionic) CONTAINER="${BASE}/ubuntu:18.04-x86_64" ;; + _focal) CONTAINER="${BASE}/ubuntu:20.04-x86_64" ;; *) guess_container ;; esac
Signature · 1 line
2.39.2
Anthony PERARD
Reviewed-byRe: [PATCH 17/12] CI: Refresh Ubuntu Focal container as 20.04-x86_64
Quoted 6 lines · Andrew Cooper
Exactly as per 18.04 (Bionic). This saves ~500M:
registry.gitlab.com/xen-project/xen/ubuntu 20.04-x86_64 1.06GB registry.gitlab.com/xen-project/xen/ubuntu focal 1.57GBSigned-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
[PATCH 18/12] CI: Add Ubuntu 22.04 (Jammy) and 24.04 (Noble) testing
In reply to Andrew CooperThe containers are exactly as per 20.04 (Focal). However, this now brings us to 5 releases * 4 build jobs worth of Ubuntu testing, which is overkill.
The oldest and newest toolchains are the most likely to find problems with new code, so reduce the middle 3 releases (18/20/22) to just a single smoke test each.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
CC: Shawn Anastasio <[email protected]>
Pipeline run showing all Ubuntu testing on staging:
https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1371106799
There are 11 jobs in total.
--- .../build/ubuntu/22.04-x86_64.dockerfile | 72 +++++++++++++++++++ .../build/ubuntu/24.04-x86_64.dockerfile | 72 +++++++++++++++++++ automation/gitlab-ci/build.yaml | 39 +++++----- automation/scripts/containerize | 2 + 4 files changed, 163 insertions(+), 22 deletions(-) create mode 100644 automation/build/ubuntu/22.04-x86_64.dockerfile create mode 100644 automation/build/ubuntu/24.04-x86_64.dockerfile
automation/build/ubuntu/22.04-x86_64.dockerfile new file +72 -0
@@ -0,0 +1,72 @@ +# syntax=docker/dockerfile:1 +FROM --platform=linux/amd64 ubuntu:22.04 +LABEL maintainer.name="The Xen Project" +LABEL maintainer.email="[email protected]" + +ENV DEBIAN_FRONTEND=noninteractive + +RUN <<EOF +#!/bin/bash + set -e + + useradd --create-home user + + apt-get -y update + + DEPS=( + # Xen + bison + build-essential + checkpolicy + clang + flex + python3-minimal + + # Tools (general) + ca-certificates + git-core + gzip + patch + perl + pkg-config + wget + # libxenguest dombuilder + libbz2-dev + libzstd-dev + liblzo2-dev + liblzma-dev + zlib1g-dev + # libacpi + acpica-tools + # libxl + uuid-dev + libyajl-dev + # RomBIOS + bcc + bin86 + # xentop + libncurses5-dev + # Python bindings + python3-dev + python3-setuptools + # Ocaml bindings/oxenstored + ocaml-nox + ocaml-findlib + + # Stubdom download/extract + bzip2 + + # Qemu build + libglib2.0-dev + libpixman-1-dev + meson + ninja-build + python3-packaging + ) + + apt-get -y --no-install-recommends install "${DEPS[@]}" + rm -rf /var/lib/apt/lists/* +EOF + +USER user +WORKDIR /build
automation/build/ubuntu/24.04-x86_64.dockerfile new file +72 -0
72 lines not on this page · raw message
automation/gitlab-ci/build.yaml +17 -22
@@ -608,45 +608,40 @@ ubuntu-16.04-x86_64-gcc-debug: variables: CONTAINER: ubuntu:16.04-x86_64 -ubuntu-18.04-x86_64-clang: - extends: .clang-x86-64-build - variables: - CONTAINER: ubuntu:18.04-x86_64 - -ubuntu-18.04-x86_64-clang-debug: - extends: .clang-x86-64-build-debug - variables: - CONTAINER: ubuntu:18.04-x86_64 - ubuntu-18.04-x86_64-gcc: extends: .gcc-x86-64-build variables: CONTAINER: ubuntu:18.04-x86_64 -ubuntu-18.04-x86_64-gcc-debug: - extends: .gcc-x86-64-build-debug - variables: - CONTAINER: ubuntu:18.04-x86_64 - ubuntu-20.04-x86_64-gcc: extends: .gcc-x86-64-build variables: CONTAINER: ubuntu:20.04-x86_64 -ubuntu-20.04-x86_64-gcc-debug: - extends: .gcc-x86-64-build-debug +ubuntu-22.04-x86_64-gcc: + extends: .gcc-x86-64-build variables: - CONTAINER: ubuntu:20.04-x86_64 + CONTAINER: ubuntu:22.04-x86_64 -ubuntu-20.04-x86_64-clang: +ubuntu-24.04-x86_64-clang: extends: .clang-x86-64-build variables: - CONTAINER: ubuntu:20.04-x86_64 + CONTAINER: ubuntu:24.04-x86_64 -ubuntu-20.04-x86_64-clang-debug: +ubuntu-24.04-x86_64-clang-debug: extends: .clang-x86-64-build-debug variables: - CONTAINER: ubuntu:20.04-x86_64 + CONTAINER: ubuntu:24.04-x86_64 + +ubuntu-24.04-x86_64-gcc: + extends: .gcc-x86-64-build + variables: + CONTAINER: ubuntu:24.04-x86_64 + +ubuntu-24.04-x86_64-gcc-debug: + extends: .gcc-x86-64-build-debug + variables: + CONTAINER: ubuntu:24.04-x86_64 opensuse-leap-15.6-clang: extends: .clang-x86-64-build
automation/scripts/containerize +2 -0
@@ -55,6 +55,8 @@ case "_${CONTAINER}" in _xenial) CONTAINER="${BASE}/ubuntu:16.04-x86_64" ;; _bionic) CONTAINER="${BASE}/ubuntu:18.04-x86_64" ;; _focal) CONTAINER="${BASE}/ubuntu:20.04-x86_64" ;; + _jammy) CONTAINER="${BASE}/ubuntu:22.04-x86_64" ;; + _noble) CONTAINER="${BASE}/ubuntu:24.04-x86_64" ;; *) guess_container ;; esac
base-commit: d46e4264c3fa68974fbb9b1bce514ea44af6d945
prerequisite-patch-id: b6875ad0bbd279067fdef31b9adf67e2a5abdf8a
prerequisite-patch-id: 5d4a01a02e7694939cdbf620a2e7509fee49dd92
prerequisite-patch-id: 65ecea93b1a320219deb8670264c3d464e00384e
prerequisite-patch-id: eb7042acce82d2bc324f8560fed3ffd57025d248
prerequisite-patch-id: 91c6663df2cfa139497e2d8b4dc17f1697f0c9f6
prerequisite-patch-id: 84c3345d580461986a3a9560735279baff21234f
prerequisite-patch-id: 5d732fc5f039fb25f7177279f63019f59acae4b1
prerequisite-patch-id: dd5006da276abc0e4da1ecb2c84d699c5bd98124
prerequisite-patch-id: 7e816655cd1066b42be5fa57a832c777c9867fc9
prerequisite-patch-id: fc25cefbee4cb69703df5c1f5b3eb317f03952af
prerequisite-patch-id: 531b3db898f54c05bff5fbcb69e0f7f62811c744
prerequisite-patch-id: ddb51b78aeaa2a6efe8035c39a128810c01aa822
prerequisite-patch-id: b4894b8120682429e1a74369c6b36bbbdc09d749
prerequisite-patch-id: c002530cb86a08b38980779a3d06cf491fdb9fae
Signature · 1 line
2.39.2
Anthony PERARD
Reviewed-byRe: [PATCH 18/12] CI: Add Ubuntu 22.04 (Jammy) and 24.04 (Noble) testing
Quoted 6 lines · Andrew Cooper
The containers are exactly as per 20.04 (Focal). However, this now brings us to 5 releases * 4 build jobs worth of Ubuntu testing, which is overkill.
The oldest and newest toolchains are the most likely to find problems with new code, so reduce the middle 3 releases (18/20/22) to just a single smoke test each.
That would mean a bit less clang build-test, but I guess that would be mostly covered by FreeBSD testing on GitHub.
I tried to find out which version of clang were been used with this patch applied:
Alpine clang version 16.0.6
Debian clang version 14.0.6
clang version 17.0.6
clang version 18.1.8
clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
Ubuntu clang version 18.1.3 (1ubuntu1)
and the versions that won't be tested anymore:
clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
clang version 10.0.0-4ubuntu1
Ubuntu clang version 14.0.0-1ubuntu1.1Quoted 1 line
Signed-off-by Andrew Cooper <[email protected]>
Anyway:
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 18/12] CI: Add Ubuntu 22.04 (Jammy) and 24.04 (Noble) testing
Quoted 23 lines · Anthony PERARD
Quoted 6 lines · Andrew Cooper
The containers are exactly as per 20.04 (Focal). However, this now brings us to 5 releases * 4 build jobs worth of Ubuntu testing, which is overkill.
The oldest and newest toolchains are the most likely to find problems with new code, so reduce the middle 3 releases (18/20/22) to just a single smoke test each.
That would mean a bit less
clangbuild-test, but I guess that would be mostly covered by FreeBSD testing on GitHub.I tried to find out which version of clang were been used with this patch applied:
Alpine clang version 16.0.6 Debian clang version 14.0.6 clang version 17.0.6 clang version 18.1.8 clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final) Ubuntu clang version 18.1.3 (1ubuntu1) and the versions that won't be tested anymore: clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final) clang version 10.0.0-4ubuntu1 Ubuntu clang version 14.0.0-1ubuntu1.1
Quoted 3 lines
Quoted 1 line
Signed-off-by Andrew Cooper <[email protected]>Anyway:
Reviewed-by Anthony PERARD <[email protected]>
Thanks.
We could intermix Clang/GCC if we really want, but tbh (still!) the only thing you want to use Clang for is stricter/better diagnostics. GCC is still a better optimising compiler for C, particularly for kernel activities.
Also, those clang versions are entirely obsolete. If we wanted to do sensible testing, we'd need to be using the nightly builds and keeping within ~18mo of the release which I think is their support window.
Also also, this testing is CC=Clang, LD=ld, which is subtly different to the FreeBSD testing which is full-stack LLVM (so LD=lld).
There is definitely room for improvement in our Clang/LLVM testing.
~Andrew
[PATCH 20/12] CI: Swap from perl to perl-base in build containers
In reply to Andrew CooperWe only need a basic perl environment, not the things that a primarily-perl project would want.
Discovered during the Fedora refresh where the difference is ~40M, but it's more modest for OpenSUSE and Ubuntu.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1372507213
--- automation/build/opensuse/leap-15.6-x86_64.dockerfile | 2 +- automation/build/opensuse/tumbleweed-x86_64.dockerfile | 2 +- automation/build/ubuntu/16.04-x86_64.dockerfile | 2 +- automation/build/ubuntu/18.04-x86_64.dockerfile | 2 +- automation/build/ubuntu/20.04-x86_64.dockerfile | 2 +- automation/build/ubuntu/22.04-x86_64.dockerfile | 2 +- automation/build/ubuntu/24.04-x86_64.dockerfile | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-)
automation/build/opensuse/leap-15.6-x86_64.dockerfile +1 -1
@@ -31,7 +31,7 @@ RUN <<EOF git-core gzip patch - perl + perl-base pkg-config wget # libxenguest dombuilder
automation/build/opensuse/tumbleweed-x86_64.dockerfile +1 -1
@@ -31,7 +31,7 @@ RUN <<EOF git-core gzip patch - perl + perl-base pkg-config wget # libxenguest dombuilder
automation/build/ubuntu/16.04-x86_64.dockerfile +1 -1
@@ -27,7 +27,7 @@ RUN <<EOF git-core gzip patch - perl + perl-base pkg-config wget # libxenguest dombuilder
automation/build/ubuntu/18.04-x86_64.dockerfile +1 -1
@@ -27,7 +27,7 @@ RUN <<EOF git-core gzip patch - perl + perl-base pkg-config wget # libxenguest dombuilder
automation/build/ubuntu/20.04-x86_64.dockerfile +1 -1
@@ -27,7 +27,7 @@ RUN <<EOF git-core gzip patch - perl + perl-base pkg-config wget # libxenguest dombuilder
automation/build/ubuntu/22.04-x86_64.dockerfile +1 -1
@@ -27,7 +27,7 @@ RUN <<EOF git-core gzip patch - perl + perl-base pkg-config wget # libxenguest dombuilder
automation/build/ubuntu/24.04-x86_64.dockerfile +1 -1
@@ -27,7 +27,7 @@ RUN <<EOF git-core gzip patch - perl + perl-base pkg-config wget # libxenguest dombuilder
Signature · 1 line
2.39.2
Re: [PATCH 20/12] CI: Swap from perl to perl-base in build containers
Quoted 5 lines · Andrew Cooper
We only need a basic perl environment, not the things that a primarily-perl project would want.
Discovered during the Fedora refresh where the difference is ~40M, but it's more modest for OpenSUSE and Ubuntu.
"perl-base" on Ubuntu is quite different from "perl-base" on OpenSUSE Leap. I haven't check Tumbleweed but is probably the same as Leap. So, I think these would deserve to be in separated patches.
On Ubuntu, installing "perl" or "perl-base" or none of them makes no difference. We install "build-essentials" which pulls "dpkg-dev" which pulls "perl", so we don't test properly if "perl-base" is enough or if we need more.
I tried to find out which Perl module we would need, and I've got this list:
- get_maintainer
Getopt::Long;
- add_maintainer
Getopt::Long;
File::Basename;
List::MoreUtils;
IO::Handle;
- stubdom/vtpmmgr:
Digest::SHA qw(sha1);
Math::BigInt only => 'GMP';
- kconfig
Getopt::Long;
- ocaml/lib/xc/abi-check
Data::Dumper;
- docs/gen-html-index
Getopt::Long;
IO::File;
File::Basename;
- docs/xen-headers
Getopt::Long;
File::Find;
IO::File;(there's also tools/examples/xeninfo.pl, I dont't if that can still work or if we need to purge it)
Then I've got whatever module is in "perl-base" pkg.
For Leap's perl-base:
getopt:long
file:basename
io:handle
io:file
file:find
data:dumper
digest:sha
So, add_maintainer.pl and the script in stubdom/vtpmmgr won't work. So that's probably fine for Leap.
For Ubuntu 22.04 (not check others, but likely about the same) there's more packages:
perl-base
getopt::long
file:basename
io:handle
io:file
perl-modules-5.34
file:find
math:bigint
libperl5.34
data:dumper
digest:sha
liblist-moreutils-perl
list:MoreUtils
librpc-xml-perl
rpc:xml("perl" pkg pulls "perl-modules-5.34" and "libperl5.34")
So with just "perl-base", the ocaml's abi-check wouldn't work, as well as the docs's script (probably ok for "docs" if we don't use containers to build them). Also add_maintainer and stubdom/vtpmmgr like for Leap.
So I would suggest to not touch the Ubuntu containers, make the change to the OpenSUSE one and maybe add the limitation to the commit message.
Cheers,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 20/12] CI: Swap from perl to perl-base in build containers
Quoted 8 lines · Anthony PERARD
Quoted 5 lines · Andrew Cooper
We only need a basic perl environment, not the things that a primarily-perl project would want.
Discovered during the Fedora refresh where the difference is ~40M, but it's more modest for OpenSUSE and Ubuntu.
"perl-base" on Ubuntu is quite different from "perl-base" on OpenSUSE Leap. I haven't check Tumbleweed but is probably the same as Leap.
/sigh - because that would have been too easy.
Quoted 6 lines
So, I think these would deserve to be in separated patches.
On Ubuntu, installing "perl" or "perl-base" or none of them makes no difference. We install "build-essentials" which pulls "dpkg-dev" which pulls "perl", so we don't test properly if "perl-base" is enough or if we need more.
Fine - lets leave the Ubuntu's alone.
Quoted 26 lines
I tried to find out which Perl module we would need, and I've got this list:
-
get_maintainerGetopt::Long; - add_maintainer Getopt::Long; File::Basename; List::MoreUtils; IO::Handle; - stubdom/vtpmmgr: Digest::SHA qw(sha1); Math::BigInt only => 'GMP'; - kconfig Getopt::Long; - ocaml/lib/xc/abi-check Data::Dumper; - docs/gen-html-index Getopt::Long; IO::File; File::Basename; - docs/xen-headers Getopt::Long; File::Find; IO::File;
Quoted 2 lines
(there's also
tools/examples/xeninfo.pl, I dont't if that can still work or if we need to purge it)
Ewww.
my %xenhosts = ("192.0.2.10" => {"port" => "9363"}, ...
is enough to demonstrate it's a special purpose script.
That file isn't referenced by the Makefile, nor is anything in the vnc/ dir, so we should purge the lot.
Quoted 13 lines
Then I've got whatever module is in "perl-base" pkg.
For Leap's perl-base:
getopt:long
file:basename
io:handle
io:file
file:find
data:dumper
digest:sha
So, add_maintainer.pl and the script in stubdom/vtpmmgr won't work. So that's probably fine for Leap.
vtpmmgr isn't built by default, so gets 0 testing in CI. I don't expect people to be using add_maintainer inside the containers either.
Quoted 18 lines
For Ubuntu 22.04 (not check others, but likely about the same) there's more packages:
perl-base getopt::long file:basename io:handle io:file perl-modules-5.34 file:find math:bigint libperl5.34 data:dumper digest:sha liblist-moreutils-perl list:MoreUtils librpc-xml-perl rpc:xml
Quoted 5 lines
("perl" pkg pulls "perl-modules-5.34" and "libperl5.34")
So with just "perl-base", the ocaml's abi-check wouldn't work, as well as the docs's script (probably ok for "docs" if we don't use containers to build them).
Hmm yeah - we wouldn't want to break that.
Quoted 5 lines
Also
add_maintainerand stubdom/vtpmmgr like for Leap.So I would suggest to not touch the Ubuntu containers, make the change to the OpenSUSE one and maybe add the limitation to the commit message.
Ok - I'll do this.
~Andrew
[PATCH 19/12] docs: Fix install-man$(1)-pages if no manpages are generated
In reply to Andrew CooperAll tools to build manpages are optional, and if none of them happen to be present, the intermediate working directory may not even be created.
Treat this as non-fatal, bringing the behaviour in line with install-html. Like the html side, it needs to be not-or to avoid Make thinking the rule has failed.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
--- docs/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
docs/Makefile +1 -1
@@ -151,7 +151,7 @@ man$(1)-pages: $$(DOC_MAN$(1)) .PHONY: install-man$(1)-pages install-man$(1)-pages: man$(1)-pages $(INSTALL_DIR) $(DESTDIR)$(mandir) - cp -r man$(1) $(DESTDIR)$(mandir) + [ ! -d man$(1) ] || cp -r man$(1) $(DESTDIR)$(mandir) # Clean .PHONY: clean-man$(1)-pages
Signature · 1 line
2.39.2
Anthony PERARD
Reviewed-byRe: [PATCH 19/12] docs: Fix install-man$(1)-pages if no manpages are generated
Quoted 8 lines · Andrew Cooper
All tools to build manpages are optional, and if none of them happen to be present, the intermediate working directory may not even be created.
Treat this as non-fatal, bringing the behaviour in line with install-html. Like the html side, it needs to be not-or to avoid Make thinking the rule has failed.
Signed-off-by Andrew Cooper <[email protected]>
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
[PATCH 21/12] CI: Refresh and upgrade the Fedora container
In reply to Andrew CooperFedora 29 is long out of date. Move forward 5 years to Fedora 39.
Inlcude all the usual improvements. Rework the container to be non-root, use heredocs for legibility, and switch to the new naming scheme.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
I know Fedora 40 the latest release, but it suffers from the same stubdom failure as Archlinux currently does. Fedora 39 builds fine.
Fedora is (correctly) blocking, so adding 40 would be bad move at this point in the 4.19 release.
https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1372042761 is a run with both 39 and 40, showing the failure.
--- automation/build/fedora/29.dockerfile | 46 ------------ automation/build/fedora/39-x86_64.dockerfile | 74 ++++++++++++++++++++ automation/gitlab-ci/build.yaml | 8 +-- 3 files changed, 78 insertions(+), 50 deletions(-) delete mode 100644 automation/build/fedora/29.dockerfile create mode 100644 automation/build/fedora/39-x86_64.dockerfile
automation/build/fedora/29.dockerfile deleted +0 -46
@@ -1,46 +0,0 @@ -# syntax=docker/dockerfile:1 -FROM --platform=linux/amd64 fedora:29 -LABEL maintainer.name="The Xen Project" \ - maintainer.email="[email protected]" - -# install Xen depends -RUN dnf -y install \ - clang \ - gcc \ - gcc-c++ \ - ncurses-devel \ - zlib-devel \ - python-devel \ - python3-devel \ - libuuid-devel \ - pkgconfig \ - flex \ - bison \ - libaio-devel \ - glib2-devel \ - yajl-devel \ - pixman-devel \ - glibc-devel \ - make \ - binutils \ - git \ - wget \ - acpica-tools \ - python-markdown \ - patch \ - checkpolicy \ - dev86 \ - xz-devel \ - bzip2 \ - nasm \ - ocaml \ - ocaml-findlib \ - golang \ - # QEMU - ninja-build \ - && dnf clean all && \ - rm -rf /var/cache/dnf - -RUN useradd --create-home user -USER user -WORKDIR /build
automation/build/fedora/39-x86_64.dockerfile new file +74 -0
74 lines not on this page · raw message
automation/gitlab-ci/build.yaml +4 -4
@@ -575,15 +575,15 @@ debian-bookworm-32-gcc-debug: variables: CONTAINER: debian:bookworm-i386 -fedora-gcc: +fedora-39-x86_64-gcc: extends: .gcc-x86-64-build variables: - CONTAINER: fedora:29 + CONTAINER: fedora:39-x86_64 -fedora-gcc-debug: +fedora-39-x86_64-gcc-debug: extends: .gcc-x86-64-build-debug variables: - CONTAINER: fedora:29 + CONTAINER: fedora:39-x86_64 ubuntu-16.04-x86_64-clang: extends: .clang-x86-64-build
Signature · 1 line
2.39.2
Re: [PATCH 21/12] CI: Refresh and upgrade the Fedora container
Quoted 4 lines · Andrew Cooper
Fedora 29 is long out of date. Move forward 5 years to Fedora 39.
Inlcude all the usual improvements. Rework the container to be non-root, use heredocs for legibility, and switch to the new naming scheme.
"clang" as been removed, but it's not mention in the commit message. The other changes are kind of "the usual improvement" yes, as been done to other containers (libaio, nasm, ...).
The patch is missing an update of the "containerize" script.
Quoted 5 lines
automation/build/fedora/39-x86_64.dockerfile new file +0
-0
...
automation/build/fedora/39-x86_64.dockerfile
Quoted 2 lines
automation/build/fedora/39-x86_64.dockerfile new file +2
-0+ + dnf -y install "${DEPS[@]}"
You might want to add --setopt=install_weak_deps=False to avoid installing "git" for example.
When running the original command by hand, dnf want to install those weak deps:
Installing weak dependencies:
apr-util-bdb
apr-util-openssl
cryptsetup-libs
diffutils
git
libbpf
libxkbcommon
mercurial
perl-NDBM_File
python3-fb-re2
python3-pip
qrencode-libs
subversion
systemd-networkd
systemd-resolved
And comparring the list of deps, these extra pkgs are installed because of weak deps:
apr
apr-util
dbus
dbus-broker
dbus-common
device-mapper
device-mapper-libs
git-core-doc
kmod-libs
libargon2
libseccomp
libserf
perl-Error
perl-File-Find
perl-Git
perl-TermReadKey
perl-lib
python3-zombie-imp
re2
subversion-libs
systemd
systemd-pam
utf8proc
xkeyboard-configSo, probably only "perl-File-Find" (can be written "perl(File::Find)" I think as well) might be needed, but only for docs/ I think, from the previous email.
It seems I go from 1.26GB to 1.18GB without those weak deps.
Cheers,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 21/12] CI: Refresh and upgrade the Fedora container
Quoted 6 lines · Anthony PERARD
Quoted 4 lines · Andrew Cooper
Fedora 29 is long out of date. Move forward 5 years to Fedora 39.
Inlcude all the usual improvements. Rework the container to be non-root, use heredocs for legibility, and switch to the new naming scheme.
"clang" as been removed, but it's not mention in the commit message.
Oops - I meant to mention that. We don't do any clang builds with this container, so I though I'd keep it slimer until we change that.
Quoted 5 lines
The
other changes are kind of "the usual improvement" yes, as been done to other containers (libaio, nasm, ...).The patch is missing an update of the "containerize" script.
Ah - I found the hunk in a different patch. I'll fix.
Quoted 11 lines
Quoted 6 lines
automation/build/fedora/39-x86_64.dockerfile new file +0
-0...
automation/build/fedora/39-x86_64.dockerfile
Quoted 2 lines
automation/build/fedora/39-x86_64.dockerfile new file +2
-0+ + dnf -y install "${DEPS[@]}"You might want to add --setopt=install_weak_deps=False to avoid installing "git" for example.
Well, git-core is installed explicitly, but point taken.
Quoted 44 lines
When running the original command by hand,
dnfwant to install those weak deps:
Installing weak dependencies:
apr-util-bdb
apr-util-openssl
cryptsetup-libs
diffutils
git
libbpf
libxkbcommon
mercurial
perl-NDBM_File
python3-fb-re2
python3-pip
qrencode-libs
subversion
systemd-networkd
systemd-resolved
And comparring the list of deps, these extra pkgs are installed because of weak deps:apr apr-util dbus dbus-broker dbus-common device-mapper device-mapper-libs git-core-doc kmod-libs libargon2 libseccomp libserf perl-Error perl-File-Find perl-Git perl-TermReadKey perl-lib python3-zombie-imp re2 subversion-libs systemd systemd-pam utf8proc xkeyboard-config
Quoted 5 lines
So, probably only "perl-File-Find" (can be written "perl(File::Find)" I think as well) might be needed, but only for docs/ I think, from the previous email.
It seems I go from 1.26GB to 1.18GB without those weak deps.
Thanks. I'll rework.
I have a specific plan for testing docs and it's going to be a dedicated container and job, as well as things being a bit more -Werror-like.
~Andrew
[PATCH 22/12] tools/examples: Remove more obsolete content
In reply to Andrew Cooperxeninfo.pl was introduced in commit 1b0a8bb57e3e ("Added xeninfo.pl, a script for collecting statistics from Xen hosts using the Xen-API") and has been touched exactly twice since to remove hardcoded IP addresses and paths.
The configuration files in vnc/* date from when we had a vendered version of Qemu living in the tree.
These have never (AFAICT) been wired into the make install rule.
---
CC: Anthony PERARD <[email protected]>
CC: Juergen Gross <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jan Beulich <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Julien Grall <[email protected]>
CC: Oleksii Kurochko <[email protected]>
--- tools/examples/vnc/Xservers | 5 - tools/examples/vnc/Xvnc-xen | 53 ------- tools/examples/xeninfo.pl | 284 ------------------------------------ 3 files changed, 342 deletions(-) delete mode 100644 tools/examples/vnc/Xservers delete mode 100755 tools/examples/vnc/Xvnc-xen delete mode 100644 tools/examples/xeninfo.pl
tools/examples/vnc/Xservers deleted +0 -5
@@ -1,5 +0,0 @@ -# Configuration lines to go in /etc/X11/xdm/Xservers to -# start Xvnc and connect back to a vncviewer in domain-0. -# See 'man xdm' under 'LOCAL SERVER SPECIFICATION' for format details. - -:1 Xvnc local /usr/X11R6/bin/Xvnc-xen :1
tools/examples/vnc/Xvnc-xen deleted +0 -53
53 lines not on this page · raw message
tools/examples/xeninfo.pl deleted +0 -284
284 lines not on this page · raw message
Signature · 1 line
2.39.2
Anthony PERARD
Reviewed-byRe: [PATCH 22/12] tools/examples: Remove more obsolete content
Quoted 3 lines · Andrew Cooper
xeninfo.pl was introduced in commit
1b0a8bb57e3e("Added xeninfo.pl, a script for collecting statistics from Xen hosts using the Xen-API") and has been touched exactly twice since to remove hardcoded IP addresses and paths.
Xen-API, is that xapi?
Quoted 2 lines
The configuration files in vnc/* date from when we had a vendered version of Qemu living in the tree.
I guess QEMU at that time didn't have VNC support? Because looks like the vnc config was supposed to be put in a guest, and xm had support for it.
Quoted 3 lines
These have never (AFAICT) been wired into the
make installrule.Signed-off-by Andrew Cooper <[email protected]>
Anyway, patch looks good:
Thanks,
Signature · 5 lines
Anthony Perard | Vates XCP-ng Developer XCP-ng & Xen Orchestra - Vates solutions web: https://vates.tech
Re: [PATCH 22/12] tools/examples: Remove more obsolete content
Quoted 5 lines · Anthony PERARD
Quoted 3 lines · Andrew Cooper
xeninfo.pl was introduced in commit
1b0a8bb57e3e("Added xeninfo.pl, a script for collecting statistics from Xen hosts using the Xen-API") and has been touched exactly twice since to remove hardcoded IP addresses and paths.Xen-API, is that xapi?
Yes. xapi is a contraction of Xen-API, but even the docs intermix the two names. https://xapi-project.github.io/xen-api/
Quoted 6 lines
Quoted 3 lines
The configuration files in vnc/* date from when we had a vendered version of Qemu living in the tree.
I guess QEMU at that time didn't have VNC support? Because looks like the vnc config was supposed to be put in a guest, and
xmhad support for it.
Thinking about it, this may have been for PV guests instead. We used to have a mode in XenServer to inject vnc configuration into a PV guest in order to get a GUI installer rather than the TUI one.
Either way, xl doesn't have --vnc, although it does have -V which seems to have subsumed this functionality.
Quoted 6 lines
Quoted 4 lines
These have never (AFAICT) been wired into the
make installrule.Signed-off-by Andrew Cooper <[email protected]>Anyway, patch looks good:
Reviewed-by Anthony PERARD <[email protected]>
Thanks,
~Andrew