CI: Fixes/cleanup in preparation for RISCV
cover CI: Fixes/cleanup in preparation for RISCV
This series is intended to avoid the need for any editing of automation/scripts/build in the main RISCV series.
https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/734584203
https://gitlab.com/xen-project/people/andyhhp/xen/-/commits/auto-build
The two Ubuntu Bionic failures are known and preexisting.
Andrew Cooper (6):
CI: Drop automation/configs/
CI: Remove guesswork about which artefacts to preserve
CI: Only calculate ./configure args if needed
CI: Express HYPERVISOR_ONLY in build.yml
CI: Fix build script when CROSS_COMPILE is in use
CI: Simplify the MUSL check
automation/configs/x86/hvm_only_config | 3 - automation/configs/x86/no_hvm_pv_config | 3 - automation/configs/x86/pv_only_config | 3 - automation/gitlab-ci/build.yaml | 2 + automation/scripts/build | 127 ++++++++++++++------------------ 5 files changed, 59 insertions(+), 79 deletions(-) delete mode 100644 automation/configs/x86/hvm_only_config delete mode 100644 automation/configs/x86/no_hvm_pv_config delete mode 100644 automation/configs/x86/pv_only_config
Signature · 1 line
2.11.0
PATCH 1/6 CI: Drop automation/configs/
Having 3 extra hypervisor builds on the end of a full build is deeply confusing to debug if one of them fails, because the .config file presented in the artefacts is not the one which caused a build failure. Also, the log tends to be truncated in the UI.
PV-only is tested as part of PV-Shim in a full build anyway, so doesn't need repeating. HVM-only and neither will come up frequently in randconfig, so drop all the logic here to simplify things.
---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>
--- automation/configs/x86/hvm_only_config | 3 --- automation/configs/x86/no_hvm_pv_config | 3 --- automation/configs/x86/pv_only_config | 3 --- automation/scripts/build | 21 --------------------- 4 files changed, 30 deletions(-) delete mode 100644 automation/configs/x86/hvm_only_config delete mode 100644 automation/configs/x86/no_hvm_pv_config delete mode 100644 automation/configs/x86/pv_only_config
automation/configs/x86/hvm_only_config deleted +0 -3
@@ -1,3 +0,0 @@ -CONFIG_HVM=y -# CONFIG_PV is not set -# CONFIG_DEBUG is not set
automation/configs/x86/no_hvm_pv_config deleted +0 -3
@@ -1,3 +0,0 @@ -# CONFIG_HVM is not set -# CONFIG_PV is not set -# CONFIG_DEBUG is not set
automation/configs/x86/pv_only_config deleted +0 -3
@@ -1,3 +0,0 @@ -CONFIG_PV=y -# CONFIG_HVM is not set -# CONFIG_DEBUG is not set
automation/scripts/build +0 -21
@@ -85,24 +85,3 @@ if [[ "${XEN_TARGET_ARCH}" != "x86_32" ]]; then cp -r dist binaries/ fi fi - -if [[ "${hypervisor_only}" == "y" ]]; then - # If we are build testing a specific Kconfig exit now, there's no point in - # testing all the possible configs. - exit 0 -fi - -# Build all the configs we care about -case ${XEN_TARGET_ARCH} in - x86_64) arch=x86 ;; - *) exit 0 ;; -esac - -cfg_dir="automation/configs/${arch}" -for cfg in `ls ${cfg_dir}`; do - echo "Building $cfg" - make -j$(nproc) -C xen clean - rm -f xen/.config - make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig - make -j$(nproc) -C xen -done
Signature · 1 line
2.11.0
Re: [PATCH 1/6] CI: Drop automation/configs/
Hi Andrew,
Quoted 8 lines · Andrew Cooper
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Having 3 extra hypervisor builds on the end of a full build is deeply confusing to debug if one of them fails, because the .config file presented in the artefacts is not the one which caused a build failure. Also, the log tends to be truncated in the UI.
Quoted 5 lines
PV-only is tested as part of PV-Shim in a full build anyway, so doesn't need repeating. HVM-only and neither will come up frequently in randconfig, so drop all the logic here to simplify things.
Signed-off-by Andrew Cooper <[email protected]>
~Michal
Re: [PATCH 1/6] CI: Drop automation/configs/
In reply to Andrew CooperQuoted 10 lines · Andrew Cooper
Having 3 extra hypervisor builds on the end of a full build is deeply confusing to debug if one of them fails, because the .config file presented in the artefacts is not the one which caused a build failure. Also, the log tends to be truncated in the UI.
PV-only is tested as part of PV-Shim in a full build anyway, so doesn't need repeating. HVM-only and neither will come up frequently in randconfig, so drop all the logic here to simplify things.
Signed-off-by Andrew Cooper <[email protected]>
Quoted 75 lines
---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/configs/x86/hvm_only_config | 3 --- automation/configs/x86/no_hvm_pv_config | 3 --- automation/configs/x86/pv_only_config | 3 --- automation/scripts/build | 21 --------------------- 4 files changed, 30 deletions(-) delete mode 100644 automation/configs/x86/hvm_only_config delete mode 100644 automation/configs/x86/no_hvm_pv_config delete mode 100644 automation/configs/x86/pv_only_configautomation/configs/x86/hvm_only_config deleted +0
-3@@ -1,3 +0,0 @@ -CONFIG_HVM=y -# CONFIG_PV is not set -# CONFIG_DEBUG is not setautomation/configs/x86/no_hvm_pv_config deleted +0
-3@@ -1,3 +0,0 @@ -# CONFIG_HVM is not set -# CONFIG_PV is not set -# CONFIG_DEBUG is not setautomation/configs/x86/pv_only_config deleted +0
-3@@ -1,3 +0,0 @@ -CONFIG_PV=y -# CONFIG_HVM is not set -# CONFIG_DEBUG is not setautomation/scripts/build +0
-21@@ -85,24 +85,3 @@ if [[ "${XEN_TARGET_ARCH}" != "x86_32" ]]; then cp -r dist binaries/ fi fi - -if [[ "${hypervisor_only}" == "y" ]]; then - # If we are build testing a specific Kconfig exit now, there's no point in - # testing all the possible configs. - exit 0 -fi - -# Build all the configs we care about -case ${XEN_TARGET_ARCH} in - x86_64) arch=x86 ;; - *) exit 0 ;; -esac - -cfg_dir="automation/configs/${arch}" -for cfg in `ls ${cfg_dir}`; do - echo "Building $cfg" - make -j$(nproc) -C xen clean - rm -f xen/.config - make -C xen KBUILD_DEFCONFIG=../../../../${cfg_dir}/${cfg} defconfig - make -j$(nproc) -C xen -doneSignature · 1 line
2.11.0
PATCH 2/6 CI: Remove guesswork about which artefacts to preserve
In reply to Andrew CooperPreserve the artefacts based on the make rune we actually ran, rather than guesswork about which rune we would have run based on other settings.
Note that the ARM qemu smoke tests depend on finding binaries/xen even from full builds. Also that the Jessie-32 containers build tools but not Xen.
This means the x86_32 builds now store relevant artefacts. No change in other configurations.
---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>
--- automation/scripts/build | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-)
automation/scripts/build +14 -8
@@ -70,18 +70,24 @@ if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then cfgargs+=("--with-system-seabios=/bin/false") fi +# Directory for the artefacts to be dumped into +mkdir binaries + if [[ "${hypervisor_only}" == "y" ]]; then + # Xen-only build make -j$(nproc) xen + + # Preserve artefacts + cp xen/xen binaries/xen else + # Full build ./configure "${cfgargs[@]}" make -j$(nproc) dist -fi -# Extract artifacts to avoid getting rewritten by customised builds -mkdir binaries -if [[ "${XEN_TARGET_ARCH}" != "x86_32" ]]; then - cp xen/xen binaries/xen - if [[ "${hypervisor_only}" != "y" ]]; then - cp -r dist binaries/ - fi + # Preserve artefacts + # Note: Some smoke tests depending on finding binaries/xen on a full build + # even though dist/ contains everything, while some containers don't even + # build Xen + cp -r dist binaries/ + if [[ -f xen/xen ]] ; then cp xen/xen binaries/xen; fi fi
Signature · 1 line
2.11.0
Re: [PATCH 2/6] CI: Remove guesswork about which artefacts to preserve
Hi Andrew,
Quoted 6 lines · Andrew Cooper
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Preserve the artefacts based on the
makerune we actually ran, rather than guesswork about which rune we would have run based on other settings.
Quoted 7 lines
Note that the ARM qemu smoke tests depend on finding binaries/xen even from full builds. Also that the Jessie-32 containers build tools but not Xen.
This means the
x86_32builds now store relevant artefacts. No change in other configurations.Signed-off-by Andrew Cooper <[email protected]>
I'd prefer to keep using "artifacts" and not "artefacts" as the former is what GitLab uses and what we use in build/test.yaml.
Apart from that:
~Michal
Re: [PATCH 2/6] CI: Remove guesswork about which artefacts to preserve
Quoted 12 lines · Michal Orzel
Hi Andrew,
Quoted 9 lines · Andrew Cooper
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Preserve the artefacts based on the
makerune we actually ran, rather than guesswork about which rune we would have run based on other settings.Note that the ARM qemu smoke tests depend on finding binaries/xen even from full builds. Also that the Jessie-32 containers build tools but not Xen.
Quoted 6 lines
Quoted 4 lines
This means the
x86_32builds now store relevant artefacts. No change in other configurations.Signed-off-by Andrew Cooper <[email protected]>I'd prefer to keep using "artifacts" and not "artefacts" as the former is what GitLab uses and what we use in
build/test.yaml.
Xen is written in British English. We're forced to compromise for external dependencies, but xen.git is mostly British not American.
~Andrew
Re: [PATCH 2/6] CI: Remove guesswork about which artefacts to preserve
Quoted 18 lines · Andrew Cooper
Quoted 15 lines · Michal Orzel
Hi Andrew,
Quoted 12 lines · Andrew Cooper
Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.
Preserve the artefacts based on the
makerune we actually ran, rather than guesswork about which rune we would have run based on other settings.Note that the ARM qemu smoke tests depend on finding binaries/xen even from full builds. Also that the Jessie-32 containers build tools but not Xen.
This means the
x86_32builds now store relevant artefacts. No change in other configurations.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Signed-off-by Andrew Cooper <[email protected]>I'd prefer to keep using "artifacts" and not "artefacts" as the former is what GitLab uses and what we use in
build/test.yaml.Xen is written in British English. We're forced to compromise for external dependencies, but xen.git is mostly British not American.
True, but from the consistency perspective and easy grepping, it is beneficial to stick to what a subsystem uses by default.
Quoted 2 lines
~Andrew
~Michal
Re: [PATCH 2/6] CI: Remove guesswork about which artefacts to preserve
In reply to Andrew CooperQuoted 46 lines · Andrew Cooper
Preserve the artefacts based on the
makerune we actually ran, rather than guesswork about which rune we would have run based on other settings.Note that the ARM qemu smoke tests depend on finding binaries/xen even from full builds. Also that the Jessie-32 containers build tools but not Xen.
This means the
x86_32builds now store relevant artefacts. No change in other configurations.Signed-off-by Andrew Cooper <[email protected]>---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/scripts/build | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-)automation/scripts/build +8
-4@@ -70,18 +70,24 @@ if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then cfgargs+=("--with-system-seabios=/bin/false") fi +# Directory for the artefacts to be dumped into +mkdir binaries + if [[ "${hypervisor_only}" == "y" ]]; then + # Xen-only build make -j$(nproc) xen + + # Preserve artefacts + cp xen/xen binaries/xen else + # Full build ./configure "${cfgargs[@]}" make -j$(nproc) dist -fi -# Extract artifacts to avoid getting rewritten by customised builds -mkdir binaries -if [[ "${XEN_TARGET_ARCH}" != "x86_32" ]]; then
automation/scripts/build if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then · line 92
Quoted 10 lines
automation/scripts/build +6
-4- cp xen/xen binaries/xen - if [[ "${hypervisor_only}" != "y" ]]; then - cp -r dist binaries/ - fi + # Preserve artefacts + # Note: Some smoke tests depending on finding binaries/xen on a full build + # even though dist/ contains everything, while some containers don't even + # build Xen + cp -r dist binaries/ + if [[ -f xen/xen ]] ; then cp xen/xen binaries/xen; fi
why the "if" ?
You could just:
cp xen/xen binaries/xen
unconditionally?
If you are OK with this change you can do it on commit
Re: [PATCH 2/6] CI: Remove guesswork about which artefacts to preserve
Quoted 59 lines · Stefano Stabellini
Quoted 46 lines · Andrew Cooper
Preserve the artefacts based on the
makerune we actually ran, rather than guesswork about which rune we would have run based on other settings.Note that the ARM qemu smoke tests depend on finding binaries/xen even from full builds. Also that the Jessie-32 containers build tools but not Xen.
This means the
x86_32builds now store relevant artefacts. No change in other configurations.Signed-off-by Andrew Cooper <[email protected]>---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/scripts/build | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-)automation/scripts/build +8
-4@@ -70,18 +70,24 @@ if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then cfgargs+=("--with-system-seabios=/bin/false") fi +# Directory for the artefacts to be dumped into +mkdir binaries + if [[ "${hypervisor_only}" == "y" ]]; then + # Xen-only build make -j$(nproc) xen + + # Preserve artefacts + cp xen/xen binaries/xen else + # Full build ./configure "${cfgargs[@]}" make -j$(nproc) dist -fi -# Extract artifacts to avoid getting rewritten by customised builds -mkdir binaries -if [[ "${XEN_TARGET_ARCH}" != "x86_32" ]]; thenautomation/scripts/build if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then · line 92
Quoted 10 lines
automation/scripts/build +6
-4- cp xen/xen binaries/xen - if [[ "${hypervisor_only}" != "y" ]]; then - cp -r dist binaries/ - fi + # Preserve artefacts + # Note: Some smoke tests depending on finding binaries/xen on a full build + # even though dist/ contains everything, while some containers don't even + # build Xen + cp -r dist binaries/ + if [[ -f xen/xen ]] ; then cp xen/xen binaries/xen; fiwhy the "if" ?
Quoted 5 lines
You could just:
cp xen/xen binaries/xen
unconditionally?
No - the script is set -e and xen/xen doesn't exist in the Jessie32 containers.
That's why the old logic had an "if not x86_32" guard (except it also guarded the regular dist -> binaries/ copy which is problematic).
At a guess, the other working option would be cp xen/xen binaries/xen || :
I don't much care which of these two gets used, but pretty much anything else results in a failed test.
~Andrew
Re: [PATCH 2/6] CI: Remove guesswork about which artefacts to preserve
Quoted 69 lines · Andrew Cooper
Quoted 59 lines · Stefano Stabellini
Quoted 46 lines · Andrew Cooper
Preserve the artefacts based on the
makerune we actually ran, rather than guesswork about which rune we would have run based on other settings.Note that the ARM qemu smoke tests depend on finding binaries/xen even from full builds. Also that the Jessie-32 containers build tools but not Xen.
This means the
x86_32builds now store relevant artefacts. No change in other configurations.Signed-off-by Andrew Cooper <[email protected]>---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/scripts/build | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-)automation/scripts/build +8
-4@@ -70,18 +70,24 @@ if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then cfgargs+=("--with-system-seabios=/bin/false") fi +# Directory for the artefacts to be dumped into +mkdir binaries + if [[ "${hypervisor_only}" == "y" ]]; then + # Xen-only build make -j$(nproc) xen + + # Preserve artefacts + cp xen/xen binaries/xen else + # Full build ./configure "${cfgargs[@]}" make -j$(nproc) dist -fi -# Extract artifacts to avoid getting rewritten by customised builds -mkdir binaries -if [[ "${XEN_TARGET_ARCH}" != "x86_32" ]]; thenautomation/scripts/build if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then · line 92
Quoted 10 lines
automation/scripts/build +6
-4- cp xen/xen binaries/xen - if [[ "${hypervisor_only}" != "y" ]]; then - cp -r dist binaries/ - fi + # Preserve artefacts + # Note: Some smoke tests depending on finding binaries/xen on a full build + # even though dist/ contains everything, while some containers don't even + # build Xen + cp -r dist binaries/ + if [[ -f xen/xen ]] ; then cp xen/xen binaries/xen; fiwhy the "if" ?
Quoted 5 lines
You could just:
cp xen/xen binaries/xen
unconditionally?
No - the script is
set -eand xen/xen doesn't exist in the Jessie32 containers.
Quoted 7 lines
That's why the old logic had an "if not
x86_32" guard (except it also guarded the regular dist -> binaries/ copy which is problematic).At a guess, the other working option would be
cp xen/xen binaries/xen || :I don't much care which of these two gets used, but pretty much anything else results in a failed test.
I didn't realize that. I think your version is this patch is better, keep it as is
PATCH 3/6 CI: Only calculate ./configure args if needed
In reply to Andrew CooperThis is purely code motion of the cfgargs construction, into the case where it is used.
No practical change.
---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>
--- automation/scripts/build | 63 ++++++++++++++++++++++++------------------------ 1 file changed, 31 insertions(+), 32 deletions(-)
automation/scripts/build +31 -32
@@ -39,37 +39,6 @@ if [[ "${XEN_TARGET_ARCH}" = "arm32" ]]; then hypervisor_only="y" fi -# build up our configure options -cfgargs=() -cfgargs+=("--enable-docs") - -if [[ "${CC}" == "clang"* ]]; then - # SeaBIOS cannot be built with clang - cfgargs+=("--with-system-seabios=/usr/share/seabios/bios.bin") - # iPXE cannot be built with clang - cfgargs+=("--with-system-ipxe=/usr/lib/ipxe/ipxe.pxe") - # newlib cannot be built with clang so we cannot build stubdoms - cfgargs+=("--disable-stubdom") -fi - -if ! test -z "$(ldd /bin/ls|grep musl|head -1)"; then - # disable --disable-werror for QEMUU when building with MUSL - cfgargs+=("--with-extra-qemuu-configure-args=\"--disable-werror\"") - # SeaBIOS doesn't build on MUSL systems - cfgargs+=("--with-system-seabios=/bin/false") -fi - -# Qemu requires Python 3.5 or later, and ninja -if ! type python3 || python3 -c "import sys; res = sys.version_info < (3, 5); exit(not(res))" \ - || ! type ninja; then - cfgargs+=("--with-system-qemu=/bin/false") -fi - -# SeaBIOS requires GCC 4.6 or later -if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then - cfgargs+=("--with-system-seabios=/bin/false") -fi - # Directory for the artefacts to be dumped into mkdir binaries
@@ -80,7 +49,37 @@ if [[ "${hypervisor_only}" == "y" ]]; then # Preserve artefacts cp xen/xen binaries/xen else - # Full build + # Full build. Figure out our ./configure options + cfgargs=() + cfgargs+=("--enable-docs") + + if [[ "${CC}" == "clang"* ]]; then + # SeaBIOS cannot be built with clang + cfgargs+=("--with-system-seabios=/usr/share/seabios/bios.bin") + # iPXE cannot be built with clang + cfgargs+=("--with-system-ipxe=/usr/lib/ipxe/ipxe.pxe") + # newlib cannot be built with clang so we cannot build stubdoms + cfgargs+=("--disable-stubdom") + fi + + if ! test -z "$(ldd /bin/ls|grep musl|head -1)"; then + # disable --disable-werror for QEMUU when building with MUSL + cfgargs+=("--with-extra-qemuu-configure-args=\"--disable-werror\"") + # SeaBIOS doesn't build on MUSL systems + cfgargs+=("--with-system-seabios=/bin/false") + fi + + # Qemu requires Python 3.5 or later, and ninja + if ! type python3 || python3 -c "import sys; res = sys.version_info < (3, 5); exit(not(res))" \ + || ! type ninja; then + cfgargs+=("--with-system-qemu=/bin/false") + fi + + # SeaBIOS requires GCC 4.6 or later + if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then + cfgargs+=("--with-system-seabios=/bin/false") + fi + ./configure "${cfgargs[@]}" make -j$(nproc) dist
Signature · 1 line
2.11.0
Re: [PATCH 3/6] CI: Only calculate ./configure args if needed
Hi Andrew,
Quoted 8 lines · Andrew Cooper
This is purely code motion of the cfgargs construction, into the case where it is used.
No practical change.
Signed-off-by Andrew Cooper <[email protected]>
~Michal
Re: [PATCH 3/6] CI: Only calculate ./configure args if needed
In reply to Andrew CooperQuoted 6 lines · Andrew Cooper
This is purely code motion of the cfgargs construction, into the case where it is used.
No practical change.
Signed-off-by Andrew Cooper <[email protected]>
Quoted 94 lines
---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/scripts/build | 63 ++++++++++++++++++++++++------------------------ 1 file changed, 31 insertions(+), 32 deletions(-)automation/scripts/build +31
-32@@ -39,37 +39,6 @@ if [[ "${XEN_TARGET_ARCH}" = "arm32" ]]; then hypervisor_only="y" fi -# build up our configure options -cfgargs=() -cfgargs+=("--enable-docs") - -if [[ "${CC}" == "clang"* ]]; then - # SeaBIOS cannot be built with clang - cfgargs+=("--with-system-seabios=/usr/share/seabios/bios.bin") - # iPXE cannot be built with clang - cfgargs+=("--with-system-ipxe=/usr/lib/ipxe/ipxe.pxe") - # newlib cannot be built with clang so we cannot build stubdoms - cfgargs+=("--disable-stubdom") -fi - -if ! test -z "$(ldd /bin/ls|grep musl|head -1)"; then - # disable --disable-werror for QEMUU when building with MUSL - cfgargs+=("--with-extra-qemuu-configure-args=\"--disable-werror\"") - # SeaBIOS doesn't build on MUSL systems - cfgargs+=("--with-system-seabios=/bin/false") -fi - -# Qemu requires Python 3.5 or later, and ninja -if ! type python3 || python3 -c "import sys; res = sys.version_info < (3, 5); exit(not(res))" \ - || ! type ninja; then - cfgargs+=("--with-system-qemu=/bin/false") -fi - -# SeaBIOS requires GCC 4.6 or later -if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then - cfgargs+=("--with-system-seabios=/bin/false") -fi - # Directory for the artefacts to be dumped into mkdir binaries@@ -80,7 +49,37 @@ if [[ "${hypervisor_only}" == "y" ]]; then # Preserve artefacts cp xen/xen binaries/xen else - # Full build + # Full build. Figure out our ./configure options + cfgargs=() + cfgargs+=("--enable-docs") + + if [[ "${CC}" == "clang"* ]]; then + # SeaBIOS cannot be built with clang + cfgargs+=("--with-system-seabios=/usr/share/seabios/bios.bin") + # iPXE cannot be built with clang + cfgargs+=("--with-system-ipxe=/usr/lib/ipxe/ipxe.pxe") + # newlib cannot be built with clang so we cannot build stubdoms + cfgargs+=("--disable-stubdom") + fi + + if ! test -z "$(ldd /bin/ls|grep musl|head -1)"; then + # disable --disable-werror for QEMUU when building with MUSL + cfgargs+=("--with-extra-qemuu-configure-args=\"--disable-werror\"") + # SeaBIOS doesn't build on MUSL systems + cfgargs+=("--with-system-seabios=/bin/false") + fi + + # Qemu requires Python 3.5 or later, and ninja + if ! type python3 || python3 -c "import sys; res = sys.version_info < (3, 5); exit(not(res))" \ + || ! type ninja; then + cfgargs+=("--with-system-qemu=/bin/false") + fi + + # SeaBIOS requires GCC 4.6 or later + if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then + cfgargs+=("--with-system-seabios=/bin/false") + fi + ./configure "${cfgargs[@]}" make -j$(nproc) distSignature · 1 line
2.11.0
PATCH 4/6 CI: Express HYPERVISOR_ONLY in build.yml
In reply to Andrew CooperWhether to build only Xen, or everything, is a property of container, toolchain and/or testcase. It is not a property of XEN_TARGET_ARCH.
Capitalise HYPERVISOR_ONLY and have it set by the debian-unstable-gcc-arm32-* testcases at the point that arm32 get matched with a container that can only build Xen.
For simplicity, retain the RANDCONFIG -> HYPERVISOR_ONLY implication.
---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>
--- automation/gitlab-ci/build.yaml | 2 ++ automation/scripts/build | 11 ++++------- 2 files changed, 6 insertions(+), 7 deletions(-)
automation/gitlab-ci/build.yaml +2 -0
@@ -516,11 +516,13 @@ debian-unstable-gcc-arm32: extends: .gcc-arm32-cross-build variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: y debian-unstable-gcc-arm32-debug: extends: .gcc-arm32-cross-build-debug variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: y debian-unstable-gcc-arm32-randconfig: extends: .gcc-arm32-cross-build
automation/scripts/build +4 -7
@@ -19,7 +19,9 @@ if [[ "${RANDCONFIG}" == "y" ]]; then fi make -j$(nproc) -C xen KCONFIG_ALLCONFIG=tools/kconfig/allrandom.config randconfig - hypervisor_only="y" + + # RANDCONFIG implies HYPERVISOR_ONLY + HYPERVISOR_ONLY="y" else echo "CONFIG_DEBUG=${debug}" > xen/.config
@@ -34,15 +36,10 @@ fi # to exit early -- bash is invoked with -e. cp xen/.config xen-config -# arm32 only cross-compiles the hypervisor -if [[ "${XEN_TARGET_ARCH}" = "arm32" ]]; then - hypervisor_only="y" -fi - # Directory for the artefacts to be dumped into mkdir binaries -if [[ "${hypervisor_only}" == "y" ]]; then +if [[ "${HYPERVISOR_ONLY}" == "y" ]]; then # Xen-only build make -j$(nproc) xen
Signature · 1 line
2.11.0
Re: [PATCH 4/6] CI: Express HYPERVISOR_ONLY in build.yml
Hi Andrew,
This is not something critical but I would like to see HYPERVISOR_ONLY set explicitly for all debian-unstable-gcc-arm32-* jobs as it is not clear for end-user of build.yaml that "the RANDCONFIG -> HYPERVISOR_ONLY implication."
~Oleksii
Quoted 75 lines · Andrew Cooper
Whether to build only Xen, or everything, is a property of container, toolchain and/or testcase. It is not a property of
XEN_TARGET_ARCH.Capitalise
HYPERVISOR_ONLYand have it set by the debian-unstable- gcc-arm32-*
testcases at the point that arm32 get matched with a container that can only
build Xen.For simplicity, retain the RANDCONFIG ->
HYPERVISOR_ONLYimplication.Signed-off-by Andrew Cooper <[email protected]>---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>
---
automation/gitlab-ci/build.yaml | 2 ++
automation/scripts/build | 11 ++++-------2 files changed, 6 insertions(+), 7 deletions(-)automation/gitlab- renamed from automation/gitlab-ci/build.yaml +0
-0
ci/build.yaml
index93d9ff69a9f2..e6a9357de3ef100644automation/gitlab-ci/build.yaml +0
-0@@ -516,11 +516,13 @@ debian-unstable-gcc-arm32:extends: .gcc-arm32-cross-build
variables:
CONTAINER: debian:unstable-arm32-gcc
+HYPERVISOR_ONLY: y
debian-unstable-gcc-arm32-debug:
extends: .gcc-arm32-cross-build-debug
variables:
CONTAINER: debian:unstable-arm32-gcc
+HYPERVISOR_ONLY: y
debian-unstable-gcc-arm32-randconfig:
extends: .gcc-arm32-cross-buildautomation/scripts/build +0
-0@@ -19,7 +19,9 @@ if [[ "${RANDCONFIG}" == "y" ]]; thenfi
make -j$(nproc) -C xen KCONFIG_ALLCONFIG=tools/kconfig/allrandom.config randconfig - hypervisor_only="y"
+
+ # RANDCONFIG impliesHYPERVISOR_ONLY
+ HYPERVISOR_ONLY="y"
else
echo "CONFIG_DEBUG=${debug}" > xen/.configautomation/scripts/build +0
-0@@ -34,15 +36,10 @@ fi # to exit early -- bash is invoked with -e.cp xen/.config xen-config
automation/scripts/build +0
-5-# arm32 only cross-compiles the hypervisor -if [[ "${XEN_TARGET_ARCH}" = "arm32" ]]; then - hypervisor_only="y" -fi -# Directory for the artefacts to be dumped into
mkdir binariesautomation/scripts/build +1
-1-if [[ "${hypervisor_only}" == "y" ]]; then +if [[ "${HYPERVISOR_ONLY}" == "y" ]]; then# Xen-only build
make -j$(nproc) xen
Re: [PATCH 4/6] CI: Express HYPERVISOR_ONLY in build.yml
In reply to Andrew CooperHi Andrew,
Quoted 5 lines · Andrew Cooper
Whether to build only Xen, or everything, is a property of container, toolchain and/or testcase. It is not a property of
XEN_TARGET_ARCH.
Quoted 7 lines
Capitalise
HYPERVISOR_ONLYand have it set by the debian-unstable-gcc-arm32-* testcases at the point that arm32 get matched with a container that can only build Xen.For simplicity, retain the RANDCONFIG ->
HYPERVISOR_ONLYimplication.Signed-off-by Andrew Cooper <[email protected]>
With regards to Oleksii comment, I do not think we should add HYPERVISOR_ONLY explicitly for randconfig debian-unstable-gcc-arm32-* jobs. Doing so would require similar change for all the randconfig jobs to be consistent and this is not needed.
~Michal
Re: [PATCH 4/6] CI: Express HYPERVISOR_ONLY in build.yml
In reply to Andrew CooperQuoted 26 lines · Andrew Cooper
Whether to build only Xen, or everything, is a property of container, toolchain and/or testcase. It is not a property of
XEN_TARGET_ARCH.Capitalise
HYPERVISOR_ONLYand have it set by the debian-unstable-gcc-arm32-* testcases at the point that arm32 get matched with a container that can only build Xen.For simplicity, retain the RANDCONFIG ->
HYPERVISOR_ONLYimplication.Signed-off-by Andrew Cooper <[email protected]>---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/gitlab-ci/build.yaml | 2 ++ automation/scripts/build | 11 ++++------- 2 files changed, 6 insertions(+), 7 deletions(-)automation/gitlab-ci/build.yaml +0
-0
automation/gitlab-ci/build.yaml debian-unstable-gcc-arm32: · line 525
Quoted 10 lines
automation/gitlab-ci/build.yaml +2
-0@@ -516,11 +516,13 @@ debian-unstable-gcc-arm32: extends: .gcc-arm32-cross-build variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: y debian-unstable-gcc-arm32-debug: extends: .gcc-arm32-cross-build-debug variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: y
can you move the setting of HYPERVISOR_ONLY to .arm32-cross-build-tmpl ?
I think that makes the most sense because .arm32-cross-build-tmpl is the one setting XEN_TARGET_ARCH and also the x86_64 tag.
Quoted 38 lines
automation/gitlab-ci/build.yaml +0
-0debian-unstable-gcc-arm32-randconfig: extends: .gcc-arm32-cross-buildautomation/scripts/build +4
-7@@ -19,7 +19,9 @@ if [[ "${RANDCONFIG}" == "y" ]]; then fi make -j$(nproc) -C xen KCONFIG_ALLCONFIG=tools/kconfig/allrandom.config randconfig - hypervisor_only="y" + + # RANDCONFIG implies HYPERVISOR_ONLY + HYPERVISOR_ONLY="y" else echo "CONFIG_DEBUG=${debug}" > xen/.config@@ -34,15 +36,10 @@ fi # to exit early -- bash is invoked with -e. cp xen/.config xen-config -# arm32 only cross-compiles the hypervisor -if [[ "${XEN_TARGET_ARCH}" = "arm32" ]]; then - hypervisor_only="y" -fi - # Directory for the artefacts to be dumped into mkdir binaries -if [[ "${hypervisor_only}" == "y" ]]; then +if [[ "${HYPERVISOR_ONLY}" == "y" ]]; then # Xen-only build make -j$(nproc) xenSignature · 1 line
2.11.0
Re: [PATCH 4/6] CI: Express HYPERVISOR_ONLY in build.yml
Quoted 33 lines · Stefano Stabellini
Quoted 31 lines · Andrew Cooper
Whether to build only Xen, or everything, is a property of container, toolchain and/or testcase. It is not a property of
XEN_TARGET_ARCH.Capitalise
HYPERVISOR_ONLYand have it set by the debian-unstable-gcc-arm32-* testcases at the point that arm32 get matched with a container that can only build Xen.For simplicity, retain the RANDCONFIG ->
HYPERVISOR_ONLYimplication.Signed-off-by Andrew Cooper <[email protected]>---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/gitlab-ci/build.yaml | 2 ++ automation/scripts/build | 11 ++++------- 2 files changed, 6 insertions(+), 7 deletions(-)automation/gitlab-ci/build.yaml +1
-0@@ -516,11 +516,13 @@ debian-unstable-gcc-arm32: extends: .gcc-arm32-cross-build variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: y
Quoted 6 lines
automation/gitlab-ci/build.yaml debian-unstable-gcc-arm32: · line 525
Quoted 5 lines
automation/gitlab-ci/build.yaml +1
-0debian-unstable-gcc-arm32-debug: extends: .gcc-arm32-cross-build-debug variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: ycan you move the setting of
HYPERVISOR_ONLYto .arm32-cross-build-tmpl ?
Not really - that's the point I'm trying to make in the commit message.
Quoted 2 lines
I think that makes the most sense because .arm32-cross-build-tmpl is the one setting
XEN_TARGET_ARCHand also thex86_64tag.
It's not about x86_64; its about the container.
Whether we can build just Xen, or everything, solely depends on the contents in debian:unstable-arm32-gcc
If we wanted to, we could update unstable-arm32-gcc's dockerfile to install the arm32 cross user libs, and drop this HYPERVISOR_ONLY restriction.
~Andrew
Re: [PATCH 4/6] CI: Express HYPERVISOR_ONLY in build.yml
Quoted 46 lines · Andrew Cooper
Quoted 33 lines · Stefano Stabellini
Quoted 31 lines · Andrew Cooper
Whether to build only Xen, or everything, is a property of container, toolchain and/or testcase. It is not a property of
XEN_TARGET_ARCH.Capitalise
HYPERVISOR_ONLYand have it set by the debian-unstable-gcc-arm32-* testcases at the point that arm32 get matched with a container that can only build Xen.For simplicity, retain the RANDCONFIG ->
HYPERVISOR_ONLYimplication.Signed-off-by Andrew Cooper <[email protected]>---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/gitlab-ci/build.yaml | 2 ++ automation/scripts/build | 11 ++++------- 2 files changed, 6 insertions(+), 7 deletions(-)automation/gitlab-ci/build.yaml +1
-0@@ -516,11 +516,13 @@ debian-unstable-gcc-arm32: extends: .gcc-arm32-cross-build variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: yQuoted 6 lines
automation/gitlab-ci/build.yaml debian-unstable-gcc-arm32: · line 525
Quoted 5 lines
automation/gitlab-ci/build.yaml +1
-0debian-unstable-gcc-arm32-debug: extends: .gcc-arm32-cross-build-debug variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: ycan you move the setting of
HYPERVISOR_ONLYto .arm32-cross-build-tmpl ?Not really - that's the point I'm trying to make in the commit message.
Quoted 2 lines
I think that makes the most sense because .arm32-cross-build-tmpl is the one setting
XEN_TARGET_ARCHand also thex86_64tag.
Quoted 8 lines
It's not about
x86_64; its about the container.Whether we can build just Xen, or everything, solely depends on the contents in debian:unstable-arm32-gcc
If we wanted to, we could update unstable-arm32-gcc's dockerfile to install the arm32 cross user libs, and drop this
HYPERVISOR_ONLYrestriction.
If it is a property of the container, shouldn't HYPERVISOR_ONLY be set every time the debian:unstable-arm32-gcc container is used? Including debian-unstable-gcc-arm32-randconfig and debian-unstable-gcc-arm32-debug-randconfig?
I realize that the other 2 jobs are randconfigs so HYPERVISOR_ONLY gets set anyway. But if HYPERVISOR_ONLY is a property of the specific container, then I think it would be best to be consistent and set HYPERVISOR_ONLY everywhere debian:unstable-arm32-gcc is used.
E.g. one day we could just randconfigs to build also the tools with a simple change to the build script and otherwise we would need to remember to also add the HYPERVISOR_ONLY tag for the other 2 jobs using debian:unstable-arm32-gcc.
Re: [PATCH 4/6] CI: Express HYPERVISOR_ONLY in build.yml
Quoted 63 lines · Stefano Stabellini
Quoted 44 lines · Andrew Cooper
Quoted 33 lines · Stefano Stabellini
Quoted 31 lines · Andrew Cooper
Whether to build only Xen, or everything, is a property of container, toolchain and/or testcase. It is not a property of
XEN_TARGET_ARCH.Capitalise
HYPERVISOR_ONLYand have it set by the debian-unstable-gcc-arm32-* testcases at the point that arm32 get matched with a container that can only build Xen.For simplicity, retain the RANDCONFIG ->
HYPERVISOR_ONLYimplication.Signed-off-by Andrew Cooper <[email protected]>---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/gitlab-ci/build.yaml | 2 ++ automation/scripts/build | 11 ++++------- 2 files changed, 6 insertions(+), 7 deletions(-)automation/gitlab-ci/build.yaml +1
-0@@ -516,11 +516,13 @@ debian-unstable-gcc-arm32: extends: .gcc-arm32-cross-build variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: yQuoted 6 lines
automation/gitlab-ci/build.yaml debian-unstable-gcc-arm32: · line 525
Quoted 5 lines
automation/gitlab-ci/build.yaml +1
-0debian-unstable-gcc-arm32-debug: extends: .gcc-arm32-cross-build-debug variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: ycan you move the setting of
HYPERVISOR_ONLYto .arm32-cross-build-tmpl ?Not really - that's the point I'm trying to make in the commit message.
Quoted 2 lines
I think that makes the most sense because .arm32-cross-build-tmpl is the one setting
XEN_TARGET_ARCHand also thex86_64tag.Quoted 8 lines
It's not about
x86_64; its about the container.Whether we can build just Xen, or everything, solely depends on the contents in debian:unstable-arm32-gcc
If we wanted to, we could update unstable-arm32-gcc's dockerfile to install the arm32 cross user libs, and drop this
HYPERVISOR_ONLYrestriction.If it is a property of the container, shouldn't
HYPERVISOR_ONLYbe set every time the debian:unstable-arm32-gcc container is used? Including debian-unstable-gcc-arm32-randconfig and debian-unstable-gcc-arm32-debug-randconfig?I realize that the other 2 jobs are randconfigs so
HYPERVISOR_ONLYgets set anyway. But ifHYPERVISOR_ONLYis a property of the specific container, then I think it would be best to be consistent and setHYPERVISOR_ONLYeverywhere debian:unstable-arm32-gcc is used.
Quoted 4 lines
E.g. one day we could just randconfigs to build also the tools with a simple change to the build script and otherwise we would need to remember to also add the
HYPERVISOR_ONLYtag for the other 2 jobs using debian:unstable-arm32-gcc.
Ok, so we want 4 HYPERVISOR_ONLY's in total, one for each instance of
CONTAINER: debian:unstable-arm32-gcc ?
~Andrew
Re: [PATCH 4/6] CI: Express HYPERVISOR_ONLY in build.yml
Quoted 64 lines · Andrew Cooper
Quoted 63 lines · Stefano Stabellini
Quoted 44 lines · Andrew Cooper
Quoted 33 lines · Stefano Stabellini
Quoted 31 lines · Andrew Cooper
Whether to build only Xen, or everything, is a property of container, toolchain and/or testcase. It is not a property of
XEN_TARGET_ARCH.Capitalise
HYPERVISOR_ONLYand have it set by the debian-unstable-gcc-arm32-* testcases at the point that arm32 get matched with a container that can only build Xen.For simplicity, retain the RANDCONFIG ->
HYPERVISOR_ONLYimplication.Signed-off-by Andrew Cooper <[email protected]>---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/gitlab-ci/build.yaml | 2 ++ automation/scripts/build | 11 ++++------- 2 files changed, 6 insertions(+), 7 deletions(-)automation/gitlab-ci/build.yaml +1
-0@@ -516,11 +516,13 @@ debian-unstable-gcc-arm32: extends: .gcc-arm32-cross-build variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: yQuoted 6 lines
automation/gitlab-ci/build.yaml debian-unstable-gcc-arm32: · line 525
Quoted 5 lines
automation/gitlab-ci/build.yaml +1
-0debian-unstable-gcc-arm32-debug: extends: .gcc-arm32-cross-build-debug variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: ycan you move the setting of
HYPERVISOR_ONLYto .arm32-cross-build-tmpl ?Not really - that's the point I'm trying to make in the commit message.
Quoted 2 lines
I think that makes the most sense because .arm32-cross-build-tmpl is the one setting
XEN_TARGET_ARCHand also thex86_64tag.Quoted 8 lines
It's not about
x86_64; its about the container.Whether we can build just Xen, or everything, solely depends on the contents in debian:unstable-arm32-gcc
If we wanted to, we could update unstable-arm32-gcc's dockerfile to install the arm32 cross user libs, and drop this
HYPERVISOR_ONLYrestriction.If it is a property of the container, shouldn't
HYPERVISOR_ONLYbe set every time the debian:unstable-arm32-gcc container is used? Including debian-unstable-gcc-arm32-randconfig and debian-unstable-gcc-arm32-debug-randconfig?I realize that the other 2 jobs are randconfigs so
HYPERVISOR_ONLYgets set anyway. But ifHYPERVISOR_ONLYis a property of the specific container, then I think it would be best to be consistent and setHYPERVISOR_ONLYeverywhere debian:unstable-arm32-gcc is used.
Quoted 7 lines
Quoted 4 lines
E.g. one day we could just randconfigs to build also the tools with a simple change to the build script and otherwise we would need to remember to also add the
HYPERVISOR_ONLYtag for the other 2 jobs using debian:unstable-arm32-gcc.Ok, so we want 4 HYPERVISOR_ONLY's in total, one for each instance of
CONTAINER: debian:unstable-arm32-gcc ?
yeah
Re: [PATCH 4/6] CI: Express HYPERVISOR_ONLY in build.yml
Quoted 65 lines · Stefano Stabellini
Quoted 64 lines · Andrew Cooper
Quoted 63 lines · Stefano Stabellini
Quoted 44 lines · Andrew Cooper
Quoted 33 lines · Stefano Stabellini
Quoted 31 lines · Andrew Cooper
Whether to build only Xen, or everything, is a property of container, toolchain and/or testcase. It is not a property of
XEN_TARGET_ARCH.Capitalise
HYPERVISOR_ONLYand have it set by the debian-unstable-gcc-arm32-* testcases at the point that arm32 get matched with a container that can only build Xen.For simplicity, retain the RANDCONFIG ->
HYPERVISOR_ONLYimplication.Signed-off-by Andrew Cooper <[email protected]>---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/gitlab-ci/build.yaml | 2 ++ automation/scripts/build | 11 ++++------- 2 files changed, 6 insertions(+), 7 deletions(-)automation/gitlab-ci/build.yaml +1
-0@@ -516,11 +516,13 @@ debian-unstable-gcc-arm32: extends: .gcc-arm32-cross-build variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: yQuoted 6 lines
automation/gitlab-ci/build.yaml debian-unstable-gcc-arm32: · line 525
Quoted 5 lines
automation/gitlab-ci/build.yaml +1
-0debian-unstable-gcc-arm32-debug: extends: .gcc-arm32-cross-build-debug variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: ycan you move the setting of
HYPERVISOR_ONLYto .arm32-cross-build-tmpl ?Not really - that's the point I'm trying to make in the commit message.
Quoted 2 lines
I think that makes the most sense because .arm32-cross-build-tmpl is the one setting
XEN_TARGET_ARCHand also thex86_64tag.Quoted 8 lines
It's not about
x86_64; its about the container.Whether we can build just Xen, or everything, solely depends on the contents in debian:unstable-arm32-gcc
If we wanted to, we could update unstable-arm32-gcc's dockerfile to install the arm32 cross user libs, and drop this
HYPERVISOR_ONLYrestriction.If it is a property of the container, shouldn't
HYPERVISOR_ONLYbe set every time the debian:unstable-arm32-gcc container is used? Including debian-unstable-gcc-arm32-randconfig and debian-unstable-gcc-arm32-debug-randconfig?I realize that the other 2 jobs are randconfigs so
HYPERVISOR_ONLYgets set anyway. But ifHYPERVISOR_ONLYis a property of the specific container, then I think it would be best to be consistent and setHYPERVISOR_ONLYeverywhere debian:unstable-arm32-gcc is used.
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
E.g. one day we could just randconfigs to build also the tools with a simple change to the build script and otherwise we would need to remember to also add the
HYPERVISOR_ONLYtag for the other 2 jobs using debian:unstable-arm32-gcc.Ok, so we want 4 HYPERVISOR_ONLY's in total, one for each instance of
CONTAINER: debian:unstable-arm32-gcc ?yeah
Can I take that as an R-by/A-by then?
~Andrew
Re: [PATCH 4/6] CI: Express HYPERVISOR_ONLY in build.yml
Quoted 70 lines · Andrew Cooper
Quoted 69 lines · Stefano Stabellini
Quoted 68 lines · Andrew Cooper
Quoted 67 lines · Stefano Stabellini
Quoted 44 lines · Andrew Cooper
Quoted 33 lines · Stefano Stabellini
Quoted 31 lines · Andrew Cooper
Whether to build only Xen, or everything, is a property of container, toolchain and/or testcase. It is not a property of
XEN_TARGET_ARCH.Capitalise
HYPERVISOR_ONLYand have it set by the debian-unstable-gcc-arm32-* testcases at the point that arm32 get matched with a container that can only build Xen.For simplicity, retain the RANDCONFIG ->
HYPERVISOR_ONLYimplication.Signed-off-by Andrew Cooper <[email protected]>---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/gitlab-ci/build.yaml | 2 ++ automation/scripts/build | 11 ++++------- 2 files changed, 6 insertions(+), 7 deletions(-)automation/gitlab-ci/build.yaml +1
-0@@ -516,11 +516,13 @@ debian-unstable-gcc-arm32: extends: .gcc-arm32-cross-build variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: yQuoted 6 lines
automation/gitlab-ci/build.yaml debian-unstable-gcc-arm32: · line 525
Quoted 5 lines
automation/gitlab-ci/build.yaml +1
-0debian-unstable-gcc-arm32-debug: extends: .gcc-arm32-cross-build-debug variables: CONTAINER: debian:unstable-arm32-gcc + HYPERVISOR_ONLY: ycan you move the setting of
HYPERVISOR_ONLYto .arm32-cross-build-tmpl ?Not really - that's the point I'm trying to make in the commit message.
Quoted 2 lines
I think that makes the most sense because .arm32-cross-build-tmpl is the one setting
XEN_TARGET_ARCHand also thex86_64tag.Quoted 8 lines
It's not about
x86_64; its about the container.Whether we can build just Xen, or everything, solely depends on the contents in debian:unstable-arm32-gcc
If we wanted to, we could update unstable-arm32-gcc's dockerfile to install the arm32 cross user libs, and drop this
HYPERVISOR_ONLYrestriction.If it is a property of the container, shouldn't
HYPERVISOR_ONLYbe set every time the debian:unstable-arm32-gcc container is used? Including debian-unstable-gcc-arm32-randconfig and debian-unstable-gcc-arm32-debug-randconfig?I realize that the other 2 jobs are randconfigs so
HYPERVISOR_ONLYgets set anyway. But ifHYPERVISOR_ONLYis a property of the specific container, then I think it would be best to be consistent and setHYPERVISOR_ONLYeverywhere debian:unstable-arm32-gcc is used.E.g. one day we could just randconfigs to build also the tools with a simple change to the build script and otherwise we would need to remember to also add the
HYPERVISOR_ONLYtag for the other 2 jobs using debian:unstable-arm32-gcc.
Quoted 5 lines
Quoted 3 lines
Quoted 2 lines
Ok, so we want 4 HYPERVISOR_ONLY's in total, one for each instance of
CONTAINER: debian:unstable-arm32-gcc ?yeah
Can I take that as an R-by/A-by then?
yep
PATCH 5/6 CI: Fix build script when CROSS_COMPILE is in use
In reply to Andrew CooperSome testcases use a cross compiler. Presently it's only arm32 and due to previous cleanup the only thing which is now wrong is printing the compiler version at the start of day.
Construct $cc to match what make will eventually choose given CROSS_COMPILE, taking care not to modify $CC. Use $cc throughout the rest of the script.
Also correct the compiler detection logic. Plain "gcc" was wrong, and "clang"* was a bodge highlighting the issue, but neither survive the CROSS_COMPILE correction. Instead, construct cc_is_{gcc,clang} booleans like we do elsewhere in the build system, by querying the --version text for gcc or clang.
While making this change, adjust cc_ver to be calculated once at the same time as cc_is_* are calculated.
---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>
--- automation/scripts/build | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-)
automation/scripts/build +14 -8
@@ -2,13 +2,12 @@ test -f /etc/os-release && cat "$_" -$CC --version +# Construct $cc such that it matches what `make` will chose when taking +# CROSS_COMPILE into account. Do not modify $CC directly, as that will cause +# `make` to double-account CROSS_COMPILE. +cc="${CROSS_COMPILE}${CC}" -# Express the compiler version as an integer. e.g. GCC 4.9.2 => 0x040902 -cc-ver() -{ - $CC -dumpversion | awk -F. '{ printf "0x%02x%02x%02x", $1, $2, $3 }' -} +$cc --version # random config or default config if [[ "${RANDCONFIG}" == "y" ]]; then
@@ -50,7 +49,14 @@ else cfgargs=() cfgargs+=("--enable-docs") - if [[ "${CC}" == "clang"* ]]; then + # booleans for which compiler is in use + cc_is_gcc="$($cc --version | grep -q gcc && echo "y" || :)" + cc_is_clang="$($cc --version | grep -q clang && echo "y" || :)" + + # The compiler version as an integer. e.g. GCC 4.9.2 => 0x040902 + cc_ver="$($cc -dumpversion | awk -F. '{ printf "0x%02x%02x%02x", $1, $2, $3 }')" + + if [[ "${cc_is_clang}" == "y" ]]; then # SeaBIOS cannot be built with clang cfgargs+=("--with-system-seabios=/usr/share/seabios/bios.bin") # iPXE cannot be built with clang
@@ -73,7 +79,7 @@ else fi # SeaBIOS requires GCC 4.6 or later - if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then + if [[ "${cc_is_gcc}" == "y" && "${cc_ver}" -lt 0x040600 ]]; then cfgargs+=("--with-system-seabios=/bin/false") fi
Signature · 1 line
2.11.0
Re: [PATCH 5/6] CI: Fix build script when CROSS_COMPILE is in use
Quoted 13 lines · Andrew Cooper
Some testcases use a cross compiler. Presently it's only arm32 and due to previous cleanup the only thing which is now wrong is printing the compiler version at the start of day.
Construct $cc to match what
makewill eventually choose givenCROSS_COMPILE, taking care not to modify $CC. Use $cc throughout the rest of the script.Also correct the compiler detection logic. Plain "gcc" was wrong, and "clang"* was a bodge highlighting the issue, but neither survive the
CROSS_COMPILEcorrection. Instead, construct cc_is_{gcc,clang} booleans like we do elsewhere in the build system, by querying the --version text for gcc or clang.
Quoted 4 lines
While making this change, adjust
cc_verto be calculated once at the same time as cc_is_* are calculated.Signed-off-by Andrew Cooper <[email protected]>
Quoted 61 lines
---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/scripts/build | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-)automation/scripts/build +14
-8@@ -2,13 +2,12 @@ test -f /etc/os-release && cat "$_" -$CC --version +# Construct $cc such that it matches what `make` will chose when taking +# CROSS_COMPILE into account. Do not modify $CC directly, as that will cause +# `make` to double-account CROSS_COMPILE. +cc="${CROSS_COMPILE}${CC}" -# Express the compiler version as an integer. e.g. GCC 4.9.2 => 0x040902 -cc-ver() -{ - $CC -dumpversion | awk -F. '{ printf "0x%02x%02x%02x", $1, $2, $3 }' -} +$cc --version # random config or default config if [[ "${RANDCONFIG}" == "y" ]]; then@@ -50,7 +49,14 @@ else cfgargs=() cfgargs+=("--enable-docs") - if [[ "${CC}" == "clang"* ]]; then + # booleans for which compiler is in use + cc_is_gcc="$($cc --version | grep -q gcc && echo "y" || :)" + cc_is_clang="$($cc --version | grep -q clang && echo "y" || :)" + + # The compiler version as an integer. e.g. GCC 4.9.2 => 0x040902 + cc_ver="$($cc -dumpversion | awk -F. '{ printf "0x%02x%02x%02x", $1, $2, $3 }')" + + if [[ "${cc_is_clang}" == "y" ]]; then # SeaBIOS cannot be built with clang cfgargs+=("--with-system-seabios=/usr/share/seabios/bios.bin") # iPXE cannot be built with clang@@ -73,7 +79,7 @@ else fi # SeaBIOS requires GCC 4.6 or later - if [[ "${CC}" == "gcc" && `cc-ver` -lt 0x040600 ]]; then + if [[ "${cc_is_gcc}" == "y" && "${cc_ver}" -lt 0x040600 ]]; then cfgargs+=("--with-system-seabios=/bin/false") fiSignature · 1 line
2.11.0
PATCH 6/6 CI: Simplify the MUSL check
In reply to Andrew CooperThere's no need to do ad-hoc string parsing. Use grep -q instead.
No functional change.
---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>
--- automation/scripts/build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
automation/scripts/build +1 -1
@@ -65,7 +65,7 @@ else cfgargs+=("--disable-stubdom") fi - if ! test -z "$(ldd /bin/ls|grep musl|head -1)"; then + if ldd /bin/ls | grep -q musl; then # disable --disable-werror for QEMUU when building with MUSL cfgargs+=("--with-extra-qemuu-configure-args=\"--disable-werror\"") # SeaBIOS doesn't build on MUSL systems
Signature · 1 line
2.11.0
Re: [PATCH 6/6] CI: Simplify the MUSL check
Quoted 5 lines · Andrew Cooper
There's no need to do ad-hoc string parsing. Use grep -q instead.
No functional change.
Signed-off-by Andrew Cooper <[email protected]>
Quoted 26 lines
---
CC: Doug Goldstein <[email protected]>
CC: Stefano Stabellini <[email protected]>
CC: Anthony PERARD <[email protected]>
CC: Michal Orzel <[email protected]>
CC: Oleksii Kurochko <[email protected]>--- automation/scripts/build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)automation/scripts/build +1
-1@@ -65,7 +65,7 @@ else cfgargs+=("--disable-stubdom") fi - if ! test -z "$(ldd /bin/ls|grep musl|head -1)"; then + if ldd /bin/ls | grep -q musl; then # disable --disable-werror for QEMUU when building with MUSL cfgargs+=("--with-extra-qemuu-configure-args=\"--disable-werror\"") # SeaBIOS doesn't build on MUSL systemsSignature · 1 line
2.11.0