gitlab: improve debuggability of jobs
cover gitlab: improve debuggability of jobs
This ensures that the main configure/make commands run have their args echoed into the build logs.
Daniel P. Berrangé (2):
gitlab: remove redundant setting of PKG_CONFIG_PATH
gitlab: show configure/make args before running
.gitlab-ci.d/buildtest-template.yml | 4 ++++ .gitlab-ci.d/crossbuild-template.yml | 24 ++++++++++++++++++------ 2 files changed, 22 insertions(+), 6 deletions(-)
Signature · 1 line
2.37.2
PATCH 1/2 gitlab: remove redundant setting of PKG_CONFIG_PATH
The PKG_CONFIG_PATH variable is not defined in GitLab CI envs and even if it was, we don't need to set it to its existing value.
--- .gitlab-ci.d/crossbuild-template.yml | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-)
.gitlab-ci.d/crossbuild-template.yml +3 -6
@@ -6,8 +6,7 @@ script: - mkdir build - cd build - - PKG_CONFIG_PATH=$PKG_CONFIG_PATH - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS + - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS --disable-user --target-list-exclude="arm-softmmu cris-softmmu i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu
@@ -32,8 +31,7 @@ script: - mkdir build - cd build - - PKG_CONFIG_PATH=$PKG_CONFIG_PATH - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS + - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS --disable-tools --enable-${ACCEL:-kvm} $EXTRA_CONFIGURE_OPTS - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
@@ -44,8 +42,7 @@ script: - mkdir build - cd build - - PKG_CONFIG_PATH=$PKG_CONFIG_PATH - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS + - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS --disable-system --target-list-exclude="aarch64_be-linux-user alpha-linux-user cris-linux-user m68k-linux-user microblazeel-linux-user nios2-linux-user or1k-linux-user ppc-linux-user sparc-linux-user
Signature · 1 line
2.37.2
Re: [PATCH 1/2] gitlab: remove redundant setting of PKG_CONFIG_PATH
Quoted 9 lines · Daniel P. Berrangé
The
PKG_CONFIG_PATHvariable is not defined in GitLab CI envs and even if it was, we don't need to set it to its existing value.Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/crossbuild-template.yml | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-)
PATCH 2/2 gitlab: show configure/make args before running
In reply to Daniel P. BerrangéWhen debugging failed jobs it is helpful to see the full configure/make args used, without having to search the gitlab config file to figure it out.
--- .gitlab-ci.d/buildtest-template.yml | 4 ++++ .gitlab-ci.d/crossbuild-template.yml | 15 +++++++++++++++ 2 files changed, 19 insertions(+)
.gitlab-ci.d/buildtest-template.yml +4 -0
@@ -13,8 +13,10 @@ - cd build - if test -n "$TARGETS"; then + echo "../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list=\"$TARGETS\"" ; ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list="$TARGETS" ; else + echo "../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS" ; ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS ; fi || { cat config.log meson-logs/meson-log.txt && exit 1; } - if test -n "$LD_JOBS";
@@ -24,6 +26,7 @@ - make -j"$JOBS" - if test -n "$MAKE_CHECK_ARGS"; then + echo "make -j\"$JOBS\" $MAKE_CHECK_ARGS" ; make -j"$JOBS" $MAKE_CHECK_ARGS ; fi
@@ -37,6 +40,7 @@ - cd build - find . -type f -exec touch {} + # Avoid recompiling by hiding ninja with NINJA=":" + - echo "make NINJA=\":\" $MAKE_CHECK_ARGS" - make NINJA=":" $MAKE_CHECK_ARGS .native_test_job_template:
.gitlab-ci.d/crossbuild-template.yml +15 -0
@@ -6,11 +6,17 @@ script: - mkdir build - cd build + - echo "../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS + --disable-user --target-list-exclude=\"arm-softmmu cris-softmmu + i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu + mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu + sparc-softmmu xtensa-softmmu $CROSS_SKIP_TARGETS\"" - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS --disable-user --target-list-exclude="arm-softmmu cris-softmmu i386-softmmu microblaze-softmmu mips-softmmu mipsel-softmmu mips64-softmmu ppc-softmmu riscv32-softmmu sh4-softmmu sparc-softmmu xtensa-softmmu $CROSS_SKIP_TARGETS" + - echo "make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS" - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS - if grep -q "EXESUF=.exe" config-host.mak; then make installer;
@@ -31,8 +37,11 @@ script: - mkdir build - cd build + - echo "../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS + --disable-tools --enable-${ACCEL:-kvm} $EXTRA_CONFIGURE_OPTS" - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS --disable-tools --enable-${ACCEL:-kvm} $EXTRA_CONFIGURE_OPTS + - echo "make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS" - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS .cross_user_build_job:
@@ -42,9 +51,15 @@ script: - mkdir build - cd build + - echo "../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS + --disable-system --target-list-exclude=\"aarch64_be-linux-user + alpha-linux-user cris-linux-user m68k-linux-user microblazeel-linux-user + nios2-linux-user or1k-linux-user ppc-linux-user sparc-linux-user + xtensa-linux-user $CROSS_SKIP_TARGETS\"" - ../configure --enable-werror --disable-docs $QEMU_CONFIGURE_OPTS --disable-system --target-list-exclude="aarch64_be-linux-user alpha-linux-user cris-linux-user m68k-linux-user microblazeel-linux-user nios2-linux-user or1k-linux-user ppc-linux-user sparc-linux-user xtensa-linux-user $CROSS_SKIP_TARGETS" + - echo "make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS" - make -j$(expr $(nproc) + 1) all check-build $MAKE_CHECK_ARGS
Signature · 1 line
2.37.2
Re: [PATCH 2/2] gitlab: show configure/make args before running
Quoted 10 lines · Daniel P. Berrangé
When debugging failed jobs it is helpful to see the full configure/make args used, without having to search the gitlab config file to figure it out.
Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/buildtest-template.yml | 4 ++++ .gitlab-ci.d/crossbuild-template.yml | 15 +++++++++++++++ 2 files changed, 19 insertions(+)
On IRC Mark suggested sh -xc "$CMD" to avoid duplication. I'm not sure how to get escaping right though.
Re: [PATCH 2/2] gitlab: show configure/make args before running
Quoted 6 lines · Stefan Hajnoczi
Quoted 5 lines · Daniel P. Berrangé
When debugging failed jobs it is helpful to see the full configure/make args used, without having to search the gitlab config file to figure it out.
Quoted 8 lines
Quoted 5 lines
Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/buildtest-template.yml | 4 ++++ .gitlab-ci.d/crossbuild-template.yml | 15 +++++++++++++++ 2 files changed, 19 insertions(+)On IRC Mark suggested sh -xc "$CMD" to avoid duplication. I'm not sure how to get escaping right though.
set -x && ../configure ... ?
r~
Re: [PATCH 2/2] gitlab: show configure/make args before running
Quoted 13 lines · Richard Henderson
Quoted 12 lines · Stefan Hajnoczi
Quoted 10 lines · Daniel P. Berrangé
When debugging failed jobs it is helpful to see the full configure/make args used, without having to search the gitlab config file to figure it out.
Signed-off-by Daniel P. Berrangé <[email protected]>--- .gitlab-ci.d/buildtest-template.yml | 4 ++++ .gitlab-ci.d/crossbuild-template.yml | 15 +++++++++++++++ 2 files changed, 19 insertions(+)
Quoted 4 lines
Quoted 2 lines
On IRC Mark suggested sh -xc "$CMD" to avoid duplication. I'm not sure how to get escaping right though.
set -x && ../configure ... ?
I tried adding 'set -x' to the script: blocks too. It turns out that gitlab has an echo '$CMD' for each line in the yaml script: section
IOW, with my patch here we get a log that looks like
$ mkdir build
$ cd build
$ if test -n "$TARGETS"; then echo "../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list=\"$TARGETS\"" ; ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list="$TARGETS" ; else echo "../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS" ; ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS ; fi || { cat config.log meson-logs/meson-log.txt && exit 1; }
../configure --enable-werror --disable-docs --disable-gcrypt --enable-nettle --enable-docs --enable-fdt=system --enable-slirp --enable-capstone --target-list="tricore-softmmu microblaze-softmmu mips-softmmu xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu"and with adding 'set -x' we get
++ echo '$ mkdir build'
$ mkdir build
++ mkdir build
++ echo '$ cd build'
$ cd build
++ cd build
++ echo '$ if test -n "$TARGETS"; then ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list="$TARGETS" ; else ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS ; fi || { cat config.log meson-logs/meson-log.txt && exit 1; }'
$ if test -n "$TARGETS"; then ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS --target-list="$TARGETS" ; else ../configure --enable-werror --disable-docs ${LD_JOBS:+--meson=git} $CONFIGURE_ARGS ; fi || { cat config.log meson-logs/meson-log.txt && exit 1; }
++ test -n 'tricore-softmmu microblaze-softmmu mips-softmmu xtensa-softmmu m68k-softmmu riscv32-softmmu ppc-softmmu sparc64-softmmu'
++ ../configure --enable-werror --disable-docs --disable-gcrypt --enable-nettle --enable-docs --enable-fdt=system --enable-slirp --enable-capstone '--target-list=tricore-softmmu microblaze-softmmu mips-softmmu xtensa-sofDoing 'set -x && configure' only marginally helps, as the 'set -x' still takes effect for all commands that come after configure too.
The other option is to not do either 'echo' not 'set -x', and instead do this:
printenv | grep -v -E "(CI_|GITLAB)"
so that the build logs has a dump of any env vars set. So when seeing
./configure $CONFIGURE_ARGS
we have a record of what $CONFIGURE_ARGS was set to.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Paolo Bonzini
AppliedQueued patch 1 while the second is figured out, thanks.
Paolo