Azure: Set the timeout for jobs to the maximum
PATCH 1/4 Azure: Set the timeout for jobs to the maximum
As per current Azure Pipelines documentation we qualify for 3600 minutes per job, if specified, as the timeout. The default unspecified timeout is 60 minutes. Rework things to specify 0 as the timeout (and so maximum allowed) so that we don't have failures due to running slightly past 60 minutes total.
--- .azure-pipelines.yml | 1 + 1 file changed, 1 insertion(+)
.azure-pipelines.yml +1 -0
@@ -468,6 +468,7 @@ stages: - stage: world_build jobs: - job: build_the_world + timeoutInMinutes: 0 # Use the maximum allowed displayName: 'Build the World' pool: vmImage: $(ubuntu_vm)
Signature · 1 line
2.34.1
PATCH 2/4 Azure: Rework build the world jobs
Now that we have 3600 minutes per build job, condense and rework things such that our overall time largely doesn't change, but we can also largely avoid having to re-tweak this job to avoid timeouts. Given that we have 10 threads, we also move a few of the specific sandbox test builds to a prior stage.
Note that while sandbox builds with address sanitization enabled (ASAN) not all tests pass, so we limit ourselves to just checking that the version test passes for now.
---
Cc: Simon Glass <[email protected]>
I believe we merged ASAN with the intention of fixing some of the found issues later, so this isn't really a functional change.
--- .azure-pipelines.yml | 100 +++++++++++-------------------------------- 1 file changed, 25 insertions(+), 75 deletions(-)
.azure-pipelines.yml +25 -75
@@ -254,9 +254,17 @@ stages: matrix: sandbox: TEST_PY_BD: "sandbox" + sandbox_asan: + TEST_PY_BD: "sandbox" + OVERRIDE: "-a ASAN" + TEST_PY_TEST_SPEC: "version" sandbox_clang: TEST_PY_BD: "sandbox" OVERRIDE: "-O clang-16" + sandbox_clang_asan: + TEST_PY_BD: "sandbox" + OVERRIDE: "-O clang-16 -a ASAN" + TEST_PY_TEST_SPEC: "version" sandbox_nolto: TEST_PY_BD: "sandbox" BUILD_ENV: "NO_LTO=1"
@@ -476,84 +484,26 @@ stages: # Use almost the same target division in .travis.yml, only merged # 3 small build jobs (arc/microblaze/xtensa) into one. matrix: - arc_nios2_m68k_microblaze_xtensa: - BUILDMAN: "arc nios2 microblaze m68k xtensa" - amlogic: - BUILDMAN: "amlogic" - arm11_arm7_arm920t_arm946es: - BUILDMAN: "arm11 arm7 arm920t arm946es" - arm926ejs: - BUILDMAN: "arm926ejs -x freescale,siemens,at91,kirkwood,omap" - at91_non_armv7: - BUILDMAN: "at91 -x armv7" - at91_non_arm926ejs: - BUILDMAN: "at91 -x arm926ejs" - boundary_engicam_toradex: - BUILDMAN: "boundary engicam toradex" - arm_bcm: - BUILDMAN: "bcm -x mips" - nxp_arm32: - BUILDMAN: "freescale -x powerpc,m68k,aarch64,ls101,ls102,ls104,ls108,ls20,lx216" - nxp_ls101x_ls108x: - BUILDMAN: "freescale&ls101 freescale&ls108" - nxp_ls102x: - BUILDMAN: "freescale&ls102 -x keymile" - nxp_ls104x: - BUILDMAN: "freescale&ls104" - nxp_ls20xx_lx216x: - BUILDMAN: "freescale&ls20 freescale&lx216" - imx6: - BUILDMAN: "mx6 -x boundary,engicam,freescale,technexion,toradex" + am33xx_at91_kirkwood_mvebu_omap: + BUILDMAN: "am33xx at91_kirkwood mvebu omap -x siemens" + amlogic_bcm_boundary_engicam_siemens_technexion_oradex: + BUILDMAN: "amlogic bcm boundary engicam siemens technexion toradex -x mips" + arm_nxp_minus_imx: + BUILDMAN: "freescale -x powerpc,m68k,imx,mx" imx: - BUILDMAN: "mx -x mx6,imx8,freescale,technexion,toradex" - imx8_imx9: - BUILDMAN: "imx8 imx9 -x engicam,technexion,toradex" - keymiles_siemens_technexion: - BUILDMAN: "keymile siemens technexion" - keystone2_keystone3: - BUILDMAN: "k2 k3 -x siemens,toradex" - sandbox_asan: - BUILDMAN: "sandbox" - OVERRIDE: "-a ASAN" - sandbox_clang_asan: - BUILDMAN: "sandbox" - OVERRIDE: "-O clang-16 -a ASAN" - samsung_socfpga_renesas: - BUILDMAN: "samsung socfpga renesas" - sun4i_sun9i: - BUILDMAN: "sun4i sun9i" - sun5i_sun6i: - BUILDMAN: "sun5i sun6i" - sun7i: - BUILDMAN: "sun7i" - sun8i: - BUILDMAN: "sun8i" - sun50i: - BUILDMAN: "sun50i" - arm_catch_all: - BUILDMAN: "arm -x arm11,arm7,arm9,aarch64,at91,bcm,freescale,kirkwood,mvebu,renesas,siemens,tegra,uniphier,mx,samsung,sunxi,am33xx,omap,toradex,socfpga,k2,k3,zynq" - sandbox_x86: - BUILDMAN: "sandbox x86" - kirkwood_mvebu_uniphier: - BUILDMAN: "kirkwood mvebu uniphier" - mips: - BUILDMAN: "mips" + BUILDMAN: "mx imx -x boundary,engicam,technexion,toradex" + rk: + BUILDMAN: "rk" + sunxi: + BUILDMAN: "sunxi" powerpc: - BUILDMAN: "powerpc -x keymile" - tegra: - BUILDMAN: "tegra -x toradex" - am33xx_omap: - BUILDMAN: "am33xx omap -x siemens" + BUILDMAN: "powerpc" + arm_catch_all: + BUILDMAN: "arm -x aarch64,am33xx,at91,bcm,ls1,kirkwood,mvebu,omap,rk,siemens,mx,sunxi,technexion,toradex" aarch64_catch_all: - BUILDMAN: "aarch64 -x amlogic,bcm,imx8,imx9,k3,tegra,ls1,ls2,lx216,mvebu,uniphier,renesas,sunxi,samsung,socfpga,rk,versal,zynq" - rk_non_rockchip_64bit: - BUILDMAN: "rk&aarch64 -x rockchip" - rk_rockchip_64bit: - BUILDMAN: "rk&aarch64&rockchip" - zynq_zynqmp_versal: - BUILDMAN: "zynq&armv7 versal zynqmp&aarch64" - riscv: - BUILDMAN: "riscv" + BUILDMAN: "aarch64 -x amlogic,bcm,engicam,imx,ls1,ls2,lx216,mvebu,rk,siemens,sunxi,toradex" + everything_but_arm_and_powerpc: + BUILDMAN: "-x arm,powerpc" steps: - script: | cat << EOF > build.sh
Signature · 1 line
2.34.1
Re: [PATCH 2/4] Azure: Rework build the world jobs
Quoted 13 lines · Tom Rini
Now that we have 3600 minutes per build job, condense and rework things such that our overall time largely doesn't change, but we can also largely avoid having to re-tweak this job to avoid timeouts. Given that we have 10 threads, we also move a few of the specific sandbox test builds to a prior stage.
Note that while sandbox builds with address sanitization enabled (ASAN) not all tests pass, so we limit ourselves to just checking that the version test passes for now.
Quoted 8 lines
---
Cc: Simon Glass <[email protected]>I believe we merged ASAN with the intention of fixing some of the found issues later, so this isn't really a functional change.
--- .azure-pipelines.yml | 100 +++++++++++-------------------------------- 1 file changed, 25 insertions(+), 75 deletions(-)
Re: [PATCH 2/4] Azure: Rework build the world jobs
In reply to Tom RiniQuoted 6 lines · Tom Rini
Now that we have 3600 minutes per build job, condense and rework things such that our overall time largely doesn't change, but we can also largely avoid having to re-tweak this job to avoid timeouts. Given that we have 10 threads, we also move a few of the specific sandbox test builds to a prior stage.
Quoted 7 lines
Note that while sandbox builds with address sanitization enabled (ASAN) not all tests pass, so we limit ourselves to just checking that the version test passes for now.
Applied to u-boot/next, thanks!
Signature · 1 line
Tom
PATCH 3/4 CI: Combine tools-only and envtools jobs
In reply to Tom RiniThese jobs are to confirm specific build targets, on a Linux host. We can safely combine these two build tests, with a make mrproper in between.
--- .azure-pipelines.yml | 13 ++----------- .gitlab-ci.yml | 12 ++++-------- 2 files changed, 6 insertions(+), 19 deletions(-)
.azure-pipelines.yml +2 -11
@@ -126,7 +126,7 @@ stages: ./tools/buildman/buildman --maintainer-check || exit 0 - job: tools_only - displayName: 'Ensure host tools build' + displayName: 'Ensure host tools and env tools build' pool: vmImage: $(ubuntu_vm) container:
@@ -135,16 +135,7 @@ stages: steps: - script: | make tools-only_config tools-only -j$(nproc) - - - job: envtools - displayName: 'Ensure env tools build' - pool: - vmImage: $(ubuntu_vm) - container: - image: $(ci_runner_image) - options: $(container_option) - steps: - - script: | + make mrproper make tools-only_config envtools -j$(nproc) - job: utils
.gitlab-ci.yml +4 -8
@@ -190,16 +190,12 @@ Check for configs without MAINTAINERS entry: - ./tools/buildman/buildman --maintainer-check || exit 0 # Ensure host tools build -Build tools-only: +Build tools-only and envtools: stage: testsuites script: - - make tools-only_config tools-only -j$(nproc) - -# Ensure env tools build -Build envtools: - stage: testsuites - script: - - make tools-only_config envtools -j$(nproc) + - make tools-only_config tools-only -j$(nproc); + make mrproper; + make tools-only_config envtools -j$(nproc) Run binman, buildman, dtoc, Kconfig and patman testsuites: stage: testsuites
Signature · 1 line
2.34.1
Re: [PATCH 3/4] CI: Combine tools-only and envtools jobs
Quoted 10 lines · Tom Rini
These jobs are to confirm specific build targets, on a Linux host. We can safely combine these two build tests, with a make mrproper in between.
Signed-off-by Tom Rini <[email protected]>--- .azure-pipelines.yml | 13 ++----------- .gitlab-ci.yml | 12 ++++-------- 2 files changed, 6 insertions(+), 19 deletions(-)
Re: [PATCH 3/4] CI: Combine tools-only and envtools jobs
In reply to Tom RiniQuoted 5 lines · Tom Rini
These jobs are to confirm specific build targets, on a Linux host. We can safely combine these two build tests, with a make mrproper in between.
Signed-off-by Tom Rini <[email protected]>
Applied to u-boot/next, thanks!
Signature · 1 line
Tom
PATCH 4/4 CI: Drop some jobs we didn't really utilize
In reply to Tom Rini- We have added more TODO/etc comments since this task was created and never focused on removing them.
- The output of sloccount isn't preserved or looked at, and if desired should be in the release stats pages instead somehow.
- The results of cppcheck aren't investigated and require modeling work to be useful to start with.
--- .azure-pipelines.yml | 32 -------------------------------- .gitlab-ci.yml | 22 ---------------------- tools/docker/Dockerfile | 2 -- 3 files changed, 56 deletions(-)
.azure-pipelines.yml +0 -32
@@ -67,16 +67,6 @@ stages: :^doc/ :^arch/arm/dts/ :^scripts/kconfig/lkc.h :^include/linux/kconfig.h :^tools/ && exit 1 || exit 0 - - job: cppcheck - displayName: 'Static code analysis with cppcheck' - pool: - vmImage: $(ubuntu_vm) - container: - image: $(ci_runner_image) - options: $(container_option) - steps: - - script: cppcheck -j$(nproc) --force --quiet --inline-suppr . - - job: docs displayName: 'Build documentation' pool:
@@ -92,28 +82,6 @@ stages: make htmldocs KDOC_WERROR=1 make infodocs - - job: todo - displayName: 'Search for TODO within source tree' - pool: - vmImage: $(ubuntu_vm) - container: - image: $(ci_runner_image) - options: $(container_option) - steps: - - script: grep -r TODO . - - script: grep -r FIXME . - - script: grep -r HACK . | grep -v HACKKIT - - - job: sloccount - displayName: 'Some statistics about the code base' - pool: - vmImage: $(ubuntu_vm) - container: - image: $(ci_runner_image) - options: $(container_option) - steps: - - script: sloccount . - - job: maintainers displayName: 'Ensure all configs have MAINTAINERS entries' pool:
.gitlab-ci.yml +0 -22
@@ -151,22 +151,6 @@ check for new CONFIG symbols outside Kconfig: :^doc/ :^arch/arm/dts/ :^scripts/kconfig/lkc.h :^include/linux/kconfig.h :^tools/ && exit 1 || exit 0 -# QA jobs for code analytics -# static code analysis with cppcheck (we can add --enable=all later) -cppcheck: - stage: testsuites - script: - - cppcheck -j$(nproc) --force --quiet --inline-suppr . - -# search for TODO within source tree -grep TODO/FIXME/HACK: - stage: testsuites - script: - - grep -r TODO . - - grep -r FIXME . - # search for HACK within source tree and ignore HACKKIT board - - grep -r HACK . | grep -v HACKKIT - # build documentation docs: stage: testsuites
@@ -177,12 +161,6 @@ docs: - make htmldocs KDOC_WERROR=1 - make infodocs -# some statistics about the code base -sloccount: - stage: testsuites - script: - - sloccount . - # ensure all configs have MAINTAINERS entries Check for configs without MAINTAINERS entry: stage: testsuites
tools/docker/Dockerfile +0 -2
@@ -42,7 +42,6 @@ RUN apt-get update && apt-get install -y \ clang-16 \ coreutils \ cpio \ - cppcheck \ curl \ device-tree-compiler \ dosfstools \
@@ -104,7 +103,6 @@ RUN apt-get update && apt-get install -y \ python3-virtualenv \ rpm2cpio \ sbsigntool \ - sloccount \ socat \ softhsm2 \ sparse \
Signature · 1 line
2.34.1
Re: [PATCH 4/4] CI: Drop some jobs we didn't really utilize
Quoted 8 lines · Tom Rini
- We have added more TODO/etc comments since this task was created and never focused on removing them.
- The output of sloccount isn't preserved or looked at, and if desired should be in the release stats pages instead somehow.
- The results of cppcheck aren't investigated and require modeling work to be useful to start with.
Quoted 6 lines
Signed-off-by Tom Rini <[email protected]>--- .azure-pipelines.yml | 32 -------------------------------- .gitlab-ci.yml | 22 ---------------------- tools/docker/Dockerfile | 2 -- 3 files changed, 56 deletions(-)
Re: [PATCH 4/4] CI: Drop some jobs we didn't really utilize
In reply to Tom RiniQuoted 9 lines · Tom Rini
- We have added more TODO/etc comments since this task was created and never focused on removing them.
- The output of sloccount isn't preserved or looked at, and if desired should be in the release stats pages instead somehow.
- The results of cppcheck aren't investigated and require modeling work to be useful to start with.Signed-off-by Tom Rini <[email protected]>Reviewed-by Simon Glass <[email protected]>
Applied to u-boot/next, thanks!
Signature · 1 line
Tom
Quoted 7 lines · Tom Rini
As per current Azure Pipelines documentation we qualify for 3600 minutes per job, if specified, as the timeout. The default unspecified timeout is 60 minutes. Rework things to specify 0 as the timeout (and so maximum allowed) so that we don't have failures due to running slightly past 60 minutes total.
Quoted 5 lines
Signed-off-by Tom Rini <[email protected]>--- .azure-pipelines.yml | 1 + 1 file changed, 1 insertion(+)
Quoted 9 lines · Tom Rini
As per current Azure Pipelines documentation we qualify for 3600 minutes per job, if specified, as the timeout. The default unspecified timeout is 60 minutes. Rework things to specify 0 as the timeout (and so maximum allowed) so that we don't have failures due to running slightly past 60 minutes total.
Applied to u-boot/master, thanks!
Signature · 1 line
Tom