Bring all testings in gitlab and travis CI to Azure Pipelines
Subsystems (4)
v2 cover Bring all testings in gitlab and travis CI to Azure Pipelines
At present we have set up a Microsoft Azure Pipelines to build U-Boot host tools for Windows. We can expand it to cover all the CI testing in gitlab and travis CI.
There are issues if we use the 'container' keyword in the pipeline, that buildman seems to not able to exit correctly and hangs forever. So we manually call docker to run the image to perform the CI tasks. Unfortunately with above configuration, the uboot user inside the container does not have the write permission to the U-Boot git repo, and this exposes several issues in the U-Boot build process.
With all such issues fixed, a complete run on Azure Pipelines takes about 2 hours and 10 minutes. It seems to be a good backup for the free travis CI service.
This series depends on
http://patchwork.ozlabs.org/project/uboot/list/?series=138998
and is avaiable at u-boot-x86/azure for testing.
Changes in v2:
- remove the unneeded debug codes
Bin Meng (5):
tools: buildman: Honor output directory when generating boards.cfg
tools: buildman: Remove useless mkdir() in Make() in test.py
arm: mvebu: Avoid generating kwbimage.cfg in the source tree .travis.yml: Remove the unneeded '&' for ls20xx buildman Bring all testings in gitlab and travis CI to Azure Pipelines
.azure-pipelines.yml | 382 +++++++++++++++++++++++++++++++++++ .travis.yml | 2 +- Makefile | 15 +- arch/arm/mach-mvebu/Makefile | 4 +- board/Marvell/db-88f6281-bp/Makefile | 4 +- board/Marvell/db-xc3-24g4xg/Makefile | 4 +- board/mikrotik/crs305-1g-4s/Makefile | 4 +- tools/buildman/control.py | 10 +- tools/buildman/test.py | 8 - 9 files changed, 409 insertions(+), 24 deletions(-)
Signature · 1 line
2.7.4
PATCH v2 1/5 tools: buildman: Honor output directory when generating boards.cfg
buildman always generates boards.cfg in the U-Boot source tree. When '-o' is given, we should generate boards.cfg to the given output directory.
---
Changes in v2: None
tools/buildman/control.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tools/buildman/control.py +5 -5
@@ -201,14 +201,14 @@ def DoBuildman(options, args, toolchains=None, make_func=None, boards=None, # Work out what subset of the boards we are building if not boards: - board_file = os.path.join(options.git, 'boards.cfg') - status = subprocess.call([os.path.join(options.git, - 'tools/genboardscfg.py')]) + board_file = os.path.join(options.output_dir, 'boards.cfg') + genboardscfg = os.path.join(options.git, 'tools/genboardscfg.py') + status = subprocess.call([genboardscfg, '-o', board_file]) if status != 0: - sys.exit("Failed to generate boards.cfg") + sys.exit("Failed to generate boards.cfg") boards = board.Boards() - boards.ReadBoards(os.path.join(options.git, 'boards.cfg')) + boards.ReadBoards(board_file) exclude = [] if options.exclude:
Signature · 1 line
2.7.4
PATCH v2 1/5 tools: buildman: Honor output directory when generating boards.cfg
Hi Bin,
Quoted 5 lines · Bin Meng
buildman always generates boards.cfg in the U-Boot source tree. When '-o' is given, we should generate boards.cfg to the given output directory.
Quoted 8 lines
Signed-off-by Bin Meng <[email protected]>---
Changes in v2: None
tools/buildman/control.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
Does this preserve existing behaviour when -o is not given? I suppose it doesn't matter so long as buildman uses the right file?
Quoted 25 lines
tools/buildman/control.py +5
-6@@ -201,14 +201,14 @@ def DoBuildman(options, args, toolchains=None, make_func=None, boards=None, # Work out what subset of the boards we are building if not boards: - board_file = os.path.join(options.git, 'boards.cfg') - status = subprocess.call([os.path.join(options.git, - 'tools/genboardscfg.py')]) + board_file = os.path.join(options.output_dir, 'boards.cfg') + genboardscfg = os.path.join(options.git, 'tools/genboardscfg.py') + status = subprocess.call([genboardscfg, '-o', board_file]) if status != 0: - sys.exit("Failed to generate boards.cfg") + sys.exit("Failed to generate boards.cfg") boards = board.Boards() - boards.ReadBoards(os.path.join(options.git, 'boards.cfg')) + boards.ReadBoards(board_file) exclude = [] if options.exclude: --
Quoted 2 lines
2.7.4
Regards,
Simon
PATCH v2 1/5 tools: buildman: Honor output directory when generating boards.cfg
Hi Simon,
Quoted 14 lines · Simon Glass
Hi Bin,
Quoted 10 lines · Bin Meng
buildman always generates boards.cfg in the U-Boot source tree. When '-o' is given, we should generate boards.cfg to the given output directory.
Signed-off-by Bin Meng <[email protected]>---
Changes in v2: None
Quoted 7 lines
Quoted 3 lines
tools/buildman/control.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)Reviewed-by Simon Glass <[email protected]>Does this preserve existing behaviour when -o is not given? I suppose
When -o is not given, the options.output_dir is the destination directory and it's "../" by default.
But I suspect we should change the default output_dir to somewhere else, in a separate patch, because arbitrarily put something in the parent directory is not a good practice.
Quoted 1 line
it doesn't matter so long as buildman uses the right file?
Yes, it doesn't matter.
Regards,
Bin
PATCH v2 1/5 tools: buildman: Honor output directory when generating boards.cfg
In reply to Bin MengQuoted 6 lines · Bin Meng
buildman always generates boards.cfg in the U-Boot source tree.
When '-o' is given, we should generate boards.cfg to the given output directory.
Signed-off-by Bin Meng <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20191030/72ee6f92/attachment.sig>
PATCH v2 2/5 tools: buildman: Remove useless mkdir() in Make() in test.py
In reply to Bin MengIn the 'Make' function, the codes tries to create a directory if current stage is 'build'. But the directory isn't used at all anywhere.
---
Changes in v2:
- remove the unneeded debug codes
tools/buildman/test.py | 8 -------- 1 file changed, 8 deletions(-)
tools/buildman/test.py +0 -8
@@ -156,14 +156,6 @@ class TestBuild(unittest.TestCase): result.return_code = commit.return_code result.stderr = (''.join(commit.error_list) % {'basedir' : base_dir + '/.bm-work/00/'}) - if stage == 'build': - target_dir = None - for arg in args: - if arg.startswith('O='): - target_dir = arg[2:] - - if not os.path.isdir(target_dir): - os.mkdir(target_dir) result.combined = result.stdout + result.stderr return result
Signature · 1 line
2.7.4
PATCH v2 2/5 tools: buildman: Remove useless mkdir() in Make() in test.py
Hi Bin,
Quoted 7 lines · Bin Meng
In the 'Make' function, the codes tries to create a directory if current stage is 'build'. But the directory isn't used at all anywhere.
Signed-off-by Bin Meng <[email protected]>
Quoted 7 lines
---
Changes in v2:
- remove the unneeded debug codestools/buildman/test.py | 8 -------- 1 file changed, 8 deletions(-)
Can you check this works OK when there is no output dir at all? E.g.
-o /path/to/a/deep/non-existent/dir
Regards,
Simon
PATCH v2 2/5 tools: buildman: Remove useless mkdir() in Make() in test.py
Hi Simon,
Quoted 16 lines · Simon Glass
Hi Bin,
Quoted 12 lines · Bin Meng
In the 'Make' function, the codes tries to create a directory if current stage is 'build'. But the directory isn't used at all anywhere.
Signed-off-by Bin Meng <[email protected]>---
Changes in v2:
- remove the unneeded debug codes
Quoted 6 lines
Quoted 2 lines
tools/buildman/test.py | 8 -------- 1 file changed, 8 deletions(-)Can you check this works OK when there is no output dir at all? E.g.
-o /path/to/a/deep/non-existent/dir
If I understand the codes correctly the "O=<path>" in args[] does not come from "-o /path/to/a/deep/non-existent/dir" but from the builderthread with some hardcoded values. The removed codes here seem to do something not very useful.
Quoted 2 lines
Reviewed-by Simon Glass <[email protected]>
Regards,
Bin
PATCH v2 2/5 tools: buildman: Remove useless mkdir() in Make() in test.py
Hi Bin,
Quoted 23 lines · Bin Meng
Hi Simon,
Quoted 19 lines · Simon Glass
Hi Bin,
Quoted 14 lines · Bin Meng
In the 'Make' function, the codes tries to create a directory if current stage is 'build'. But the directory isn't used at all anywhere.
Signed-off-by Bin Meng <[email protected]>---
Changes in v2:
- remove the unneeded debug codestools/buildman/test.py | 8 -------- 1 file changed, 8 deletions(-)
Quoted 8 lines
Quoted 3 lines
Can you check this works OK when there is no output dir at all? E.g.
-o /path/to/a/deep/non-existent/dir
If I understand the codes correctly the "O=<path>" in args[] does not come from "-o /path/to/a/deep/non-existent/dir" but from the builderthread with some hardcoded values. The removed codes here seem to do something not very useful.
That's fine, and TBH if we hit a problem later due to this we should add a unit test to cover it.
Regards,
Simon
PATCH v2 2/5 tools: buildman: Remove useless mkdir() in Make() in test.py
In reply to Bin MengQuoted 5 lines · Bin Meng
In the 'Make' function, the codes tries to create a directory if current stage is 'build'. But the directory isn't used at all anywhere.
Signed-off-by Bin Meng <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20191030/49ec7f89/attachment.sig>
PATCH v2 3/5 arm: mvebu: Avoid generating kwbimage.cfg in the source tree
In reply to Bin MengAt present some boards generate kwbimage.cfg in the source tree during the build. This breaks buildman testing on some systems where the source tree is read-only. Update makefile rules to generate it in the build tree instead.
Note some other boards have the kwbimage.cfg file written in advance, hence we need check if the file exists in the build tree first, otherwise we fall back to one in the source tree.
---
Changes in v2: None
Makefile | 15 +++++++++++++-- arch/arm/mach-mvebu/Makefile | 4 ++-- board/Marvell/db-88f6281-bp/Makefile | 4 ++-- board/Marvell/db-xc3-24g4xg/Makefile | 4 ++-- board/mikrotik/crs305-1g-4s/Makefile | 4 ++-- 5 files changed, 21 insertions(+), 10 deletions(-)
Makefile +13 -2
@@ -1276,10 +1276,21 @@ endif MKIMAGEFLAGS_u-boot-dtb.img = $(MKIMAGEFLAGS_u-boot.img) -MKIMAGEFLAGS_u-boot.kwb = -n $(srctree)/$(CONFIG_SYS_KWD_CONFIG:"%"=%) \ +# Some boards have the kwbimage.cfg file written in advance, while some +# other boards generate it on the fly during the build in the build tree. +# Let's check if the file exists in the build tree first, otherwise we +# fall back to use the one in the source tree. +KWD_CONFIG_FILE = $(shell \ + if [ -f $(objtree)/$(CONFIG_SYS_KWD_CONFIG:"%"=%) ]; then \ + echo -n $(objtree)/$(CONFIG_SYS_KWD_CONFIG:"%"=%); \ + else \ + echo -n $(srctree)/$(CONFIG_SYS_KWD_CONFIG:"%"=%); \ + fi) + +MKIMAGEFLAGS_u-boot.kwb = -n $(KWD_CONFIG_FILE) \ -T kwbimage -a $(CONFIG_SYS_TEXT_BASE) -e $(CONFIG_SYS_TEXT_BASE) -MKIMAGEFLAGS_u-boot-spl.kwb = -n $(srctree)/$(CONFIG_SYS_KWD_CONFIG:"%"=%) \ +MKIMAGEFLAGS_u-boot-spl.kwb = -n $(KWD_CONFIG_FILE) \ -T kwbimage -a $(CONFIG_SYS_TEXT_BASE) -e $(CONFIG_SYS_TEXT_BASE) \ $(if $(KEYDIR),-k $(KEYDIR))
arch/arm/mach-mvebu/Makefile +2 -2
@@ -58,10 +58,10 @@ KWB_REPLACE += SEC_FUSE_DUMP KWB_CFG_SEC_FUSE_DUMP = a38x endif -$(src)/kwbimage.cfg: $(src)/kwbimage.cfg.in include/autoconf.mk \ +$(obj)/kwbimage.cfg: $(src)/kwbimage.cfg.in include/autoconf.mk \ include/config/auto.conf $(Q)sed -ne '$(foreach V,$(KWB_REPLACE),s/^#@$(V)/$(V) $(KWB_CFG_$(V))/;)p' \ - <$< >$(dir $<)$(@F) + <$< >$(dir $@)$(@F) endif # CONFIG_SPL_BUILD obj-y += gpio.o
board/Marvell/db-88f6281-bp/Makefile +2 -2
@@ -4,9 +4,9 @@ obj-y := db-88f6281-bp.o extra-y := kwbimage.cfg quiet_cmd_sed = SED $@ - cmd_sed = sed $(SEDFLAGS_$(@F)) $< >$(dir $<)$(@F) + cmd_sed = sed $(SEDFLAGS_$(@F)) $< >$(dir $@)$(@F) SEDFLAGS_kwbimage.cfg = -e "s/^\#@BOOT_FROM.*/BOOT_FROM $(if $(CONFIG_CMD_NAND),nand,spi)/" -$(src)/kwbimage.cfg: $(src)/kwbimage.cfg.in include/autoconf.mk \ +$(obj)/kwbimage.cfg: $(src)/kwbimage.cfg.in include/autoconf.mk \ include/config/auto.conf $(call if_changed,sed)
board/Marvell/db-xc3-24g4xg/Makefile +2 -2
@@ -6,9 +6,9 @@ obj-y := db-xc3-24g4xg.o extra-y := kwbimage.cfg quiet_cmd_sed = SED $@ - cmd_sed = sed $(SEDFLAGS_$(@F)) $< >$(dir $<)$(@F) + cmd_sed = sed $(SEDFLAGS_$(@F)) $< >$(dir $@)$(@F) SEDFLAGS_kwbimage.cfg =-e "s|^BINARY.*|BINARY $(srctree)/$(@D)/binary.0 0000005b 00000068|" -$(src)/kwbimage.cfg: $(src)/kwbimage.cfg.in include/autoconf.mk \ +$(obj)/kwbimage.cfg: $(src)/kwbimage.cfg.in include/autoconf.mk \ include/config/auto.conf $(call if_changed,sed)
board/mikrotik/crs305-1g-4s/Makefile +2 -2
@@ -6,9 +6,9 @@ obj-y := crs305-1g-4s.o extra-y := kwbimage.cfg quiet_cmd_sed = SED $@ - cmd_sed = sed $(SEDFLAGS_$(@F)) $< >$(dir $<)$(@F) + cmd_sed = sed $(SEDFLAGS_$(@F)) $< >$(dir $@)$(@F) SEDFLAGS_kwbimage.cfg =-e "s|^BINARY.*|BINARY $(srctree)/$(@D)/binary.0 0000005b 00000068|" -$(src)/kwbimage.cfg: $(src)/kwbimage.cfg.in include/autoconf.mk \ +$(obj)/kwbimage.cfg: $(src)/kwbimage.cfg.in include/autoconf.mk \ include/config/auto.conf $(call if_changed,sed)
Signature · 1 line
2.7.4
PATCH v2 3/5 arm: mvebu: Avoid generating kwbimage.cfg in the source tree
Quoted 15 lines · Bin Meng
At present some boards generate kwbimage.cfg in the source tree during the build. This breaks buildman testing on some systems where the source tree is read-only. Update makefile rules to generate it in the build tree instead.
Note some other boards have the kwbimage.cfg file written in advance, hence we need check if the file exists in the build tree first, otherwise we fall back to one in the source tree.
Signed-off-by Bin Meng <[email protected]>Reviewed-by Chris Packham <[email protected]>Reviewed-by Stefan Roese <[email protected]>---
Quoted 9 lines
Changes in v2: None
Makefile | 15 +++++++++++++-- arch/arm/mach-mvebu/Makefile | 4 ++-- board/Marvell/db-88f6281-bp/Makefile | 4 ++-- board/Marvell/db-xc3-24g4xg/Makefile | 4 ++-- board/mikrotik/crs305-1g-4s/Makefile | 4 ++-- 5 files changed, 21 insertions(+), 10 deletions(-)
PATCH v2 3/5 arm: mvebu: Avoid generating kwbimage.cfg in the source tree
In reply to Bin MengQuoted 5 lines · Bin Meng
At present some boards generate kwbimage.cfg in the source tree during the build. This breaks buildman testing on some systems where the source tree is read-only. Update makefile rules to generate it in the build tree instead.
Quoted 7 lines
Note some other boards have the kwbimage.cfg file written in advance, hence we need check if the file exists in the build tree first, otherwise we fall back to one in the source tree.
Signed-off-by Bin Meng <[email protected]>Reviewed-by Chris Packham <[email protected]>Reviewed-by Stefan Roese <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20191030/bb719ffb/attachment.sig>
PATCH v2 4/5 .travis.yml: Remove the unneeded '&' for ls20xx buildman
In reply to Bin Meng---
Changes in v2: None
.travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
.travis.yml +1 -1
@@ -200,7 +200,7 @@ matrix: - BUILDMAN="freescale&aarch64&ls108" - name: "buildman NXP AArch64 LS20xx" env: - - BUILDMAN="freescale&aarch64&&ls20" + - BUILDMAN="freescale&aarch64&ls20" - name: "buildman NXP AArch64 LX216x" env: - BUILDMAN="freescale&aarch64&lx216"
Signature · 1 line
2.7.4
PATCH v2 4/5 .travis.yml: Remove the unneeded '&' for ls20xx buildman
Quoted 10 lines · Bin Meng
Signed-off-by Bin Meng <[email protected]>Reviewed-by Tom Rini <[email protected]>---
Changes in v2: None
.travis.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 1 line
.travis.yml +0
-0
PATCH v2 4/5 .travis.yml: Remove the unneeded '&' for ls20xx buildman
In reply to Bin MengQuoted 2 lines · Bin Meng
Signed-off-by Bin Meng <[email protected]>Reviewed-by Tom Rini <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20191030/f2f277ae/attachment.sig>
PATCH v2 5/5 Bring all testings in gitlab and travis CI to Azure Pipelines
In reply to Bin MengThis expands current Azure Pipelines Windows host tools build testing to cover all the CI testing in gitlab and travis CI.
Note for some unknown reason, the 'container' cannot be used for any jobs that have buildman, for buildman does not exit properly and hangs the job forever. As a workaround, we manually call docker to run the image to perform the CI tasks.
A complete run on Azure Pipelines takes about 2 hours and 10 minutes.
---
See the build result at:
https://dev.azure.com/bmeng/GitHub/_build/results?buildId=109
Changes in v2: None
.azure-pipelines.yml | 382 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 382 insertions(+)
.azure-pipelines.yml +382 -0
@@ -1,5 +1,12 @@ variables: windows_vm: vs2015-win2012r2 + ubuntu_vm: ubuntu-18.04 + ci_runner_image: trini/u-boot-gitlab-ci-runner:bionic-20190912.1-03Oct2019 + # Add '-u 0' options for Azure pipelines, otherwise we get "permission + # denied" error when it tries to "useradd -m -u 1001 vsts_azpcontainer", + # since our $(ci_runner_image) user is not root. + container_option: -u 0 + work_dir: /u jobs: - job: tools_only_windows
@@ -36,3 +43,378 @@ jobs: MSYSTEM: MSYS # Tell MSYS2 not to ‘cd’ our startup directory to HOME CHERE_INVOKING: yes + + - job: cppcheck + displayName: 'Static code analysis with cppcheck' + pool: + vmImage: $(ubuntu_vm) + container: + image: $(ci_runner_image) + options: $(container_option) + steps: + - script: cppcheck --force --quiet --inline-suppr . + + - job: todo + displayName: 'Search for TODO within source tree' + pool: + vmImage: $(ubuntu_vm) + container: + image: $(ci_runner_image) + options: $(container_option) + steps: + - script: grep -r TODO . + - script: grep -r FIXME . + - script: grep -r HACK . | grep -v HACKKIT + + - job: sloccount + displayName: 'Some statistics about the code base' + pool: + vmImage: $(ubuntu_vm) + container: + image: $(ci_runner_image) + options: $(container_option) + steps: + - script: sloccount . + + - job: maintainers + displayName: 'Ensure all configs have MAINTAINERS entries' + pool: + vmImage: $(ubuntu_vm) + container: + image: $(ci_runner_image) + options: $(container_option) + steps: + - script: | + if [ `./tools/genboardscfg.py -f 2>&1 | wc -l` -ne 0 ]; then exit 1; fi + + - job: tools_only + displayName: 'Ensure host tools build' + pool: + vmImage: $(ubuntu_vm) + container: + image: $(ci_runner_image) + options: $(container_option) + steps: + - script: | + make tools-only_config tools-only -j$(nproc) + + - job: envtools + displayName: 'Ensure env tools build' + pool: + vmImage: $(ubuntu_vm) + container: + image: $(ci_runner_image) + options: $(container_option) + steps: + - script: | + make tools-only_config envtools -j$(nproc) + + - job: utils + displayName: 'Run binman, buildman, dtoc and patman testsuites' + pool: + vmImage: $(ubuntu_vm) + steps: + - script: | + cat << EOF > build.sh + set -ex + cd ${WORK_DIR} + EOF + cat << "EOF" >> build.sh + git config --global user.name "Azure Pipelines" + git config --global user.email bmeng.cn at gmail.com + export USER=azure + virtualenv /tmp/venv + . /tmp/venv/bin/activate + pip install pyelftools + export UBOOT_TRAVIS_BUILD_DIR=/tmp/.bm-work/sandbox_spl + export PYTHONPATH=${UBOOT_TRAVIS_BUILD_DIR}/scripts/dtc/pylibfdt + export PATH=${UBOOT_TRAVIS_BUILD_DIR}/scripts/dtc:${PATH} + ./tools/buildman/buildman -o /tmp -P sandbox_spl + ./tools/binman/binman --toolpath ${UBOOT_TRAVIS_BUILD_DIR}/tools test + ./tools/buildman/buildman -t + ./tools/dtoc/dtoc -t + ./tools/patman/patman --test + EOF + cat build.sh + # We cannot use "container" like other jobs above, as buildman + # seems to hang forever with pre-configured "container" environment + docker run -v $PWD:$(work_dir) $(ci_runner_image) /bin/bash $(work_dir)/build.sh + + - job: test_py + displayName: 'test.py' + pool: + vmImage: $(ubuntu_vm) + strategy: + matrix: + sandbox: + TEST_PY_BD: "sandbox" + BUILDMAN: "^sandbox$" + sandbox_spl: + TEST_PY_BD: "sandbox_spl" + TEST_PY_TEST_SPEC: "test_ofplatdata" + BUILDMAN: "^sandbox_spl$" + sandbox_flattree: + TEST_PY_BD: "sandbox_flattree" + BUILDMAN: "^sandbox_flattree$" + evb_ast2500: + TEST_PY_BD: "evb-ast2500" + TEST_PY_ID: "--id qemu" + BUILDMAN: "^evb-ast2500$" + vexpress_ca15_tc2: + TEST_PY_BD: "vexpress_ca15_tc2" + TEST_PY_ID: "--id qemu" + BUILDMAN: "^vexpress_ca15_tc2$" + vexpress_ca9x4: + TEST_PY_BD: "vexpress_ca9x4" + TEST_PY_ID: "--id qemu" + BUILDMAN: "^vexpress_ca9x4$" + integratorcp_cm926ejs: + TEST_PY_BD: "integratorcp_cm926ejs" + TEST_PY_ID: "--id qemu" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^integratorcp_cm926ejs$" + qemu_arm: + TEST_PY_BD: "qemu_arm" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^qemu_arm$" + qemu_arm64: + TEST_PY_BD: "qemu_arm64" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^qemu_arm64$" + qemu_mips: + TEST_PY_BD: "qemu_mips" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^qemu_mips$" + qemu_mipsel: + TEST_PY_BD: "qemu_mipsel" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^qemu_mipsel$" + qemu_mips64: + TEST_PY_BD: "qemu_mips64" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^qemu_mips64$" + qemu_mips64el: + TEST_PY_BD: "qemu_mips64el" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^qemu_mips64el$" + qemu_ppce500: + TEST_PY_BD: "qemu-ppce500" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^qemu-ppce500$" + qemu_riscv64: + TEST_PY_BD: "qemu-riscv64" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^qemu-riscv64$" + qemu_x86: + TEST_PY_BD: "qemu-x86" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^qemu-x86$" + qemu_x86_64: + TEST_PY_BD: "qemu-x86_64" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^qemu-x86_64$" + zynq_zc702: + TEST_PY_BD: "zynq_zc702" + TEST_PY_ID: "--id qemu" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^zynq_zc702$" + xilinx_versal_virt: + TEST_PY_BD: "xilinx_versal_virt" + TEST_PY_ID: "--id qemu" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^xilinx_versal_virt$" + xtfpga: + TEST_PY_BD: "xtfpga" + TEST_PY_ID: "--id qemu" + TEST_PY_TEST_SPEC: "not sleep" + BUILDMAN: "^xtfpga$" + steps: + - script: | + cat << EOF > test.sh + set -ex + # make environment variables available as tests are running inside a container + export WORK_DIR="${WORK_DIR}" + export TEST_PY_BD="${TEST_PY_BD}" + export TEST_PY_ID="${TEST_PY_ID}" + export TEST_PY_TEST_SPEC="${TEST_PY_TEST_SPEC}" + export BUILDMAN="${BUILDMAN}" + EOF + cat << "EOF" >> test.sh + # the below corresponds to .gitlab-ci.yml "before_script" + cd ${WORK_DIR} + git clone --depth=1 git://github.com/swarren/uboot-test-hooks.git /tmp/uboot-test-hooks + ln -s travis-ci /tmp/uboot-test-hooks/bin/`hostname` + ln -s travis-ci /tmp/uboot-test-hooks/py/`hostname` + virtualenv /tmp/venv + . /tmp/venv/bin/activate + pip install pytest==2.8.7 + pip install python-subunit + pip install coverage + grub-mkimage --prefix=\"\" -o ~/grub_x86.efi -O i386-efi normal echo lsefimmap lsefi lsefisystab efinet tftp minicmd + grub-mkimage --prefix=\"\" -o ~/grub_x64.efi -O x86_64-efi normal echo lsefimmap lsefi lsefisystab efinet tftp minicmd + mkdir ~/grub2-arm + cd ~/grub2-arm; wget -O - http://download.opensuse.org/ports/armv7hl/distribution/leap/42.2/repo/oss/suse/armv7hl/grub2-arm-efi-2.02~beta2-87.1.armv7hl.rpm | rpm2cpio | cpio -di + mkdir ~/grub2-arm64 + cd ~/grub2-arm64; wget -O - http://download.opensuse.org/ports/aarch64/distribution/leap/42.2/repo/oss/suse/aarch64/grub2-arm64-efi-2.02~beta2-87.1.aarch64.rpm | rpm2cpio | cpio -di + # the below corresponds to .gitlab-ci.yml "script" + cd ${WORK_DIR} + if [[ "${BUILDMAN}" != "" ]]; then + ret=0; + tools/buildman/buildman -o /tmp -P -E ${BUILDMAN} ${OVERRIDE} || ret=$?; + if [[ $ret -ne 0 && $ret -ne 129 ]]; then + tools/buildman/buildman -o /tmp -sdeP ${BUILDMAN}; + exit $ret; + fi; + fi + export UBOOT_TRAVIS_BUILD_DIR=/tmp/.bm-work/${TEST_PY_BD}; + export PATH=/opt/qemu/bin:/tmp/uboot-test-hooks/bin:/usr/bin:/bin; + export PYTHONPATH=/tmp/uboot-test-hooks/py/travis-ci; + if [[ "${TEST_PY_BD}" != "" ]]; then + ./test/py/test.py --bd ${TEST_PY_BD} ${TEST_PY_ID} -k "${TEST_PY_TEST_SPEC:-not a_test_which_does_not_exist}" --build-dir "$UBOOT_TRAVIS_BUILD_DIR"; + ret=$?; + if [[ $ret -ne 0 ]]; then + exit $ret; + fi; + fi + # the below corresponds to .gitlab-ci.yml "after_script" + rm -rf ~/grub2* /tmp/uboot-test-hooks /tmp/venv + EOF + cat test.sh + # make current directory writeable to uboot user inside the container + # as sandbox testing need create files like spi flash images, etc. + # (TODO: clean up this in the future) + chmod 777 . + docker run -v $PWD:$(work_dir) $(ci_runner_image) /bin/bash $(work_dir)/test.sh + + - job: build_the_world + displayName: 'Build the World' + pool: + vmImage: $(ubuntu_vm) + strategy: + # Use almost the same target division in .travis.yml, only merged + # 4 small build jobs (arc/microblaze/nds32/xtensa) into one. + matrix: + arc_microblaze_nds32_xtensa: + BUILDMAN: "arc microblaze nds32 xtensa" + arm11_arm7_arm920t_arm946es: + BUILDMAN: "arm11 arm7 arm920t arm946es" + arm926ejs: + BUILDMAN: "arm926ejs -x freescale,siemens,at91,kirkwood,spear,omap" + at91_non_armv7: + BUILDMAN: "at91 -x armv7" + at91_non_arm926ejs: + BUILDMAN: "at91 -x arm926ejs" + boundary_engicam_toradex: + BUILDMAN: "boundary engicam toradex" + arm_bcm: + BUILDMAN: "bcm -x mips" + nxp_arm32: + BUILDMAN: "freescale -x powerpc,m68k,aarch64" + nxp_aarch64_ls101x: + BUILDMAN: "freescale&aarch64&ls101" + nxp_aarch64_ls102x: + BUILDMAN: "freescale&aarch64&ls102" + nxp_aarch64_ls104x: + BUILDMAN: "freescale&aarch64&ls104" + nxp_aarch64_ls108x: + BUILDMAN: "freescale&aarch64&ls108" + nxp_aarch64_ls20xx: + BUILDMAN: "freescale&aarch64&ls20" + nxp_aarch64_lx216x: + BUILDMAN: "freescale&aarch64&lx216" + imx6: + BUILDMAN: "mx6 -x boundary,engicam,freescale,technexion,toradex" + imx: + BUILDMAN: "mx -x mx6,freescale,technexion,toradex" + keystone2_keystone3: + BUILDMAN: "k2 k3" + samsung_socfpga: + BUILDMAN: "samsung socfpga" + spear: + BUILDMAN: "spear" + sun4i: + BUILDMAN: "sun4i" + sun5i: + BUILDMAN: "sun5i" + sun6i: + BUILDMAN: "sun6i" + sun7i: + BUILDMAN: "sun7i" + sun8i_32bit: + BUILDMAN: "sun8i&armv7" + sun8i_64bit: + BUILDMAN: "sun8i&aarch64" + sun9i: + BUILDMAN: "sun9i" + sun50i: + BUILDMAN: "sun50i" + arm_catch_all: + BUILDMAN: "arm -x arm11,arm7,arm9,aarch64,at91,bcm,freescale,kirkwood,mvebu,siemens,tegra,uniphier,mx,samsung,sunxi,am33xx,omap,rockchip,toradex,socfpga,k2,k3,zynq" + sandbox_x86: + BUILDMAN: "sandbox x86" + technexion: + BUILDMAN: "technexion" + kirkwood: + BUILDMAN: "kirkwood" + mvebu: + BUILDMAN: "mvebu" + m68k: + BUILDMAN: "m68k" + mips: + BUILDMAN: "mips" + non_fsl_ppc: + BUILDMAN: "powerpc -x freescale" + mpc85xx_freescale: + BUILDMAN: "mpc85xx&freescale -x t208xrdb -x t4qds -x t102* -x p1_p2_rdb_pc -x p1010rdb -x corenet_ds -x b4860qds -x bsc91*" + t208xrdb_corenet_ds: + BUILDMAN: "t208xrdb corenet_ds" + fsl_ppc: + BUILDMAN: "t4qds b4860qds mpc83xx&freescale mpc86xx&freescale" + t102x: + BUILDMAN: "t102*" + p1_p2_rdb_pc: + BUILDMAN: "p1_p2_rdb_pc" + p1010rdb_bsc91: + BUILDMAN: "p1010rdb bsc91" + siemens: + BUILDMAN: "siemens" + tegra: + BUILDMAN: "tegra -x toradex" + am33xx_no_siemens: + BUILDMAN: "am33xx -x siemens" + omap: + BUILDMAN: "omap" + uniphier: + BUILDMAN: "uniphier" + aarch64_catch_all: + BUILDMAN: "aarch64 -x bcm,k3,tegra,ls1,ls2,mvebu,uniphier,sunxi,samsung,rockchip,versal,zynq" + rockchip: + BUILDMAN: "rockchip" + sh: + BUILDMAN: "sh -x arm" + zynq: + BUILDMAN: "zynq&armv7" + zynqmp_versal: + BUILDMAN: "versal|zynqmp&aarch64" + riscv: + BUILDMAN: "riscv" + steps: + - script: | + cat << EOF > build.sh + set -ex + cd ${WORK_DIR} + # make environment variables available as tests are running inside a container + export BUILDMAN="${BUILDMAN}" + EOF + cat << "EOF" >> build.sh + if [[ "${BUILDMAN}" != "" ]]; then + ret=0; + tools/buildman/buildman -o /tmp -P -E ${BUILDMAN} ${OVERRIDE} || ret=$?; + if [[ $ret -ne 0 && $ret -ne 129 ]]; then + tools/buildman/buildman -o /tmp -sdeP ${BUILDMAN}; + exit $ret; + fi; + fi + EOF + cat build.sh + docker run -v $PWD:$(work_dir) $(ci_runner_image) /bin/bash $(work_dir)/build.sh
Signature · 1 line
2.7.4
PATCH v2 5/5 Bring all testings in gitlab and travis CI to Azure Pipelines
Quoted 8 lines · Bin Meng
This expands current Azure Pipelines Windows host tools build testing to cover all the CI testing in gitlab and travis CI.
Note for some unknown reason, the 'container' cannot be used for any jobs that have buildman, for buildman does not exit properly and hangs the job forever. As a workaround, we manually call docker to run the image to perform the CI tasks.
Quoted 4 lines
A complete run on Azure Pipelines takes about 2 hours and 10 minutes.
Signed-off-by Bin Meng <[email protected]>
Applied to u-boot/master, thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://lists.denx.de/pipermail/u-boot/attachments/20191030/4fd2f3a3/attachment.sig>