ci: azure: Update to use stages
PATCH 1/4 ci: azure: Update to use stages
Follow what we do in GitLab CI where we break the jobs up in to stages such that if earlier and often quicker sanity tests fail we don't run everything else.
--- .azure-pipelines.yml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
.azure-pipelines.yml +7 -1
@@ -9,7 +9,9 @@ variables: container_option: -u 0 work_dir: /u -jobs: +stages: +- stage: testsuites + jobs: - job: tools_only_windows displayName: 'Ensure host tools build for Windows' pool:
@@ -199,6 +201,8 @@ jobs: export PATH=/opt/gcc-11.1.0-nolibc/arm-linux-gnueabi/bin:$PATH test/nokia_rx51_test.sh +- stage: test_py + jobs: - job: test_py displayName: 'test.py' pool:
@@ -381,6 +385,8 @@ jobs: # Some tests using libguestfs-tools need the fuse device to run docker run "$@" --device /dev/fuse:/dev/fuse -v $PWD:$(work_dir) $(ci_runner_image) /bin/bash $(work_dir)/test.sh +- stage: world_build + jobs: - job: build_the_world displayName: 'Build the World' pool:
Signature · 1 line
2.25.1
PATCH 2/4 CI, Dockerfile: Update to latest "focal" tag
Bring us to the focial-20220105 tag and rebuild our images on top of this.
--- .azure-pipelines.yml | 2 +- .gitlab-ci.yml | 2 +- tools/docker/Dockerfile | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-)
.azure-pipelines.yml +1 -1
@@ -2,7 +2,7 @@ variables: windows_vm: windows-2019 ubuntu_vm: ubuntu-18.04 macos_vm: macOS-10.15 - ci_runner_image: trini/u-boot-gitlab-ci-runner:focal-20211006-14Nov2021 + ci_runner_image: trini/u-boot-gitlab-ci-runner:focal-20220105-10Jan2022 # Add '-u 0' options for Azure pipelines, otherwise we get "permission # denied" error when it tries to "useradd -m -u 1001 vsts_azpcontainer", # since our $(ci_runner_image) user is not root.
.gitlab-ci.yml +1 -1
@@ -2,7 +2,7 @@ # Grab our configured image. The source for this is found at: # https://source.denx.de/u-boot/gitlab-ci-runner -image: trini/u-boot-gitlab-ci-runner:focal-20211006-14Nov2021 +image: trini/u-boot-gitlab-ci-runner:focal-20220105-10Jan2022 # We run some tests in different order, to catch some failures quicker. stages:
tools/docker/Dockerfile +1 -1
@@ -2,7 +2,7 @@ # This Dockerfile is used to build an image containing basic stuff to be used # to build U-Boot and run our test suites. -FROM ubuntu:focal-20211006 +FROM ubuntu:focal-20220105 MAINTAINER Tom Rini <[email protected]> LABEL Description=" This image is for building U-Boot inside a container"
Signature · 1 line
2.25.1
Re: [PATCH 2/4] CI, Dockerfile: Update to latest "focal" tag
Quoted 2 lines · Tom Rini
Bring us to the focial-20220105 tag and rebuild our images on top of
s/focial/focal
Re: [PATCH 2/4] CI, Dockerfile: Update to latest "focal" tag
Quoted 5 lines · Fabio Estevam
Quoted 2 lines · Tom Rini
Bring us to the focial-20220105 tag and rebuild our images on top of
s/focial/focal
Whoops, thanks.
Signature · 1 line
Tom
Re: [PATCH 2/4] CI, Dockerfile: Update to latest "focal" tag
In reply to Tom RiniQuoted 4 lines · Tom Rini
Bring us to the focal-20220105 tag and rebuild our images on top of this.
Signed-off-by Tom Rini <[email protected]>
Applied to u-boot/master, thanks!
Signature · 1 line
Tom
PATCH 3/4 binman: Have faked binaries live in the output directory.
In reply to Tom RiniIn general, and for Azure specifically, we need to have files created in the output directory and cannot assume a writable source directory. Rework the faked blob support to put the faked binary in to the output directory and then stop the test from deleting the now non-existent file.
--- tools/binman/etype/blob.py | 4 +++- tools/binman/ftest.py | 1 - 2 files changed, 3 insertions(+), 2 deletions(-)
tools/binman/etype/blob.py +3 -1
@@ -39,11 +39,13 @@ class Entry_blob(Entry): def ObtainContents(self): if self.allow_fake and not pathlib.Path(self._filename).is_file(): + self._filename = tools.GetOutputFilename(self._filename) with open(self._filename, "wb") as out: out.truncate(1024) self.faked = True + else: + self._filename = self.GetDefaultFilename() - self._filename = self.GetDefaultFilename() self._pathname = tools.GetInputFilename(self._filename, self.external and self.section.GetAllowMissing()) # Allow the file to be missing
tools/binman/ftest.py +0 -1
@@ -4675,7 +4675,6 @@ class TestFunctional(unittest.TestCase): err = stderr.getvalue() self.assertRegex(err, "Image '.*' has faked external blobs and is non-functional: .*") - os.remove('binman_faking_test_blob') def testVersion(self): """Test we can get the binman version"""
Signature · 1 line
2.25.1
Re: [PATCH 3/4] binman: Have faked binaries live in the output directory.
Hi Tom,
Quoted 7 lines · Tom Rini
In general, and for Azure specifically, we need to have files created in the output directory and cannot assume a writable source directory. Rework the faked blob support to put the faked binary in to the output directory and then stop the test from deleting the now non-existent file.
Quoted 7 lines
Cc Heiko Thiery <[email protected]>Cc Simon Glass <[email protected]>Signed-off-by Tom Rini <[email protected]>--- tools/binman/etype/blob.py | 4 +++- tools/binman/ftest.py | 1 - 2 files changed, 3 insertions(+), 2 deletions(-)
Please see this:
https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/
Regards,
Simon
Re: [PATCH 3/4] binman: Have faked binaries live in the output directory.
Quoted 18 lines · Simon Glass
Hi Tom,
Quoted 7 lines · Tom Rini
In general, and for Azure specifically, we need to have files created in the output directory and cannot assume a writable source directory. Rework the faked blob support to put the faked binary in to the output directory and then stop the test from deleting the now non-existent file.
Quoted 7 lines
Cc Heiko Thiery <[email protected]>Cc Simon Glass <[email protected]>Signed-off-by Tom Rini <[email protected]>--- tools/binman/etype/blob.py | 4 +++- tools/binman/ftest.py | 1 - 2 files changed, 3 insertions(+), 2 deletions(-)
Quoted 3 lines
Please see this:
https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/
Ah, I thought you might have had it fixed but I didn't read well enough. My only concern is I would like to get Azure passing again ASAP. Should I just grab that patch by itself then?
Signature · 1 line
Tom
Re: [PATCH 3/4] binman: Have faked binaries live in the output directory.
Hi Tom,
Quoted 20 lines · Tom Rini
Quoted 18 lines · Simon Glass
Hi Tom,
Quoted 14 lines · Tom Rini
In general, and for Azure specifically, we need to have files created in the output directory and cannot assume a writable source directory. Rework the faked blob support to put the faked binary in to the output directory and then stop the test from deleting the now non-existent file.
Cc Heiko Thiery <[email protected]>Cc Simon Glass <[email protected]>Signed-off-by Tom Rini <[email protected]>--- tools/binman/etype/blob.py | 4 +++- tools/binman/ftest.py | 1 - 2 files changed, 3 insertions(+), 2 deletions(-)
Quoted 7 lines
Quoted 3 lines
Please see this:
https://patchwork.ozlabs.org/project/uboot/patch/[email protected]/
Ah, I thought you might have had it fixed but I didn't read well enough. My only concern is I would like to get Azure passing again ASAP. Should I just grab that patch by itself then?
Yes, but I just sent v2 rebased to master (two patches).
Regards,
Simon
PATCH 4/4 .readthedocs.yml: Set our requirements file
In reply to Tom RiniWe provide a requirements.txt file for doc building, but had not been configuring readthedocs to know where it is.
--- .readthedocs.yml | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
.readthedocs.yml +4 -5
@@ -12,8 +12,7 @@ sphinx: # Optionally build your docs in additional formats such as PDF and ePub formats: [] -# Optionally set the version of Python and requirements required to build your docs -# python: -# version: 3.7 -# install: -# - requirements: docs/requirements.txt +# Explicitly set the requirements file +python: + install: + - requirements: doc/sphinx/requirements.txt
Signature · 1 line
2.25.1
Re: [PATCH 4/4] .readthedocs.yml: Set our requirements file
Quoted 15 lines · Tom Rini
We provide a requirements.txt file for doc building, but had not been configuring readthedocs to know where it is.
Cc Heinrich Schuchardt <[email protected]>Signed-off-by Tom Rini <[email protected]>---
.readthedocs.yml | 9 ++++----- > 1 file changed, 4 insertions(+), 5 deletions(-).readthedocs.yml +0
-0@@ -12,8 +12,7 @@ sphinx: # Optionally build your docs in additional formats such as PDF and ePub formats: []
.readthedocs.yml sphinx: · line 18
Quoted 10 lines
.readthedocs.yml +4
-5-# Optionally set the version of Python and requirements required to build your docs -# python: -# version: 3.7 -# install: -# - requirements: docs/requirements.txt +# Explicitly set the requirements file +python: + install: + - requirements: doc/sphinx/requirements.txt
I could not get it running without install python3-six:
# .readthedocs.yml # Read the Docs configuration file # See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
# Required
version: 2
build:
os: "ubuntu-20.04"
apt_packages:
- python3-six
tools:
python: "3.9"
# Build documentation in the docs/ directory with Sphinx sphinx:
configuration: doc/conf.py
# Optionally build your docs in additional formats such as PDF and ePub
formats: []
python:
install:
- requirements: doc/sphinx/requirements.txt
Best regards
Heinrich
Quoted 8 lines · Tom Rini
Follow what we do in GitLab CI where we break the jobs up in to stages such that if earlier and often quicker sanity tests fail we don't run everything else.
Signed-off-by Tom Rini <[email protected]>--- .azure-pipelines.yml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
I want to note that in my testing this didn't change the overall run-time of the CI job.
Signature · 1 line
Tom
Quoted 5 lines · Tom Rini
Follow what we do in GitLab CI where we break the jobs up in to stages such that if earlier and often quicker sanity tests fail we don't run everything else.
Signed-off-by Tom Rini <[email protected]>
Applied to u-boot/master, thanks!
Signature · 1 line
Tom