Skip to contentSkip to conversation

support/run-tests: add a mode to only download emulator builtin binaries

PATCH In review Romain Naour 2 participants 4 messages Jul 17, 2023 – Jul 18, 2023

lore
Share & tools
lore
#0 lore raw

PATCH 1/3 support/run-tests: add a mode to only download emulator builtin binaries

Currently, emulator builtin binaries are downloaded after the rootfs build by each tests running the emulator (Qemu) on armv5 or armv7.

Due to download quota on the server hosting theses binaries [1] some GitLab jobs are failing due to "Connection reset by peer" [2]:

ConnectionResetError: [Errno 104] Connection reset by peer

In a follow up commit, we'll add the possibility to download only once all emulator builtin binaries to pass them as build artifacts.

As such, we will need to make sure that run-tests only download theses binaries, and that it does not run the testsuite.

Add such a mode, with a new option, --prepare-only.

[1] http://autobuild.buildroot.net/artefacts
[2] https://gitlab.com/buildroot.org/buildroot/-/jobs/4409032417

---
 support/testing/run-tests | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
support/testing/run-tests +14 -0
index bf40019362..e8c540f370 100755
@@ -8,6 +8,8 @@ import nose2
 
 from infra.basetest import BRConfigTest
 
+import infra
+
 
 def main():
     parser = argparse.ArgumentParser(description='Run Buildroot tests')
@@ -23,6 +25,8 @@ def main():
                         help='output directory')
     parser.add_argument('-d', '--download',
                         help='download directory')
+    parser.add_argument('-p', '--prepare-only', action='store_true',
+                        help='download emulator builtin binaries')
     parser.add_argument('-k', '--keep',
                         help='keep build directories',
                         action='store_true')
@@ -60,6 +64,16 @@ def main():
 
     BRConfigTest.downloaddir = os.path.abspath(args.download)
 
+    if args.prepare_only:
+        emulator_builtin_binaries = ["kernel-vexpress-5.10.7",
+                                     "vexpress-v2p-ca9-5.10.7.dtb",
+                                     "kernel-versatile-5.10.7",
+                                     "versatile-pb-5.10.7.dtb"]
+        print("Downloading emulator builtin binaries")
+        for binary in emulator_builtin_binaries:
+            infra.download(BRConfigTest.downloaddir, binary)
+        return 0
+
     if args.output is None:
         print("Missing output directory, please use -o/--output")
         print("")
Signature · 6 lines
2.41.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

PATCH 2/3 gitlab-ci.yml: add stage explicitely to each jobs

"stages" is curently not defined in the .gitlab-ci.yml file, so the default pipeline stages are:

.pre
build
test
deploy
.post

Since any jobs specify a stage, all jobs are assigned the "test" stage [1]. All other stages defined by default are not used in the gitlab-ci pipeline, they remain hidden.

In order to introduce a new custom stage, add the "test" stage explicitely.

[1] https://docs.gitlab.com/ee/ci/yaml/#stages

---
 support/misc/gitlab-ci.yml.in | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
support/misc/gitlab-ci.yml.in +12 -1
index cbb6e555ce..d8d6b3f410 100644
@@ -1,27 +1,36 @@
+stages:
+  - test
+
 before_script:
   - git config --global --add safe.directory ${CI_PROJECT_DIR}
 
 .check-check-package_base:
+    stage: test
     script:
         - python3 -m pytest -v utils/checkpackagelib/
 
 .check-check-symbol_base:
+    stage: test
     script:
         - python3 -m pytest -v utils/checksymbolslib/
 
 .check-DEVELOPERS_base:
+    stage: test
     script:
         - utils/get-developers -v
 
 .check-package_base:
+    stage: test
     script:
         - make check-package
 
 .check-symbol_base:
+    stage: test
     script:
         - utils/check-symbols
 
 .defconfig_check:
+    stage: test
     script:
         - DEFCONFIG_NAME=$(echo ${CI_JOB_NAME} | sed -e 's,_check$,,g')
         - echo "Configure Buildroot for ${DEFCONFIG_NAME}"
@@ -42,6 +51,7 @@ before_script:
       }
 
 .defconfig_base:
+    stage: test
     script:
         - DEFCONFIG_NAME=${CI_JOB_NAME}
         - OUTPUT_DIR=output
@@ -69,6 +79,7 @@ before_script:
             - runtime-test.log
 
 .runtime_test_base:
+    stage: test
     # Keep build directories so the rootfs can be an artifact of the job. The
     # runner will clean up those files for us.
     # Multiply every emulator timeout by 10 to avoid sporadic failures in
@@ -86,7 +97,7 @@ before_script:
             - test-output/*/images/*
 
 .test_pkg:
-    stage: build
+    stage: test
     script:
         - OUTPUT_DIR=${CI_JOB_NAME}
         - echo "Configure Buildroot for ${OUTPUT_DIR}"
Signature · 6 lines
2.41.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#2 lore raw

PATCH 3/3 gitlab-ci.yml: pass emulator builtin binaries as artifacts

In reply to Romain Naour

Notes: We can't use runtime_test_download job from the parent pipeline (generate-gitlab-ci) since the artifacts archive size is limited to 5MB. So introduce a new custom stage named "download" executed before "test" stage. test-dl directory that contain downloaded files can be an artifact of the job passed to all jobs of next stages.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/4409032417

Runtime tested:
https://gitlab.com/kubu93/buildroot/-/pipelines/934319226

---
 support/misc/gitlab-ci.yml.in          | 11 +++++++++++
 support/scripts/generate-gitlab-ci-yml |  1 +
 2 files changed, 12 insertions(+)
support/misc/gitlab-ci.yml.in +11 -0
index d8d6b3f410..446132846f 100644
@@ -1,4 +1,5 @@
 stages:
+  - download
   - test
 
 before_script:
@@ -78,6 +79,16 @@ before_script:
             - output/build/*/.config
             - runtime-test.log
 
+.runtime_test_download:
+    stage: download
+    # Keep test-dl directory so the downloaded files can be an artifact of
+    # the job passed to all jobs of next stages.
+    script: ./support/testing/run-tests -d test-dl/ --prepare-only
+    artifacts:
+        when: always
+        paths:
+            - test-dl/
+
 .runtime_test_base:
     stage: test
     # Keep build directories so the rootfs can be an artifact of the job. The
support/scripts/generate-gitlab-ci-yml +1 -0
index ea4340f47c..2d922b9eb2 100755
@@ -132,6 +132,7 @@ gen_tests() {
     fi
 
     if ${do_runtime:-false}; then
+        printf 'runtime_test_download: { extends: .runtime_test_download }\n'
         printf '%s: { extends: .runtime_test_base }\n' "${runtimes[@]}"
     fi
 
Signature · 6 lines
2.41.0

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#3 lore raw
In reply to Romain Naour
Quoted 16 lines · On Mon, 17 Jul 2023 23:33:57 +0200 Romain Naour <[email protected]> wrote:

Currently, emulator builtin binaries are downloaded after the rootfs build by each tests running the emulator (Qemu) on armv5 or armv7.

Due to download quota on the server hosting theses binaries [1] some GitLab jobs are failing due to "Connection reset by peer" [2]:

ConnectionResetError: [Errno 104] Connection reset by peer

In a follow up commit, we'll add the possibility to download only once all emulator builtin binaries to pass them as build artifacts.

As such, we will need to make sure that run-tests only download theses binaries, and that it does not run the testsuite.

Add such a mode, with a new option, --prepare-only.

Quoted 7 lines

[1] http://autobuild.buildroot.net/artefacts
[2] https://gitlab.com/buildroot.org/buildroot/-/jobs/4409032417

---
 support/testing/run-tests | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

Thanks, series applied. I'm not an expert in Gitlab CI sorcery, so I trust you that it is the right way of doing things :-)

Thomas

Signature · 7 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot