Skip to contentSkip to conversation

support: script to build-test packages

PATCH New Yann E. MORIN 3 participants 10 messages Feb 7, 2017

lore
Share & tools
lore
#0 lore raw

cover support: script to build-test packages

Hello All!

As Thomas Petazzoni initially wrote (and slightly edited to match the new script):

This has been often asked, and yesterday I needed it, so I did a quick and dirty hack. The following script takes as argument a defconfig fragment that defines a bunch of Buildroot configuration options, and will built this fragment against all the toolchain configurations tested in the autobuilders.

Example:

    $ cat template
    BR2_PACKAGE_JACK2=y
    BR2_PACKAGE_JACK2_LEGACY=y
    BR2_PACKAGE_JACK2_DBUS=y
    $ ./support/scripts/test-pkg -c template -d ~/br-test-jack2 -p jack2
             armv5-ctng-linux-gnueabi : download config, olddefconfig, dirclean, build: OK
           armv7-ctng-linux-gnueabihf : download config, olddefconfig, dirclean, build: OK
                     br-aarch64-glibc : download config, olddefconfig, dirclean, build: OK
                        br-arcle-hs38 : download config, olddefconfig, dirclean, build: OK
                         br-arm-basic : download config, olddefconfig, SKIP
               br-arm-cortex-a9-glibc : download config, olddefconfig, dirclean, build: OK
                br-arm-cortex-a9-musl : download config, olddefconfig, dirclean, build: OK
                br-arm-cortex-m4-full : download config, olddefconfig, SKIP
                          br-arm-full : download config, olddefconfig, dirclean, build: OK
                 br-arm-full-nothread : download config, olddefconfig, SKIP
                   br-arm-full-static : download config, olddefconfig, SKIP
                         br-bfin-full : download config, olddefconfig, SKIP
                br-i386-pentium4-full : download config, olddefconfig, dirclean, build: OK
             br-i386-pentium-mmx-musl : download config, olddefconfig, dirclean, build: OK
                    br-m68k-5208-full : download config, olddefconfig, SKIP
                   br-m68k-68040-full : download config, olddefconfig, dirclean, build: OK
                 br-microblazeel-full : download config, olddefconfig, dirclean, build: OK
                   br-mips64-n64-full : download config, olddefconfig, dirclean, build: OK
              br-mips32r6-el-hf-glibc : download config, olddefconfig, dirclean, build: OK
              br-mips64r6-el-hf-glibc : download config, olddefconfig, dirclean, build: OK
                   br-mipsel-o32-full : download config, olddefconfig, dirclean, build: OK
                       br-nios2-glibc : download config, olddefconfig, dirclean, build: OK
            br-powerpc-603e-basic-cpp : download config, olddefconfig, SKIP
          br-powerpc64le-power8-glibc : download config, olddefconfig, dirclean, build: OK
            br-powerpc64-power7-glibc : download config, olddefconfig, dirclean, build: OK
               br-powerpc-e500mc-full : download config, olddefconfig, dirclean, build: OK
                          br-sh4-full : download config, olddefconfig, dirclean, build: OK
                      br-sparc-uclibc : download config, olddefconfig, SKIP
                     br-sparc64-glibc : download config, olddefconfig, dirclean, build: OK
                 br-x86-64-core2-full : download config, olddefconfig, dirclean, build: OK
                       br-x86-64-musl : download config, olddefconfig, dirclean, build: OK
                       br-xtensa-full : download config, olddefconfig, dirclean, build: OK
                  i686-ctng-linux-gnu : download config, olddefconfig, dirclean, build: OK
                       linaro-aarch64 : download config, olddefconfig, dirclean, build: FAIL
                           linaro-arm : download config, olddefconfig, dirclean, build: FAIL
          mips64el-ctng_n32-linux-gnu : download config, olddefconfig, dirclean, build: OK
          mips64el-ctng_n64-linux-gnu : download config, olddefconfig, dirclean, build: FAIL
     powerpc-ctng_e500v2-linux-gnuspe : download config, olddefconfig, dirclean, build: OK
                  sourcery-arm-armv4t : download config, olddefconfig, dirclean, build: OK
                         sourcery-arm : download config, olddefconfig, dirclean, build: OK
                  sourcery-arm-thumb2 : download config, olddefconfig, dirclean, build: OK
                      sourcery-mips64 : download config, olddefconfig, dirclean, build: OK
                        sourcery-mips : download config, olddefconfig, dirclean, build: OK
                       sourcery-nios2 : download config, olddefconfig, dirclean, build: OK
                          sourcery-sh : download config, olddefconfig, dirclean, build: FAIL
                      sourcery-x86-64 : download config, olddefconfig, dirclean, build: OK
                         sourcery-x86 : download config, olddefconfig, dirclean, build: OK
        x86_64-ctng_locales-linux-gnu : download config, olddefconfig, dirclean, build: OK
    8 configurations were skipped
    4 configurations would not build

So, the jack2 package, with both its "legacy" and "dbus" backends has been built in all those configurations.

It says "OK" when the build was successful, "FAIL" when it failed and "SKIP" when for some reason the options could not be enabled for the current toolchain configuration (for example if this package is disabled on this architecture, or if a toolchain feature is missing, etc.).

Note that the script excludes the "internal toolchain" configurations, because they take too long to build.

One nice trick is that you can re-run the script, and it will simply restart the build. So after fixing your package, if you want to test it again, you don't have to clean everything and restart from scratch:
just remove the build directory of your package in all output directories, and restart the script.

Regards,
Yann E. MORIN.

The following changes since commit c03f112b63327409c6ebb301d2fa8d873233e0da

python-service-identity: bump version to 16.0.0 (2017-02-07 14:25:23 +0100)

are available in the git repository at:

git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to 3a2dfc3a5dcf4389d2b65b91de95124385241a18

supprt/test-pkg: add option to limit the number of tests (2017-02-07 14:41:50 +0100)

----------------------------------------------------------------
Yann E. MORIN (4):
support/scripts: add script to test a package support/test-pkg: store lines missing from resulting configuraiton support/test-pkg: report number and types of failures supprt/test-pkg: add option to limit the number of tests

 support/scripts/test-pkg | 214 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 214 insertions(+)
 create mode 100755 support/scripts/test-pkg
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#1 lore raw

PATCH 1/4 support/scripts: add script to test a package

This script helps in testing that a pacakge builds fine on a wide range of architectures and toolchains: BE/LE, 32/64-bit, musl/glibc/uclibc...

[yann.morin.1998 at free.fr:
- completely rewrite the script from Thomas, with help from Luca
]

Cc Luca Ceresoli <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
---
 support/scripts/test-pkg | 164 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 164 insertions(+)
 create mode 100755 support/scripts/test-pkg
support/scripts/test-pkg new file +164 -0
new file mode 100755
index 0000000..daebe62
@@ -0,0 +1,164 @@
+#!/bin/bash
+set -e
+
+TOOLCHAINS_BASE_URL='http://autobuild.buildroot.org/toolchains/configs'
+
+main() {
+    local o O opts
+    local cfg dir pkg toolchain
+    local -a toolchains
+
+    o='hc:d:p:'
+    O='help,config-snippet:build-dir:package:'
+    opts="$( getopt -n "${my_name}" -o "${o}" -l "${O}" -- "${@}"  )"
+    eval set -- "${opts}"
+
+    while [ ${#} -gt 0 ]; do
+        case "${1}" in
+        (-h|--help)
+            help; exit 0
+            ;;
+        (-c|--config-snippet)
+            cfg="${2}"; shift 2
+            ;;
+        (-d|--build-dir)
+            dir="${2}"; shift 2
+            ;;
+        (-p|--package)
+            pkg="${2}"; shift 2
+            ;;
+        (--)
+            shift; break
+            ;;
+        esac
+    done
+    if [ -z "${cfg}" ]; then
+        printf "error: no config snippet specified\n" >&2; exit 1
+    fi
+    if [ -z "${dir}" ]; then
+        dir="${HOME}/br-test-pkg"
+    fi
+
+    # Extract the names of the toolchains
+    # http://server/path/to/name.config,arch,libc  -->  name
+    toolchains=( $( curl -s "${TOOLCHAINS_BASE_URL}/toolchain-configs.csv" \
+                    |sed -r -e 's/,.*//; s:.*/(.*)\.config:\1:; /internal/d;'
+                  )
+               )
+
+    if [ ${#toolchains[@]} -eq 0 ]; then
+        printf "error: no toolchain found (networking issue?)\n" >&2; exit 1
+    fi
+
+    for toolchain in "${toolchains[@]}"; do
+        build_one "${dir}" "${toolchain}" "${cfg}" "${pkg}"
+    done
+}
+
+build_one() {
+    local dir="${1}"
+    local toolchain="${2}"
+    local cfg="${3}"
+    local pkg="${4}"
+    local line
+
+    printf "%40s: " "${toolchain}"
+
+    dir="${dir}/${toolchain}"
+    mkdir -p "${dir}"
+
+    printf "download config"
+    if ! curl -s "${TOOLCHAINS_BASE_URL}/${toolchain}.config" >"${dir}/.config"; then
+        printf ": FAILED\n"
+        return
+    fi
+
+    cat >>"${dir}/.config" <<-_EOF_
+	BR2_INIT_NONE=y
+	BR2_SYSTEM_BIN_SH_NONE=y
+	# BR2_PACKAGE_BUSYBOX is not set
+	# BR2_TARGET_ROOTFS_TAR is not set
+	_EOF_
+    cat "${cfg}" >>"${dir}/.config"
+
+    printf ", olddefconfig"
+    if ! make O="${dir}" olddefconfig >/dev/null 2>&1; then
+        printf ": FAILED\n"
+        return
+    fi
+    # We want all the options from the snippet to be present as-is (set
+    # or not set) in the actual .config; if one of them is not, it means
+    # some dependency from the toolchain or arch is not available, in
+    # which case this config is untestable and we skip it.
+    while read line; do
+        if ! grep "^${line}\$" "${dir}/.config" >/dev/null 2>&1; then
+            printf ", SKIPPED\n"
+            return
+        fi
+    done <"${cfg}"
+
+    if [ -n "${pkg}" ]; then
+        printf ", dirclean"
+        if ! make O="${dir}" "${pkg}-dirclean" >> "${dir}/logfile" 2>&1; then
+            printf ": FAILED\n"
+            return
+        fi
+    fi
+
+    printf ", build"
+    # shellcheck disable=SC2086
+    if ! make O="${dir}" ${pkg} >> "${dir}/logfile" 2>&1; then
+        printf ": FAILED\n"
+        return
+    fi
+
+    printf ": OK\n"
+}
+
+help() {
+    cat <<_EOF_
+test-pkg: test-build a package against various toolchains and architectures
+
+The supplied config snippet is appended to each toolchain config, the
+resulting configuration is checked that it still contains all options
+specified in the snippet; if any is missing, the build is skipped, on the
+assumption that the package under test requires a toolchain or architecture
+feature that is missing.
+
+The list of toolchains is retrieved from Buildroot autobuilders, available
+at ${TOOLCHAINS_BASE_URL}.
+
+In case failures are noticed, you can fix the package and just re-run the
+same command again; it will re-run the test where it failed. If you did
+specify a package (with -p), the package build dir will be removed first.
+
+Options
+
+    -h, --help
+        Print this help.
+
+    -c CFG, --config-snippet CFG
+        Use the CFG file as the source for the config snippet. This file
+        should contain all the config options required to build a package.
+
+    -d DIR, --build-dir DIR
+        Do the builds in directory DIR, one sub-dir per toolchain.
+
+    -p PKG, --package PKG
+        Test-build the package PKG, by running 'make PKG'; if not specified,
+        just runs 'make'.
+
+Example:
+
+    Testing libcec would require a config snippet that contains:
+        BR2_PACKAGE_LIBCEC=y
+
+    Testing libcurl with openSSL support would require a snippet such as:
+        BR2_PACKAGE_OPENSSL=y
+        BR2_PACKAGE_LIBCURL=y
+
+_EOF_
+}
+
+my_name="${0##*/}"
+main "${@}"
Signature · 1 line
2.7.4
#2 lore raw

PATCH 1/4 support/scripts: add script to test a package

Hi,

Quoted 10 lines · Yann E. MORIN

This script helps in testing that a pacakge builds fine on a wide range of architectures and toolchains: BE/LE, 32/64-bit, musl/glibc/uclibc...

[yann.morin.1998 at free.fr:
- completely rewrite the script from Thomas, with help from Luca
]

Cc Luca Ceresoli <[email protected]>
Cc Thomas De Schampheleire <[email protected]>

Just two very minor nits about wording. For the rest the diff from the version I already reviewed is very limited anyway.

[...]

support/scripts/test-pkg line 123

Quoted 6 lines
support/scripts/test-pkg new file +6 -0
+help() {
+    cat <<_EOF_
+test-pkg: test-build a package against various toolchains and architectures
+
+The supplied config snippet is appended to each toolchain config, the
+resulting configuration is checked that it still contains all options

"is checked that it still contains" seems wrong to me. I'd reword it as "is checked to ensure it still contains".

support/scripts/test-pkg line 128

Quoted 5 lines
support/scripts/test-pkg new file +5 -0
+specified in the snippet; if any is missing, the build is skipped, on the
+assumption that the package under test requires a toolchain or architecture
+feature that is missing.
+
+The list of toolchains is retrieved from Buildroot autobuilders, available

from Buildroot autobuilders -> from the Buildroot autobuilders

With these fixed:

Signature · 1 line
Luca
#3 lore raw

PATCH 1/4 support/scripts: add script to test a package

Quoted 37 lines · Luca Ceresoli

Hi,

Quoted 10 lines · Yann E. MORIN

This script helps in testing that a pacakge builds fine on a wide range of architectures and toolchains: BE/LE, 32/64-bit, musl/glibc/uclibc...

[yann.morin.1998 at free.fr:
- completely rewrite the script from Thomas, with help from Luca
]

Cc Luca Ceresoli <[email protected]>
Cc Thomas De Schampheleire <[email protected]>

Just two very minor nits about wording. For the rest the diff from the version I already reviewed is very limited anyway.

[...]

support/scripts/test-pkg line 123

Quoted 6 lines
support/scripts/test-pkg new file +6 -0
+help() {
+    cat <<_EOF_
+test-pkg: test-build a package against various toolchains and architectures
+
+The supplied config snippet is appended to each toolchain config, the
+resulting configuration is checked that it still contains all options

"is checked that it still contains" seems wrong to me. I'd reword it as "is checked to ensure it still contains".

support/scripts/test-pkg line 128

Quoted 5 lines
support/scripts/test-pkg new file +5 -0
+specified in the snippet; if any is missing, the build is skipped, on the
+assumption that the package under test requires a toolchain or architecture
+feature that is missing.
+
+The list of toolchains is retrieved from Buildroot autobuilders, available

from Buildroot autobuilders -> from the Buildroot autobuilders

Quoted 3 lines

With these fixed:

With suggestions from Luca fixed:

#4 lore raw

PATCH 2/4 support/test-pkg: store lines missing from resulting configuraiton

In reply to Yann E. MORIN

When a build is skipped, store the lines from the config snippet, that are missing in the resulting configuration, in a fie in the build directory, for the user to inspect.

Cc Thomas Petazzoni <[email protected]>
Cc Luca Ceresoli <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
---
 support/scripts/test-pkg | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
support/scripts/test-pkg +12 -3
index daebe62..67f73b3 100755
@@ -60,7 +60,8 @@ build_one() {
     local toolchain="${2}"
     local cfg="${3}"
     local pkg="${4}"
-    local line
+    local line skip
+    local -a missing_lines
 
     printf "%40s: " "${toolchain}"
 
@@ -90,12 +91,20 @@ build_one() {
     # or not set) in the actual .config; if one of them is not, it means
     # some dependency from the toolchain or arch is not available, in
     # which case this config is untestable and we skip it.
+    skip=false
     while read line; do
         if ! grep "^${line}\$" "${dir}/.config" >/dev/null 2>&1; then
-            printf ", SKIPPED\n"
-            return
+            missing_lines+=( "${line}" )
+            skip=true
         fi
     done <"${cfg}"
+    if ${skip}; then
+        for line in "${missing_lines[@]}"; do
+            printf "%s\n" "${line}"
+        done >"${dir}/missing.config"
+        printf ", SKIPPED\n"
+        return
+    fi
 
     if [ -n "${pkg}" ]; then
         printf ", dirclean"
Signature · 1 line
2.7.4
#5 lore raw

PATCH 2/4 support/test-pkg: store lines missing from resulting configuraiton

Hi,

Quoted 2 lines · Yann E. MORIN

When a build is skipped, store the lines from the config snippet, that are missing in the resulting configuration, in a fie in the build

fie -> file

Quoted 34 lines

directory, for the user to inspect.

Cc Thomas Petazzoni <[email protected]>
Cc Luca Ceresoli <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
---
 support/scripts/test-pkg | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)
support/scripts/test-pkg +4 -3
index daebe62..67f73b3 100755
@@ -60,7 +60,8 @@ build_one() {
     local toolchain="${2}"
     local cfg="${3}"
     local pkg="${4}"
-    local line
+    local line skip
+    local -a missing_lines
 
     printf "%40s: " "${toolchain}"
 
@@ -90,12 +91,20 @@ build_one() {
     # or not set) in the actual .config; if one of them is not, it means
     # some dependency from the toolchain or arch is not available, in
     # which case this config is untestable and we skip it.
+    skip=false
     while read line; do
         if ! grep "^${line}\$" "${dir}/.config" >/dev/null 2>&1; then
-            printf ", SKIPPED\n"
-            return
+            missing_lines+=( "${line}" )

support/scripts/test-pkg build_one() { · line 107

Quoted 10 lines
support/scripts/test-pkg +8 -0
+            skip=true
         fi
     done <"${cfg}"
+    if ${skip}; then
+        for line in "${missing_lines[@]}"; do
+            printf "%s\n" "${line}"
+        done >"${dir}/missing.config"
+        printf ", SKIPPED\n"
+        return
+    fi

Nice feature, but the array-based implementation seems a more complex than needed.

What about:

+ skip=false

     rm -f "${dir}/missing.config"
     while read line; do
         if ! grep "^${line}\$" "${dir}/.config" >/dev/null 2>&1; then
-            printf ", SKIPPED\n"
-            return
+            echo "${line}" >>"${dir}/missing.config"
+            skip=true
         fi
     done <"${cfg}"

+ if ${skip}; then
+ printf ", SKIPPED\n"
+ return
+ fi

This is more concise, does not use arrays (so it's easier to understand to shell newbies). It also has the effect of removing the missing.config file@the next iteration, so you won't find it there when the config step succeeded.

Signature · 1 line
Luca
#6 lore raw

PATCH 3/4 support/test-pkg: report number and types of failures

In reply to Yann E. MORIN
Cc Thomas Petazzoni <[email protected]>
Cc Luca Ceresoli <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
---
 support/scripts/test-pkg | 42 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 36 insertions(+), 6 deletions(-)
support/scripts/test-pkg +36 -6
index 67f73b3..604a759 100755
@@ -6,6 +6,7 @@ TOOLCHAINS_BASE_URL='http://autobuild.buildroot.org/toolchains/configs'
 main() {
     local o O opts
     local cfg dir pkg toolchain
+    local ret nb_dl nb_cfg nb_skip nb_clean nb_build
     local -a toolchains
 
     o='hc:d:p:'
@@ -50,9 +51,38 @@ main() {
         printf "error: no toolchain found (networking issue?)\n" >&2; exit 1
     fi
 
+    nb_dl=0
+    nb_cfg=0
+    nb_skip=0
+    nb_clean=0
+    nb_build=0
     for toolchain in "${toolchains[@]}"; do
-        build_one "${dir}" "${toolchain}" "${cfg}" "${pkg}"
+        build_one "${dir}" "${toolchain}" "${cfg}" "${pkg}" && ret=0 || ret=${?}
+        case ${ret} in
+        (0) ;;
+        (1) : $((nb_dl++));;
+        (2) : $((nb_cfg++));;
+        (3) : $((nb_skip++));;
+        (4) : $((nb_clean++));;
+        (5) : $((nb_build++));;
+        esac
     done
+
+    if [ ${nb_dl} -ne 0 ]; then
+        printf "%d configurations could not be downloaded\n" ${nb_dl}
+    fi
+    if [ ${nb_cfg} -ne 0 ]; then
+        printf "%d configurations could not be applied\n" ${nb_cfg}
+    fi
+    if [ ${nb_skip} -ne 0 ]; then
+        printf "%d configurations were skipped\n" ${nb_skip}
+    fi
+    if [ ${nb_clean} -ne 0 ]; then
+        printf "%d configurations could not be dircleaned\n" ${nb_clean}
+    fi
+    if [ ${nb_build} -ne 0 ]; then
+        printf "%d configurations would not build\n" ${nb_build}
+    fi
 }
 
 build_one() {
@@ -71,7 +101,7 @@ build_one() {
     printf "download config"
     if ! curl -s "${TOOLCHAINS_BASE_URL}/${toolchain}.config" >"${dir}/.config"; then
         printf ": FAILED\n"
-        return
+        return 1
     fi
 
     cat >>"${dir}/.config" <<-_EOF_
@@ -85,7 +115,7 @@ build_one() {
     printf ", olddefconfig"
     if ! make O="${dir}" olddefconfig >/dev/null 2>&1; then
         printf ": FAILED\n"
-        return
+        return 2
     fi
     # We want all the options from the snippet to be present as-is (set
     # or not set) in the actual .config; if one of them is not, it means
@@ -103,14 +133,14 @@ build_one() {
             printf "%s\n" "${line}"
         done >"${dir}/missing.config"
         printf ", SKIPPED\n"
-        return
+        return 3
     fi
 
     if [ -n "${pkg}" ]; then
         printf ", dirclean"
         if ! make O="${dir}" "${pkg}-dirclean" >> "${dir}/logfile" 2>&1; then
             printf ": FAILED\n"
-            return
+            return 4
         fi
     fi
 
@@ -118,7 +148,7 @@ build_one() {
     # shellcheck disable=SC2086
     if ! make O="${dir}" ${pkg} >> "${dir}/logfile" 2>&1; then
         printf ": FAILED\n"
-        return
+        return 5
     fi
 
     printf ": OK\n"
Signature · 1 line
2.7.4
#7 lore raw

PATCH 3/4 support/test-pkg: report number and types of failures

Hi,

Quoted 47 lines · Yann E. MORIN
Cc Thomas Petazzoni <[email protected]>
Cc Luca Ceresoli <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
---
 support/scripts/test-pkg | 42 ++++++++++++++++++++++++++++++++++++------
 1 file changed, 36 insertions(+), 6 deletions(-)
support/scripts/test-pkg +21 -1
index 67f73b3..604a759 100755
@@ -6,6 +6,7 @@ TOOLCHAINS_BASE_URL='http://autobuild.buildroot.org/toolchains/configs'
 main() {
     local o O opts
     local cfg dir pkg toolchain
+    local ret nb_dl nb_cfg nb_skip nb_clean nb_build
     local -a toolchains
 
     o='hc:d:p:'
@@ -50,9 +51,38 @@ main() {
         printf "error: no toolchain found (networking issue?)\n" >&2; exit 1
     fi
 
+    nb_dl=0
+    nb_cfg=0
+    nb_skip=0
+    nb_clean=0
+    nb_build=0
     for toolchain in "${toolchains[@]}"; do
-        build_one "${dir}" "${toolchain}" "${cfg}" "${pkg}"
+        build_one "${dir}" "${toolchain}" "${cfg}" "${pkg}" && ret=0 || ret=${?}
+        case ${ret} in
+        (0) ;;
+        (1) : $((nb_dl++));;
+        (2) : $((nb_cfg++));;
+        (3) : $((nb_skip++));;
+        (4) : $((nb_clean++));;
+        (5) : $((nb_build++));;
+        esac
     done
+
+    if [ ${nb_dl} -ne 0 ]; then
+        printf "%d configurations could not be downloaded\n" ${nb_dl}
+    fi
+    if [ ${nb_cfg} -ne 0 ]; then
+        printf "%d configurations could not be applied\n" ${nb_cfg}

support/scripts/test-pkg main() { · line 85

Quoted 10 lines
support/scripts/test-pkg +10 -0
+    fi
+    if [ ${nb_skip} -ne 0 ]; then
+        printf "%d configurations were skipped\n" ${nb_skip}
+    fi
+    if [ ${nb_clean} -ne 0 ]; then
+        printf "%d configurations could not be dircleaned\n" ${nb_clean}
+    fi
+    if [ ${nb_build} -ne 0 ]; then
+        printf "%d configurations would not build\n" ${nb_build}
+    fi

The script gets pretty verbose, so it would be nice if we had a way to avoid the lines that are almost equal. Maybe an associative array, heh. But in the end it would probably result in poor readability, so:

Signature · 1 line
Luca
#8 lore raw

PATCH 4/4 supprt/test-pkg: add option to limit the number of tests

In reply to Yann E. MORIN

Sometimes, it interesting to have a global overview of whether the package builds at all or not, rather than test on all toolchains.

Add an option that allows testing on a limited set of randomly choosen toolchains.

Cc Thomas Petazzoni <[email protected]>
Cc Luca Ceresoli <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
---
 support/scripts/test-pkg | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)
support/scripts/test-pkg +15 -4
index 604a759..0b4615d 100755
@@ -5,15 +5,16 @@ TOOLCHAINS_BASE_URL='http://autobuild.buildroot.org/toolchains/configs'
 
 main() {
     local o O opts
-    local cfg dir pkg toolchain
+    local cfg dir pkg random toolchain
     local ret nb_dl nb_cfg nb_skip nb_clean nb_build
     local -a toolchains
 
-    o='hc:d:p:'
-    O='help,config-snippet:build-dir:package:'
+    o='hc:d:p:r:'
+    O='help,config-snippet:build-dir:package:,random::'
     opts="$( getopt -n "${my_name}" -o "${o}" -l "${O}" -- "${@}"  )"
     eval set -- "${opts}"
 
+    random=0
     while [ ${#} -gt 0 ]; do
         case "${1}" in
         (-h|--help)
@@ -28,6 +29,9 @@ main() {
         (-p|--package)
             pkg="${2}"; shift 2
             ;;
+        (-r|--random)
+            random="${2}"; shift 2
+            ;;
         (--)
             shift; break
             ;;
@@ -43,7 +47,10 @@ main() {
     # Extract the names of the toolchains
     # http://server/path/to/name.config,arch,libc  -->  name
     toolchains=( $( curl -s "${TOOLCHAINS_BASE_URL}/toolchain-configs.csv" \
-                    |sed -r -e 's/,.*//; s:.*/(.*)\.config:\1:; /internal/d;'
+                    |sed -r -e 's/,.*//; s:.*/(.*)\.config:\1:; /internal/d;' \
+                    |if [ ${random} -gt 0 ]; then \
+                        sort -R |head -n ${random} |sort
+                     fi
                   )
                )
 
@@ -187,6 +194,10 @@ Options
         Test-build the package PKG, by running 'make PKG'; if not specified,
         just runs 'make'.
 
+    -r N, --random N
+        Limit the tests to the N randomly selected toolchains, instead of
+        building all.
+
 Example:
 
     Testing libcec would require a config snippet that contains:
Signature · 1 line
2.7.4
#9 lore raw

PATCH 4/4 supprt/test-pkg: add option to limit the number of tests

All,

On 2017-02-07 14:53 +0100, Yann E. MORIN spake thusly:

Quoted 5 lines

Sometimes, it interesting to have a global overview of whether the package builds at all or not, rather than test on all toolchains.

Add an option that allows testing on a limited set of randomly choosen toolchains.

Forget that patch, there is an issue when the option is not provided. :-(

Regards,
Yann E. MORIN.

Quoted 68 lines
Cc Thomas Petazzoni <[email protected]>
Cc Luca Ceresoli <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
---
 support/scripts/test-pkg | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)
support/scripts/test-pkg +15 -4
index 604a759..0b4615d 100755
@@ -5,15 +5,16 @@ TOOLCHAINS_BASE_URL='http://autobuild.buildroot.org/toolchains/configs'
 
 main() {
     local o O opts
-    local cfg dir pkg toolchain
+    local cfg dir pkg random toolchain
     local ret nb_dl nb_cfg nb_skip nb_clean nb_build
     local -a toolchains
 
-    o='hc:d:p:'
-    O='help,config-snippet:build-dir:package:'
+    o='hc:d:p:r:'
+    O='help,config-snippet:build-dir:package:,random::'
     opts="$( getopt -n "${my_name}" -o "${o}" -l "${O}" -- "${@}"  )"
     eval set -- "${opts}"
 
+    random=0
     while [ ${#} -gt 0 ]; do
         case "${1}" in
         (-h|--help)
@@ -28,6 +29,9 @@ main() {
         (-p|--package)
             pkg="${2}"; shift 2
             ;;
+        (-r|--random)
+            random="${2}"; shift 2
+            ;;
         (--)
             shift; break
             ;;
@@ -43,7 +47,10 @@ main() {
     # Extract the names of the toolchains
     # http://server/path/to/name.config,arch,libc  -->  name
     toolchains=( $( curl -s "${TOOLCHAINS_BASE_URL}/toolchain-configs.csv" \
-                    |sed -r -e 's/,.*//; s:.*/(.*)\.config:\1:; /internal/d;'
+                    |sed -r -e 's/,.*//; s:.*/(.*)\.config:\1:; /internal/d;' \
+                    |if [ ${random} -gt 0 ]; then \
+                        sort -R |head -n ${random} |sort
+                     fi
                   )
                )
 
@@ -187,6 +194,10 @@ Options
         Test-build the package PKG, by running 'make PKG'; if not specified,
         just runs 'make'.
 
+    -r N, --random N
+        Limit the tests to the N randomly selected toolchains, instead of
+        building all.
+
 Example:
 
     Testing libcec would require a config snippet that contains:
Signature · 1 line
2.7.4
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'