Skip to contentSkip to conversation

support/test-pkg: fixes and enhancements

PATCH Superseded Yann E. MORIN 2 participants 13 messages Feb 12, 2017

lore
Share & tools
lore
#0 lore raw

v4 cover support/test-pkg: fixes and enhancements

Hello All!

This series brins a few fixes and improvements to the test-pkg script.

First, it fixes the cioding style, to be more in-line with what is used in Buildroot.

Changes v2 -> v3:
- drop patches already applied
- simplify the config check (Cam)
- be less verbose (Thomas)
- fix coding style (Thomas)

Changes v2 -> v3:
- simplify the code to store missing config lines (Luca)
- properly run when -r is not provided
- add 5th path to use an alternate list
- add 6th patch to print the progress [n/N] for each toolchain

Regards,
Yann E. MORIN.

The following changes since commit 8ac3f121f0ccc31a79b08c473e19e4d4ae922942

wiringpi: fix static linking (2017-02-12 15:03:02 +0100)

are available in the git repository at:

git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to 05e8a8e4a66b175edcd8f6250c583051f77eeae9

support/test-pkg: print number of toolchain and progress (2017-02-12 15:18:08 +0100)

----------------------------------------------------------------
Yann E. MORIN (7):
supprt/test-pkg: fix code style
support/scripts: check the config snippet exists support/test-pkg: simplify the config check support/test-pkg: be less verbose
support/test-pkg: report number and types of failures support/test-pkg: add option to use an alternate list of toolchains support/test-pkg: print number of toolchain and progress

 support/scripts/test-pkg | 125 ++++++++++++++++++++++++++++++-----------------
 1 file changed, 81 insertions(+), 44 deletions(-)
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#1 lore raw

PATCH v4 1/7 supprt/test-pkg: fix code style

---
 support/scripts/test-pkg | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
support/scripts/test-pkg +10 -10
index b867da8..9066fa0 100755
@@ -10,7 +10,7 @@ main() {
 
     o='hc:d:p:r:'
     O='help,config-snippet:build-dir:package:,random:'
-    opts="$( getopt -n "${my_name}" -o "${o}" -l "${O}" -- "${@}"  )"
+    opts="$(getopt -n "${my_name}" -o "${o}" -l "${O}" -- "${@}")"
     eval set -- "${opts}"
 
     random=0
@@ -46,14 +46,14 @@ main() {
     # Extract the URLs of the toolchains; drop internal toolchains
     # E.g.: http://server/path/to/name.config,arch,libc
     #  -->  http://server/path/to/name.config
-    toolchains=( $( curl -s "${TOOLCHAINS_URL}" \
-                    |sed -r -e 's/,.*//; /internal/d;' \
-                    |if [ ${random} -gt 0 ]; then \
-                        sort -R |head -n ${random}
-                     else
-                        cat
-                     fi |sort
-                  )
+    toolchains=($(curl -s "${TOOLCHAINS_URL}" \
+                  |sed -r -e 's/,.*//; /internal/d;' \
+                  |if [ ${random} -gt 0 ]; then \
+                      sort -R |head -n ${random}
+                   else
+                      cat
+                   fi |sort
+                 )
                )
 
     if [ ${#toolchains[@]} -eq 0 ]; then
@@ -73,7 +73,7 @@ build_one() {
     local toolchain line skip
 
     # Using basename(1) on a URL works nicely
-    toolchain="$( basename "${url}" .config )"
+    toolchain="$(basename "${url}" .config)"
 
     printf "%40s: " "${toolchain}"
 
Signature · 1 line
2.7.4
#2 lore raw

PATCH v4 1/7 supprt/test-pkg: fix code style

Hello,

Quoted 5 lines · Yann E. MORIN
---
 support/scripts/test-pkg | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

Applied to master after fixing the typo in the commit title. Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#3 lore raw

PATCH v4 2/7 support/scripts: check the config snippet exists

In reply to Yann E. MORIN
---
 support/scripts/test-pkg | 3 +++
 1 file changed, 3 insertions(+)
support/scripts/test-pkg +3 -0
index 9066fa0..11ec18c 100755
@@ -39,6 +39,9 @@ main() {
     if [ -z "${cfg}" ]; then
         printf "error: no config snippet specified\n" >&2; exit 1
     fi
+    if [ ! -e "${cfg}" ]; then
+        printf "error: %s: no such file\n" "${cfg}" >&2; exit 1
+    fi
     if [ -z "${dir}" ]; then
         dir="${HOME}/br-test-pkg"
     fi
Signature · 1 line
2.7.4
#4 lore raw

PATCH v4 2/7 support/scripts: check the config snippet exists

Hello,

Quoted 4 lines · Yann E. MORIN
---
 support/scripts/test-pkg | 3 +++
 1 file changed, 3 insertions(+)

Applied to master, after adjusting the commit title to support/test-pkg, in order to be consistent with other patches in the series.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#5 lore raw

PATCH v4 3/7 support/test-pkg: simplify the config check

In reply to Yann E. MORIN

Use comm(1) to check that all our config options are properly set in the resulting configuration, rather than our canned and fragile code.

---
 support/scripts/test-pkg | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)
support/scripts/test-pkg +5 -9
index 11ec18c..a1d272a 100755
@@ -73,7 +73,7 @@ build_one() {
     local url="${2}"
     local cfg="${3}"
     local pkg="${4}"
-    local toolchain line skip
+    local toolchain
 
     # Using basename(1) on a URL works nicely
     toolchain="$(basename "${url}" .config)"
@@ -106,14 +106,10 @@ build_one() {
     # or not set) in the actual .config; if one of them is not, it means
     # some dependency from the toolchain or arch is not available, in
     # which case this config is untestable and we skip it.
-    skip=false
-    while read line; do
-        if ! grep "^${line}\$" "${dir}/.config" >/dev/null 2>&1; then
-            printf "%s\n" "${line}"
-            skip=true
-        fi
-    done <"${cfg}" >"${dir}/missing.config"
-    if ${skip}; then
+    # We don't care about the locale to sort in, as long as both sort are
+    # done in the same locale.
+    comm -23 <(sort "${cfg}") <(sort "${dir}/.config") >"${dir}/missing.config"
+    if [ -s "${dir}/missing.config" ]; then
         printf ", SKIPPED\n"
         return
     fi
Signature · 1 line
2.7.4
#6 lore raw

PATCH v4 3/7 support/test-pkg: simplify the config check

Hello,

Quoted 8 lines · Yann E. MORIN

Use comm(1) to check that all our config options are properly set in the resulting configuration, rather than our canned and fragile code.

---
 support/scripts/test-pkg | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)

Applied to master, thanks.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#7 lore raw

PATCH v4 4/7 support/test-pkg: be less verbose

In reply to Yann E. MORIN
---
 support/scripts/test-pkg | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)
support/scripts/test-pkg +6 -10
index a1d272a..d9ae5c5 100755
@@ -83,9 +83,8 @@ build_one() {
     dir="${dir}/${toolchain}"
     mkdir -p "${dir}"
 
-    printf "download config"
     if ! curl -s "${url}" >"${dir}/.config"; then
-        printf ": FAILED\n"
+        printf "FAILED\n"
         return
     fi
 
@@ -97,9 +96,8 @@ build_one() {
 	_EOF_
     cat "${cfg}" >>"${dir}/.config"
 
-    printf ", olddefconfig"
     if ! make O="${dir}" olddefconfig >/dev/null 2>&1; then
-        printf ": FAILED\n"
+        printf "FAILED\n"
         return
     fi
     # We want all the options from the snippet to be present as-is (set
@@ -110,28 +108,26 @@ build_one() {
     # done in the same locale.
     comm -23 <(sort "${cfg}") <(sort "${dir}/.config") >"${dir}/missing.config"
     if [ -s "${dir}/missing.config" ]; then
-        printf ", SKIPPED\n"
+        printf "SKIPPED\n"
         return
     fi
     # Remove file, it's empty anyway.
     rm -f "${dir}/missing.config"
 
     if [ -n "${pkg}" ]; then
-        printf ", dirclean"
         if ! make O="${dir}" "${pkg}-dirclean" >> "${dir}/logfile" 2>&1; then
-            printf ": FAILED\n"
+            printf "FAILED\n"
             return
         fi
     fi
 
-    printf ", build"
     # shellcheck disable=SC2086
     if ! make O="${dir}" ${pkg} >> "${dir}/logfile" 2>&1; then
-        printf ": FAILED\n"
+        printf "FAILED\n"
         return
     fi
 
-    printf ": OK\n"
+    printf "OK\n"
 }
 
 help() {
Signature · 1 line
2.7.4
#8 lore raw

PATCH v4 4/7 support/test-pkg: be less verbose

Hello,

Quoted 5 lines · Yann E. MORIN
---
 support/scripts/test-pkg | 16 ++++++----------
 1 file changed, 6 insertions(+), 10 deletions(-)

Applied to master, thanks.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#9 lore raw

PATCH v4 5/7 support/test-pkg: report number and types of failures

In reply to Yann E. MORIN
Cc Thomas Petazzoni <[email protected]>
Cc Luca Ceresoli <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
---
 support/scripts/test-pkg | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
support/scripts/test-pkg +18 -6
index d9ae5c5..a040ce1 100755
@@ -6,6 +6,7 @@ TOOLCHAINS_URL='http://autobuild.buildroot.org/toolchains/configs/toolchain-conf
 main() {
     local o O opts
     local cfg dir pkg random toolchain
+    local ret nb nb_skip nb_fail
     local -a toolchains
 
     o='hc:d:p:r:'
@@ -63,9 +64,20 @@ main() {
         printf "error: no toolchain found (networking issue?)\n" >&2; exit 1
     fi
 
+    nb=0
+    nb_skip=0
+    nb_fail=0
     for toolchain in "${toolchains[@]}"; do
-        build_one "${dir}" "${toolchain}" "${cfg}" "${pkg}"
+        build_one "${dir}" "${toolchain}" "${cfg}" "${pkg}" && ret=0 || ret=${?}
+        case ${ret} in
+        (0) ;;
+        (1) : $((nb_skip++));;
+        (2) : $((nb_fail++));;
+        esac
+        : $((nb++))
     done
+
+    printf "%d builds, %d skipped, %d failed\n" ${nb} ${nb_skip} ${nb_fail}
 }
 
 build_one() {
@@ -85,7 +97,7 @@ build_one() {
 
     if ! curl -s "${url}" >"${dir}/.config"; then
         printf "FAILED\n"
-        return
+        return 2
     fi
 
     cat >>"${dir}/.config" <<-_EOF_
@@ -98,7 +110,7 @@ build_one() {
 
     if ! make O="${dir}" olddefconfig >/dev/null 2>&1; then
         printf "FAILED\n"
-        return
+        return 2
     fi
     # We want all the options from the snippet to be present as-is (set
     # or not set) in the actual .config; if one of them is not, it means
@@ -109,7 +121,7 @@ build_one() {
     comm -23 <(sort "${cfg}") <(sort "${dir}/.config") >"${dir}/missing.config"
     if [ -s "${dir}/missing.config" ]; then
         printf "SKIPPED\n"
-        return
+        return 1
     fi
     # Remove file, it's empty anyway.
     rm -f "${dir}/missing.config"
@@ -117,14 +129,14 @@ build_one() {
     if [ -n "${pkg}" ]; then
         if ! make O="${dir}" "${pkg}-dirclean" >> "${dir}/logfile" 2>&1; then
             printf "FAILED\n"
-            return
+            return 2
         fi
     fi
 
     # shellcheck disable=SC2086
     if ! make O="${dir}" ${pkg} >> "${dir}/logfile" 2>&1; then
         printf "FAILED\n"
-        return
+        return 2
     fi
 
     printf "OK\n"
Signature · 1 line
2.7.4
#10 lore raw

PATCH v4 5/7 support/test-pkg: report number and types of failures

Hello,

Quoted 7 lines · Yann E. MORIN
Cc Thomas Petazzoni <[email protected]>
Cc Luca Ceresoli <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
---
 support/scripts/test-pkg | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)

Applied to master, thanks.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#11 lore raw

PATCH v4 6/7 support/test-pkg: add option to use an alternate list of toolchains

In reply to Yann E. MORIN

For now, testing a package requires network access. However, there are situations where everything is already cached locally (especially the toolchains tarballs) and network is not available (e.g. in the train, travelling back from FOSDEM...)

Alternatively, one may also want to test against a subset of the default toolchains (e.g. the ones known to have a specific issue).

Add an option to use an alternate URL, which can be remote or a path to a local file.

---
 support/scripts/test-pkg | 47 +++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 41 insertions(+), 6 deletions(-)
support/scripts/test-pkg +41 -6
index a040ce1..9626c1a 100755
@@ -5,16 +5,17 @@ TOOLCHAINS_URL='http://autobuild.buildroot.org/toolchains/configs/toolchain-conf
 
 main() {
     local o O opts
-    local cfg dir pkg random toolchain
+    local cfg dir pkg random url toolchain
     local ret nb nb_skip nb_fail
     local -a toolchains
 
-    o='hc:d:p:r:'
-    O='help,config-snippet:build-dir:package:,random:'
+    o='hc:d:p:r:t:'
+    O='help,config-snippet:build-dir:package:,random:,toolchains:'
     opts="$(getopt -n "${my_name}" -o "${o}" -l "${O}" -- "${@}")"
     eval set -- "${opts}"
 
     random=0
+    url="${TOOLCHAINS_URL}"
     while [ ${#} -gt 0 ]; do
         case "${1}" in
         (-h|--help)
@@ -32,6 +33,9 @@ main() {
         (-r|--random)
             random="${2}"; shift 2
             ;;
+        (-t|--toolchains)
+            url="${2}"; shift 2
+            ;;
         (--)
             shift; break
             ;;
@@ -47,10 +51,16 @@ main() {
         dir="${HOME}/br-test-pkg"
     fi
 
+    # Transform local paths to URI to make curl happy and simplify
+    # our code path
+    case "${url}" in
+    (/*)    url="file://${url}";;
+    esac
+
     # Extract the URLs of the toolchains; drop internal toolchains
     # E.g.: http://server/path/to/name.config,arch,libc
     #  -->  http://server/path/to/name.config
-    toolchains=($(curl -s "${TOOLCHAINS_URL}" \
+    toolchains=($(curl -s "${url}" \
                   |sed -r -e 's/,.*//; /internal/d;' \
                   |if [ ${random} -gt 0 ]; then \
                       sort -R |head -n ${random}
@@ -87,6 +97,12 @@ build_one() {
     local pkg="${4}"
     local toolchain
 
+    # Transform local paths to URI to make curl happy and simplify
+    # our code path
+    case "${url}" in
+    (/*)    url="file://${url}";;
+    esac
+
     # Using basename(1) on a URL works nicely
     toolchain="$(basename "${url}" .config)"
 
@@ -156,8 +172,22 @@ In case failures are noticed, you can fix the package and just re-run the
 same command again; it will re-run the test where it failed. If you did
 specify a package (with -p), the package build dir will be removed first.
 
-The list of toolchains is retrieved from the Buildroot autobuilders, available
-at ${TOOLCHAINS_URL}.
+Unless specified with -t, the list of toolchains is retrieved from the
+Buildroot autobuilders, available at:
+    ${TOOLCHAINS_URL}
+
+The list of toolchains should contain the URLs to all toolchains, one per
+line, along with the architecture and C library used, separated by commas,
+"URL,ARCH,LIBC" (only the first field, URL, is used by this script). For
+example:
+
+    https://server/path/to/toolchain-1.config,arm,glibc
+    /path/to/local-toolchain.config,i386,musl
+
+The URL for each toolchain should point to a .config file that contains
+only the toolchain and architecture settings. URLs that contain the string
+'internal' are skipped, on the assumption that the configuration would
+build an internal toolchain (which takes a lot of time).
 
 Options:
 
@@ -179,6 +209,11 @@ Options:
         Limit the tests to the N randomly selected toolchains, instead of
         building with all toolchains.
 
+    -t URL, --toolchains URL
+        Use the toolchains described at URL instead of the toolchains used
+        by the Buildroot autobuilders (see above). URL can be a path to a
+        local file.
+
 Example:
 
     Testing libcec would require a config snippet that contains:
Signature · 1 line
2.7.4
#12 lore raw

PATCH v4 7/7 support/test-pkg: print number of toolchain and progress

In reply to Yann E. MORIN
Cc Thomas De Schampheleire <[email protected]>
---
 support/scripts/test-pkg | 25 ++++++++++---------------
 1 file changed, 10 insertions(+), 15 deletions(-)
support/scripts/test-pkg +10 -15
index 9626c1a..8d47415 100755
@@ -6,7 +6,7 @@ TOOLCHAINS_URL='http://autobuild.buildroot.org/toolchains/configs/toolchain-conf
 main() {
     local o O opts
     local cfg dir pkg random url toolchain
-    local ret nb nb_skip nb_fail
+    local ret nb nb_skip nb_fail nb_tc
     local -a toolchains
 
     o='hc:d:p:r:t:'
@@ -70,21 +70,25 @@ main() {
                  )
                )
 
-    if [ ${#toolchains[@]} -eq 0 ]; then
+    nb_tc="${#toolchains[@]}"
+    if [ ${nb_tc} -eq 0 ]; then
         printf "error: no toolchain found (networking issue?)\n" >&2; exit 1
     fi
+    printf "Found %d toolchains\n" ${nb_tc}
 
     nb=0
     nb_skip=0
     nb_fail=0
     for toolchain in "${toolchains[@]}"; do
+        : $((nb++))
+        printf "%40s [%*d/%d]: " "$(basename "${toolchain}" .config)" \
+                                 ${#nb_tc} ${nb} ${nb_tc}
         build_one "${dir}" "${toolchain}" "${cfg}" "${pkg}" && ret=0 || ret=${?}
         case ${ret} in
-        (0) ;;
-        (1) : $((nb_skip++));;
-        (2) : $((nb_fail++));;
+        (0) printf "OK\n";;
+        (1) : $((nb_skip++)); printf "SKIPPED\n";;
+        (2) : $((nb_fail++)); printf "FAILED\n";;
         esac
-        : $((nb++))
     done
 
     printf "%d builds, %d skipped, %d failed\n" ${nb} ${nb_skip} ${nb_fail}
@@ -106,13 +110,10 @@ build_one() {
     # Using basename(1) on a URL works nicely
     toolchain="$(basename "${url}" .config)"
 
-    printf "%40s: " "${toolchain}"
-
     dir="${dir}/${toolchain}"
     mkdir -p "${dir}"
 
     if ! curl -s "${url}" >"${dir}/.config"; then
-        printf "FAILED\n"
         return 2
     fi
 
@@ -125,7 +126,6 @@ build_one() {
     cat "${cfg}" >>"${dir}/.config"
 
     if ! make O="${dir}" olddefconfig >/dev/null 2>&1; then
-        printf "FAILED\n"
         return 2
     fi
     # We want all the options from the snippet to be present as-is (set
@@ -136,7 +136,6 @@ build_one() {
     # done in the same locale.
     comm -23 <(sort "${cfg}") <(sort "${dir}/.config") >"${dir}/missing.config"
     if [ -s "${dir}/missing.config" ]; then
-        printf "SKIPPED\n"
         return 1
     fi
     # Remove file, it's empty anyway.
@@ -144,18 +143,14 @@ build_one() {
 
     if [ -n "${pkg}" ]; then
         if ! make O="${dir}" "${pkg}-dirclean" >> "${dir}/logfile" 2>&1; then
-            printf "FAILED\n"
             return 2
         fi
     fi
 
     # shellcheck disable=SC2086
     if ! make O="${dir}" ${pkg} >> "${dir}/logfile" 2>&1; then
-        printf "FAILED\n"
         return 2
     fi
-
-    printf "OK\n"
 }
 
 help() {
Signature · 1 line
2.7.4