Run tests in CI
Subsystems (2)
v4 cover Run tests in CI
Hi,
changes v3->v4:
* don't run on cross compile build
* add 2 commits to fix tests on (tst_net.sh on dash and tst_strstatus.c on alpine)
* partly rewrite runtest.sh
tested
https://github.com/pevik/ltp/actions/runs/1026124767
Kind regards,
Petr
Petr Vorel (7):
tst_device: Require root
tst_net.sh: Declare prefix variable as empty tst_strstatus.c: Use musl compatible status number
lib: Add script for running tests
make: Add make test{, -c, -shell} targets build.sh: Add support for make test
CI: Run also make test
.github/workflows/ci.yml | 5 ++ Makefile | 23 +++++ build.sh | 22 ++++- lib/newlib_tests/runtest.sh | 148 +++++++++++++++++++++++++++++++ lib/newlib_tests/tst_device.c | 1 + lib/newlib_tests/tst_strstatus.c | 3 +- testcases/lib/tst_net.sh | 2 +- 7 files changed, 201 insertions(+), 3 deletions(-) create mode 100755 lib/newlib_tests/runtest.sh
Signature · 1 line
2.32.0
PATCH v4 1/7 tst_device: Require root
as it's actually required for successful run:
tst_device.c:100: TINFO: Not allowed to open /dev/loop-control. Are you root?: EACCES (13) tst_device.c:139: TINFO: No free devices found
tst_device.c:335: TBROK: Failed to acquire device
---
The same as in v3.
lib/newlib_tests/tst_device.c | 1 + 1 file changed, 1 insertion(+)
lib/newlib_tests/tst_device.c +1 -0
@@ -40,6 +40,7 @@ static void do_test(void) }static struct tst_test test = { + .needs_root =1 , .needs_device =1 , .dev_min_size =300 , .test_all = do_test,
Signature · 1 line
2.32.0
PATCH v4 1/7 tst_device: Require root
Hi!
Obviously correct.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH v4 2/7 tst_net.sh: Declare prefix variable as empty
In reply to Petr Vorelin tst_ipaddr_un(). This is required to fix problem on dash, which (unlike bash and busybox ash implementation) takes value of previously defined variable, i.e. guarding with local does not work:
$ foo=ee; bar() { local foo; echo "foo: '$foo'"; }; bar
foo: 'ee'It requires declare it as empty:
$ foo=ee; bar() { local foo=; echo "foo: '$foo'"; }; bar
foo: ''This problem seems to be on various dash versions from 0.5.8 to 0.5.11+git20200708+dd9ef66-5 (on Debian oldstable, stable, testing). It'd require to set all variables, but now fix just prefix, which is used in build.sh and it's quite common name.
---
New in v4.
testcases/lib/tst_net.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
testcases/lib/tst_net.sh +2 -2
@@ -1,7 +1,7 @@ #!/bin/sh # SPDX-License-Identifier: GPL-2.0-or-later # Copyright (c) 2014-2017 Oracle and/or its affiliates. All Rights Reserved. -# Copyright (c) 2016-2019 Petr Vorel <[email protected]> +# Copyright (c) 2016-2021 Petr Vorel <[email protected]> # Author: Alexey Kodanev <[email protected]> [ -n "$TST_LIB_NET_LOADED" ] && return 0
@@ -408,7 +408,7 @@ tst_ipaddr_un() local max_net_id=$default_max local min_net_id=0 - local counter host_id host_range is_counter max_host_id min_host_id net_id prefix tmp type + local counter host_id host_range is_counter max_host_id min_host_id net_id prefix= tmp type local OPTIND while getopts "c:h:n:p" opt; do
Signature · 1 line
2.32.0
PATCH v4 2/7 tst_net.sh: Declare prefix variable as empty
Hi!
It starts to look like it would be easier to write a shell just for LTP and stick to it :-).
Anyways:
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH v4 3/7 tst_strstatus.c: Use musl compatible status number
In reply to Petr Vorelmusl since commit 41c632824c08 ("fix definitions of WIFSTOPPED and WIFSIGNALED to support up to signal 127") returns 1 on WIFSIGNALED(0xff). Thus test with WIFSIGNALED(0x1ff), which is compatible for both glibc and musl.
---
New in v4.
lib/newlib_tests/tst_strstatus.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
lib/newlib_tests/tst_strstatus.c +2 -1
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0-or-later /* * Copyright (c) 2017 Cyril Hrubis <[email protected]> +* Copyright (c) 2019-2021 Petr Vorel <[email protected]> */ /*
@@ -18,7 +19,7 @@ static struct tcase { {0x0001, "killed by SIGHUP"}, {0x137f, "is stopped"}, {0xffff, "is resumed"}, - {0xff, "invalid status 0xff"}, + {0x1ff, "invalid status 0x1ff"}, };static void do_test (unsigned int n)
Signature · 1 line
2.32.0
PATCH v4 3/7 tst_strstatus.c: Use musl compatible status number
PATCH v4 4/7 lib: Add script for running tests
In reply to Petr VorelFor now run only tests which TPASS or TCONF.
Disabled also problematic tests:
* tst_bool_expr: for some reason killed after testing:
/__w/ltp/ltp/lib/newlib_tests/tst_bool_expr.c:41: TINFO: Parsing 'A ( B )' A ( B )
Summary:
passed 24 failed 0 broken 0 skipped 0
warnings 0
PATH: '/__w/ltp/ltp/../ltp-build/testcases/lib:/__w/ltp/ltp/lib/newlib_tests/../../testcases/lib/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
DEBUG: 0
/__w/ltp/ltp/lib/tst_test.c:1363: TBROK: Test killed by SIGSEGV!
* tst_fuzzy_sync01: sporadically fails:../../include/tst_fuzzy_sync.h:685: TINFO: Exceeded execution loops, requesting exit
tst_fuzzy_sync01.c:227: TFAIL: acs:3 act:1 art:1 | =:23 -:46 +:2999931
...
Summary:
passed 21
failed 3
---
Partly rewritten since v3.
lib/newlib_tests/runtest.sh | 148 ++++++++++++++++++++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100755 lib/newlib_tests/runtest.sh
lib/newlib_tests/runtest.sh new file +148 -0
@@ -0,0 +1,148 @@ +#!/bin/sh +# Copyright (c) 2021 Petr Vorel <[email protected]> + +LTP_C_API_TESTS="${LTP_C_API_TESTS:-test05 test07 test09 test12 test15 test18 +test_exec test_timer tst_res_hexd tst_strstatus tst_fuzzy_sync02 tst_fuzzy_sync03}" + +LTP_SHELL_API_TESTS="${LTP_SHELL_API_TESTS:-shell/tst_check_driver.sh shell/net/*.sh}" + +cd $(dirname $0) +PATH="$PWD/../../testcases/lib/:$PATH" + +. tst_ansi_color.sh + +usage() +{ + cat << EOF +Usage: $0 [-b DIR ] [-c|-s] +-b DIR build directory (required for out-of-tree build) +-c run C API tests only +-s run shell API tests only +-h print this help +EOF +} + +# custom version +tst_flag2mask() +{ + case "$1" in + TPASS) return 0;; + TFAIL) return 1;; + TBROK) return 2;; + TWARN) return 4;; + TINFO) return 16;; + TCONF) return 32;; + esac +} + +# custom version +tst_res() +{ + if [ $# -eq 0 ]; then + echo >&2 + return + fi + + local res="$1" + shift + + tst_color_enabled + local color=$? + + printf "runtest " >&2 + tst_print_colored $res "$res: " >&2 + echo "$@" >&2 + +} + +# custom version +tst_brk() +{ + local res="$1" + shift + + tst_flag2mask "$res" + local mask=$? + + tst_res + tst_res $res $@ + + exit $mask +} + +run_tests() +{ + local target="$1" + local i ret tconf tpass vars + + eval vars="\$LTP_${target}_API_TESTS" + + tst_res TINFO "=== Run $target tests ===" + + for i in $vars; do + tst_res TINFO "* $i" + ./$i + ret=$? + + case $ret in + 0) tpass="$tpass $i";; + 1) tst_brk TFAIL "$i failed with TFAIL";; + 2) tst_brk TFAIL "$i failed with TBROK";; + 4) tst_brk TFAIL "$i failed with TWARN";; + 32) tconf="$tconf $i";; + 127) tst_brk TBROK "Error: file not found (wrong PATH? out-of-tree build without -b?), exit code: $ret";; + *) tst_brk TBROK "Error: unknown failure, exit code: $ret";; + esac + tst_res + done + + [ -z "$tpass" ] && tpass=" none" + [ -z "$tconf" ] && tconf=" none" + + tst_res TINFO "=== $target TEST RESULTS ===" + tst_res TINFO "Tests exited with TPASS:$tpass" + tst_res TINFO "Tests exited with TCONF:$tconf" + tst_res +} + +run_c_tests() +{ + if [ "$builddir" ]; then + cd $builddir/lib/newlib_tests + fi + + run_tests "C" + + if [ "$builddir" ]; then + cd - + fi +} + +run_shell_tests() +{ + run_tests "SHELL" +} + +builddir= +run= +while getopts b:chs opt; do + case $opt in + 'h') usage; exit 0;; + 'b') builddir=$OPTARG; PATH="$builddir/testcases/lib:$PATH";; + 'c') run="c";; + 's') run="s";; + *) usage; tst_brk TBROK "Error: invalid option";; + esac +done + +tst_res TINFO "PATH='$PATH'" + +if [ -z "$run" -o "$run" = "c" ]; then + run_c_tests +fi + +if [ -z "$run" -o "$run" = "s" ]; then + run_shell_tests +fi + +tst_res TPASS "No test failed"
Signature · 1 line
2.32.0
PATCH v4 4/7 lib: Add script for running tests
Hi!
Quoted 10 lines
For now run only tests which TPASS or TCONF.
Disabled also problematic tests:
*
tst_bool_expr: for some reason killed after testing:/__w/ltp/ltp/lib/newlib_tests/tst_bool_expr.c:41: TINFO: Parsing 'A ( B )' A ( B )
Summary:
Quoted 8 lines
passed 24 failed 0 broken 0 skipped 0warnings 0
PATH: '/__w/ltp/ltp/../ltp-build/testcases/lib:/__w/ltp/ltp/lib/newlib_tests/../../testcases/lib/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
DEBUG: 0
/__w/ltp/ltp/lib/tst_test.c:1363: TBROK: Test killed by SIGSEGV!
I've executed the test under valgrind and found some "conditional jump depends on uninitialized value", which is because we do not clear the priv pointer for newly added tokens. Does this patch fix it for you?
lib/tst_bool_expr.c +1 -0
@@ -55,6 +55,7 @@ static int new_tok(struct tst_expr_tok **last, const char *tok, size_t tok_len) (*last)->tok = tok; (*last)->tok_len = tok_len; (*last)->op =char_to_op (tok[0 ]); + (*last)->priv =NULL ; (*last)++;return 1 ;
Quoted 7 lines
*
tst_fuzzy_sync01: sporadically fails:../../include/tst_fuzzy_sync.h:685: TINFO: Exceeded execution loops, requesting exit
tst_fuzzy_sync01.c:227: TFAIL: acs:3 act:1 art:1 | =:23 -:46 +:2999931
...
Summary:
passed 21
failed 3
Not sure what we can do here, I guess that timings would be hard to fix on VMs that run the tests.
Quoted 75 lines
Signed-off-by Petr Vorel <[email protected]>---
Partly rewritten since v3.lib/newlib_tests/runtest.sh | 148 ++++++++++++++++++++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100755 lib/newlib_tests/runtest.shlib/newlib_tests/runtest.sh new file +61
-0@@ -0,0 +1,148 @@ +#!/bin/sh +# Copyright (c) 2021 Petr Vorel <[email protected]> + +LTP_C_API_TESTS="${LTP_C_API_TESTS:-test05 test07 test09 test12 test15 test18 +test_exec test_timer tst_res_hexd tst_strstatus tst_fuzzy_sync02 tst_fuzzy_sync03}" + +LTP_SHELL_API_TESTS="${LTP_SHELL_API_TESTS:-shell/tst_check_driver.sh shell/net/*.sh}" + +cd $(dirname $0) +PATH="$PWD/../../testcases/lib/:$PATH" + +. tst_ansi_color.sh + +usage() +{ + cat << EOF +Usage: $0 [-b DIR ] [-c|-s] +-b DIR build directory (required for out-of-tree build) +-c run C API tests only +-s run shell API tests only +-h print this help +EOF +} + +# custom version +tst_flag2mask() +{ + case "$1" in + TPASS) return 0;; + TFAIL) return 1;; + TBROK) return 2;; + TWARN) return 4;; + TINFO) return 16;; + TCONF) return 32;; + esac +} + +# custom version +tst_res() +{ + if [ $# -eq 0 ]; then + echo >&2 + return + fi + + local res="$1" + shift + + tst_color_enabled + local color=$? + + printf "runtest " >&2 + tst_print_colored $res "$res: " >&2 + echo "$@" >&2 + +} + +# custom version +tst_brk() +{ + local res="$1"
lib/newlib_tests/runtest.sh line 71
Quoted 10 lines
lib/newlib_tests/runtest.sh new file +10
-0+ shift + + tst_flag2mask "$res" + local mask=$? + + tst_res + tst_res $res $@ + + exit $mask +}
I'm not sure that we should call these function tst_res and tst_brk it only confuses everything since these are different from the ones in the test library.
Quoted 12 lines
lib/newlib_tests/runtest.sh new file +12
-0+run_tests() +{ + local target="$1" + local i ret tconf tpass vars + + eval vars="\$LTP_${target}_API_TESTS" + + tst_res TINFO "=== Run $target tests ===" + + for i in $vars; do + tst_res TINFO "* $i" + ./$i
lib/newlib_tests/runtest.sh line 93
Quoted 10 lines
lib/newlib_tests/runtest.sh new file +10
-0+ ret=$? + + case $ret in + 0) tpass="$tpass $i";; + 1) tst_brk TFAIL "$i failed with TFAIL";; + 2) tst_brk TFAIL "$i failed with TBROK";; + 4) tst_brk TFAIL "$i failed with TWARN";; + 32) tconf="$tconf $i";; + 127) tst_brk TBROK "Error: file not found (wrong PATH? out-of-tree build without -b?), exit code: $ret";; + *) tst_brk TBROK "Error: unknown failure, exit code: $ret";;
Why do we exit on failure here?
We should just increase the fail counters and go ahead with next test.
Quoted 57 lines
lib/newlib_tests/runtest.sh new file +54
-0+ esac + tst_res + done + + [ -z "$tpass" ] && tpass=" none" + [ -z "$tconf" ] && tconf=" none" + + tst_res TINFO "=== $target TEST RESULTS ===" + tst_res TINFO "Tests exited with TPASS:$tpass" + tst_res TINFO "Tests exited with TCONF:$tconf" + tst_res +} + +run_c_tests() +{ + if [ "$builddir" ]; then + cd $builddir/lib/newlib_tests + fi + + run_tests "C" + + if [ "$builddir" ]; then + cd - + fi +} + +run_shell_tests() +{ + run_tests "SHELL" +} + +builddir= +run= +while getopts b:chs opt; do + case $opt in + 'h') usage; exit 0;; + 'b') builddir=$OPTARG; PATH="$builddir/testcases/lib:$PATH";; + 'c') run="c";; + 's') run="s";; + *) usage; tst_brk TBROK "Error: invalid option";; + esac +done + +tst_res TINFO "PATH='$PATH'" + +if [ -z "$run" -o "$run" = "c" ]; then + run_c_tests +fi + +if [ -z "$run" -o "$run" = "s" ]; then + run_shell_tests +fi + +tst_res TPASS "No test failed"Signature · 1 line
2.32.0
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH v4 4/7 lib: Add script for running tests
Quoted 29 lines
Hi!
Quoted 10 lines
For now run only tests which TPASS or TCONF.
Disabled also problematic tests:
*
tst_bool_expr: for some reason killed after testing:/__w/ltp/ltp/lib/newlib_tests/tst_bool_expr.c:41: TINFO: Parsing 'A ( B )' A ( B )
Summary:
Quoted 8 lines
passed 24 failed 0 broken 0 skipped 0warnings 0
PATH: '/__w/ltp/ltp/../ltp-build/testcases/lib:/__w/ltp/ltp/lib/newlib_tests/../../testcases/lib/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
DEBUG: 0
/__w/ltp/ltp/lib/tst_test.c:1363: TBROK: Test killed by SIGSEGV!I've executed the test under valgrind and found some "conditional jump depends on uninitialized value", which is because we do not clear the priv pointer for newly added tokens. Does this patch fix it for you?
lib/tst_bool_expr.c +0
-0
lib/tst_bool_expr.c static int new_tok(struct tst_expr_tok **last, const char *tok, size_t tok_len) · line 59
Quoted 5 lines
lib/tst_bool_expr.c +1
-0@@ -55,6 +55,7 @@ static int new_tok(struct tst_expr_tok **last, const char *tok, size_t tok_len) (*last)->tok = tok; (*last)->tok_len = tok_len; (*last)->op =char_to_op (tok[0 ]); + (*last)->priv =NULL ; (*last)++;
Thanks! It looks like it helped (but few jobs haven't finished yet).
https://github.com/pevik/ltp/actions/runs/1026771350
Will you merge this fix yourself please?
Quoted 10 lines
lib/tst_bool_expr.c +0
-0return 1 ;Quoted 7 lines
*
tst_fuzzy_sync01: sporadically fails:../../include/tst_fuzzy_sync.h:685: TINFO: Exceeded execution loops, requesting exit
tst_fuzzy_sync01.c:227: TFAIL: acs:3 act:1 art:1 | =:23 -:46 +:2999931
...
Summary:
passed 21
failed 3
Quoted 2 lines
Not sure what we can do here, I guess that timings would be hard to fix on VMs that run the tests.
If I remember correctly Richie suggested that FAIL is also OK. He said only TBROK and TCONF is a problem. I'd prefer to fuzzy sync tests which always pass, but after this effort I can work on API tests metadata, which would allow also this.
...
Quoted 30 lines
Quoted 30 lines
(quoted diff) +30
-0+# custom version +tst_res() +{ + if [ $# -eq 0 ]; then + echo >&2 + return + fi + + local res="$1" + shift + + tst_color_enabled + local color=$? + + printf "runtest " >&2 + tst_print_colored $res "$res: " >&2 + echo "$@" >&2 + +} + +# custom version +tst_brk() +{ + local res="$1" + shift + + tst_flag2mask "$res" + local mask=$? + + tst_res
Quoted 8 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ tst_res $res $@ + + exit $mask +}I'm not sure that we should call these function
tst_resandtst_brkit only confuses everything since these are different from the ones in the test library.
OK, I'll rename it (runtest_res() and runtest_brk()).
Quoted 25 lines
Quoted 12 lines
(quoted diff) +12
-0+run_tests() +{ + local target="$1" + local i ret tconf tpass vars + + eval vars="\$LTP_${target}_API_TESTS" + + tst_res TINFO "=== Run $target tests ===" + + for i in $vars; do + tst_res TINFO "* $i" + ./$iReview
Quoted 10 lines
(quoted diff) +10
-0+ ret=$? + + case $ret in + 0) tpass="$tpass $i";; + 1) tst_brk TFAIL "$i failed with TFAIL";; + 2) tst_brk TFAIL "$i failed with TBROK";; + 4) tst_brk TFAIL "$i failed with TWARN";; + 32) tconf="$tconf $i";; + 127) tst_brk TBROK "Error: file not found (wrong PATH? out-of-tree build without -b?), exit code: $ret";; + *) tst_brk TBROK "Error: unknown failure, exit code: $ret";;Why do we exit on failure here?
Quoted 1 line
We should just increase the fail counters and go ahead with next test.
I quit here because you know how hard is to find error in very long log file. Also why to waste developer time when some test failed? Similar approach make has. But sure, I can continue here and print summary at the end.
Thanks for your review!
I merged the first 3 commits.
Kind regards,
Petr
PATCH v4 4/7 lib: Add script for running tests
Hi!
Quoted 3 lines
Thanks! It looks like it helped (but few jobs haven't finished yet).
https://github.com/pevik/ltp/actions/runs/1026771350
Will you merge this fix yourself please?
It was obvious bug in the code, so I pushed it with yours Reported-by and Tested-by.
Quoted 7 lines
Quoted 2 lines
Not sure what we can do here, I guess that timings would be hard to fix on VMs that run the tests.
If I remember correctly Richie suggested that FAIL is also OK. He said only TBROK and TCONF is a problem. I'd prefer to fuzzy sync tests which always pass, but after this effort I can work on API tests metadata, which would allow also this.
Another possibility would be relaxing the timings on VMs. I guess that we could change the treshold for reaching the criticall section to 10 on VMs.
Quoted 65 lines
Quoted 30 lines
Quoted 30 lines
(quoted diff) +30
-0+# custom version +tst_res() +{ + if [ $# -eq 0 ]; then + echo >&2 + return + fi + + local res="$1" + shift + + tst_color_enabled + local color=$? + + printf "runtest " >&2 + tst_print_colored $res "$res: " >&2 + echo "$@" >&2 + +} + +# custom version +tst_brk() +{ + local res="$1" + shift + + tst_flag2mask "$res" + local mask=$? + + tst_resQuoted 8 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ tst_res $res $@ + + exit $mask +}I'm not sure that we should call these function
tst_resandtst_brkit only confuses everything since these are different from the ones in the test library.OK, I'll rename it (
runtest_res()andruntest_brk()).Quoted 24 lines
Quoted 12 lines
(quoted diff) +12
-0+run_tests() +{ + local target="$1" + local i ret tconf tpass vars + + eval vars="\$LTP_${target}_API_TESTS" + + tst_res TINFO "=== Run $target tests ===" + + for i in $vars; do + tst_res TINFO "* $i" + ./$iReview
Quoted 10 lines
(quoted diff) +10
-0+ ret=$? + + case $ret in + 0) tpass="$tpass $i";; + 1) tst_brk TFAIL "$i failed with TFAIL";; + 2) tst_brk TFAIL "$i failed with TBROK";; + 4) tst_brk TFAIL "$i failed with TWARN";; + 32) tconf="$tconf $i";; + 127) tst_brk TBROK "Error: file not found (wrong PATH? out-of-tree build without -b?), exit code: $ret";; + *) tst_brk TBROK "Error: unknown failure, exit code: $ret";;Why do we exit on failure here?
Quoted 5 lines
Quoted 1 line
We should just increase the fail counters and go ahead with next test.
I quit here because you know how hard is to find error in very long log file. Also why to waste developer time when some test failed? Similar approach make has. But sure, I can continue here and print summary at the end.
As long as we run it on CI we really should run all tests.
Signature · 2 lines
Cyril Hrubis [email protected]
PATCH v4 4/7 lib: Add script for running tests
Hi all,
...
Quoted 11 lines
Quoted 7 lines
Quoted 2 lines
Not sure what we can do here, I guess that timings would be hard to fix on VMs that run the tests.
If I remember correctly Richie suggested that FAIL is also OK. He said only TBROK and TCONF is a problem. I'd prefer to fuzzy sync tests which always pass, but after this effort I can work on API tests metadata, which would allow also this.
Another possibility would be relaxing the timings on VMs. I guess that we could change the treshold for reaching the criticall section to 10 on VMs.
I'd be for it, we have checks for VM. @Richie, any opinion about it?
Kind regards,
Petr
PATCH v4 5/7 make: Add make test{, -c, -shell} targets
In reply to Petr VorelFor testing C and shell API.
---
changes v3->v4:
Don't run on cross-compile.
Makefile | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
Makefile +23 -0
@@ -192,6 +192,29 @@ $(INSTALL_TARGETS): $(INSTALL_DIR) $(DESTDIR)/$(bindir) ## Install install: $(INSTALL_TARGETS) +## Test +define _test + @set -e; $(top_srcdir)/lib/newlib_tests/runtest.sh -b $(abs_builddir) $(1) +endef + +test: lib-all +ifneq ($(build),$(host)) + $(error running tests on cross-compile build not supported) +endif + $(call _test) + +test-c: lib-all +ifneq ($(build),$(host)) + $(error running tests on cross-compile build not supported) +endif + $(call _test,-c) + +test-shell: lib-all +ifneq ($(build),$(host)) + $(error running tests on cross-compile build not supported) +endif + $(call _test,-s) + ## Help .PHONY: help help:
Signature · 1 line
2.32.0
PATCH v4 5/7 make: Add make test{, -c, -shell} targets
PATCH v4 6/7 build.sh: Add support for make test
In reply to Petr Vorel---
changes v3->v4:
Don't run on cross-compile.
build.sh | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-)
build.sh +21 -1
@@ -119,6 +119,17 @@ build_out_tree() make $MAKE_OPTS_OUT_TREE } +test_in_tree() +{ + make test +} + +test_out_tree() +{ + cd $BUILD_DIR + make $MAKE_OPTS_OUT_TREE test +} + install_in_tree() { make $MAKE_OPTS install
@@ -165,6 +176,7 @@ RUN: autotools run only 'make autotools' configure run only 'configure' build run only 'make' +test run only 'make test' (not supported for cross-compile build) install run only 'make install' Default configure options:
@@ -192,7 +204,7 @@ while getopts "c:hio:p:r:t:" opt; do esac;; p) prefix="$OPTARG";; r) case "$OPTARG" in - autotools|configure|build|install) run="$OPTARG";; + autotools|configure|build|test|install) run="$OPTARG";; *) echo "Wrong run type '$OPTARG'" >&2; usage; exit 1;; esac;; t) case "$OPTARG" in
@@ -218,6 +230,14 @@ if [ -z "$run" -o "$run" = "build" ]; then eval build_${tree}_tree fi +if [ -z "$run" -o "$run" = "test" ]; then + if [ "$build" = "cross" ]; then + echo "cross-compile build, skipping running tests" >&2 + else + eval test_${tree}_tree + fi +fi + if [ -z "$run" -o "$run" = "install" ]; then if [ "$install" = 1 ]; then eval install_${tree}_tree
Signature · 1 line
2.32.0
PATCH v4 6/7 build.sh: Add support for make test
PATCH v4 7/7 CI: Run also make test
In reply to Petr Vorelon all targets
---
changes v3->v4:
Add $BUILD to not run on cross-compile.
.github/workflows/ci.yml | 5 +++++ 1 file changed, 5 insertions(+)
.github/workflows/ci.yml +5 -0
@@ -151,6 +151,11 @@ jobs: - name: Compile run: ./build.sh -r build -o ${TREE:-in} + - name: Test + run: | + case "$VARIANT" in cross-compile*) BUILD="cross";; i386) BUILD="32";; *) BUILD="native";; esac + ./build.sh -r test -o ${TREE:-in} -t $BUILD + - name: Install run: | if [ "$MAKE_INSTALL" = 1 ]; then INSTALL_OPT="-i"; fi
Signature · 1 line
2.32.0