make test: Run C API tests
cover make test: Run C API tests
Hi,
this is very simple way to add 'make test' target. It seems tests are outdated and some of them maybe not meant to be unit shell tests. I removed tst_safe_macros as it's failing and not sufficient enough. We should write proper test for it.
Current implementation is still failing: after 'make test' run, there is still failing tst_checkpoint_wait_timeout.
tst_checkpoint_wait_timeout 1 TBROK : tst_checkpoint.c:144: tst_checkpoint_wait_timeout.c:46: tst_checkpoint_wait(0, 10000): errno=ETIMEDOUT(110): Connection timed out tst_checkpoint_wait_timeout 2 TBROK : tst_checkpoint.c:144: Remaining cases broken
Kind regards,
Petr
Petr Vorel (2):
tests: Remove tst_safe_macros.c
make: Add test target
Makefile | 3 +++ lib/tests/Makefile | 3 +++ lib/tests/test.sh | 32 +++++++++++++++++++++++++++++ lib/tests/tst_safe_macros.c | 40 ------------------------------------- 4 files changed, 38 insertions(+), 40 deletions(-) create mode 100755 lib/tests/test.sh delete mode 100644 lib/tests/tst_safe_macros.c
Signature · 1 line
2.18.0
RFC PATCH 1/2 tests: Remove tst_safe_macros.c
It's broken and outdated (doesn't really test safe macros features).
--- lib/tests/tst_safe_macros.c | 40 ------------------------------------- 1 file changed, 40 deletions(-) delete mode 100644 lib/tests/tst_safe_macros.c
lib/tests/tst_safe_macros.c deleted +0 -40
@@ -1,40 +0,0 @@ -#include "test.h" -#include "safe_macros.h" - -char *TCID ="test_safe_macros" ; -int TST_TOTAL =1 ; - -int fd =-1 ; - -void cleanup (void ) -{ -SAFE_CLOSE (NULL , fd); -SAFE_UNLINK (NULL , __FILE__"~" ); -tst_resm (TINFO ,"got here" ); -} - -int main(int argc LTP_ATTRIBUTE_UNUSED, char **argv) -{ -char buf[10 ]; -int fds[2 ]; - - buf[9 ] ='\0' ; - -if (system ("cp " __FILE__" " __FILE__"~" )) { -fprintf (stderr,"error: could not cp file\n" ); -return 1 ; - } -printf ("%s\n" ,SAFE_BASENAME (NULL , *argv)); -printf ("%s\n" ,SAFE_DIRNAME (NULL , *argv)); - fd =SAFE_OPEN (cleanup, __FILE__"~" ,O_RDWR ); -SAFE_READ (cleanup,0 , fd, buf,9 ); -printf ("buf: %s\n" , buf); -SAFE_READ (cleanup,1 , fd, buf,9 ); -printf ("buf: %s\n" , buf); -SAFE_WRITE (cleanup,0 ,-1 , buf,9 ); -SAFE_WRITE (NULL ,0 , fd, buf,9 ); -SAFE_WRITE (NULL ,1 , fd, buf,9 ); -SAFE_PIPE (NULL , fds); - -return 0 ; -}
Signature · 1 line
2.18.0
RFC PATCH 2/2 make: Add test target
In reply to Petr Vorel--- Makefile | 3 +++ lib/tests/Makefile | 3 +++ lib/tests/test.sh | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 38 insertions(+) create mode 100755 lib/tests/test.sh
Makefile +3 -0
@@ -89,6 +89,9 @@ MAKE_TARGETS := $(addsuffix -all,$(filter-out lib,$(COMMON_TARGETS))) # overtaxed one, or one where -j => 1 was specified. all: $(addsuffix -all,$(COMMON_TARGETS)) Version +test: + cd lib/tests && make $@ + $(MAKE_TARGETS): lib-all .PHONY: include-all include-install
lib/tests/Makefile +3 -0
@@ -7,4 +7,7 @@ LDLIBS += -lltp tst_cleanup_once: CFLAGS += -pthread +test: + ./test.sh $(MAKE_TARGETS) + include $(top_srcdir)/include/mk/generic_leaf_target.mk
lib/tests/test.sh new file +32 -0
@@ -0,0 +1,32 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2016-2018 Petr Vorel <[email protected]> + +set -e + +for i in $@; do + echo "=== Testing '$i' ===" + case $i in + tst_checkpoint_wake_timeout|tst_record_childstatus) + if [ "$i" = "tst_record_childstatus" ]; then + echo "NOTE: expecting fail the test" + ./$i || [ $? -eq 1 ] + fi + ;; + tst_device) + if [ $(id -u) -ne 0 ]; then + echo "WARN: not root, skip the test" + else + ./$i + fi + ;; + *) + ./$i + ;; + esac + echo +done + +echo "END OF TESTING" + +# vim: set ft=sh ts=4 sts=4 sw=4 expandtab :
Signature · 1 line
2.18.0
RFC PATCH 2/2 make: Add test target
----- Original Message -----
Quoted 7 lines
Signed-off-by Petr Vorel <[email protected]>--- Makefile | 3 +++ lib/tests/Makefile | 3 +++ lib/tests/test.sh | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 38 insertions(+) create mode 100755 lib/tests/test.sh
lib/tests/test.sh line 14
Quoted 9 lines
lib/tests/test.sh new file +9
-0+ +for i in $@; do + echo "=== Testing '$i' ===" + case $i in + tst_checkpoint_wake_timeout|tst_record_childstatus) + if [ "$i" = "tst_record_childstatus" ]; then + echo "NOTE: expecting fail the test" + ./$i || [ $? -eq 1 ] + fi
I'd prefer we fix the tests and make them return 0 if everything goes as expected.
Regards,
Jan
RFC PATCH 2/2 make: Add test target
In reply to Petr VorelHi!
Quoted 44 lines
Makefile +3
-0@@ -89,6 +89,9 @@ MAKE_TARGETS := $(addsuffix -all,$(filter-out lib,$(COMMON_TARGETS))) # overtaxed one, or one where -j => 1 was specified. all: $(addsuffix -all,$(COMMON_TARGETS)) Version +test: + cd lib/tests && make $@ + $(MAKE_TARGETS): lib-all .PHONY: include-all include-installlib/tests/Makefile +3
-0@@ -7,4 +7,7 @@ LDLIBS += -lltp tst_cleanup_once: CFLAGS += -pthread +test: + ./test.sh $(MAKE_TARGETS) + include $(top_srcdir)/include/mk/generic_leaf_target.mklib/tests/test.sh new file +12
-0@@ -0,0 +1,32 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (c) 2016-2018 Petr Vorel <[email protected]> + +set -e + +for i in $@; do + echo "=== Testing '$i' ===" + case $i in + tst_checkpoint_wake_timeout|tst_record_childstatus) + if [ "$i" = "tst_record_childstatus" ]; then + echo "NOTE: expecting fail the test"
lib/tests/test.sh line 22
Quoted 10 lines
lib/tests/test.sh new file +10
-0+ ./$i || [ $? -eq 1 ] + fi + ;; + tst_device) + if [ $(id -u) -ne 0 ]; then + echo "WARN: not root, skip the test" + else + ./$i + fi + ;;
Listing the testcases does not scale that much. I wonder how we can do better.
Also new library tests are stored in newlib_tests directory.
lib/tests/test.sh line 32
Quoted 10 lines
lib/tests/test.sh new file +10
-0+ *) + ./$i + ;; + esac + echo +done + +echo "END OF TESTING" + +# vim: set ft=sh ts=4 sts=4 sw=4 expandtab :
I wonder if we can try to check output of these tests against expected output. Most of the testcases have nice and stable output that is always the same. Some of the include pid in the test output that will change with each iteration and some, for instance the thread safety related tests for the new library, produce the output lines in random order.
So maybe the solution would be grepping for pre-defined patterns in the test output in the case that the test output is not stable.
Signature · 2 lines
Cyril Hrubis [email protected]
RFC PATCH 2/2 make: Add test target
In reply to Petr VorelHi Jan,
Quoted 11 lines
----- Original Message -----
Quoted 10 lines
Signed-off-by Petr Vorel <[email protected]>--- Makefile | 3 +++ lib/tests/Makefile | 3 +++ lib/tests/test.sh | 32 ++++++++++++++++++++++++++++++++ 3 files changed, 38 insertions(+) create mode 100755 lib/tests/test.shlib/tests/test.sh new file +3
-0+ +for i in $@; do + echo "=== Testing '$i' ==="
Quoted 8 lines
lib/tests/test.sh line 14
Quoted 6 lines
lib/tests/test.sh new file +6
-0+ case $i in + tst_checkpoint_wake_timeout|tst_record_childstatus) + if [ "$i" = "tst_record_childstatus" ]; then + echo "NOTE: expecting fail the test" + ./$i || [ $? -eq 1 ] + fiI'd prefer we fix the tests and make them return 0 if everything goes as expected.
Agree. But I took different folder. As Cyril pointed out, tests in lib/tests/ are for legacy API. I'm going to test new ones from lib/newlib_tests/.
As Christian works on #312 "Unit testing the shell library" [1], I'd like if both shell and C tests had the same approach of testing.
I wonder whether to keep these tests in lib/tests/. At least some of them aren't meant to be for unit testing (due expecting failing), nobody runs them, ...
[1] https://github.com/linux-test-project/ltp/issues/312
Quoted 2 lines
Regards,
Jan