Skip to contentSkip to conversation

build: make check target

RFC PATCH Superseded Petr Vorel 4 participants 9 messages Jun 3, 2021 – Jun 4, 2021 Kbuild

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Petr Vorel

author
#0 lore raw

cover build: make check target

Hi,

actually, there are 3 targets: check, check-c, check-shell.

Previous attempt: make: Add make check target + run.sh [1]. This is built on the top of patchset tst_net.sh: Allow to skip initialization [2]. Hope this time library check will made it :).

I plan to wrap tests with a script in the future, which will parse expected output, i.e. approach from #312 [3].

It's not ready due several problems, but for curious here is a test run:
https://github.com/pevik/ltp/runs/2739826773

1) I made something wrong, I have no idea how to fix this error:
make[1]: Entering directory 'ltp/lib' ../include/mk/generic_trunk_target.inc:105: warning: overriding recipe for target 'check-c' ../include/mk/generic_leaf_target.inc:110: warning: ignoring old recipe for target 'check-c' ../include/mk/generic_trunk_target.inc:105: warning: overriding recipe for target 'check-shell' ../include/mk/generic_leaf_target.inc:118: warning: ignoring old recipe for target 'check-shell'

2) I obviously have problem with out-of-tree build
https://github.com/pevik/ltp/runs/2739826612?check_suite_focus=true
where path of shell tests (which are in src directory only) is obviously
broken:
make[2]: *** No rule to make target 'shell/tst_check_driver.sh', needed by 'check-shell'. Stop. 20
make[1]: *** [/__w/ltp/ltp/include/mk/generic_trunk_target.inc:105: check-shell] Error 2 21
make[1]: Leaving directory '/__w/ltp/ltp-build/lib'

3) tst_bool_expr fails on some systems when run via make check:
https://github.com/pevik/ltp/runs/2739826773?check_suite_focus=true
But it runs ok when running normally. It might be dash issue (it's on Debian).

Kind regards,
Petr

[1] https://patchwork.ozlabs.org/project/ltp/patch/[email protected]/
[2] https://patchwork.ozlabs.org/project/ltp/list/?series=247078
[3] https://github.com/linux-test-project/ltp/issues/312

Petr Vorel (3):
make: Add make check{,-c,-shell} targets
build.sh: Add support for make check
CI: Run also make check

 .github/workflows/ci.yml            |  3 +++
 Makefile                            | 21 ++++++++++++++++++++-
 build.sh                            | 18 +++++++++++++++++-
 include/mk/generic_leaf_target.inc  | 25 ++++++++++++++++++++++---
 include/mk/generic_trunk_target.inc |  4 ++--
 lib/newlib_tests/Makefile           |  8 ++++++++
 6 files changed, 72 insertions(+), 7 deletions(-)
Signature · 1 line
2.31.1
author
#1 lore raw

RFC PATCH 1/3 make: Add make check{,-c,-shell} targets

For testing C and shell API.

---
 Makefile                            | 21 ++++++++++++++++++++-
 include/mk/generic_leaf_target.inc  | 25 ++++++++++++++++++++++---
 include/mk/generic_trunk_target.inc |  4 ++--
 lib/newlib_tests/Makefile           |  8 ++++++++
 4 files changed, 52 insertions(+), 6 deletions(-)
Makefile +20 -1
index 56812d77b..acea4551e 100644
@@ -75,10 +75,14 @@ endif
 INSTALL_TARGETS		+= $(COMMON_TARGETS)
 CLEAN_TARGETS		+= $(COMMON_TARGETS) lib libs
 BOOTSTRAP_TARGETS	:= $(sort $(COMMON_TARGETS) $(CLEAN_TARGETS) $(INSTALL_TARGETS))
+TEST_TARGETS		:= lib
+TEST_SHELL_TARGETS	:= lib
 
 CLEAN_TARGETS		:= $(addsuffix -clean,$(CLEAN_TARGETS))
 INSTALL_TARGETS		:= $(addsuffix -install,$(INSTALL_TARGETS))
 MAKE_TARGETS		:= $(addsuffix -all,$(filter-out lib,$(COMMON_TARGETS)))
+TEST_TARGETS		:= $(addsuffix -check-c,$(TEST_TARGETS))
+TEST_SHELL_TARGETS	:= $(addsuffix -check-shell,$(TEST_SHELL_TARGETS))
 
 # There's no reason why we should run `all' twice. Otherwise we're just wasting
 # 3+ mins of useful CPU cycles on a modern machine, and even more time on an
@@ -113,6 +117,14 @@ $(filter-out include-clean,$(CLEAN_TARGETS))::
 	-$(MAKE) -C "$(subst -clean,,$@)" \
 		 -f "$(abs_top_srcdir)/$(subst -clean,,$@)/Makefile" clean
 
+$(TEST_TARGETS): lib-all
+	$(MAKE) -C "$(subst -check-c,,$@)" \
+		-f "$(abs_top_srcdir)/$(subst -check-c,,$@)/Makefile" check-c
+
+$(TEST_SHELL_TARGETS):
+	$(MAKE) -C "$(subst -check-shell,,$@)" \
+		-f "$(abs_top_srcdir)/$(subst -check-shell,,$@)/Makefile" check-shell
+
 # Just like everything depends on include-all / -install, we need to get rid
 # of include last to ensure that things won't be monkey screwed up. Only do
 # this if we're invoking clean or a subclean directly though.
@@ -189,8 +201,15 @@ INSTALL_TARGETS		+= $(addprefix $(DESTDIR)/$(bindir)/,$(BINDIR_INSTALL_SCRIPTS))
 
 $(INSTALL_TARGETS): $(INSTALL_DIR) $(DESTDIR)/$(bindir)
 
+.PHONY: check check-c check-shell
+
+## Check
+check-c: $(TEST_TARGETS)
+check-shell: $(TEST_SHELL_TARGETS)
+check: check-c check-shell
+
 ## Install
-install: $(INSTALL_TARGETS)
+install: lib-all $(INSTALL_TARGETS)
 
 ## Help
 .PHONY: help
include/mk/generic_leaf_target.inc +22 -3
index 64953f89a..d53623456 100644
@@ -63,8 +63,12 @@
 #				  real files or directories). This will automatically append
 #				  adds the .o suffix to all files referenced by
 #				  $(MAKE_TARGETS)) to CLEAN_TARGETS, if MAKE_TARGETS wasn't
-#				  defined (see
-#				  $(MAKE_TARGETS)).
+#				  defined (see $(MAKE_TARGETS)).
+#
+# $(TEST_TARGETS)		: LTP C API tests.
+#
+# $(TEST_SHELL_TARGETS)		: LTP shell API tests.
+#
 # $(INSTALL_MODE)		: What mode should we using when calling
 # 				  install(1)?
 #
@@ -92,7 +96,7 @@
 # INSTALL_DIR			:= $(libdir)
 #
 
-.PHONY: all clean install
+.PHONY: all check-c check-shell clean install
 
 ifneq ($(strip $(MAKE_TARGETS)),)
 $(MAKE_TARGETS) += $(HOST_MAKE_TARGETS)
@@ -102,6 +106,21 @@ $(MAKE_TARGETS): | $(MAKE_DEPS)
 
 all: $(MAKE_TARGETS)
 
+check-c: $(TEST_TARGETS)
+	@set -e; echo; echo "===== Test C API ====="; \
+	for i in $(TEST_TARGETS); do \
+	    echo; echo "* $$i"; \
+	    echo "PATH $(top_srcdir)/testcases/lib:$$PATH"; \
+	    PATH="$(top_srcdir)/testcases/lib:$$PATH" ./$$i || [ $$? -eq 32 ]; \
+	done
+
+check-shell: $(TEST_SHELL_TARGETS)
+	@set -e; echo; echo "===== Test shell API ====="; \
+	for i in $(TEST_SHELL_TARGETS); do \
+	    echo; echo "* $$i"; \
+	    PATH="$(abs_top_srcdir)/testcases/lib:$(abs_top_builddir)/testcases/lib:$$PATH" $(abs_srcdir)/$$i || [ $$? -eq 32 ]; \
+	done
+
 clean:: $(CLEAN_DEPS)
 	-$(RM) -f -r $(CLEAN_TARGETS)
 
include/mk/generic_trunk_target.inc +2 -2
index fc59f944f..d195fe3bd 100644
@@ -48,7 +48,7 @@
 
 include $(top_srcdir)/include/mk/functions.mk
 
-RECURSIVE_TARGETS		?= all install
+RECURSIVE_TARGETS		?= all check check-c check-shell install
 
 $(eval $(get_make_dirs))
 
@@ -71,7 +71,7 @@ trunk-install: $(INSTALL_FILES)
 # Avoid creating duplicate .PHONY references to all, clean, and install. IIRC,
 # I've seen some indeterministic behavior when one does this in the past with
 # GNU Make...
-.PHONY: $(filter-out $(RECURSIVE_TARGETS),all clean install)
+.PHONY: $(filter-out $(RECURSIVE_TARGETS),all check clean install)
 all: trunk-all
 
 clean:: trunk-clean
lib/newlib_tests/Makefile +8 -0
index 30ca6810c..182ef5689 100644
@@ -19,5 +19,13 @@ ifeq ($(ANDROID),1)
 FILTER_OUT_MAKE_TARGETS	+= test08
 endif
 
+# C API tests (which exit 0)
+TEST_TARGETS += test0[579] test1[2568] test_exec test_timer
+TEST_TARGETS += tst_bool_expr tst_res_hexd tst_strstatus
+TEST_TARGETS += tst_fuzzy_sync0[1-2]
+
+# shell API tests (which exit 0)
+TEST_SHELL_TARGETS += shell/tst_check_driver.sh
+TEST_SHELL_TARGETS += shell/net/*.sh
 
 include $(top_srcdir)/include/mk/generic_leaf_target.mk
Signature · 1 line
2.31.1
#2 lore raw

RFC PATCH 1/3 make: Add make check{,-c,-shell} targets

Hi!

Quoted 116 lines

For testing C and shell API.

---
 Makefile                            | 21 ++++++++++++++++++++-
 include/mk/generic_leaf_target.inc  | 25 ++++++++++++++++++++++---
 include/mk/generic_trunk_target.inc |  4 ++--
 lib/newlib_tests/Makefile           |  8 ++++++++
 4 files changed, 52 insertions(+), 6 deletions(-)
Makefile +20 -1
index 56812d77b..acea4551e 100644
@@ -75,10 +75,14 @@ endif
 INSTALL_TARGETS		+= $(COMMON_TARGETS)
 CLEAN_TARGETS		+= $(COMMON_TARGETS) lib libs
 BOOTSTRAP_TARGETS	:= $(sort $(COMMON_TARGETS) $(CLEAN_TARGETS) $(INSTALL_TARGETS))
+TEST_TARGETS		:= lib
+TEST_SHELL_TARGETS	:= lib
 
 CLEAN_TARGETS		:= $(addsuffix -clean,$(CLEAN_TARGETS))
 INSTALL_TARGETS		:= $(addsuffix -install,$(INSTALL_TARGETS))
 MAKE_TARGETS		:= $(addsuffix -all,$(filter-out lib,$(COMMON_TARGETS)))
+TEST_TARGETS		:= $(addsuffix -check-c,$(TEST_TARGETS))
+TEST_SHELL_TARGETS	:= $(addsuffix -check-shell,$(TEST_SHELL_TARGETS))
 
 # There's no reason why we should run `all' twice. Otherwise we're just wasting
 # 3+ mins of useful CPU cycles on a modern machine, and even more time on an
@@ -113,6 +117,14 @@ $(filter-out include-clean,$(CLEAN_TARGETS))::
 	-$(MAKE) -C "$(subst -clean,,$@)" \
 		 -f "$(abs_top_srcdir)/$(subst -clean,,$@)/Makefile" clean
 
+$(TEST_TARGETS): lib-all
+	$(MAKE) -C "$(subst -check-c,,$@)" \
+		-f "$(abs_top_srcdir)/$(subst -check-c,,$@)/Makefile" check-c
+
+$(TEST_SHELL_TARGETS):
+	$(MAKE) -C "$(subst -check-shell,,$@)" \
+		-f "$(abs_top_srcdir)/$(subst -check-shell,,$@)/Makefile" check-shell
+
 # Just like everything depends on include-all / -install, we need to get rid
 # of include last to ensure that things won't be monkey screwed up. Only do
 # this if we're invoking clean or a subclean directly though.
@@ -189,8 +201,15 @@ INSTALL_TARGETS		+= $(addprefix $(DESTDIR)/$(bindir)/,$(BINDIR_INSTALL_SCRIPTS))
 
 $(INSTALL_TARGETS): $(INSTALL_DIR) $(DESTDIR)/$(bindir)
 
+.PHONY: check check-c check-shell
+
+## Check
+check-c: $(TEST_TARGETS)
+check-shell: $(TEST_SHELL_TARGETS)
+check: check-c check-shell
+
 ## Install
-install: $(INSTALL_TARGETS)
+install: lib-all $(INSTALL_TARGETS)
 
 ## Help
 .PHONY: help
include/mk/generic_leaf_target.inc +22 -3
index 64953f89a..d53623456 100644
@@ -63,8 +63,12 @@
 #				  real files or directories). This will automatically append
 #				  adds the .o suffix to all files referenced by
 #				  $(MAKE_TARGETS)) to CLEAN_TARGETS, if MAKE_TARGETS wasn't
-#				  defined (see
-#				  $(MAKE_TARGETS)).
+#				  defined (see $(MAKE_TARGETS)).
+#
+# $(TEST_TARGETS)		: LTP C API tests.
+#
+# $(TEST_SHELL_TARGETS)		: LTP shell API tests.
+#
 # $(INSTALL_MODE)		: What mode should we using when calling
 # 				  install(1)?
 #
@@ -92,7 +96,7 @@
 # INSTALL_DIR			:= $(libdir)
 #
 
-.PHONY: all clean install
+.PHONY: all check-c check-shell clean install
 
 ifneq ($(strip $(MAKE_TARGETS)),)
 $(MAKE_TARGETS) += $(HOST_MAKE_TARGETS)
@@ -102,6 +106,21 @@ $(MAKE_TARGETS): | $(MAKE_DEPS)
 
 all: $(MAKE_TARGETS)
 
+check-c: $(TEST_TARGETS)
+	@set -e; echo; echo "===== Test C API ====="; \
+	for i in $(TEST_TARGETS); do \
+	    echo; echo "* $$i"; \
+	    echo "PATH $(top_srcdir)/testcases/lib:$$PATH"; \
+	    PATH="$(top_srcdir)/testcases/lib:$$PATH" ./$$i || [ $$? -eq 32 ]; \
+	done
+
+check-shell: $(TEST_SHELL_TARGETS)
+	@set -e; echo; echo "===== Test shell API ====="; \
+	for i in $(TEST_SHELL_TARGETS); do \
+	    echo; echo "* $$i"; \
+	    PATH="$(abs_top_srcdir)/testcases/lib:$(abs_top_builddir)/testcases/lib:$$PATH" $(abs_srcdir)/$$i || [ $$? -eq 32 ]; \
+	done
+
 clean:: $(CLEAN_DEPS)
 	-$(RM) -f -r $(CLEAN_TARGETS)
 
include/mk/generic_trunk_target.inc +0 -0
index fc59f944f..d195fe3bd 100644

include/mk/generic_trunk_target.inc line 51

Quoted 5 lines
include/mk/generic_trunk_target.inc +1 -1
@@ -48,7 +48,7 @@
 
 include $(top_srcdir)/include/mk/functions.mk
 
-RECURSIVE_TARGETS		?= all install
+RECURSIVE_TARGETS		?= all check check-c check-shell install

If you add a target to RECURSIVE_TARGETS such target automatically traverses directories, that what the target is for. See the rule at the end of the generic_trunk_target.inc. Which means that you redefine targets if you define the same targets to loop and run the tests above.

I guess that you want add another target that depends on check targets to actually run the tests?

Maybe:

check-c: check-c-run

check-c-run: $(TEST_TARGETS)

	for i in $(TEST_TARGETS); do
		...
	done

in order to run the tests.

In this case the check* targets would be used for directory traversal and the check-c-run would be used to run the tests.

include/mk/generic_trunk_target.inc trunk-install: $(INSTALL_FILES) · line 74

Quoted 8 lines
include/mk/generic_trunk_target.inc +1 -1
 $(eval $(get_make_dirs))
 
@@ -71,7 +71,7 @@ trunk-install: $(INSTALL_FILES)
 # Avoid creating duplicate .PHONY references to all, clean, and install. IIRC,
 # I've seen some indeterministic behavior when one does this in the past with
 # GNU Make...
-.PHONY: $(filter-out $(RECURSIVE_TARGETS),all clean install)
+.PHONY: $(filter-out $(RECURSIVE_TARGETS),all check clean install)

The check is already in RECURSIVE_TARGETS

Quoted 24 lines
include/mk/generic_trunk_target.inc +0 -0
 all: trunk-all
 
 clean:: trunk-clean
lib/newlib_tests/Makefile +8 -0
index 30ca6810c..182ef5689 100644
@@ -19,5 +19,13 @@ ifeq ($(ANDROID),1)
 FILTER_OUT_MAKE_TARGETS	+= test08
 endif
 
+# C API tests (which exit 0)
+TEST_TARGETS += test0[579] test1[2568] test_exec test_timer
+TEST_TARGETS += tst_bool_expr tst_res_hexd tst_strstatus
+TEST_TARGETS += tst_fuzzy_sync0[1-2]
+
+# shell API tests (which exit 0)
+TEST_SHELL_TARGETS += shell/tst_check_driver.sh
+TEST_SHELL_TARGETS += shell/net/*.sh
 
 include $(top_srcdir)/include/mk/generic_leaf_target.mk
Signature · 1 line
2.31.1
Signature · 2 lines
Cyril Hrubis
[email protected]
author
#3 lore raw

RFC PATCH 1/3 make: Add make check{,-c,-shell} targets

Hi Cyril,

...

Quoted 5 lines
Quoted 4 lines

+++ b/include/mk/generic_trunk_target.inc

include/mk/generic_trunk_target.inc +0 -0
@@ -48,7 +48,7 @@
 
 include $(top_srcdir)/include/mk/functions.mk
Quoted 7 lines

include/mk/generic_trunk_target.inc line 51

Quoted 2 lines
include/mk/generic_trunk_target.inc +1 -1
-RECURSIVE_TARGETS		?= all install
+RECURSIVE_TARGETS		?= all check check-c check-shell install

If you add a target to RECURSIVE_TARGETS such target automatically traverses directories, that what the target is for. See the rule at the end of the generic_trunk_target.inc. Which means that you redefine targets if you define the same targets to loop and run the tests above.

Ah, thanks!

Quoted 8 lines

I guess that you want add another target that depends on check targets to actually run the tests?

Maybe:

check-c: check-c-run

check-c-run: $(TEST_TARGETS)

Quoted 8 lines
	for i in $(TEST_TARGETS); do
		...
	done

in order to run the tests.

In this case the check* targets would be used for directory traversal and the check-c-run would be used to run the tests.

Sounds reasonable.

Quoted 10 lines

include/mk/generic_trunk_target.inc trunk-install: $(INSTALL_FILES) · line 74

Quoted 8 lines
include/mk/generic_trunk_target.inc +1 -1
 $(eval $(get_make_dirs))
 
@@ -71,7 +71,7 @@ trunk-install: $(INSTALL_FILES)
 # Avoid creating duplicate .PHONY references to all, clean, and install. IIRC,
 # I've seen some indeterministic behavior when one does this in the past with
 # GNU Make...
-.PHONY: $(filter-out $(RECURSIVE_TARGETS),all clean install)
+.PHONY: $(filter-out $(RECURSIVE_TARGETS),all check clean install)

The check is already in RECURSIVE_TARGETS

Thanks!

I'll send v2 on Monday, which also use test* as target name instead of check* as we agreed on Richie's Libclang based analyzer patchset [1].

Kind regards,
Petr

[1] https://lists.linux.it/pipermail/ltp/2021-June/023008.html

#4 lore raw

RFC PATCH 1/3 make: Add make check{,-c,-shell} targets

In reply to Petr Vorel
Quoted 4 lines · Petr Vorel

For testing C and shell API.

Why not just add a single ?make check? target that calls the C and shell targets to match what most open source projects do in terms of testing? Also, continuing on in light of errors seems like a good default behavior to have as it would prevent a single error/failure from gathering all other results.

My 2 cents,
-Enji
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linux.it/pipermail/ltp/attachments/20210604/1b27c9a2/attachment-0001.sig>

author
#5 lore raw

RFC PATCH 1/3 make: Add make check{,-c,-shell} targets

Hi Enji,

Quoted 5 lines
Quoted 3 lines · Petr Vorel

For testing C and shell API.

Why not just add a single ?make check? target that calls the C and shell targets to match what most open source projects do in terms of testing? Also, continuing on in light of errors seems like a good default behavior to have as it would prevent a single error/failure from gathering all other results.

Yes, the original approach in this patchset is make check,
which consists of two make-c and make-shell. One of tst_fuzzy_sync checks is quite long and in longer term the number of tests for both C and shell API might grow to prolong the testing. Thus one may appreciate to test for local development just one of these two.

Or, if you mean how it has been implemented (adding tests into variables), I plan to add docparse tests (into docparse/tests/) - i.e. into different directory.

Also, result of Richie's Libclang based analyzer patchset [1], where he also added make check, we decided to use make test here [2].

Quoted 1 line

My 2 cents,

Thanks, comments are always welcome.

Quoted 1 line

-Enji

Kind regards,
Petr

[1] https://lists.linux.it/pipermail/ltp/2021-June/023008.html
[2] https://lists.linux.it/pipermail/ltp/2021-June/023021.html

author
#6 lore raw

RFC PATCH 2/3 build.sh: Add support for make check

In reply to Petr Vorel
---
 build.sh | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)
build.sh +17 -1
index 240ce8e68..c892fa87b 100755
@@ -119,6 +119,17 @@ build_out_tree()
 	make $MAKE_OPTS_OUT_TREE
 }
 
+check_in_tree()
+{
+	make check
+}
+
+check_out_tree()
+{
+	cd $BUILD_DIR
+	make $MAKE_OPTS_OUT_TREE check
+}
+
 install_in_tree()
 {
 	make $MAKE_OPTS install
@@ -165,6 +176,7 @@ RUN:
 autotools   run only 'make autotools'
 configure   run only 'configure'
 build       run only 'make'
+check       run only 'make check'
 install     run only 'make install'
 
 Default configure options:
@@ -192,7 +204,7 @@ while getopts "c:hio:p:r:t:" opt; do
 		esac;;
 	p) prefix="$OPTARG";;
 	r) case "$OPTARG" in
-		autotools|configure|build|install) run="$OPTARG";;
+		autotools|configure|build|check|install) run="$OPTARG";;
 		*) echo "Wrong run type '$OPTARG'" >&2; usage; exit 1;;
 		esac;;
 	t) case "$OPTARG" in
@@ -218,6 +230,10 @@ if [ -z "$run" -o "$run" = "build" ]; then
 	eval build_${tree}_tree
 fi
 
+if [ -z "$run" -o "$run" = "check" ]; then
+	eval check_${tree}_tree
+fi
+
 if [ -z "$run" -o "$run" = "install" ]; then
 	if [ "$install" = 1 ]; then
 		eval install_${tree}_tree
Signature · 1 line
2.31.1
author
#7 lore raw

RFC PATCH 3/3 CI: Run also make check

In reply to Petr Vorel

ona all targets

---
 .github/workflows/ci.yml | 3 +++
 1 file changed, 3 insertions(+)
.github/workflows/ci.yml +3 -0
index 185f49e41..01e2578fa 100644
@@ -151,6 +151,9 @@ jobs:
     - name: Compile
       run: ./build.sh -r build -o ${TREE:-in}
 
+    - name: Test
+      run: ./build.sh -r check -o ${TREE:-in}
+
     - name: Install
       run: |
         if [ "$MAKE_INSTALL" = 1 ]; then INSTALL_OPT="-i"; fi
Signature · 1 line
2.31.1
#8 lore raw

cover build: make check target

In reply to Petr Vorel

Hello Petr,

Quoted 3 lines · Petr Vorel

Hi,

actually, there are 3 targets: check, check-c, check-shell.

Usually, if you only want to make a sub group of tests. Then you cd into the sub directory and do 'make'.

So why do we need three targets?

Just an idea, but maybe in a future patch this could run some actual tests as well? A lot of the old syscall tests would be fine to run in CI. More likely if they fail it is due to LTP library changes than a kernel bug on the host.

Quoted 55 lines

Previous attempt: make: Add make check target + run.sh [1]. This is built on the top of patchset tst_net.sh: Allow to skip initialization [2]. Hope this time library check will made it :).

I plan to wrap tests with a script in the future, which will parse expected output, i.e. approach from #312 [3].

It's not ready due several problems, but for curious here is a test run:
https://github.com/pevik/ltp/runs/2739826773

1) I made something wrong, I have no idea how to fix this error:
make[1]: Entering directory 'ltp/lib' ../include/mk/generic_trunk_target.inc:105: warning: overriding recipe for target 'check-c' ../include/mk/generic_leaf_target.inc:110: warning: ignoring old recipe for target 'check-c' ../include/mk/generic_trunk_target.inc:105: warning: overriding recipe for target 'check-shell' ../include/mk/generic_leaf_target.inc:118: warning: ignoring old recipe for target 'check-shell'

2) I obviously have problem with out-of-tree build
https://github.com/pevik/ltp/runs/2739826612?check_suite_focus=true
where path of shell tests (which are in src directory only) is obviously
broken:
make[2]: *** No rule to make target 'shell/tst_check_driver.sh', needed by 'check-shell'. Stop. 20
make[1]: *** [/__w/ltp/ltp/include/mk/generic_trunk_target.inc:105: check-shell] Error 2 21
make[1]: Leaving directory '/__w/ltp/ltp-build/lib'

3) tst_bool_expr fails on some systems when run via make check:
https://github.com/pevik/ltp/runs/2739826773?check_suite_focus=true
But it runs ok when running normally. It might be dash issue (it's on Debian).

Kind regards,
Petr

[1] https://patchwork.ozlabs.org/project/ltp/patch/[email protected]/
[2] https://patchwork.ozlabs.org/project/ltp/list/?series=247078
[3] https://github.com/linux-test-project/ltp/issues/312

Petr Vorel (3):
make: Add make check{,-c,-shell} targets
build.sh: Add support for make check
CI: Run also make check

 .github/workflows/ci.yml            |  3 +++
 Makefile                            | 21 ++++++++++++++++++++-
 build.sh                            | 18 +++++++++++++++++-
 include/mk/generic_leaf_target.inc  | 25 ++++++++++++++++++++++---
 include/mk/generic_trunk_target.inc |  4 ++--
 lib/newlib_tests/Makefile           |  8 ++++++++
 6 files changed, 72 insertions(+), 7 deletions(-)
Signature · 1 line
2.31.1
Signature · 2 lines
Thank you,
Richard.