don't run sparse{c,i} tests when sparse-llvm is disabled

PATCH In review Ramsay Jones 2 participants 6 messages Aug 4, 2014 – Oct 2, 2014 Sparse checker

lore

Share & tools

lore

Ramsay Jones

author

2014-08-04 18:38 UTC #0 lore raw

PATCH 6/10 don't run sparse{c,i} tests when sparse-llvm is disabled

Signed-off-by Ramsay Jones <[email protected]>

---
 validation/test-suite | 25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)

validation/test-suite +23 -2

index 3c011c6..0142701 100755
@@ -7,12 +7,18 @@ default_cmd="sparse \$file"
 tests_list=`find . -name '*.c' | sed -e 's#^\./\(.*\)#\1#' | sort`
 prog_name=`basename $0`
 
+if [ ! -x "$default_path/sparse-llvm" ]; then
+	disabled_cmds="sparsec sparsei sparse-llvm"
+fi
+
 # counts:
 #	- tests that have not been converted to test-suite format
+#	- tests that are disabled
 #	- tests that passed
 #	- tests that failed
 #	- tests that failed but are known to fail
 unhandled_tests=0
+disabled_tests=0
 ok_tests=0
 ko_tests=0
 known_ko_tests=0

@@ -80,6 +86,7 @@ echo "    help                       prints usage"
 #	- 0 if the test passed,
 #	- 1 if it failed,
 #	- 2 if it is not a "test-suite" test.
+#	- 3 if the test is disabled.
 do_test()
 {
 	test_failed=0

@@ -95,8 +102,6 @@ do_test()
 	fi
 	test_name=$last_result
 
-	echo "     TEST    $test_name ($file)"
-
 	# does the test provide a specific command ?
 	cmd=`eval echo $default_path/$default_cmd`
 	get_value "check-command" $file

@@ -104,6 +109,19 @@ do_test()
 		last_result=`echo $last_result | sed -e 's/^ *//'`
 		cmd=`eval echo $default_path/$last_result`
 	fi
+
+	# check for disabled commands
+	for i in $disabled_cmds; do
+		case "$cmd" in
+		*$i*)
+			disabled_tests=`expr $disabled_tests + 1`
+			return 3
+			;;
+		esac
+	done
+
+	echo "     TEST    $test_name ($file)"
+
 	verbose "Using command       : $cmd"
 
 	# grab the expected exit value

@@ -168,6 +186,9 @@ do_test_suite()
 	if [ "$unhandled_tests" -ne "0" ]; then
 		echo "$unhandled_tests tests could not be handled by $prog_name"
 	fi
+	if [ "$disabled_tests" -ne "0" ]; then
+		echo "$disabled_tests tests were disabled"
+	fi
 }
 
 ##

Signature · 1 line

2.0.0

Christopher Li

maintainer

2014-09-27 04:29 UTC #1 lore raw

validation/test-suite default_cmd="sparse \$file" · line 12

Quoted 3 lines · Ramsay Jones

validation/test-suite +3 -0

+if [ ! -x "$default_path/sparse-llvm" ]; then
+       disabled_cmds="sparsec sparsei sparse-llvm"
+fi

I think this can be simplified as if the test command was not found, then that test will be disabled. It doesn't need to be sparse llvm, it can be other
test program.

We don't need to set an disable_cmds list then check against that list.

Chris

Ramsay Jones

author

2014-09-27 13:37 UTC #2 lore raw

Quoted 9 lines · Christopher Li

validation/test-suite default_cmd="sparse \$file" · line 12
Quoted 3 lines · Ramsay Jones
validation/test-suite +3 -0
+if [ ! -x "$default_path/sparse-llvm" ]; then
+       disabled_cmds="sparsec sparsei sparse-llvm"
+fi
I think this can be simplified as if the test command was not found, then that test will be disabled. It doesn't need to be sparse llvm, it can be other
test program.

Unfortunately, sparcec and sparsei are the commands that are used by the tests (I included sparse-llvm in the list in case any _future_ tests used it directly), and they do exist and are executable. They simply will not work if sparse-llvm is not built.

I did think about removing the x-bit on sparse{c,i} in git, enabling it again in the worktree if sparse-llvm was built, but I was concerned that would not work on cygwin and it would possibly complicate installation. ie I didn't try it out seriously. This patch seemed the best solution to me.

[Another idea I didn't try: name the sparse{c,i} scripts sparse{c,i}-in and rename them as part of a successful build of sparse-llvm]

BTW, the 'validation/backend/sum.c' fails for me on cygwin (64-bit), but I think it is actually a problem with the llvm interpreter lli. (it works fine on my old 32-bit Linux laptop).

ATB,
Ramsay Jones

Christopher Li

maintainer

2014-09-30 03:26 UTC #3 lore raw

Quoted 8 lines · Ramsay Jones

On 27/09/14 05:29, Christopher Li wrote:
Unfortunately, sparcec and sparsei are the commands that are used by the tests (I included sparse-llvm in the list in case any _future_ tests used it directly), and they do exist and are executable. They simply will not work if sparse-llvm is not built.

I see. That make sense.

We just need a way to specify the dependency of test commands. e.g. sparsec depends on sparse-llvm. So if sparsec does not exist, or sparse-llvm does not exist, the test will be skipped.

I am thinking some thing in the test-suite like:

set_deps sparsec sparse-llvm
set_deps sparsei sparse-llvm

...
get_deps basename cmd
dep_cmd=$last_result

Quoted 4 lines

Note that I renamed sparse{c,i} to sparse{c,i}-in. It may have been better to name then sparse{c,i}.sh, since they are not really templates to be 'processed' to create a build product. dunno.

I prefer specify the dependency. Another copy is not necessary.

Chris

Christopher Li

maintainer

2014-10-02 03:44 UTC #4 lore raw

Quoted 8 lines · Christopher Li

I am thinking some thing in the test-suite like:
set_deps sparsec sparse-llvm
set_deps sparsei sparse-llvm
...
get_deps basename cmd
dep_cmd=$last_result

Actually, I try that, it is a little bit too complicated.

Your first approach is fine.

I end up using your V1 patch, with a small change to do exact match rather than a wild cast. The wild cast has danger that match a sub word of the command. e.g. if you try to disable command "sp", it will disable "sparse" as well.

Chris

validation/test-suite +4 -4

index 1c94aa8..784ff6b 100755
@@ -111,13 +111,13 @@ do_test()
        fi
 
        # check for disabled commands
+       set -- $cmd
+       base_cmd=`basename $1`
        for i in $disabled_cmds; do
-               case "$cmd" in
-               *$i*)
+               if [ "$i" == "$base_cmd" ] ; then
                        disabled_tests=`expr $disabled_tests + 1`
                        return 3
-                       ;;
-               esac
+               fi
        done
 
        echo "     TEST    $test_name ($file)"

Ramsay Jones

author

2014-10-02 10:31 UTC #5 lore raw

Quoted 10 lines · Christopher Li

Quoted 8 lines · Christopher Li
I am thinking some thing in the test-suite like:
set_deps sparsec sparse-llvm
set_deps sparsei sparse-llvm
...
get_deps basename cmd
dep_cmd=$last_result

Quoted 7 lines

Actually, I try that, it is a little bit too complicated.
Your first approach is fine.
I end up using your V1 patch, with a small change to do exact match rather than a wild cast. The wild cast has danger that match a sub word of the command. e.g. if you try to disable command "sp", it will disable "sparse" as well.

Ah yes, good catch!

ATB,
Ramsay Jones