Use same test names in the default and the TAP13 output format

Since commit 6e1d3752d7ca ("tap13: list testcases individually") the comment is no longer valid. Therefore let's remove it.

Signed-off-by Marc Hartmayer <[email protected]>

---
 scripts/runtime.bash | 3 ---
 1 file changed, 3 deletions(-)

scripts/runtime.bash +0 -3

index c88e246245a6..caa4c5ba18cc 100644
@@ -53,9 +53,6 @@ skip_nodefault()
 
 function print_result()
 {
-    # output test results in a TAP format
-    # https://testanything.org/tap-version-13-specification.html
-
     local status="$1"
     local testname="$2"
     local summary="$3"

Signature · 1 line

2.25.4

Since commit 6e1d3752d7ca ("tap13: list testcases individually") the comment is no longer valid. Therefore let's remove it.

Signed-off-by Marc Hartmayer <[email protected]>

---
 scripts/runtime.bash | 3 ---
 1 file changed, 3 deletions(-)

Use the same test names in the TAP13 output as in the default output format. This makes the output more consistent. To achieve this, we need to pass the test name as an argument to the function process_test_output.

Before this change:

$ ./run_tests.sh
PASS selftest-setup (14 tests)
...

vs.

$ ./run_tests.sh -t
TAP version 13
ok 1 - selftest: true
ok 2 - selftest: argc == 3
...

After this change:

$ ./run_tests.sh
PASS selftest-setup (14 tests)
...

vs.

$ ./run_tests.sh -t
TAP version 13
ok 1 - selftest-setup: true
ok 2 - selftest-setup: argc == 3
...

While at it, introduce a local variable kernel in RUNTIME_log_stdout since this makes the function easier to read.

Signed-off-by Marc Hartmayer <[email protected]>

---
 run_tests.sh         | 15 +++++++++------
 scripts/runtime.bash |  6 +++---
 2 files changed, 12 insertions(+), 9 deletions(-)

run_tests.sh +9 -6

index 01e36dcfa06e..b5812336866f 100755
@@ -81,18 +81,19 @@ if [[ $tap_output == "no" ]]; then
     postprocess_suite_output() { cat; }
 else
     process_test_output() {
+        local testname="$1"
         CR=$'\r'
         while read -r line; do
             line="${line%$CR}"
             case "${line:0:4}" in
                 PASS)
-                    echo "ok TEST_NUMBER - ${line#??????}" >&3
+                    echo "ok TEST_NUMBER - ${testname}:${line#*:*:}" >&3
                     ;;
                 FAIL)
-                    echo "not ok TEST_NUMBER - ${line#??????}" >&3
+                    echo "not ok TEST_NUMBER - ${testname}:${line#*:*:}" >&3
                     ;;
                 SKIP)
-                    echo "ok TEST_NUMBER - ${line#??????} # skip" >&3
+                    echo "ok TEST_NUMBER - ${testname}:${line#*:*:} # skip" >&3
                     ;;
                 *)
                     ;;

@@ -114,12 +115,14 @@ else
     }
 fi
 
-RUNTIME_log_stderr () { process_test_output; }
+RUNTIME_log_stderr () { process_test_output "$1"; }
 RUNTIME_log_stdout () {
+    local testname="$1"
     if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then
-        ./scripts/pretty_print_stacks.py $1 | process_test_output
+        local kernel="$2"
+        ./scripts/pretty_print_stacks.py "$kernel" | process_test_output "$testname"
     else
-        process_test_output
+        process_test_output "$testname"
     fi
 }
 

scripts/runtime.bash +3 -3

index caa4c5ba18cc..294e6b15a5e2 100644
@@ -140,10 +140,10 @@ function run()
     # extra_params in the config file may contain backticks that need to be
     # expanded, so use eval to start qemu.  Use "> >(foo)" instead of a pipe to
     # preserve the exit status.
-    summary=$(eval $cmdline 2> >(RUNTIME_log_stderr) \
-                             > >(tee >(RUNTIME_log_stdout $kernel) | extract_summary))
+    summary=$(eval $cmdline 2> >(RUNTIME_log_stderr $testname) \
+                             > >(tee >(RUNTIME_log_stdout $testname $kernel) | extract_summary))
     ret=$?
-    [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $kernel)
+    [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $testname $kernel)
 
     if [ $ret -eq 0 ]; then
         print_result "PASS" $testname "$summary"

Signature · 1 line

2.25.4

Use the same test names in the TAP13 output as in the default output format. This makes the output more consistent. To achieve this, we need to pass the test name as an argument to the function process_test_output.

Before this change:

$ ./run_tests.sh
PASS selftest-setup (14 tests)
...

vs.

$ ./run_tests.sh -t
TAP version 13
ok 1 - selftest: true
ok 2 - selftest: argc == 3
...

After this change:

$ ./run_tests.sh
PASS selftest-setup (14 tests)
...

vs.

$ ./run_tests.sh -t
TAP version 13
ok 1 - selftest-setup: true
ok 2 - selftest-setup: argc == 3

...

While at it, introduce a local variable kernel in RUNTIME_log_stdout since this makes the function easier to read.

Signed-off-by Marc Hartmayer <[email protected]>

---
 run_tests.sh         | 15 +++++++++------
 scripts/runtime.bash |  6 +++---
 2 files changed, 12 insertions(+), 9 deletions(-)

run_tests.sh +9 -6

index 01e36dcfa06e..b5812336866f 100755
@@ -81,18 +81,19 @@ if [[ $tap_output == "no" ]]; then
     postprocess_suite_output() { cat; }
 else
     process_test_output() {
+        local testname="$1"
         CR=$'\r'
         while read -r line; do
             line="${line%$CR}"
             case "${line:0:4}" in
                 PASS)
-                    echo "ok TEST_NUMBER - ${line#??????}" >&3
+                    echo "ok TEST_NUMBER - ${testname}:${line#*:*:}" >&3
                     ;;
                 FAIL)
-                    echo "not ok TEST_NUMBER - ${line#??????}" >&3
+                    echo "not ok TEST_NUMBER - ${testname}:${line#*:*:}" >&3
                     ;;
                 SKIP)
-                    echo "ok TEST_NUMBER - ${line#??????} # skip" >&3
+                    echo "ok TEST_NUMBER - ${testname}:${line#*:*:} # skip" >&3
                     ;;
                 *)
                     ;;

@@ -114,12 +115,14 @@ else
     }
 fi
 
-RUNTIME_log_stderr () { process_test_output; }
+RUNTIME_log_stderr () { process_test_output "$1"; }
 RUNTIME_log_stdout () {
+    local testname="$1"
     if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then
-        ./scripts/pretty_print_stacks.py $1 | process_test_output
+        local kernel="$2"
+        ./scripts/pretty_print_stacks.py "$kernel" | process_test_output "$testname"
     else
-        process_test_output
+        process_test_output "$testname"
     fi
 }
 

scripts/runtime.bash +3 -3

index caa4c5ba18cc..294e6b15a5e2 100644
@@ -140,10 +140,10 @@ function run()
     # extra_params in the config file may contain backticks that need to be
     # expanded, so use eval to start qemu.  Use "> >(foo)" instead of a pipe to
     # preserve the exit status.
-    summary=$(eval $cmdline 2> >(RUNTIME_log_stderr) \
-                             > >(tee >(RUNTIME_log_stdout $kernel) | extract_summary))
+    summary=$(eval $cmdline 2> >(RUNTIME_log_stderr $testname) \
+                             > >(tee >(RUNTIME_log_stdout $testname $kernel) | extract_summary))
     ret=$?
-    [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $kernel)
+    [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $testname $kernel)
 
     if [ $ret -eq 0 ]; then
         print_result "PASS" $testname "$summary"

Quoted 23 lines · Marc Hartmayer

Use the same test names in the TAP13 output as in the default output format. This makes the output more consistent. To achieve this, we need to pass the test name as an argument to the function process_test_output.

Before this change:

$ ./run_tests.sh
PASS selftest-setup (14 tests)
...

vs.

$ ./run_tests.sh -t
TAP version 13
ok 1 - selftest: true
ok 2 - selftest: argc == 3
...

After this change:

$ ./run_tests.sh
PASS selftest-setup (14 tests)
...

Quoted 6 lines

vs.

$ ./run_tests.sh -t
TAP version 13
ok 1 - selftest-setup: true
ok 2 - selftest-setup: argc == 3

This doesn't work for me, we can't just drop prefixes.

I.e. it needs to be "testname: prefix1: prefix2: prefixn: Test text"
selftest-setup: selftest: true