Skip to contentSkip to conversation

report: add support for the xunit-quiet format

PATCH In review Theodore Ts'o 2 participants 2 messages Jul 8, 2022 – Jul 10, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 report: add support for the xunit-quiet format

The xunit-quiet format exludes the NNN.{full,dmesg,bad} files in <system-out> and <system-err> nodes.

For test runners that save the enttire results directory to preserve all of the test artifacts, capturing the NNN.{full,dmesg,bad} in the results.xml file is redundant. In addition, if the NNN.bad is too large, it can cause the junitparser python library to refuse to parse the XML file to prevent potential denial of service attacks[1]. A simple way to avoid this problem is to simply to have xfstests to omit the <system-out> and <system-err> nodes.

[1] https://gitlab.com/gitlab-org/gitlab/-/issues/268035

---

Note: this patch depends on David Disseldorp's "add option to rerun failed tests" series.

 check         |  2 +-
 common/report | 20 ++++++++++++++------
 2 files changed, 15 insertions(+), 7 deletions(-)
check +1 -1
index 27ebb0c2..d4d436c0 100755
@@ -75,7 +75,7 @@ check options
     -I <n>		iterate the test list <n> times, but stops iterating further in case of any test failure
     -d			dump test output to stdout
     -b			brief test summary
-    -R fmt[,fmt]	generate report in formats specified. Supported format: [xunit]
+    -R fmt[,fmt]	generate report in formats specified. Supported formats: xunit, xunit-quiet
     --large-fs		optimise scratch device for large filesystems
     -s section		run only specified section from config file
     -S section		exclude the specified section from the config file
common/report +14 -6
index 5ca41bc4..cdf36b3e 100644
@@ -71,6 +71,12 @@ _xunit_make_testcase_report()
 	local test_name="$2"
 	local test_status="$3"
 	local test_time="$4"
+	local report_format="$5"
+	local quiet
+
+	if [ "$report_format" = xunit-quiet ]; then
+	    quiet=yes
+	fi
 
 	# TODO: other places may also win if no-section mode will be named like 'default/global'
 	if [ $sect_name == '-no-sections-' ]; then
@@ -104,14 +110,16 @@ _xunit_make_testcase_report()
 			_err_msg="Test $test_name failed, reason unknown"
 		fi
 		echo -e "\t\t<failure message=\"$_err_msg\" type=\"TestFail\" />" >> $report
-		if [ -s "$full_file" ]; then
+		if [ -z "$quiet" -a -s "$full_file" ]; then
 			echo -e "\t\t<system-out>" >> $report
 			printf	'<![CDATA[\n' >>$report
 			cat "$full_file" | tr -dc '[:print:][:space:]' | encode_xml >>$report
 			printf ']]>\n'	>>$report
 			echo -e "\t\t</system-out>" >> $report
 		fi
-		if [ -f "$dmesg_file" ]; then
+		if [ -n "$quiet" ]; then
+		   :
+		elif [ -f "$dmesg_file" ]; then
 			echo -e "\t\t<system-err>" >> $report
 			printf	'<![CDATA[\n' >>$report
 			cat "$dmesg_file" | tr -dc '[:print:][:space:]' | encode_xml >>$report
@@ -144,7 +152,7 @@ _make_section_report()
 	local sect_time="$5"
 	for report in $REPORT_LIST; do
 		case "$report" in
-		"xunit")
+		"xunit"|"xunit-quiet")
 			_xunit_make_section_report "$sect_name" "$tests_count" \
 						   "$bad_count" "$notrun_count" \
 						   "$sect_time"
@@ -164,9 +172,9 @@ _make_testcase_report()
 	local test_time="$4"
 	for report in $REPORT_LIST; do
 		case "$report" in
-		"xunit")
+		"xunit"|"xunit-quiet")
 			_xunit_make_testcase_report "$sect_name" "$test_seq" \
-						    "$test_status" "$test_time"
+						    "$test_status" "$test_time" "$report"
 			;;
 		*)
 			_dump_err "report format '$report' is not supported"
@@ -178,7 +186,7 @@ _make_testcase_report()
 _assert_report_list() {
 	for report in $REPORT_LIST; do
 		case "$report" in
-		"xunit")
+		"xunit"|"xunit-quiet")
 			;;
 		*)
 			_fatal "report format '$report' is not supported"
Signature · 1 line
2.31.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 11 lines · Theodore Ts'o

The xunit-quiet format exludes the NNN.{full,dmesg,bad} files in <system-out> and <system-err> nodes.

For test runners that save the enttire results directory to preserve all of the test artifacts, capturing the NNN.{full,dmesg,bad} in the results.xml file is redundant. In addition, if the NNN.bad is too large, it can cause the junitparser python library to refuse to parse the XML file to prevent potential denial of service attacks[1]. A simple way to avoid this problem is to simply to have xfstests to omit the <system-out> and <system-err> nodes.

Quoted 3 lines

[1] https://gitlab.com/gitlab-org/gitlab/-/issues/268035

Looks fine. A minor nit below...

Quoted 27 lines

---

Note: this patch depends on David Disseldorp's "add option to rerun failed tests" series.

 check         |  2 +-
 common/report | 20 ++++++++++++++------
 2 files changed, 15 insertions(+), 7 deletions(-)
check +1 -1
index 27ebb0c2..d4d436c0 100755
@@ -75,7 +75,7 @@ check options
     -I <n>		iterate the test list <n> times, but stops iterating further in case of any test failure
     -d			dump test output to stdout
     -b			brief test summary
-    -R fmt[,fmt]	generate report in formats specified. Supported format: [xunit]
+    -R fmt[,fmt]	generate report in formats specified. Supported formats: xunit, xunit-quiet
     --large-fs		optimise scratch device for large filesystems
     -s section		run only specified section from config file
     -S section		exclude the specified section from the config file
common/report +0 -0
index 5ca41bc4..cdf36b3e 100644

common/report _xunit_make_testcase_report() · line 78

Quoted 8 lines
common/report +5 -0
@@ -71,6 +71,12 @@ _xunit_make_testcase_report()
 	local test_name="$2"
 	local test_status="$3"
 	local test_time="$4"
+	local report_format="$5"
+	local quiet
+
+	if [ "$report_format" = xunit-quiet ]; then
+	    quiet=yes
        ^^^^

Nit: xfstests mostly uses tab based indentation. Same for ':' lower down