selftests: sync test conversion to TAP13

PATCH In review Shuah Khan 3 participants 6 messages Jul 24, 2017 – Jul 28, 2017 Kselftest

Share & tools

Shuah Khan

author

2017-07-24 21:07 UTC #0 lore raw

cover selftests: sync test conversion to TAP13

This patch series includes patches to convert sync test to use TAP13 ksft framework. In addition, fix to sync test to differentiate between unsupported feature and access error when a non-root user runs it.

Updated kfst framework to return counters for sync test to use to print the final pass or fail summary message. Updated ksft+print_cnts() to print counters to summarize the test results.

Shuah Khan (3):
selftests: sync: differentiate between sync unsupported and access
errors
selftests: kselftest framework: add API to return pass/fail/* counts
selftests: sync: convert to use TAP13 ksft framework

 tools/testing/selftests/kselftest.h      | 10 +++++
 tools/testing/selftests/sync/sync_test.c | 71 +++++++++++++++++++++-----------
 tools/testing/selftests/sync/synctest.h  |  3 +-
 3 files changed, 58 insertions(+), 26 deletions(-)

Signature · 1 line

2.11.0

Shuah Khan

author

2017-07-24 21:08 UTC #1 lore raw

PATCH 1/3 selftests: sync: differentiate between sync unsupported and access errors

Sync test doesn't differentiate between sync unsupported and test run by non-root user and treats both as unsupported cases.

Fix it to add handling for these two different scenarios.

Signed-off-by Shuah Khan <[email protected]>

---
 tools/testing/selftests/sync/sync_test.c | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

tools/testing/selftests/sync/sync_test.c +19 -4

index 62fa666e501a..86ae45ad0347 100644
@@ -31,6 +31,7 @@
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <sys/wait.h>
+#include <errno.h>
 
 #include "synctest.h"

@@ -56,18 +57,32 @@ static int run_test(int (*test)(void), char *name)
 static int sync_api_supported(void)
 {
 	struct stat sbuf;
+	int ret;
+
+	ret = stat("/sys/kernel/debug/sync/sw_sync", &sbuf);
+	if (!ret)
+		return 0;
+
+	if (errno == ENOENT) {
+		printf("SKIP: Sync framework not supported by kernel\n");
+		exit(0);
+	}
+	if (errno == EACCES) {
+		printf("SKIP: Run Sync test as root.\n");
+		exit(0);
+	}
+
+	perror("stat");
+	exit(ret);
 
-	return 0 == stat("/sys/kernel/debug/sync/sw_sync", &sbuf);
 }
 
 int main(void)
 {
 	int err = 0;
 
-	if (!sync_api_supported()) {
-		printf("SKIP: Sync framework not supported by kernel\n");
+	if (!sync_api_supported())
 		return 0;
-	}
 
 	printf("[RUN]\tTesting sync framework\n");

Signature · 1 line

2.11.0

Shuah Khan

author

2017-07-24 21:08 UTC #2 lore raw

PATCH 2/3 selftests: kselftest framework: add API to return pass/fail/* counts

In reply to Shuah Khan

Some tests print final pass/fail message based on fail count. Add ksft_get_*_cnt() API to kselftest framework to return counts.

Update ksft_print_cnts() to print the test results summary message with individual pass, fail, ... counters.

Signed-off-by Shuah Khan <[email protected]>

---
 tools/testing/selftests/kselftest.h | 10 ++++++++++
 1 file changed, 10 insertions(+)

tools/testing/selftests/kselftest.h +10 -0

index 08e90c2cc5cb..45bf25905279 100644
@@ -45,6 +45,12 @@ static inline void ksft_inc_xfail_cnt(void) { ksft_cnt.ksft_xfail++; }
 static inline void ksft_inc_xpass_cnt(void) { ksft_cnt.ksft_xpass++; }
 static inline void ksft_inc_xskip_cnt(void) { ksft_cnt.ksft_xskip++; }
 
+static inline int ksft_get_pass_cnt(void) { return ksft_cnt.ksft_pass; }
+static inline int ksft_get_fail_cnt(void) { return ksft_cnt.ksft_fail; }
+static inline int ksft_get_xfail_cnt(void) { return ksft_cnt.ksft_xfail; }
+static inline int ksft_get_xpass_cnt(void) { return ksft_cnt.ksft_xpass; }
+static inline int ksft_get_xskip_cnt(void) { return ksft_cnt.ksft_xskip; }
+
 static inline void ksft_print_header(void)
 {
 	printf("TAP version 13\n");

@@ -52,6 +58,10 @@ static inline void ksft_print_header(void)
 
 static inline void ksft_print_cnts(void)
 {
+	printf("Pass %d Fail %d Xfail %d Xpass %d Skip %d\n",
+		ksft_cnt.ksft_pass, ksft_cnt.ksft_fail,
+		ksft_cnt.ksft_xfail, ksft_cnt.ksft_xpass,
+		ksft_cnt.ksft_xskip);
 	printf("1..%d\n", ksft_test_num());
 }

Signature · 1 line

2.11.0

Shuah Khan

author

2017-07-24 21:08 UTC #3 lore raw

PATCH 3/3 selftests: sync: convert to use TAP13 ksft framework

In reply to Shuah Khan

Convert test to use TAP13 ksft framework. Output after conversion:

TAP version 13
# [RUN] Testing sync framework
ok 1 [RUN] test_alloc_timeline
ok 2 [RUN] test_alloc_fence
ok 3 [RUN] test_alloc_fence_negative
ok 4 [RUN] test_fence_one_timeline_wait
ok 5 [RUN] test_fence_one_timeline_merge
ok 6 [RUN] test_fence_merge_same_fence
ok 7 [RUN] test_fence_multi_timeline_wait
ok 8 [RUN] test_stress_two_threads_shared_timeline
ok 9 [RUN] test_consumer_stress_multi_producer_single_consumer
ok 10 [RUN] test_merge_stress_random_merge
Pass 10 Fail 0 Xfail 0 Xpass 0 Skip 0
1..10

Signed-off-by Shuah Khan <[email protected]>

---
 tools/testing/selftests/sync/sync_test.c | 72 +++++++++++++++++---------------
 tools/testing/selftests/sync/synctest.h  |  3 +-
 2 files changed, 41 insertions(+), 34 deletions(-)

tools/testing/selftests/sync/sync_test.c +39 ~~-33~~

index 86ae45ad0347..7f7938263c5c 100644
@@ -32,76 +32,82 @@
 #include <sys/stat.h>
 #include <sys/wait.h>
 #include <errno.h>
+#include <string.h>
 
+#include "../kselftest.h"
 #include "synctest.h"
 
 static int run_test(int (*test)(void), char *name)
 {
 	int result;
 	pid_t childpid;
+	int ret;
 
 	fflush(stdout);
 	childpid = fork();
 
 	if (childpid) {
 		waitpid(childpid, &result, 0);
-		if (WIFEXITED(result))
-			return WEXITSTATUS(result);
+		if (WIFEXITED(result)) {
+			ret = WEXITSTATUS(result);
+			if (!ret)
+				ksft_test_result_pass("[RUN]\t%s\n", name);
+			else
+				ksft_test_result_fail("[RUN]\t%s\n", name);
+			return ret;
+		}
 		return 1;
 	}
 
-	printf("[RUN]\tExecuting %s\n", name);
 	exit(test());
 }
 
-static int sync_api_supported(void)
+static void sync_api_supported(void)
 {
 	struct stat sbuf;
 	int ret;
 
 	ret = stat("/sys/kernel/debug/sync/sw_sync", &sbuf);
 	if (!ret)
-		return 0;
+		return;
 
-	if (errno == ENOENT) {
-		printf("SKIP: Sync framework not supported by kernel\n");
-		exit(0);
-	}
-	if (errno == EACCES) {
-		printf("SKIP: Run Sync test as root.\n");
-		exit(0);
-	}
+	if (errno == ENOENT)
+		ksft_exit_skip("Sync framework not supported by kernel\n");
 
-	perror("stat");
-	exit(ret);
+	if (errno == EACCES)
+		ksft_exit_skip("Run Sync test as root.\n");
 
+	ksft_exit_fail_msg("stat failed on /sys/kernel/debug/sync/sw_sync: %s",
+				strerror(errno));
 }
 
 int main(void)
 {
-	int err = 0;
+	int err;
+
+	ksft_print_header();
 
-	if (!sync_api_supported())
-		return 0;
+	sync_api_supported();
 
-	printf("[RUN]\tTesting sync framework\n");
+	ksft_print_msg("[RUN]\tTesting sync framework\n");
 
-	err += RUN_TEST(test_alloc_timeline);
-	err += RUN_TEST(test_alloc_fence);
-	err += RUN_TEST(test_alloc_fence_negative);
+	RUN_TEST(test_alloc_timeline);
+	RUN_TEST(test_alloc_fence);
+	RUN_TEST(test_alloc_fence_negative);
 
-	err += RUN_TEST(test_fence_one_timeline_wait);
-	err += RUN_TEST(test_fence_one_timeline_merge);
-	err += RUN_TEST(test_fence_merge_same_fence);
-	err += RUN_TEST(test_fence_multi_timeline_wait);
-	err += RUN_TEST(test_stress_two_threads_shared_timeline);
-	err += RUN_TEST(test_consumer_stress_multi_producer_single_consumer);
-	err += RUN_TEST(test_merge_stress_random_merge);
+	RUN_TEST(test_fence_one_timeline_wait);
+	RUN_TEST(test_fence_one_timeline_merge);
+	RUN_TEST(test_fence_merge_same_fence);
+	RUN_TEST(test_fence_multi_timeline_wait);
+	RUN_TEST(test_stress_two_threads_shared_timeline);
+	RUN_TEST(test_consumer_stress_multi_producer_single_consumer);
+	RUN_TEST(test_merge_stress_random_merge);
 
+	err = ksft_get_fail_cnt();
 	if (err)
-		printf("[FAIL]\tsync errors: %d\n", err);
-	else
-		printf("[OK]\tsync\n");
+		ksft_exit_fail_msg("%d out of %d sync tests failed\n",
+					err, ksft_test_num());
 
-	return !!err;
+	/* need this return to keep gcc happy */
+	return ksft_exit_pass();
 }

tools/testing/selftests/sync/synctest.h +2 -1

index e7d1d57dba7a..90a8e5369914 100644
@@ -29,10 +29,11 @@
 #define SELFTESTS_SYNCTEST_H
 
 #include <stdio.h>
+#include "../kselftest.h"
 
 #define ASSERT(cond, msg) do { \
 	if (!(cond)) { \
-		printf("[ERROR]\t%s", (msg)); \
+		ksft_print_msg("[ERROR]\t%s", (msg)); \
 		return 1; \
 	} \
 } while (0)

Signature · 1 line

2.11.0

Gustavo Padovan

Reviewed-by on the v1 series

2017-07-28 18:25 UTC #4 lore raw

In reply to Shuah Khan

Hi Shuah,

Thank you for your patches.

Quoted 17 lines · Shuah Khan

This patch series includes patches to convert sync test to use TAP13 ksft framework. In addition, fix to sync test to differentiate between
unsupported feature and access error when a non-root user runs it.
Updated kfst framework to return counters for sync test to use to print
the final pass or fail summary message. Updated ksft+print_cnts() to print counters to summarize the test results.
Shuah Khan (3):
selftests: sync: differentiate between sync unsupported and access errors
selftests: kselftest framework: add API to return pass/fail/* counts
selftests: sync: convert to use TAP13 ksft framework

Quoted 5 lines

tools/testing/selftests/kselftest.h | 10 +++++ tools/testing/selftests/sync/sync_test.c | 71 +++++++++++++++++++++-
----------
tools/testing/selftests/sync/synctest.h | 3 +-
 3 files changed, 58 insertions(+), 26 deletions(-)

All looks good to me, so for the 3 patches:

Reviewed-by Gustavo Padovan <[email protected]>

Gustavo

Signature · 2 lines

Gustavo Padovan
Collabora Ltd.

Shuah Khan

maintainer

2017-07-28 19:12 UTC #5 lore raw

Quoted 28 lines · Gustavo Padovan

Hi Shuah,
Thank you for your patches.
Quoted 17 lines · Shuah Khan
This patch series includes patches to convert sync test to use TAP13 ksft framework. In addition, fix to sync test to differentiate between
unsupported feature and access error when a non-root user runs it.
Updated kfst framework to return counters for sync test to use to print
the final pass or fail summary message. Updated ksft+print_cnts() to print counters to summarize the test results.
Shuah Khan (3):
selftests: sync: differentiate between sync unsupported and access
errors
selftests: kselftest framework: add API to return pass/fail/*
counts
selftests: sync: convert to use TAP13 ksft framework
Quoted 5 lines
 tools/testing/selftests/kselftest.h      | 10 +++++
 tools/testing/selftests/sync/sync_test.c | 71 +++++++++++++++++++++-
----------
 tools/testing/selftests/sync/synctest.h  |  3 +-
 3 files changed, 58 insertions(+), 26 deletions(-)

Quoted 5 lines

All looks good to me, so for the 3 patches:
Reviewed-by Gustavo Padovan <[email protected]>
Gustavo

Thanks Gustavo. I will get them in to linux-ksefltest next for 4.14-rc1 with your Reviwed-by.

-- Shuah