Skip to contentSkip to conversation

kasan: test: Improve failure message in KUNIT_EXPECT_KASAN_FAIL()

PATCH Superseded David Gow 2 participants 3 messages Jun 4, 2021 Library code

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

David Gow

author
#0 lore raw

PATCH 1/1 kasan: test: Improve failure message in KUNIT_EXPECT_KASAN_FAIL()

The KUNIT_EXPECT_KASAN_FAIL() macro currently uses KUNIT_EXPECT_EQ() to compare fail_data.report_expected and fail_data.report_found. This always gave a somewhat useless error message on failure, but the addition of extra compile-time checking with READ_ONCE() has caused it to get much longer, and be truncated before anything useful is displayed.

Instead, just check fail_data.report_found by hand (we've just test report_expected to 'true'), and print a better failure message with KUNIT_FAIL()

Beforehand, a failure in:
KUNIT_EXPECT_KASAN_FAIL(test, ((volatile char *)area)[3100]); would looked like:
[22:00:34] [FAILED] vmalloc_oob

[22:00:34]     # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:991
[22:00:34]     Expected ({ do { extern void __compiletime_assert_705(void) __attribute__((__error__("Unsupported access size for {READ,WRITE}_ONCE()."))); if (!((sizeof(fail_data.report_expected) == sizeof(char) || sizeof(fail_data.repp
[22:00:34]     not ok 45 - vmalloc_oob

With this change, it instead looks like:
[22:04:04] [FAILED] vmalloc_oob

[22:04:04]     # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:993
[22:04:04]     KASAN failure expected in "((volatile char *)area)[3100]", but none occurred
[22:04:04]     not ok 45 - vmalloc_oob

---

Stumbled across this because the vmalloc_oob test is failing (i.e., KASAN isn't picking up an error) under qemu on my system, and the message above was horrifying. (I'll file a Bugzilla bug for the test failure today.)

Cheers,
-- David

 lib/test_kasan.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
lib/test_kasan.c +5 -3
index cacbbbdef768..deda13c9d9ff 100644
@@ -98,9 +98,11 @@ static void kasan_test_exit(struct kunit *test)
 	barrier();							\
 	expression;							\
 	barrier();							\
-	KUNIT_EXPECT_EQ(test,						\
-			READ_ONCE(fail_data.report_expected),		\
-			READ_ONCE(fail_data.report_found));		\
+	if (READ_ONCE(fail_data.report_found) == false) {		\
+		KUNIT_FAIL(test, KUNIT_SUBTEST_INDENT "KASAN failure "	\
+				"expected in \"" #expression		\
+				 "\", but none occurred");		\
+	}								\
 	if (IS_ENABLED(CONFIG_KASAN_HW_TAGS)) {				\
 		if (READ_ONCE(fail_data.report_found))			\
 			kasan_enable_tagging_sync();			\
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
#1 lore raw
Quoted 43 lines · David Gow' via kasan-dev

The KUNIT_EXPECT_KASAN_FAIL() macro currently uses KUNIT_EXPECT_EQ() to compare fail_data.report_expected and fail_data.report_found. This always gave a somewhat useless error message on failure, but the addition of extra compile-time checking with READ_ONCE() has caused it to get much longer, and be truncated before anything useful is displayed.

Instead, just check fail_data.report_found by hand (we've just test report_expected to 'true'), and print a better failure message with KUNIT_FAIL()

Beforehand, a failure in:
KUNIT_EXPECT_KASAN_FAIL(test, ((volatile char *)area)[3100]); would looked like:
[22:00:34] [FAILED] vmalloc_oob

[22:00:34]     # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:991
[22:00:34]     Expected ({ do { extern void __compiletime_assert_705(void) __attribute__((__error__("Unsupported access size for {READ,WRITE}_ONCE()."))); if (!((sizeof(fail_data.report_expected) == sizeof(char) || sizeof(fail_data.repp
[22:00:34]     not ok 45 - vmalloc_oob

With this change, it instead looks like:
[22:04:04] [FAILED] vmalloc_oob

[22:04:04]     # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:993
[22:04:04]     KASAN failure expected in "((volatile char *)area)[3100]", but none occurred
[22:04:04]     not ok 45 - vmalloc_oob

---

Stumbled across this because the vmalloc_oob test is failing (i.e., KASAN isn't picking up an error) under qemu on my system, and the message above was horrifying. (I'll file a Bugzilla bug for the test failure today.)

Cheers,
-- David

 lib/test_kasan.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
lib/test_kasan.c +0 -0
index cacbbbdef768..deda13c9d9ff 100644

lib/test_kasan.c static void kasan_test_exit(struct kunit *test) · line 102

Quoted 5 lines
lib/test_kasan.c +0 -2
@@ -98,9 +98,11 @@ static void kasan_test_exit(struct kunit *test)
        barrier();                                                      \
        expression;                                                     \
        barrier();                                                      \
-       KUNIT_EXPECT_EQ(test,                                           \
-                       READ_ONCE(fail_data.report_expected),           \

What do we have fail_data.report_expected for? Could we remove it now? I think it's unused now.

lib/test_kasan.c static void kasan_test_exit(struct kunit *test) · line 101

Quoted 2 lines
lib/test_kasan.c +1 -1
-                       READ_ONCE(fail_data.report_found));             \
+       if (READ_ONCE(fail_data.report_found) == false) {               \

if (!READ_ONCE(fail_data.report_found)) {
?

lib/test_kasan.c static void kasan_test_exit(struct kunit *test) · line 105

Quoted 4 lines
lib/test_kasan.c +4 -0
+               KUNIT_FAIL(test, KUNIT_SUBTEST_INDENT "KASAN failure "  \
+                               "expected in \"" #expression            \
+                                "\", but none occurred");              \
+       }                                                               \

Thanks,
-- Marco

author
#2 lore raw
Quoted 41 lines · Marco Elver
Quoted 38 lines · David Gow' via kasan-dev

The KUNIT_EXPECT_KASAN_FAIL() macro currently uses KUNIT_EXPECT_EQ() to compare fail_data.report_expected and fail_data.report_found. This always gave a somewhat useless error message on failure, but the addition of extra compile-time checking with READ_ONCE() has caused it to get much longer, and be truncated before anything useful is displayed.

Instead, just check fail_data.report_found by hand (we've just test report_expected to 'true'), and print a better failure message with KUNIT_FAIL()

Beforehand, a failure in:
KUNIT_EXPECT_KASAN_FAIL(test, ((volatile char *)area)[3100]); would looked like:
[22:00:34] [FAILED] vmalloc_oob

[22:00:34]     # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:991
[22:00:34]     Expected ({ do { extern void __compiletime_assert_705(void) __attribute__((__error__("Unsupported access size for {READ,WRITE}_ONCE()."))); if (!((sizeof(fail_data.report_expected) == sizeof(char) || sizeof(fail_data.repp
[22:00:34]     not ok 45 - vmalloc_oob

With this change, it instead looks like:
[22:04:04] [FAILED] vmalloc_oob

[22:04:04]     # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:993
[22:04:04]     KASAN failure expected in "((volatile char *)area)[3100]", but none occurred
[22:04:04]     not ok 45 - vmalloc_oob

---

Stumbled across this because the vmalloc_oob test is failing (i.e., KASAN isn't picking up an error) under qemu on my system, and the message above was horrifying. (I'll file a Bugzilla bug for the test failure today.)

Cheers,
-- David

 lib/test_kasan.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
Quoted 14 lines

lib/test_kasan.c static void kasan_test_exit(struct kunit *test) · line 102

Quoted 10 lines
lib/test_kasan.c +0 -2
index cacbbbdef768..deda13c9d9ff 100644
@@ -98,9 +98,11 @@ static void kasan_test_exit(struct kunit *test)
        barrier();                                                      \
        expression;                                                     \
        barrier();                                                      \
-       KUNIT_EXPECT_EQ(test,                                           \
-                       READ_ONCE(fail_data.report_expected),           \

What do we have fail_data.report_expected for? Could we remove it now? I think it's unused now.

I thought this was being used in kasan_update_kunit_status() (in mm/kasan/report.c), but it looks like I was mistaken. We should be able to get rid of it, then/

Quoted 6 lines

lib/test_kasan.c static void kasan_test_exit(struct kunit *test) · line 101

Quoted 2 lines
lib/test_kasan.c +1 -1
-                       READ_ONCE(fail_data.report_found));             \
+       if (READ_ONCE(fail_data.report_found) == false) {               \

if (!READ_ONCE(fail_data.report_found)) {
?

I'll change this for v2.

Quoted 7 lines

lib/test_kasan.c static void kasan_test_exit(struct kunit *test) · line 105

Quoted 4 lines
lib/test_kasan.c +4 -0
+               KUNIT_FAIL(test, KUNIT_SUBTEST_INDENT "KASAN failure "  \
+                               "expected in \"" #expression            \
+                                "\", but none occurred");              \
+       }                                                               \

Thanks,
-- Marco