kasan: test: Improve failure message in KUNIT_EXPECT_KASAN_FAIL()
Subsystems (4)
PATCH v3 1/1 kasan: test: Improve failure message in KUNIT_EXPECT_KASAN_FAIL()
The KUNIT_EXPECT_KASAN_FAIL() macro currently uses KUNIT_EXPECT_EQ() to compare fail_data.report_expected and fail_data.report_found. This always gave a somewhat useless error message on failure, but the addition of extra compile-time checking with READ_ONCE() has caused it to get much longer, and be truncated before anything useful is displayed.
Instead, just check fail_data.report_found by hand (we've just set report_expected to 'true'), and print a better failure message with KUNIT_FAIL(). Because of this, report_expected is no longer used anywhere, and can be removed.
Beforehand, a failure in:KUNIT_EXPECT_KASAN_FAIL(test, ((volatile char *)area)[3100]); would have looked like:
[22:00:34] [FAILED] vmalloc_oob
[22:00:34] # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:991
[22:00:34] Expected ({ do { extern void __compiletime_assert_705(void) __attribute__((__error__("Unsupported access size for {READ,WRITE}_ONCE()."))); if (!((sizeof(fail_data.report_expected) == sizeof(char) || sizeof(fail_data.repp
[22:00:34] not ok 45 - vmalloc_oobWith this change, it instead looks like:
[22:04:04] [FAILED] vmalloc_oob
[22:04:04] # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:993 [22:04:04] KASAN failure expected in "((volatile char *)area)[3100]", but none occurred [22:04:04] not ok 45 - vmalloc_oob
Also update the example failure in the documentation to reflect this.
---
Changes since v2:
https://lkml.org/lkml/2021/6/4/1264
- Update the example error in the documentation
Changes since v1:
https://groups.google.com/g/kasan-dev/c/CbabdwoXGlE
- Remove fail_data.report_expected now that it's unused.
- Use '!' instead of '== false' in the comparison.
- Minor typo fixes in the commit message.
The test failure being used as an example is tracked in:
https://bugzilla.kernel.org/show_bug.cgi?id=213335
Documentation/dev-tools/kasan.rst | 9 ++++----- include/linux/kasan.h | 1 - lib/test_kasan.c | 11 +++++------ 3 files changed, 9 insertions(+), 12 deletions(-)
Documentation/dev-tools/kasan.rst +4 -5
@@ -447,11 +447,10 @@ When a test fails due to a failed ``kmalloc``:: When a test fails due to a missing KASAN report:: - # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629 - Expected kasan_data->report_expected == kasan_data->report_found, but - kasan_data->report_expected == 1 - kasan_data->report_found == 0 - not ok 28 - kmalloc_double_kzfree + # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:974 + KASAN failure expected in "kfree_sensitive(ptr)", but none occurred + not ok 44 - kmalloc_double_kzfree + At the end the cumulative status of all KASAN tests is printed. On success::
include/linux/kasan.h +0 -1
@@ -17,7 +17,6 @@ struct task_struct;/* kasan_data struct is used in KUnit tests for KASAN expected failures */ struct kunit_kasan_expectation { -bool report_expected;bool report_found; };
lib/test_kasan.c +5 -6
@@ -55,7 +55,6 @@ static int kasan_test_init(struct kunit *test) multishot =kasan_save_enable_multi_shot ();kasan_set_tagging_report_once (false); fail_data.report_found = false; - fail_data.report_expected = false;kunit_add_named_resource (test,NULL ,NULL , &resource,"kasan_data" , &fail_data);return 0 ;
@@ -94,20 +93,20 @@ static void kasan_test_exit(struct kunit *test) !kasan_async_mode_enabled()) \migrate_disable (); \KUNIT_EXPECT_FALSE (test,READ_ONCE (fail_data.report_found)); \ -WRITE_ONCE (fail_data.report_expected, true); \barrier (); \ expression; \barrier (); \ -KUNIT_EXPECT_EQ (test, \ -READ_ONCE (fail_data.report_expected), \ -READ_ONCE (fail_data.report_found)); \ +if (!READ_ONCE (fail_data.report_found)) { \ +KUNIT_FAIL (test,KUNIT_SUBTEST_INDENT "KASAN failure " \ + "expected in \"" #expression \ +"\", but none occurred" ); \ + } \if (IS_ENABLED (CONFIG_KASAN_HW_TAGS )) { \if (READ_ONCE (fail_data.report_found)) \kasan_enable_tagging_sync (); \migrate_enable (); \ } \WRITE_ONCE (fail_data.report_found, false); \ -WRITE_ONCE (fail_data.report_expected, false); \ } while (0)#define KASAN_TEST_NEEDS_CONFIG_ON (test, config) do { \
Signature · 1 line
2.32.0.rc1.229.g3e70b5a671-goog
Quoted 122 lines · David Gow' via kasan-dev
The
KUNIT_EXPECT_KASAN_FAIL()macro currently usesKUNIT_EXPECT_EQ()to compare fail_data.report_expected and fail_data.report_found. This always gave a somewhat useless error message on failure, but the addition of extra compile-time checking withREAD_ONCE()has caused it to get much longer, and be truncated before anything useful is displayed.Instead, just check fail_data.report_found by hand (we've just set
report_expectedto 'true'), and print a better failure message withKUNIT_FAIL(). Because of this,report_expectedis no longer used anywhere, and can be removed.Beforehand, a failure in:
KUNIT_EXPECT_KASAN_FAIL(test, ((volatile char *)area)[3100]); would have looked like:
[22:00:34] [FAILED]vmalloc_oob[22:00:34] # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:991 [22:00:34] Expected ({ do { extern void __compiletime_assert_705(void) __attribute__((__error__("Unsupported access size for {READ,WRITE}_ONCE()."))); if (!((sizeof(fail_data.report_expected) == sizeof(char) || sizeof(fail_data.repp [22:00:34] not ok 45 - vmalloc_oobWith this change, it instead looks like:
[22:04:04] [FAILED]vmalloc_oob[22:04:04] # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:993 [22:04:04] KASAN failure expected in "((volatile char *)area)[3100]", but none occurred [22:04:04] not ok 45 - vmalloc_oobAlso update the example failure in the documentation to reflect this.
Signed-off-by David Gow <[email protected]>---
Changes since v2:
https://lkml.org/lkml/2021/6/4/1264
- Update the example error in the documentationChanges since v1:
https://groups.google.com/g/kasan-dev/c/CbabdwoXGlE
- Remove fail_data.report_expected now that it's unused.
- Use '!' instead of '== false' in the comparison.
- Minor typo fixes in the commit message.The test failure being used as an example is tracked in:
https://bugzilla.kernel.org/show_bug.cgi?id=213335Documentation/dev-tools/kasan.rst | 9 ++++----- include/linux/kasan.h | 1 - lib/test_kasan.c | 11 +++++------ 3 files changed, 9 insertions(+), 12 deletions(-)Documentation/dev-tools/kasan.rst +4
-5@@ -447,11 +447,10 @@ When a test fails due to a failed ``kmalloc``:: When a test fails due to a missing KASAN report:: - # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629 - Expected kasan_data->report_expected == kasan_data->report_found, but - kasan_data->report_expected == 1 - kasan_data->report_found == 0 - not ok 28 - kmalloc_double_kzfree + # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:974 + KASAN failure expected in "kfree_sensitive(ptr)", but none occurred + not ok 44 - kmalloc_double_kzfree + At the end the cumulative status of all KASAN tests is printed. On success::include/linux/kasan.h +0
-1@@ -17,7 +17,6 @@ struct task_struct;/* kasan_data struct is used in KUnit tests for KASAN expected failures */ struct kunit_kasan_expectation { -bool report_expected;bool report_found; };lib/test_kasan.c +5
-7@@ -55,7 +55,6 @@ static int kasan_test_init(struct kunit *test) multishot =kasan_save_enable_multi_shot ();kasan_set_tagging_report_once (false); fail_data.report_found = false; - fail_data.report_expected = false;kunit_add_named_resource (test,NULL ,NULL , &resource,"kasan_data" , &fail_data);return 0 ;@@ -94,20 +93,20 @@ static void kasan_test_exit(struct kunit *test) !kasan_async_mode_enabled()) \migrate_disable (); \KUNIT_EXPECT_FALSE (test,READ_ONCE (fail_data.report_found)); \ -WRITE_ONCE (fail_data.report_expected, true); \barrier (); \ expression; \barrier (); \ -KUNIT_EXPECT_EQ (test, \ -READ_ONCE (fail_data.report_expected), \ -READ_ONCE (fail_data.report_found)); \ +if (!READ_ONCE (fail_data.report_found)) { \ +KUNIT_FAIL (test,KUNIT_SUBTEST_INDENT "KASAN failure " \ + "expected in \"" #expression \ +"\", but none occurred" ); \ + } \if (IS_ENABLED (CONFIG_KASAN_HW_TAGS )) { \if (READ_ONCE (fail_data.report_found)) \kasan_enable_tagging_sync (); \migrate_enable (); \ } \WRITE_ONCE (fail_data.report_found, false); \ -WRITE_ONCE (fail_data.report_expected, false); \ } while (0)#define KASAN_TEST_NEEDS_CONFIG_ON (test, config) do { \ --
Quoted 1 line
2.32.0.rc1.229.g3e70b5a671-goog
Thanks!
Marco Elver
Reviewed-byQuoted 123 lines · Andrey Konovalov
Quoted 120 lines · David Gow' via kasan-dev
The
KUNIT_EXPECT_KASAN_FAIL()macro currently usesKUNIT_EXPECT_EQ()to compare fail_data.report_expected and fail_data.report_found. This always gave a somewhat useless error message on failure, but the addition of extra compile-time checking withREAD_ONCE()has caused it to get much longer, and be truncated before anything useful is displayed.Instead, just check fail_data.report_found by hand (we've just set
report_expectedto 'true'), and print a better failure message withKUNIT_FAIL(). Because of this,report_expectedis no longer used anywhere, and can be removed.Beforehand, a failure in:
KUNIT_EXPECT_KASAN_FAIL(test, ((volatile char *)area)[3100]); would have looked like:
[22:00:34] [FAILED]vmalloc_oob[22:00:34] # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:991 [22:00:34] Expected ({ do { extern void __compiletime_assert_705(void) __attribute__((__error__("Unsupported access size for {READ,WRITE}_ONCE()."))); if (!((sizeof(fail_data.report_expected) == sizeof(char) || sizeof(fail_data.repp [22:00:34] not ok 45 - vmalloc_oobWith this change, it instead looks like:
[22:04:04] [FAILED]vmalloc_oob[22:04:04] # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:993 [22:04:04] KASAN failure expected in "((volatile char *)area)[3100]", but none occurred [22:04:04] not ok 45 - vmalloc_oobAlso update the example failure in the documentation to reflect this.
Signed-off-by David Gow <[email protected]>---
Changes since v2:
https://lkml.org/lkml/2021/6/4/1264
- Update the example error in the documentationChanges since v1:
https://groups.google.com/g/kasan-dev/c/CbabdwoXGlE
- Remove fail_data.report_expected now that it's unused.
- Use '!' instead of '== false' in the comparison.
- Minor typo fixes in the commit message.The test failure being used as an example is tracked in:
https://bugzilla.kernel.org/show_bug.cgi?id=213335Documentation/dev-tools/kasan.rst | 9 ++++----- include/linux/kasan.h | 1 - lib/test_kasan.c | 11 +++++------ 3 files changed, 9 insertions(+), 12 deletions(-)Documentation/dev-tools/kasan.rst +4
-5@@ -447,11 +447,10 @@ When a test fails due to a failed ``kmalloc``:: When a test fails due to a missing KASAN report:: - # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629 - Expected kasan_data->report_expected == kasan_data->report_found, but - kasan_data->report_expected == 1 - kasan_data->report_found == 0 - not ok 28 - kmalloc_double_kzfree + # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:974 + KASAN failure expected in "kfree_sensitive(ptr)", but none occurred + not ok 44 - kmalloc_double_kzfree + At the end the cumulative status of all KASAN tests is printed. On success::include/linux/kasan.h +0
-1@@ -17,7 +17,6 @@ struct task_struct;/* kasan_data struct is used in KUnit tests for KASAN expected failures */ struct kunit_kasan_expectation { -bool report_expected;bool report_found; };lib/test_kasan.c +5
-6@@ -55,7 +55,6 @@ static int kasan_test_init(struct kunit *test) multishot =kasan_save_enable_multi_shot ();kasan_set_tagging_report_once (false); fail_data.report_found = false; - fail_data.report_expected = false;kunit_add_named_resource (test,NULL ,NULL , &resource,"kasan_data" , &fail_data);return 0 ;@@ -94,20 +93,20 @@ static void kasan_test_exit(struct kunit *test) !kasan_async_mode_enabled()) \migrate_disable (); \KUNIT_EXPECT_FALSE (test,READ_ONCE (fail_data.report_found)); \ -WRITE_ONCE (fail_data.report_expected, true); \barrier (); \ expression; \barrier (); \ -KUNIT_EXPECT_EQ (test, \ -READ_ONCE (fail_data.report_expected), \ -READ_ONCE (fail_data.report_found)); \ +if (!READ_ONCE (fail_data.report_found)) { \ +KUNIT_FAIL (test,KUNIT_SUBTEST_INDENT "KASAN failure " \ + "expected in \"" #expression \ +"\", but none occurred" ); \ + } \if (IS_ENABLED (CONFIG_KASAN_HW_TAGS )) { \if (READ_ONCE (fail_data.report_found)) \kasan_enable_tagging_sync (); \migrate_enable (); \ } \WRITE_ONCE (fail_data.report_found, false); \ -WRITE_ONCE (fail_data.report_expected, false); \ }while (0 )
Quoted 5 lines
Quoted 3 lines
lib/test_kasan.c +0
-1#define KASAN_TEST_NEEDS_CONFIG_ON (test, config) do { \ --2.32.0.rc1.229.g3e70b5a671-goog
Reviewed-by Andrey Konovalov <[email protected]>
Quoted 22 lines · David Gow
The
KUNIT_EXPECT_KASAN_FAIL()macro currently usesKUNIT_EXPECT_EQ()to compare fail_data.report_expected and fail_data.report_found. This always gave a somewhat useless error message on failure, but the addition of extra compile-time checking withREAD_ONCE()has caused it to get much longer, and be truncated before anything useful is displayed.Instead, just check fail_data.report_found by hand (we've just set
report_expectedto 'true'), and print a better failure message withKUNIT_FAIL(). Because of this,report_expectedis no longer used anywhere, and can be removed.Beforehand, a failure in:
KUNIT_EXPECT_KASAN_FAIL(test, ((volatile char *)area)[3100]); would have looked like:
[22:00:34] [FAILED]vmalloc_oob[22:00:34] # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:991 [22:00:34] Expected ({ do { extern void __compiletime_assert_705(void) __attribute__((__error__("Unsupported access size for {READ,WRITE}_ONCE()."))); if (!((sizeof(fail_data.report_expected) == sizeof(char) || sizeof(fail_data.repp [22:00:34] not ok 45 - vmalloc_oobWith this change, it instead looks like:
[22:04:04] [FAILED]vmalloc_oob
Quoted 7 lines
[22:04:04] # vmalloc_oob: EXPECTATION FAILED at lib/test_kasan.c:993 [22:04:04] KASAN failure expected in "((volatile char *)area)[3100]", but none occurred [22:04:04] not ok 45 - vmalloc_oobAlso update the example failure in the documentation to reflect this.
Signed-off-by David Gow <[email protected]>
Nice work!