KUnit-KASAN Integration

From: Patricia Alfonso <[email protected]>

Include documentation on how to test KASAN using CONFIG_TEST_KASAN_KUNIT and CONFIG_TEST_KASAN_MODULE.

Signed-off-by Patricia Alfonso <[email protected]>

Reviewed-by Dmitry Vyukov <[email protected]>

Signed-off-by David Gow <[email protected]>

---
 Documentation/dev-tools/kasan.rst | 70 +++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)

Documentation/dev-tools/kasan.rst +70 -1

index c652d740735d..b4b109d88f9e 100644
@@ -281,3 +281,73 @@ unmapped. This will require changes in arch-specific code.
 
 This allows ``VMAP_STACK`` support on x86, and can simplify support of
 architectures that do not have a fixed module region.
+
+CONFIG_TEST_KASAN_KUNIT & CONFIG_TEST_KASAN_MODULE
+--------------------------------------------------
+
+``CONFIG_TEST_KASAN_KUNIT`` utilizes the KUnit Test Framework for testing.
+This means each test focuses on a small unit of functionality and
+there are a few ways these tests can be run.
+
+Each test will print the KASAN report if an error is detected and then
+print the number of the test and the status of the test:
+
+pass::
+
+        ok 28 - kmalloc_double_kzfree
+or, if kmalloc failed::
+
+        # kmalloc_large_oob_right: ASSERTION FAILED at lib/test_kasan.c:163
+        Expected ptr is not null, but is
+        not ok 4 - kmalloc_large_oob_right
+or, if a KASAN report was expected, but not found::
+
+        # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629
+        Expected kasan_data->report_expected == kasan_data->report_found, but
+        kasan_data->report_expected == 1
+        kasan_data->report_found == 0
+        not ok 28 - kmalloc_double_kzfree
+
+All test statuses are tracked as they run and an overall status will
+be printed at the end::
+
+        ok 1 - kasan_kunit_test
+
+or::
+
+        not ok 1 - kasan_kunit_test
+
+(1) Loadable Module
+~~~~~~~~~~~~~~~~~~~~
+
+With ``CONFIG_KUNIT`` enabled, ``CONFIG_TEST_KASAN_KUNIT`` can be built as
+a loadable module and run on any architecture that supports KASAN
+using something like insmod or modprobe.
+
+(2) Built-In
+~~~~~~~~~~~~~
+
+With ``CONFIG_KUNIT`` built-in, ``CONFIG_TEST_KASAN_KUNIT`` can be built-in
+on any architecure that supports KASAN. These and any other KUnit
+tests enabled will run and print the results at boot as a late-init
+call.
+
+(3) Using kunit_tool
+~~~~~~~~~~~~~~~~~~~~~
+
+With ``CONFIG_KUNIT`` and ``CONFIG_TEST_KASAN_KUNIT`` built-in, we can also
+use kunit_tool to see the results of these along with other KUnit
+tests in a more readable way. This will not print the KASAN reports
+of tests that passed. Use `KUnit documentation <https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html>`_ for more up-to-date
+information on kunit_tool.
+
+.. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html
+
+``CONFIG_TEST_KASAN_MODULE`` is a set of KASAN tests that could not be
+converted to KUnit. These tests can be run only as a module with
+``CONFIG_TEST_KASAN_MODULE`` built as a loadable module and
+``CONFIG_KASAN`` built-in. The type of error expected and the
+function being run is printed before the expression expected to give
+an error. Then the error is printed, if found, and that test
+should be interpretted to pass only if the error was the one expected
+by the test.
--

2.26.2.303.gf8c07b1a785-goog

--
You received this message because you are subscribed to the Google Groups "kasan-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20200424061342.212535-5-davidgow%40google.com.

This patchset contains everything needed to integrate KASAN and KUnit.

KUnit will be able to:
(1) Fail tests when an unexpected KASAN error occurs
(2) Pass tests when an expected KASAN error occurs

Convert KASAN tests to KUnit with the exception of copy_user_test because KUnit is unable to test those.

Add documentation on how to run the KASAN tests with KUnit and what to expect when running these tests.

This patchset depends on:
- "[PATCH v3 kunit-next 0/2] kunit: extend kunit resources API" [1]
- "[PATCH v3 0/3] Fix some incompatibilites between KASAN and FORTIFY_SOURCE" [2]

Changes from v6:
- Rebased on top of kselftest/kunit
- Rebased on top of Daniel Axtens' fix for FORTIFY_SOURCE incompatibilites [2]
- Removed a redundant report_enabled() check.
- Fixed some places with out of date Kconfig names in the documentation.

Changes from v5:
- Split out the panic_on_warn changes to a separate patch.
- Fix documentation to fewer to the new Kconfig names.
- Fix some changes which were in the wrong patch.
- Rebase on top of kselftest/kunit (currently identical to 5.7-rc1)

Changes from v4:
- KASAN no longer will panic on errors if both panic_on_warn and kasan_multishot are enabled.
- As a result, the KASAN tests will no-longer disable panic_on_warn.
- This also means panic_on_warn no-longer needs to be exported.
- The use of temporary "kasan_data" variables has been cleaned up somewhat.
- A potential refcount/resource leak should multiple KASAN errors appear during an assertion was fixed.
- Some wording changes to the KASAN test Kconfig entries.

Changes from v3:
- KUNIT_SET_KASAN_DATA and KUNIT_DO_EXPECT_KASAN_FAIL have been combined and included in KUNIT_DO_EXPECT_KASAN_FAIL() instead.
- Reordered logic in kasan_update_kunit_status() in report.c to be easier to read.
- Added comment to not use the name "kasan_data" for any kunit tests outside of KUNIT_EXPECT_KASAN_FAIL().

Changes since v2:
- Due to Alan's changes in [1], KUnit can be built as a module.
- The name of the tests that could not be run with KUnit has been changed to be more generic: test_kasan_module.
- Documentation on how to run the new KASAN tests and what to expect when running them has been added.
- Some variables and functions are now static.
- Now save/restore panic_on_warn in a similar way to kasan_multi_shot and renamed the init/exit functions to be more generic to accommodate.
- Due to [3] in kasan_strings, kasan_memchr, and kasan_memcmp will fail if CONFIG_AMD_MEM_ENCRYPT is enabled so return early and print message explaining this circumstance.
- Changed preprocessor checks to C checks where applicable.

Changes since v1:
- Make use of Alan Maguire's suggestion to use his patch that allows static resources for integration instead of adding a new attribute to the kunit struct
- All KUNIT_EXPECT_KASAN_FAIL statements are local to each test
- The definition of KUNIT_EXPECT_KASAN_FAIL is local to the test_kasan.c file since it seems this is the only place this will be used.
- Integration relies on KUnit being builtin
- copy_user_test has been separated into its own file since KUnit is unable to test these. This can be run as a module just as before, using CONFIG_TEST_KASAN_USER
- The addition to the current task has been separated into its own patch as this is a significant enough change to be on its own.

[1] https://lore.kernel.org/linux-kselftest/[email protected]/T/#t
[2] https://lkml.org/lkml/2020/4/23/708
[3] https://bugzilla.kernel.org/show_bug.cgi?id=206337

David Gow (1):
mm: kasan: Do not panic if both panic_on_warn and kasan_multishot set

Patricia Alfonso (4):
Add KUnit Struct to Current Task
KUnit: KASAN Integration

  KASAN: Port KASAN Tests to KUnit
  KASAN: Testing Documentation

 Documentation/dev-tools/kasan.rst |  70 +++
 include/kunit/test.h              |   5 +
 include/linux/kasan.h             |   6 +
 include/linux/sched.h             |   4 +
 lib/Kconfig.kasan                 |  18 +-
 lib/Makefile                      |   3 +-
 lib/kunit/test.c                  |  13 +-
 lib/test_kasan.c                  | 688 +++++++++++++-----------------
 lib/test_kasan_module.c           |  76 ++++
 mm/kasan/report.c                 |  34 +-
 10 files changed, 514 insertions(+), 403 deletions(-)
 create mode 100644 lib/test_kasan_module.c

Signature · 1 line

2.26.2.303.gf8c07b1a785-goog

Quoted 17 lines · David Gow

This patchset contains everything needed to integrate KASAN and KUnit.

KUnit will be able to:
(1) Fail tests when an unexpected KASAN error occurs
(2) Pass tests when an expected KASAN error occurs

Convert KASAN tests to KUnit with the exception of copy_user_test because KUnit is unable to test those.

Add documentation on how to run the KASAN tests with KUnit and what to expect when running these tests.

This patchset depends on:
- "[PATCH v3 kunit-next 0/2] kunit: extend kunit resources API" [1]
- "[PATCH v3 0/3] Fix some incompatibilites between KASAN and FORTIFY_SOURCE" [2]

Quoted 8 lines

Changes from v6:
- Rebased on top of kselftest/kunit
- Rebased on top of Daniel Axtens' fix for FORTIFY_SOURCE incompatibilites [2]
- Removed a redundant report_enabled() check.
- Fixed some places with out of date Kconfig names in the documentation.

Sorry for the delay in getting to this; I retested the series with the above patchsets pre-applied; all looks good now, thanks! Looks like Daniel's patchset has a v4 so I'm not sure if that will have implications for applying your changes on top of it (haven't tested it yet myself).

For the series feel free to add

Tested-by Alan Maguire <[email protected]>

I'll try and take some time to review v7 shortly, but I wanted to confirm the issues I saw went away first in case you're blocked. The only remaining issue I see is that we'd need the named resource patchset to land first; it would be good to ensure the API it provides is solid so you won't need to respin.

Thanks!

Alan

Quoted 6 lines

Changes from v5:
- Split out the panic_on_warn changes to a separate patch.
- Fix documentation to fewer to the new Kconfig names.
- Fix some changes which were in the wrong patch.
- Rebase on top of kselftest/kunit (currently identical to 5.7-rc1)

Quoted 45 lines · Alan Maguire

Quoted 17 lines · David Gow

This patchset contains everything needed to integrate KASAN and KUnit.

KUnit will be able to:
(1) Fail tests when an unexpected KASAN error occurs
(2) Pass tests when an expected KASAN error occurs

Convert KASAN tests to KUnit with the exception of copy_user_test because KUnit is unable to test those.

Add documentation on how to run the KASAN tests with KUnit and what to expect when running these tests.

This patchset depends on:
- "[PATCH v3 kunit-next 0/2] kunit: extend kunit resources API" [1]
- "[PATCH v3 0/3] Fix some incompatibilites between KASAN and FORTIFY_SOURCE" [2]

Quoted 8 lines

Changes from v6:
- Rebased on top of kselftest/kunit
- Rebased on top of Daniel Axtens' fix for FORTIFY_SOURCE incompatibilites [2]
- Removed a redundant report_enabled() check.
- Fixed some places with out of date Kconfig names in the documentation.

Sorry for the delay in getting to this; I retested the series with the above patchsets pre-applied; all looks good now, thanks! Looks like Daniel's patchset has a v4 so I'm not sure if that will have implications for applying your changes on top of it (haven't tested it yet myself).

For the series feel free to add

Tested-by Alan Maguire <[email protected]>

I'll try and take some time to review v7 shortly, but I wanted to confirm the issues I saw went away first in case you're blocked. The only remaining issue I see is that we'd need the named resource patchset to land first; it would be good to ensure the API it provides is solid so you won't need to respin.

Quoted 11 lines

Thanks!

Alan

Quoted 6 lines

Changes from v5:
- Split out the panic_on_warn changes to a separate patch.
- Fix documentation to fewer to the new Kconfig names.
- Fix some changes which were in the wrong patch.
- Rebase on top of kselftest/kunit (currently identical to 5.7-rc1)

Hi Brendan,

Is this series ready to go inot Linux 5.8-rc1? Let me know. Probably needs rebase on top of kselftest/kunit. I applied patches from David and Vitor

thanks,
-- Shuah

Quoted 59 lines · shuah

Quoted 45 lines · Alan Maguire

Quoted 17 lines · David Gow

This patchset contains everything needed to integrate KASAN and KUnit.

KUnit will be able to:
(1) Fail tests when an unexpected KASAN error occurs
(2) Pass tests when an expected KASAN error occurs

Convert KASAN tests to KUnit with the exception of copy_user_test because KUnit is unable to test those.

Add documentation on how to run the KASAN tests with KUnit and what to expect when running these tests.

This patchset depends on:
- "[PATCH v3 kunit-next 0/2] kunit: extend kunit resources API" [1]
- "[PATCH v3 0/3] Fix some incompatibilites between KASAN and FORTIFY_SOURCE" [2]

Quoted 8 lines

Changes from v6:
- Rebased on top of kselftest/kunit
- Rebased on top of Daniel Axtens' fix for FORTIFY_SOURCE incompatibilites [2]
- Removed a redundant report_enabled() check.
- Fixed some places with out of date Kconfig names in the documentation.

Sorry for the delay in getting to this; I retested the series with the above patchsets pre-applied; all looks good now, thanks! Looks like Daniel's patchset has a v4 so I'm not sure if that will have implications for applying your changes on top of it (haven't tested it yet myself).

For the series feel free to add

Tested-by Alan Maguire <[email protected]>

I'll try and take some time to review v7 shortly, but I wanted to confirm the issues I saw went away first in case you're blocked. The only remaining issue I see is that we'd need the named resource patchset to land first; it would be good to ensure the API it provides is solid so you won't need to respin.

Quoted 11 lines

Thanks!

Alan

Quoted 6 lines

Changes from v5:
- Split out the panic_on_warn changes to a separate patch.
- Fix documentation to fewer to the new Kconfig names.
- Fix some changes which were in the wrong patch.
- Rebase on top of kselftest/kunit (currently identical to 5.7-rc1)

Quoted 9 lines

Hi Brendan,

Is this series ready to go inot Linux 5.8-rc1? Let me know. Probably needs rebase on top of kselftest/kunit. I applied patches from David and Vitor

thanks,
-- Shuah

Hi Shuah,

I think the only things holding this up are the missing dependencies:
the "extend kunit resources API" patches[1] for KUnit (which look ready to me), and the "Fix some incompatibilities between KASAN and FORTIFY_SOURCE" changes[2] on the KASAN side (which also seem ready).

This patchset may need a (likely rather trivial) rebase on top of whatever versions of those end up merged: I'm happy to do that if necessary.

Cheers,
-- David

[1]: https://lore.kernel.org/linux-kselftest/[email protected]/T/#t
[2]: http://lkml.iu.edu/hypermail/linux/kernel/2004.3/00735.html

Quoted 85 lines · David Gow

Quoted 59 lines · shuah

Quoted 45 lines · Alan Maguire

Quoted 17 lines · David Gow

This patchset contains everything needed to integrate KASAN and KUnit.

KUnit will be able to:
(1) Fail tests when an unexpected KASAN error occurs
(2) Pass tests when an expected KASAN error occurs

Convert KASAN tests to KUnit with the exception of copy_user_test because KUnit is unable to test those.

Add documentation on how to run the KASAN tests with KUnit and what to expect when running these tests.

This patchset depends on:
- "[PATCH v3 kunit-next 0/2] kunit: extend kunit resources API" [1]
- "[PATCH v3 0/3] Fix some incompatibilites between KASAN and

    FORTIFY_SOURCE" [2]

Quoted 8 lines

Changes from v6:
- Rebased on top of kselftest/kunit
- Rebased on top of Daniel Axtens' fix for FORTIFY_SOURCE incompatibilites [2]
- Removed a redundant report_enabled() check.
- Fixed some places with out of date Kconfig names in the documentation.

Sorry for the delay in getting to this; I retested the series with the above patchsets pre-applied; all looks good now, thanks! Looks like Daniel's patchset has a v4 so I'm not sure if that will have implications for applying your changes on top of it (haven't tested it yet myself).

For the series feel free to add

Tested-by Alan Maguire <[email protected]>

I'll try and take some time to review v7 shortly, but I wanted to confirm the issues I saw went away first in case you're blocked. The only remaining issue I see is that we'd need the named resource patchset to land first; it would be good to ensure the API it provides is solid so you won't need to respin.

Quoted 11 lines

Thanks!

Alan

Quoted 6 lines

Changes from v5:
- Split out the panic_on_warn changes to a separate patch.
- Fix documentation to fewer to the new Kconfig names.
- Fix some changes which were in the wrong patch.
- Rebase on top of kselftest/kunit (currently identical to 5.7-rc1)

Quoted 9 lines

Hi Brendan,

Is this series ready to go inot Linux 5.8-rc1? Let me know. Probably needs rebase on top of kselftest/kunit. I applied patches from David and Vitor

thanks,
-- Shuah

Hi Shuah,

I think the only things holding this up are the missing dependencies:
the "extend kunit resources API" patches[1] for KUnit (which look ready to me), and the "Fix some incompatibilities between KASAN and FORTIFY_SOURCE" changes[2] on the KASAN side (which also seem ready).

This patchset may need a (likely rather trivial) rebase on top of whatever versions of those end up merged: I'm happy to do that if necessary.

Cheers,
-- David

Quoted 2 lines

[1]: https://lore.kernel.org/linux-kselftest/[email protected]/T/#t
[2]: http://lkml.iu.edu/hypermail/linux/kernel/2004.3/00735.html

As David pointed out, this series is waiting on its dependencies. Sorry, I thought the "extend KUnit resources API" patchset was ready to go, but I realized I only gave a reviewed-by to one of the patches. Both have been reviewed now, but one patch needs a minor fix.

As for other patches, the patches from David, Vitor, and Anders should cover everything. Thanks!