kunit: create a centralized executor to dispatch all KUnit tests
Subsystems (10)
v5 cover kunit: create a centralized executor to dispatch all KUnit tests
## TL;DR
This patchset adds a centralized executor to dispatch tests rather than relying on late_initcall to schedule each test suite separately along with a couple of new features that depend on it.
## What am I trying to do?
Conceptually, I am trying to provide a mechanism by which test suites can be grouped together so that they can be reasoned about collectively. The last two of three patches in this series add features which depend on this:
PATCH 09/12 Prints out a test plan[1] right before KUnit tests are run;
this is valuable because it makes it possible for a test harness to detect whether the number of tests run matches the number of tests expected to be run, ensuring that no tests silently failed. The test plan includes a count of tests that will run. With the centralized executor, the tests are located in a single data structure and thus can be counted.
PATCH 10/12 Add a new kernel command-line option which allows the user
to specify that the kernel poweroff, halt, or reboot after completing all KUnit tests; this is very handy for running KUnit tests on UML or a VM so that the UML/VM process exits cleanly immediately after running all tests without needing a special initramfs. The centralized executor provides a definitive point when all tests have completed and the poweroff, halt, or reboot could occur.
In addition, by dispatching tests from a single location, we can guarantee that all KUnit tests run after late_init is complete, which was a concern during the initial KUnit patchset review (this has not been a problem in practice, but resolving with certainty is nevertheless desirable).
Other use cases for this exist, but the above features should provide an idea of the value that this could provide.
## Changes since last revision:
- Fixed a compilation error in the centralized executor patch (07/12). I had forgotten to test the patches when building as modules. I verified that works now.
- I accidentally merged patches 09/12 and 10/12 in the previous revision (v4), and made them separate patches again.
## Changes since v3:
- On the last revision I got some messages from 0day that showed that this patchset didn't work on several architectures, one issue that this patchset addresses is that we were aligning both memory segments as well as structures in the segments to specific byte boundaries which was incorrect.
- The issue mentioned above also caused me to test on additional architectures which revealed that some architectures other than UML do not use the default init linker section macro that most architectures use. There are now several new patches (2, 3, 4, and 6).
- Fixed a formatting consistency issue in the kernel params documentation patch (11/12).
- Add a brief blurb on how and when the kunit_test_suite macro works.
## Remaining work to be done:
The only architecture for which I was able to get a compiler, but was apparently unable to get KUnit into a section that the executor to see was m68k - not sure why.
Alan Maguire (1):
kunit: test: create a single centralized executor for all tests
Brendan Higgins (10):
vmlinux.lds.h: add linker section for KUnit test suites
arch: arm64: add linker section for KUnit test suites
arch: microblaze: add linker section for KUnit test suites
arch: powerpc: add linker section for KUnit test suites
arch: um: add linker section for KUnit test suites
arch: xtensa: add linker section for KUnit test suites
init: main: add KUnit to kernel init
kunit: test: add test plan to KUnit TAP format
Documentation: Add kunit_shutdown to kernel-parameters.txt
Documentation: kunit: add a brief blurb about kunit_test_suite
David Gow (1):
kunit: Add 'kunit_shutdown' option
.../admin-guide/kernel-parameters.txt | 8 ++ Documentation/dev-tools/kunit/usage.rst | 5 ++ arch/arm64/kernel/vmlinux.lds.S | 3 + arch/microblaze/kernel/vmlinux.lds.S | 4 + arch/powerpc/kernel/vmlinux.lds.S | 4 + arch/um/include/asm/common.lds.S | 4 + arch/xtensa/kernel/vmlinux.lds.S | 4 + include/asm-generic/vmlinux.lds.h | 8 ++ include/kunit/test.h | 76 +++++++++++++----- init/main.c | 4 + lib/kunit/Makefile | 3 +- lib/kunit/executor.c | 63 +++++++++++++++ lib/kunit/test.c | 13 +-- tools/testing/kunit/kunit_kernel.py | 2 +- tools/testing/kunit/kunit_parser.py | 74 ++++++++++++++--- .../test_is_test_passed-all_passed.log | Bin 1562 -> 1567 bytes .../test_data/test_is_test_passed-crash.log | Bin 3016 -> 3021 bytes .../test_data/test_is_test_passed-failure.log | Bin 1700 -> 1705 bytes 18 files changed, 227 insertions(+), 48 deletions(-) create mode 100644 lib/kunit/executor.c
These patches are available for download with dependencies here:
https://kunit-review.googlesource.com/c/linux/+/3829
[1] https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md#the-plan
[2] https://patchwork.kernel.org/patch/11383635/
base-commit: 4333a9b0b67bb4e8bcd91bdd80da80b0ec151162
prerequisite-patch-id: 2d4b5aa9fa8ada9ae04c8584b47c299a822b9455
prerequisite-patch-id: 582b6d9d28ce4b71628890ec832df6522ca68de0
Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
PATCH v5 1/12 vmlinux.lds.h: add linker section for KUnit test suites
Add a linker section where KUnit can put references to its test suites. This patch is the first step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate late_initcall.
--- include/asm-generic/vmlinux.lds.h | 8 ++++++++ 1 file changed, 8 insertions(+)
include/asm-generic/vmlinux.lds.h +8 -0
@@ -881,6 +881,13 @@ KEEP(*(.con_initcall.init)) \ __con_initcall_end = .; +/* Alignment must be consistent with (kunit_suite *) in include/kunit/test.h */ +#define KUNIT_TEST_SUITES \ + . = ALIGN(8); \ + __kunit_suites_start = .; \ + KEEP(*(.kunit_test_suites)) \ + __kunit_suites_end = .; +#ifdef CONFIG_BLK_DEV_INITRD #define INIT_RAM_FS \ . = ALIGN(4); \
@@ -1056,6 +1063,7 @@INIT_CALLS \ CON_INITCALL \INIT_RAM_FS \ + KUNIT_TEST_SUITES \ }#define BSS_SECTION (sbss_align, bss_align, stop_align) \
Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
Kees Cook
NAKRe: [PATCH v5 01/12] vmlinux.lds.h: add linker section for KUnit test suites
Quoted 18 lines · Brendan Higgins
Add a linker section where KUnit can put references to its test suites. This patch is the first step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate
late_initcall.Co-developed-by Iurii Zaikin <[email protected]>Signed-off-by Iurii Zaikin <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- include/asm-generic/vmlinux.lds.h | 8 ++++++++ 1 file changed, 8 insertions(+)include/asm-generic/vmlinux.lds.h +0
-0
include/asm-generic/vmlinux.lds.h line 884
Quoted 4 lines
include/asm-generic/vmlinux.lds.h +1
-0@@ -881,6 +881,13 @@ KEEP(*(.con_initcall.init)) \ __con_initcall_end = .; +/* Alignment must be consistent with (kunit_suite *) in include/kunit/test.h */
Nit on naming:
include/asm-generic/vmlinux.lds.h line 885
Quoted 1 line
include/asm-generic/vmlinux.lds.h +1
-0+#define KUNIT_TEST_SUITES \
I would call this KUNIT_TABLE to maintain the same names as other things of this nature.
Quoted 9 lines
include/asm-generic/vmlinux.lds.h +5
-0+ . = ALIGN(8); \ + __kunit_suites_start = .; \ + KEEP(*(.kunit_test_suites)) \ + __kunit_suites_end = .; +#ifdef CONFIG_BLK_DEV_INITRD #define INIT_RAM_FS \ . = ALIGN(4); \
include/asm-generic/vmlinux.lds.h line 1067
Quoted 5 lines
include/asm-generic/vmlinux.lds.h +1
-0@@ -1056,6 +1063,7 @@INIT_CALLS \ CON_INITCALL \INIT_RAM_FS \ + KUNIT_TEST_SUITES \ }
Nack: this must be in INIT_DATA, not in INIT_DATA_SECTION. Not all architectures use the INIT_DATA_SECTION macro (e.g. arm64), but everything uses INIT_DATA.
Signature · 1 line
Kees Cook
Re: [PATCH v5 01/12] vmlinux.lds.h: add linker section for KUnit test suites
Quoted 42 lines · Kees Cook
Quoted 18 lines · Brendan Higgins
Add a linker section where KUnit can put references to its test suites. This patch is the first step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate
late_initcall.Co-developed-by Iurii Zaikin <[email protected]>Signed-off-by Iurii Zaikin <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- include/asm-generic/vmlinux.lds.h | 8 ++++++++ 1 file changed, 8 insertions(+)include/asm-generic/vmlinux.lds.h +0
-0include/asm-generic/vmlinux.lds.h line 884
Quoted 4 lines
include/asm-generic/vmlinux.lds.h +1
-0@@ -881,6 +881,13 @@ KEEP(*(.con_initcall.init)) \ __con_initcall_end = .; +/* Alignment must be consistent with (kunit_suite *) in include/kunit/test.h */ Nit on naming:
include/asm-generic/vmlinux.lds.h line 885
Quoted 1 line
include/asm-generic/vmlinux.lds.h +1
-0+#define KUNIT_TEST_SUITES \I would call this
KUNIT_TABLEto maintain the same names as other things of this nature.Quoted 10 lines
include/asm-generic/vmlinux.lds.h +5
-0+ . = ALIGN(8); \ + __kunit_suites_start = .; \ + KEEP(*(.kunit_test_suites)) \ + __kunit_suites_end = .; +#ifdef CONFIG_BLK_DEV_INITRD #define INIT_RAM_FS \ . = ALIGN(4); \@@ -1056,6 +1063,7 @@INIT_CALLS \
Quoted 8 lines
include/asm-generic/vmlinux.lds.h line 1067
Quoted 4 lines
include/asm-generic/vmlinux.lds.h +1
-0CON_INITCALL \INIT_RAM_FS \ + KUNIT_TEST_SUITES \ }Nack: this must be in
INIT_DATA, not inINIT_DATA_SECTION. Not all architectures use theINIT_DATA_SECTIONmacro (e.g. arm64), but everything usesINIT_DATA.
Oh, maybe that would eliminate the need for the other linkerscript patches? That would be nice.
Alright, will fix.
Re: [PATCH v5 01/12] vmlinux.lds.h: add linker section for KUnit test suites
Quoted 52 lines · Brendan Higgins
Quoted 42 lines · Kees Cook
Quoted 18 lines · Brendan Higgins
Add a linker section where KUnit can put references to its test suites. This patch is the first step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate
late_initcall.Co-developed-by Iurii Zaikin <[email protected]>Signed-off-by Iurii Zaikin <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- include/asm-generic/vmlinux.lds.h | 8 ++++++++ 1 file changed, 8 insertions(+)include/asm-generic/vmlinux.lds.h +0
-0include/asm-generic/vmlinux.lds.h line 884
Quoted 4 lines
include/asm-generic/vmlinux.lds.h +1
-0@@ -881,6 +881,13 @@ KEEP(*(.con_initcall.init)) \ __con_initcall_end = .; +/* Alignment must be consistent with (kunit_suite *) in include/kunit/test.h */ Nit on naming:
include/asm-generic/vmlinux.lds.h line 885
Quoted 1 line
include/asm-generic/vmlinux.lds.h +1
-0+#define KUNIT_TEST_SUITES \I would call this
KUNIT_TABLEto maintain the same names as other things of this nature.Quoted 10 lines
include/asm-generic/vmlinux.lds.h +5
-0+ . = ALIGN(8); \ + __kunit_suites_start = .; \ + KEEP(*(.kunit_test_suites)) \ + __kunit_suites_end = .; +#ifdef CONFIG_BLK_DEV_INITRD #define INIT_RAM_FS \ . = ALIGN(4); \@@ -1056,6 +1063,7 @@INIT_CALLS \Quoted 8 lines
include/asm-generic/vmlinux.lds.h line 1067
Quoted 4 lines
include/asm-generic/vmlinux.lds.h +1
-0CON_INITCALL \INIT_RAM_FS \ + KUNIT_TEST_SUITES \ }Nack: this must be in
INIT_DATA, not inINIT_DATA_SECTION. Not all architectures use theINIT_DATA_SECTIONmacro (e.g. arm64), but everything usesINIT_DATA.
Quoted 2 lines
Oh, maybe that would eliminate the need for the other linkerscript patches? That would be nice.
Curious, did changing it as Kees suggest fix it for m68k?
Luis
_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
Re: [PATCH v5 01/12] vmlinux.lds.h: add linker section for KUnit test suites
Quoted 54 lines · Luis Chamberlain
Quoted 52 lines · Brendan Higgins
Quoted 50 lines · Kees Cook
Quoted 18 lines · Brendan Higgins
Add a linker section where KUnit can put references to its test suites. This patch is the first step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate
late_initcall.Co-developed-by Iurii Zaikin <[email protected]>Signed-off-by Iurii Zaikin <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- include/asm-generic/vmlinux.lds.h | 8 ++++++++ 1 file changed, 8 insertions(+)include/asm-generic/vmlinux.lds.h +0
-0include/asm-generic/vmlinux.lds.h line 884
Quoted 4 lines
include/asm-generic/vmlinux.lds.h +1
-0@@ -881,6 +881,13 @@ KEEP(*(.con_initcall.init)) \ __con_initcall_end = .; +/* Alignment must be consistent with (kunit_suite *) in include/kunit/test.h */ Nit on naming:
include/asm-generic/vmlinux.lds.h line 885
Quoted 1 line
include/asm-generic/vmlinux.lds.h +1
-0+#define KUNIT_TEST_SUITES \I would call this
KUNIT_TABLEto maintain the same names as other things of this nature.Quoted 9 lines
include/asm-generic/vmlinux.lds.h +5
-0+ . = ALIGN(8); \ + __kunit_suites_start = .; \ + KEEP(*(.kunit_test_suites)) \ + __kunit_suites_end = .; +#ifdef CONFIG_BLK_DEV_INITRD #define INIT_RAM_FS \ . = ALIGN(4); \include/asm-generic/vmlinux.lds.h line 1067
Quoted 5 lines
include/asm-generic/vmlinux.lds.h +1
-0@@ -1056,6 +1063,7 @@INIT_CALLS \ CON_INITCALL \INIT_RAM_FS \ + KUNIT_TEST_SUITES \ }Nack: this must be in
INIT_DATA, not inINIT_DATA_SECTION. Not all architectures use theINIT_DATA_SECTIONmacro (e.g. arm64), but everything usesINIT_DATA.
Quoted 2 lines
Quoted 2 lines
Oh, maybe that would eliminate the need for the other linkerscript patches? That would be nice.
Sorry for the delayed response. I got pulled into some other things.
Quoted 1 line
Curious, did changing it as Kees suggest fix it for m68k?
It did! There are still some architectures I cannot test due to a lack of GCC or QEMU support, but it seems to work on everything else now.
PATCH v5 2/12 arch: arm64: add linker section for KUnit test suites
In reply to Brendan HigginsAdd a linker section to arm64 where KUnit can put references to its test suites. This patch is an early step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate late_initcall.
--- arch/arm64/kernel/vmlinux.lds.S | 3 +++ 1 file changed, 3 insertions(+)
arch/arm64/kernel/vmlinux.lds.S +3 -0
@@ -181,6 +181,9 @@ SECTIONS INIT_RAM_FS *(.init.rodata.* .init.bss) /* from the EFI stub */ } + .kunit_test_suites : { + KUNIT_TEST_SUITES + } .exit.data : { EXIT_DATA }
Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
Re: [PATCH v5 02/12] arch: arm64: add linker section for KUnit test suites
Quoted 15 lines · Brendan Higgins
Add a linker section to arm64 where KUnit can put references to its test suites. This patch is an early step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate
late_initcall.Signed-off-by Brendan Higgins <[email protected]>--- arch/arm64/kernel/vmlinux.lds.S | 3 +++ 1 file changed, 3 insertions(+)arch/arm64/kernel/vmlinux.lds.S +0
-0
arch/arm64/kernel/vmlinux.lds.S SECTIONS · line 186
Quoted 6 lines
arch/arm64/kernel/vmlinux.lds.S +3
-0@@ -181,6 +181,9 @@ SECTIONS INIT_RAM_FS *(.init.rodata.* .init.bss) /* from the EFI stub */ } + .kunit_test_suites : { + KUNIT_TEST_SUITES + }
See my reply to 01/12. Then this patch can be dropped. :)
Quoted 6 lines
arch/arm64/kernel/vmlinux.lds.S +0
-0.exit.data : { EXIT_DATA }Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
Signature · 1 line
Kees Cook
Re: [PATCH v5 02/12] arch: arm64: add linker section for KUnit test suites
Quoted 12 lines · Kees Cook
Quoted 10 lines · Brendan Higgins
Add a linker section to arm64 where KUnit can put references to its test suites. This patch is an early step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate
late_initcall.Signed-off-by Brendan Higgins <[email protected]>--- arch/arm64/kernel/vmlinux.lds.S | 3 +++ 1 file changed, 3 insertions(+)
Quoted 13 lines
arch/arm64/kernel/vmlinux.lds.S SECTIONS · line 186
Quoted 11 lines
arch/arm64/kernel/vmlinux.lds.S +3
-0@@ -181,6 +181,9 @@ SECTIONS INIT_RAM_FS *(.init.rodata.* .init.bss) /* from the EFI stub */ } + .kunit_test_suites : { + KUNIT_TEST_SUITES + }See my reply to 01/12. Then this patch can be dropped. :)
Sweet, presumably this one and others.
Quoted 9 lines
Quoted 6 lines
arch/arm64/kernel/vmlinux.lds.S +0
-1.exit.data : { EXIT_DATA } --2.27.0.212.ge8ba1cc988-goog
--
Kees Cook
PATCH v5 3/12 arch: microblaze: add linker section for KUnit test suites
In reply to Brendan HigginsAdd a linker section to microblaze where KUnit can put references to its test suites. This patch is an early step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate late_initcall.
--- arch/microblaze/kernel/vmlinux.lds.S | 4 ++++ 1 file changed, 4 insertions(+)
arch/microblaze/kernel/vmlinux.lds.S +4 -0
@@ -128,6 +128,10 @@ SECTIONS { __init_end = .; + .kunit_test_suites : { + KUNIT_TEST_SUITES + } + .bss ALIGN (PAGE_SIZE) : AT(ADDR(.bss) - LOAD_OFFSET) { /* page aligned when MMU used */ __bss_start = . ;
Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
PATCH v5 4/12 arch: powerpc: add linker section for KUnit test suites
In reply to Brendan HigginsAdd a linker section to powerpc where KUnit can put references to its test suites. This patch is an early step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate late_initcall.
--- arch/powerpc/kernel/vmlinux.lds.S | 4 ++++ 1 file changed, 4 insertions(+)
arch/powerpc/kernel/vmlinux.lds.S +4 -0
@@ -202,6 +202,10 @@ SECTIONS CON_INITCALL } + .kunit_test_suites : { + KUNIT_TEST_SUITES + } + . = ALIGN(8); __ftr_fixup : AT(ADDR(__ftr_fixup) - LOAD_OFFSET) { __start___ftr_fixup = .;
Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
PATCH v5 5/12 arch: um: add linker section for KUnit test suites
In reply to Brendan HigginsAdd a linker section to UML where KUnit can put references to its test suites. This patch is an early step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate late_initcall.
--- arch/um/include/asm/common.lds.S | 4 ++++ 1 file changed, 4 insertions(+)
arch/um/include/asm/common.lds.S +4 -0
@@ -52,6 +52,10 @@ CON_INITCALL } + .kunit_test_suites : { + KUNIT_TEST_SUITES + } + .exitcall : { __exitcall_begin = .; *(.exitcall.exit)
Signature · 7 lines
2.27.0.212.ge8ba1cc988-goog _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH v5 6/12 arch: xtensa: add linker section for KUnit test suites
In reply to Brendan HigginsAdd a linker section to xtensa where KUnit can put references to its test suites. This patch is an early step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate late_initcall.
--- arch/xtensa/kernel/vmlinux.lds.S | 4 ++++ 1 file changed, 4 insertions(+)
arch/xtensa/kernel/vmlinux.lds.S +4 -0
@@ -216,6 +216,10 @@ SECTIONS INIT_RAM_FS } + .kunit_test_suites : { + KUNIT_TEST_SUITES + } + PERCPU_SECTION(XCHAL_ICACHE_LINESIZE) /* We need this dummy segment here */
Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
Re: [PATCH v5 06/12] arch: xtensa: add linker section for KUnit test suites
Quoted 8 lines · Brendan Higgins
Add a linker section to xtensa where KUnit can put references to its test suites. This patch is an early step in transitioning to dispatching all KUnit tests from a centralized executor rather than having each as its own separate
late_initcall.Signed-off-by Brendan Higgins <[email protected]>--- arch/xtensa/kernel/vmlinux.lds.S | 4 ++++
If you ever find yourself modifying multiple arch linker scripts for a series, something has gone wrong. ;)
Signature · 1 line
Kees Cook
PATCH v5 7/12 kunit: test: create a single centralized executor for all tests
In reply to Brendan HigginsFrom: Alan Maguire <[email protected]>
Add a centralized executor to dispatch tests rather than relying on late_initcall to schedule each test suite separately. Centralized execution is for built-in tests only; modules will execute tests when loaded.
--- include/kunit/test.h | 67 +++++++++++++++++++++++++++++--------------- lib/kunit/Makefile | 3 +- lib/kunit/executor.c | 28 ++++++++++++++++++ lib/kunit/test.c | 2 +- 4 files changed, 76 insertions(+), 24 deletions(-) create mode 100644 lib/kunit/executor.c
include/kunit/test.h +45 -22
@@ -224,7 +224,7 @@ size_t kunit_suite_num_test_cases(struct kunit_suite *suite);unsigned int kunit_test_case_num (struct kunit_suite *suite,struct kunit_case *test_case); -int __kunit_test_suites_init (struct kunit_suite **suites); +int __kunit_test_suites_init (struct kunit_suite *const *const suites);void __kunit_test_suites_exit (struct kunit_suite **suites);
@@ -237,34 +237,57 @@ void __kunit_test_suites_exit(struct kunit_suite **suites);* Registers @suites_list with the test framework. See &struct kunit_suite for * more information. * -* When builtin, KUnit tests are all run as late_initcalls; this means -* that they cannot test anything where tests must run at a different init -* phase. One significant restriction resulting from this is that KUnit -* cannot reliably test anything that is initialize in the late_init phase; -* another is that KUnit is useless to test things that need to be run in -* an earlier init phase. -* -* An alternative is to build the tests as a module. Because modules -* do not support multiple late_initcall()s, we need to initialize an -* array of suites for a module. -* -* TODO([email protected]): Don't run all KUnit tests as -* late_initcalls. I have some future work planned to dispatch all KUnit -* tests from the same place, and at the very least to do so after -* everything else is definitely initialized. +* If a test suite is built-in, module_init() gets translated into +* an initcall which we don't want as the idea is that for builtins +* the executor will manage execution. So ensure we do not define +* module_{init|exit} functions for the builtin case when registering +* suites via kunit_test_suites() below. */ -#define kunit_test_suites(suites_list...) \ - static struct kunit_suite *suites[] = {suites_list, NULL}; \ - static int kunit_test_suites_init(void) \ +#ifdef MODULE +#define kunit_test_suites_for_module (__suites) \ + static int __init kunit_test_suites_init(void) \ { \ - return __kunit_test_suites_init(suites); \ + return __kunit_test_suites_init(__suites); \ } \ - late_initcall(kunit_test_suites_init); \ + module_init(kunit_test_suites_init); \ + \ static void __exit kunit_test_suites_exit(void) \ { \ - return __kunit_test_suites_exit(suites); \ + return __kunit_test_suites_exit(__suites); \ } \ module_exit(kunit_test_suites_exit) +#else +#define kunit_test_suites_for_module (__suites) +#endif /* MODULE */ + +#define __kunit_test_suites (unique_array, unique_suites, ...) \ + static struct kunit_suite *unique_array[] = { __VA_ARGS__, NULL }; \ + kunit_test_suites_for_module(unique_array); \ + static struct kunit_suite **unique_suites \ + __used __section(.kunit_test_suites) = unique_array + +/** +* kunit_test_suites() - used to register one or more &struct kunit_suite +* with KUnit. +* +* @suites: a statically allocated list of &struct kunit_suite. +* +* Registers @suites with the test framework. See &struct kunit_suite for +* more information. +* +* When builtin, KUnit tests are all run via executor; this is done +* by placing the array of struct kunit_suite * in the .kunit_test_suites +* ELF section. +* +* An alternative is to build the tests as a module. Because modules do not +* support multiple initcall()s, we need to initialize an array of suites for a +* module. +* +*/ +#define kunit_test_suites (...) \ + __kunit_test_suites(__UNIQUE_ID(array), \ + __UNIQUE_ID(suites), \ + __VA_ARGS__)#define kunit_test_suite (suite) kunit_test_suites(&suite)
lib/kunit/Makefile +2 -1
@@ -3,7 +3,8 @@ obj-$(CONFIG_KUNIT) += kunit.o kunit-objs += test.o \ string-stream.o \ assert.o \ - try-catch.o + try-catch.o \ + executor.o ifeq ($(CONFIG_KUNIT_DEBUGFS),y) kunit-objs += debugfs.o
lib/kunit/executor.c new file +28 -0
@@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <kunit/test.h> + +/* +* These symbols point to the .kunit_test_suites section and are defined in +* include/asm-generic/vmlinux.lds.h, and consequently must be extern. +*/ +extern struct kunit_suite *const *const __kunit_suites_start[]; +extern struct kunit_suite *const *const __kunit_suites_end[]; + +#if IS_BUILTIN (CONFIG_KUNIT ) + +static int kunit_run_all_tests (void ) +{ +struct kunit_suite *const *const *suites; + +for (suites = __kunit_suites_start; + suites < __kunit_suites_end; + suites++) +__kunit_test_suites_init (*suites); + +return 0 ; +} + +late_initcall (kunit_run_all_tests); + +#endif /* IS_BUILTIN(CONFIG_KUNIT) */
lib/kunit/test.c +1 -1
@@ -380,7 +380,7 @@ static void kunit_init_suite(struct kunit_suite *suite)kunit_debugfs_create_suite (suite); } -int __kunit_test_suites_init (struct kunit_suite **suites) +int __kunit_test_suites_init (struct kunit_suite *const *const suites) {unsigned int i;
Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
Re: [PATCH v5 07/12] kunit: test: create a single centralized executor for all tests
Quoted 133 lines · Brendan Higgins
From: Alan Maguire <[email protected]>
Add a centralized executor to dispatch tests rather than relying on
late_initcallto schedule each test suite separately. Centralized execution is for built-in tests only; modules will execute tests when loaded.Signed-off-by Alan Maguire <[email protected]>Co-developed-by Iurii Zaikin <[email protected]>Signed-off-by Iurii Zaikin <[email protected]>Co-developed-by Brendan Higgins <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- include/kunit/test.h | 67 +++++++++++++++++++++++++++++--------------- lib/kunit/Makefile | 3 +- lib/kunit/executor.c | 28 ++++++++++++++++++ lib/kunit/test.c | 2 +- 4 files changed, 76 insertions(+), 24 deletions(-) create mode 100644 lib/kunit/executor.cinclude/kunit/test.h +45
-22@@ -224,7 +224,7 @@ size_t kunit_suite_num_test_cases(struct kunit_suite *suite);unsigned int kunit_test_case_num (struct kunit_suite *suite,struct kunit_case *test_case); -int __kunit_test_suites_init (struct kunit_suite **suites); +int __kunit_test_suites_init (struct kunit_suite *const *const suites);void __kunit_test_suites_exit (struct kunit_suite **suites);@@ -237,34 +237,57 @@ void __kunit_test_suites_exit(struct kunit_suite **suites);* Registers @suites_list with the test framework. See &struct kunit_suite for * more information. * -* When builtin, KUnit tests are all run as late_initcalls; this means -* that they cannot test anything where tests must run at a different init -* phase. One significant restriction resulting from this is that KUnit -* cannot reliably test anything that is initialize in the late_init phase; -* another is that KUnit is useless to test things that need to be run in -* an earlier init phase. -* -* An alternative is to build the tests as a module. Because modules -* do not support multiple late_initcall()s, we need to initialize an -* array of suites for a module. -* -* TODO([email protected]): Don't run all KUnit tests as -* late_initcalls. I have some future work planned to dispatch all KUnit -* tests from the same place, and at the very least to do so after -* everything else is definitely initialized. +* If a test suite is built-in, module_init() gets translated into +* an initcall which we don't want as the idea is that for builtins +* the executor will manage execution. So ensure we do not define +* module_{init|exit} functions for the builtin case when registering +* suites via kunit_test_suites() below. */ -#define kunit_test_suites(suites_list...) \ - static struct kunit_suite *suites[] = {suites_list, NULL}; \ - static int kunit_test_suites_init(void) \ +#ifdef MODULE +#define kunit_test_suites_for_module (__suites) \ + static int __init kunit_test_suites_init(void) \ { \ - return __kunit_test_suites_init(suites); \ + return __kunit_test_suites_init(__suites); \ } \ - late_initcall(kunit_test_suites_init); \ + module_init(kunit_test_suites_init); \ + \ static void __exit kunit_test_suites_exit(void) \ { \ - return __kunit_test_suites_exit(suites); \ + return __kunit_test_suites_exit(__suites); \ } \ module_exit(kunit_test_suites_exit) +#else +#define kunit_test_suites_for_module (__suites) +#endif /* MODULE */ + +#define __kunit_test_suites (unique_array, unique_suites, ...) \ + static struct kunit_suite *unique_array[] = { __VA_ARGS__, NULL }; \ + kunit_test_suites_for_module(unique_array); \ + static struct kunit_suite **unique_suites \ + __used __section(.kunit_test_suites) = unique_array + +/** +* kunit_test_suites() - used to register one or more &struct kunit_suite +* with KUnit. +* +* @suites: a statically allocated list of &struct kunit_suite. +* +* Registers @suites with the test framework. See &struct kunit_suite for +* more information. +* +* When builtin, KUnit tests are all run via executor; this is done +* by placing the array of struct kunit_suite * in the .kunit_test_suites +* ELF section. +* +* An alternative is to build the tests as a module. Because modules do not +* support multiple initcall()s, we need to initialize an array of suites for a +* module. +* +*/ +#define kunit_test_suites (...) \ + __kunit_test_suites(__UNIQUE_ID(array), \ + __UNIQUE_ID(suites), \ + __VA_ARGS__)#define kunit_test_suite (suite) kunit_test_suites(&suite)lib/kunit/Makefile +2
-1@@ -3,7 +3,8 @@ obj-$(CONFIG_KUNIT) += kunit.o kunit-objs += test.o \ string-stream.o \ assert.o \ - try-catch.o + try-catch.o \ + executor.o ifeq ($(CONFIG_KUNIT_DEBUGFS),y) kunit-objs += debugfs.olib/kunit/executor.c new file +0
-0
lib/kunit/executor.c line 10
Quoted 10 lines
lib/kunit/executor.c new file +10
-0@@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <kunit/test.h> + +/* +* These symbols point to the .kunit_test_suites section and are defined in +* include/asm-generic/vmlinux.lds.h, and consequently must be extern. +*/ +extern struct kunit_suite *const *const __kunit_suites_start[]; +extern struct kunit_suite *const *const __kunit_suites_end[];
I would expect these to be in include/asm-generic/sections.h but I guess it's not required.
Signature · 1 line
Kees Cook
Re: [PATCH v5 07/12] kunit: test: create a single centralized executor for all tests
Quoted 140 lines · Kees Cook
Quoted 138 lines · Brendan Higgins
From: Alan Maguire <[email protected]>
Add a centralized executor to dispatch tests rather than relying on
late_initcallto schedule each test suite separately. Centralized execution is for built-in tests only; modules will execute tests when loaded.Signed-off-by Alan Maguire <[email protected]>Co-developed-by Iurii Zaikin <[email protected]>Signed-off-by Iurii Zaikin <[email protected]>Co-developed-by Brendan Higgins <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- include/kunit/test.h | 67 +++++++++++++++++++++++++++++--------------- lib/kunit/Makefile | 3 +- lib/kunit/executor.c | 28 ++++++++++++++++++ lib/kunit/test.c | 2 +- 4 files changed, 76 insertions(+), 24 deletions(-) create mode 100644 lib/kunit/executor.cinclude/kunit/test.h +45
-22@@ -224,7 +224,7 @@ size_t kunit_suite_num_test_cases(struct kunit_suite *suite);unsigned int kunit_test_case_num (struct kunit_suite *suite,struct kunit_case *test_case); -int __kunit_test_suites_init (struct kunit_suite **suites); +int __kunit_test_suites_init (struct kunit_suite *const *const suites);void __kunit_test_suites_exit (struct kunit_suite **suites);@@ -237,34 +237,57 @@ void __kunit_test_suites_exit(struct kunit_suite **suites);* Registers @suites_list with the test framework. See &struct kunit_suite for * more information. * -* When builtin, KUnit tests are all run as late_initcalls; this means -* that they cannot test anything where tests must run at a different init -* phase. One significant restriction resulting from this is that KUnit -* cannot reliably test anything that is initialize in the late_init phase; -* another is that KUnit is useless to test things that need to be run in -* an earlier init phase. -* -* An alternative is to build the tests as a module. Because modules -* do not support multiple late_initcall()s, we need to initialize an -* array of suites for a module. -* -* TODO([email protected]): Don't run all KUnit tests as -* late_initcalls. I have some future work planned to dispatch all KUnit -* tests from the same place, and at the very least to do so after -* everything else is definitely initialized. +* If a test suite is built-in, module_init() gets translated into +* an initcall which we don't want as the idea is that for builtins +* the executor will manage execution. So ensure we do not define +* module_{init|exit} functions for the builtin case when registering +* suites via kunit_test_suites() below. */ -#define kunit_test_suites(suites_list...) \ - static struct kunit_suite *suites[] = {suites_list, NULL}; \ - static int kunit_test_suites_init(void) \ +#ifdef MODULE +#define kunit_test_suites_for_module (__suites) \ + static int __init kunit_test_suites_init(void) \ { \ - return __kunit_test_suites_init(suites); \ + return __kunit_test_suites_init(__suites); \ } \ - late_initcall(kunit_test_suites_init); \ + module_init(kunit_test_suites_init); \ + \ static void __exit kunit_test_suites_exit(void) \ { \ - return __kunit_test_suites_exit(suites); \ + return __kunit_test_suites_exit(__suites); \ } \ module_exit(kunit_test_suites_exit) +#else +#define kunit_test_suites_for_module (__suites) +#endif /* MODULE */ + +#define __kunit_test_suites (unique_array, unique_suites, ...) \ + static struct kunit_suite *unique_array[] = { __VA_ARGS__, NULL }; \ + kunit_test_suites_for_module(unique_array); \ + static struct kunit_suite **unique_suites \ + __used __section(.kunit_test_suites) = unique_array + +/** +* kunit_test_suites() - used to register one or more &struct kunit_suite +* with KUnit. +* +* @suites: a statically allocated list of &struct kunit_suite. +* +* Registers @suites with the test framework. See &struct kunit_suite for +* more information. +* +* When builtin, KUnit tests are all run via executor; this is done +* by placing the array of struct kunit_suite * in the .kunit_test_suites +* ELF section. +* +* An alternative is to build the tests as a module. Because modules do not +* support multiple initcall()s, we need to initialize an array of suites for a +* module. +* +*/ +#define kunit_test_suites (...) \ + __kunit_test_suites(__UNIQUE_ID(array), \ + __UNIQUE_ID(suites), \ + __VA_ARGS__)#define kunit_test_suite (suite) kunit_test_suites(&suite)lib/kunit/Makefile +2
-1@@ -3,7 +3,8 @@ obj-$(CONFIG_KUNIT) += kunit.o kunit-objs += test.o \ string-stream.o \ assert.o \ - try-catch.o + try-catch.o \ + executor.o ifeq ($(CONFIG_KUNIT_DEBUGFS),y) kunit-objs += debugfs.olib/kunit/executor.c new file +5
-0@@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <kunit/test.h> + +/*
Quoted 8 lines
lib/kunit/executor.c line 10
Quoted 5 lines
lib/kunit/executor.c new file +5
-0+* These symbols point to the .kunit_test_suites section and are defined in +* include/asm-generic/vmlinux.lds.h, and consequently must be extern. +*/ +extern struct kunit_suite *const *const __kunit_suites_start[]; +extern struct kunit_suite *const *const __kunit_suites_end[];I would expect these to be in
include/asm-generic/sections.hbut I guess it's not required.
I don't have strong opinions either way, but I think this is less clutter since KUnit is the only one that uses it.
Quoted 1 line
Reviewed-by Kees Cook <[email protected]>
Thanks!
PATCH v5 8/12 init: main: add KUnit to kernel init
In reply to Brendan HigginsRemove KUnit from init calls entirely, instead call directly from kernel_init().
--- include/kunit/test.h | 9 +++++++++ init/main.c | 4 ++++ lib/kunit/executor.c | 4 +--- 3 files changed, 14 insertions(+), 3 deletions(-)
include/kunit/test.h +9 -0
@@ -228,6 +228,15 @@ int __kunit_test_suites_init(struct kunit_suite * const * const suites);void __kunit_test_suites_exit (struct kunit_suite **suites); +#if IS_BUILTIN (CONFIG_KUNIT ) +int kunit_run_all_tests (void ); +#else +static inline int kunit_run_all_tests (void ) +{ +return 0 ; +} +#endif /* IS_BUILTIN(CONFIG_KUNIT) */ +/** * kunit_test_suites() - used to register one or more &struct kunit_suite * with KUnit.
init/main.c +4 -0
@@ -106,6 +106,8 @@#define CREATE_TRACE_POINTS #include <trace/events/initcall.h> +#include <kunit/test.h> +static int kernel_init (void *);extern void init_IRQ (void );
@@ -1504,6 +1506,8 @@ static noinline void __init kernel_init_freeable(void)do_basic_setup (); +kunit_run_all_tests (); +console_on_rootfs ();/*
lib/kunit/executor.c +1 -3
@@ -11,7 +11,7 @@ extern struct kunit_suite * const * const __kunit_suites_end[];#if IS_BUILTIN (CONFIG_KUNIT ) -static int kunit_run_all_tests (void ) +int kunit_run_all_tests (void ) {struct kunit_suite *const *const *suites;
@@ -23,6 +23,4 @@ static int kunit_run_all_tests(void) return 0; } -late_initcall (kunit_run_all_tests); -#endif /* IS_BUILTIN(CONFIG_KUNIT) */
Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
Re: [PATCH v5 08/12] init: main: add KUnit to kernel init
Quoted 7 lines · Brendan Higgins
Remove KUnit from init calls entirely, instead call directly from
kernel_init().Co-developed-by Alan Maguire <[email protected]>Signed-off-by Alan Maguire <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>
Signature · 1 line
Kees Cook
Re: [PATCH v5 08/12] init: main: add KUnit to kernel init
In reply to Brendan HigginsQuoted 2 lines · Brendan Higgins
Remove KUnit from init calls entirely, instead call directly from
kernel_init().
The commit log does not explain *why*.
Quoted 4 lines
Co-developed-by Alan Maguire <[email protected]>Signed-off-by Alan Maguire <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>
Other than that:
Luis
PATCH v5 9/12 kunit: test: add test plan to KUnit TAP format
In reply to Brendan HigginsTAP 14 allows an optional test plan to be emitted before the start of the start of testing[1]; this is valuable because it makes it possible for a test harness to detect whether the number of tests run matches the number of tests expected to be run, ensuring that no tests silently failed.
Link[1]: https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md#the-plan
--- lib/kunit/executor.c | 17 ++++ lib/kunit/test.c | 11 --- tools/testing/kunit/kunit_parser.py | 76 +++++++++++++++--- .../test_is_test_passed-all_passed.log | Bin 1562 -> 1567 bytes .../test_data/test_is_test_passed-crash.log | Bin 3016 -> 3021 bytes .../test_data/test_is_test_passed-failure.log | Bin 1700 -> 1705 bytes 6 files changed, 80 insertions(+), 24 deletions(-)
lib/kunit/executor.c +17 -0
@@ -11,10 +11,27 @@ extern struct kunit_suite * const * const __kunit_suites_end[];#if IS_BUILTIN (CONFIG_KUNIT ) +static void kunit_print_tap_header (void ) +{ +struct kunit_suite *const *const *suites, *const *subsuite; +int num_of_suites =0 ; + +for (suites = __kunit_suites_start; + suites < __kunit_suites_end; + suites++) +for (subsuite = *suites; *subsuite !=NULL ; subsuite++) + num_of_suites++; + +pr_info ("TAP version 14\n" ); +pr_info ("1..%d\n" , num_of_suites); +} +int kunit_run_all_tests (void ) {struct kunit_suite *const *const *suites; +kunit_print_tap_header (); +for (suites = __kunit_suites_start; suites < __kunit_suites_end; suites++)
lib/kunit/test.c +0 -11
@@ -19,16 +19,6 @@ static void kunit_set_failure(struct kunit *test)WRITE_ONCE (test->success, false); } -static void kunit_print_tap_version (void ) -{ -static bool kunit_has_printed_tap_version; - -if (!kunit_has_printed_tap_version) { -pr_info ("TAP version 14\n" ); - kunit_has_printed_tap_version = true; - } -} -/* * Append formatted message to log, size of which is limited to * KUNIT_LOG_SIZE bytes (including null terminating byte).
@@ -68,7 +58,6 @@ EXPORT_SYMBOL_GPL(kunit_suite_num_test_cases);static void kunit_print_subtest_start (struct kunit_suite *suite) { -kunit_print_tap_version ();kunit_log (KERN_INFO , suite,KUNIT_SUBTEST_INDENT "# Subtest: %s" , suite->name); kunit_log(KERN_INFO, suite, KUNIT_SUBTEST_INDENT "1..%zd",
tools/testing/kunit/kunit_parser.py +63 -13
138 lines not on this page · raw message
tools/testing/kunit/test_data/test_is_test_passed-all_passed.log binary +0 -0
tools/testing/kunit/test_data/test_is_test_passed-crash.log binary +0 -0
tools/testing/kunit/test_data/test_is_test_passed-failure.log binary +0 -0
Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
Re: [PATCH v5 09/12] kunit: test: add test plan to KUnit TAP format
Quoted 9 lines · Brendan Higgins
TAP 14 allows an optional test plan to be emitted before the start of the start of testing[1]; this is valuable because it makes it possible for a test harness to detect whether the number of tests run matches the number of tests expected to be run, ensuring that no tests silently failed.
Link[1]: https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md#the-plan
Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>
Look good, except...
tools/testing/kunit/test_data/test_is_test_passed-failure.log
Quoted 26 lines
tools/testing/kunit/test_data/test_is_test_passed-all_passed.log binary +0
-0tools/testing/kunit/test_data/test_is_test_passed-crash.log binary +0
-0tools/testing/kunit/test_data/test_is_test_passed-failure.log binary +0
-0
What is happening here?? Those logs appear as text to me. Why did git freak out?
Signature · 1 line
Kees Cook
Re: [PATCH v5 09/12] kunit: test: add test plan to KUnit TAP format
Quoted 38 lines · Kees Cook
Quoted 9 lines · Brendan Higgins
TAP 14 allows an optional test plan to be emitted before the start of the start of testing[1]; this is valuable because it makes it possible for a test harness to detect whether the number of tests run matches the number of tests expected to be run, ensuring that no tests silently failed.
Link[1]: https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md#the-plan
Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>Look good, except...
Quoted 24 lines
tools/testing/kunit/test_data/test_is_test_passed-all_passed.log binary +0
-0tools/testing/kunit/test_data/test_is_test_passed-crash.log binary +0
-0tools/testing/kunit/test_data/test_is_test_passed-failure.log binary +0
-0
Quoted 5 lines
Quoted 2 lines
delta 23
ecmZ3<yM&i7#4$jjEVZaOGe1wk&}5_VG&TTPhX-Z=What is happening here?? Those logs appear as text to me. Why did git freak out?
That's because this is all test data; it's all plaintext, but out of necessity some of the test data is kind of munged up and causes checkpatch to complain, so Shuah asked us to mark it as binary since it isn't actually code and so checkpatch will stop flagging it.
PATCH v5 10/12 kunit: Add 'kunit_shutdown' option
In reply to Brendan HigginsFrom: David Gow <[email protected]>
Add a new kernel command-line option, 'kunit_shutdown', which allows the user to specify that the kernel poweroff, halt, or reboot after completing all KUnit tests; this is very handy for running KUnit tests on UML or a VM so that the UML/VM process exits cleanly immediately after running all tests without needing a special initramfs.
--- lib/kunit/executor.c | 20 ++++++++++++++++++++ tools/testing/kunit/kunit_kernel.py | 2 +- tools/testing/kunit/kunit_parser.py | 2 +- 3 files changed, 22 insertions(+), 2 deletions(-)
lib/kunit/executor.c +20 -0
@@ -1,5 +1,6 @@// SPDX-License-Identifier: GPL-2.0 +#include <linux/reboot.h> #include <kunit/test.h> /*
@@ -11,6 +12,23 @@ extern struct kunit_suite * const * const __kunit_suites_end[];#if IS_BUILTIN (CONFIG_KUNIT ) +static char *kunit_shutdown; +core_param (kunit_shutdown, kunit_shutdown, charp,0644 ); + +static void kunit_handle_shutdown (void ) +{ +if (!kunit_shutdown) +return ; + +if (!strcmp (kunit_shutdown,"poweroff" )) +kernel_power_off (); +else if (!strcmp (kunit_shutdown,"halt" )) +kernel_halt (); +else if (!strcmp (kunit_shutdown,"reboot" )) +kernel_restart (NULL ); + +} +static void kunit_print_tap_header (void ) {struct kunit_suite *const *const *suites, *const *subsuite;
@@ -37,6 +55,8 @@ int kunit_run_all_tests(void) suites++)__kunit_test_suites_init (*suites); +kunit_handle_shutdown (); +return 0 ; }
tools/testing/kunit/kunit_kernel.py +1 -1
@@ -172,7 +172,7 @@ class LinuxSourceTree(object): return self.validate_config(build_dir) def run_kernel(self, args=[], build_dir='', timeout=None): - args.extend(['mem=1G']) + args.extend(['mem=1G', 'kunit_shutdown=halt']) outfile = 'test.log' self._ops.linux_bin(args, timeout, build_dir, outfile) subprocess.call(['stty', 'sane'])
tools/testing/kunit/kunit_parser.py +1 -1
@@ -49,7 +49,7 @@ class TestStatus(Enum): kunit_start_re = re.compile(r'TAP version [0-9]+$') kunit_end_re = re.compile('(List of all partitions:|' - 'Kernel panic - not syncing: VFS:)') + 'Kernel panic - not syncing: VFS:|reboot: System halted)') def isolate_kunit_output(kernel_output): started = False
Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
Re: [PATCH v5 10/12] kunit: Add 'kunit_shutdown' option
Quoted 38 lines · Brendan Higgins
From: David Gow <[email protected]>
Add a new kernel command-line option, '
kunit_shutdown', which allows the user to specify that the kernel poweroff, halt, or reboot after completing all KUnit tests; this is very handy for running KUnit tests on UML or a VM so that the UML/VM process exits cleanly immediately after running all tests without needing a special initramfs.Signed-off-by David Gow <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- lib/kunit/executor.c | 20 ++++++++++++++++++++ tools/testing/kunit/kunit_kernel.py | 2 +- tools/testing/kunit/kunit_parser.py | 2 +- 3 files changed, 22 insertions(+), 2 deletions(-)lib/kunit/executor.c +7
-0@@ -1,5 +1,6 @@// SPDX-License-Identifier: GPL-2.0 +#include <linux/reboot.h> #include <kunit/test.h> /* @@ -11,6 +12,23 @@ extern struct kunit_suite * const * const __kunit_suites_end[];#if IS_BUILTIN (CONFIG_KUNIT ) +static char *kunit_shutdown; +core_param (kunit_shutdown, kunit_shutdown, charp,0644 ); + +static void kunit_handle_shutdown (void ) +{ +if (!kunit_shutdown)
lib/kunit/executor.c extern struct kunit_suite * const * const __kunit_suites_end[]; · line 30
Quoted 10 lines
lib/kunit/executor.c +10
-0+return ; + +if (!strcmp (kunit_shutdown,"poweroff" )) +kernel_power_off (); +else if (!strcmp (kunit_shutdown,"halt" )) +kernel_halt (); +else if (!strcmp (kunit_shutdown,"reboot" )) +kernel_restart (NULL ); + +}
If you have patches that do something just before the initrd, and then you add more patches to shut down immediately after an initrd, people may ask you to just use an initrd instead of filling the kernel with these changes...
I mean, I get it, but it's not hard to make an initrd that poke a sysctl to start the tests...
In fact, you don't even need a initrd to poke sysctls these days.
Signature · 1 line
Kees Cook
Re: [PATCH v5 10/12] kunit: Add 'kunit_shutdown' option
Quoted 56 lines · Kees Cook
Quoted 38 lines · Brendan Higgins
From: David Gow <[email protected]>
Add a new kernel command-line option, '
kunit_shutdown', which allows the user to specify that the kernel poweroff, halt, or reboot after completing all KUnit tests; this is very handy for running KUnit tests on UML or a VM so that the UML/VM process exits cleanly immediately after running all tests without needing a special initramfs.Signed-off-by David Gow <[email protected]>Signed-off-by Brendan Higgins <[email protected]>Reviewed-by Stephen Boyd <[email protected]>--- lib/kunit/executor.c | 20 ++++++++++++++++++++ tools/testing/kunit/kunit_kernel.py | 2 +- tools/testing/kunit/kunit_parser.py | 2 +- 3 files changed, 22 insertions(+), 2 deletions(-)lib/kunit/executor.c +7
-0@@ -1,5 +1,6 @@// SPDX-License-Identifier: GPL-2.0 +#include <linux/reboot.h> #include <kunit/test.h> /* @@ -11,6 +12,23 @@ extern struct kunit_suite * const * const __kunit_suites_end[];#if IS_BUILTIN (CONFIG_KUNIT ) +static char *kunit_shutdown; +core_param (kunit_shutdown, kunit_shutdown, charp,0644 ); + +static void kunit_handle_shutdown (void ) +{ +if (!kunit_shutdown)lib/kunit/executor.c extern struct kunit_suite * const * const __kunit_suites_end[]; · line 30
Quoted 10 lines
lib/kunit/executor.c +10
-0+return ; + +if (!strcmp (kunit_shutdown,"poweroff" )) +kernel_power_off (); +else if (!strcmp (kunit_shutdown,"halt" )) +kernel_halt (); +else if (!strcmp (kunit_shutdown,"reboot" )) +kernel_restart (NULL ); + +}If you have patches that do something just before the initrd, and then you add more patches to shut down immediately after an initrd, people may ask you to just use an initrd instead of filling the kernel with these changes...
Quoted 4 lines
I mean, I get it, but it's not hard to make an initrd that poke a sysctl to start the tests...
In fact, you don't even need a initrd to poke sysctls these days.
True, but it is nice to not have to maintain an initramfs for each architecture that you want to test. Still, I see your point. If we can find a convenient way to distribute the needed dependencies for running KUnit on each non-UML architecture then I think I can abandon this change.
So how about this: I will drop this patch from this patchset and move it up to the follow up patchset that adds multiarchitecture support to kunit_tool. There we can address the problem of how to best track the necessary dependencies including possibly intitramfss.
_______________________________________________
linux-um mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-um
PATCH v5 11/12 Documentation: Add kunit_shutdown to kernel-parameters.txt
In reply to Brendan HigginsAdd kunit_shutdown, an option to specify that the kernel shutsdown after running KUnit tests, to the kernel-parameters.txt documentation.
--- Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++ 1 file changed, 8 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +8 -0
@@ -2183,6 +2183,14 @@ 0: force disabled 1: force enabled + kunit_shutdown=[KERNEL UNIT TESTING FRAMEWORK] Shutdown kernel after + running built-in tests. Tests configured as modules will + not be run. + Default: (flag not present) don't shutdown + poweroff: poweroff the kernel after running tests + halt: halt the kernel after running tests + reboot: reboot the kernel after running tests + kvm.ignore_msrs=[KVM] Ignore guest accesses to unhandled MSRs. Default is 0 (don't ignore, but inject #GP)
Signature · 1 line
2.27.0.212.ge8ba1cc988-goog
PATCH v5 12/12 Documentation: kunit: add a brief blurb about kunit_test_suite
In reply to Brendan HigginsAdd a brief blurb saying how and when the kunit_test_suite() macro works to the usage documentation.
--- Documentation/dev-tools/kunit/usage.rst | 5 +++++ 1 file changed, 5 insertions(+)
Documentation/dev-tools/kunit/usage.rst +5 -0
@@ -211,6 +211,11 @@ KUnit test framework. .. note:: A test case will only be run if it is associated with a test suite. +``kunit_test_suite(...)`` is a macro which tells the linker to put the specified +test suite in a special linker section so that it can be run by KUnit either +after late_init, or when the test module is loaded (depending on whether the +test was built in or not). + For more information on these types of things see the :doc:`api/test`. Isolating Behavior
Signature · 7 lines
2.27.0.212.ge8ba1cc988-goog _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Quoted 3 lines · Brendan Higgins
This patchset adds a centralized executor to dispatch tests rather than relying on
late_initcallto schedule each test suite separately along with a couple of new features that depend on it.
So, the new section looks fine to me (modulo the INIT_DATA change). The plumbing to start the tests, though, I think is redundant. Why not just add a sysctl that starts all known tests?
That way you don't need the plumbing into init/main.c, and you can have a mode where builtin tests can be started on a fully booted system too.
i.e. boot with "sysctl.kernel.kunit=start" or when fully booted with "echo start > /proc/sys/kernel/kunit"
And instead of the kunit-specific halt/reboot stuff, how about moving /proc/sysrq-trigger into /proc/sys instead? Then you (or anything) could do:
sysctl.kernel.kunit=start sysctl.kernel.sysrq-trigger=b
Signature · 1 line
Kees Cook
Quoted 5 lines · Kees Cook
Quoted 3 lines · Brendan Higgins
This patchset adds a centralized executor to dispatch tests rather than relying on
late_initcallto schedule each test suite separately along with a couple of new features that depend on it.
Sorry it took so long to reply. I got sucked into some other stuff again.
Quoted 3 lines
So, the new section looks fine to me (modulo the
INIT_DATAchange). The plumbing to start the tests, though, I think is redundant. Why not just add a sysctl that starts all known tests?
We already have that; however, we use debugfs to start the tests - same difference. I just find it convenient to not have to build and then maintain a userland for each architecture. It's also really nice that KUnit "just works out of the box" - you don't have to download anything other than the kernel source, and you don't need to do any steps outside of just run "kuit.py run". That seems like a big advantage to me.
Quoted 11 lines
That way you don't need the plumbing into
init/main.c, and you can have a mode where builtin tests can be started on a fully booted system too.i.e. boot with "sysctl.kernel.kunit=start" or when fully booted with "echo start >
/proc/sys/kernel/kunit"And instead of the kunit-specific halt/reboot stuff, how about moving
/proc/sysrq-triggerinto/proc/sysinstead? Then you (or anything) could do:sysctl.kernel.kunit=start sysctl.kernel.sysrq-trigger=b
I think it might be harder to make a case for the reboot stuff without the stuff I am working on outside of this patchset. I think I will probably drop that patch from this patchset and reintroduce it later.