Skip to contentSkip to conversation

kcsan: test: use new suite_{init,exit} support

PATCH In review Marco Elver 4 participants 8 messages May 4, 2022 – May 12, 2022 KCSAN

lore
Share & tools
lore

Marco Elver

author maintainer
#0 lore raw

PATCH 1/1 kcsan: test: use new suite_{init,exit} support

Use the newly added suite_{init,exit} support for suite-wide init and cleanup. This avoids the unsupported method by which the test used to do suite-wide init and cleanup (avoiding issues such as missing TAP headers, and possible future conflicts).

---
This patch should go on the -kselftest/kunit branch, where this new support currently lives, including a similar change to the KFENCE test.

---
 kernel/kcsan/kcsan_test.c | 31 +++++++++++++------------------
 1 file changed, 13 insertions(+), 18 deletions(-)
kernel/kcsan/kcsan_test.c +13 -18
index a36fca063a73..59560b5e1d9c 100644
@@ -1565,14 +1565,6 @@ static void test_exit(struct kunit *test)
 	torture_cleanup_end();
 }
 
-static struct kunit_suite kcsan_test_suite = {
-	.name = "kcsan",
-	.test_cases = kcsan_test_cases,
-	.init = test_init,
-	.exit = test_exit,
-};
-static struct kunit_suite *kcsan_test_suites[] = { &kcsan_test_suite, NULL };
-
 __no_kcsan
 static void register_tracepoints(struct tracepoint *tp, void *ignore)
 {
@@ -1588,11 +1580,7 @@ static void unregister_tracepoints(struct tracepoint *tp, void *ignore)
 		tracepoint_probe_unregister(tp, probe_console, NULL);
 }
 
-/*
- * We only want to do tracepoints setup and teardown once, therefore we have to
- * customize the init and exit functions and cannot rely on kunit_test_suite().
- */
-static int __init kcsan_test_init(void)
+static int kcsan_suite_init(struct kunit_suite *suite)
 {
 	/*
 	 * Because we want to be able to build the test as a module, we need to
@@ -1600,18 +1588,25 @@ static int __init kcsan_test_init(void)
 	 * won't work here.
 	 */
 	for_each_kernel_tracepoint(register_tracepoints, NULL);
-	return __kunit_test_suites_init(kcsan_test_suites);
+	return 0;
 }
 
-static void kcsan_test_exit(void)
+static void kcsan_suite_exit(struct kunit_suite *suite)
 {
-	__kunit_test_suites_exit(kcsan_test_suites);
 	for_each_kernel_tracepoint(unregister_tracepoints, NULL);
 	tracepoint_synchronize_unregister();
 }
 
-late_initcall_sync(kcsan_test_init);
-module_exit(kcsan_test_exit);
+static struct kunit_suite kcsan_test_suite = {
+	.name = "kcsan",
+	.test_cases = kcsan_test_cases,
+	.init = test_init,
+	.exit = test_exit,
+	.suite_init = kcsan_suite_init,
+	.suite_exit = kcsan_suite_exit,
+};
+
+kunit_test_suites(&kcsan_test_suite);
 
 MODULE_LICENSE("GPL v2");
 MODULE_AUTHOR("Marco Elver <[email protected]>");
Signature · 1 line
2.36.0.464.gb9c8b46e94-goog
Reviewed-by on v1 1/1
#1 lore raw
Quoted 11 lines · Marco Elver

Use the newly added suite_{init,exit} support for suite-wide init and cleanup. This avoids the unsupported method by which the test used to do suite-wide init and cleanup (avoiding issues such as missing TAP headers, and possible future conflicts).

---
This patch should go on the -kselftest/kunit branch, where this new support currently lives, including a similar change to the KFENCE test.
---

Thanks! This is working for me. I ran it as a builtin using kunit_tool under (I had to add an x86_64-smp architecture), then use:
./tools/testing/kunit/kunit.py run --arch=x86_64-smp --kconfig_add=CONFIG_KCSAN=y --kconfig_add=CONFIG_DEBUG_KERNEL=y --timeout 900 'kcsan'

To add the x86_64 smp architecture, I added a file ./tools/testing/kunit/qemu_configs/x86_64-smp.py, which was a copy of x86_64.py but with 'CONFIG_SMP=y' added to XXXX and '-smp 16' added to YYYY.
It took about 10 minutes on my system, so the default 5 minute timeout definitely wasn't enough.

(It's maybe worth noting that kunit_tool's output is pretty ugly when this isn't running on an SMP system, as the skipped subtests -- plus the "no tests run" errors -- take up a lot of space on the screen. That's possibly something we should consider when we look further into how the kunit_tool NO_TEST result works. Not really related to this change (or even this test) though.)

No complaints about the patch: I'm just really glad to see things migrate off custom init/exit code!

-- David

Quoted 69 lines
 kernel/kcsan/kcsan_test.c | 31 +++++++++++++------------------
 1 file changed, 13 insertions(+), 18 deletions(-)
kernel/kcsan/kcsan_test.c +13 -19
index a36fca063a73..59560b5e1d9c 100644
@@ -1565,14 +1565,6 @@ static void test_exit(struct kunit *test)
        torture_cleanup_end();
 }
 
-static struct kunit_suite kcsan_test_suite = {
-       .name = "kcsan",
-       .test_cases = kcsan_test_cases,
-       .init = test_init,
-       .exit = test_exit,
-};
-static struct kunit_suite *kcsan_test_suites[] = { &kcsan_test_suite, NULL };
-
 __no_kcsan
 static void register_tracepoints(struct tracepoint *tp, void *ignore)
 {
@@ -1588,11 +1580,7 @@ static void unregister_tracepoints(struct tracepoint *tp, void *ignore)
                tracepoint_probe_unregister(tp, probe_console, NULL);
 }
 
-/*
- * We only want to do tracepoints setup and teardown once, therefore we have to
- * customize the init and exit functions and cannot rely on kunit_test_suite().
- */
-static int __init kcsan_test_init(void)
+static int kcsan_suite_init(struct kunit_suite *suite)
 {
        /*
         * Because we want to be able to build the test as a module, we need to
@@ -1600,18 +1588,25 @@ static int __init kcsan_test_init(void)
         * won't work here.
         */
        for_each_kernel_tracepoint(register_tracepoints, NULL);
-       return __kunit_test_suites_init(kcsan_test_suites);
+       return 0;
 }
 
-static void kcsan_test_exit(void)
+static void kcsan_suite_exit(struct kunit_suite *suite)
 {
-       __kunit_test_suites_exit(kcsan_test_suites);
        for_each_kernel_tracepoint(unregister_tracepoints, NULL);
        tracepoint_synchronize_unregister();
 }
 
-late_initcall_sync(kcsan_test_init);
-module_exit(kcsan_test_exit);
+static struct kunit_suite kcsan_test_suite = {
+       .name = "kcsan",
+       .test_cases = kcsan_test_cases,
+       .init = test_init,
+       .exit = test_exit,
+       .suite_init = kcsan_suite_init,
+       .suite_exit = kcsan_suite_exit,
+};
+
+kunit_test_suites(&kcsan_test_suite);
 
 MODULE_LICENSE("GPL v2");
 MODULE_AUTHOR("Marco Elver <[email protected]>");
--

2.36.0.464.gb9c8b46e94-goog

author maintainer
#2 lore raw
Quoted 20 lines · David Gow
Quoted 11 lines · Marco Elver

Use the newly added suite_{init,exit} support for suite-wide init and cleanup. This avoids the unsupported method by which the test used to do suite-wide init and cleanup (avoiding issues such as missing TAP headers, and possible future conflicts).

---
This patch should go on the -kselftest/kunit branch, where this new support currently lives, including a similar change to the KFENCE test.
---

Thanks! This is working for me. I ran it as a builtin using kunit_tool under (I had to add an x86_64-smp architecture), then use:
./tools/testing/kunit/kunit.py run --arch=x86_64-smp --kconfig_add=CONFIG_KCSAN=y --kconfig_add=CONFIG_DEBUG_KERNEL=y --timeout 900 'kcsan'

Quoted 6 lines

To add the x86_64 smp architecture, I added a file ./tools/testing/kunit/qemu_configs/x86_64-smp.py, which was a copy of x86_64.py but with 'CONFIG_SMP=y' added to XXXX and '-smp 16' added to YYYY.
It took about 10 minutes on my system, so the default 5 minute timeout definitely wasn't enough.

The trick to reduce the KCSAN test time is to set CONFIG_KCSAN_REPORT_ONCE_IN_MS=100 or lower. So should you consider a special KUnit config, I'd add that.

Quoted 11 lines

(It's maybe worth noting that kunit_tool's output is pretty ugly when this isn't running on an SMP system, as the skipped subtests -- plus the "no tests run" errors -- take up a lot of space on the screen. That's possibly something we should consider when we look further into how the kunit_tool NO_TEST result works. Not really related to this change (or even this test) though.)

No complaints about the patch: I'm just really glad to see things migrate off custom init/exit code!

Thank you!

Quoted 71 lines

-- David

Quoted 69 lines
 kernel/kcsan/kcsan_test.c | 31 +++++++++++++------------------
 1 file changed, 13 insertions(+), 18 deletions(-)
kernel/kcsan/kcsan_test.c +13 -19
index a36fca063a73..59560b5e1d9c 100644
@@ -1565,14 +1565,6 @@ static void test_exit(struct kunit *test)
        torture_cleanup_end();
 }
 
-static struct kunit_suite kcsan_test_suite = {
-       .name = "kcsan",
-       .test_cases = kcsan_test_cases,
-       .init = test_init,
-       .exit = test_exit,
-};
-static struct kunit_suite *kcsan_test_suites[] = { &kcsan_test_suite, NULL };
-
 __no_kcsan
 static void register_tracepoints(struct tracepoint *tp, void *ignore)
 {
@@ -1588,11 +1580,7 @@ static void unregister_tracepoints(struct tracepoint *tp, void *ignore)
                tracepoint_probe_unregister(tp, probe_console, NULL);
 }
 
-/*
- * We only want to do tracepoints setup and teardown once, therefore we have to
- * customize the init and exit functions and cannot rely on kunit_test_suite().
- */
-static int __init kcsan_test_init(void)
+static int kcsan_suite_init(struct kunit_suite *suite)
 {
        /*
         * Because we want to be able to build the test as a module, we need to
@@ -1600,18 +1588,25 @@ static int __init kcsan_test_init(void)
         * won't work here.
         */
        for_each_kernel_tracepoint(register_tracepoints, NULL);
-       return __kunit_test_suites_init(kcsan_test_suites);
+       return 0;
 }
 
-static void kcsan_test_exit(void)
+static void kcsan_suite_exit(struct kunit_suite *suite)
 {
-       __kunit_test_suites_exit(kcsan_test_suites);
        for_each_kernel_tracepoint(unregister_tracepoints, NULL);
        tracepoint_synchronize_unregister();
 }
 
-late_initcall_sync(kcsan_test_init);
-module_exit(kcsan_test_exit);
+static struct kunit_suite kcsan_test_suite = {
+       .name = "kcsan",
+       .test_cases = kcsan_test_cases,
+       .init = test_init,
+       .exit = test_exit,
+       .suite_init = kcsan_suite_init,
+       .suite_exit = kcsan_suite_exit,
+};
+
+kunit_test_suites(&kcsan_test_suite);
 
 MODULE_LICENSE("GPL v2");
 MODULE_AUTHOR("Marco Elver <[email protected]>");
--

2.36.0.464.gb9c8b46e94-goog

#3 lore raw
Quoted 22 lines · Marco Elver
Quoted 20 lines · David Gow
Quoted 11 lines · Marco Elver

Use the newly added suite_{init,exit} support for suite-wide init and cleanup. This avoids the unsupported method by which the test used to do suite-wide init and cleanup (avoiding issues such as missing TAP headers, and possible future conflicts).

---
This patch should go on the -kselftest/kunit branch, where this new support currently lives, including a similar change to the KFENCE test.
---

Thanks! This is working for me. I ran it as a builtin using kunit_tool under (I had to add an x86_64-smp architecture), then use:
./tools/testing/kunit/kunit.py run --arch=x86_64-smp --kconfig_add=CONFIG_KCSAN=y --kconfig_add=CONFIG_DEBUG_KERNEL=y --timeout 900 'kcsan'

Quoted 4 lines
Quoted 4 lines

To add the x86_64 smp architecture, I added a file ./tools/testing/kunit/qemu_configs/x86_64-smp.py, which was a copy of x86_64.py but with 'CONFIG_SMP=y' added to XXXX and '-smp 16' added to YYYY.

(Whoops, forgot to copy this in properly: XXXX was 'kconfig' and YYYY was 'extra_qemu_params'.)

The x86_64-smp.py file ends up looking like this:
---8<---
from ..qemu_config import QemuArchParams

QEMU_ARCH = QemuArchParams(linux_arch='x86_64',

                          kconfig='''
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SMP=y
                          ''',
                          qemu_arch='x86_64',
                          kernel_path='arch/x86/boot/bzImage',
                          kernel_command_line='console=ttyS0',
                          extra_qemu_params=['-smp 16'])

---8<---

Quoted 7 lines
Quoted 2 lines

It took about 10 minutes on my system, so the default 5 minute timeout definitely wasn't enough.

The trick to reduce the KCSAN test time is to set CONFIG_KCSAN_REPORT_ONCE_IN_MS=100 or lower. So should you consider a special KUnit config, I'd add that.

Ah: it might be worth adding a dedicated kcsan .kunitconfig, in which case this would be helpful. It'd also need the SMP qemu config above before it's particularly useful, and 16 was a randomly-picked number of CPUs -- not sure if there's a better default.

If you're likely to use it, though, we can definitely add it in. I'm sure there'll eventually be other uses for an SMP config under kunit_tool, too.

Quoted 85 lines
Quoted 11 lines

(It's maybe worth noting that kunit_tool's output is pretty ugly when this isn't running on an SMP system, as the skipped subtests -- plus the "no tests run" errors -- take up a lot of space on the screen. That's possibly something we should consider when we look further into how the kunit_tool NO_TEST result works. Not really related to this change (or even this test) though.)

No complaints about the patch: I'm just really glad to see things migrate off custom init/exit code!

Thank you!

Quoted 71 lines

-- David

Quoted 69 lines
 kernel/kcsan/kcsan_test.c | 31 +++++++++++++------------------
 1 file changed, 13 insertions(+), 18 deletions(-)
kernel/kcsan/kcsan_test.c +13 -19
index a36fca063a73..59560b5e1d9c 100644
@@ -1565,14 +1565,6 @@ static void test_exit(struct kunit *test)
        torture_cleanup_end();
 }
 
-static struct kunit_suite kcsan_test_suite = {
-       .name = "kcsan",
-       .test_cases = kcsan_test_cases,
-       .init = test_init,
-       .exit = test_exit,
-};
-static struct kunit_suite *kcsan_test_suites[] = { &kcsan_test_suite, NULL };
-
 __no_kcsan
 static void register_tracepoints(struct tracepoint *tp, void *ignore)
 {
@@ -1588,11 +1580,7 @@ static void unregister_tracepoints(struct tracepoint *tp, void *ignore)
                tracepoint_probe_unregister(tp, probe_console, NULL);
 }
 
-/*
- * We only want to do tracepoints setup and teardown once, therefore we have to
- * customize the init and exit functions and cannot rely on kunit_test_suite().
- */
-static int __init kcsan_test_init(void)
+static int kcsan_suite_init(struct kunit_suite *suite)
 {
        /*
         * Because we want to be able to build the test as a module, we need to
@@ -1600,18 +1588,25 @@ static int __init kcsan_test_init(void)
         * won't work here.
         */
        for_each_kernel_tracepoint(register_tracepoints, NULL);
-       return __kunit_test_suites_init(kcsan_test_suites);
+       return 0;
 }
 
-static void kcsan_test_exit(void)
+static void kcsan_suite_exit(struct kunit_suite *suite)
 {
-       __kunit_test_suites_exit(kcsan_test_suites);
        for_each_kernel_tracepoint(unregister_tracepoints, NULL);
        tracepoint_synchronize_unregister();
 }
 
-late_initcall_sync(kcsan_test_init);
-module_exit(kcsan_test_exit);
+static struct kunit_suite kcsan_test_suite = {
+       .name = "kcsan",
+       .test_cases = kcsan_test_cases,
+       .init = test_init,
+       .exit = test_exit,
+       .suite_init = kcsan_suite_init,
+       .suite_exit = kcsan_suite_exit,
+};
+
+kunit_test_suites(&kcsan_test_suite);
 
 MODULE_LICENSE("GPL v2");
 MODULE_AUTHOR("Marco Elver <[email protected]>");
--

2.36.0.464.gb9c8b46e94-goog

author maintainer
#4 lore raw
Quoted 55 lines · David Gow
Quoted 22 lines · Marco Elver
Quoted 20 lines · David Gow
Quoted 11 lines · Marco Elver

Use the newly added suite_{init,exit} support for suite-wide init and cleanup. This avoids the unsupported method by which the test used to do suite-wide init and cleanup (avoiding issues such as missing TAP headers, and possible future conflicts).

---
This patch should go on the -kselftest/kunit branch, where this new support currently lives, including a similar change to the KFENCE test.
---

Thanks! This is working for me. I ran it as a builtin using kunit_tool under (I had to add an x86_64-smp architecture), then use:
./tools/testing/kunit/kunit.py run --arch=x86_64-smp --kconfig_add=CONFIG_KCSAN=y --kconfig_add=CONFIG_DEBUG_KERNEL=y --timeout 900 'kcsan'

Quoted 4 lines
Quoted 4 lines

To add the x86_64 smp architecture, I added a file ./tools/testing/kunit/qemu_configs/x86_64-smp.py, which was a copy of x86_64.py but with 'CONFIG_SMP=y' added to XXXX and '-smp 16' added to YYYY.

(Whoops, forgot to copy this in properly: XXXX was 'kconfig' and YYYY was 'extra_qemu_params'.)

The x86_64-smp.py file ends up looking like this:
---8<---
from ..qemu_config import QemuArchParams

QEMU_ARCH = QemuArchParams(linux_arch='x86_64',

                          kconfig='''
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SMP=y
                          ''',
                          qemu_arch='x86_64',
                          kernel_path='arch/x86/boot/bzImage',
                          kernel_command_line='console=ttyS0',
                          extra_qemu_params=['-smp 16'])

---8<---

Quoted 7 lines
Quoted 2 lines

It took about 10 minutes on my system, so the default 5 minute timeout definitely wasn't enough.

The trick to reduce the KCSAN test time is to set CONFIG_KCSAN_REPORT_ONCE_IN_MS=100 or lower. So should you consider a special KUnit config, I'd add that.

Quoted 8 lines

Ah: it might be worth adding a dedicated kcsan .kunitconfig, in which case this would be helpful. It'd also need the SMP qemu config above before it's particularly useful, and 16 was a randomly-picked number of CPUs -- not sure if there's a better default.

If you're likely to use it, though, we can definitely add it in. I'm sure there'll eventually be other uses for an SMP config under kunit_tool, too.

I currently have some other frankenscript to run it, but I wouldn't mind just using kunit_tool to do so. So having real SMP support there would be very useful.

Thanks,
-- Marco

#5 lore raw
In reply to David Gow
Quoted 37 lines · David Gow
Quoted 22 lines · Marco Elver
Quoted 20 lines · David Gow
Quoted 11 lines · Marco Elver

Use the newly added suite_{init,exit} support for suite-wide init and cleanup. This avoids the unsupported method by which the test used to do suite-wide init and cleanup (avoiding issues such as missing TAP headers, and possible future conflicts).

---
This patch should go on the -kselftest/kunit branch, where this new support currently lives, including a similar change to the KFENCE test.
---

Thanks! This is working for me. I ran it as a builtin using kunit_tool under (I had to add an x86_64-smp architecture), then use:
./tools/testing/kunit/kunit.py run --arch=x86_64-smp --kconfig_add=CONFIG_KCSAN=y --kconfig_add=CONFIG_DEBUG_KERNEL=y --timeout 900 'kcsan'

Quoted 4 lines
Quoted 4 lines

To add the x86_64 smp architecture, I added a file ./tools/testing/kunit/qemu_configs/x86_64-smp.py, which was a copy of x86_64.py but with 'CONFIG_SMP=y' added to XXXX and '-smp 16' added to YYYY.

(Whoops, forgot to copy this in properly: XXXX was 'kconfig' and YYYY was 'extra_qemu_params'.)

The x86_64-smp.py file ends up looking like this:
---8<---
from ..qemu_config import QemuArchParams

QEMU_ARCH = QemuArchParams(linux_arch='x86_64',

Quoted 9 lines
                          kconfig='''
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SMP=y
                          ''',
                          qemu_arch='x86_64',
                          kernel_path='arch/x86/boot/bzImage',
                          kernel_command_line='console=ttyS0',
                          extra_qemu_params=['-smp 16'])

You'd want ['-smp', '16'].
Otherwise this config will be broken by
https://lore.kernel.org/linux-kselftest/[email protected]/

Reviewed-by on v1 1/1
#6 lore raw
In reply to Marco Elver
Quoted 7 lines · Marco Elver

Use the newly added suite_{init,exit} support for suite-wide init and cleanup. This avoids the unsupported method by which the test used to do suite-wide init and cleanup (avoiding issues such as missing TAP headers, and possible future conflicts).

Adding '-smp 16' and CONFIG_KCSAN_REPORT_ONCE_IN_MS=100 as you suggested below, I was able to get it running under kunit.py with the following results:
Testing complete. Passed: 168, Failed: 0, Crashed: 0, Skipped: 1, Errors: 0 Elapsed time: 92.642s total, 0.003s configuring, 4.592s building, 88.009s running

Nice!