Skip to contentSkip to conversation

kunit: tool: Add x86_64-smp architecture for SMP testing

PATCH In review David Gow 4 participants 21 messages May 18, 2022 – Jul 15, 2022

Subsystems (3)
lore
Share & tools
lore

David Gow

author
#0 lore raw

PATCH 1/2 kunit: tool: Add x86_64-smp architecture for SMP testing

Add a new QEMU config for kunit_tool, x86_64-smp, which provides an 8-cpu SMP setup. No other kunit_tool configurations provide an SMP setup, so this is the best bet for testing things like KCSAN, which require a multicore/multi-cpu system.

The choice of 8 CPUs is pretty arbitrary: it's enough to get tests like KCSAN to run with a nontrivial number of worker threads, while still working relatively quickly on older machines.

---

This is based off the discussion in:
https://groups.google.com/g/kasan-dev/c/A7XzC2pXRC8

---
 tools/testing/kunit/qemu_configs/x86_64-smp.py | 13 +++++++++++++
 1 file changed, 13 insertions(+)
 create mode 100644 tools/testing/kunit/qemu_configs/x86_64-smp.py
tools/testing/kunit/qemu_configs/x86_64-smp.py new file +13 -0
new file mode 100644
index 000000000000..a95623f5f8b7
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='x86_64',
+			   kconfig='''
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SMP=y
+			   ''',
+			   qemu_arch='x86_64',
+			   kernel_path='arch/x86/boot/bzImage',
+			   kernel_command_line='console=ttyS0',
+			   extra_qemu_params=['-smp', '8'])
Signature · 1 line
2.36.0.550.gb090851708-goog
author
#1 lore raw

PATCH 2/2 kcsan: test: Add a .kunitconfig to run KCSAN tests

Add a .kunitconfig file, which provides a default, working config for running the KCSAN tests. Note that it needs to run on an SMP machine, so to run under kunit_tool, the x86_64-smp qemu-based setup should be used:
./tools/testing/kunit/kunit.py run --arch=x86_64-smp --kunitconfig=kernel/kcsan

---
 kernel/kcsan/.kunitconfig | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100644 kernel/kcsan/.kunitconfig
kernel/kcsan/.kunitconfig new file +20 -0
new file mode 100644
index 000000000000..a8a815b1eb73
@@ -0,0 +1,20 @@
+# Note that the KCSAN tests need to run on an SMP setup.
+# Under kunit_tool, this can be done by using the x86_64-smp
+# qemu-based architecture:
+# ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan --arch=x86_64-smp
+
+CONFIG_KUNIT=y
+
+CONFIG_DEBUG_KERNEL=y
+
+CONFIG_KCSAN=y
+CONFIG_KCSAN_KUNIT_TEST=y
+
+# Needed for test_barrier_nothreads
+CONFIG_KCSAN_STRICT=y
+CONFIG_KCSAN_WEAK_MEMORY=y
+
+# This prevents the test from timing out on many setups. Feel free to remove
+# (or alter) this, in conjunction with setting a different test timeout with,
+# for example, the --timeout kunit_tool option.
+CONFIG_KCSAN_REPORT_ONCE_IN_MS=100
Signature · 1 line
2.36.0.550.gb090851708-goog
maintainer Reviewed-by on v1 2/2
#2 lore raw

Re: [PATCH 2/2] kcsan: test: Add a .kunitconfig to run KCSAN tests

Quoted 6 lines · David Gow

Add a .kunitconfig file, which provides a default, working config for running the KCSAN tests. Note that it needs to run on an SMP machine, so to run under kunit_tool, the x86_64-smp qemu-based setup should be used:
./tools/testing/kunit/kunit.py run --arch=x86_64-smp --kunitconfig=kernel/kcsan

Thanks for adding this.

Quoted 16 lines
---
 kernel/kcsan/.kunitconfig | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100644 kernel/kcsan/.kunitconfig
kernel/kcsan/.kunitconfig new file +5 -0
new file mode 100644
index 000000000000..a8a815b1eb73
@@ -0,0 +1,20 @@
+# Note that the KCSAN tests need to run on an SMP setup.
+# Under kunit_tool, this can be done by using the x86_64-smp
+# qemu-based architecture:
+# ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan --arch=x86_64-smp
+

kernel/kcsan/.kunitconfig line 15

Quoted 10 lines
kernel/kcsan/.kunitconfig new file +10 -0
+CONFIG_KUNIT=y
+
+CONFIG_DEBUG_KERNEL=y
+
+CONFIG_KCSAN=y
+CONFIG_KCSAN_KUNIT_TEST=y
+
+# Needed for test_barrier_nothreads
+CONFIG_KCSAN_STRICT=y
+CONFIG_KCSAN_WEAK_MEMORY=y

Note, KCSAN_STRICT implies KCSAN_WEAK_MEMORY.

Also, a bunch of the test cases' outcomes depend on KCSAN's "strictness". I think to cover the various combinations would be too complex, but we can just settle on testing KCSAN_STRICT=y.

The end result is the same, but you could drop the CONFIG_KCSAN_WEAK_MEMORY=y line, and let the latest KCSAN_STRICT defaults decide (I don't expect them to change any time soon).

If you want it to be more explicit, it's also fine leaving the CONFIG_KCSAN_WEAK_MEMORY=y line in.

Quoted 7 lines
kernel/kcsan/.kunitconfig new file +4 -0
+# This prevents the test from timing out on many setups. Feel free to remove
+# (or alter) this, in conjunction with setting a different test timeout with,
+# for example, the --timeout kunit_tool option.
+CONFIG_KCSAN_REPORT_ONCE_IN_MS=100
Signature · 1 line
2.36.0.550.gb090851708-goog
author
#3 lore raw

Re: [PATCH 2/2] kcsan: test: Add a .kunitconfig to run KCSAN tests

Quoted 42 lines · Marco Elver
Quoted 6 lines · David Gow

Add a .kunitconfig file, which provides a default, working config for running the KCSAN tests. Note that it needs to run on an SMP machine, so to run under kunit_tool, the x86_64-smp qemu-based setup should be used:
./tools/testing/kunit/kunit.py run --arch=x86_64-smp --kunitconfig=kernel/kcsan

Thanks for adding this.

Quoted 16 lines
---
 kernel/kcsan/.kunitconfig | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100644 kernel/kcsan/.kunitconfig
kernel/kcsan/.kunitconfig new file +5 -0
new file mode 100644
index 000000000000..a8a815b1eb73
@@ -0,0 +1,20 @@
+# Note that the KCSAN tests need to run on an SMP setup.
+# Under kunit_tool, this can be done by using the x86_64-smp
+# qemu-based architecture:
+# ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan --arch=x86_64-smp
+

kernel/kcsan/.kunitconfig line 15

Quoted 10 lines
kernel/kcsan/.kunitconfig new file +10 -0
+CONFIG_KUNIT=y
+
+CONFIG_DEBUG_KERNEL=y
+
+CONFIG_KCSAN=y
+CONFIG_KCSAN_KUNIT_TEST=y
+
+# Needed for test_barrier_nothreads
+CONFIG_KCSAN_STRICT=y
+CONFIG_KCSAN_WEAK_MEMORY=y

Note, KCSAN_STRICT implies KCSAN_WEAK_MEMORY.

Quoted 3 lines

Also, a bunch of the test cases' outcomes depend on KCSAN's "strictness". I think to cover the various combinations would be too complex, but we can just settle on testing KCSAN_STRICT=y.

It's definitely possible to either have multiple .kunitconfigs, each of which could have slightly different setups, e.g.:
- kernel/kcsan/.kunitconfig (defualt)
- kernel/kcsan/strict.kunitconfig (passed explicitly when desired)

Equally, if we got rid of KCSAN_STRICT in the .kunitconfig, you could override it with --kconfig_add, e.g.
- ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan --arch=x86_64-smp
- ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan --arch=x86_64-smp --kconfig_add CONFIG_KSCAN_STRICT=y

Quoted 6 lines

The end result is the same, but you could drop the CONFIG_KCSAN_WEAK_MEMORY=y line, and let the latest KCSAN_STRICT defaults decide (I don't expect them to change any time soon).

If you want it to be more explicit, it's also fine leaving the CONFIG_KCSAN_WEAK_MEMORY=y line in.

Do you have a preference here? Or to get rid of both and default to the non-strict version mentioned above?

Quoted 8 lines
Quoted 8 lines
kernel/kcsan/.kunitconfig new file +4 -1
+# This prevents the test from timing out on many setups. Feel free to remove
+# (or alter) this, in conjunction with setting a different test timeout with,
+# for example, the --timeout kunit_tool option.
+CONFIG_KCSAN_REPORT_ONCE_IN_MS=100
--

2.36.0.550.gb090851708-goog

maintainer
#4 lore raw

Re: [PATCH 2/2] kcsan: test: Add a .kunitconfig to run KCSAN tests

Quoted 64 lines · David Gow
Quoted 42 lines · Marco Elver
Quoted 6 lines · David Gow

Add a .kunitconfig file, which provides a default, working config for running the KCSAN tests. Note that it needs to run on an SMP machine, so to run under kunit_tool, the x86_64-smp qemu-based setup should be used:
./tools/testing/kunit/kunit.py run --arch=x86_64-smp --kunitconfig=kernel/kcsan

Thanks for adding this.

Quoted 16 lines
---
 kernel/kcsan/.kunitconfig | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100644 kernel/kcsan/.kunitconfig
kernel/kcsan/.kunitconfig new file +5 -0
new file mode 100644
index 000000000000..a8a815b1eb73
@@ -0,0 +1,20 @@
+# Note that the KCSAN tests need to run on an SMP setup.
+# Under kunit_tool, this can be done by using the x86_64-smp
+# qemu-based architecture:
+# ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan --arch=x86_64-smp
+

kernel/kcsan/.kunitconfig line 15

Quoted 10 lines
kernel/kcsan/.kunitconfig new file +10 -0
+CONFIG_KUNIT=y
+
+CONFIG_DEBUG_KERNEL=y
+
+CONFIG_KCSAN=y
+CONFIG_KCSAN_KUNIT_TEST=y
+
+# Needed for test_barrier_nothreads
+CONFIG_KCSAN_STRICT=y
+CONFIG_KCSAN_WEAK_MEMORY=y

Note, KCSAN_STRICT implies KCSAN_WEAK_MEMORY.

Quoted 3 lines

Also, a bunch of the test cases' outcomes depend on KCSAN's "strictness". I think to cover the various combinations would be too complex, but we can just settle on testing KCSAN_STRICT=y.

It's definitely possible to either have multiple .kunitconfigs, each of which could have slightly different setups, e.g.:
- kernel/kcsan/.kunitconfig (defualt)
- kernel/kcsan/strict.kunitconfig (passed explicitly when desired)

Equally, if we got rid of KCSAN_STRICT in the .kunitconfig, you could override it with --kconfig_add, e.g.
- ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan --arch=x86_64-smp
- ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan --arch=x86_64-smp --kconfig_add CONFIG_KSCAN_STRICT=y

Quoted 4 lines

The end result is the same, but you could drop the CONFIG_KCSAN_WEAK_MEMORY=y line, and let the latest KCSAN_STRICT defaults decide (I don't expect them to change any time soon).

Quoted 5 lines
Quoted 2 lines

If you want it to be more explicit, it's also fine leaving the CONFIG_KCSAN_WEAK_MEMORY=y line in.

Do you have a preference here? Or to get rid of both and default to the non-strict version mentioned above?

I'd keep it simple for now, and remove both lines i.e. make non-strict the default. It's easy to just run with --kconfig_add CONFIG_KCSAN_STRICT=y, along with other variations. I know that rcutoruture uses KCSAN_STRICT=y by default, so it's already getting coverage there. ;-)

Thanks,
-- Marco

#5 lore raw

Re: [PATCH 2/2] kcsan: test: Add a .kunitconfig to run KCSAN tests

Quoted 5 lines · Marco Elver

I'd keep it simple for now, and remove both lines i.e. make non-strict the default. It's easy to just run with --kconfig_add CONFIG_KCSAN_STRICT=y, along with other variations. I know that rcutoruture uses KCSAN_STRICT=y by default, so it's already getting coverage there. ;-)

David decided to drop the parent patch (the new QEMU config) now --qemu_args was merged into the kunit tree.
Did we want a standalone v2 of this patch?

Based on Marco's comments, we'd change:
* drop CONFIG_KCSAN_STRICT=y per this comment [1]
* drop CONFIG_KCSAN_WEAK_MEMORY per previous comments Then for --qemu_args changes:
* add CONFIG_SMP=y explicitly to this file
* update the comment to show to include --qemu_args="-smp 8"

Does this sound right?

[1] Note: there's also patches in kunit now so you could do --kconfig_add=CONFIG_KCSAN_STRICT=n to explicitly disable it. This wasn't possible before. Does that change what we want for the default?

Daniel

maintainer
#6 lore raw

Re: [PATCH 2/2] kcsan: test: Add a .kunitconfig to run KCSAN tests

Quoted 12 lines · Daniel Latypov
Quoted 5 lines · Marco Elver

I'd keep it simple for now, and remove both lines i.e. make non-strict the default. It's easy to just run with --kconfig_add CONFIG_KCSAN_STRICT=y, along with other variations. I know that rcutoruture uses KCSAN_STRICT=y by default, so it's already getting coverage there. ;-)

David decided to drop the parent patch (the new QEMU config) now --qemu_args was merged into the kunit tree.
Did we want a standalone v2 of this patch?

Quoted 8 lines

Based on Marco's comments, we'd change:
* drop CONFIG_KCSAN_STRICT=y per this comment [1]
* drop CONFIG_KCSAN_WEAK_MEMORY per previous comments Then for --qemu_args changes:
* add CONFIG_SMP=y explicitly to this file
* update the comment to show to include --qemu_args="-smp 8"

Does this sound right?

Yes, sounds good to me, and thanks for remembering this. I'd prefer a close-to-default config.

Quoted 3 lines

[1] Note: there's also patches in kunit now so you could do --kconfig_add=CONFIG_KCSAN_STRICT=n to explicitly disable it. This wasn't possible before. Does that change what we want for the default?

I'd just have KCSAN_STRICT=n by default, and if desired it can be added per kconfig_add just the same way.

Thanks,
-- Marco

#7 lore raw

Re: [PATCH 2/2] kcsan: test: Add a .kunitconfig to run KCSAN tests

Quoted 26 lines · Marco Elver
Quoted 12 lines · Daniel Latypov
Quoted 5 lines · Marco Elver

I'd keep it simple for now, and remove both lines i.e. make non-strict the default. It's easy to just run with --kconfig_add CONFIG_KCSAN_STRICT=y, along with other variations. I know that rcutoruture uses KCSAN_STRICT=y by default, so it's already getting coverage there. ;-)

David decided to drop the parent patch (the new QEMU config) now --qemu_args was merged into the kunit tree.
Did we want a standalone v2 of this patch?

Quoted 8 lines

Based on Marco's comments, we'd change:
* drop CONFIG_KCSAN_STRICT=y per this comment [1]
* drop CONFIG_KCSAN_WEAK_MEMORY per previous comments Then for --qemu_args changes:
* add CONFIG_SMP=y explicitly to this file
* update the comment to show to include --qemu_args="-smp 8"

Does this sound right?

Yes, sounds good to me, and thanks for remembering this. I'd prefer a close-to-default config.

Quoted 6 lines
Quoted 3 lines

[1] Note: there's also patches in kunit now so you could do --kconfig_add=CONFIG_KCSAN_STRICT=n to explicitly disable it. This wasn't possible before. Does that change what we want for the default?

I'd just have KCSAN_STRICT=n by default, and if desired it can be added per kconfig_add just the same way.

Ack.
So concretely, so then a final result like this?

$ cat kernel/kcsan/.kunitconfig
# Note that the KCSAN tests need to run on an SMP setup.
# Under kunit_tool, this can be done by using the x86_64-smp
# qemu-based architecture:
# ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan
--arch=x86_64 --qemu_args='-smp 8'

CONFIG_KUNIT=y

CONFIG_DEBUG_KERNEL=y

CONFIG_KCSAN=y
CONFIG_KCSAN_KUNIT_TEST=y

# Need some level of concurrency to test a concurrency sanitizer. CONFIG_SMP=y

# This prevents the test from timing out on many setups. Feel free to remove # (or alter) this, in conjunction with setting a different test timeout with,

# for example, the --timeout kunit_tool option.
CONFIG_KCSAN_REPORT_ONCE_IN_MS=100
#8 lore raw

Re: [PATCH 2/2] kcsan: test: Add a .kunitconfig to run KCSAN tests

Quoted 7 lines · Daniel Latypov

Ack.
So concretely, so then a final result like this?

$ cat kernel/kcsan/.kunitconfig
# Note that the KCSAN tests need to run on an SMP setup.
# Under kunit_tool, this can be done by using the x86_64-smp
# qemu-based architecture:

Oops, this bit would need to be updated to something like:

# Under kunit_tool, this can be done by using --qemu_args:

Quoted 17 lines
# ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan
--arch=x86_64 --qemu_args='-smp 8'

CONFIG_KUNIT=y

CONFIG_DEBUG_KERNEL=y

CONFIG_KCSAN=y
CONFIG_KCSAN_KUNIT_TEST=y

# Need some level of concurrency to test a concurrency sanitizer. CONFIG_SMP=y

# This prevents the test from timing out on many setups. Feel free to remove # (or alter) this, in conjunction with setting a different test timeout with,

# for example, the --timeout kunit_tool option.
CONFIG_KCSAN_REPORT_ONCE_IN_MS=100
author
#9 lore raw

Re: [PATCH 2/2] kcsan: test: Add a .kunitconfig to run KCSAN tests

Quoted 5 lines · Daniel Latypov
Quoted 3 lines · Daniel Latypov

Ack.
So concretely, so then a final result like this?

Quoted 26 lines
Quoted 4 lines
$ cat kernel/kcsan/.kunitconfig
# Note that the KCSAN tests need to run on an SMP setup.
# Under kunit_tool, this can be done by using the x86_64-smp
# qemu-based architecture:

Oops, this bit would need to be updated to something like:

# Under kunit_tool, this can be done by using --qemu_args:

Quoted 17 lines
# ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan
--arch=x86_64 --qemu_args='-smp 8'

CONFIG_KUNIT=y

CONFIG_DEBUG_KERNEL=y

CONFIG_KCSAN=y
CONFIG_KCSAN_KUNIT_TEST=y

# Need some level of concurrency to test a concurrency sanitizer. CONFIG_SMP=y

# This prevents the test from timing out on many setups. Feel free to remove # (or alter) this, in conjunction with setting a different test timeout with,

# for example, the --timeout kunit_tool option.
CONFIG_KCSAN_REPORT_ONCE_IN_MS=100

Thanks everyone. I've sent out a v2 with just this patch here:
https://lore.kernel.org/linux-kselftest/[email protected]/

I expect we'll take it in via the KUnit branch, as it's most useful with the --qemu_args option.

Cheers,
-- David

#10 lore raw

Re: [PATCH 2/2] kcsan: test: Add a .kunitconfig to run KCSAN tests

In reply to David Gow

kernel/kcsan/.kunitconfig line 4

Quoted 10 lines · David Gow
kernel/kcsan/.kunitconfig new file +4 -0
new file mode 100644
index 000000000000..a8a815b1eb73
@@ -0,0 +1,20 @@
+# Note that the KCSAN tests need to run on an SMP setup.
+# Under kunit_tool, this can be done by using the x86_64-smp
+# qemu-based architecture:
+# ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan --arch=x86_64-smp

Just noting here, if we go with --qemu_args [1], then we'd change this to
--arch=x86_64 --qemu_args='-smp 8'
and then probably add
CONFIG_SMP=y
to this file.

[1] https://lore.kernel.org/linux-kselftest/[email protected]

Quoted 6 lines
kernel/kcsan/.kunitconfig new file +6 -0
+
+CONFIG_KUNIT=y
+
+CONFIG_DEBUG_KERNEL=y
+
+CONFIG_KCSAN=y

kernel/kcsan/.kunitconfig line 20

Quoted 10 lines
kernel/kcsan/.kunitconfig new file +10 -0
+CONFIG_KCSAN_KUNIT_TEST=y
+
+# Needed for test_barrier_nothreads
+CONFIG_KCSAN_STRICT=y
+CONFIG_KCSAN_WEAK_MEMORY=y
+
+# This prevents the test from timing out on many setups. Feel free to remove
+# (or alter) this, in conjunction with setting a different test timeout with,
+# for example, the --timeout kunit_tool option.
+CONFIG_KCSAN_REPORT_ONCE_IN_MS=100

Tangent:

Ah this reminds me, unfortunately you can't use --kconfig_add to overwrite this atm.
Right now, it'll just blindly try to append and then complain that one of the two copies of the option is missing.

That might be a feature to look into.
Or at least, we can maybe give a better error message.

E.g. with the default kunitconfig, the error currently looks like # Try to overwrite CONFIG_KUNIT_ALL_TESTS=y

$ ./tools/testing/kunit/kunit.py config --kconfig_add=CONFIG_KUNIT_ALL_TESTS=m
...
ERROR:root:Not all Kconfig options selected in kunitconfig were in the
generated .config.
This is probably due to unsatisfied dependencies.
Missing: CONFIG_KUNIT_ALL_TESTS=m
Acked-by on v1 2/2
#11 lore raw

Re: [PATCH 2/2] kcsan: test: Add a .kunitconfig to run KCSAN tests

In reply to David Gow
Quoted 7 lines · David Gow

Add a .kunitconfig file, which provides a default, working config for running the KCSAN tests. Note that it needs to run on an SMP machine, so to run under kunit_tool, the x86_64-smp qemu-based setup should be used:
./tools/testing/kunit/kunit.py run --arch=x86_64-smp --kunitconfig=kernel/kcsan

Ack, but I think Marco settled on removing CONFIG_KCSAN_STRICT=y and CONFIG_KCSAN_WEAK_MEMORY=y.

maintainer Acked-by on v1 1/2
#12 lore raw
In reply to David Gow
Quoted 10 lines · David Gow' via KUnit Development

Add a new QEMU config for kunit_tool, x86_64-smp, which provides an 8-cpu SMP setup. No other kunit_tool configurations provide an SMP setup, so this is the best bet for testing things like KCSAN, which require a multicore/multi-cpu system.

The choice of 8 CPUs is pretty arbitrary: it's enough to get tests like KCSAN to run with a nontrivial number of worker threads, while still working relatively quickly on older machines.

Quoted 36 lines

---

This is based off the discussion in:
https://groups.google.com/g/kasan-dev/c/A7XzC2pXRC8

---
 tools/testing/kunit/qemu_configs/x86_64-smp.py | 13 +++++++++++++
 1 file changed, 13 insertions(+)
 create mode 100644 tools/testing/kunit/qemu_configs/x86_64-smp.py
tools/testing/kunit/qemu_configs/x86_64-smp.py new file +13 -0
new file mode 100644
index 000000000000..a95623f5f8b7
@@ -0,0 +1,13 @@
+# SPDX-License-Identifier: GPL-2.0
+from ..qemu_config import QemuArchParams
+
+QEMU_ARCH = QemuArchParams(linux_arch='x86_64',
+			   kconfig='''
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_SMP=y
+			   ''',
+			   qemu_arch='x86_64',
+			   kernel_path='arch/x86/boot/bzImage',
+			   kernel_command_line='console=ttyS0',
+			   extra_qemu_params=['-smp', '8'])
Signature · 6 lines
2.36.0.550.gb090851708-goog

-- 
You received this message because you are subscribed to the Google Groups "KUnit Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/kunit-dev/20220518073232.526443-1-davidgow%40google.com.
#13 lore raw
In reply to David Gow
Quoted 10 lines · David Gow' via KUnit Development

Add a new QEMU config for kunit_tool, x86_64-smp, which provides an 8-cpu SMP setup. No other kunit_tool configurations provide an SMP setup, so this is the best bet for testing things like KCSAN, which require a multicore/multi-cpu system.

The choice of 8 CPUs is pretty arbitrary: it's enough to get tests like KCSAN to run with a nontrivial number of worker threads, while still working relatively quickly on older machines.

Since it's arbitrary, I somewhat prefer the idea of leaving up entirely to the caller
i.e.

$ kunit.py run --kconfig_add=CONFIG_SMP=y --qemu_args '-smp 8'

We could add CONFIG_SMP=y to the default qemu_configs/*.py and do

$ kunit.py run --qemu_args '-smp 8'
but I'd prefer the first, even if it is more verbose.

Marco, does this seem reasonable from your perspective?

I think that a new --qemu_args would be generically useful for adhoc use and light enough that people won't need to add qemu_configs much. E.g. I can see people wanting multiple NUMA nodes, a specific -cpu, and so on.

maintainer
#14 lore raw
Quoted 17 lines · Daniel Latypov
Quoted 10 lines · David Gow' via KUnit Development

Add a new QEMU config for kunit_tool, x86_64-smp, which provides an 8-cpu SMP setup. No other kunit_tool configurations provide an SMP setup, so this is the best bet for testing things like KCSAN, which require a multicore/multi-cpu system.

The choice of 8 CPUs is pretty arbitrary: it's enough to get tests like KCSAN to run with a nontrivial number of worker threads, while still working relatively quickly on older machines.

Since it's arbitrary, I somewhat prefer the idea of leaving up entirely to the caller
i.e.

Quoted 7 lines
$ kunit.py run --kconfig_add=CONFIG_SMP=y --qemu_args '-smp 8'

We could add CONFIG_SMP=y to the default qemu_configs/*.py and do

$ kunit.py run --qemu_args '-smp 8'
but I'd prefer the first, even if it is more verbose.

Marco, does this seem reasonable from your perspective?

Either way works. But I wouldn't mind a sane default though, where that default can be overridden with custom number of CPUs.

Quoted 3 lines

I think that a new --qemu_args would be generically useful for adhoc use and light enough that people won't need to add qemu_configs much. E.g. I can see people wanting multiple NUMA nodes, a specific -cpu, and so on.

#15 lore raw
Quoted 21 lines · Marco Elver
Quoted 19 lines · Daniel Latypov
Quoted 10 lines · David Gow' via KUnit Development

Add a new QEMU config for kunit_tool, x86_64-smp, which provides an 8-cpu SMP setup. No other kunit_tool configurations provide an SMP setup, so this is the best bet for testing things like KCSAN, which require a multicore/multi-cpu system.

The choice of 8 CPUs is pretty arbitrary: it's enough to get tests like KCSAN to run with a nontrivial number of worker threads, while still working relatively quickly on older machines.

Since it's arbitrary, I somewhat prefer the idea of leaving up entirely to the caller
i.e.

$ kunit.py run --kconfig_add=CONFIG_SMP=y --qemu_args '-smp 8'
Quoted 9 lines
Quoted 5 lines

We could add CONFIG_SMP=y to the default qemu_configs/*.py and do

$ kunit.py run --qemu_args '-smp 8'
but I'd prefer the first, even if it is more verbose.

Marco, does this seem reasonable from your perspective?

Either way works. But I wouldn't mind a sane default though, where that default can be overridden with custom number of CPUs.

Ack.
Let me clean up what I have for --qemu_args and send it out for discussion.

One downside I see to adding more qemu_configs is that --arch now becomes more kunit-specific.
Before, a user could assume "oh, it's just what I pass in to make ARCH=...". This new "--arch=x86_64-smp" violates that. I don't personally see it being that confusing, but I still worry.

author
#17 lore raw
In reply to Marco Elver
Quoted 21 lines · Marco Elver
Quoted 19 lines · Daniel Latypov
Quoted 10 lines · David Gow' via KUnit Development

Add a new QEMU config for kunit_tool, x86_64-smp, which provides an 8-cpu SMP setup. No other kunit_tool configurations provide an SMP setup, so this is the best bet for testing things like KCSAN, which require a multicore/multi-cpu system.

The choice of 8 CPUs is pretty arbitrary: it's enough to get tests like KCSAN to run with a nontrivial number of worker threads, while still working relatively quickly on older machines.

Since it's arbitrary, I somewhat prefer the idea of leaving up entirely to the caller
i.e.

$ kunit.py run --kconfig_add=CONFIG_SMP=y --qemu_args '-smp 8'
Quoted 9 lines
Quoted 5 lines

We could add CONFIG_SMP=y to the default qemu_configs/*.py and do

$ kunit.py run --qemu_args '-smp 8'
but I'd prefer the first, even if it is more verbose.

Marco, does this seem reasonable from your perspective?

Either way works. But I wouldn't mind a sane default though, where that default can be overridden with custom number of CPUs.

I tend to agree that having both would be nice: I think there are enough useful "machine configs" that trying to maintain, e.g, a 1:1 mapping with kernel architectures is going to leave a bunch of things on the table, particularly as we add more tests for, e.g., drivers and specific CPU models.

The problem, of course, is that the --kconfig_add flags don't allow us to override anything explicitly stated in either the kunitconfig or qemu_config (and I imagine there could be problems with --qemu_config, too).

Quoted 3 lines
Quoted 3 lines

I think that a new --qemu_args would be generically useful for adhoc use and light enough that people won't need to add qemu_configs much. E.g. I can see people wanting multiple NUMA nodes, a specific -cpu, and so on.

#18 lore raw
Quoted 6 lines · David Gow

I tend to agree that having both would be nice: I think there are enough useful "machine configs" that trying to maintain, e.g, a 1:1 mapping with kernel architectures is going to leave a bunch of things on the table, particularly as we add more tests for, e.g., drivers and specific CPU models.

I agree that we don't necessarily need to maintain a 1:1 mapping. But I feel like we should have a pretty convincing reason for doing so, e.g. support for a CPU that requires we add in a bunch of kconfigs.

This particular one feels simple enough to me.
Given we already have to put specific instructions in the kcsan/.kunitconfig, I don't know if there's much of a difference in cost between these two commands

$ ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan
--arch=x86_64-smp
$ ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan
--arch=x86_64 --kconfig_add CONFIG_SMP=y --qemu_args "-smp 8"

I've generally learned to prefer more explicit commands like the second, even if they're quite a bit longer.
But I have the following biases
* I use FZF heavily, so I don't re-type long commands much
* I'm the person who proposed --kconfig_add and --qemu_args, so of course I'd think the longer form is easy to understand. so I'm not in a position to object to this change.

Changing topics:
Users can overwrite the '-smp 8' here via --qemu_args [1], so I'm much less worried about hard-coding any specific value in this file anymore.
And given that, I think a more "natural" value for this file would be "-smp 2". I think anything that needs more than that should explicitly should --qemu_args.

Thoughts?

[1] tested with --qemu_args='-smp 4' --qemu_args='-smp 8' and I see the following in the test.log
smpboot: Allowing 8 CPUs, 0 hotplug CPUs
so QEMU respects the last value passed in, as expected.

Quoted 5 lines

The problem, of course, is that the --kconfig_add flags don't allow us to override anything explicitly stated in either the kunitconfig or qemu_config (and I imagine there could be problems with --qemu_config, too).

This patch would fix that.
https://lore.kernel.org/linux-kselftest/[email protected]

It introduces an overwriting priority of
* --kconfig_add
* kunitconfig / --kunitconfig
* qemu_config

#19 lore raw
Quoted 9 lines · Daniel Latypov
Quoted 6 lines · David Gow

I tend to agree that having both would be nice: I think there are enough useful "machine configs" that trying to maintain, e.g, a 1:1 mapping with kernel architectures is going to leave a bunch of things on the table, particularly as we add more tests for, e.g., drivers and specific CPU models.

Quoted 4 lines

I agree that we don't necessarily need to maintain a 1:1 mapping. But I feel like we should have a pretty convincing reason for doing so, e.g. support for a CPU that requires we add in a bunch of kconfigs.

Agreed. That being said, if we have a good convention for archs that are not in arch/, then it should be OK. The biggest thing is that all archs passed into ARCH=, if supported, should have a default with the same value for kunittool; as long as that is the case, I don't think anyone will get confused.

Quoted 9 lines

This particular one feels simple enough to me.
Given we already have to put specific instructions in the kcsan/.kunitconfig, I don't know if there's much of a difference in cost between these two commands

$ ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan
--arch=x86_64-smp
$ ./tools/testing/kunit/kunit.py run --kunitconfig=kernel/kcsan
--arch=x86_64 --kconfig_add CONFIG_SMP=y --qemu_args "-smp 8"

Also agree.

Quoted 2 lines

I've generally learned to prefer more explicit commands like the second, even if they're quite a bit longer.

I agree, but I think I learned this from you :-)

Quoted 2 lines

But I have the following biases
* I use FZF heavily, so I don't re-type long commands much

Same.

Quoted 3 lines

* I'm the person who proposed --kconfig_add and --qemu_args, so of course I'd think the longer form is easy to understand. so I'm not in a position to object to this change.

Yeah, I think I am a bit biased on this too, but I don't terribly care one way or the other.

Quoted 8 lines

Changing topics:
Users can overwrite the '-smp 8' here via --qemu_args [1], so I'm much less worried about hard-coding any specific value in this file anymore.
And given that, I think a more "natural" value for this file would be "-smp 2". I think anything that needs more than that should explicitly should --qemu_args.

Thoughts?

If we have time, we could bring this topic up at LPC?

Quoted 18 lines

[1] tested with --qemu_args='-smp 4' --qemu_args='-smp 8' and I see the following in the test.log
smpboot: Allowing 8 CPUs, 0 hotplug CPUs
so QEMU respects the last value passed in, as expected.

Quoted 5 lines

The problem, of course, is that the --kconfig_add flags don't allow us to override anything explicitly stated in either the kunitconfig or qemu_config (and I imagine there could be problems with --qemu_config, too).

This patch would fix that.
https://lore.kernel.org/linux-kselftest/[email protected]

It introduces an overwriting priority of
* --kconfig_add
* kunitconfig / --kunitconfig
* qemu_config

Reviewed-by on v1 1/2
#20 lore raw
In reply to David Gow
Quoted 11 lines · David Gow' via KUnit Development

Add a new QEMU config for kunit_tool, x86_64-smp, which provides an 8-cpu SMP setup. No other kunit_tool configurations provide an SMP setup, so this is the best bet for testing things like KCSAN, which require a multicore/multi-cpu system.

The choice of 8 CPUs is pretty arbitrary: it's enough to get tests like KCSAN to run with a nontrivial number of worker threads, while still working relatively quickly on older machines.

I know there is some discussion on this patch, but I think this patch is good as implemented; we could always delete this config if we change our policies later.