genirq: Add kunit tests for depth counts
PATCH v4 1/1 genirq: Add kunit tests for depth counts
There have been a few bugs and/or misunderstandings about the reference counting, and startup/shutdown behaviors in the IRQ core and related CPU hotplug code. These 4 test cases try to capture a few interesting cases.
* irq_disable_depth_test: basic request/disable/enable sequence
* irq_free_disabled_test: request/disable/free/re-request sequence - this catches errors on previous revisions of my work
* irq_cpuhotplug_test: exercises managed-affinity IRQ + CPU hotplug. This captures a problematic test case that I've fixed. This test requires CONFIG_SMP and a hotpluggable CPU#1.
* irq_shutdown_depth_test: exercises similar behavior from irq_cpuhotplug_test, but directly using irq_*() APIs instead of going through CPU hotplug. This still requires CONFIG_SMP, because managed-affinity is stubbed out (and not all APIs are even present) without it.
Note the use of 'imply SMP': ARCH=um doesn't support SMP, and kunit is often exercised there. Thus, 'imply' will force SMP on where possible (such as ARCH=x86_64), but leave it off where it's not.
Behavior on various SMP and ARCH configurations:
$ tools/testing/kunit/kunit.py run 'irq_test_cases*' --arch x86_64 --qemu_args '-smp 2' [...] [11:12:24] Testing complete. Ran 4 tests: passed: 4
$ tools/testing/kunit/kunit.py run 'irq_test_cases*' --arch x86_64 [...] [11:13:27] [SKIPPED] irq_cpuhotplug_test [11:13:27] ================= [PASSED] irq_test_cases ================== [11:13:27] ============================================================ [11:13:27] Testing complete. Ran 4 tests: passed: 3, skipped: 1
# default: ARCH=um
$ tools/testing/kunit/kunit.py run 'irq_test_cases*' [11:14:26] [SKIPPED] irq_shutdown_depth_test [11:14:26] [SKIPPED] irq_cpuhotplug_test [11:14:26] ================= [PASSED] irq_test_cases ================== [11:14:26] ============================================================ [11:14:26] Testing complete. Ran 4 tests: passed: 2, skipped: 2
Without commit 788019eb559f ("genirq: Retain disable depth for managed interrupts across CPU hotplug"), we fail as follows:
[11:18:55] =============== irq_test_cases (4 subtests) ================ [11:18:55] [PASSED] irq_disable_depth_test
[11:18:55] [PASSED] irq_free_disabled_test
[11:18:55] # irq_shutdown_depth_test: EXPECTATION FAILED at kernel/irq/irq_test.c:147
[11:18:55] Expected desc->depth == 1, but
[11:18:55] desc->depth == 0 (0x0)
[11:18:55] ------------[ cut here ]------------
[11:18:55] Unbalanced enable for IRQ 26
[11:18:55] WARNING: CPU: 1 PID: 36 at kernel/irq/manage.c:792 __enable_irq+0x36/0x60
...
[11:18:55] [FAILED] irq_shutdown_depth_test
[11:18:55] #1
[11:18:55] # irq_cpuhotplug_test: EXPECTATION FAILED at kernel/irq/irq_test.c:202 [11:18:55] Expected irqd_is_activated(data) to be false, but is true [11:18:55] # irq_cpuhotplug_test: EXPECTATION FAILED at kernel/irq/irq_test.c:203 [11:18:55] Expected irqd_is_started(data) to be false, but is true [11:18:55] # irq_cpuhotplug_test: EXPECTATION FAILED at kernel/irq/irq_test.c:204 [11:18:55] Expected desc->depth == 1, but
[11:18:55] desc->depth == 0 (0x0)
[11:18:55] ------------[ cut here ]------------
[11:18:55] Unbalanced enable for IRQ 27
[11:18:55] WARNING: CPU: 0 PID: 38 at kernel/irq/manage.c:792 __enable_irq+0x36/0x60
...
[11:18:55] [FAILED] irq_cpuhotplug_test
[11:18:55] # module: irq_test
[11:18:55] # irq_test_cases: pass:2 fail:2 skip:0 total:4
[11:18:55] # Totals: pass:2 fail:2 skip:0 total:4
[11:18:55] ================= [FAILED] irq_test_cases ================== [11:18:55] ============================================================ [11:18:55] Testing complete. Ran 4 tests: passed: 2, failed: 2
---
Changes in v4:
* depend on KUNIT=y: CONFIG_IRQ_KUNIT_TEST=y is incompatible with CONFIG_KUNIT=m -- same problem as in commit 35c57fc3f8ea ("kunit:
building kunit as a module breaks allmodconfig")
Changes in v3:
* send as standalone patch, since dependent patch was merged
* make IRQ_KUNIT_TEST bool; we depend on a few internal functions that are not exported to modules. Reported:
https://lore.kernel.org/oe-kbuild-all/[email protected]/
Changes in v2:
* add request_irq()/disable_irq()/free_irq()/request_irq() test sequence
* clean up more resources in tests
* move tests to patch 2 (i.e., after bugs are fixed and tests pass)
* adapt to irq_startup_managed() (new API)
kernel/irq/Kconfig | 11 ++ kernel/irq/Makefile | 1 + kernel/irq/irq_test.c | 229 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 241 insertions(+) create mode 100644 kernel/irq/irq_test.c
kernel/irq/Kconfig +11 -0
@@ -144,6 +144,17 @@ config GENERIC_IRQ_DEBUGFS config GENERIC_IRQ_KEXEC_CLEAR_VM_FORWARD bool +config IRQ_KUNIT_TEST + bool "KUnit tests for IRQ management APIs" if !KUNIT_ALL_TESTS + depends on KUNIT=y + default KUNIT_ALL_TESTS + imply SMP + help + This option enables KUnit tests for the IRQ subsystem API. These are + only for development and testing, not for regular kernel use cases. + + If unsure, say N. + endmenu config GENERIC_IRQ_MULTI_HANDLER
kernel/irq/Makefile +1 -0
@@ -19,3 +19,4 @@ obj-$(CONFIG_GENERIC_IRQ_IPI_MUX) += ipi-mux.o obj-$(CONFIG_SMP) += affinity.o obj-$(CONFIG_GENERIC_IRQ_DEBUGFS) += debugfs.o obj-$(CONFIG_GENERIC_IRQ_MATRIX_ALLOCATOR) += matrix.o +obj-$(CONFIG_IRQ_KUNIT_TEST) += irq_test.o
kernel/irq/irq_test.c new file +229 -0
229 lines not on this page · raw message
Signature · 1 line
2.49.0.1151.ga128411c76-goog
The following commit has been merged into the irq/core branch of tip:
The following commit has been merged into the irq/core branch of tip:
Commit-ID: 66067c3c8a1ee097e9c30e8bbd643b12ba54a6b0 Gitweb: https://git.kernel.org/tip/66067c3c8a1ee097e9c30e8bbd643b12ba54a6b0 Author: Brian Norris <[email protected]> AuthorDate: Thu, 22 May 2025 14:08:01 -07:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Fri, 13 Jun 2025 15:24:44 +02:00
genirq: Add kunit tests for depth counts
There have been a few bugs and/or misunderstandings about the reference counting, and startup/shutdown behaviors in the IRQ core and related CPU hotplug code. These 4 test cases try to capture a few interesting cases.
* irq_disable_depth_test: basic request/disable/enable sequence
* irq_free_disabled_test: request/disable/free/re-request sequence - this catches errors on previous revisions of my work
* irq_cpuhotplug_test: exercises managed-affinity IRQ + CPU hotplug. This captures a problematic test case which was fixed recently. This test requires CONFIG_SMP and a hotpluggable CPU#1.
* irq_shutdown_depth_test: exercises similar behavior from irq_cpuhotplug_test, but directly using irq_*() APIs instead of going through CPU hotplug. This still requires CONFIG_SMP, because managed-affinity is stubbed out (and not all APIs are even present) without it.
Note the use of 'imply SMP': ARCH=um doesn't support SMP, and kunit is often exercised there. Thus, 'imply' will force SMP on where possible (such as ARCH=x86_64), but leave it off where it's not.
Behavior on various SMP and ARCH configurations:
$ tools/testing/kunit/kunit.py run 'irq_test_cases*' --arch x86_64 --qemu_args '-smp 2' [...] [11:12:24] Testing complete. Ran 4 tests: passed: 4
$ tools/testing/kunit/kunit.py run 'irq_test_cases*' --arch x86_64 [...] [11:13:27] [SKIPPED] irq_cpuhotplug_test [11:13:27] ================= [PASSED] irq_test_cases ================== [11:13:27] ============================================================ [11:13:27] Testing complete. Ran 4 tests: passed: 3, skipped: 1
# default: ARCH=um
$ tools/testing/kunit/kunit.py run 'irq_test_cases*' [11:14:26] [SKIPPED] irq_shutdown_depth_test [11:14:26] [SKIPPED] irq_cpuhotplug_test [11:14:26] ================= [PASSED] irq_test_cases ================== [11:14:26] ============================================================ [11:14:26] Testing complete. Ran 4 tests: passed: 2, skipped: 2
Without commit 788019eb559f ("genirq: Retain disable depth for managed interrupts across CPU hotplug"), this fails as follows:
[11:18:55] =============== irq_test_cases (4 subtests) ================ [11:18:55] [PASSED] irq_disable_depth_test
[11:18:55] [PASSED] irq_free_disabled_test
[11:18:55] # irq_shutdown_depth_test: EXPECTATION FAILED at kernel/irq/irq_test.c:147
[11:18:55] Expected desc->depth == 1, but
[11:18:55] desc->depth == 0 (0x0)
[11:18:55] ------------[ cut here ]------------
[11:18:55] Unbalanced enable for IRQ 26
[11:18:55] WARNING: CPU: 1 PID: 36 at kernel/irq/manage.c:792 __enable_irq+0x36/0x60
...
[11:18:55] [FAILED] irq_shutdown_depth_test
[11:18:55] #1
[11:18:55] # irq_cpuhotplug_test: EXPECTATION FAILED at kernel/irq/irq_test.c:202 [11:18:55] Expected irqd_is_activated(data) to be false, but is true [11:18:55] # irq_cpuhotplug_test: EXPECTATION FAILED at kernel/irq/irq_test.c:203 [11:18:55] Expected irqd_is_started(data) to be false, but is true [11:18:55] # irq_cpuhotplug_test: EXPECTATION FAILED at kernel/irq/irq_test.c:204 [11:18:55] Expected desc->depth == 1, but
[11:18:55] desc->depth == 0 (0x0)
[11:18:55] ------------[ cut here ]------------
[11:18:55] Unbalanced enable for IRQ 27
[11:18:55] WARNING: CPU: 0 PID: 38 at kernel/irq/manage.c:792 __enable_irq+0x36/0x60
...
[11:18:55] [FAILED] irq_cpuhotplug_test
[11:18:55] # module: irq_test
[11:18:55] # irq_test_cases: pass:2 fail:2 skip:0 total:4
[11:18:55] # Totals: pass:2 fail:2 skip:0 total:4
[11:18:55] ================= [FAILED] irq_test_cases ================== [11:18:55] ============================================================ [11:18:55] Testing complete. Ran 4 tests: passed: 2, failed: 2
--- kernel/irq/Kconfig | 11 ++- kernel/irq/Makefile | 1 +- kernel/irq/irq_test.c | 229 +++++++++++++++++++++++++++++++++++++++++- 3 files changed, 241 insertions(+) create mode 100644 kernel/irq/irq_test.c
kernel/irq/Kconfig +11 -0
@@ -144,6 +144,17 @@ config GENERIC_IRQ_DEBUGFS config GENERIC_IRQ_KEXEC_CLEAR_VM_FORWARD bool +config IRQ_KUNIT_TEST + bool "KUnit tests for IRQ management APIs" if !KUNIT_ALL_TESTS + depends on KUNIT=y + default KUNIT_ALL_TESTS + imply SMP + help + This option enables KUnit tests for the IRQ subsystem API. These are + only for development and testing, not for regular kernel use cases. + + If unsure, say N. + endmenu config GENERIC_IRQ_MULTI_HANDLER
kernel/irq/Makefile +1 -0
@@ -19,3 +19,4 @@ obj-$(CONFIG_GENERIC_IRQ_IPI_MUX) += ipi-mux.o obj-$(CONFIG_SMP) += affinity.o obj-$(CONFIG_GENERIC_IRQ_DEBUGFS) += debugfs.o obj-$(CONFIG_GENERIC_IRQ_MATRIX_ALLOCATOR) += matrix.o +obj-$(CONFIG_IRQ_KUNIT_TEST) += irq_test.o
kernel/irq/irq_test.c new file +229 -0
229 lines not on this page · raw message
Hi Brian,
Quoted 13 lines · Brian Norris
There have been a few bugs and/or misunderstandings about the reference counting, and startup/shutdown behaviors in the IRQ core and related CPU hotplug code. These 4 test cases try to capture a few interesting cases.
*
irq_disable_depth_test: basic request/disable/enable sequence*
irq_free_disabled_test: request/disable/free/re-request sequence - this catches errors on previous revisions of my work*
irq_cpuhotplug_test: exercises managed-affinity IRQ + CPU hotplug. This captures a problematic test case that I've fixed. This test requiresCONFIG_SMPand a hotpluggable CPU#1.
Quoted 6 lines
*
irq_shutdown_depth_test: exercises similar behavior fromirq_cpuhotplug_test, but directly using irq_*() APIs instead of going through CPU hotplug. This still requiresCONFIG_SMP, because managed-affinity is stubbed out (and not all APIs are even present) without it.
This test triggers warning tracebacks for me.
[ 5.766715] ok 2 irq_free_disabled_test
[ 5.769030]
[ 5.769106] ========================================================
[ 5.769159] WARNING: possible irq lock inversion dependency detected
[ 5.769355] 6.16.0-11743-g6bcdbd62bd56 #1 Tainted: G N
[ 5.769413] --------------------------------------------------------
[ 5.769465] kunit_try_catch/122 just changed the state of lock:
[ 5.769532] ffffffffb81ace18 (irq_resend_lock){+...}-{2:2}, at: clear_irq_resend+0x14/0x70
[ 5.769899] but this lock was taken by another, HARDIRQ-safe lock in the past:
[ 5.769967] (&irq_desc_lock_class){-.-.}-{2:2}
[ 5.769989]
[ 5.769989]
[ 5.769989] and interrupts could create inverse lock ordering between them.
...
[ 5.776956] ret_from_fork_asm+0x1a/0x30
[ 5.776983] </TASK>
[ 5.778916] # irq_shutdown_depth_test: pass:1 fail:0 skip:0 total:1
[ 5.778953] ok 3 irq_shutdown_depth_testIs this on purpose ?
Thanks,
Guenter
PATCH 1/1 genirq/test: Resolve irq lock inversion warnings
irq_shutdown_and_deactivate() is normally called with the descriptor lock held, and interrupts disabled. Nested a few levels down, it grabs the global irq_resend_lock. Lockdep rightfully complains [1].
Grab the descriptor lock, and disable interrupts, to resolve the complaint.
Tested with:
tools/testing/kunit/kunit.py run 'irq_test_cases*' \
--arch x86_64 --qemu_args '-smp 2' \
--kconfig_add CONFIG_DEBUG_KERNEL=y \
--kconfig_add CONFIG_PROVE_LOCKING=y \
--raw_output=all[1]
========================================================
WARNING: possible irq lock inversion dependency detected
6.16.0-11743-g6bcdbd62bd56 #2 Tainted: G N
--------------------------------------------------------
kunit_try_catch/40 just changed the state of lock:ffffffff898b1538 (irq_resend_lock){+...}-{2:2}, at: clear_irq_resend+0x14/0x70 but this lock was taken by another, HARDIRQ-safe lock in the past:
(&irq_desc_lock_class){-.-.}-{2:2}
and interrupts could create inverse lock ordering between them.
other info that might help us debug this:
Possible interrupt unsafe locking scenario:
CPU0 CPU1
---- ----
lock(irq_resend_lock);
local_irq_disable();
lock(&irq_desc_lock_class);
lock(irq_resend_lock);
<Interrupt>
lock(&irq_desc_lock_class);[...]
... key at: [<ffffffff898b1538>] irq_resend_lock+0x18/0x60
... acquired at:
__lock_acquire+0x82b/0x2620 lock_acquire+0xc7/0x2c0 _raw_spin_lock+0x2b/0x40 clear_irq_resend+0x14/0x70 irq_shutdown_and_deactivate+0x29/0x80 irq_shutdown_depth_test+0x1ce/0x600 kunit_try_run_case+0x90/0x120 kunit_generic_run_threadfn_adapter+0x1c/0x40 kthread+0xf3/0x200 ret_from_fork+0x140/0x1b0 ret_from_fork_asm+0x1a/0x30
[ 5.766715] ok 2 irq_free_disabled_test
[ 5.769030]
[ 5.769106] ========================================================
[ 5.769159] WARNING: possible irq lock inversion dependency detected
[ 5.769355] 6.16.0-11743-g6bcdbd62bd56 #1 Tainted: G N
[ 5.769413] --------------------------------------------------------
[ 5.769465] kunit_try_catch/122 just changed the state of lock:
[ 5.769532] ffffffffb81ace18 (irq_resend_lock){+...}-{2:2}, at: clear_irq_resend+0x14/0x70
[ 5.769899] but this lock was taken by another, HARDIRQ-safe lock in the past:
[ 5.769967] (&irq_desc_lock_class){-.-.}-{2:2}
[ 5.769989]
[ 5.769989]
[ 5.769989] and interrupts could create inverse lock ordering between them.
...
[ 5.776956] ret_from_fork_asm+0x1a/0x30
[ 5.776983] </TASK>
[ 5.778916] # irq_shutdown_depth_test: pass:1 fail:0 skip:0 total:1
[ 5.778953] ok 3 irq_shutdown_depth_test---
Quoted 10 lines · Guenter Roeck
Hi Brian,
Quoted 6 lines · Brian Norris
*
irq_shutdown_depth_test: exercises similar behavior fromirq_cpuhotplug_test, but directly using irq_*() APIs instead of going through CPU hotplug. This still requiresCONFIG_SMP, because managed-affinity is stubbed out (and not all APIs are even present) without it.This test triggers warning tracebacks for me.
[...]
Quoted 1 line
Is this on purpose ?
No. I think it's an artifact of trying to imitate CPU hotplug, but doing so insufficiently. I believe the surrounding patch fixes things.
Let me know if it helps. Thanks for the report.
kernel/irq/irq_test.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/irq/irq_test.c +3 -1
@@ -1,5 +1,6 @@// SPDX-License-Identifier: LGPL-2.1+ +#include <linux/cleanup.h> #include <linux/cpu.h> #include <linux/cpumask.h> #include <linux/interrupt.h>
@@ -134,7 +135,8 @@ static void irq_shutdown_depth_test(struct kunit *test)disable_irq (virq);KUNIT_EXPECT_EQ (test, desc->depth,1 ); -irq_shutdown_and_deactivate (desc); + scoped_guard(raw_spinlock_irqsave, &desc->lock) +irq_shutdown_and_deactivate (desc);KUNIT_EXPECT_FALSE (test,irqd_is_activated (data));KUNIT_EXPECT_FALSE (test,irqd_is_started (data));
Signature · 1 line
2.50.1.565.gc32cd1483b-goog
Re: [PATCH] genirq/test: Resolve irq lock inversion warnings
Quoted 74 lines · Brian Norris
irq_shutdown_and_deactivate()is normally called with the descriptor lock held, and interrupts disabled. Nested a few levels down, it grabs the globalirq_resend_lock. Lockdep rightfully complains [1].Grab the descriptor lock, and disable interrupts, to resolve the complaint.
Tested with:
tools/testing/kunit/kunit.py run 'irq_test_cases*' \ --arch x86_64 --qemu_args '-smp 2' \ --kconfig_add CONFIG_DEBUG_KERNEL=y \ --kconfig_add CONFIG_PROVE_LOCKING=y \ --raw_output=all[1]
========================================================
WARNING: possible irq lock inversion dependency detected
6.16.0-11743-g6bcdbd62bd56 #2 Tainted: G N
--------------------------------------------------------
kunit_try_catch/40 just changed the state of lock:ffffffff898b1538(irq_resend_lock){+...}-{2:2}, at:clear_irq_resend+0x14/0x70but this lock was taken by another, HARDIRQ-safe lock in the past:
(&irq_desc_lock_class){-.-.}-{2:2}and interrupts could create inverse lock ordering between them.
other info that might help us debug this:
Possible interrupt unsafe locking scenario:CPU0 CPU1 ---- ---- lock(irq_resend_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(irq_resend_lock); <Interrupt> lock(&irq_desc_lock_class);[...]
... key at: [<
ffffffff898b1538>]irq_resend_lock+0x18/0x60
... acquired at:__lock_acquire+0x82b/0x2620 lock_acquire+0xc7/0x2c0 _raw_spin_lock+0x2b/0x40 clear_irq_resend+0x14/0x70 irq_shutdown_and_deactivate+0x29/0x80 irq_shutdown_depth_test+0x1ce/0x600 kunit_try_run_case+0x90/0x120 kunit_generic_run_threadfn_adapter+0x1c/0x40 kthread+0xf3/0x200 ret_from_fork+0x140/0x1b0 ret_from_fork_asm+0x1a/0x30[ 5.766715] ok 2 irq_free_disabled_test [ 5.769030] [ 5.769106] ======================================================== [ 5.769159] WARNING: possible irq lock inversion dependency detected [ 5.769355] 6.16.0-11743-g6bcdbd62bd56 #1 Tainted: G N [ 5.769413] -------------------------------------------------------- [ 5.769465] kunit_try_catch/122 just changed the state of lock: [ 5.769532] ffffffffb81ace18 (irq_resend_lock){+...}-{2:2}, at: clear_irq_resend+0x14/0x70 [ 5.769899] but this lock was taken by another, HARDIRQ-safe lock in the past: [ 5.769967] (&irq_desc_lock_class){-.-.}-{2:2} [ 5.769989] [ 5.769989] [ 5.769989] and interrupts could create inverse lock ordering between them. ... [ 5.776956] ret_from_fork_asm+0x1a/0x30 [ 5.776983] </TASK> [ 5.778916] # irq_shutdown_depth_test: pass:1 fail:0 skip:0 total:1 [ 5.778953] ok 3 irq_shutdown_depth_test
Quoted 4 lines
Fixes 66067c3c8a1e ("genirq: Add kunit tests for depth counts")Reported-by Guenter Roeck <[email protected]>Signed-off-by Brian Norris <[email protected]>
Thanks for the quick turnaround!
Guenter
[tip: irq/urgent] genirq/test: Resolve irq lock inversion warnings
In reply to Brian NorrisThe following commit has been merged into the irq/urgent branch of tip:
The following commit has been merged into the irq/urgent branch of tip:
Commit-ID: 5b65258229117995eb6c4bd74995e15fb5f2cfe3 Gitweb: https://git.kernel.org/tip/5b65258229117995eb6c4bd74995e15fb5f2cfe3 Author: Brian Norris <[email protected]> AuthorDate: Tue, 05 Aug 2025 11:32:20 -07:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Wed, 06 Aug 2025 10:29:48 +02:00
genirq/test: Resolve irq lock inversion warnings
irq_shutdown_and_deactivate() is normally called with the descriptor lock held, and interrupts disabled. Nested a few levels down, it grabs the global irq_resend_lock. Lockdep rightfully complains when interrupts are not disabled:
CPU0 CPU1
---- ----
lock(irq_resend_lock);
local_irq_disable();
lock(&irq_desc_lock_class);
lock(irq_resend_lock);
<Interrupt>
lock(&irq_desc_lock_class);...
_raw_spin_lock+0x2b/0x40 clear_irq_resend+0x14/0x70 irq_shutdown_and_deactivate+0x29/0x80 irq_shutdown_depth_test+0x1ce/0x600 kunit_try_run_case+0x90/0x120
Grab the descriptor lock and disable interrupts, to resolve the problem.
--- kernel/irq/irq_test.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/irq/irq_test.c +3 -1
@@ -1,5 +1,6 @@// SPDX-License-Identifier: LGPL-2.1+ +#include <linux/cleanup.h> #include <linux/cpu.h> #include <linux/cpumask.h> #include <linux/interrupt.h>
@@ -134,7 +135,8 @@ static void irq_shutdown_depth_test(struct kunit *test)disable_irq (virq);KUNIT_EXPECT_EQ (test, desc->depth,1 ); -irq_shutdown_and_deactivate (desc); + scoped_guard(raw_spinlock_irqsave, &desc->lock) +irq_shutdown_and_deactivate (desc);KUNIT_EXPECT_FALSE (test,irqd_is_activated (data));KUNIT_EXPECT_FALSE (test,irqd_is_started (data));
Hi Brian,
Quoted 23 lines · Brian Norris
There have been a few bugs and/or misunderstandings about the reference counting, and startup/shutdown behaviors in the IRQ core and related CPU hotplug code. These 4 test cases try to capture a few interesting cases.
*
irq_disable_depth_test: basic request/disable/enable sequence*
irq_free_disabled_test: request/disable/free/re-request sequence - this catches errors on previous revisions of my work*
irq_cpuhotplug_test: exercises managed-affinity IRQ + CPU hotplug. This captures a problematic test case that I've fixed. This test requiresCONFIG_SMPand a hotpluggable CPU#1.*
irq_shutdown_depth_test: exercises similar behavior fromirq_cpuhotplug_test, but directly using irq_*() APIs instead of going through CPU hotplug. This still requiresCONFIG_SMP, because managed-affinity is stubbed out (and not all APIs are even present) without it.Note the use of 'imply SMP': ARCH=um doesn't support SMP, and kunit is often exercised there. Thus, 'imply' will force SMP on where possible (such as ARCH=x86_64), but leave it off where it's not.
...
kernel/irq/Kconfig config GENERIC_IRQ_DEBUGFS · line 157
Quoted 12 lines
kernel/irq/Kconfig +11
-0+config IRQ_KUNIT_TEST + bool "KUnit tests for IRQ management APIs" if !KUNIT_ALL_TESTS + depends on KUNIT=y + default KUNIT_ALL_TESTS + imply SMP + help + This option enables KUnit tests for the IRQ subsystem API. These are + only for development and testing, not for regular kernel use cases. + + If unsure, say N. +
The new code calls irq_domain_alloc_descs(), making it dependent on IRQ_DOMAIN. However, specifying that dependency directly is not possible:
config IRQ_KUNIT_TEST
bool "KUnit tests for IRQ management APIs" if !KUNIT_ALL_TESTS depends on KUNIT=y
+ depends on IRQ_DOMAIN
default KUNIT_ALL_TESTS
imply SMP
help
results in:
*** Default configuration is based on 'defconfig'
error: recursive dependency detected!
symbol SMP is implied by IRQ_KUNIT_TEST
symbol IRQ_KUNIT_TEST depends on IRQ_DOMAIN symbol IRQ_DOMAIN is selected by IRQ_DOMAIN_HIERARCHY symbol IRQ_DOMAIN_HIERARCHY is selected by GENERIC_IRQ_IPI symbol GENERIC_IRQ_IPI depends on SMP
This is seen with alpha configurations such as alpha:defconfig after adding the IRQ_DOMAIN dependency.
I have no idea how to resolve this. For now I disabled IRQ_KUNIT_TEST for my alpha test builds.
Guenter
Quoted 22 lines · Guenter Roeck
The new code calls
irq_domain_alloc_descs(), making it dependent onIRQ_DOMAIN. However, specifying that dependency directly is not possible:config
IRQ_KUNIT_TEST
bool "KUnit tests for IRQ management APIs" if !KUNIT_ALL_TESTS depends on KUNIT=y
+ depends onIRQ_DOMAIN
defaultKUNIT_ALL_TESTS
imply SMP
helpresults in:
*** Default configuration is based on 'defconfig'
error: recursive dependency detected!
symbol SMP is implied byIRQ_KUNIT_TEST
symbolIRQ_KUNIT_TESTdepends onIRQ_DOMAINsymbolIRQ_DOMAINis selected byIRQ_DOMAIN_HIERARCHYsymbolIRQ_DOMAIN_HIERARCHYis selected byGENERIC_IRQ_IPIsymbolGENERIC_IRQ_IPIdepends on SMP
Quoted 5 lines
This is seen with alpha configurations such as alpha:defconfig after adding the
IRQ_DOMAINdependency.I have no idea how to resolve this. For now I disabled
IRQ_KUNIT_TESTfor my alpha test builds.
How about 'select'? That's the usual way IRQ_DOMAIN is managed anyway.
It builds for me, but my distro doesn't provide an Alpha QEMU, so I can't test it.
kernel/irq/Kconfig +1 -0
@@ -148,6 +148,7 @@ config IRQ_KUNIT_TEST bool "KUnit tests for IRQ management APIs" if !KUNIT_ALL_TESTS depends on KUNIT=y default KUNIT_ALL_TESTS + select IRQ_DOMAIN imply SMP help This option enables KUnit tests for the IRQ subsystem API. These are
Quoted 26 lines · Brian Norris
Quoted 25 lines · Guenter Roeck
The new code calls
irq_domain_alloc_descs(), making it dependent onIRQ_DOMAIN. However, specifying that dependency directly is not possible:config
IRQ_KUNIT_TEST
bool "KUnit tests for IRQ management APIs" if !KUNIT_ALL_TESTS depends on KUNIT=y
+ depends onIRQ_DOMAIN
defaultKUNIT_ALL_TESTS
imply SMP
helpresults in:
*** Default configuration is based on 'defconfig'
error: recursive dependency detected!
symbol SMP is implied byIRQ_KUNIT_TEST
symbolIRQ_KUNIT_TESTdepends onIRQ_DOMAINsymbolIRQ_DOMAINis selected byIRQ_DOMAIN_HIERARCHYsymbolIRQ_DOMAIN_HIERARCHYis selected byGENERIC_IRQ_IPIsymbolGENERIC_IRQ_IPIdepends on SMPThis is seen with alpha configurations such as alpha:defconfig after adding the
IRQ_DOMAINdependency.
Quoted 8 lines
Quoted 2 lines
I have no idea how to resolve this. For now I disabled
IRQ_KUNIT_TESTfor my alpha test builds.How about 'select'? That's the usual way
IRQ_DOMAINis managed anyway.It builds for me, but my distro doesn't provide an Alpha QEMU, so I can't test it.
I can try, but the irq test code fails for me all over the place, so I am not sure if it is worth it.
From my current upstream test results (6.17-rc1):
Build results:
total: 162 pass: 162 fail: 0 Qemu test results: total: 637 pass: 637 fail: 0 Unit test results: pass: 640017 fail: 649
The failures are all from the irq test code. I didn't have time to analyze it. You can find details at https://kerneltests.org/builders in the "master" column if you have time.
Note that "imply SMP" does not make SMP mandatory. I can still disable it after enabling IRQ_KUNIT_TEST on x86. Frankly I don't really understand what "imply" is supposed to be useful for.
Guenter
Quoted 10 lines
kernel/irq/Kconfig +1
-0@@ -148,6 +148,7 @@ config IRQ_KUNIT_TEST bool "KUnit tests for IRQ management APIs" if !KUNIT_ALL_TESTS depends on KUNIT=y default KUNIT_ALL_TESTS + select IRQ_DOMAIN imply SMP help This option enables KUnit tests for the IRQ subsystem API. These are
Quoted 12 lines · Guenter Roeck
I can try, but the irq test code fails for me all over the place, so I am not sure if it is worth it.
From my current upstream test results (6.17-rc1):
Build results:
total: 162 pass: 162 fail: 0 Qemu test results: total: 637 pass: 637 fail: 0 Unit test results: pass: 640017 fail: 649
Quoted 3 lines
The failures are all from the irq test code. I didn't have time to analyze it. You can find details at https://kerneltests.org/builders in the "master" column if you have time.
I can't replicate all of those easily, since I don't have tooling ready for some of the more esoteric architectures. But many of those look like they boil down to a single oversight: that some architectures default to IRQ_NOREQUEST, and so I need to throw this onto the fake IRQs I set up:
irq_settings_clr_norequest(desc);
With that, I can pass on ARCH=arm:
tools/testing/kunit/kunit.py run 'irq_test_cases*' --arch arm \
--qemu_args '-smp 2' --cross_compile arm-linux-gnueabi-I'm less sure about the ARCH=parisc{,64} ones, but I think that boils down to missing CONFIG_SPARSE_IRQ. I think I can skip tests in the !SPARSE_IRQ case.
Quoted 2 lines
Note that "imply SMP" does not make SMP mandatory. I can still disable it after enabling
IRQ_KUNIT_TESTon x86.
Right, that's intentional. There are a few CONFIG_SMP-conditional bits in the tests, since many users likely run on ARCH=um, which does not have an SMP build.
Quoted 2 lines
Frankly I don't really understand what "imply"
is supposed to be useful for.
Documentation/kbuild/kconfig-language.rst calls it a "weak reverse dependency" (i.e., a weak "select") and:
This is useful e.g. with multiple drivers that want to indicate their ability to hook into a secondary subsystem while allowing the user to configure that subsystem out without also having to unset these drivers.
Basically, I want SMP when it's available, but I don't want to force it when not. And it means on KUNIT_ALL_TESTS builds that otherwise didn't make an opinionated choice for CONFIG_SMP, we get CONFIG_SMP=y. The latter point is relevant to tools/testing/kunit/kunit.py.
I have a patch series to clean up a little while fixing the errors you've pointed me at. I'll probably send it out next week.
Thanks,
Brian