scheduling with irqs disabled: swapper
Hey Ingo, All,
We had the following bug reported on bootup on one of our boxes (it
was a 4way I believe) running -rt22. So far it seems to be a one-off but I figured I'd post it to see if anyone had a clue.
thanks
-john
...
usbmon: debugfs is not available
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid drivers/usb/input/hid-core.c: v2.6:USB HID core driver
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
NET: Registered protocol family 2
BUG: scheduling with irqs disabled: swapper/0x00010002/0 caller is rt_lock_slowlock+0xd2/0x139 [<c030fb85>] schedule+0x60/0xd0 (8) [<c0310787>] rt_lock_slowlock+0xd2/0x139 (12) [<c0310e2d>] __lock_text_start+0x1d/0x1e (72) [<c022a57f>] i8042_interrupt+0x2c/0x1f0 (4) [<c013edb1>] misrouted_irq+0x85/0x118 (36) [<c013ef08>] note_interrupt+0x43/0x9f (36) [<c013e395>] __do_IRQ+0xc2/0xf9 (16)
Quoted 4 lines · john stultz
Hey Ingo, All,
We had the following bug reported on bootup on one of our boxes (it
was a 4way I believe) running -rt22. So far it seems to be a one-off but I figured I'd post it to see if anyone had a clue.
I'm assuming this is a i386. Also I'm assuming that frame pointers was not compiled in since the stack is a little suspicious.
Anyway, could you show the /proc/interrupts of this machine. I'm curious if the i8042 isn't sharing an interrupt with something with NODELAY in it.
-- Steve
Quoted 18 lines
...
usbmon: debugfs is not available
usbcore: registered new driver hiddev
usbcore: registered new driver usbhiddrivers/usb/input/hid-core.c: v2.6:USB HID core driver
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
NET: Registered protocol family 2BUG: scheduling with irqs disabled: swapper/0x00010002/0 caller is rt_lock_slowlock+0xd2/0x139 [<c030fb85>] schedule+0x60/0xd0 (8) [<c0310787>] rt_lock_slowlock+0xd2/0x139 (12) [<c0310e2d>] __lock_text_start+0x1d/0x1e (72) [<c022a57f>] i8042_interrupt+0x2c/0x1f0 (4) [<c013edb1>] misrouted_irq+0x85/0x118 (36) [<c013ef08>] note_interrupt+0x43/0x9f (36) [<c013e395>] __do_IRQ+0xc2/0xf9 (16)
Quoted 6 lines · Steven Rostedt
Quoted 4 lines · john stultz
Hey Ingo, All,
We had the following bug reported on bootup on one of our boxes (it
was a 4way I believe) running -rt22. So far it seems to be a one-off but I figured I'd post it to see if anyone had a clue.
Quoted 6 lines
I'm assuming this is a i386. Also I'm assuming that frame pointers was not compiled in since the stack is a little suspicious.
Anyway, could you show the
/proc/interruptsof this machine. I'm curious if the i8042 isn't sharing an interrupt with something with NODELAY in it.
Here ya go:
CPU0 CPU1 CPU2 CPU3
0: 8796 3868607 275 531673 IO-APIC-edge [........N/ 0] pit 2: 0 0 0 0 XT-PIC [........N/ 0] cascade 3: 5 620 2 229 IO-APIC-edge [........./ 63] serial 8: 0 1 0 0 IO-APIC-edge [........./ 0] rtc 11: 0 0 0 0 IO-APIC-edge [........./ 0] acpi 19: 120 0 0 1 IO-APIC-level [........./ 0] ohci_hcd:usb1, ohci_hcd:usb2 24: 57 9 5 46795 IO-APIC-level [........./ 0] eth0 26: 1396 14537 0 702 IO-APIC-level [........./ 0] ioc0
NMI: 0 0 0 0
LOC: 6907796 4419008 4415669 4413513
ERR: 0
MIS: 0
thanks
-john
Quoted 24 lines · john stultz
Quoted 6 lines · Steven Rostedt
Quoted 4 lines · john stultz
Hey Ingo, All,
We had the following bug reported on bootup on one of our boxes (it
was a 4way I believe) running -rt22. So far it seems to be a one-off but I figured I'd post it to see if anyone had a clue.Quoted 6 lines
I'm assuming this is a i386. Also I'm assuming that frame pointers was not compiled in since the stack is a little suspicious.
Anyway, could you show the
/proc/interruptsof this machine. I'm curious if the i8042 isn't sharing an interrupt with something with NODELAY in it.Here ya go:
CPU0 CPU1 CPU2 CPU30: 8796 3868607 275 531673 IO-APIC-edge [........N/ 0] pit 2: 0 0 0 0 XT-PIC [........N/ 0] cascade 3: 5 620 2 229 IO-APIC-edge [........./ 63] serial 8: 0 1 0 0 IO-APIC-edge [........./ 0] rtc 11: 0 0 0 0 IO-APIC-edge [........./ 0] acpi 19: 120 0 0 1 IO-APIC-level [........./ 0] ohci_hcd:usb1, ohci_hcd:usb2 24: 57 9 5 46795 IO-APIC-level [........./ 0] eth0 26: 1396 14537 0 702 IO-APIC-level [........./ 0] ioc0
Quoted 4 lines
NMI: 0 0 0 0
LOC: 6907796 4419008 4415669 4413513
ERR: 0
MIS: 0
Thanks, but I was looking more into the code, and I'm wondering... Does this machine have "irqfixup" or "irqpoll" set in the kernel command line?
I think that -rt doesn't support it yet. That is, it can call a handler from interrupt context, which should have been a thread.
Let me know if that was the case.
-- Steve
Quoted 8 lines · Steven Rostedt
Thanks, but I was looking more into the code, and I'm wondering... Does this machine have "irqfixup" or "irqpoll" set in the kernel command line?
I think that -rt doesn't support it yet. That is, it can call a handler from interrupt context, which should have been a thread.
Let me know if that was the case.
the backtrace shows misrouted_irq(), which is only called if "irqfixup" is enabled. That indeed isnt supported in -rt yet.
Ingo
Quoted 6 lines · Ingo Molnar
Quoted 4 lines · Steven Rostedt
Thanks, but I was looking more into the code, and I'm wondering... Does this machine have "irqfixup" or "irqpoll" set in the kernel command line?
Quoted 7 lines
Quoted 4 lines
I think that -rt doesn't support it yet. That is, it can call a handler from interrupt context, which should have been a thread.
Let me know if that was the case.
the backtrace shows
misrouted_irq(), which is only called if "irqfixup" is enabled. That indeed isnt supported in -rt yet.
Ugh. You and Steven are right. We've been bitten by this a few times, but we thought we got rid of that option on all of our boxes. I guess one slipped by.
Anyway, thanks for pointing that out. Would you consider a patch like the following so that folks don't continue to slip over this?
thanks
-john
kernel/irq/spurious.c +10 -0
@@ -194,6 +194,11 @@ __setup("noirqdebug", noirqdebug_setup);static int __init irqfixup_setup (char *str) { +#ifdef CONFIG_PREEMPT_RT +printk (KERN_WARNING "irqfixup boot option not supported " +"w/ CONFIG_PREEMT_RT\n" ); +return 1 ; +#endif irqfixup =1 ;printk (KERN_WARNING "Misrouted IRQ fixup support enabled.\n" );printk (KERN_WARNING "This may impact system performance.\n" );
@@ -204,6 +209,11 @@ __setup("irqfixup", irqfixup_setup);static int __init irqpoll_setup (char *str) { +#ifdef CONFIG_PREEMPT_RT +printk (KERN_WARNING "irqpoll boot option not supported " +"w/ CONFIG_PREEMT_RT\n" ); +return 1 ; +#endif irqfixup =2 ;printk (KERN_WARNING "Misrouted IRQ fixup and polling support " "enabled\n" );
Quoted 9 lines · john stultz
Quoted 2 lines
the backtrace shows
misrouted_irq(), which is only called if "irqfixup" is enabled. That indeed isnt supported in -rt yet.Ugh. You and Steven are right. We've been bitten by this a few times, but we thought we got rid of that option on all of our boxes. I guess one slipped by.
Anyway, thanks for pointing that out. Would you consider a patch like the following so that folks don't continue to slip over this?
sure, i've applied it. (fixed a small typo in the printout)
Ingo