Skip to contentSkip to conversation

CPEI gets warning at kernel/irq/migration.c:27/move_masked_irq()

PATCH In review Hidetoshi Seto 3 participants 4 messages Dec 5, 2006 – Dec 7, 2006 IRQ

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 CPEI gets warning at kernel/irq/migration.c:27/move_masked_irq()

Hi,

While running my MCA test (hardware error injection) on 2.6.19, I got some warning like following:

Quoted 13 lines
BUG: warning at kernel/irq/migration.c:27/move_masked_irq()

Call Trace:
 [<a000000100013d20>] show_stack+0x40/0xa0
                                sp=e00000006b2578d0 bsp=e00000006b2510b0
 [<a000000100013db0>] dump_stack+0x30/0x60
                                sp=e00000006b257aa0 bsp=e00000006b251098
 [<a0000001000de430>] move_masked_irq+0xb0/0x240
                                sp=e00000006b257aa0 bsp=e00000006b251070
 [<a0000001000de6a0>] move_native_irq+0xe0/0x180
                                sp=e00000006b257aa0 bsp=e00000006b251040
 [<a00000010004ff50>] iosapic_end_level_irq+0x30/0xe0
                                sp=e00000006b257aa0 bsp=e00000006b251020
Quoted 8 lines
 [<a0000001000d94d0>] __do_IRQ+0x170/0x400
                                sp=e00000006b257aa0 bsp=e00000006b250fd8
 [<a0000001000116f0>] ia64_handle_irq+0x1b0/0x260
                                sp=e00000006b257aa0 bsp=e00000006b250fa8
 [<a00000010000c3a0>] ia64_leave_kernel+0x0/0x280
                                sp=e00000006b257aa0 bsp=e00000006b250fa8
 [<a000000100690cf0>] _spin_unlock_irqrestore+0x30/0x60
                                sp=e00000006b257c70 bsp=e00000006b250f90

It comes from:

[kernel/irq/migration.c]
26 if (CHECK_IRQ_PER_CPU(desc->status)) {
27 WARN_ON(1);
28 return;
29 }

By putting some printk in kernel, I found that irqbalance is trying to move CPEI which is handled as PER_CPU irq. That's why.

CPEI(Corrected Platform Error Interrupt) is ia64 specific irq, is allowed to pin to particular processor which selected by the platform, and even it is PER_CPU but it has set_affinity handler (=iosapic_set_affinity) as same as other IO-SAPIC-level interrupts. (I don't know why, but I guess that there would be typical situation where the handler for migration is needed, such as hotplug - the processor going to be offline/hot-removed.)

To shut up this warning, there are 2 way at least:
a) fix CPEI stuff
b) prohibit setting affinity to PER_CPU irq

I'm not sure what stuff of CPEI need to be fixed, but I think that returning error to attempting move PER_CPU irq is useful for all applications since it will never work.

Following small patch takes b) style.
It works, the warning disappeared and irqbalance still runs well.

Thanks,
H.Seto

---
 kernel/irq/proc.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/irq/proc.c +2 -1
Index: linux-2.6.19/kernel/irq/proc.c
===================================================================
@@ -54,7 +54,8 @@ static int irq_affinity_write_proc(struc
 	unsigned int irq = (int)(long)data, full_count = count, err;
 	cpumask_t new_value, tmp;
 
-	if (!irq_desc[irq].chip->set_affinity || no_irq_affinity)
+	if (!irq_desc[irq].chip->set_affinity || no_irq_affinity ||
+				CHECK_IRQ_PER_CPU(irq_desc[irq].status))
 		return -EIO;
 
 	err = cpumask_parse_user(buffer, count, new_value);
#1 lore raw
Quoted 70 lines · On Tue, 05 Dec 2006 16:35:06 +0900 Hidetoshi Seto…

Hi,

While running my MCA test (hardware error injection) on 2.6.19, I got some warning like following:

Quoted 13 lines
BUG: warning at kernel/irq/migration.c:27/move_masked_irq()

Call Trace:
 [<a000000100013d20>] show_stack+0x40/0xa0
                                spà0000006b2578d0 bspà0000006b2510b0
 [<a000000100013db0>] dump_stack+0x30/0x60
                                spà0000006b257aa0 bspà0000006b251098
 [<a0000001000de430>] move_masked_irq+0xb0/0x240
                                spà0000006b257aa0 bspà0000006b251070
 [<a0000001000de6a0>] move_native_irq+0xe0/0x180
                                spà0000006b257aa0 bspà0000006b251040
 [<a00000010004ff50>] iosapic_end_level_irq+0x30/0xe0
                                spà0000006b257aa0 bspà0000006b251020
Quoted 8 lines
 [<a0000001000d94d0>] __do_IRQ+0x170/0x400
                                spà0000006b257aa0 bspà0000006b250fd8
 [<a0000001000116f0>] ia64_handle_irq+0x1b0/0x260
                                spà0000006b257aa0 bspà0000006b250fa8
 [<a00000010000c3a0>] ia64_leave_kernel+0x0/0x280
                                spà0000006b257aa0 bspà0000006b250fa8
 [<a000000100690cf0>] _spin_unlock_irqrestore+0x30/0x60
                                spà0000006b257c70 bspà0000006b250f90

It comes from:

[kernel/irq/migration.c]
26 if (CHECK_IRQ_PER_CPU(desc->status)) {
27 WARN_ON(1);
28 return;
29 }

By putting some printk in kernel, I found that irqbalance is trying to move CPEI which is handled as PER_CPU irq. That's why.

CPEI(Corrected Platform Error Interrupt) is ia64 specific irq, is allowed to pin to particular processor which selected by the platform, and even it is PER_CPU but it has set_affinity handler (=iosapic_set_affinity) as same as other IO-SAPIC-level interrupts. (I don't know why, but I guess that there would be typical situation where the handler for migration is needed, such as hotplug - the processor going to be offline/hot-removed.)

To shut up this warning, there are 2 way at least:
a) fix CPEI stuff
b) prohibit setting affinity to PER_CPU irq

I'm not sure what stuff of CPEI need to be fixed, but I think that returning error to attempting move PER_CPU irq is useful for all applications since it will never work.

Following small patch takes b) style.
It works, the warning disappeared and irqbalance still runs well.

Thanks,
H.Seto

---
 kernel/irq/proc.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Index: linux-2.6.19/kernel/irq/proc.c =================================> --- linux-2.6.19.orig/kernel/irq/proc.c +++ linux-2.6.19/kernel/irq/proc.c

kernel/irq/proc.c static int irq_affinity_write_proc(struc · line 59

Quoted 7 lines
kernel/irq/proc.c +2 -1
@@ -54,7 +54,8 @@ static int irq_affinity_write_proc(struc
 	unsigned int irq = (int)(long)data, full_count = count, err;
 	cpumask_t new_value, tmp;
 
-	if (!irq_desc[irq].chip->set_affinity || no_irq_affinity)
+	if (!irq_desc[irq].chip->set_affinity || no_irq_affinity ||
+				CHECK_IRQ_PER_CPU(irq_desc[irq].status))
 		return -EIO;

It'd be nice if we could just teach the userspace balancer to not try to move perpcu IRQs?

otoh, the patch is super-cheap. Arjan?

#2 lore raw
Quoted 5 lines

It'd be nice if we could just teach the userspace balancer to not try to move perpcu IRQs?

otoh, the patch is super-cheap. Arjan?

I can fix irqbalance no problem, however I like the kernel approach as well, since it's not just irqbalance that moves irqs, sysadmins tend to do it as well.... so how about both?

One thing we probably should do, and that would help irqbalance immensely, is to export a bitmask for which cpus an interrupt CAN go to, next to where the current binding interface is. I'll check into that

Hidetoshi: would it be possible to send me a /proc/interrupts file of that machine?

Signature · 2 lines
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org
#3 lore raw
Quoted 8 lines · Arjan van de Ven
Quoted 4 lines

It'd be nice if we could just teach the userspace balancer to not try to move perpcu IRQs?

otoh, the patch is super-cheap. Arjan?

I can fix irqbalance no problem, however I like the kernel approach as well, since it's not just irqbalance that moves irqs, sysadmins tend to do it as well.... so how about both?

Thanks for positive comments.

Quoted 6 lines

One thing we probably should do, and that would help irqbalance immensely, is to export a bitmask for which cpus an interrupt CAN go to, next to where the current binding interface is. I'll check into that

Hidetoshi: would it be possible to send me a /proc/interrupts file of that machine?

Here you are.
This machine is not special. I guess all ia64 machine will show few or more but almost same lines. CPEI is #30.

$ cat /proc/interrupts
           CPU0       CPU1
 28:          0          0          LSAPIC  cpe_poll
 29:          0          0          LSAPIC  cmc_poll
 30:          0          0  IO-SAPIC-level  cpe_hndlr
 31:          0          0          LSAPIC  cmc_hndlr
 34:       4643    1881067   IO-SAPIC-edge  ide0
 39:          0          0  IO-SAPIC-level  acpi
 45:        148        560   IO-SAPIC-edge  serial
 48:          0          0  IO-SAPIC-level  uhci_hcd
 49:          0          0  IO-SAPIC-level  uhci_hcd
 50:          0          0  IO-SAPIC-level  ehci_hcd
 51:      21131    1257790  IO-SAPIC-level  eth0
 53:      21778     458314  IO-SAPIC-level  ioc0
 54:          0         44  IO-SAPIC-level  ioc1
232:          0          0          LSAPIC  mca_rdzv
238:          0          0          LSAPIC  perfmon
239:  214832233  214832003          LSAPIC  timer
240:          0          0          LSAPIC  mca_wkup
254:     209193        611          LSAPIC  IPI
ERR:          0

Thanks,
H.Seto