Skip to contentSkip to conversation

kexec: Avoid migration of already disabled

PATCH Jay Lan 3 participants 7 messages Jan 31, 2007 – Feb 1, 2007

lore
Share & tools
lore

Jay Lan

author
#0 lore raw
Quoted 6 lines · Magnus Damm

kexec: Avoid migration of already disabled irqs (ia64)

This patch fixes up ia64 kexec support for HP rx2620 hardware. It does this by skipping migration of already disabled irqs. This is most likely a problem on other ia64 platforms as well, but I've only tested this on one machine so far.

I have not seen this problem on SN systems.

Cheers,
- jay

Quoted 33 lines

The full story is that handle_bad_irq() gets invoked before starting the new kernel without this patch. This seems to happen when fixup_irqs() calls generic_handle_irq() on already migrated (and disabled) irqs. So by avoiding migration of disabled irqs we stay away of handle_bad_irq().

---

Tested using kexec-tools-testing 7792798a79b78a5d566f70c9f00237d050b01350 on HP rx2620 hardware.

Applies on top of 2.6.20-rc6.

 arch/ia64/kernel/irq.c |    3 +++
 1 file changed, 3 insertions(+)
arch/ia64/kernel/irq.c +3 -0
@@ -122,6 +122,9 @@ static void migrate_irqs(void)
 	for (irq=0; irq < NR_IRQS; irq++) {
 		desc = irq_desc + irq;
 
+		if (desc->status = IRQ_DISABLED)
+			continue;
+
 		/*
 		 * No handling for now.
 		 * TBD: Implement a disable function so we can now

_______________________________________________
fastboot mailing list
[email protected]
https://lists.osdl.org/mailman/listinfo/fastboot

#1 lore raw

Re: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)

Quoted 9 lines · Jay Lan
Quoted 6 lines · Magnus Damm

kexec: Avoid migration of already disabled irqs (ia64)

This patch fixes up ia64 kexec support for HP rx2620 hardware. It does this by skipping migration of already disabled irqs. This is most likely a problem on other ia64 platforms as well, but I've only tested this on one machine so far.

I have not seen this problem on SN systems.

Ok, thanks. Let me give you more details.

When I perform "kexec -e" the following output appears on my serial console (with my patch applied).

ACPI: PCI interrupt for device 0000:20:02.1 disabled
GSI 30 (level, low) -> CPU 1 (0x0200) vector 53 unregistered
ACPI: PCI interrupt for device 0000:20:02.0 disabled
GSI 29 (level, low) -> CPU 0 (0x0000) vector 52 unregistered Starting new kernel
CPU 1 is now offline
CPU 2 is now offline
CPU 3 is now offline
Linux version 2.6.20-rc6 (damm@localhost) (gcc version 3.4.5) #1 SMP Tue Jan 30 16:59:54 JST 2007

Without the patch the kernel tries to migrate already disabled interrupts which results in this:

ACPI: PCI interrupt for device 0000:20:02.1 disabled GSI 30 (level, low) -> CPU 1 (0x0200) vector 53 unregistered
ACPI: PCI interrupt for device 0000:20:02.0 disabled GSI 29 (level, low) -> CPU 0 (0x0000) vector 52 unregistered Starting new kernel

BUG: at arch/ia64/kernel/irq.c:155 migrate_irqs()

Call Trace:
 [<a000000100012e10>] show_stack+0x50/0xa0
                                spà000040fb7f7b20 bspà000040fb7f0d60
 [<a000000100012e90>] dump_stack+0x30/0x60
                                spà000040fb7f7cf0 bspà000040fb7f0d48
 [<a000000100011610>] fixup_irqs+0x490/0x680
                                spà000040fb7f7cf0 bspà000040fb7f0d08
 [<a000000100055300>] __cpu_disable+0x5c0/0x660
                                spà000040fb7f7d80 bspà000040fb7f0cb8
 [<a0000001000d21c0>] take_cpu_down+0x20/0x80
                                spà000040fb7f7dc0 bspà000040fb7f0ca0
 [<a0000001000dcd90>] do_stop+0x250/0x360
                                spà000040fb7f7dc0 bspà000040fb7f0c60
 [<a0000001000bf9b0>] kthread+0x230/0x2a0
                                spà000040fb7f7dd0 bspà000040fb7f0c20
 [<a000000100015290>] kernel_thread_helper+0xd0/0x100
                                spà000040fb7f7e30 bspà000040fb7f0bf0
 [<a0000001000094c0>] start_kernel_thread+0x20/0x40
                                spà000040fb7f7e30 bspà000040fb7f0bf0

irq 53, desc: a0000001007d1080, depth: 0, count: 3, unhandled: 0 ->handle_irq(): 0000000000000000, 0x0
->chip(): a000000100831fa8, no_irq_chip+0x0/0x80 ->action(): 0000000000000000
IRQ_DISABLED set
Unexpected irq vector 0x35 on CPU 1!
CPU 1 is now offline
...
(more or less same thing on CPU2 and CPU3 as well)

This is how my /proc/interrupts look:

/ # cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3

 28:          1          1          1          1          LSAPIC  cpe_poll
 29:          0          0          0          0          LSAPIC  cmc_poll
 31:          0          0          0          0          LSAPIC  cmc_hndlr
 48:          0          0          0          0  IO-SAPIC-level  acpi
 49:          0         52          0          0  IO-SAPIC-level  serial
 52:        104          0          0          0  IO-SAPIC-level  eth0
232:          0          0          0          0          LSAPIC  mca_rdzv
238:          0          0          0          0          LSAPIC  perfmon
239:       7027       6979       7055       6916          LSAPIC  timer
240:          0          0          0          0          LSAPIC  mca_wkup
253:        145        106        766        689          LSAPIC  resched
254:         21         50         51         34          LSAPIC  IPI

ERR: 0

Are IOSAPIC interrupts routed to multiples CPUs in your case? Do you get any "ACPI: PCI interrupt for device nnn disabled" messages?

Thanks!

/ magnus

#2 lore raw

RE: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)

In reply to Jay Lan
Quoted 18 lines

-----Original Message-----
From: Magnus Damm [mailto:[email protected]]
Sent: 2007Äê1ÔÂ31ÈÕ 11:50
To: Jay Lan
Cc: Magnus Damm; [email protected]; Luck, Tony; Zou, Nanhai; [email protected]
Subject: Re: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)

Quoted 8 lines · Jay Lan
Quoted 6 lines · Magnus Damm

kexec: Avoid migration of already disabled irqs (ia64)

This patch fixes up ia64 kexec support for HP rx2620 hardware. It does this by skipping migration of already disabled irqs. This is most likely a problem on other ia64 platforms as well, but I've only tested this on one machine so far.

Quoted 6 lines
Quoted 1 line

I have not seen this problem on SN systems.

Ok, thanks. Let me give you more details.

When I perform "kexec -e" the following output appears on my serial console (with my patch applied).

This patch is correct I think. I assume you will also see this bug when trying to offline a CPU by echo 0 > /sys/devices/system/cpu/cpuX/online

However it will not be triggered in crash dump case which has been tested heavily on different kind of platforms. We do not migrate IRQ at the time of crash. I guess that is why Jay has never seen it on SN platforms.

Thanks
Zou Nan hai

#3 lore raw

Re: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)

In reply to Jay Lan

Hi Nan hai,

Quoted 9 lines · Nanhai
Quoted 9 lines · Jay Lan
Quoted 8 lines
Quoted 6 lines · Magnus Damm

kexec: Avoid migration of already disabled irqs (ia64)

This patch fixes up ia64 kexec support for HP rx2620 hardware. It does this by skipping migration of already disabled irqs. This is most likely a problem on other ia64 platforms as well, but I've only tested this on one machine so far.

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

I have not seen this problem on SN systems.

Ok, thanks. Let me give you more details.

When I perform "kexec -e" the following output appears on my serial console (with my patch applied).

This patch is correct I think. I assume you will also see this bug when trying to offline a CPU by echo 0 > /sys/devices/system/cpu/cpuX/online

You are right, I can trigger the bug that way too. And the bug goes away with the patch. Excellent!

Quoted 1 line

However it will not be triggered in crash dump case which has been tested heavily on different kind of platforms. We do not migrate IRQ at the time of crash. I guess that is why Jay has never seen it on SN platforms.

Booting into a crash kernel does not seem to work on this ia64 box unfortunately. I'm about to investigate why, but I thought trying and fixing kexec was a good first step.

Thanks for the help!

/ magnus

#4 lore raw

RE: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)

In reply to Jay Lan
Quoted 37 lines

-----Original Message-----
From: Magnus Damm [mailto:[email protected]]
Sent: 2007Äê1ÔÂ31ÈÕ 13:58
To: Zou, Nanhai
Cc: Jay Lan; Magnus Damm; [email protected]; Luck, Tony; [email protected]
Subject: Re: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)

Hi Nan hai,

Quoted 5 lines · Nanhai
Quoted 5 lines · Jay Lan
Quoted 4 lines
Quoted 3 lines · Magnus Damm

kexec: Avoid migration of already disabled irqs (ia64)

This patch fixes up ia64 kexec support for HP rx2620 hardware. It does

this

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

by skipping migration of already disabled irqs. This is most likely

a problem

Quoted 11 lines
Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

on other ia64 platforms as well, but I've only tested this on one machine so far.

I have not seen this problem on SN systems.

Ok, thanks. Let me give you more details.

When I perform "kexec -e" the following output appears on my serial console (with my patch applied).

This patch is correct I think. I assume you will also see this bug when trying

to offline a CPU by echo 0 > /sys/devices/system/cpu/cpuX/online

You are right, I can trigger the bug that way too. And the bug goes away with the patch. Excellent!

Quoted 8 lines
Quoted 1 line

However it will not be triggered in crash dump case which has been tested

heavily on different kind of platforms. We do not migrate IRQ at the time of crash. I guess that is why Jay has never seen it on SN platforms.

Booting into a crash kernel does not seem to work on this ia64 box unfortunately. I'm about to investigate why, but I thought trying and fixing kexec was a good first step.

That is probably caused by HP's IOMMU engine not shutting down at crash time....,
Please try add machvec=dig option for the crash dump kernel.
However we still need to find a light way to correctly shutdown IOMMU engine on those HP platforms.

Thanks
Zou Nan hai

Quoted 4 lines

Thanks for the help!

/ magnus

author
#5 lore raw
In reply to Jay Lan
Quoted 22 lines · Magnus Damm

Hi Nan hai,

Quoted 19 lines · Nanhai
Quoted 5 lines · Jay Lan
Quoted 4 lines
Quoted 3 lines · Magnus Damm

kexec: Avoid migration of already disabled irqs (ia64)

This patch fixes up ia64 kexec support for HP rx2620 hardware.

It does this

Quoted 1 line
Quoted 1 line
Quoted 1 line

by skipping migration of already disabled irqs. This is most

likely a problem

Quoted 1 line
Quoted 1 line
Quoted 1 line

on other ia64 platforms as well, but I've only tested this on

one machine

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

so far.

I have not seen this problem on SN systems.

Ok, thanks. Let me give you more details.

When I perform "kexec -e" the following output appears on my serial console (with my patch applied).

Quoted 6 lines
Quoted 3 lines

This patch is correct I think. I assume you will also see this bug
when trying to offline a CPU by echo 0 > /sys/devices/system/cpu/cpuX/online

You are right, I can trigger the bug that way too. And the bug goes away with the patch. Excellent!

I tried on a 2p SN system with both '-l' followed by '-e' and also Nan-hai's trigger command. Neither triggered the problem. But, on the other hand, the system worked fine with your patch also. :)

Thanks,
- jay

Quoted 13 lines
Quoted 5 lines

However it will not be triggered in crash dump case which has been tested heavily on different kind of platforms. We do not migrate IRQ at the time of crash. I guess that is why Jay has never seen it on SN platforms.

Booting into a crash kernel does not seem to work on this ia64 box unfortunately. I'm about to investigate why, but I thought trying and fixing kexec was a good first step.

Thanks for the help!

/ magnus

#6 lore raw

Re: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)

In reply to Jay Lan
Quoted 8 lines · Nanhai
Quoted 5 lines

From: Magnus Damm [mailto:[email protected]]
Booting into a crash kernel does not seem to work on this ia64 box unfortunately. I'm about to investigate why, but I thought trying and fixing kexec was a good first step.

That is probably caused by HP's IOMMU engine not shutting down at crash time....,
Please try add machvec=dig option for the crash dump kernel.
However we still need to find a light way to correctly shutdown IOMMU engine on those HP platforms.

It does unfortunately not seem to help. Regardless of SMP or UP kernel.

I'll try to track down where things lock up. Thanks.

/ magnus