kexec: Avoid migration of already disabled
Quoted 6 lines · Magnus Damm
kexec: Avoid migration of already disabled irqs (ia64)
This patch fixes up ia64 kexec support for HP rx2620 hardware. It does this by skipping migration of already disabled irqs. This is most likely a problem on other ia64 platforms as well, but I've only tested this on one machine so far.
I have not seen this problem on SN systems.
Cheers,
- jay
Quoted 33 lines
The full story is that
handle_bad_irq()gets invoked before starting the new kernel without this patch. This seems to happen whenfixup_irqs()callsgeneric_handle_irq()on already migrated (and disabled) irqs. So by avoiding migration of disabled irqs we stay away ofhandle_bad_irq().Signed-off-by Magnus Damm <[email protected]>---
Tested using kexec-tools-testing
7792798a79b78a5d566f70c9f00237d050b01350on HP rx2620 hardware.Applies on top of 2.6.20-rc6.
arch/ia64/kernel/irq.c | 3 +++ 1 file changed, 3 insertions(+)arch/ia64/kernel/irq.c +3
-0@@ -122,6 +122,9 @@ static void migrate_irqs(void)for (irq=0 ; irq <NR_IRQS ; irq++) { desc = irq_desc + irq; +if (desc->status =IRQ_DISABLED ) +continue ; +/* * No handling for now. * TBD: Implement a disable function so we can now _______________________________________________
fastboot mailing list
[email protected]
https://lists.osdl.org/mailman/listinfo/fastboot
Re: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)
Quoted 9 lines · Jay Lan
Quoted 6 lines · Magnus Damm
kexec: Avoid migration of already disabled irqs (ia64)
This patch fixes up ia64 kexec support for HP rx2620 hardware. It does this by skipping migration of already disabled irqs. This is most likely a problem on other ia64 platforms as well, but I've only tested this on one machine so far.
I have not seen this problem on SN systems.
Ok, thanks. Let me give you more details.
When I perform "kexec -e" the following output appears on my serial console (with my patch applied).
ACPI: PCI interrupt for device 0000:20:02.1 disabled
GSI 30 (level, low) -> CPU 1 (0x0200) vector 53 unregistered
ACPI: PCI interrupt for device 0000:20:02.0 disabled
GSI 29 (level, low) -> CPU 0 (0x0000) vector 52 unregistered Starting new kernel
CPU 1 is now offline
CPU 2 is now offline
CPU 3 is now offline
Linux version 2.6.20-rc6 (damm@localhost) (gcc version 3.4.5) #1 SMP Tue Jan 30 16:59:54 JST 2007
Without the patch the kernel tries to migrate already disabled interrupts which results in this:
ACPI: PCI interrupt for device 0000:20:02.1 disabled GSI 30 (level, low) -> CPU 1 (0x0200) vector 53 unregistered
ACPI: PCI interrupt for device 0000:20:02.0 disabled GSI 29 (level, low) -> CPU 0 (0x0000) vector 52 unregistered Starting new kernel
BUG: at arch/ia64/kernel/irq.c:155 migrate_irqs()
Call Trace:
[<a000000100012e10>] show_stack+0x50/0xa0
spà000040fb7f7b20 bspà000040fb7f0d60
[<a000000100012e90>] dump_stack+0x30/0x60
spà000040fb7f7cf0 bspà000040fb7f0d48
[<a000000100011610>] fixup_irqs+0x490/0x680
spà000040fb7f7cf0 bspà000040fb7f0d08
[<a000000100055300>] __cpu_disable+0x5c0/0x660
spà000040fb7f7d80 bspà000040fb7f0cb8
[<a0000001000d21c0>] take_cpu_down+0x20/0x80
spà000040fb7f7dc0 bspà000040fb7f0ca0
[<a0000001000dcd90>] do_stop+0x250/0x360
spà000040fb7f7dc0 bspà000040fb7f0c60
[<a0000001000bf9b0>] kthread+0x230/0x2a0
spà000040fb7f7dd0 bspà000040fb7f0c20
[<a000000100015290>] kernel_thread_helper+0xd0/0x100
spà000040fb7f7e30 bspà000040fb7f0bf0
[<a0000001000094c0>] start_kernel_thread+0x20/0x40
spà000040fb7f7e30 bspà000040fb7f0bf0irq 53, desc: a0000001007d1080, depth: 0, count: 3, unhandled: 0 ->handle_irq(): 0000000000000000, 0x0
->chip(): a000000100831fa8, no_irq_chip+0x0/0x80 ->action(): 0000000000000000
IRQ_DISABLED set
Unexpected irq vector 0x35 on CPU 1!
CPU 1 is now offline
...
(more or less same thing on CPU2 and CPU3 as well)
This is how my /proc/interrupts look:
/ # cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
28: 1 1 1 1 LSAPIC cpe_poll 29: 0 0 0 0 LSAPIC cmc_poll 31: 0 0 0 0 LSAPIC cmc_hndlr 48: 0 0 0 0 IO-SAPIC-level acpi 49: 0 52 0 0 IO-SAPIC-level serial 52: 104 0 0 0 IO-SAPIC-level eth0 232: 0 0 0 0 LSAPIC mca_rdzv 238: 0 0 0 0 LSAPIC perfmon 239: 7027 6979 7055 6916 LSAPIC timer 240: 0 0 0 0 LSAPIC mca_wkup 253: 145 106 766 689 LSAPIC resched 254: 21 50 51 34 LSAPIC IPI
ERR: 0
Are IOSAPIC interrupts routed to multiples CPUs in your case? Do you get any "ACPI: PCI interrupt for device nnn disabled" messages?
Thanks!
/ magnus
RE: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)
In reply to Jay LanQuoted 18 lines
-----Original Message-----
From: Magnus Damm [mailto:[email protected]]
Sent: 2007Äê1ÔÂ31ÈÕ 11:50
To: Jay Lan
Cc: Magnus Damm; [email protected]; Luck, Tony; Zou, Nanhai; [email protected]
Subject: Re: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)Quoted 8 lines · Jay Lan
Quoted 6 lines · Magnus Damm
kexec: Avoid migration of already disabled irqs (ia64)
This patch fixes up ia64 kexec support for HP rx2620 hardware. It does this by skipping migration of already disabled irqs. This is most likely a problem on other ia64 platforms as well, but I've only tested this on one machine so far.
Quoted 6 lines
Quoted 1 line
I have not seen this problem on SN systems.
Ok, thanks. Let me give you more details.
When I perform "kexec -e" the following output appears on my serial console (with my patch applied).
This patch is correct I think. I assume you will also see this bug when trying to offline a CPU by echo 0 > /sys/devices/system/cpu/cpuX/online
However it will not be triggered in crash dump case which has been tested heavily on different kind of platforms. We do not migrate IRQ at the time of crash. I guess that is why Jay has never seen it on SN platforms.
Thanks
Zou Nan hai
Re: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)
In reply to Jay LanHi Nan hai,
Quoted 9 lines · Nanhai
Quoted 9 lines · Jay Lan
Quoted 8 lines
Quoted 6 lines · Magnus Damm
kexec: Avoid migration of already disabled irqs (ia64)
This patch fixes up ia64 kexec support for HP rx2620 hardware. It does this by skipping migration of already disabled irqs. This is most likely a problem on other ia64 platforms as well, but I've only tested this on one machine so far.
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
I have not seen this problem on SN systems.
Ok, thanks. Let me give you more details.
When I perform "kexec -e" the following output appears on my serial console (with my patch applied).
This patch is correct I think. I assume you will also see this bug when trying to offline a CPU by echo 0 >
/sys/devices/system/cpu/cpuX/online
You are right, I can trigger the bug that way too. And the bug goes away with the patch. Excellent!
Quoted 1 line
However it will not be triggered in crash dump case which has been tested heavily on different kind of platforms. We do not migrate IRQ at the time of crash. I guess that is why Jay has never seen it on SN platforms.
Booting into a crash kernel does not seem to work on this ia64 box unfortunately. I'm about to investigate why, but I thought trying and fixing kexec was a good first step.
Thanks for the help!
/ magnus
RE: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)
In reply to Jay LanQuoted 37 lines
-----Original Message-----
From: Magnus Damm [mailto:[email protected]]
Sent: 2007Äê1ÔÂ31ÈÕ 13:58
To: Zou, Nanhai
Cc: Jay Lan; Magnus Damm; [email protected]; Luck, Tony; [email protected]
Subject: Re: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)Hi Nan hai,
Quoted 5 lines · Nanhai
Quoted 5 lines · Jay Lan
Quoted 4 lines
Quoted 3 lines · Magnus Damm
kexec: Avoid migration of already disabled irqs (ia64)
This patch fixes up ia64 kexec support for HP rx2620 hardware. It does
this
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
by skipping migration of already disabled irqs. This is most likely
a problem
Quoted 11 lines
Quoted 9 lines
Quoted 4 lines
Quoted 2 lines
on other ia64 platforms as well, but I've only tested this on one machine so far.
I have not seen this problem on SN systems.
Ok, thanks. Let me give you more details.
When I perform "kexec -e" the following output appears on my serial console (with my patch applied).
This patch is correct I think. I assume you will also see this bug when trying
to offline a CPU by echo 0 >
/sys/devices/system/cpu/cpuX/onlineYou are right, I can trigger the bug that way too. And the bug goes away with the patch. Excellent!
Quoted 8 lines
Quoted 1 line
However it will not be triggered in crash dump case which has been tested
heavily on different kind of platforms. We do not migrate IRQ at the time of crash. I guess that is why Jay has never seen it on SN platforms.
Booting into a crash kernel does not seem to work on this ia64 box unfortunately. I'm about to investigate why, but I thought trying and fixing kexec was a good first step.
That is probably caused by HP's IOMMU engine not shutting down at crash time....,
Please try add machvec=dig option for the crash dump kernel.
However we still need to find a light way to correctly shutdown IOMMU engine on those HP platforms.
Thanks
Zou Nan hai
Quoted 4 lines
Thanks for the help!
/ magnus
Quoted 22 lines · Magnus Damm
Hi Nan hai,
Quoted 19 lines · Nanhai
Quoted 5 lines · Jay Lan
Quoted 4 lines
Quoted 3 lines · Magnus Damm
kexec: Avoid migration of already disabled irqs (ia64)
This patch fixes up ia64 kexec support for HP rx2620 hardware.
It does this
Quoted 1 line
Quoted 1 line
Quoted 1 line
by skipping migration of already disabled irqs. This is most
likely a problem
Quoted 1 line
Quoted 1 line
Quoted 1 line
on other ia64 platforms as well, but I've only tested this on
one machine
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
so far.
I have not seen this problem on SN systems.
Ok, thanks. Let me give you more details.
When I perform "kexec -e" the following output appears on my serial console (with my patch applied).
Quoted 6 lines
Quoted 3 lines
This patch is correct I think. I assume you will also see this bug
when trying to offline a CPU by echo 0 >/sys/devices/system/cpu/cpuX/onlineYou are right, I can trigger the bug that way too. And the bug goes away with the patch. Excellent!
I tried on a 2p SN system with both '-l' followed by '-e' and also Nan-hai's trigger command. Neither triggered the problem. But, on the other hand, the system worked fine with your patch also. :)
Thanks,
- jay
Quoted 13 lines
Quoted 5 lines
However it will not be triggered in crash dump case which has been tested heavily on different kind of platforms. We do not migrate IRQ at the time of crash. I guess that is why Jay has never seen it on SN platforms.
Booting into a crash kernel does not seem to work on this ia64 box unfortunately. I'm about to investigate why, but I thought trying and fixing kexec was a good first step.
Thanks for the help!
/ magnus
Re: [Fastboot] [PATCH] kexec: Avoid migration of already disabled irqs (ia64)
In reply to Jay LanQuoted 8 lines · Nanhai
Quoted 5 lines
From: Magnus Damm [mailto:[email protected]]
Booting into a crash kernel does not seem to work on this ia64 box unfortunately. I'm about to investigate why, but I thought trying and fixing kexec was a good first step.That is probably caused by HP's IOMMU engine not shutting down at crash time....,
Please try add machvec=dig option for the crash dump kernel.
However we still need to find a light way to correctly shutdown IOMMU engine on those HP platforms.
It does unfortunately not seem to help. Regardless of SMP or UP kernel.
I'll try to track down where things lock up. Thanks.
/ magnus