Skip to contentSkip to conversation

Fix for broken kexec on panic

PATCH In review Vivek Goyal 5 participants 7 messages Feb 24, 2005

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Fix for broken kexec on panic

Hi,

Kexec on panic is broken on i386 in 2.6.11-rc3-mm2 because of re-organization of boot memory allocator initialization code. Primary kernel does not boot if kexec is enabled and crashkernel=X@Y command line parameter is passed. After re-organization, kexec is trying to call reserve_bootmem before boot memory allocator has initialized.

This patch fixes the problem. I have moved the call to reserved_bootmem() for kexec for both discontig and contig memory into new setup_bootmem_allocator().

This patch has been generated against 2.6.11-rc4-mm1

Thanks
Vivek

Kexec on panic is broken on i386 in 2.6.11-rc3-mm2 because of re-organisation of boot memory allocator code. Primary kernel does not boot if kexec is enabled and crashkernel=X@Y command line parameter is passed. After re-organization, kexec is trying to call reserve_bootmem before boot memory allocator has initialized.
This patch fixes the problem. I have moved the call to reserved_bootmem() for kexec for both discontig and contig memory into new setup_bootmem_allocator().

---

 linux-2.6.11-rc4-mm1-root/arch/i386/kernel/setup.c |   10 +++++-----
 linux-2.6.11-rc4-mm1-root/arch/i386/mm/discontig.c |    5 -----
 2 files changed, 5 insertions(+), 10 deletions(-)
arch/i386/kernel/setup.c renamed from arch/i386/kernel/setup.c~kexec-reserve-bootmem-fix +5 -5
diff -puN arch/i386/kernel/setup.c~kexec-reserve-bootmem-fix arch/i386/kernel/setup.c
@@ -987,11 +987,6 @@ unsigned long __init find_max_low_pfn(vo
 			printk(KERN_ERR "ignoring highmem size on non-highmem kernel!\n");
 #endif
 	}
-#ifdef CONFIG_KEXEC
-	if (crashk_res.start != crashk_res.end)
-		reserve_bootmem(crashk_res.start,
-			crashk_res.end - crashk_res.start + 1);
-#endif
 	return max_low_pfn;
 }
 
@@ -1174,6 +1169,11 @@ void __init setup_bootmem_allocator(void
 		}
 	}
 #endif
+#ifdef CONFIG_KEXEC
+	if (crashk_res.start != crashk_res.end)
+		reserve_bootmem(crashk_res.start,
+			crashk_res.end - crashk_res.start + 1);
+#endif
 }
 
 /*
arch/i386/mm/discontig.c renamed from arch/i386/mm/discontig.c~kexec-reserve-bootmem-fix +0 -5
diff -puN arch/i386/mm/discontig.c~kexec-reserve-bootmem-fix arch/i386/mm/discontig.c
@@ -265,11 +265,6 @@ unsigned long __init setup_memory(void)
 		find_max_pfn_node(nid);
 
 	NODE_DATA(0)->bdata = &node0_bdata;
-#ifdef CONFIG_KEXEC
-	if (crashk_res.start != crashk_res.end)
-		reserve_bootmem(crashk_res.start,
-			crashk_res.end - crashk_res.start + 1);
-#endif
 	setup_bootmem_allocator();
 	return max_low_pfn;
 }

_

#1 lore raw
Quoted 3 lines · Vivek Goyal

Kexec on panic is broken on i386 in 2.6.11-rc3-mm2 because of
re-organization of boot memory allocator initialization code.

OK...

Where are we up to with these patches, btw? Do you consider them close-to-complete? Do you have a feel for what proportion of machines will work correctly?

#2 lore raw
Quoted 10 lines · Andrew Morton
Quoted 3 lines · Vivek Goyal

Kexec on panic is broken on i386 in 2.6.11-rc3-mm2 because of
re-organization of boot memory allocator initialization code.

OK...

Where are we up to with these patches, btw? Do you consider them close-to-complete? Do you have a feel for what proportion of machines will work correctly?

After the rework of kexec patches, there is very minimal kernel code needed for kdump and most of the code is in user space kexec-tools. The changes needed in kexec-tools to load the crashdump kernel and generate ELF headers, for x86 architecture are done and will be posted for comments today by Vivek.

Currently the work remaining is to capture the old-kernel memory during second kernel boot up. There is some lack of consensus whether this functionality should go in kernel-space (/proc/vmcore) or user-space (a separate utility which can be run from initrd). Before the last kexec rework, kdump has the facility to do /proc/vmcore and now it has to be re-done accordingly. There is some code already done by Eric to do it in user-space. We are evaluating both the approaches and should arrive at the conclusion asap.

Thanks
Maneesh

Signature · 6 lines
Maneesh Soni
Linux Technology Center, 
IBM India Software Labs,
Bangalore, India
email: [email protected]
Phone: 91-80-25044990
#3 lore raw
Quoted 8 lines · Maneesh Soni
Quoted 7 lines · Andrew Morton
Quoted 3 lines · Vivek Goyal

Kexec on panic is broken on i386 in 2.6.11-rc3-mm2 because of
re-organization of boot memory allocator initialization code.

OK...

Quoted 8 lines
Quoted 3 lines

Where are we up to with these patches, btw? Do you consider them close-to-complete? Do you have a feel for what proportion of machines will work correctly?

After the rework of kexec patches, there is very minimal kernel code needed for kdump and most of the code is in user space kexec-tools. The changes needed in kexec-tools to load the crashdump kernel and generate ELF headers, for x86 architecture are done and will be posted for comments today by Vivek.

Cool.

Quoted 7 lines

Currently the work remaining is to capture the old-kernel memory during second kernel boot up. There is some lack of consensus whether this functionality should go in kernel-space (/proc/vmcore) or user-space (a separate utility which can be run from initrd). Before the last kexec rework, kdump has the facility to do /proc/vmcore and now it has to be re-done accordingly. There is some code already done by Eric to do it in user-space. We are evaluating both the approaches and should arrive at the conclusion asap.

Do you have a pointer to your user space kdump stuff? I have never seen it.

How to configure this and the usability issues are interesting. There is no fundamental reason the code needs to live in a ramdisk. We are back in a fully functional kernel after all. In this case a ramdisk/initramfs is useful for the same reason a ramdisk with a rescue disk is useful. It is possible the normal root filesystem is
corrupt. A ramdisk allows you to have a known good copy of your
tools.

Eric

#4 lore raw
Quoted 29 lines · Eric W. Biederman
Quoted 8 lines · Maneesh Soni
Quoted 7 lines · Andrew Morton
Quoted 3 lines · Vivek Goyal

Kexec on panic is broken on i386 in 2.6.11-rc3-mm2 because of
re-organization of boot memory allocator initialization code.

OK...

Quoted 8 lines
Quoted 3 lines

Where are we up to with these patches, btw? Do you consider them close-to-complete? Do you have a feel for what proportion of machines will work correctly?

After the rework of kexec patches, there is very minimal kernel code needed for kdump and most of the code is in user space kexec-tools. The changes needed in kexec-tools to load the crashdump kernel and generate ELF headers, for x86 architecture are done and will be posted for comments today by Vivek.

Cool.

Quoted 7 lines

Currently the work remaining is to capture the old-kernel memory during second kernel boot up. There is some lack of consensus whether this functionality should go in kernel-space (/proc/vmcore) or user-space (a separate utility which can be run from initrd). Before the last kexec rework, kdump has the facility to do /proc/vmcore and now it has to be re-done accordingly. There is some code already done by Eric to do it in user-space. We are evaluating both the approaches and should arrive at the conclusion asap.

Quoted 2 lines

Do you have a pointer to your user space kdump stuff? I have never seen it.

Actually I meant user space utility done by you. IMO, /proc/vmcore option as saves one from the hassel of a _new_ user space tool to configure / capture crash dumps. Though a user-space tool in addition to /proc/vmcore can be useful also in a badly crashed system.

Quoted 9 lines

How to configure this and the usability issues are interesting. There is no fundamental reason the code needs to live in a ramdisk. We are back in a fully functional kernel after all. In this case a ramdisk/initramfs is useful for the same reason a ramdisk with a rescue disk is useful. It is possible the normal root filesystem is
corrupt. A ramdisk allows you to have a known good copy of your
tools.

Eric

Signature · 6 lines
Maneesh Soni
Linux Technology Center, 
IBM India Software Labs,
Bangalore, India
email: [email protected]
Phone: 91-80-25044990
#5 lore raw
In reply to Andrew Morton
Quoted 8 lines · Andrew Morton
Quoted 3 lines · Vivek Goyal

Kexec on panic is broken on i386 in 2.6.11-rc3-mm2 because of
re-organization of boot memory allocator initialization code.

OK...

Where are we up to with these patches, btw?

Currently we are in the middle of a reorganization that places more of the work in user space, making the solution more robust.

Quoted 1 line

Do you consider them close-to-complete?

If you consider complete working code yes. Design wise I believe we are complete except proving out the pieces.

Off the top of my head the current todo looks like:
- kexec on panic user space preparation Basically generating ELF headers of a CORE dump before we crash.

- Moving ioapic setup on x86 and x86_64 into init_IRQs, where it belongs. Currently when using IOAPICs we use them for everything except calibrating the delay loop. Where we assume the legacy interrupt controller is setup. When initiating kexec from panic() we don't do a clean shutdown so that is not the case and SMP is broken.

Once those two pieces are in place and tested we should be able to drop all of the crashdump-* patches. As well as the crash_shutdown patches that touch the apics.

Of course there will still be lots of pieces left to make the drivers
more robust, and to make things easier to use. But all of those
are evolutionary improvements, not core architecture things.

Quoted 2 lines

Do you have a feel for what proportion of machines will work correctly?

Yes.

For the non panic case I need to get ioapic virtual wire setup working on ACPI systems. And I need to fix the ACPI using interrupts on the shutdown path bug.

Most device drivers work without being touched.

So we should work fine on x86 and x86_64 systems using either the legacy interrupt controller or IOAPICs.

So it should be most of x86 and x86_64 systems, working
with no more effort than I have described. And as device
drivers are fixed even more systems.

In addition there is nothing non-portable about the architecture, although avoiding firmware calls on non-embedded ports can be a very challenging modification to the boot process. So we should be able to start picking up most machines on other architectures like ppc, ppc64 and ia64 as soon the ports are complete.

Quoted 1 line

From a user perspective things are going to be rough for a while

as things all the final kinks get worked out. But things will be largely usable and we should be able to get usable bug reports. Of course that is the level where we start seeing a whole new class of bug reports :)

The biggest theoretical gotcha are systems with hot plug memory. As we are memorizing the memory map and storing it in a safe along with the recovery kernel before a crash occurs.

Basically we are in pretty good shape except for systems like an SGI-Altix or an IBM-s390. Hot-plug memory, multi-terabyte core dump sizes, and weird SMP architectures are problems that we don't/won't cope with well. With only hotplug memory requiring a modification of responsibilities to handle.

Eric

#6 lore raw
In reply to Vivek Goyal
Quoted 11 lines · Vivek Goyal

Kexec on panic is broken on i386 in 2.6.11-rc3-mm2 because of re-organization of boot memory allocator initialization code. Primary kernel does not boot if kexec is enabled and crashkernel=X@Y command line parameter is passed. After re-organization, kexec is trying to call reserve_bootmem before boot memory allocator has initialized.

This patch fixes the problem. I have moved the call to reserved_bootmem() for kexec for both discontig and contig memory into new setup_bootmem_allocator().

This patch has been generated against 2.6.11-rc4-mm1

Looks like a good change, especially since it reduces the total amount of code (and the size of your patch).

Although, to make any potential merging easier, it is almost always better to put those kinds of things in functions #ifdef'd in a header. The fact that there are other #ifdefs in setup_bootmem_allocator() is a partial excuse, but not a very good one. :)

-- Dave