overview
Subsystems (16)
cover overview
Andrew the following patchset is against 2.6.11-rc1-mm1 with all of the kexec patches removed. The list of removed patches is included below.
This patchsset is a major refresh of the kexec on panic functionality in the kernel. The primary aim of which was to take the requirements capture of the kernel crashdump patches and start integrating the functionality cleanly into the kexec patches.
Major accomplishments:
- Compat syscall support has been added.
- The crashdump capture code has been separated from the kexec on panic code.
- The kernel to jump to on panic is now loaded in place.
- A long standing bug that allowed 2 sources pages to copy data
to a single destination page has been caught and fixed.
- Support for loading an x86_64 kernel in a reserved of memory has been completed.
The crashdump code is currently slightly broken. I have attempted to minimize the breakage so things can quick be made to work again.
With respect to a final design discussion there are two remaining open issues. The first is how little hardware shutdown we can get away with in the kernel that is panicing. I believe we can reduce this to a simply NMI to the other cpus telling them to stop. This has been address as a major concern in previous conversations.
The second is an issue is the most significant with respect to the design of a kernel based crash dump capture implementation. How does the crashdump capture process discover relevant information about the kernel that just crashed? There are two options.
1) As represented by the current crashdump patches the crashdump kernel and the kernel in which it loads are kept in sync so that it has uptodate versions all of crashed kernels data structures because it is built from the same source. So it only needs to find the address of the data structures it would like to look at.
2) The relevant information if it is available when sys_kexec_load is called is exported to user space, or the machine_crash_shutdown method marshalls what little information must be captured when the machine dies in a well known standard format (most likely ELF notes). Allowing the crashdump capture process to simply pass on the information or utilize it as appropriate.
If the second method can successfully represent all of the interesting information then we can allow kernel version skew, between the two kernels, and potentially implement the entire crash dump capture process in user space.
As best as I have been able to discover the interesting information includes. The cpu state (registers) at the time of the crash/panic. The list of memory regions the kernel that has crashed was using. And potentially the list of pages dedicated to kernel data as opposed to user space, so the the people with insane amounts of memory (1TB+) don't require unmanagely large core files.
Andrew earlier when asked about the possibility of merghing the kexec
Quoted 6 lines · on panic code you
I don't want us to be in a position of merging all that code and then finding out that it cannot be made to work "sufficiently well", forcing us to revert it and find a new crashdump solution. You guys know far better than I when we will reach that threshold. If the kexec/dump developers can say "yup, this is going to work (because X)" then I'm happy.
So here is my subjective view.
- This code needs to sit in a development tree for a little while to shake out whatever bugs still linger from my massive refactoring.
- Through the kexec patches the code and design appears to be sound. Given that machine_kexec is little more than a jump there are few possible implementations that will be able to use it. The only exception I can see are running special dump drivers from the kernel that crashed, and I believe no one thinks the that will work well.
- Once we finish sorting out the best way to get information out of the kernel that crashed I think we will have a complete architecture that is largely portable to any architecture.
In the interests of full disclosure my main interesting is using the kernel as a bootloader for other kernels and that has been working fairly for years now :)
Eric
# Patches to remove from 2.6.11-rc1-mm1 before applying this patchset:
#
assign_irq_vector-section-fix.patch kexec-i8259-shutdowni386.patch kexec-i8259-shutdown-x86_64.patch kexec-apic-virtwire-on-shutdowni386patch.patch kexec-apic-virtwire-on-shutdownx86_64.patch kexec-ioapic-virtwire-on-shutdowni386.patch kexec-apic-virt-wire-fix.patch kexec-ioapic-virtwire-on-shutdownx86_64.patch kexec-e820-64bit.patch
kexec-kexec-generic.patch
kexec-ide-spindown-fix.patch
kexec-ifdef-cleanup.patch
kexec-machine_shutdownx86_64.patch
kexec-kexecx86_64.patch
kexec-kexecx86_64-4level-fix.patch kexec-kexecx86_64-4level-fix-unfix.patch kexec-machine_shutdowni386.patch
kexec-kexeci386.patch
kexec-use_mm.patch kexec-loading-kernel-from-non-default-offset.patch kexec-loading-kernel-from-non-default-offset-fix.patch kexec-enabling-co-existence-of-normal-kexec-kernel-and-panic-kernel.patch kexec-ppc-support.patch
#kexec-kexecppc.patch
#
crashdump-documentation.patch crashdump-memory-preserving-reboot-using-kexec.patch crashdump-memory-preserving-reboot-using-kexec-fix.patch kdump-config_discontigmem-fix.patch crashdump-routines-for-copying-dump-pages.patch crashdump-routines-for-copying-dump-pages-kmap-fiddle.patch crashdump-kmap-build-fix.patch crashdump-register-snapshotting-before-kexec-boot.patch crashdump-elf-format-dump-file-access.patch crashdump-linear-raw-format-dump-file-access.patch crashdump-minor-bug-fixes-to-kexec-crashdump-code.patch crashdump-cleanups-to-the-kexec-based-crashdump-code.patch
#
x86-rename-apic_mode_exint.patch
x86-local-apic-fix.patch
#
PATCH 1/29 x86-rename-apic_mode_exint
From: "Maciej W. Rozycki" <[email protected]>
Rename APIC_MODE_EXINT to APIC_MODE_EXTINT - I think it should be named after what the mode is called in documentation.
From: "Eric W. Biederman" <[email protected]>
I have reduced this patch to just the name change in the header. And integrated the changes into the patches that add those lines. Otherwise I ran into some ugly dependencies.
---
asm-i386/apicdef.h | 2 +- asm-x86_64/apicdef.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
include/asm-i386/apicdef.h +1 -1
@@ -90,7 +90,7 @@#define SET_APIC_DELIVERY_MODE (x,y) (((x)&~0x700)|((y)<<8))#define APIC_MODE_FIXED 0x0#define APIC_MODE_NMI 0x4 -#define APIC_MODE_EXINT 0x7 +#define APIC_MODE_EXTINT 0x7#define APIC_LVT1 0x360#define APIC_LVTERR 0x370#define APIC_TMICT 0x380
include/asm-x86_64/apicdef.h +1 -1
@@ -94,7 +94,7 @@#define SET_APIC_DELIVERY_MODE (x,y) (((x)&~0x700)|((y)<<8))#define APIC_MODE_FIXED 0x0#define APIC_MODE_NMI 0x4 -#define APIC_MODE_EXINT 0x7 +#define APIC_MODE_EXTINT 0x7#define APIC_LVT1 0x360#define APIC_LVTERR 0x370#define APIC_TMICT 0x380
PATCH 2/29 x86-local-apic-fix
From: "Maciej W. Rozycki" <[email protected]>
Fix a kexec problem whcih causes local APIC detection failure.
The problem is detect_init_APIC() is called early, before the command line have been processed. Therefore "lapic" (and "nolapic") have not been seen, yet.
---
arch/i386/kernel/apic.c | 25 +++++-------------------- arch/i386/kernel/setup.c | 11 +++++++++++ include/asm-i386/apic.h | 13 +++++++++++++ 3 files changed, 29 insertions(+), 20 deletions(-)
arch/i386/kernel/apic.c +5 -20
@@ -41,6 +41,11 @@#include "io_ports.h" /* +* Knob to control our willingness to enable the local APIC. +*/ +int enable_local_apic__initdata =0 ;/* -1=force-disable, +1=force-enable */ + +/* * Debug level */ int apic_verbosity;
@@ -666,26 +671,6 @@* Detect and enable local APICs on non-SMP boards. * Original code written by Keir Fraser. */ - -/* -* Knob to control our willingness to enable the local APIC. -*/ -int enable_local_apic__initdata =0 ;/* -1=force-disable, +1=force-enable */ - -static int __init lapic_disable (char *str) -{ - enable_local_apic =-1 ; -clear_bit (X86_FEATURE_APIC , boot_cpu_data.x86_capability); -return 0 ; -} -__setup ("nolapic" , lapic_disable); - -static int __init lapic_enable (char *str) -{ - enable_local_apic =1 ; -return 0 ; -} -__setup ("lapic" , lapic_enable);static int __init apic_set_verbosity (char *str) {
arch/i386/kernel/setup.c +11 -0
@@ -41,6 +41,7 @@#include <linux/init.h> #include <linux/edd.h> #include <video/edid.h> +#include <asm/apic.h> #include <asm/e820.h> #include <asm/mpspec.h> #include <asm/setup.h>
@@ -813,6 +814,16 @@disable_ioapic_setup ();#endif /* CONFIG_X86_LOCAL_APIC */ #endif /* CONFIG_ACPI_BOOT */ + +#ifdef CONFIG_X86_LOCAL_APIC +/* enable local APIC */ + else if (!memcmp(from, "lapic", 5)) +lapic_enable (); + +/* disable local APIC */ + else if (!memcmp(from, "nolapic", 6)) +lapic_disable (); +#endif /* CONFIG_X86_LOCAL_APIC */ /* * highmem=size forces highmem to be exactly 'size' bytes.
include/asm-i386/apic.h +13 -0
@@ -5,6 +5,7 @@#include <linux/pm.h> #include <asm/fixmap.h> #include <asm/apicdef.h> +#include <asm/processor.h> #include <asm/system.h> #define Dprintk(x...)
@@ -16,7 +17,19 @@#define APIC_VERBOSE 1#define APIC_DEBUG 2 +extern int enable_local_apic;extern int apic_verbosity; + +static inline void lapic_disable (void ) +{ + enable_local_apic =-1 ; +clear_bit (X86_FEATURE_APIC , boot_cpu_data.x86_capability); +} + +static inline void lapic_enable (void ) +{ + enable_local_apic =1 ; +}/* * Define the default level of output to be very little
PATCH 3/29 x86_64-e820-64bit
From: Eric W. Biederman <[email protected]>
It is ok to reserve resources > 4G on x86_64 struct resource is 64bit now :)
---
e820.c | 2 -- 1 files changed, 2 deletions(-)
arch/x86_64/kernel/e820.c +0 -2
@@ -185,8 +185,6 @@int i;for (i =0 ; i < e820.nr_map; i++) {struct resource *res; -if (e820.map[i].addr + e820.map[i].size >0x100000000ULL ) -continue ; res =alloc_bootmem_low (sizeof (struct resource ));switch (e820.map[i].type) {case E820_RAM : res->name ="System RAM" ;break ;
PATCH 4/29 x86-i8259-shutdown
From: Eric W. Biederman <[email protected]>
This patch disables interrupt generation from the legacy pic on reboot. Now that there is a sys_device class it should not be called while drivers are still using interrupts.
There is a report about this breaking ACPI power off on some systems.
http://bugme.osdl.org/show_bug.cgi?id=4041
However the final comment seems to exhonorate this code. So until I get more information I believe that was a false positive.
---
i8259.c | 12 ++++++++++++ 1 files changed, 12 insertions(+)
arch/i386/kernel/i8259.c +12 -0
@@ -270,10 +270,22 @@return 0 ; } +static int i8259A_shutdown (struct sys_device *dev) +{ +/* Put the i8259A into a quiescent state that +* the kernel initialization code can get it +* out of. +*/ +outb (0xff ,0x21 );/* mask all of 8259A-1 */ +outb (0xff ,0xA1 );/* mask all of 8259A-1 */ +return 0 ; +} +static struct sysdev_class i8259_sysdev_class = {set_kset_name ("i8259" ), .suspend = i8259A_suspend, .resume = i8259A_resume, + .shutdown = i8259A_shutdown, };static struct sys_device device_i8259A = {
PATCH 5/29 x86_64-i8259-shutdown
From: Eric W. Biederman <[email protected]
The following patch simply adds a shutdown method to the x86_64 i8259 code.
---
i8259.c | 12 ++++++++++++ 1 files changed, 12 insertions(+)
arch/x86_64/kernel/i8259.c +12 -0
@@ -416,10 +416,22 @@return 0 ; } +static int i8259A_shutdown (struct sys_device *dev) +{ +/* Put the i8259A into a quiescent state that +* the kernel initialization code can get it +* out of. +*/ +outb (0xff ,0x21 );/* mask all of 8259A-1 */ +outb (0xff ,0xA1 );/* mask all of 8259A-1 */ +return 0 ; +} +static struct sysdev_class i8259_sysdev_class = {set_kset_name ("i8259" ), .suspend = i8259A_suspend, .resume = i8259A_resume, + .shutdown = i8259A_shutdown, };static struct sys_device device_i8259A = {
PATCH 6/29 x86-apic-virtwire-on-shutdown
When coming out of apic mode attempt to set the appropriate apic back into virtual wire mode. This improves on previous versions of this patch by by never setting bot the local apic and the ioapic into veritual wire mode.
This code looks at data from the mptable to see if an ioapic has an ExtInt input to make this decision. A future improvement is to figure out which apic or ioapic was in virtual wire mode at boot time and to remember it. That is potentially a more accurate method, of selecting which apic to place in virutal wire mode.
---
arch/i386/kernel/apic.c | 38 +++++++++++++++++++++++++++++++++++++- arch/i386/kernel/io_apic.c | 33 ++++++++++++++++++++++++++++++++- include/asm-i386/apic.h | 2 +- include/asm-i386/apicdef.h | 1 + 4 files changed, 71 insertions(+), 3 deletions(-)
arch/i386/kernel/apic.c +37 -1
@@ -211,7 +211,7 @@enable_apic_mode (); } -void disconnect_bsp_APIC (void ) +void disconnect_bsp_APIC (int virt_wire_setup) {if (pic_mode) {/*
@@ -224,6 +224,42 @@ "entering PIC mode.\n");outb (0x70 ,0x22 );outb (0x00 ,0x23 ); + } +else { +/* Go back to Virtual Wire compatibility mode */ +unsigned long value; + +/* For the spurious interrupt use vector F, and enable it */ + value =apic_read (APIC_SPIV ); + value &= ~APIC_VECTOR_MASK ; + value |=APIC_SPIV_APIC_ENABLED ; + value |=0xf ; +apic_write_around (APIC_SPIV , value); + +if (!virt_wire_setup) { +/* For LVT0 make it edge triggered, active high, external and enabled */ + value =apic_read (APIC_LVT0 ); + value &= ~(APIC_MODE_MASK |APIC_SEND_PENDING | +APIC_INPUT_POLARITY |APIC_LVT_REMOTE_IRR | +APIC_LVT_LEVEL_TRIGGER |APIC_LVT_MASKED ); + value |=APIC_LVT_REMOTE_IRR |APIC_SEND_PENDING ; + value =SET_APIC_DELIVERY_MODE (value,APIC_MODE_EXTINT ); +apic_write_around (APIC_LVT0 , value); + } +else { +/* Disable LVT0 */ +apic_write_around (APIC_LVT0 ,APIC_LVT_MASKED ); + } + +/* For LVT1 make it edge triggered, active high, nmi and enabled */ + value =apic_read (APIC_LVT1 ); + value &= ~( +APIC_MODE_MASK |APIC_SEND_PENDING | +APIC_INPUT_POLARITY |APIC_LVT_REMOTE_IRR | +APIC_LVT_LEVEL_TRIGGER |APIC_LVT_MASKED ); + value |=APIC_LVT_REMOTE_IRR |APIC_SEND_PENDING ; + value =SET_APIC_DELIVERY_MODE (value,APIC_MODE_NMI ); +apic_write_around (APIC_LVT1 , value); } }
arch/i386/kernel/io_apic.c +32 -1
@@ -1631,12 +1631,43 @@*/ void disable_IO_APIC (void ) { +int pin;/* * Clear the IO-APIC before rebooting: */ clear_IO_APIC (); -disconnect_bsp_APIC (); +/* +* If the i82559 is routed through an IOAPIC +* Put that IOAPIC in virtual wire mode +* so legacy interrups can be delivered. +*/ + pin =find_isa_irq_pin (0 , mp_ExtINT); +if (pin !=-1 ) { +struct IO_APIC_route_entry entry; +unsigned long flags; + +memset (&entry,0 ,sizeof (entry)); + entry.mask =0 ;/* Enabled */ + entry.trigger =0 ;/* Edge */ + entry.irr =0 ; + entry.polarity =0 ;/* High */ + entry.delivery_status =0 ; + entry.dest_mode =0 ;/* Physical */ + entry.delivery_mode =7 ;/* ExtInt */ + entry.vector =0 ; + entry.dest.physical.physical_dest =0 ; + + +/* +* Add it to the IO-APIC irq-routing table: +*/ +spin_lock_irqsave (&ioapic_lock, flags); +io_apic_write (0 ,0x11 +2 *pin, *(((int *)&entry)+1 )); +io_apic_write (0 ,0x10 +2 *pin, *(((int *)&entry)+0 )); +spin_unlock_irqrestore (&ioapic_lock, flags); + } +disconnect_bsp_APIC (pin !=-1 ); }/*
include/asm-i386/apic.h +1 -1
@@ -100,7 +100,7 @@extern int get_maxlvt (void );extern void clear_local_APIC (void );extern void connect_bsp_APIC (void ); -extern void disconnect_bsp_APIC (void ); +extern void disconnect_bsp_APIC (int virt_wire_setup);extern void disable_local_APIC (void );extern void lapic_shutdown (void );extern int verify_local_APIC (void );
include/asm-i386/apicdef.h +1 -0
@@ -86,6 +86,7 @@#define APIC_LVT_REMOTE_IRR (1<<14)#define APIC_INPUT_POLARITY (1<<13)#define APIC_SEND_PENDING (1<<12) +#define APIC_MODE_MASK 0x700#define GET_APIC_DELIVERY_MODE (x) (((x)>>8)&0x7)#define SET_APIC_DELIVERY_MODE (x,y) (((x)&~0x700)|((y)<<8))#define APIC_MODE_FIXED 0x0
PATCH 7/29 x86_64-apic-virtwire-on-shutdown
When coming out of apic mode attempt to set the appropriate apic back into virtual wire mode. This improves on previous versions of this patch by by never setting bot the local apic and the ioapic into veritual wire mode.
This code looks at data from the mptable to see if an ioapic has an ExtInt input to make this decision. A future improvement is to figure out which apic or ioapic was in virtual wire mode at boot time and to remember it. That is potentially a more accurate method, of selecting which apic to place in virutal wire mode.
---
arch/x86_64/kernel/apic.c | 38 +++++++++++++++++++++++++++++++++++++- arch/x86_64/kernel/io_apic.c | 36 ++++++++++++++++++++++++++++++++++-- include/asm-x86_64/apic.h | 2 +- 3 files changed, 72 insertions(+), 4 deletions(-)
arch/x86_64/kernel/apic.c +37 -1
@@ -132,7 +132,7 @@ } } -void disconnect_bsp_APIC (void ) +void disconnect_bsp_APIC (int virt_wire_setup) {if (pic_mode) {/*
@@ -144,6 +144,42 @@apic_printk (APIC_QUIET ,"disabling APIC mode, entering PIC mode.\n" );outb (0x70 ,0x22 );outb (0x00 ,0x23 ); + } +else { +/* Go back to Virtual Wire compatibility mode */ +unsigned long value; + +/* For the spurious interrupt use vector F, and enable it */ + value =apic_read (APIC_SPIV ); + value &= ~APIC_VECTOR_MASK ; + value |=APIC_SPIV_APIC_ENABLED ; + value |=0xf ; +apic_write_around (APIC_SPIV , value); + +if (!virt_wire_setup) { +/* For LVT0 make it edge triggered, active high, external and enabled */ + value =apic_read (APIC_LVT0 ); + value &= ~(APIC_MODE_MASK |APIC_SEND_PENDING | +APIC_INPUT_POLARITY |APIC_LVT_REMOTE_IRR | +APIC_LVT_LEVEL_TRIGGER |APIC_LVT_MASKED ); + value |=APIC_LVT_REMOTE_IRR |APIC_SEND_PENDING ; + value =SET_APIC_DELIVERY_MODE (value,APIC_MODE_EXTINT ); +apic_write_around (APIC_LVT0 , value); + } +else { +/* Disable LVT0 */ +apic_write_around (APIC_LVT0 ,APIC_LVT_MASKED ); + } + +/* For LVT1 make it edge triggered, active high, nmi and enabled */ + value =apic_read (APIC_LVT1 ); + value &= ~( +APIC_MODE_MASK |APIC_SEND_PENDING | +APIC_INPUT_POLARITY |APIC_LVT_REMOTE_IRR | +APIC_LVT_LEVEL_TRIGGER |APIC_LVT_MASKED ); + value |=APIC_LVT_REMOTE_IRR |APIC_SEND_PENDING ; + value =SET_APIC_DELIVERY_MODE (value,APIC_MODE_NMI ); +apic_write_around (APIC_LVT1 , value); } }
arch/x86_64/kernel/io_apic.c +34 -2
@@ -327,7 +327,7 @@/* * Find the pin to which IRQ[irq] (ISA) is connected */ -static int __init find_isa_irq_pin (int irq,int type) +static int find_isa_irq_pin (int irq,int type) {int i;
@@ -1125,12 +1125,44 @@*/ void disable_IO_APIC (void ) { +int pin;/* * Clear the IO-APIC before rebooting: */ clear_IO_APIC (); -disconnect_bsp_APIC (); +/* +* If the i82559 is routed through an IOAPIC +* Put that IOAPIC in virtual wire mode +* so legacy interrups can be delivered. +*/ + pin =find_isa_irq_pin (0 , mp_ExtINT); +if (pin !=-1 ) { +struct IO_APIC_route_entry entry; +unsigned long flags; + +memset (&entry,0 ,sizeof (entry)); + entry.mask =0 ;/* Enabled */ + entry.trigger =0 ;/* Edge */ + entry.irr =0 ; + entry.polarity =0 ;/* High */ + entry.delivery_status =0 ; + entry.dest_mode =0 ;/* Physical */ + entry.delivery_mode =7 ;/* ExtInt */ + entry.vector =0 ; + entry.dest.physical.physical_dest =0 ; + + +/* +* Add it to the IO-APIC irq-routing table: +*/ +spin_lock_irqsave (&ioapic_lock, flags); +io_apic_write (0 ,0x11 +2 *pin, *(((int *)&entry)+1 )); +io_apic_write (0 ,0x10 +2 *pin, *(((int *)&entry)+0 )); +spin_unlock_irqrestore (&ioapic_lock, flags); + } + +disconnect_bsp_APIC (pin !=-1 ); }/*
include/asm-x86_64/apic.h +1 -1
@@ -77,7 +77,7 @@extern int get_maxlvt (void );extern void clear_local_APIC (void );extern void connect_bsp_APIC (void ); -extern void disconnect_bsp_APIC (void ); +extern void disconnect_bsp_APIC (int virt_wire_setup);extern void disable_local_APIC (void );extern int verify_local_APIC (void );extern void cache_APIC_registers (void );
PATCH 8/29 vmlinux-fix-physical-addrs
In vmlinux.lds.h the code is carefull to define every section so vmlinux properly reports the correct physical load address of code, as well as it's virtual address.
The new SECURITY_INIT definition fails to follow that convention and and causes incorrect physical address to appear in the vmlinux if there are any security initcalls.
This patch updates the SECURITY_INIT to follow the convention in the rest of the file.
---
vmlinux.lds.h | 2 +- 1 files changed, 1 insertion(+), 1 deletion(-)
include/asm-generic/vmlinux.lds.h +1 -1
@@ -73,7 +73,7 @@ }#define SECURITY_INIT \ - .security_initcall.init : { \ + .security_initcall.init : AT(ADDR(.security_initcall.init) - LOAD_OFFSET) { \ VMLINUX_SYMBOL(__security_initcall_start) = .; \ *(.security_initcall.init) \ VMLINUX_SYMBOL(__security_initcall_end) = .; \
PATCH 9/29 x86-vmlinux-fix-physical-addrs
The vmlinux on i386 does not report the correct physical address of the kernel. Instead in the physical address field it currently reports the virtual address of the kernel.
This is patch is a bug fix that corrects vmlinux to report the proper physical addresses.
This is potentially a help for crash dump analysis tools.
This definitiely allows bootloaders that load vmlinux as a standard ELF executable. Bootloaders directly loading vmlinux become of practical importance when we consider the kexec on panic case.
---
vmlinux.lds.S | 59 +++++++++++++++++++++++++++++++++++++--------------------- 1 files changed, 38 insertions(+), 21 deletions(-)
arch/i386/kernel/vmlinux.lds.S +38 -21
@@ -2,20 +2,23 @@ * Written by Martin Mares <[email protected]>; */ +#define LOAD_OFFSET __PAGE_OFFSET + #include <asm-generic/vmlinux.lds.h> #include <asm/thread_info.h> #include <asm/page.h> OUTPUT_FORMAT("elf32-i386", "elf32-i386", "elf32-i386") OUTPUT_ARCH(i386) -ENTRY(startup_32) +ENTRY(phys_startup_32) jiffies = jiffies_64; SECTIONS { - . = __PAGE_OFFSET + 0x100000; + . = LOAD_OFFSET + 0x100000; + phys_startup_32 = startup_32 - LOAD_OFFSET; /* read-only */ _text = .; /* Text and read-only data */ - .text : { + .text : AT(ADDR(.text) - LOAD_OFFSET) { *(.text) SCHED_TEXT LOCK_TEXT
@@ -27,49 +30,55 @@ . = ALIGN(16); /* Exception table */ __start___ex_table = .; - __ex_table : { *(__ex_table) } + __ex_table : AT(ADDR(__ex_table) - LOAD_OFFSET) { *(__ex_table) } __stop___ex_table = .; RODATA /* writeable */ - .data : { /* Data */ + .data : AT(ADDR(.data) - LOAD_OFFSET) { /* Data */ *(.data) CONSTRUCTORS } . = ALIGN(4096); __nosave_begin = .; - .data_nosave : { *(.data.nosave) } + .data_nosave : AT(ADDR(.data_nosave) - LOAD_OFFSET) { *(.data.nosave) } . = ALIGN(4096); __nosave_end = .; . = ALIGN(4096); - .data.page_aligned : { *(.data.idt) } + .data.page_aligned : AT(ADDR(.data.page_aligned) - LOAD_OFFSET) { + *(.data.idt) + } . = ALIGN(32); - .data.cacheline_aligned : { *(.data.cacheline_aligned) } + .data.cacheline_aligned : AT(ADDR(.data.cacheline_aligned) - LOAD_OFFSET) { + *(.data.cacheline_aligned) + } _edata = .; /* End of data section */ . = ALIGN(THREAD_SIZE); /* init_task */ - .data.init_task : { *(.data.init_task) } + .data.init_task : AT(ADDR(.data.init_task) - LOAD_OFFSET) { + *(.data.init_task) + } /* will be freed after init */ . = ALIGN(4096); /* Init code and data */ __init_begin = .; - .init.text : { + .init.text : AT(ADDR(.init.text) - LOAD_OFFSET) { _sinittext = .; *(.init.text) _einittext = .; } - .init.data : { *(.init.data) } + .init.data : AT(ADDR(.init.data) - LOAD_OFFSET) { *(.init.data) } . = ALIGN(16); __setup_start = .; - .init.setup : { *(.init.setup) } + .init.setup : AT(ADDR(.init.setup) - LOAD_OFFSET) { *(.init.setup) } __setup_end = .; __initcall_start = .; - .initcall.init : { + .initcall.init : AT(ADDR(.initcall.init) - LOAD_OFFSET) { *(.initcall1.init) *(.initcall2.init) *(.initcall3.init)
@@ -80,33 +89,41 @@ } __initcall_end = .; __con_initcall_start = .; - .con_initcall.init : { *(.con_initcall.init) } + .con_initcall.init : AT(ADDR(.con_initcall.init) - LOAD_OFFSET) { + *(.con_initcall.init) + } __con_initcall_end = .; SECURITY_INIT . = ALIGN(4); __alt_instructions = .; - .altinstructions : { *(.altinstructions) } + .altinstructions : AT(ADDR(.altinstructions) - LOAD_OFFSET) { + *(.altinstructions) + } __alt_instructions_end = .; - .altinstr_replacement : { *(.altinstr_replacement) } + .altinstr_replacement : AT(ADDR(.altinstr_replacement) - LOAD_OFFSET) { + *(.altinstr_replacement) + } /* .exit.text is discard at runtime, not link time, to deal with references from .altinstructions and .eh_frame */ - .exit.text : { *(.exit.text) } - .exit.data : { *(.exit.data) } + .exit.text : AT(ADDR(.exit.text) - LOAD_OFFSET) { *(.exit.text) } + .exit.data : AT(ADDR(.exit.data) - LOAD_OFFSET) { *(.exit.data) } . = ALIGN(4096); __initramfs_start = .; - .init.ramfs : { *(.init.ramfs) } + .init.ramfs : AT(ADDR(.init.ramfs) - LOAD_OFFSET) { *(.init.ramfs) } __initramfs_end = .; . = ALIGN(32); __per_cpu_start = .; - .data.percpu : { *(.data.percpu) } + .data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { *(.data.percpu) } __per_cpu_end = .; . = ALIGN(4096); __init_end = .; /* freed after init ends here */ __bss_start = .; /* BSS */ - .bss : { + .bss.page_aligned : AT(ADDR(.bss.page_aligned) - LOAD_OFFSET) { *(.bss.page_aligned) + } + .bss : AT(ADDR(.bss) - LOAD_OFFSET) { *(.bss) } . = ALIGN(4);
PATCH 10/29 x86_64-vmlinux-fix-physical-addrs
The vmlinux on x86_64 does not report the correct physical address of the kernel. Instead in the physical address field it currently reports the virtual address of the kernel.
This is patch is a bug fix that corrects vmlinux to report the proper physical addresses.
This is potentially a help for crash dump analysis tools.
This definitiely allows bootloaders that load vmlinux as a standard ELF executable. Bootloaders directly loading vmlinux become of practical importance when we consider the kexec on panic case.
---
Makefile | 2 kernel/vmlinux.lds.S | 130 ++++++++++++++++++++++++++++++++++----------------- 2 files changed, 88 insertions(+), 44 deletions(-)
arch/x86_64/Makefile +1 -1
@@ -35,7 +35,7 @@ LDFLAGS := -m elf_x86_64 OBJCOPYFLAGS := -O binary -R .note -R .comment -S -LDFLAGS_vmlinux := -e stext +LDFLAGS_vmlinux := CHECKFLAGS += -D__x86_64__ -m64
arch/x86_64/kernel/vmlinux.lds.S +87 -43
217 lines not on this page · raw message
PATCH 11/29 x86_64-entry64
To enable bootloaders to boot to directly load the x86_64 vmlinux and to enable the x86_64 kernel to switch into 64bit mode earlier this patch refactors the x86_64 entry code so there is a native 64bit entry point to the kernel.
I ran this by Andi Kleen and he agreed it looks fairly sane.
---
head.S | 112 ++++++++++++++++++++++++++++++++-------------------------- smpboot.c | 2 - trampoline.S | 24 +++--------- vmlinux.lds.S | 3 + 4 files changed, 70 insertions(+), 71 deletions(-)
arch/x86_64/kernel/head.S +62 -50
@@ -26,6 +26,7 @@ .text .code32 + .globl startup_32 /* %bx: 1 if coming from smp trampoline on secondary cpu */ startup_32:
@@ -37,11 +38,13 @@ * There is no stack until we set one up. */ - movl %ebx,%ebp /* Save trampoline flag */ - + /* Initialize the %ds segment register */ movl $__KERNEL_DS,%eax movl %eax,%ds - + + /* Load new GDT with the 64bit segments using 32bit descriptor */ + lgdt pGDT32 - __START_KERNEL_map + /* If the CPU doesn't support CPUID this will double fault. * Unfortunately it is hard to check for CPUID without a stack. */
@@ -57,16 +60,13 @@ btl $29, %edx jnc no_long_mode - movl %edx,%edi - /* * Prepare for entering 64bits mode */ - /* Enable PAE mode and PGE */ + /* Enable PAE mode */ xorl %eax, %eax btsl $5, %eax - btsl $7, %eax movl %eax, %cr4 /* Setup early boot stage 4 level pagetables */
@@ -79,14 +79,6 @@ /* Enable Long Mode */ btsl $_EFER_LME, %eax - /* Enable System Call */ - btsl $_EFER_SCE, %eax - - /* No Execute supported? */ - btl $20,%edi - jnc 1f - btsl $_EFER_NX, %eax -1: /* Make changes effective */ wrmsr
@@ -94,38 +86,69 @@ xorl %eax, %eax btsl $31, %eax /* Enable paging and in turn activate Long Mode */ btsl $0, %eax /* Enable protected mode */ - btsl $1, %eax /* Enable MP */ - btsl $4, %eax /* Enable ET */ - btsl $5, %eax /* Enable NE */ - btsl $16, %eax /* Enable WP */ - btsl $18, %eax /* Enable AM */ /* Make changes effective */ movl %eax, %cr0 - jmp reach_compatibility_mode -reach_compatibility_mode: - /* * At this point we're in long mode but in 32bit compatibility mode * with EFER.LME = 1, CS.L = 0, CS.D = 1 (and in turn - * EFER.LMA = 1). Now we want to jump in 64bit mode, to do that we load + * EFER.LMA = 1). Now we want to jump in 64bit mode, to do that we use * the new gdt/idt that has __KERNEL_CS with CS.L = 1. */ - - testw %bp,%bp /* secondary CPU? */ - jnz second - - /* Load new GDT with the 64bit segment using 32bit descriptor */ - movl $(pGDT32 - __START_KERNEL_map), %eax - lgdt (%eax) - -second: - movl $(ljumpvector - __START_KERNEL_map), %eax - /* Finally jump in 64bit mode */ - ljmp *(%eax) + ljmp $__KERNEL_CS, $(startup_64 - __START_KERNEL_map) .code64 .org 0x100 -reach_long64: + .globl startup_64 +startup_64: + /* We come here either from startup_32 + * or directly from a 64bit bootloader. + * Since we may have come directly from a bootloader we + * reload the page tables here. + */ + + /* Enable PAE mode and PGE */ + xorq %rax, %rax + btsq $5, %rax + btsq $7, %rax + movq %rax, %cr4 + + /* Setup early boot stage 4 level pagetables. */ + movq $(init_level4_pgt - __START_KERNEL_map), %rax + movq %rax, %cr3 + + /* Check if nx is implemented */ + movl $0x80000001, %eax + cpuid + movl %edx,%edi + + /* Setup EFER (Extended Feature Enable Register) */ + movl $MSR_EFER, %ecx + rdmsr + + /* Enable System Call */ + btsl $_EFER_SCE, %eax + + /* No Execute supported? */ + btl $20,%edi + jnc 1f + btsl $_EFER_NX, %eax +1: + /* Make changes effective */ + wrmsr + + /* Setup cr0 */ + xorq %rax, %rax + btsq $31, %rax /* Enable paging */ + btsq $0, %rax /* Enable protected mode */ + btsq $1, %rax /* Enable MP */ + btsq $4, %rax /* Enable ET */ + btsq $5, %rax /* Enable NE */ + btsq $16, %rax /* Enable WP */ + btsq $18, %rax /* Enable AM */ + /* Make changes effective */ + movq %rax, %cr0 + + /* Setup a boot time stack */ movq init_rsp(%rip),%rsp /* zero EFLAGS after setting rsp */
@@ -198,13 +221,8 @@ .org 0xf00 .globl pGDT32 pGDT32: - .word gdt32_end-gdt_table32 - .long gdt_table32-__START_KERNEL_map - -.org 0xf10 -ljumpvector: - .long reach_long64-__START_KERNEL_map - .word __KERNEL_CS + .word gdt_end-cpu_gdt_table + .long cpu_gdt_table-__START_KERNEL_map ENTRY(stext) ENTRY(_stext)
@@ -334,12 +352,6 @@ .endr #endif -ENTRY(gdt_table32) - .quad 0x0000000000000000 /* This one is magic */ - .quad 0x0000000000000000 /* unused */ - .quad 0x00af9a000000ffff /* __KERNEL_CS */ -gdt32_end: - /* We need valid kernel segments for data and code in long mode too * IRET will check the segment types kkeil 2000/10/28 * Also sysret mandates a special GDT layout
arch/x86_64/kernel/smpboot.c +0 -2
@@ -91,8 +91,6 @@static unsigned long __init setup_trampoline (void ) {void *tramp =__va (SMP_TRAMPOLINE_BASE ); -extern volatile __u32 tramp_gdt_ptr; - tramp_gdt_ptr =__pa_symbol (&cpu_gdt_table);memcpy (tramp, trampoline_data, trampoline_end - trampoline_data);return virt_to_phys (tramp); }
arch/x86_64/kernel/trampoline.S +6 -18
@@ -37,7 +37,6 @@ mov %cs, %ax # Code and data in the same place mov %ax, %ds - mov $1, %bx # Flag an SMP trampoline cli # We should be safe anyway movl $0xA5A5A5A5, trampoline_data - r_base
@@ -46,31 +45,20 @@ lidt idt_48 - r_base # load idt with 0, 0 lgdt gdt_48 - r_base # load gdt with whatever is appropriate - movw $__KERNEL_DS,%ax - movw %ax,%ds - movw %ax,%es - xor %ax, %ax inc %ax # protected mode (PE) bit lmsw %ax # into protected mode - jmp flush_instr -flush_instr: - ljmpl $__KERNEL32_CS, $0x00100000 - # jump to startup_32 in arch/x86_64/kernel/head.S - + # flaush prefetch and jump to startup_32 in arch/x86_64/kernel/head.S + ljmpl $__KERNEL32_CS, $(startup_32-__START_KERNEL_map) + + # Careful these need to be in the same 64K segment as the above; idt_48: .word 0 # idt limit = 0 .word 0, 0 # idt base = 0L gdt_48: - .short 0x0800 # gdt limit = 2048, 256 GDT entries - .globl tramp_gdt_ptr -tramp_gdt_ptr: - .long 0 # gdt base = gdt (first SMP CPU) - # this is filled in by C because the 64bit - # linker doesn't support absolute 32bit - # relocations. - + .short __KERNEL32_CS + 7 # gdt limit + .long cpu_gdt_table-__START_KERNEL_map .globl trampoline_end trampoline_end:
arch/x86_64/kernel/vmlinux.lds.S +2 -1
@@ -10,11 +10,12 @@ OUTPUT_FORMAT("elf64-x86-64", "elf64-x86-64", "elf64-x86-64") OUTPUT_ARCH(i386:x86-64) -ENTRY(stext) +ENTRY(phys_startup_64) jiffies_64 = jiffies; SECTIONS { . = __START_KERNEL; + phys_startup_64 = startup_64 - LOAD_OFFSET; _text = .; /* Text and read-only data */ .text : AT(ADDR(.text) - LOAD_OFFSET) { *(.text)
PATCH 12/29 x86-config-kernel-start
For one kernel to report a crash another kernel has created we need to have 2 kernels loaded simultaneously in memory. To accomplish this the two kernels need to built to run at different physical addresses.
This patch adds the CONFIG_PHYSICAL_START option to the x86 kernel so we can do just that. You need to know what you are doing and the ramifications are before changing this value, and most users won't care so I have made it depend on CONFIG_EMBEDDED
bzImage kernels will work and run at a different address when compiled with this option but they will still load at 1MB. If you need a kernel loaded at a different address as well you need to boot a vmlinux.
---
arch/i386/Kconfig | 11 +++++++++++ arch/i386/boot/compressed/head.S | 7 ++++--- arch/i386/boot/compressed/misc.c | 7 ++++--- arch/i386/kernel/vmlinux.lds.S | 2 +- include/asm-i386/page.h | 3 +++ 5 files changed, 23 insertions(+), 7 deletions(-)
arch/i386/Kconfig +11 -0
@@ -890,6 +890,17 @@ source "drivers/perfctr/Kconfig" +config PHYSICAL_START + hex "Physical address where the kernel is loaded" if EMBEDDED + default "0x100000" + help + This gives the physical address where the kernel is loaded. + Primarily used in the case of kexec on panic where the + fail safe kernel needs to run at a different address than + the panic-ed kernel. + + Don't change this unless you know what you are doing. + endmenu
arch/i386/boot/compressed/head.S +4 -3
@@ -25,6 +25,7 @@ #include <linux/linkage.h> #include <asm/segment.h> +#include <asm/page.h> .globl startup_32
@@ -74,7 +75,7 @@ popl %esi # discard address popl %esi # real mode pointer xorl %ebx,%ebx - ljmp $(__BOOT_CS), $0x100000 + ljmp $(__BOOT_CS), $__PHYSICAL_START /* * We come here, if we were loaded high.
@@ -99,7 +100,7 @@ popl %ecx # lcount popl %edx # high_buffer_start popl %eax # hcount - movl $0x100000,%edi + movl $__PHYSICAL_START,%edi cli # make sure we don't get interrupted ljmp $(__BOOT_CS), $0x1000 # and jump to the move routine
@@ -124,5 +125,5 @@ movsl movl %ebx,%esi # Restore setup pointer xorl %ebx,%ebx - ljmp $(__BOOT_CS), $0x100000 + ljmp $(__BOOT_CS), $__PHYSICAL_START move_routine_end:
arch/i386/boot/compressed/misc.c +4 -3
@@ -14,6 +14,7 @@#include <linux/tty.h> #include <video/edid.h> #include <asm/io.h> +#include <asm/page.h> /* * gzip declarations
@@ -309,7 +310,7 @@#else if ((RM_ALT_MEM_K >RM_EXT_MEM_K ?RM_ALT_MEM_K :RM_EXT_MEM_K ) <1024 )error ("Less than 2MB of memory" );#endif - output_data = (char *)0x100000 ;/* Points to 1M */ + output_data = (char *)__PHYSICAL_START;/* Normally Points to 1M */ free_mem_end_ptr = (long )real_mode; }
@@ -334,8 +335,8 @@ low_buffer_size = low_buffer_end -LOW_BUFFER_START ; high_loaded =1 ; free_mem_end_ptr = (long )high_buffer_start; -if ( (0x100000 + low_buffer_size) > ((ulg )high_buffer_start)) { - high_buffer_start = (uch *)(0x100000 + low_buffer_size); +if ( (__PHYSICAL_START + low_buffer_size) > ((ulg )high_buffer_start)) { + high_buffer_start = (uch *)(__PHYSICAL_START + low_buffer_size); mv->hcount =0 ;/* say: we need not to move high_buffer */ }else mv->hcount =-1 ;
arch/i386/kernel/vmlinux.lds.S +1 -1
@@ -14,7 +14,7 @@ jiffies = jiffies_64; SECTIONS { - . = LOAD_OFFSET + 0x100000; + . = __KERNEL_START; phys_startup_32 = startup_32 - LOAD_OFFSET; /* read-only */ _text = .; /* Text and read-only data */
include/asm-i386/page.h +3 -0
@@ -122,9 +122,12 @@#ifdef __ASSEMBLY__#define __PAGE_OFFSET (0xC0000000) +#define __PHYSICAL_START CONFIG_PHYSICAL_START#else #define __PAGE_OFFSET (0xC0000000UL) +#define __PHYSICAL_START ((unsigned long)CONFIG_PHYSICAL_START)#endif +#define __KERNEL_START (__PAGE_OFFSET + __PHYSICAL_START)#define PAGE_OFFSET ((unsigned long)__PAGE_OFFSET)
PATCH 13/29 x86_64-config-kernel-start
For one kernel to report a crash another kernel has created we need to have 2 kernels loaded simultaneously in memory. To accomplish this the two kernels need to built to run at different physical addresses.
This patch adds the CONFIG_PHYSICAL_START option to the x86_64 kernel so we can do just that. You need to know what you are doing and the ramifications are before changing this value, and most users won't care so I have made it depend on CONFIG_EMBEDDED
bzImage kernels will work and run at a different address when compiled with this option but they will still load at 1MB. If you need a kernel loaded at a different address as well you need to boot a vmlinux.
---
arch/x86_64/Kconfig | 11 +++++++++++ arch/x86_64/boot/compressed/head.S | 7 ++++--- arch/x86_64/boot/compressed/misc.c | 7 ++++--- arch/x86_64/kernel/head.S | 18 +++++++++--------- include/asm-x86_64/page.h | 6 ++++-- 5 files changed, 32 insertions(+), 17 deletions(-)
arch/x86_64/Kconfig +11 -0
@@ -359,6 +359,17 @@ help Additional support for intel specific MCE features such as the thermal monitor. + +config PHYSICAL_START + hex "Physical address where the kernel is loaded" if EMBEDDED + default "0x100000" + help + This gives the physical address where the kernel is loaded. + Primarily used in the case of kexec on panic where the + fail safe kernel needs to run at a different address than + the panic-ed kernel. + + Don't change this unless you know what you are doing. endmenu #
arch/x86_64/boot/compressed/head.S +4 -3
@@ -28,6 +28,7 @@ #include <linux/linkage.h> #include <asm/segment.h> +#include <asm/page.h> .code32 .globl startup_32
@@ -77,7 +78,7 @@ jnz 3f addl $8,%esp xorl %ebx,%ebx - ljmp $(__KERNEL_CS), $0x100000 + ljmp $(__KERNEL_CS), $__PHYSICAL_START /* * We come here, if we were loaded high.
@@ -103,7 +104,7 @@ popl %ecx # lcount popl %edx # high_buffer_start popl %eax # hcount - movl $0x100000,%edi + movl $__PHYSICAL_START,%edi cli # make sure we don't get interrupted ljmp $(__KERNEL_CS), $0x1000 # and jump to the move routine
@@ -128,7 +129,7 @@ movsl movl %ebx,%esi # Restore setup pointer xorl %ebx,%ebx - ljmp $(__KERNEL_CS), $0x100000 + ljmp $(__KERNEL_CS), $__PHYSICAL_START move_routine_end:
arch/x86_64/boot/compressed/misc.c +4 -3
@@ -11,6 +11,7 @@#include "miscsetup.h" #include <asm/io.h> +#include <asm/page.h> /* * gzip declarations
@@ -284,7 +285,7 @@#else if ((ALT_MEM_K >EXT_MEM_K ?ALT_MEM_K :EXT_MEM_K ) <1024 )error ("Less than 2MB of memory" );#endif - output_data = (char *)0x100000 ;/* Points to 1M */ + output_data = (char *)__PHYSICAL_START;/* Normally Points to 1M */ free_mem_end_ptr = (long )real_mode; }
@@ -307,8 +308,8 @@ low_buffer_size = low_buffer_end -LOW_BUFFER_START ; high_loaded =1 ; free_mem_end_ptr = (long )high_buffer_start; -if ( (0x100000 + low_buffer_size) > ((ulg )high_buffer_start)) { - high_buffer_start = (uch *)(0x100000 + low_buffer_size); +if ( (__PHYSICAL_START + low_buffer_size) > ((ulg )high_buffer_start)) { + high_buffer_start = (uch *)(__PHYSICAL_START + low_buffer_size); mv->hcount =0 ;/* say: we need not to move high_buffer */ }else mv->hcount =-1 ;
arch/x86_64/kernel/head.S +9 -9
@@ -235,23 +235,23 @@ */ .org 0x1000 ENTRY(init_level4_pgt) - .quad 0x0000000000102007 /* -> level3_ident_pgt */ + .quad 0x0000000000002007 + __PHYSICAL_START /* -> level3_ident_pgt */ .fill 255,8,0 - .quad 0x000000000010a007 + .quad 0x000000000000a007 + __PHYSICAL_START .fill 254,8,0 /* (2^48-(2*1024*1024*1024))/(2^39) = 511 */ - .quad 0x0000000000103007 /* -> level3_kernel_pgt */ + .quad 0x0000000000003007 + __PHYSICAL_START /* -> level3_kernel_pgt */ .org 0x2000 ENTRY(level3_ident_pgt) - .quad 0x0000000000104007 + .quad 0x0000000000004007 + __PHYSICAL_START .fill 511,8,0 .org 0x3000 ENTRY(level3_kernel_pgt) .fill 510,8,0 /* (2^48-(2*1024*1024*1024)-((2^39)*511))/(2^30) = 510 */ - .quad 0x0000000000105007 /* -> level2_kernel_pgt */ + .quad 0x0000000000005007 + __PHYSICAL_START /* -> level2_kernel_pgt */ .fill 1,8,0 .org 0x4000
@@ -324,17 +324,17 @@ .org 0xa000 ENTRY(level3_physmem_pgt) - .quad 0x0000000000105007 /* -> level2_kernel_pgt (so that __va works even before pagetable_init) */ + .quad 0x0000000000005007 + __PHYSICAL_START /* -> level2_kernel_pgt (so that __va works even before pagetable_init) */ .org 0xb000 #ifdef CONFIG_ACPI_SLEEP ENTRY(wakeup_level4_pgt) - .quad 0x0000000000102007 /* -> level3_ident_pgt */ + .quad 0x0000000000002007 + __PHYSICAL_START /* -> level3_ident_pgt */ .fill 255,8,0 - .quad 0x000000000010a007 + .quad 0x000000000000a007 + __PHYSICAL_START .fill 254,8,0 /* (2^48-(2*1024*1024*1024))/(2^39) = 511 */ - .quad 0x0000000000103007 /* -> level3_kernel_pgt */ + .quad 0x0000000000003007 + __PHYSICAL_START /* -> level3_kernel_pgt */ #endif .data
include/asm-x86_64/page.h +4 -2
@@ -65,12 +65,14 @@extern unsigned long vm_data_default_flags, vm_data_default_flags32;extern unsigned long vm_force_exec32; -#define __START_KERNEL 0xffffffff80100000UL +#define __PHYSICAL_START ((unsigned long)CONFIG_PHYSICAL_START) +#define __START_KERNEL (__START_KERNEL_map + __PHYSICAL_START)#define __START_KERNEL_map 0xffffffff80000000UL#define __PAGE_OFFSET 0xffff810000000000UL#else -#define __START_KERNEL 0xffffffff80100000 +#define __PHYSICAL_START CONFIG_PHYSICAL_START +#define __START_KERNEL (__START_KERNEL_map + __PHYSICAL_START)#define __START_KERNEL_map 0xffffffff80000000#define __PAGE_OFFSET 0xffff810000000000#endif /* !__ASSEMBLY__ */
PATCH 14/29 kexec-kexec-generic
This patch introduces the architecture independent implementation the sys_kexec_load, the compat_sys_kexec_load system calls.
Kexec on panic support has been integrated into the core patch and is relatively clean.
In addition the hopefully architecture independent option crashkernel=size@location has been docuemented. It's purpose is to reserve space for the panic kernel to live, and where no DMA transfer will ever be setup to access.
---
Documentation/kernel-parameters.txt | 4 MAINTAINERS | 11 include/linux/kexec.h | 128 ++++ include/linux/reboot.h | 3 include/linux/syscalls.h | 5 kernel/Makefile | 1 kernel/kexec.c | 1036 ++++++++++++++++++++++++++++++++++++ kernel/panic.c | 11 kernel/sys.c | 20 kernel/sys_ni.c | 2 10 files changed, 1219 insertions(+), 2 deletions(-)
Documentation/kernel-parameters.txt +4 -0
@@ -341,6 +341,10 @@ cpia_pp= [HW,PPT] Format: { parport<nr> | auto | none } + crashkernel=nn[KMG]@ss[KMG] + [KNL] Reserve a chunk of physical memory to + hold a kernel to switch to with kexec on panic. + cs4232= [HW,OSS] Format: <io>,<irq>,<dma>,<dma2>,<mpuio>,<mpuirq>
MAINTAINERS +11 -0
@@ -1318,6 +1318,17 @@ L: [email protected] S: Maintained +KEXEC +P: Eric Biederman +P: Randy Dunlap +M: [email protected] +M: [email protected] +W: http://www.xmission.com/~ebiederm/files/kexec/ +W: http://developer.osdl.org/rddunlap/kexec/ +L: [email protected] +L: [email protected] +S: Maintained + LANMEDIA WAN CARD DRIVER P: Andrew Stanley-Jones M: [email protected]
include/linux/kexec.h +128 -0
128 lines not on this page · raw message
include/linux/reboot.h +3 -0
@@ -51,6 +51,9 @@extern void machine_halt (void );extern void machine_power_off (void ); +extern void machine_shutdown (void ); +extern void machine_crash_shutdown (void ); +#endif #endif /* _LINUX_REBOOT_H */
include/linux/syscalls.h +3 -2
@@ -159,8 +159,9 @@asmlinkage long sys_reboot (int magic1,int magic2,unsigned int cmd,void __user *arg);asmlinkage long sys_restart_syscall (void ); -asmlinkage long sys_kexec_load (void *entry,unsigned long nr_segments, -struct kexec_segment *segments,unsigned long flags); +asmlinkage long sys_kexec_load (unsigned long entry, +unsigned long nr_segments,struct kexec_segment __user *segments, +unsigned long flags);asmlinkage long sys_exit (int error_code); asmlinkage void sys_exit_group(int error_code);
kernel/Makefile +1 -0
@@ -17,6 +17,7 @@ obj-$(CONFIG_KALLSYMS) += kallsyms.o obj-$(CONFIG_PM) += power/ obj-$(CONFIG_BSD_PROCESS_ACCT) += acct.o +obj-$(CONFIG_KEXEC) += kexec.o obj-$(CONFIG_LTT) += ltt-core.o obj-$(CONFIG_COMPAT) += compat.o obj-$(CONFIG_CPUSETS) += cpuset.o
kernel/kexec.c +1036 -0
1,036 lines not on this page · raw message
kernel/panic.c +11 -0
@@ -18,6 +18,7 @@#include <linux/sysrq.h> #include <linux/interrupt.h> #include <linux/nmi.h> +#include <linux/kexec.h> int panic_timeout;int panic_on_oops;
@@ -71,7 +72,17 @@printk (KERN_EMERG "Kernel panic - not syncing: %s\n" ,buf);bust_spinlocks (0 ); +/* If we have crashed and we have a crash kernel loaded +* let it handle everything else. +* Do we want to call this before we try to display a message? +*/ +crash_kexec (); +#ifdef CONFIG_SMP +/* Note smp_send_stop is the usual smp shutdown function, which +* unfortunately means it may not be hardened to work in a panic +* situation. +*/ smp_send_stop ();#endif
kernel/sys.c +20 -0
@@ -16,6 +16,8 @@#include <linux/init.h> #include <linux/highuid.h> #include <linux/fs.h> +#include <linux/kernel.h> +#include <linux/kexec.h> #include <linux/workqueue.h> #include <linux/device.h> #include <linux/key.h>
@@ -433,6 +435,24 @@machine_restart (buffer);break ; +#ifdef CONFIG_KEXEC +case LINUX_REBOOT_CMD_KEXEC : + { +struct kimage *image; + image =xchg (&kexec_image,0 ); +if (!image) { +unlock_kernel (); +return -EINVAL ; + } +notifier_call_chain (&reboot_notifier_list,SYS_RESTART ,NULL ); + system_state =SYSTEM_RESTART ; +device_shutdown (); +printk (KERN_EMERG "Starting new kernel\n" ); +machine_shutdown (); +machine_kexec (image); +break ; + } +#endif #ifdef CONFIG_SOFTWARE_SUSPEND case LINUX_REBOOT_CMD_SW_SUSPEND : {
kernel/sys_ni.c +2 -0
@@ -18,6 +18,8 @@ cond_syscall(sys_lookup_dcookie) cond_syscall(sys_swapon) cond_syscall(sys_swapoff) +cond_syscall(sys_kexec_load) +cond_syscall(compat_sys_kexec_load) cond_syscall(sys_init_module) cond_syscall(sys_delete_module) cond_syscall(sys_socketpair)
PATCH 15/29 x86-machine_shutdown
Factor out the apic and smp shutdown code from machine_restart so it can be called by in the kexec reboot path as well.
By switching to the bootstrap cpu by default on reboot I can delete/simplify some motherboard fixups well.
---
reboot.c | 82 +++++++++++++++++++-------------------------------------------- 1 files changed, 26 insertions(+), 56 deletions(-)
arch/i386/kernel/reboot.c +26 -56
@@ -23,7 +23,6 @@int reboot_thru_bios;#ifdef CONFIG_SMP -int reboot_smp =0 ;static int reboot_cpu =-1 ;/* shamelessly grabbed from lib/vsprintf.c for readability */ #define is_digit (c) ((c) >= '0' && (c) <= '9')
@@ -46,7 +45,6 @@break ;#ifdef CONFIG_SMP case 's' :/* "smp" reboot by executing reset on BSP or other CPU*/ - reboot_smp =1 ;if (is_digit (*(str+1 ))) { reboot_cpu = (int ) (*(str+1 ) -'0' );if (is_digit (*(str+2 )))
@@ -85,33 +83,9 @@return 0 ; } -/* -* Some machines require the "reboot=s" commandline option, this quirk makes that automatic. -*/ -static int __init set_smp_reboot (struct dmi_system_id *d) -{ -#ifdef CONFIG_SMP -if (!reboot_smp) { - reboot_smp =1 ; -printk (KERN_INFO "%s series board detected. Selecting SMP-method for reboots.\n" , d->ident); - } -#endif -return 0 ; -} - -/* -* Some machines require the "reboot=b,s" commandline option, this quirk makes that automatic. -*/ -static int __init set_smp_bios_reboot (struct dmi_system_id *d) -{ -set_smp_reboot (d); -set_bios_reboot (d); -return 0 ; -} -static struct dmi_system_id __initdata reboot_dmi_table[] = { {/* Handle problems with rebooting on Dell 1300's */ - .callback = set_smp_bios_reboot, + .callback = set_bios_reboot, .ident ="Dell PowerEdge 1300" , .matches = {DMI_MATCH (DMI_SYS_VENDOR ,"Dell Computer Corporation" ),
@@ -295,41 +269,32 @@ : "i" ((void *) (0x1000 - sizeof (real_mode_switch) - 100))); } -void machine_restart (char * __unused) +void machine_shutdown (void ) {#ifdef CONFIG_SMP -int cpuid; - - cpuid =GET_APIC_ID (apic_read (APIC_ID )); - -if (reboot_smp) { - -/* check to see if reboot_cpu is valid -if its not, default to the BSP */ -if ((reboot_cpu ==-1 ) || - (reboot_cpu > (NR_CPUS -1 )) || - !physid_isset (cpuid, phys_cpu_present_map)) - reboot_cpu = boot_cpu_physical_apicid; - - reboot_smp =0 ;/* use this as a flag to only go through this once*/ -/* re-run this function on the other CPUs -it will fall though this section since we have -cleared reboot_smp, and do the reboot if it is the -correct CPU, otherwise it halts. */ -if (reboot_cpu != cpuid) -smp_call_function ((void *)machine_restart ,NULL ,1 ,0 ); +int reboot_cpu_id; + +/* The boot cpu is always logical cpu 0 */ + reboot_cpu_id =0 ; + +/* See if there has been given a command line override */ +if ((reboot_cpu_id !=-1 ) && (reboot_cpu <NR_CPUS ) && +cpu_isset (reboot_cpu, cpu_online_map)) { + reboot_cpu_id = reboot_cpu; } -/* if reboot_cpu is still -1, then we want a tradional reboot, -and if we are not running on the reboot_cpu,, halt */ -if ((reboot_cpu !=-1 ) && (cpuid != reboot_cpu)) { -for (;;) - __asm__ __volatile__ ("hlt" ); +/* Make certain the cpu I'm rebooting on is online */ +if (!cpu_isset (reboot_cpu_id, cpu_online_map)) { + reboot_cpu_id =smp_processor_id (); } -/* -* Stop all CPUs and turn off local APICs and the IO-APIC, so -* other OSs see a clean IRQ state. + +/* Make certain I only run on the appropriate processor */ +set_cpus_allowed (current,cpumask_of_cpu (reboot_cpu_id)); + +/* O.K. Now that I'm on the appropriate processor, stop +* all of the others, and disable their local APICs. */ +smp_send_stop ();#endif /* CONFIG_SMP */
@@ -338,6 +303,11 @@#ifdef CONFIG_X86_IO_APIC disable_IO_APIC ();#endif +} + +void machine_restart (char * __unused) +{ +machine_shutdown ();if (!reboot_thru_bios) {if (efi_enabled) {
PATCH 16/29 x86-kexec
This is the i386 implementation of kexec.
---
arch/i386/Kconfig | 17 ++ arch/i386/kernel/Makefile | 1 arch/i386/kernel/crash.c | 42 +++++++ arch/i386/kernel/entry.S | 2 arch/i386/kernel/machine_kexec.c | 220 +++++++++++++++++++++++++++++++++++++ arch/i386/kernel/relocate_kernel.S | 120 ++++++++++++++++++++ include/asm-i386/kexec.h | 28 ++++ 7 files changed, 429 insertions(+), 1 deletion(-)
arch/i386/Kconfig +17 -0
@@ -901,6 +901,23 @@ Don't change this unless you know what you are doing. +config KEXEC + bool "kexec system call (EXPERIMENTAL)" + depends on EXPERIMENTAL + help + kexec is a system call that implements the ability to shutdown your + current kernel, and to start another kernel. It is like a reboot + but it is indepedent of the system firmware. And like a reboot + you can start any kernel with it, not just Linux. + + The name comes from the similiarity to the exec system call. + + It is an ongoing process to be certain the hardware in a machine + is properly shutdown, so do not be surprised if this code does not + initially work for you. It may help to enable device hotplugging + support. As of this writing the exact hardware interface is + strongly in flux, so no good recommendation can be made. + endmenu
arch/i386/kernel/Makefile +1 -0
@@ -24,6 +24,7 @@ obj-$(CONFIG_X86_MPPARSE) += mpparse.o obj-$(CONFIG_X86_LOCAL_APIC) += apic.o nmi.o obj-$(CONFIG_X86_IO_APIC) += io_apic.o +obj-$(CONFIG_KEXEC) += machine_kexec.o relocate_kernel.o crash.o obj-$(CONFIG_X86_NUMAQ) += numaq.o obj-$(CONFIG_X86_SUMMIT_NUMA) += summit.o obj-$(CONFIG_KPROBES) += kprobes.o
arch/i386/kernel/crash.c +42 -0
42 lines not on this page · raw message
arch/i386/kernel/entry.S +1 -1
@@ -911,7 +911,7 @@ .long sys_mq_timedreceive /* 280 */ .long sys_mq_notify .long sys_mq_getsetattr - .long sys_ni_syscall /* reserved for kexec */ + .long sys_kexec_load .long sys_waitid .long sys_ni_syscall /* 285 */ /* available */ .long sys_add_key
arch/i386/kernel/machine_kexec.c +220 -0
220 lines not on this page · raw message
arch/i386/kernel/relocate_kernel.S +120 -0
120 lines not on this page · raw message
include/asm-i386/kexec.h +28 -0
@@ -0,0 +1,28 @@ +#ifndef _I386_KEXEC_H +#define _I386_KEXEC_H + +#include <asm/fixmap.h> + +/* +* KEXEC_SOURCE_MEMORY_LIMIT maximum page get_free_page can return. +* I.e. Maximum page that is mapped directly into kernel memory, +* and kmap is not required. +* +* Someone correct me if FIXADDR_START - PAGEOFFSET is not the correct +* calculation for the amount of memory directly mappable into the +* kernel memory space. +*/ + +/* Maximum physical address we can use pages from */ +#define KEXEC_SOURCE_MEMORY_LIMIT (-1UL) +/* Maximum address we can reach in physical address mode */ +#define KEXEC_DESTINATION_MEMORY_LIMIT (-1UL) +/* Maximum address we can use for the control code buffer */ +#define KEXEC_CONTROL_MEMORY_LIMIT TASK_SIZE + +#define KEXEC_CONTROL_CODE_SIZE 4096 + +/* The native architecture */ +#define KEXEC_ARCH KEXEC_ARCH_386 + +#endif /* _I386_KEXEC_H */
PATCH 17/29 x86-crashkernel
This is the x86 implementation of the crashkernel option. It reserves a window of memory very early in the bootup process, so we never use it for anything but the kernel to switch to when the running kernel panics.
In addition to reserving this memory a resource structure is registered so looking at /proc/iomem it is clear what happened to that memory.
ISSUES:
Is it possible to implement this in a architecture generic way? What should be done with architectures that always use an iommu and thus don't report their RAM memory resources in /proc/iomem?
---
kernel/efi.c | 4 ++++ kernel/setup.c | 30 ++++++++++++++++++++++++++++++ mm/discontig.c | 6 ++++++ 3 files changed, 40 insertions(+)
arch/i386/kernel/efi.c +4 -0
@@ -30,6 +30,7 @@#include <linux/ioport.h> #include <linux/module.h> #include <linux/efi.h> +#include <linux/kexec.h> #include <asm/setup.h> #include <asm/io.h>
@@ -596,6 +597,9 @@if (md->type ==EFI_CONVENTIONAL_MEMORY ) {request_resource (res, code_resource);request_resource (res, data_resource); +#ifdef CONFIG_KEXEC +request_resource (res, &crashk_res); +#endif } } }
arch/i386/kernel/setup.c +30 -0
@@ -40,6 +40,7 @@#include <linux/efi.h> #include <linux/init.h> #include <linux/edd.h> +#include <linux/kexec.h> #include <video/edid.h> #include <asm/apic.h> #include <asm/e820.h>
@@ -825,6 +826,27 @@lapic_disable ();#endif /* CONFIG_X86_LOCAL_APIC */ +#ifdef CONFIG_KEXEC +/* crashkernel=size@addr specifies the location to reserve for +* a crash kernel. By reserving this memory we guarantee +* that linux never set's it up as a DMA target. +* Useful for holding code to do something appropriate +* after a kernel panic. +*/ + else if (!memcmp(from, "crashkernel=", 12)) { +unsigned long size, base; + size =memparse (from+12 , &from); +if (*from =='@' ) { + base =memparse (from+1 , &from); +/* FIXME: Do I want a sanity check +* to validate the memory range? +*/ + crashk_res.start = base; + crashk_res.end = base + size -1 ; + } + } +#endif +/* * highmem=size forces highmem to be exactly 'size' bytes. * This works even on boxes that have no highmem otherwise.
@@ -1128,6 +1150,11 @@ } }#endif +#ifdef CONFIG_KEXEC +if (crashk_res.start != crashk_res.end) { +reserve_bootmem (crashk_res.start, crashk_res.end - crashk_res.start +1 ); + } +#endif return max_low_pfn; }#else
@@ -1167,6 +1194,9 @@*/ request_resource (res, code_resource);request_resource (res, data_resource); +#ifdef CONFIG_KEXEC +request_resource (res, &crashk_res); +#endif } } }
arch/i386/mm/discontig.c +6 -0
@@ -29,6 +29,7 @@#include <linux/highmem.h> #include <linux/initrd.h> #include <linux/nodemask.h> +#include <linux/kexec.h> #include <asm/e820.h> #include <asm/setup.h> #include <asm/mmzone.h>
@@ -365,6 +366,11 @@ system_max_low_pfn << PAGE_SHIFT); initrd_start = 0; } + } +#endif +#ifdef CONFIG_KEXEC +if (crashk_res.start != crashk_res.end) { +reserve_bootmem (crashk_res.start, crashk_res.end - crashk_res.start +1 ); }#endif return system_max_low_pfn;
PATCH 18/29 x86_64-machine_shutdown
Factor out the apic and smp shutdown code from machine_restart so it can be called by in the kexec reboot path as well.
---
reboot.c | 62 +++++++++++++++++++++++++++++++++----------------------------- 1 files changed, 33 insertions(+), 29 deletions(-)
arch/x86_64/kernel/reboot.c +33 -29
@@ -66,41 +66,47 @@__setup ("reboot=" , reboot_setup); -#ifdef CONFIG_SMP -static void smp_halt (void ) +static inline void kb_wait (void ) { -int cpuid =safe_smp_processor_id (); -static int first_entry =1 ; +int i; + +for (i=0 ; i<0x10000 ; i++) +if ((inb_p (0x64 ) &0x02 ) ==0 ) +break ; +} -if (reboot_force) -return ; +void machine_shutdown (void ) +{ +/* Stop the cpus and apics */ +#ifdef CONFIG_SMP +int reboot_cpu_id; -if (first_entry) { - first_entry =0 ; -smp_call_function ((void *)machine_restart,NULL ,1 ,0 ); - } - -smp_stop_cpu (); +/* The boot cpu is always logical cpu 0 */ + reboot_cpu_id =0 ; -/* AP calling this. Just halt */ -if (cpuid != boot_cpu_id) { -for (;;) - asm("hlt" ); +/* Make certain the cpu I'm about to reboot on is online */ +if (!cpu_isset (reboot_cpu_id, cpu_online_map)) { + reboot_cpu_id =smp_processor_id (); } -/* Wait for all other CPUs to have run smp_stop_cpu */ -while (!cpus_empty (cpu_online_map)) -rep_nop (); -} +/* Make certain I only run on the appropriate processor */ +set_cpus_allowed (current,cpumask_of_cpu (reboot_cpu_id)); + +/* O.K Now that I'm on the appropriate processor, +* stop all of the others. +*/ +smp_send_stop ();#endif -static inline void kb_wait (void ) -{ -int i; +local_irq_disable (); -for (i=0 ; i<0x10000 ; i++) -if ((inb_p (0x64 ) &0x02 ) ==0 ) -break ; +#ifndef CONFIG_SMP +disable_local_APIC (); +#endif + +disable_IO_APIC (); + +local_irq_enable (); }void machine_restart (char * __unused)
@@ -109,9 +115,7 @@printk ("machine restart\n" ); -#ifdef CONFIG_SMP -smp_halt (); -#endif +machine_shutdown ();if (!reboot_force) {local_irq_disable ();
PATCH 19/29 x86_64-kexec
This is the x86_64 implementation of machine kexec.
32bit compatibility support has been implemented, and machine_kexec has been enhanced to not care about the changing internal kernel paget table structures.
---
arch/x86_64/Kconfig | 17 ++ arch/x86_64/ia32/ia32entry.S | 2 arch/x86_64/kernel/Makefile | 1 arch/x86_64/kernel/crash.c | 40 +++++ arch/x86_64/kernel/machine_kexec.c | 245 +++++++++++++++++++++++++++++++++++ arch/x86_64/kernel/relocate_kernel.S | 143 ++++++++++++++++++++ include/asm-x86_64/kexec.h | 28 ++++ include/asm-x86_64/unistd.h | 2 8 files changed, 476 insertions(+), 2 deletions(-)
arch/x86_64/Kconfig +17 -0
@@ -370,6 +370,23 @@ the panic-ed kernel. Don't change this unless you know what you are doing. + +config KEXEC + bool "kexec system call (EXPERIMENTAL)" + depends on EXPERIMENTAL + help + kexec is a system call that implements the ability to shutdown your + current kernel, and to start another kernel. It is like a reboot + but it is indepedent of the system firmware. And like a reboot + you can start any kernel with it, not just Linux. + + The name comes from the similiarity to the exec system call. + + It is an ongoing process to be certain the hardware in a machine + is properly shutdown, so do not be surprised if this code does not + initially work for you. It may help to enable device hotplugging + support. As of this writing the exact hardware interface is + strongly in flux, so no good recommendation can be made. endmenu #
arch/x86_64/ia32/ia32entry.S +1 -1
@@ -589,7 +589,7 @@ .quad compat_sys_mq_timedreceive /* 280 */ .quad compat_sys_mq_notify .quad compat_sys_mq_getsetattr - .quad quiet_ni_syscall /* reserved for kexec */ + .quad compat_sys_kexec_load .quad sys32_waitid .quad quiet_ni_syscall /* sys_altroot */ .quad sys_add_key
arch/x86_64/kernel/Makefile +1 -0
@@ -20,6 +20,7 @@ obj-$(CONFIG_X86_LOCAL_APIC) += apic.o nmi.o obj-$(CONFIG_X86_IO_APIC) += io_apic.o mpparse.o \ genapic.o genapic_cluster.o genapic_flat.o +obj-$(CONFIG_KEXEC) += machine_kexec.o relocate_kernel.o crash.o obj-$(CONFIG_PM) += suspend.o obj-$(CONFIG_SOFTWARE_SUSPEND) += suspend_asm.o obj-$(CONFIG_CPU_FREQ) += cpufreq/
arch/x86_64/kernel/crash.c +40 -0
@@ -0,0 +1,40 @@ +/* +* Architecture specific (x86_64) functions for kexec based crash dumps. +* +* Created by: Hariprasad Nellitheertha ([email protected]) +* +* Copyright (C) IBM Corporation, 2004. All rights reserved. +* +*/ + +#include <linux/init.h> +#include <linux/types.h> +#include <linux/kernel.h> +#include <linux/smp.h> +#include <linux/irq.h> +#include <linux/reboot.h> +#include <linux/kexec.h> +#include <linux/elf.h> +#include <linux/elfcore.h> + +#include <asm/processor.h> +#include <asm/hardirq.h> +#include <asm/nmi.h> +#include <asm/hw_irq.h> + +#define MAX_NOTE_BYTES 1024 +typedef u32 note_buf_t [MAX_NOTE_BYTES /4 ]; + +note_buf_t crash_notes[NR_CPUS ]; + +void machine_crash_shutdown (void ) +{ +/* This function is only called after the system +* has paniced or is otherwise in a critical state. +* The minimum amount of code to allow a kexec'd kernel +* to run successfully needs to happen here. +* +* In practice this means shooting down the other cpus in +* an SMP system. +*/ +}
arch/x86_64/kernel/machine_kexec.c +245 -0
245 lines not on this page · raw message
arch/x86_64/kernel/relocate_kernel.S +143 -0
143 lines not on this page · raw message
include/asm-x86_64/kexec.h +28 -0
@@ -0,0 +1,28 @@ +#ifndef _X86_64_KEXEC_H +#define _X86_64_KEXEC_H + +#include <asm/page.h> +#include <asm/proto.h> + +/* +* KEXEC_SOURCE_MEMORY_LIMIT maximum page get_free_page can return. +* I.e. Maximum page that is mapped directly into kernel memory, +* and kmap is not required. +* +* So far x86_64 is limited to 40 physical address bits. +*/ + +/* Maximum physical address we can use pages from */ +#define KEXEC_SOURCE_MEMORY_LIMIT (0xFFFFFFFFFFUL) +/* Maximum address we can reach in physical address mode */ +#define KEXEC_DESTINATION_MEMORY_LIMIT (0xFFFFFFFFFFUL) +/* Maximum address we can use for the control pages */ +#define KEXEC_CONTROL_MEMORY_LIMIT (0xFFFFFFFFFFUL) + +/* Allocate one page for the pdp and the second for the code */ +#define KEXEC_CONTROL_CODE_SIZE (4096UL + 4096UL) + +/* The native architecture */ +#define KEXEC_ARCH KEXEC_ARCH_X86_64 + +#endif /* _X86_64_KEXEC_H */
include/asm-x86_64/unistd.h +1 -1
@@ -554,7 +554,7 @@#define __NR_mq_getsetattr 245 __SYSCALL(__NR_mq_getsetattr, sys_mq_getsetattr)#define __NR_kexec_load 246 -__SYSCALL(__NR_kexec_load, sys_ni_syscall) +__SYSCALL(__NR_kexec_load, sys_kexec_load)#define __NR_waitid 247 __SYSCALL(__NR_waitid, sys_waitid)#define __NR_add_key 248
PATCH 20/29 x86_64-crashkernel
This is the x86_64 implementation of the crashkernel option. It reserves a window of memory very early in the bootup process, so we never use it for anything but the kernel to switch to when the running kernel panics.
In addition to reserving this memory a resource structure is registered so looking at /proc/iomem it is clear what happened to that memory.
ISSUES:
Is it possible to implement this in a architecture generic way? What should be done with architectures that always use an iommu and thus don't report their RAM memory resources in /proc/iomem?
---
e820.c | 4 ++++ setup.c | 27 +++++++++++++++++++++++++++ 2 files changed, 31 insertions(+)
arch/x86_64/kernel/e820.c +4 -0
@@ -10,6 +10,7 @@#include <linux/bootmem.h> #include <linux/ioport.h> #include <linux/string.h> +#include <linux/kexec.h> #include <asm/page.h> #include <asm/e820.h> #include <asm/proto.h>
@@ -204,6 +205,9 @@*/ request_resource (res, &code_resource);request_resource (res, &data_resource); +#ifdef CONFIG_KEXEC +request_resource (res, &crashk_res); +#endif } } }
arch/x86_64/kernel/setup.c +27 -0
@@ -40,6 +40,7 @@#include <linux/acpi.h> #include <linux/kallsyms.h> #include <linux/edd.h> +#include <linux/kexec.h> #include <asm/mtrr.h> #include <asm/uaccess.h> #include <asm/system.h>
@@ -321,6 +322,27 @@if (!memcmp (from,"noexec=" ,7 ))nonx_setup (from +7 ); +#ifdef CONFIG_KEXEC +/* crashkernel=size@addr specifies the location to reserve for +* a crash kernel. By reserving this memory we guarantee +* that linux never set's it up as a DMA target. +* Useful for holding code to do something appropriate +* after a kernel panic. +*/ + else if (!memcmp(from, "crashkernel=", 12)) { +unsigned long size, base; + size =memparse (from+12 , &from); +if (*from =='@' ) { + base =memparse (from+1 , &from); +/* FIXME: Do I want a sanity check +* to validate the memory range? +*/ + crashk_res.start = base; + crashk_res.end = base + size -1 ; + } + } +#endif + next_char: c = *(from++);if (!c)
@@ -574,6 +596,11 @@ (unsigned long)(end_pfn << PAGE_SHIFT)); initrd_start = 0; } + } +#endif +#ifdef CONFIG_KEXEC +if (crashk_res.start != crashk_res.end) { +reserve_bootmem (crashk_res.start, crashk_res.end - crashk_res.start +1 ); }#endif paging_init ();
PATCH 21/29 kexec-ppc-support
I have tweaked this patch slightly to handle an empty list of pages to relocate passed to relocate_new_kernel. And I have added ppc_md.machine_crash_shutdown. To keep up with the changes in the generic kexec infrastructure.
From: Albert Herranz <[email protected]>
The following patch adds support for kexec on the ppc32 platform.
Non-OpenFirmware based platforms are likely to work directly without additional changes on the kernel side. The kexec-tools userland package may need to be slightly updated, though.
For OpenFirmware based machines, additional work is still needed on the kernel side before kexec support is ready. Benjamin Herrenschmidt is kindly working on that part.
In order for a ppc platform to use the kexec kernel services it must implement some ppc_md hooks. Otherwise, kexec will be explicitly disabled, as suggested by benh.
There are 3+1 new ppc_md hooks that a platform supporting kexec may implement. Two of them are mandatory for kexec to work. See include/asm-ppc/machdep.h for details.
- machine_kexec_prepare(image)
This function is called to make any arrangements to the image before it is loaded.
This hook _MUST_ be provided by a platform in order to activate kexec support for that platform. Otherwise, the platform is considered to not support kexec and the kexec_load system call will fail (that makes all existing platforms by default non-kexec'able).
- machine_kexec_cleanup(image)
This function is called to make any cleanups on image after the loaded image data it is freed. This hook is optional. A platform may or may not provide this hook.
- machine_kexec(image)
This function is called to perform the _actual_ kexec. This hook _MUST_ be provided by a platform in order to activate kexec support for that platform.
If a platform provides machine_kexec_prepare but forgets to provide machine_kexec, a kexec will fall back to a reboot.
A ready-to-use machine_kexec_simple() generic function is provided to, hopefully, simplify kexec adoption for embedded platforms. A platform may call this function from its specific machine_kexec hook, like this:
void myplatform_kexec(struct kimage *image)
{
machine_kexec_simple(image);
}- machine_shutdown()
This function is called to perform any machine specific shutdowns, not already done by drivers. This hook is optional. A platform may or may not provide this hook.
An example (trimmed) platform specific module for a platform supporting kexec through the existing machine_kexec_simple follows:
/* ... */
#ifdef CONFIG_KEXEC
int myplatform_kexec_prepare(struct kimage *image)
{
/* here, we can place additional preparations*/
return 0; /* yes, we support kexec */
}
void myplatform_kexec(struct kimage *image)
{
machine_kexec_simple(image);
}#endif /* CONFIG_KEXEC */
/* ... */
void __init
platform_init(unsigned long r3, unsigned long r4,
unsigned long r5,
unsigned long r6, unsigned long r7)
{
/* ... */
#ifdef CONFIG_KEXEC
ppc_md.machine_kexec_prepare =myplatform_kexec_prepare;
ppc_md.machine_kexec =myplatform_kexec;
#endif /* CONFIG_KEXEC */
/* ... */
}
The kexec ppc kernel support has been heavily tested on the GameCube Linux port, and, as reported in the fastboot mailing list, it has been tested too on a Moto 82xx ppc by Rick Richardson.
---
arch/ppc/Kconfig | 20 ++++++ arch/ppc/kernel/Makefile | 1 arch/ppc/kernel/machine_kexec.c | 121 +++++++++++++++++++++++++++++++++++++ arch/ppc/kernel/misc.S | 2 arch/ppc/kernel/relocate_kernel.S | 123 ++++++++++++++++++++++++++++++++++++++ include/asm-ppc/kexec.h | 38 +++++++++++ include/asm-ppc/machdep.h | 31 +++++++++ 7 files changed, 335 insertions(+), 1 deletion(-)
arch/ppc/Kconfig +20 -0
@@ -198,6 +198,26 @@ here. Saying Y here will not hurt performance (on any machine) but will increase the size of the kernel. +config KEXEC + bool "kexec system call (EXPERIMENTAL)" + depends on EXPERIMENTAL + help + kexec is a system call that implements the ability to shutdown your + current kernel, and to start another kernel. It is like a reboot + but it is indepedent of the system firmware. And like a reboot + you can start any kernel with it, not just Linux. + + The name comes from the similiarity to the exec system call. + + It is an ongoing process to be certain the hardware in a machine + is properly shutdown, so do not be surprised if this code does not + initially work for you. It may help to enable device hotplugging + support. As of this writing the exact hardware interface is + strongly in flux, so no good recommendation can be made. + + In the GameCube implementation, kexec allows you to load and + run DOL files, including kernel and homebrew DOLs. + source "drivers/cpufreq/Kconfig" config CPU_FREQ_PMAC
arch/ppc/kernel/Makefile +1 -0
@@ -25,6 +25,7 @@ obj-$(CONFIG_TAU) += temp.o obj-$(CONFIG_ALTIVEC) += vecemu.o vector.o obj-$(CONFIG_FSL_BOOKE) += perfmon_fsl_booke.o +obj-$(CONFIG_KEXEC) += machine_kexec.o relocate_kernel.o ifndef CONFIG_MATH_EMULATION obj-$(CONFIG_8xx) += softemu8xx.o
arch/ppc/kernel/machine_kexec.c +121 -0
121 lines not on this page · raw message
arch/ppc/kernel/misc.S +1 -1
@@ -1450,7 +1450,7 @@ .long sys_mq_timedreceive /* 265 */ .long sys_mq_notify .long sys_mq_getsetattr - .long sys_ni_syscall /* 268 reserved for sys_kexec_load */ + .long sys_kexec_load .long sys_add_key .long sys_request_key /* 270 */ .long sys_keyctl
arch/ppc/kernel/relocate_kernel.S +123 -0
123 lines not on this page · raw message
include/asm-ppc/kexec.h +38 -0
@@ -0,0 +1,38 @@ +#ifndef _PPC_KEXEC_H +#define _PPC_KEXEC_H + +#ifdef CONFIG_KEXEC + +/* +* KEXEC_SOURCE_MEMORY_LIMIT maximum page get_free_page can return. +* I.e. Maximum page that is mapped directly into kernel memory, +* and kmap is not required. +* +* Someone correct me if FIXADDR_START - PAGEOFFSET is not the correct +* calculation for the amount of memory directly mappable into the +* kernel memory space. +*/ + +/* Maximum physical address we can use pages from */ +#define KEXEC_SOURCE_MEMORY_LIMIT (-1UL) +/* Maximum address we can reach in physical address mode */ +#define KEXEC_DESTINATION_MEMORY_LIMIT (-1UL) +/* Maximum address we can use for the control code buffer */ +#define KEXEC_CONTROL_MEMORY_LIMIT TASK_SIZE + +#define KEXEC_CONTROL_CODE_SIZE 4096 + +/* The native architecture */ +#define KEXEC_ARCH KEXEC_ARCH_PPC + +#ifndef __ASSEMBLY__ + +struct kimage ; + +extern void machine_kexec_simple (struct kimage *image); + +#endif /* __ASSEMBLY__ */ + +#endif /* CONFIG_KEXEC */ + +#endif /* _PPC_KEXEC_H */
include/asm-ppc/machdep.h +31 -0
@@ -4,6 +4,7 @@#include <linux/config.h> #include <linux/init.h> +#include <linux/kexec.h> #include <asm/setup.h>
@@ -106,6 +107,36 @@/* functions for dealing with other cpus */ struct smp_ops_t *smp_ops;#endif /* CONFIG_SMP */ + +#ifdef CONFIG_KEXEC +/* Called to shutdown machine specific hardware not already controlled +* by other drivers. +* XXX Should we move this one out of kexec scope? +*/ +void (*machine_shutdown)(void ); + +/* Called to do the minimal shutdown needed to run a kexec'd kernel +* to run successfully. +* XXX Should we move this one out of kexec scope? +*/ +void (*machine_crash_shutdown)(void ); + +/* Called to do what every setup is needed on image and the +* reboot code buffer. Returns 0 on success. +* Provide your own (maybe dummy) implementation if your platform +* claims to support kexec. +*/ +int (*machine_kexec_prepare)(struct kimage *image); + +/* Called to handle any machine specific cleanup on image */ +void (*machine_kexec_cleanup)(struct kimage *image); + +/* Called to perform the _real_ kexec. +* Do NOT allocate memory or fail here. We are past the point of +* no return. +*/ +void (*machine_kexec)(struct kimage *image); +#endif /* CONFIG_KEXEC */ };extern struct machdep_calls ppc_md;
PATCH 22/29 x86-crash_shutdown-nmi-shootdown
One of the dangers when switching from one kernel to another is what happens to all of the other cpus that were running
in the crashed kernel. In an attempt to avoid that problem
this patch adds a nmi handler and attempts to shoot down the other cpus by sending them non maskable interrupts.
The code then waits for 1 second or until all known cpus have stopped running and then jumps from the running kernel that has crashed to the kernel in reserved memory.
The kernel spin loop is used for the delay as that should behave continue to be safe even in after a crash.
---
crash.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 56 insertions(+)
arch/i386/kernel/crash.c +56 -0
@@ -23,12 +23,65 @@#include <asm/hardirq.h> #include <asm/nmi.h> #include <asm/hw_irq.h> +#include <mach_ipi.h> #define MAX_NOTE_BYTES 1024typedef u32 note_buf_t [MAX_NOTE_BYTES /4 ];note_buf_t crash_notes[NR_CPUS ]; +#ifdef CONFIG_SMP +static atomic_t waiting_for_crash_ipi; + +static int crash_nmi_callback (struct pt_regs *regs,int cpu) +{ +local_irq_disable (); +atomic_dec (&waiting_for_crash_ipi); +/* Assume hlt works */ + __asm__("hlt" ); +for (;;); +return 1 ; +} + +/* +* By using the NMI code instead of a vector we just sneak thru the +* word generator coming out with just what we want. AND it does +* not matter if clustered_apic_mode is set or not. +*/ +static void smp_send_nmi_allbutself (void ) +{ +send_IPI_allbutself (APIC_DM_NMI ); +} + +static void nmi_shootdown_cpus (void ) +{ +unsigned long msecs; +atomic_set (&waiting_for_crash_ipi,num_online_cpus () -1 ); + +/* Would it be better to replace the trap vector here? */ +set_nmi_callback (crash_nmi_callback); +/* Ensure the new callback function is set before sending +* out the NMI +*/ +wmb (); + +smp_send_nmi_allbutself (); + + msecs =1000 ;/* Wait at most a second for the other cpus to stop */ +while ((atomic_read (&waiting_for_crash_ipi) >0 ) && msecs) { +mdelay (1 ); + msecs--; + } + +/* Leave the nmi callback set */ +} +#else +static void nmi_shootdown_cpus (void ) +{ +/* There are no cpus to shootdown */ +} +#endif +void machine_crash_shutdown (void ) {/* This function is only called after the system
@@ -39,4 +92,7 @@* In practice this means shooting down the other cpus in * an SMP system. */ +/* The kernel is broken so disable interrupts */ +local_irq_disable (); +nmi_shootdown_cpus (); }
PATCH 23/29 x86-crash_shutdown-snapshot-registers
After the kernel panics if we wish to generate an entire machine core file it is very nice to know the register state at the time the machine crashed.
After long discussion it was realized that if you are going to be saving the information anyway it is reasonable to store the information in a format that it will be used and recognized in so the register state is stored in the standard ELF note format.
---
crash.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 80 insertions(+)
arch/i386/kernel/crash.c +80 -0
@@ -30,12 +30,91 @@note_buf_t crash_notes[NR_CPUS ]; +static u32 *append_elf_note (u32 *buf, +char *name,unsigned type,void *data,size_t data_len) +{ +struct elf_note note; + note.n_namesz =strlen (name) +1 ; + note.n_descsz = data_len; + note.n_type = type; +memcpy (buf, ¬e,sizeof (note)); + buf += (sizeof (note) +3 )/4 ; +memcpy (buf, name, note.n_namesz); + buf += (note.n_namesz +3 )/4 ; +memcpy (buf, data, note.n_descsz); + buf += (note.n_descsz +3 )/4 ; +return buf; +} + +static void final_note (u32 *buf) +{ +struct elf_note note; + note.n_namesz =0 ; + note.n_descsz =0 ; + note.n_type =0 ; +memcpy (buf, ¬e,sizeof (note)); +} + + +static void crash_save_this_cpu (struct pt_regs *regs,int cpu) +{ +struct elf_prstatus prstatus; +u32 *buf; +if ((cpu <0 ) || (cpu >=NR_CPUS )) { +return ; + } +/* Using ELF notes here is opportunistic. +* I need a well defined structure format +* for the data I pass, and I need tags +* on the data to indicate what information I have +* squirrelled away. ELF notes happen to provide +* all of that that no need to invent something new. +*/ + buf = &crash_notes[cpu][0 ]; +memset (&prstatus,0 ,sizeof (prstatus)); + prstatus.pr_pid = current->pid; +elf_core_copy_regs (&prstatus.pr_reg, regs); + buf =append_elf_note (buf,"CORE" ,NT_PRSTATUS , + &prstatus,sizeof (prstatus)); + +final_note (buf); +} + +static void crash_get_current_regs (struct pt_regs *regs) +{ + __asm__ __volatile__("movl %%ebx,%0" :"=m" (regs->ebx)); + __asm__ __volatile__("movl %%ecx,%0" :"=m" (regs->ecx)); + __asm__ __volatile__("movl %%edx,%0" :"=m" (regs->edx)); + __asm__ __volatile__("movl %%esi,%0" :"=m" (regs->esi)); + __asm__ __volatile__("movl %%edi,%0" :"=m" (regs->edi)); + __asm__ __volatile__("movl %%ebp,%0" :"=m" (regs->ebp)); + __asm__ __volatile__("movl %%eax,%0" :"=m" (regs->eax)); + __asm__ __volatile__("movl %%esp,%0" :"=m" (regs->esp)); + __asm__ __volatile__("movw %%ss, %%ax;" :"=a" (regs->xss)); + __asm__ __volatile__("movw %%cs, %%ax;" :"=a" (regs->xcs)); + __asm__ __volatile__("movw %%ds, %%ax;" :"=a" (regs->xds)); + __asm__ __volatile__("movw %%es, %%ax;" :"=a" (regs->xes)); + __asm__ __volatile__("pushfl; popl %0" :"=m" (regs->eflags)); + + regs->eip = (unsigned long )current_text_addr (); +} + +static void crash_save_self (void ) +{ +struct pt_regs regs; +int cpu; + cpu =smp_processor_id (); +crash_get_current_regs (®s); +crash_save_this_cpu (®s, cpu); +} +#ifdef CONFIG_SMP static atomic_t waiting_for_crash_ipi;static int crash_nmi_callback (struct pt_regs *regs,int cpu) {local_irq_disable (); +crash_save_this_cpu (regs, cpu);atomic_dec (&waiting_for_crash_ipi);/* Assume hlt works */ __asm__("hlt" );
@@ -95,4 +174,5 @@/* The kernel is broken so disable interrupts */ local_irq_disable ();nmi_shootdown_cpus (); +crash_save_self (); }
PATCH 24/29 x86-crash_shutdown-apic-shutdown
This patch should not make it to the stable kernel without a being reviewed a lot more. It is unclear how much a hardned kernel can take when it comes to misconfigured apics. So since a normal kernel has problems this patch does a clean shutdown.
It is my expectation this patch will be dropped from future generations of the kexec work. But for the moment it is a crutch to keep from breaking everything.
---
crash.c | 7 +++++++ 1 files changed, 7 insertions(+)
arch/i386/kernel/crash.c +7 -0
@@ -23,6 +23,7 @@#include <asm/hardirq.h> #include <asm/nmi.h> #include <asm/hw_irq.h> +#include <asm/apic.h> #include <mach_ipi.h> #define MAX_NOTE_BYTES 1024
@@ -115,6 +116,7 @@ {local_irq_disable ();crash_save_this_cpu (regs, cpu); +disable_local_APIC ();atomic_dec (&waiting_for_crash_ipi);/* Assume hlt works */ __asm__("hlt" );
@@ -153,6 +155,7 @@ }/* Leave the nmi callback set */ +disable_local_APIC (); }#else static void nmi_shootdown_cpus (void )
@@ -174,5 +177,9 @@/* The kernel is broken so disable interrupts */ local_irq_disable ();nmi_shootdown_cpus (); +lapic_shutdown (); +#if defined(CONFIG_X86_IO_APIC ) +disable_IO_APIC (); +#endif crash_save_self (); }
PATCH 25/29 crashdump-documentation
I have addressed the worst of the documentation changes that come about from the current refacatoring.
From: Hariprasad Nellitheertha <[email protected]>
This patch contains the documentation for the kexec based crash dump tool.
Signed off by Hariprasad Nellitheertha <[email protected]>
---
00-INDEX | 2 + kdump.txt | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+)
Documentation/00-INDEX +2 -0
@@ -140,6 +140,8 @@ - info on the in-kernel binary support for Java(tm). kbuild/ - directory with info about the kernel build process. +kdumpt.txt + - mini HowTo on getting the crash dump code to work. kernel-doc-nano-HOWTO.txt - mini HowTo on generation and location of kernel documentation files. kernel-docs.txt
Documentation/kdump.txt +98 -0
98 lines not on this page · raw message
PATCH 26/29 crashdump-memory-preserving-reboot-using-kexec
With the recent refactoring of the kexec code this patch is a shadow of it's former self. The user space code in /sbin/kexec has been enhanced so it can contain copy the first 640k. And the strong tying between the crashdump capturecode paths and the kexec on panic code paths have been removed.
From: Hariprasad Nellitheertha <[email protected]>
This patch contains the code that does the memory preserving reboot. It copies over the first 640k into a backup region before handing over to kexec. The second kernel will boot using only the backup region.
Signed off by Hariprasad Nellitheertha <[email protected]> Signed off by Adam Litke <[email protected]>
---
arch/i386/Kconfig | 21 +++++++++++++++++++++ arch/i386/kernel/setup.c | 8 ++++++++ include/asm-i386/crash_dump.h | 20 ++++++++++++++++++++ include/linux/bootmem.h | 3 +++ include/linux/crash_dump.h | 10 ++++++++++ kernel/Makefile | 1 + kernel/crash_dump.c | 13 +++++++++++++ mm/bootmem.c | 7 +++++++ 8 files changed, 83 insertions(+)
arch/i386/Kconfig +21 -0
@@ -918,6 +918,27 @@ support. As of this writing the exact hardware interface is strongly in flux, so no good recommendation can be made. +config CRASH_DUMP + bool "kernel crash dumps (EXPERIMENTAL)" + depends on EMBEDDED + depends on EXPERIMENTAL + help + Generate crash dump after being started by kexec. + +config BACKUP_BASE + int "location from where the crash dumping kernel will boot (MB)" + depends on CRASH_DUMP + default 16 + help + This is the location where the second kernel will boot from. + +config BACKUP_SIZE + int "Size of memory used by the crash dumping kernel (MB)" + depends on CRASH_DUMP + range 16 64 + default 32 + help + The size of the second kernel's memory. endmenu
arch/i386/kernel/setup.c +8 -0
@@ -51,6 +51,7 @@#include <asm/io_apic.h> #include <asm/ist.h> #include <asm/io.h> +#include <asm/crash_dump.h> #include "setup_arch_pre.h" #include <bios_ebda.h>
@@ -713,6 +714,13 @@if (to != command_line) to--;if (!memcmp (from+7 ,"exactmap" ,8 )) { +#ifdef CONFIG_CRASH_DUMP +/* If we are doing a crash dump, we +* still need to know the real mem +* size. +*/ +set_saved_max_pfn (); +#endif from +=8 +7 ; e820.nr_map =0 ; userdef =1 ;
include/asm-i386/crash_dump.h +20 -0
@@ -0,0 +1,20 @@ +/* asm-i386/crash_dump.h */ +#include <linux/bootmem.h> + +#ifdef CONFIG_CRASH_DUMP +extern unsigned long __init find_max_low_pfn (void ); +extern void __init find_max_pfn (void ); + +#define CRASH_BACKUP_BASE ((unsigned long)CONFIG_BACKUP_BASE * 0x100000) +#define CRASH_BACKUP_SIZE ((unsigned long)CONFIG_BACKUP_SIZE * 0x100000) +#define CRASH_RELOCATE_SIZE 0xa0000 + +static inline void set_saved_max_pfn (void ) +{ +find_max_pfn (); + saved_max_pfn =find_max_low_pfn (); +} + +#else +#define set_saved_max_pfn () do { } while(0) +#endif
include/linux/bootmem.h +3 -0
@@ -21,6 +21,9 @@* highest page */ extern unsigned long max_pfn; +#ifdef CONFIG_CRASH_DUMP +extern unsigned long saved_max_pfn; +#endif /* * node_bootmem_map is a map pointer - the bits represent all physical
include/linux/crash_dump.h +10 -0
@@ -0,0 +1,10 @@ +#include <linux/kexec.h> +#include <linux/smp_lock.h> +#include <linux/device.h> +#ifdef CONFIG_CRASH_DUMP +#include <asm/crash_dump.h> +#endif + +#ifdef CONFIG_CRASH_DUMP +#else +#endif
kernel/Makefile +1 -0
@@ -29,6 +29,7 @@ obj-$(CONFIG_KPROBES) += kprobes.o obj-$(CONFIG_SYSFS) += ksysfs.o obj-$(CONFIG_GENERIC_HARDIRQS) += irq/ +obj-$(CONFIG_CRASH_DUMP) += crash_dump.o ifneq ($(CONFIG_IA64),y) # According to Alan Modra <[email protected]>, the -fno-omit-frame-pointer is
kernel/crash_dump.c +13 -0
@@ -0,0 +1,13 @@ +/* +* kernel/crash_dump.c - Memory preserving reboot related code. +* +* Created by: Hariprasad Nellitheertha ([email protected]) +* Copyright (C) IBM Corporation, 2004. All rights reserved +*/ + +#include <linux/smp_lock.h> +#include <linux/errno.h> +#include <linux/proc_fs.h> +#include <asm/io.h> +#include <asm/uaccess.h> +
mm/bootmem.c +7 -0
@@ -28,6 +28,13 @@unsigned long max_low_pfn;unsigned long min_low_pfn;unsigned long max_pfn; +#ifdef CONFIG_CRASH_DUMP +/* +* If we have booted due to a crash, max_pfn will be a very low value. We need +* to know the amount of memory that the previous kernel used. +*/ +unsigned long saved_max_pfn; +#endif EXPORT_SYMBOL (max_pfn);/* This is exported so * dma_get_required_mask(), which uses
PATCH 27/29 crashdump-routines-for-copying-dump-pages
kernel/crash.c has been renamed kernel/crash_dump.c to clarify it's purpose.
From: Hariprasad Nellitheertha <[email protected]>
This patch provides the interfaces necessary to read the dump contents, treating it as a high memory device.
Signed off by Hariprasad Nellitheertha <[email protected]>
---
arch/i386/mm/highmem.c | 18 ++++++++++++++++++ include/asm-i386/highmem.h | 1 + include/linux/highmem.h | 1 + kernel/crash_dump.c | 33 +++++++++++++++++++++++++++++++++ 4 files changed, 53 insertions(+)
arch/i386/mm/highmem.c +18 -0
@@ -74,6 +74,24 @@preempt_check_resched (); } +/* This is the same as kmap_atomic() but can map memory that doesn't +* have a struct page associated with it. +*/ +void *kmap_atomic_pfn (unsigned long pfn,enum km_type type) +{ +enum fixed_addresses idx; +unsigned long vaddr; + +inc_preempt_count (); + + idx = type +KM_TYPE_NR *smp_processor_id (); + vaddr =__fix_to_virt (FIX_KMAP_BEGIN + idx); +set_pte (kmap_pte-idx,pfn_pte (pfn, kmap_prot)); +__flush_tlb_one (vaddr); + +return (void *) vaddr; +} +struct page *kmap_atomic_to_page (char *ptr) {unsigned long idx, vaddr = (unsigned long )ptr;
include/asm-i386/highmem.h +1 -0
@@ -72,6 +72,7 @@void kunmap (struct page *page);char *kmap_atomic (struct page *page,enum km_type type);void kunmap_atomic (char *kvaddr,enum km_type type); +void *kmap_atomic_pfn (unsigned long pfn,enum km_type type);struct page *kmap_atomic_to_page (char *ptr);#define flush_cache_kmaps () do { } while (0)
include/linux/highmem.h +1 -0
@@ -28,6 +28,7 @@#define kmap_atomic (page, idx) ((char *)page_address(page))#define kunmap_atomic (addr, idx) do { char *p = addr; (void)p; } while (0) +#define kmap_atomic_pfn (pfn, idx) page_address(pfn_to_page(pfn))#define kmap_atomic_to_page (ptr) virt_to_page(ptr)#endif /* CONFIG_HIGHMEM */
kernel/crash_dump.c +33 -0
@@ -8,6 +8,39 @@#include <linux/smp_lock.h> #include <linux/errno.h> #include <linux/proc_fs.h> +#include <linux/bootmem.h> +#include <linux/highmem.h> +#include <linux/crash_dump.h> +#include <asm/io.h> #include <asm/uaccess.h> +/* +* Copy a page from "oldmem". For this page, there is no pte mapped +* in the current kernel. We stitch up a pte, similar to kmap_atomic. +*/ +ssize_t copy_oldmem_page (unsigned long pfn,char *buf, +size_t csize,int userbuf) +{ +void *page, *vaddr; + +if (!csize) +return 0 ; + + page =kmalloc (PAGE_SIZE ,GFP_KERNEL ); + + vaddr =kmap_atomic_pfn (pfn,KM_PTE0 ); +copy_page (page, vaddr); +kunmap_atomic (vaddr,KM_PTE0 ); + +if (userbuf) { +if (copy_to_user (buf, page, csize)) { +kfree (page); +return -EFAULT ; + } + }else +memcpy (buf, page, csize); +kfree (page); + +return 0 ; +}
PATCH 28/29 crashdump-elf-format-dump-file-access
This patch has been refactored to more closely match the prevailing style in the affected files. And to clearly indicate the dependency between /proc/kcore and proc/vmcore.c
From: Hariprasad Nellitheertha <[email protected]>
This patch contains the code that provides an ELF format interface to the previous kernel's memory post kexec reboot.
Signed off by Hariprasad Nellitheertha <[email protected]>
---
fs/proc/Makefile | 3 fs/proc/kcore.c | 10 - fs/proc/proc_misc.c | 8 + fs/proc/vmcore.c | 239 +++++++++++++++++++++++++++++++++++++++++++++ include/linux/crash_dump.h | 13 ++ 5 files changed, 267 insertions(+), 6 deletions(-)
fs/proc/Makefile +2 -1
@@ -10,5 +10,6 @@ proc-y += inode.o root.o base.o generic.o array.o \ kmsg.o proc_tty.o proc_misc.o -proc-$(CONFIG_PROC_KCORE) += kcore.o +kcore-$(CONFIG_CRASH_DUMP) += vmcore.o +proc-$(CONFIG_PROC_KCORE) += kcore.o $(kcore-y) proc-$(CONFIG_PROC_DEVICETREE) += proc_devtree.o
fs/proc/kcore.c +5 -5
@@ -97,7 +97,7 @@/* * determine size of ELF note */ -static int notesize (struct memelfnote *en) +int notesize (struct memelfnote *en) {int sz;
@@ -112,7 +112,7 @@/* * store a note in the header buffer */ -static char *storenote (struct memelfnote *men,char *bufp) +char *storenote (struct memelfnote *men,char *bufp) {struct elf_note en;
@@ -139,7 +139,7 @@* store an ELF coredump header in the supplied buffer * nphdr is the number of elf_phdr to insert */ -static void elf_kcore_store_hdr (char *bufp,int nphdr,int dataoff) +void elf_kcore_store_hdr (char *bufp,int nphdr,int dataoff,struct kcore_list *clist) {struct elf_prstatus prstatus;/* NT_PRSTATUS */ struct elf_prpsinfo prpsinfo;/* NT_PRPSINFO */
@@ -191,7 +191,7 @@ nhdr->p_align =0 ;/* setup ELF PT_LOAD program header for every area */ -for (m=kclist; m; m=m->next) { +for (m=clist; m; m=m->next) { phdr = (struct elf_phdr *) bufp; bufp +=sizeof (struct elf_phdr ); offset +=sizeof (struct elf_phdr );
@@ -287,7 +287,7 @@return -ENOMEM ; }memset (elf_buf,0 , elf_buflen); -elf_kcore_store_hdr (elf_buf, nphdr, elf_buflen); +elf_kcore_store_hdr (elf_buf, nphdr, elf_buflen, kclist);read_unlock (&kclist_lock);if (copy_to_user (buffer, elf_buf + *fpos, tsz)) {kfree (elf_buf);
fs/proc/proc_misc.c +8 -0
@@ -44,6 +44,7 @@#include <linux/jiffies.h> #include <linux/sysrq.h> #include <linux/vmalloc.h> +#include <linux/crash_dump.h> #include <asm/uaccess.h> #include <asm/pgtable.h> #include <asm/io.h>
@@ -598,6 +599,13 @@ proc_root_kcore->size = (size_t)high_memory - PAGE_OFFSET + PAGE_SIZE; } +# ifdef CONFIG_CRASH_DUMP + entry =create_proc_entry ("vmcore" ,S_IRUSR ,NULL ); +if (entry) { + entry->proc_fops = &proc_vmcore_operations; + entry->size = (size_t )(saved_max_pfn <<PAGE_SHIFT ); + } +# endif #endif #ifdef CONFIG_MAGIC_SYSRQ entry =create_proc_entry ("sysrq-trigger" ,S_IWUSR ,NULL );
fs/proc/vmcore.c +239 -0
239 lines not on this page · raw message
include/linux/crash_dump.h +13 -0
@@ -1,10 +1,23 @@#include <linux/kexec.h> #include <linux/smp_lock.h> #include <linux/device.h> +#include <linux/proc_fs.h> #ifdef CONFIG_CRASH_DUMP #include <asm/crash_dump.h> #endif +extern unsigned long saved_max_pfn; +extern struct memelfnote memelfnote; +extern int notesize (struct memelfnote *); +extern char *storenote (struct memelfnote *,char *); +extern void elf_kcore_store_hdr (char *,int ,int ,struct kcore_list *); +#ifdef CONFIG_CRASH_DUMP +extern struct file_operations proc_vmcore_operations; +extern struct proc_dir_entry *proc_vmcore; + +extern ssize_t copy_oldmem_page (unsigned long ,char *,size_t ,int ); +extern void crash_create_proc_entry (void );#else +#define crash_create_proc_entry () do { } while(0)#endif
PATCH 29/29 crashdump-linear-raw-format-dump-file-access
From: Hariprasad Nellitheertha <[email protected]>
This patch contains the code that enables us to access the previous kernel's memory as /dev/oldmem.
---
Documentation/devices.txt | 1 drivers/char/mem.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+)
Documentation/devices.txt +1 -0
@@ -100,6 +100,7 @@ 9 = /dev/urandom Faster, less secure random number gen. 10 = /dev/aio Asyncronous I/O notification interface 11 = /dev/kmsg Writes to this come out as printk's + 12 = /dev/oldmem Access to crash dump from kexec kernel 1 block RAM disk 0 = /dev/ram0 First RAM disk 1 = /dev/ram1 Second RAM disk
drivers/char/mem.c +74 -0
110 lines not on this page · raw message
Re: [PATCH 19/29] x86_64-kexec
In reply to Eric W. Biederman"Eric W. Biederman" <[email protected]> writes:
[note an extensive review of all this code, but from a quick read...]
arch/x86_64/kernel/machine_kexec.c line 157
Quoted 11 lines
arch/x86_64/kernel/machine_kexec.c +11
-0+ +static void load_segments (void ) +{ + __asm__ __volatile__ ( + "\tmovl $"STR(__KERNEL_DS)",%eax\n" +"\tmovl %eax,%ds\n" +"\tmovl %eax,%es\n" +"\tmovl %eax,%ss\n" +"\tmovl %eax,%fs\n" +"\tmovl %eax,%gs\n" + );
This misses an clobber for "eax"
-Andi
Re: [PATCH 19/29] x86_64-kexec
In reply to Eric W. Biedermanarch/x86_64/kernel/crash.c line 28
Quoted 6 lines · Eric W. Biederman
...
arch/x86_64/kernel/crash.c +1
-0+note_buf_t crash_notes[NR_CPUS ];
After your patches, this global variable stays completely unused on x86_64 (for the i386 version, you added a usage).
cu
Adrian
BTW: Is external usage for crash_notes planned, or can it become static
on both architectures?
Signature · 4 lines
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon SeedRe: [Fastboot] Re: [PATCH 19/29] x86_64-kexec
Quoted 13 lines · Adrian Bunk
Quoted 2 lines · Eric W. Biederman
...
---
linux-2.6.11-rc1-mm1-nokexec-x86_64-machine_shutdown/arch/x86_64/kernel/crash.cWed Dec 31 17:00:00 1969Quoted 1 line
+++
linux-2.6.11-rc1-mm1-nokexec-x86_64-kexec/arch/x86_64/kernel/crash.cTueJan 18 23:14:06 2005Quoted 3 lines
...
+note_buf_t crash_notes[NR_CPUS];
...
Quoted 8 lines
After your patches, this global variable stays completely unused on
x86_64(for the i386 version, you added a usage).cu
AdrianBTW: Is external usage for
crash_notesplanned, or can it become static
on both architectures?
A sharp eye. That array is a key part of an ongoing conversation.
To analyze why a kernel crashed you need some information, beyond simply the contents of memory at the time of the crash.
If that information is not static and obtainable at the time of the crash machine_crash_shutdown() needs to capture that information.
For the format of the information that crashed we can either use some random structure, that you need to know to read kernel debug information to interpret. Or we can use a standard format, reducing the need for magic in the interpretation process. The introduction of crash_notes is the first step is switching to using a standard format, for the data to remove unnecessary dependencies between a kernel and the tools that analyze it after it has crashed.
crash_notes is designed to be a set of per cpu buffers that hold information captured just after a kernel has crashed. So the usage is expected to be very external. How we communicate the address of these per cpu buffers to analysis tools still needs to be addressed. /proc/kallsyms?
As for internal users those will come when machine_crash_shutdown becomes more than a noop on x86_64.
Eric
Re: [PATCH 16/29] x86-kexec
In reply to Eric W. BiedermanHello Eric,
Quoted 1 line · Eric W. Biederman
This is the i386 implementation of kexec.
I tried these patches on an i386 box with kexec-tools-1.99. kexec-ing with vmlinux works fine but bzImage still doesnt go through. Is there a newer kexec-tools package that we need to use this with (to take care of the "purgatory code getting overwritten" problem you had identified).
Regards, Hari
Re: [Fastboot] Re: [PATCH 16/29] x86-kexec
Quoted 9 lines · Hariprasad Nellitheertha
Hello Eric,
Quoted 1 line · Eric W. Biederman
This is the i386 implementation of kexec.
I tried these patches on an i386 box with kexec-tools-1.99. kexec-ing with vmlinux works fine but bzImage still doesnt go through. Is there a newer kexec-tools package that we need to use this with (to take care of the "purgatory code getting overwritten" problem you had identified).
Yes. I will release the 2.0 version shortly. I need to give it a code review before I put it out. So sometime later today.
Eric
Re: [PATCH 6/29] x86-apic-virtwire-on-shutdown
In reply to Eric W. BiedermanQuoted 11 lines · Eric W. Biederman
When coming out of apic mode attempt to set the appropriate apic back into virtual wire mode. This improves on previous versions of this patch by by never setting bot the local apic and the ioapic into veritual wire mode.
This code looks at data from the mptable to see if an ioapic has an ExtInt input to make this decision. A future improvement is to figure out which apic or ioapic was in virtual wire mode at boot time and to remember it. That is potentially a more accurate method, of selecting which apic to place in virutal wire mode.
The call to find_isa_irq_pin() will always fail on ACPI-enabled systems, so this patch is a NO-OP unless the system is booted in MPS mode.
Do we really want to be adding this complexity for obsolete systems? Are there systems that fail without this patch?
-Len
Re: [PATCH 6/29] x86-apic-virtwire-on-shutdown
Quoted 13 lines · Len Brown
Quoted 11 lines · Eric W. Biederman
When coming out of apic mode attempt to set the appropriate apic back into virtual wire mode. This improves on previous versions of this patch by by never setting bot the local apic and the ioapic into veritual wire mode.
This code looks at data from the mptable to see if an ioapic has an ExtInt input to make this decision. A future improvement is to figure out which apic or ioapic was in virtual wire mode at boot time and to remember it. That is potentially a more accurate method, of selecting which apic to place in virutal wire mode.
Quoted 5 lines
The call to
find_isa_irq_pin()will always fail on ACPI-enabled systems, so this patch is a NO-OP unless the system is booted in MPS mode.Do we really want to be adding this complexity for obsolete systems? Are there systems that fail without this patch?
Yes there are bleeding edge systems that fail without this patch. And I have them. That is why I wrote the code.
I do agree that find_isa_irq_pin is a suboptimal way to get this information, looking at the ioapics at boot time would be better. However it works for me, the code is not wrong, and as you said usually the code becomes a noop.
If I can find the appropriate place in the boot path to examine the ioapics before they get stomped I am more than willing to write code that will handle this even in the presence of acpi data.
In addition this code is not a complete noop because when find_isa_irq_pin fails it does put the local apic in virtual wire mode.
Eric
Re: [PATCH 6/29] x86-apic-virtwire-on-shutdown
Quoted 19 lines · Eric W. Biederman
Quoted 3 lines · Len Brown
Quoted 2 lines · Eric W. Biederman
When coming out of apic mode attempt to set the appropriate apic back into virtual wire mode. This improves on previous
versions
Quoted 1 line
Quoted 1 line
of this patch by by never setting bot the local apic and the
ioapic
Quoted 6 lines
Quoted 6 lines
into veritual wire mode.
This code looks at data from the mptable to see if an ioapic has an ExtInt input to make this decision. A future improvement is to figure out which apic or ioapic was in virtual wire mode at boot time and to remember it. That is potentially a more
accurate
Quoted 4 lines
Quoted 2 lines
method, of selecting which apic to place in virutal wire mode.
The call to
find_isa_irq_pin()will always fail on ACPI-enabled
Quoted 8 lines
systems,
Quoted 4 lines
so this patch is a NO-OP unless the system is booted in MPS mode.
Do we really want to be adding this complexity for obsolete systems? Are there systems that fail without this patch?
Yes there are bleeding edge systems that fail without this patch. And I have them. That is why I wrote the code.
What bleeding edge system support MPS and does not support ACPI?
Quoted 8 lines
I do agree that
find_isa_irq_pinis a suboptimal way to get this information, looking at the ioapics at boot time would be better. However it works for me, the code is not wrong, and as you said usually the code becomes a noop.If I can find the appropriate place in the boot path to examine the ioapics before they get stomped I am more than willing to write code that will handle this even in the presence of acpi data.
I belive we don't touch the IO_APICS in either MPS or ACPI mode before setup_IO_APIC.
Quoted 3 lines
In addition this code is not a complete noop because when
find_isa_irq_pinfails it does put the local apic in virtual wire mode.
If the goal of this patch is to restore the hardware to the state that it was before Linux scribbed on it, then it might be a better ideal to save/restore the actual register values the BIOS gave us rather than writing hard-coded values, no?
-Len
Re: [PATCH 6/29] x86-apic-virtwire-on-shutdown
Quoted 5 lines · Len Brown
Quoted 2 lines · Eric W. Biederman
Yes there are bleeding edge systems that fail without this patch. And I have them. That is why I wrote the code.
What bleeding edge system support MPS and does not support ACPI?
All I was talking about is the hardware here. Last I looked the errata on the E7520 E7525 and E7530 chipsets listed using the IOAPIC in virtual wire mode the only way to get them to get the system to work stably if you did not have an SMP kernel. If there is an updated errata work around I'd love to hear it.
The fact that ACPI is quite common is one of the reasons this code path needs more work.
I have not seen the problems with ACPI as I have yet to see a compelling reason to turn it on. And my few experiences with it have lead me to put acpi=off on my kernel command-line by default.
Quoted 2 lines
I belive we don't touch the
IO_APICSin either MPS or ACPI mode beforesetup_IO_APIC.
Thanks I will have a lookup. That sounds like a likely place.
Quoted 4 lines
If the goal of this patch is to restore the hardware to the state that it was before Linux scribbed on it, then it might be a better ideal to save/restore the actual register values the BIOS gave us rather than writing hard-coded values, no?
Nope that is not the goal. The goal is to place system devices in a state close enough to pc compatibility mode that an unpatched kernel will start. A secondary goal is to place the system in a state where the firmware is likely not to have problems.
As the apics are architectural hardware and we only touch the bits we understand there is no reason for us to be modest and pretend we don't know what we are doing. The architecture defines a very narrow set of states that are valid when you are not using the apics. The question is only which of those states will work? pic_mode, virt_wire mode in local apic, virt_wire mode in ioapic. Looking at the hardware is probably the most consistently reliable method of determining that as the kernel will not boot if the firmware sets it up wrong.
Eric
Re: [PATCH 4/29] x86-i8259-shutdown
In reply to Eric W. BiedermanQuoted 10 lines · Eric W. Biederman
From: Eric W. Biederman <[email protected]>
This patch disables interrupt generation from the legacy pic on reboot. Now that there is a
sys_deviceclass it should not be called while drivers are still using interrupts.There is a report about this breaking ACPI power off on some systems.
http://bugme.osdl.org/show_bug.cgi?id=4041
However the final comment seems to exhonorate this code. So until I get more information I believe that was a false positive.
No, the last comment in the bug report
(davej says that there were poweroff problems in FC) does not exhonerate this patch.
All it says is that there are additional poweroff bugs out there.
-Len
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 12 lines · Len Brown
Quoted 10 lines · Eric W. Biederman
From: Eric W. Biederman <[email protected]>
This patch disables interrupt generation from the legacy pic on reboot. Now that there is a
sys_deviceclass it should not be called while drivers are still using interrupts.There is a report about this breaking ACPI power off on some systems.
http://bugme.osdl.org/show_bug.cgi?id=4041
However the final comment seems to exhonorate this code. So until I get more information I believe that was a false positive.
Quoted 4 lines
No, the last comment in the bug report
(davej says that there were poweroff problems in FC) does not exhonerate this patch.
All it says is that there are additional poweroff bugs out there.
Indeed. Since dropping the kexec bits from the Fedora kernel, the 'hangs at poweroff' bug went away for a lot of folks, but there still remain some people affected by some other regression.
https://bugzilla.redhat.com/beta/show_bug.cgi?id=acpi_power_off
has the gory details.
Dave
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 18 lines · Dave Jones
Quoted 12 lines · Len Brown
Quoted 10 lines · Eric W. Biederman
From: Eric W. Biederman <[email protected]>
This patch disables interrupt generation from the legacy pic on reboot. Now that there is a
sys_deviceclass it should not be called while drivers are still using interrupts.There is a report about this breaking ACPI power off on some systems.
http://bugme.osdl.org/show_bug.cgi?id=4041
However the final comment seems to exhonorate this code. So until I get more information I believe that was a false positive.Quoted 4 lines
No, the last comment in the bug report
(davej says that there were poweroff problems in FC) does not exhonerate this patch.
All it says is that there are additional poweroff bugs out there.
Quoted 5 lines
Indeed. Since dropping the kexec bits from the Fedora kernel, the 'hangs at poweroff' bug went away for a lot of folks, but there still remain some people affected by some other regression.
https://bugzilla.redhat.com/beta/show_bug.cgi?id=acpi_power_off
has the gory details.
Ok. I misunderstood that one then. I thought a separate fix had cured the bug. With the kexec bits remaining.
Eric
Re: [PATCH 4/29] x86-i8259-shutdown
In reply to Dave JonesQuoted 13 lines · Dave Jones
Quoted 12 lines · Len Brown
Quoted 10 lines · Eric W. Biederman
From: Eric W. Biederman <[email protected]>
This patch disables interrupt generation from the legacy pic on reboot. Now that there is a
sys_deviceclass it should not be called while drivers are still using interrupts.There is a report about this breaking ACPI power off on some systems.
http://bugme.osdl.org/show_bug.cgi?id=4041
However the final comment seems to exhonorate this code. So until I get more information I believe that was a false positive.
Quoted 4 lines
Quoted 4 lines
No, the last comment in the bug report
(davej says that there were poweroff problems in FC) does not exhonerate this patch.
All it says is that there are additional poweroff bugs out there.
So I will ask again, as I did when Andrew first pointed this in my direction. What code path in the kernel could possibly care if we disable the i8259 after we have disabled all of the other hardware in the system.
The i8259 is a system device so everything else is shutoff first (by design). I have one vague reference that this is a/the problem but I don't have anything to go on with respect to fixing it.
I don't have a system that reproduces this.
Eric
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 4 lines · Eric W. Biederman
So I will ask again, as I did when Andrew first pointed this in my direction. What code path in the kernel could possibly care if we disable the i8259 after we have disabled all of the other hardware in the system.
This may be a foolish question, but, are there possibly any code paths in the *BIOS* that could care?
-Barry K. Nathan <[email protected]>
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 8 lines · Barry K. Nathan
Quoted 4 lines · Eric W. Biederman
So I will ask again, as I did when Andrew first pointed this in my direction. What code path in the kernel could possibly care if we disable the i8259 after we have disabled all of the other hardware in the system.
This may be a foolish question, but, are there possibly any code paths in the *BIOS* that could care?
Fairly unlikely at this point, as the state we have traditionally reprogrammed the i8259 to, delivers interrupts to different vectors
than the firmware uses. So I don't see how telling it not
to deliver interrupts where the firmware won't expect them is likely to change things.
It could be that ACPI AML code is trying something at an inappropriate time. But I can not even find the ACPI soft power code path. pm_power_off never seems to get hooked.
Or it could one of the other kexec related patches for all I know.
Until I get a good data point or a reproducer I can't do anything. It doesn't even make sense to drop the patch because then I won't get a good data point. And I won't know if similar symptoms crop of if I need to do something else.
Eric
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 8 lines · Eric W. Biederman
Quoted 6 lines · Barry K. Nathan
Quoted 4 lines · Eric W. Biederman
So I will ask again, as I did when Andrew first pointed this in my direction. What code path in the kernel could possibly care if we disable the i8259 after we have disabled all of the other hardware in the system.
Quoted 8 lines
Quoted 2 lines
This may be a foolish question, but, are there possibly any code paths in the *BIOS* that could care?
Fairly unlikely at this point, as the state we have traditionally reprogrammed the i8259 to, delivers interrupts to different vectors
than the firmware uses. So I don't see how telling it not
to deliver interrupts where the firmware won't expect them is likely to change things.
FWIW I've noticed something quirky. I guess this is only likely to show up in a contrived scenario, but I have actually reproduced this (by accident, even), so maybe it's worth mentioning.
1. System is booted with both APM and ACPI disabled. (To be exact, ACPI is not compiled into the kernel I tested in this case, and APM is, but the BIOS lacks APM support.)
2. Ultra-minimal /sbin/init does shutdown syscall.
3. "Shutdown: hda
Power down."
You (or I, at least) would expect things to stop here, but we enter the Twilight Zone instead:
4. "Kernel panic - not syncing: Attempted to kill init!" (Nothing this weird ever happens if I have ACPI enabled. In that case it either freezes or properly shuts down.)
5. The admin (that's me) then tries to reboot box without resorting to the power or reset buttons. (Actually the chassis in this case doesn't have a reset button, and I want to avoid unnecessarily power-cycling the thing.)
Without the i8259 shutdown patch applied, I can reboot with Alt-SysRq-B. With the patch, the computer doesn't respond to Alt-SysRq-B and I have to use the power button.
If anyone's interested, the source code to my minimal /sbin/init is here:
http://bugme.osdl.org/attachment.cgi?id=4398&action=view
Quoted 5 lines
It could be that ACPI AML code is trying something at an inappropriate time. But I can not even find the ACPI soft power code path.
pm_power_offnever seems to get hooked.Or it could one of the other kexec related patches for all I know.
The problem occurs even with no other kexec patches applied. And applying all kexec patches except the i8259 shutdown patch fails to reproduce the problem.
Quoted 4 lines
Until I get a good data point or a reproducer I can't do anything. It doesn't even make sense to drop the patch because then I won't get a good data point. And I won't know if similar symptoms crop of if I need to do something else.
It's only happening on 1 of 4 tested boxes here, too (the other three are not affected, but they don't have the same motherboard either). :(
-Barry K. Nathan <[email protected]>
Re: [PATCH 4/29] x86-i8259-shutdown
Ok Looking at the code I have finally tracked where apci_power_off() lives. drives/acpi/poweroff/sleep.c And it has companions in drivers/acpi/hardware/hwsleep.c
Why I did not find this when I wall looking for things that set pm_power_off earlier I haven't a clue because it showed up this time.
One of the functions called by acpi_enter_sleep_state_prep has this to say. /******************************************************************************
*
* FUNCTION: acpi_enter_sleep_state_prep
*
* PARAMETERS: sleep_state - Which sleep state to enter
*
* RETURN: Status
*
* DESCRIPTION: Prepare to enter a system sleep state (see ACPI 2.0 spec p 231)
* This function must execute with interrupts enabled. * We break sleeping into 2 stages so that OSPM can handle * various OS-specific tasks between the two steps.
* ******************************************************************************/
Since I clearly have interrupts disabled at this point we clearly have an issue. Not if we were using apics we would never have had interrupts enabled when we got here so this code as coded is also broken in an SMP context.
Cool I wonder how many more bugs work on kexec can flush out?
So the question becomes where is a good place to call acpi_enter_sleep_state_prep()?
Quoted 29 lines · Barry K. Nathan
Quoted 8 lines · Eric W. Biederman
Quoted 6 lines · Barry K. Nathan
Quoted 4 lines · Eric W. Biederman
So I will ask again, as I did when Andrew first pointed this in my direction. What code path in the kernel could possibly care if we disable the i8259 after we have disabled all of the other hardware in the system.
Quoted 8 lines
Quoted 2 lines
This may be a foolish question, but, are there possibly any code paths in the *BIOS* that could care?
Fairly unlikely at this point, as the state we have traditionally reprogrammed the i8259 to, delivers interrupts to different vectors
than the firmware uses. So I don't see how telling it not
to deliver interrupts where the firmware won't expect them is likely to change things.FWIW I've noticed something quirky. I guess this is only likely to show up in a contrived scenario, but I have actually reproduced this (by accident, even), so maybe it's worth mentioning.
1. System is booted with both APM and ACPI disabled. (To be exact, ACPI is not compiled into the kernel I tested in this case, and APM is, but the BIOS lacks APM support.)
2. Ultra-minimal/sbin/initdoes shutdown syscall.
3. "Shutdown: hda
Power down."
Quoted 2 lines
You (or I, at least) would expect things to stop here, but we enter the Twilight Zone instead:
So would I.
Quoted 3 lines
4. "Kernel panic - not syncing: Attempted to kill init!" (Nothing this weird ever happens if I have ACPI enabled. In that case it either freezes or properly shuts down.)
What lines are printed just before that? It would be nice to know which part of the kernel panic if we can.
At least if you can reproduce this that would be nice.
I wonder if APM is really disabled at this point. I guess since I have found the ACPI bug we can save the APM bug for later.
Quoted 8 lines
5. The admin (that's me) then tries to reboot box without resorting to the power or reset buttons. (Actually the chassis in this case doesn't have a reset button, and I want to avoid unnecessarily power-cycling the thing.)
Without the i8259 shutdown patch applied, I can reboot with Alt-SysRq-B. With the patch, the computer doesn't respond to Alt-SysRq-B and I have to use the power button.
That is probably the one legitimate glitch with this patch. It disables interrupts so the keyboard becomes unresponsive. Of course if the system is really powered down that is not an issue but...
Quoted 3 lines
If anyone's interested, the source code to my minimal
/sbin/initis here:
http://bugme.osdl.org/attachment.cgi?id=4398&action=view
Interesting.
I am wondering if you see the same symptoms if you make this your /init in a initcpio.gz. I am wondering if some of the issues might be related to something not being shutdown properly.
Quoted 9 lines
Quoted 5 lines
It could be that ACPI AML code is trying something at an inappropriate time. But I can not even find the ACPI soft power code path.
pm_power_offnever seems to get hooked.Or it could one of the other kexec related patches for all I know.
The problem occurs even with no other kexec patches applied. And applying all kexec patches except the i8259 shutdown patch fails to reproduce the problem.
Very odd. I was wondering how many test cases had run, before I found
the acpi bug above. If the problem was not 100% the correlation with this patch might have been a fluke. I just want to be certain.
Quoted 7 lines
Quoted 4 lines
Until I get a good data point or a reproducer I can't do anything. It doesn't even make sense to drop the patch because then I won't get a good data point. And I won't know if similar symptoms crop of if I need to do something else.
It's only happening on 1 of 4 tested boxes here, too (the other three are not affected, but they don't have the same motherboard either). :(
Well if you can help me track this down I would appreciate it.
Do you think you would have time to try some test patches?
Eric
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 1 line · Eric W. Biederman
Cool I wonder how many more bugs work on kexec can flush out?
BTW, I got your other e-mail with the patch. I'll see if I can try the patch sometime in the next few hours.
Quoted 8 lines
Quoted 5 lines
3. "Shutdown: hda
Power down."You (or I, at least) would expect things to stop here, but we enter the Twilight Zone instead:
So would I.
Quoted 8 lines
Quoted 3 lines
4. "Kernel panic - not syncing: Attempted to kill init!" (Nothing this weird ever happens if I have ACPI enabled. In that case it either freezes or properly shuts down.)
What lines are printed just before that? It would be nice to know which part of the kernel panic if we can.
At least if you can reproduce this that would be nice.
100% reproducible for me:
---begin quote---
Shutdown: hda
Power down.
Kernel panic - not syncing: Attempted to kill init!
---end quote---
Quoted 2 lines
I wonder if APM is really disabled at this point. I guess since I have found the ACPI bug we can save the APM bug for later.
Earlier in boot it says "apm: bios not found" or something like that. I can try again without ACPI or APM compiled into the kernel at all and see what happens. (IIRC, on all of my systems with working APM, the patch doesn't introduce any problems -- shutdown just works.)
Quoted 9 lines
Quoted 3 lines
If anyone's interested, the source code to my minimal
/sbin/initis here:
http://bugme.osdl.org/attachment.cgi?id=4398&action=viewInteresting.
I am wondering if you see the same symptoms if you make this your /init in a initcpio.gz. I am wondering if some of the issues might be related to something not being shutdown properly.
Yeah, I need to give that a try, but it's possible that I won't be able to do that for a couple of days.
Quoted 7 lines
Quoted 3 lines
The problem occurs even with no other kexec patches applied. And applying all kexec patches except the i8259 shutdown patch fails to reproduce the problem.
Very odd. I was wondering how many test cases had run, before I found
the acpi bug above. If the problem was not 100% the correlation with this patch might have been a fluke. I just want to be certain.
For me, it's absolute 100% correlation over 20-30 test runs or so (unfortunately it didn't occur to me to keep an exact count).
Quoted 3 lines
Well if you can help me track this down I would appreciate it.
Do you think you would have time to try some test patches?
Yes, I should have time to test patches.
-Barry K. Nathan <[email protected]>
Re: [PATCH 4/29] x86-i8259-shutdown
In reply to Barry K. NathanCould you try this patch on your system with acpi that is having problems.
The patch needs some work before it goes into a mainline kernel as I have hacked the call to acpi_power_off_prepare into roughly the proper position in the call chain instead of use a proper hook. But I can't quickly find an existing hook in the proper location.
Eric
drivers/acpi/sleep/poweroff.c +18 -2
@@ -7,18 +7,34 @@#include <linux/pm.h> #include <linux/init.h> +#include <linux/kernel.h> #include <acpi/acpi_bus.h> #include <linux/sched.h> #include "sleep.h" +static void acpi_power_off_prepare (void ) +{ +if (system_state ==SYSTEM_POWER_OFF ) { +acpi_wakeup_gpe_poweroff_prepare (); +acpi_enter_sleep_state_prep (ACPI_STATE_S5 ); + } +} + +void do_acpi_power_off_prepare (void ) +{ +if (!acpi_disabled) { +apci_power_offf_prepare (); + } +} +static void acpi_power_off (void ) {printk ("%s called\n" ,__FUNCTION__); +#if 0 /* This should be made redundant by other patches.. */ /* Some SMP machines only can poweroff in boot CPU */ set_cpus_allowed (current,cpumask_of_cpu (0 )); -acpi_wakeup_gpe_poweroff_prepare (); -acpi_enter_sleep_state_prep (ACPI_STATE_S5 ); +#endif ACPI_DISABLE_IRQS ();acpi_enter_sleep_state (ACPI_STATE_S5 ); }
drivers/base/power/shutdown.c +6 -0
@@ -62,6 +62,12 @@ }up_write (&devices_subsys.rwsem); +#if 1 + { +extern void do_acpi_power_off_prepare (void ); +do_acpi_power_off_prepare (); + } +#endif sysdev_shutdown (); }
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 8 lines · Eric W. Biederman
Could you try this patch on your system with acpi that is having problems.
The patch needs some work before it goes into a mainline kernel as I have hacked the call to
acpi_power_off_prepareinto roughly the proper position in the call chain instead of use a proper hook. But I can't quickly find an existing hook in the proper location.
I had to fix a couple of typos ("apci" and "offf") to get it to compile. Once I did that, the patch made shutdown work again.
-Barry K. Nathan <[email protected]>
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 4 lines · Barry K. Nathan
Quoted 3 lines · Eric W. Biederman
Could you try this patch on your system with acpi that is having problems.
Quoted 8 lines
Quoted 5 lines
The patch needs some work before it goes into a mainline kernel as I have hacked the call to
acpi_power_off_prepareinto roughly the proper position in the call chain instead of use a proper hook. But I can't quickly find an existing hook in the proper location.I had to fix a couple of typos ("apci" and "offf") to get it to compile. Once I did that, the patch made shutdown work again.
Thanks. Now I just need to come up with the good version unless one of the acpi guys wants to volunteer.
Eric
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 12 lines · Eric W. Biederman
Quoted 10 lines · Barry K. Nathan
Quoted 8 lines · Eric W. Biederman
Could you try this patch on your system with acpi that is having problems.
The patch needs some work before it goes into a mainline kernel as I have hacked the call to
acpi_power_off_prepareinto roughly the proper position in the call chain instead of use a proper hook. But I can't quickly find an existing hook in the proper location.
Quoted 5 lines
Quoted 2 lines
I had to fix a couple of typos ("apci" and "offf") to get it to compile. Once I did that, the patch made shutdown work again.
Thanks. Now I just need to come up with the good version unless one of the acpi guys wants to volunteer.
On my box this patch breaks shutdown instead, while it was working without it on -rc2-mm1.
I have an Asus A7V8X motherboard with a VIA VT8377 (KT400) north bridge and a VT8235 south bridge (according to lspci). The IO-APIC is used for interrupt routing.
Sytse
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 6 lines · Sytse Wielinga
On my box this patch breaks shutdown instead, while it was working without it on -rc2-mm1.
I have an Asus A7V8X motherboard with a VIA VT8377 (KT400) north bridge and a VT8235 south bridge (according to lspci). The IO-APIC is used for interrupt routing.
Hmm. The patch had a couple of hard coded assumptions about the configuration (using ACPI etc), but I don't think it was significant enough to break anything. You have a UP board and a K7 processor so my removal of set_cpus_allowed that should not affect anything.
But you are using an SMP kernel or at least the apic support.
Are you using ACPI poweroff?
How does the kernel shutdown fail?
Eric
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 9 lines · Eric W. Biederman
Quoted 6 lines · Sytse Wielinga
On my box this patch breaks shutdown instead, while it was working without it on -rc2-mm1.
I have an Asus A7V8X motherboard with a VIA VT8377 (KT400) north bridge and a VT8235 south bridge (according to lspci). The IO-APIC is used for interrupt routing.
Quoted 6 lines
Hmm. The patch had a couple of hard coded assumptions about the configuration (using ACPI etc), but I don't think it was significant enough to break anything. You have a UP board and a K7 processor so my removal of
set_cpus_allowedthat should not affect anything.But you are using an SMP kernel or at least the apic support.
Yes, I have only one processor but I am using the IO-APIC.
Quoted 1 line
Are you using ACPI poweroff?
Yes.
Quoted 1 line
How does the kernel shutdown fail?
It halts after saying 'acpi_power_off called'. Strangely, it only breaks when using the Alt-SysRq-O poweroff function. Shutting down normally still powers off the system (and does print 'acpi_power_off called'). I think it must have something to do with the IDE devices not having powered down before acpi_power_off is called, but I haven't seen the code so I have no idea what really causes it to break.
Sytse
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 8 lines · Sytse Wielinga
Quoted 1 line · Eric W. Biederman
How does the kernel shutdown fail?
It halts after saying '
acpi_power_offcalled'. Strangely, it only breaks when using the Alt-SysRq-O poweroff function. Shutting down normally still powers off the system (and does print 'acpi_power_offcalled'). I think it must have something to do with the IDE devices not having powered down beforeacpi_power_offis called, but I haven't seen the code so I have no idea what really causes it to break.
I am starting to hate the poor factoring of all of this stuff in the kernel.
kernel/power/poweroff.c re-implements the wheel it comes to doing poweroff a system. Instead of doing a graceful power off it skips calling the powerdown notifer and calling device_shutdown.
Since I moved the acpi prepare for powerdown in device_shutdown it makes sense that code path would now fail.
Do you know if there is any deliberate reason Alt-SysRq-O skips doing a normal device shutdown work?
If not I think I will just extract a common factor from kernel/sys.c/sys_reboot(CMD_POWER_OFF);
And have both code paths call it.
Eric
Re: [PATCH 4/29] x86-i8259-shutdown
Quoted 2 lines · Eric W. Biederman
Do you know if there is any deliberate reason Alt-SysRq-O skips doing a normal device shutdown work?
I would guess that it's intended for use when things are so messed up that all you want to do is cut power ASAP. But, that's just a guess.
-Barry K. Nathan <[email protected]>
PATCH 1/1 Reserving backup region for kexec based crashdumps.
In reply to Eric W. BiedermanHi Andrew,
Following patch is against 2.6.11-rc1-mm2.
As mentioned by following note from Eric, crashdump code is currently broken.
Quoted 3 lines
The crashdump code is currently slightly broken. I have attempted to minimize the breakage so things can quick be made to work again.
We have started doing changes to make crashdump up and running again. Following are few identified items to be done.
1. Reserve the backup region (640k) during kernel bootup.
2. Copy the data to backup region during crash.(moved to kexec user space code, patch posted in separate mail)
3. Prepare elf headers while loading kexec panic kernel and store in reserved memory area.
4. Pass required information to crashdump kernel, which parses it and exports through /proc/vmcore. (may be user space utility, open to discussion)
Following patch implements item 1) in the list. Soon we shall be rolling out the patches for rest.
Thanks
Vivek
This patch adds support for reserving 640k memory as backup region as required by crashdump kernel for x86.
---
---
linux-2.6.11-rc1-mm2-kexec-eric-root/arch/i386/kernel/setup.c | 8 ++++++++ linux-2.6.11-rc1-mm2-kexec-eric-root/include/linux/kexec.h | 6 +++++- linux-2.6.11-rc1-mm2-kexec-eric-root/kernel/kexec.c | 8 ++++++++ 3 files changed, 21 insertions(+), 1 deletion(-)
arch/i386/kernel/setup.c renamed from arch/i386/kernel/setup.c~crashdump-x86-reserve-640k-memory +8 -0
@@ -1159,6 +1159,13 @@ static unsigned long __init setup_memory#ifdef CONFIG_KEXEC if (crashk_res.start != crashk_res.end) {reserve_bootmem (crashk_res.start, crashk_res.end - crashk_res.start +1 ); + +#define CRASH_DUMP_BACKUP 0xa0000 +/* Reserve another 640Kb for crashdump backup. */ + crashdumpk_res.start = crashk_res.end +1 ; + crashdumpk_res.end = crashdumpk_res.start + +CRASH_DUMP_BACKUP -1 ; +reserve_bootmem (crashdumpk_res.start,CRASH_DUMP_BACKUP ); }#endif return max_low_pfn;
@@ -1202,6 +1209,7 @@ legacy_init_iomem_resources(struct resourequest_resource (res, data_resource);#ifdef CONFIG_KEXEC request_resource (res, &crashk_res); +request_resource (res, &crashdumpk_res);#endif } }
include/linux/kexec.h renamed from include/linux/kexec.h~crashdump-x86-reserve-640k-memory +5 -1
@@ -79,7 +79,7 @@ struct kimage {unsigned long control_page;/* Flags to indicate special processing */ - int type : 1; + unsigned int type : 1;#define KEXEC_TYPE_DEFAULT 0#define KEXEC_TYPE_CRASH 1 };
@@ -122,6 +122,10 @@ extern struct kimage *kexec_crash_image;*/ extern struct resource crashk_res; +/* Location of backup region to hold the crashdump kernel data. +*/ +extern struct resource crashdumpk_res; +#else /* !CONFIG_KEXEC */ static inline void crash_kexec (void ) { }#endif /* CONFIG_KEXEC */
kernel/kexec.c renamed from kernel/kexec.c~crashdump-x86-reserve-640k-memory +8 -0
@@ -32,6 +32,14 @@ struct resource crashk_res = { .flags = IORESOURCE_BUSY | IORESOURCE_MEM }; +/* Location of the backup area for the crash dump kernel */ +struct resource crashdumpk_res = { + .name ="Crash Dump Backup" , + .start =0 , + .end =0 , + .flags =IORESOURCE_BUSY |IORESOURCE_MEM +}; +/* * When kexec transitions to the new kernel there is a one-to-one * mapping between physical and virtual addresses. On processors
_
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 7 lines · Vivek Goyal
Hi Andrew,
Following patch is against 2.6.11-rc1-mm2.
As mentioned by following note from Eric, crashdump code is currently broken.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
The crashdump code is currently slightly broken. I have attempted to minimize the breakage so things can quick be made to work again.
We have started doing changes to make crashdump up and running again. Following are few identified items to be done.
1. Reserve the backup region (640k) during kernel bootup.
Why do we need a separate region for this?
It should be simple enough to take 640 out of the area kexec reserves for the crash dump kernel. That is what the previous code implemented.
Quoted 2 lines
2. Copy the data to backup region during crash.(moved to kexec user space code, patch posted in separate mail)
Thanks by and large it looks sane, it won't work yet the but it is moving in the right direction.
include/linux/kexec.h struct kimage { · line 82
Quoted 9 lines
+++
linux-2.6.11-rc1-mm2-kexec-eric-root/include/linux/kexec.h2005-01-20 13:55:33.000000000 +0530include/linux/kexec.h renamed from include/linux/kexec.h~crashdump-x86-reserve-640k-memory +1
-1@@ -79,7 +79,7 @@ struct kimage {unsigned long control_page;/* Flags to indicate special processing */ - int type : 1; + unsigned int type : 1;
That looks like a sane bug fix. Having values of 0 and -1 is quite what I was expecting...
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 15 lines · Eric W. Biederman
Quoted 13 lines · Vivek Goyal
Hi Andrew,
Following patch is against 2.6.11-rc1-mm2.
As mentioned by following note from Eric, crashdump code is currently broken.
Quoted 3 lines
The crashdump code is currently slightly broken. I have attempted to minimize the breakage so things can quick be made to work again.
We have started doing changes to make crashdump up and running again. Following are few identified items to be done.
Quoted 6 lines
Quoted 1 line
1. Reserve the backup region (640k) during kernel bootup.
Why do we need a separate region for this?
It should be simple enough to take 640 out of the area kexec reserves for the crash dump kernel. That is what the previous code implemented.
Previous code also reserved the backup memory region after crash kernel region. It is just a matter of interpretation. What I understand that crash kernel reserved region represents something where one can load the panic kernel directly and new kernel can use this memory region for memory allocation.
I don't want to steal the backup region from crash kernel region otherwise, I shall have to boot the crash kernel with some strange values like memmap=(32M-640k)@16M (symbolically) to prevent crash kernel overwriting backup region. Why to make user aware of location of backup region.
Alternatively, this can be managed by reserving this backup region again in crash kernel to avoid any stomping. May be pass backup region location to new kernel through parameter segment or through command line but don't see a strong reason for doing that.
Thanks
Vivek
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
On deeper review your patch as it stands is incomplete. In particular you don't provide a way to either hardcode or dynamically set the area you are attempt to reserve to hold the backup region.
Quoted 11 lines · Vivek Goyal
Quoted 4 lines · Eric W. Biederman
Why do we need a separate region for this?
It should be simple enough to take 640 out of the area kexec reserves for the crash dump kernel. That is what the previous code implemented.
Previous code also reserved the backup memory region after crash kernel region. It is just a matter of interpretation. What I understand that crash kernel reserved region represents something where one can load the panic kernel directly and new kernel can use this memory region for memory allocation.
Yes the reservation is a hunk of memory reserved for use by the crashdump process, or whatever happens after panic. It is up to the loaded code to define how that memory is used. purgatory.ro is a legitimate part of that loaded code.
Quoted 5 lines
I don't want to steal the backup region from crash kernel region otherwise, I shall have to boot the crash kernel with some strange values like memmap=(32M-640k)@16M (symbolically) to prevent crash kernel overwriting backup region. Why to make user aware of location of backup region.
Making the user aware of the region makes it one more thing for the user to be aware of and to manually manage. Based on what was passed as crashkernel=... We should be able to automate all of the rest of it. So a weird memmap= line should not be hard.
I will have to wait and see but it would not surprise me if we settled on a fixed address per architecture for the reservation to make it easier for various users.
On that note we probably want to move the magic that we are doing for crashdumps into the linux loader (i.e. x86-linux-setup.c ) in kexec-tools, as most of these pieces are specific to taking a crashdump with linux. Not that I expect we will be doing it with anything else but...
Quoted 4 lines
Alternatively, this can be managed by reserving this backup region again in crash kernel to avoid any stomping. May be pass backup region location to new kernel through parameter segment or through command line but don't see a strong reason for doing that.
Probably the biggest reason for doing it in one reservation is that it happens to be an implementation detail of the crashdump capture kernel. If that kernel is not SMP I believe you can safely leave the first 640k alone. I know at least one other effort has had success in that area.
In general it is not good to make unnecessary implementation details between two pieces of software be part of their interface.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 3 lines · Eric W. Biederman
On deeper review your patch as it stands is incomplete. In particular you don't provide a way to either hardcode or dynamically set the area you are attempt to reserve to hold the backup region.
Well. Here is the new patch. This one steals the 640k from top of memory region reserved for crash kernel.
A new command line parameter (crashbackup=) has been introduced for crash dump kernels. This parameter specifies the location of backup region from where to retrieve the backup data.
Thanks
Vivek
This patch adds support for reserving 640k memory as backup region as required by crashdump kernel for x86.
---
---
linux-2.6.11-rc1-root/Documentation/kernel-parameters.txt | 5 ++ linux-2.6.11-rc1-root/arch/i386/kernel/setup.c | 26 ++++++++++++-- linux-2.6.11-rc1-root/arch/i386/mm/discontig.c | 8 ++++ linux-2.6.11-rc1-root/include/linux/crash_dump.h | 1 linux-2.6.11-rc1-root/include/linux/kexec.h | 6 ++- linux-2.6.11-rc1-root/kernel/crash_dump.c | 3 + linux-2.6.11-rc1-root/kernel/kexec.c | 8 ++++ 7 files changed, 54 insertions(+), 3 deletions(-)
arch/i386/kernel/setup.c renamed from arch/i386/kernel/setup.c~crashdump-x86-reserve-640k-memory +24 -2
@@ -41,6 +41,7 @@#include <linux/init.h> #include <linux/edd.h> #include <linux/kexec.h> +#include <linux/crash_dump.h> #include <video/edid.h> #include <asm/apic.h> #include <asm/e820.h>
@@ -51,7 +52,6 @@#include <asm/io_apic.h> #include <asm/ist.h> #include <asm/io.h> -#include <asm/crash_dump.h> #include "setup_arch_pre.h" #include <bios_ebda.h>
@@ -852,7 +852,20 @@ static void __init parse_cmdline_early ( } }#endif - +#ifdef CONFIG_CRASH_DUMP +/* crashbackup=size@addr specifies the location of backup +* region where, crashed kernel has stored some backup data. +*/ + else if (!memcmp(from, "crashbackup=", 12)) { +unsigned long size, base; + size =memparse (from+12 , &from); +if (*from =='@' ) { + base =memparse (from+1 , &from); + crashbackup_start = base; + crashbackup_end = base + size -1 ; + } + } +#endif /* * highmem=size forces highmem to be exactly 'size' bytes. * This works even on boxes that have no highmem otherwise.
@@ -1159,6 +1172,14 @@ static unsigned long __init setup_memory#ifdef CONFIG_KEXEC if (crashk_res.start != crashk_res.end) {reserve_bootmem (crashk_res.start, crashk_res.end - crashk_res.start +1 ); + +#define CRASHDUMP_BACKUP_SZ 0xa0000 +/* Steal 640K from top of reserved region for crash kernel */ +if ((crashk_res.end - crashk_res.start) >CRASHDUMP_BACKUP_SZ ) { + crashdumpk_res.end = crashk_res.end; + crashk_res.end = crashk_res.end -CRASHDUMP_BACKUP_SZ ; + crashdumpk_res.start = crashk_res.end +1 ; + } }#endif return max_low_pfn;
@@ -1202,6 +1223,7 @@ legacy_init_iomem_resources(struct resourequest_resource (res, data_resource);#ifdef CONFIG_KEXEC request_resource (res, &crashk_res); +request_resource (res, &crashdumpk_res);#endif } }
arch/i386/mm/discontig.c renamed from arch/i386/mm/discontig.c~crashdump-x86-reserve-640k-memory +8 -0
@@ -29,6 +29,7 @@#include <linux/highmem.h> #include <linux/initrd.h> #include <linux/nodemask.h> +#include <linux/ioport.h> #include <linux/kexec.h> #include <asm/e820.h> #include <asm/setup.h>
@@ -371,6 +372,13 @@ unsigned long __init setup_memory(void)#ifdef CONFIG_KEXEC if (crashk_res.start != crashk_res.end) {reserve_bootmem (crashk_res.start, crashk_res.end - crashk_res.start +1 ); +#define CRASHDUMP_BACKUP_SZ 0xa0000 +/* Steal 640K from top of reserved region for crash kernel */ +if ((crashk_res.end - crashk_res.start) >CRASHDUMP_BACKUP_SZ ) { + crashdumpk_res.end = crashk_res.end; + crashk_res.end = crashk_res.end -CRASHDUMP_BACKUP_SZ ; + crashdumpk_res.start = crashk_res.end +1 ; + } }#endif return system_max_low_pfn;
Documentation/kernel-parameters.txt renamed from Documentation/kernel-parameters.txt~crashdump-x86-reserve-640k-memory +5 -0
@@ -27,6 +27,7 @@ restrictions referred to are that the re APM Advanced Power Management support is enabled. AX25 Appropriate AX.25 support is enabled. CD Appropriate CD support is enabled. + CRASHDUMP Crash Dump support is enabled. DEVFS devfs support is enabled. DRM Direct Rendering Management support is enabled. EDD BIOS Enhanced Disk Drive Services (EDD) is enabled
@@ -345,6 +346,10 @@ running once the system is up. [KNL] Reserve a chunk of physical memory to hold a kernel to switch to with kexec on panic. + crashbackup=nn[KMG]@ss[KMG] + [KNL, CRASHDUMP] Specifies the location of crash data + backup region. + cs4232= [HW,OSS] Format: <io>,<irq>,<dma>,<dma2>,<mpuio>,<mpuirq>
include/linux/crash_dump.h renamed from include/linux/crash_dump.h~crashdump-x86-reserve-640k-memory +1 -0
@@ -15,6 +15,7 @@ extern void elf_kcore_store_hdr(char *,#ifdef CONFIG_CRASH_DUMP extern struct file_operations proc_vmcore_operations;extern struct proc_dir_entry *proc_vmcore; +extern unsigned long crashbackup_start, crashbackup_end;extern ssize_t copy_oldmem_page (unsigned long ,char *,size_t ,int );extern void crash_create_proc_entry (void );
include/linux/kexec.h renamed from include/linux/kexec.h~crashdump-x86-reserve-640k-memory +5 -1
@@ -79,7 +79,7 @@ struct kimage {unsigned long control_page;/* Flags to indicate special processing */ - int type : 1; + unsigned int type : 1;#define KEXEC_TYPE_DEFAULT 0#define KEXEC_TYPE_CRASH 1 };
@@ -122,6 +122,10 @@ extern struct kimage *kexec_crash_image;*/ extern struct resource crashk_res; +/* Location of backup region to hold the crashdump kernel data. +*/ +extern struct resource crashdumpk_res; +#else /* !CONFIG_KEXEC */ static inline void crash_kexec (void ) { }#endif /* CONFIG_KEXEC */
kernel/crash_dump.c renamed from kernel/crash_dump.c~crashdump-x86-reserve-640k-memory +3 -0
@@ -15,6 +15,9 @@#include <asm/io.h> #include <asm/uaccess.h> +/* Location of crashdump backup region if there is one. */ +unsigned long crashbackup_start, crashbackup_end; +/* * Copy a page from "oldmem". For this page, there is no pte mapped * in the current kernel. We stitch up a pte, similar to kmap_atomic.
kernel/kexec.c renamed from kernel/kexec.c~crashdump-x86-reserve-640k-memory +8 -0
@@ -32,6 +32,14 @@ struct resource crashk_res = { .flags = IORESOURCE_BUSY | IORESOURCE_MEM }; +/* Location of the backup area for the crash dump kernel */ +struct resource crashdumpk_res = { + .name ="Crash Dump Backup" , + .start =0 , + .end =0 , + .flags =IORESOURCE_BUSY |IORESOURCE_MEM +}; +/* * When kexec transitions to the new kernel there is a one-to-one * mapping between physical and virtual addresses. On processors
_
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Right now I am very frustrated with reviewing any of the crashdump patches. When I make comments usually things change just enough that what I said is addressed but things are addressed very much at a surface level. Which means that if I think any kind of substantial change is needed the only way I seem to be able to communicate that is by actually implementing it myself.
Code that works today is great it does manages the job of requirements
capture. But just throwing code together when you are dealing
with fundamental interface boundaries is not a good way to build a sustainable design. And with the crashdump code I want an interface that is at least as simple and as stable as the syscall interface.
At the very least if a patch is just a snapshot of your development process up for comment and you are going to continue on making headway please say as much. If I know the code is quite possibly going to change in some pretty fundamental ways I can stop worrying about it. This patch is certainly nothing I would want for more than a couple of day hack, in my personal development tree.
I will try once again...
There is evil intermingling and false dependency sharing between the dying kernel and the crash capture kernel in this patch, and virtually all of the code is unnecessary. I have already addressed why.
Quoted 11 lines · Vivek Goyal
Quoted 3 lines · Eric W. Biederman
On deeper review your patch as it stands is incomplete. In particular you don't provide a way to either hardcode or dynamically set the area you are attempt to reserve to hold the backup region.
Well. Here is the new patch. This one steals the 640k from top of memory region reserved for crash kernel.
A new command line parameter (crashbackup=) has been introduced for crash dump kernels. This parameter specifies the location of backup region from where to retrieve the backup data.
What is wrong with user space doing all of the extra space reservation?
Could you send this fairly obvious kexec fix, as a separate patch?
Quoted 10 lines
diff -puN
include/linux/kexec.h~crashdump-x86-reserve-640k-memoryinclude/linux/kexec.h--- linux-2.6.11-rc1/include/linux/kexec.h~crashdump-x86-reserve-640k-memory
2005-01-22 14:16:27.000000000 +0530+++
linux-2.6.11-rc1-root/include/linux/kexec.h2005-01-22 14:16:27.000000000 +0530
include/linux/kexec.h struct kimage { · line 85
Quoted 8 lines
include/linux/kexec.h renamed from include/linux/kexec.h~crashdump-x86-reserve-640k-memory +1
-1@@ -79,7 +79,7 @@ struct kimage {unsigned long control_page;/* Flags to indicate special processing */ - int type : 1; + unsigned int type : 1;#define KEXEC_TYPE_DEFAULT 0#define KEXEC_TYPE_CRASH 1 };
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 3 lines · [email protected] (Eric W. Biederman) wrote:
There is evil intermingling and false dependency sharing between
the dying kernel and the crash capture kernel in this patch,
yikes! I'll drop it from -mm while we have a rethink.
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
In reply to Eric W. BiedermanHi Eric,
It looks like we are looking at things a little differently. I see a portion of the picture in your mind, but obviously not entirely.
Perhaps, we need to step back and iron out in specific terms what the interface between the two kernels should be in the crash dump case, and the distribution of responsibility between kernel, user space and the user.
[BTW, the patch was intended as a step in development up for comment early enough to be able to get agreement on the interface and think issues through to more completeness before going too far. Sorry, if that wasn't apparent.]
When you say "evil intermingling", I'm guessing you mean the "crashbackup=" boot parameter ? If so, then yes, I agree it'd be nice to find a way around it that doesn't push hardcoding elsewhere.
Let me explain the interface/approach I was looking at.
1.First kernel reserves some area of memory for crash/capture kernel as specified by crashkernel=X@Y boot time parameter.
2.First kernel marks the top 640K of this area as backup area. (If architecture needs it.) This is sort of a hardcoding and probably this space reservation can be managed from user space as well as mentioned by you in this mail below.
3. Location of backup region is exported through /proc/iomem which can be read by user space utility to pass this information to purgatory code to determine where to copy the first 640K.
Note that we do not make any additional reservation for the backup region. We carve this out from the top of the already reserved region and export it through /proc/iomem so that the user space code and the capture kernel code need not make any assumptions about where this region is located.
4. Once the capture kernel boots, it needs to know the location of backup region for two purposes.
a. It should not overwrite the backup region.
b. There needs to be a way for the capture tool to access the original
contents of the backed up region
Boot time parameter crashbackup=A@B has been provided to pass this information to capture kernel. This parameter is valid only for capture kernel and becomes effective only if CONFIG_CRASH_DUMP is enabled.
Quoted 2 lines
What is wrong with user space doing all of the extra space reservation?
Just for clarity, are you suggesting kexec-tools creating an additional segment for the backup region and pass the information to kernel.
There is no problem in doing reservation from user space except one. How does the user and in-turn capture kernel come to know the location of backup region, assuming that the user is going to provide the exactmap for capture kernel to boot into.
Just a thought, is it a good idea for kexec-tools to be creating and passing memmap parameters doing appropriate adjustment for backup region.
I had another question. How is the starting location of elf headers communicated to capture tool? Is parameter segment a good idea? or some hardcoding?
Another approach can be that backup area information is encoded in elf headers and capture kernel is booted with modified memmap (User gets backup region information from /proc/iomem) and capture tool can extract backup area information from elf headers as stored by first kernel.
Could you please elaborate a little more on what aspect of your view differs from the above.
Thanks
Vivek
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
For the guys on ppc, and other architectures that have all of their cpu memory behind an iommu. I propose we create a /proc/cpumem which is the subset of /proc/iomem that deals with RAM. In any event as something like that is straight forward to implement I will assume the existence of the functionality and we can attack the details when we do the merge the first of those architectures into the kernel.
Quoted 11 lines · Vivek Goyal
Hi Eric,
It looks like we are looking at things a little differently. I see a portion of the picture in your mind, but obviously not entirely.
Perhaps, we need to step back and iron out in specific terms what the interface between the two kernels should be in the crash dump case, and the distribution of responsibility between kernel, user space and the user.
Quoted 4 lines
[BTW, the patch was intended as a step in development up for comment early enough to be able to get agreement on the interface and think issues through to more completeness before going too far. Sorry, if that wasn't apparent.]
It wasn't quite, and the fact that Andrew picked it up added to the confusion.
Quoted 4 lines
When you say "evil intermingling", I'm guessing you mean the "crashbackup=" boot parameter ? If so, then yes, I agree it'd be nice to find a way around it that doesn't push hardcoding elsewhere.
I believe there are some alternatives to crashbackup= in the crashdump capture kernel. But as long as that code is running in the kernel we can't do a lot better.
However for the primary kernel it has no need to know that we even have a backup region, nor does it need to know about the size of the backup region. That can all be handled with the single reservation, we have now.
/sbin/kexec which makes the backup needs to know about it and it needs to pass that information on. But the primary kernel does not.
The largest reason I am sensitive to this issue is that if you are not booting an SMP kernel I don't believe we need a backup region on x86 at all. If we can remove that dependency I want the freedom to do that without having to modify the primary kernel. Or if we discover we need to preserve other things like the ACPI, mp and pirq tables I don't want to require patching the kernel just so I can copy those and preserve them.
Quoted 5 lines
Let me explain the interface/approach I was looking at.
1.First kernel reserves some area of memory for crash/capture kernel as specified by crashkernel=X@Y boot time parameter.
Quoted 8 lines
2.First kernel marks the top 640K of this area as backup area. (If architecture needs it.) This is sort of a hardcoding and probably this space reservation can be managed from user space as well as mentioned by you in this mail below.
3. Location of backup region is exported through
/proc/iomemwhich can be read by user space utility to pass this information to purgatory code to determine where to copy the first 640K.
And 1-3 can be done in /sbin/kexec. And if it is done there we can increase our freedom of implementation in the crashdump capture process quite a bit.
Quoted 9 lines
Note that we do not make any additional reservation for the backup region. We carve this out from the top of the already reserved region and export it through
/proc/iomemso that the user space code and the capture kernel code need not make any assumptions about where this region is located.4. Once the capture kernel boots, it needs to know the location of backup region for two purposes.
Quoted 8 lines
a. It should not overwrite the backup region.
b. There needs to be a way for the capture tool to access the original
contents of the backed up regionBoot time parameter crashbackup=A@B has been provided to pass this information to capture kernel. This parameter is valid only for capture kernel and becomes effective only if
CONFIG_CRASH_DUMPis enabled.
But that is not what you implemented. crashbackup= was an alternative to the carving out of 640K in parts 1-3.
Quoted 5 lines
Quoted 2 lines
What is wrong with user space doing all of the extra space reservation?
Just for clarity, are you suggesting kexec-tools creating an additional segment for the backup region and pass the information to kernel.
Yes, having kexec create a bss segment for the backup region would be a good idea. It will keep us from stomping on the kernel trampoline (think the identity mapped x86_64 page tables here) by accident.
Quoted 8 lines
There is no problem in doing reservation from user space except one. How does the user and in-turn capture kernel come to know the location of backup region, assuming that the user is going to provide the exactmap for capture kernel to boot into.
Just a thought, is it a good idea for kexec-tools to be creating and passing memmap parameters doing appropriate adjustment for backup region.
Exactly. Having /sbin/kexec do this instead of the user doing this manually is a much simpler solution than we have now.
Quoted 3 lines
I had another question. How is the starting location of elf headers communicated to capture tool? Is parameter segment a good idea? or some hardcoding?
I recognize the need for that information. But I do not recognize the need for it to be an ELF header (we do need something conceptually close). If we don't have regions of the memory map appearing and disappearing dynamically we can get this information from /proc/iomem, before the crash and store it in one of the data segments that we checksum.
Quoted 8 lines
Another approach can be that backup area information is encoded in elf headers and capture kernel is booted with modified memmap (User gets backup region information from
/proc/iomem) and capture tool can extract backup area information from elf headers as stored by first kernel.Could you please elaborate a little more on what aspect of your view differs from the above.
See above.
The direction I would take if I was to take to implementing this the crashdump functionality is something different.
Instead of patching crashdump functionality into the kernel, I would create a subdirectory in kexec-tools called crashdump and put in the source for a user-space program that could run as init. In addition I would but in the code to generate and initramfs cpio.gz archive of that program. And I would build the program against uclibc, klibc or one of the other libc variants that actually allows building static binaries. Unless something has changed recently glibc does not all for truly static binaries as it dynamically open /lib/libnss*
Given the pain of building against an external library that is not widely distributed I would probably take a snapshot of the code and place it in crashdump/libc in the kexec-tools source. Taking a snapshot of frequently used libraries is commonly done with the gnu toolchain and is wonderfully effective in resolving painful dependencies.
The crashdump /init would just mmap /dev/mem to read the raw memory.
Quoted 1 line
From there it would generate the core file.
When kexec'ing a panic kernel I would simply have /sbin/kexec unconditionally load that cpio.gz as the initrd and things would work.
The large advantage of doing all of this in user space is that it moves all of the crashdump policy into user space and into one source tree, for simplified maintenance.
However as long as we gracefully handle the interface between the primary kernel and the capture kernel we can switch mechanisms for actually taking the crash dump, kernel based or user space as seems most sane.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Hi Eric,
However for the primary kernel it has no need to know that we
Quoted 6 lines
even have a backup region, nor does it need to know about the size of the backup region. That can all be handled with the single reservation, we have now.
/sbin/kexecwhich makes the backup needs to know about it and it needs to pass that information on. But the primary kernel does not.
Agreed. Primary kernel need not to be aware of backup region and reservation of this region can be better managed from user space.
Quoted 6 lines
Quoted 3 lines
Boot time parameter crashbackup=A@B has been provided to pass this information to capture kernel. This parameter is valid only for capture kernel and becomes effective only if
CONFIG_CRASH_DUMPis enabled.But that is not what you implemented. crashbackup= was an alternative to the carving out of 640K in parts 1-3.
Not really. crashbackup= is not being used for carving out backup region. It is just used for passing the address of this region to second kernel. That's why it has been put under CONFIG_CRASH_DUMP.
Quoted 26 lines
Quoted 5 lines
Quoted 2 lines
What is wrong with user space doing all of the extra space reservation?
Just for clarity, are you suggesting kexec-tools creating an additional segment for the backup region and pass the information to kernel.
Yes, having kexec create a bss segment for the backup region would be a good idea. It will keep us from stomping on the kernel trampoline (think the identity mapped
x86_64page tables here) by accident.Quoted 8 lines
There is no problem in doing reservation from user space except one. How does the user and in-turn capture kernel come to know the location of backup region, assuming that the user is going to provide the exactmap for capture kernel to boot into.
Just a thought, is it a good idea for kexec-tools to be creating and passing memmap parameters doing appropriate adjustment for backup region.
Exactly. Having
/sbin/kexecdo this instead of the user doing this manually is a much simpler solution than we have now.Quoted 3 lines
I had another question. How is the starting location of elf headers communicated to capture tool? Is parameter segment a good idea? or some hardcoding?
Quoted 7 lines
I recognize the need for that information. But I do not recognize the need for it to be an ELF header (we do need something conceptually close). If we don't have regions of the memory map appearing and disappearing dynamically we can get this information from
/proc/iomem, before the crash and store it in one of the data segments that we checksum.
This looks good. So memory regions are parsed from /proc/iomem and this information is put in one data segment and stored in reserved region during panic kernel load time.
But I am unable to co-relate as to how the capture tool (even if its all in user space) gets to know the address of this segment (or for that matter, the bss segment created for backup region). Am I missing something obvious.
Quoted 27 lines
The direction I would take if I was to take to implementing this the crashdump functionality is something different.
Instead of patching crashdump functionality into the kernel, I would create a subdirectory in kexec-tools called crashdump and put in the source for a user-space program that could run as init. In addition I would but in the code to generate and initramfs cpio.gz archive of that program. And I would build the program against uclibc, klibc or one of the other libc variants that actually allows building static binaries. Unless something has changed recently glibc does not all for truly static binaries as it dynamically open
/lib/libnss*Given the pain of building against an external library that is not widely distributed I would probably take a snapshot of the code and place it in crashdump/libc in the kexec-tools source. Taking a snapshot of frequently used libraries is commonly done with the gnu toolchain and is wonderfully effective in resolving painful dependencies.
The crashdump /init would just mmap
/dev/memto read the raw memory.Quoted 1 line
From there it would generate the core file.
When kexec'ing a panic kernel I would simply have
/sbin/kexecunconditionally load that cpio.gz as the initrd and things would work.
Quoted 8 lines
The large advantage of doing all of this in user space is that it moves all of the crashdump policy into user space and into one source tree, for simplified maintenance.
However as long as we gracefully handle the interface between the primary kernel and the capture kernel we can switch mechanisms for actually taking the crash dump, kernel based or user space as seems most sane.
This seems to be a good direction.
Thanks
Vivek
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 8 lines · Vivek Goyal
Hi Eric,
However for the primary kernel it has no need to know that we
Quoted 4 lines
even have a backup region, nor does it need to know about the size of the backup region. That can all be handled with the single reservation, we have now.
Quoted 6 lines
Quoted 2 lines
/sbin/kexecwhich makes the backup needs to know about it and it needs to pass that information on. But the primary kernel does not.Agreed. Primary kernel need not to be aware of backup region and reservation of this region can be better managed from user space.
Good. It sound like we are pretty much back on the same page then.
Quoted 11 lines
Quoted 6 lines
Quoted 3 lines
Boot time parameter crashbackup=A@B has been provided to pass this information to capture kernel. This parameter is valid only for capture kernel and becomes effective only if
CONFIG_CRASH_DUMPis enabled.But that is not what you implemented. crashbackup= was an alternative to the carving out of 640K in parts 1-3.
Not really. crashbackup= is not being used for carving out backup region. It is just used for passing the address of this region to second kernel. That's why it has been put under
CONFIG_CRASH_DUMP.
Ok I missed a piece in your patch. You have crashdumpk_res, andcrashbackup_start, crashbackup_end. And I missed the fact that
they were different variables as they dealt with the same concept.
So that patch actually should have been three patches. The one line bug fix. The crashdumpk_res bit (which I strongly object to) and the crashbackup_start/_end bit. The fact that all three were in the same patch is a reviewing and maintenance pain.
Please in the future do not include code that runs in the primary kernel and crashdump specific code that runs in the capture kernel in the same patch.
Quoted 8 lines
This looks good. So memory regions are parsed from
/proc/iomemand this information is put in one data segment and stored in reserved region during panic kernel load time.But I am unable to co-relate as to how the capture tool (even if its all in user space) gets to know the address of this segment (or for that matter, the bss segment created for backup region). Am I missing something obvious.
There are a lot of choices at that point.
Place the data in the on the kernel command line, and pick it up from /proc/cmdline.
Place the data in a file on the initramfs.
Place the data in a user space data segment.
Quoted 7 lines
Quoted 4 lines
However as long as we gracefully handle the interface between the primary kernel and the capture kernel we can switch mechanisms for actually taking the crash dump, kernel based or user space as seems most sane.
This seems to be a good direction.
Cool.
One of the ideas worth exploring is to see about stabilizing the
other side of this interface as well. That is we should explore
providing a fixed interface coming out of purgatory.ro to the new kernel and it's user space (i.e. the ELF header like thing). I think we are quite close to that point already. And this goes back to your question of how do we let the capture kernel/user space know where to look.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Well, trying to put the already discussed ideas together. I was planning to work on following design. Please comment.
Crashed Kernel <-->Capture Kernel(or User Space) Interface:
----------------------------------------------------------
The whole idea is that Crash image is represented in ELF Core format. These ELF Headers are prepared by kexec-tools user space and put in one segment. Address of start of image is passed to the capture kernel(or user space) using one command line (eg. crashimage=). Now either kernel space or user space can parse the elf headers and extract required information and export final kernel elf core image.
Quoted 9 lines
[email protected] wrote:
If we were using an ELF header I would include onePT_NOTEprogram header per cpu (Giving each cpu it's own area to mess around in). And I would use onePT_LOADsegment per possible memory zone. So in the worst case (current sgi altix) (MAX_NUMNODES=256, MAX_NR_ZONES=3, MAX_NR_CPUS=1024) 256*3+1024 = 1792 program
headers. At 56 bytes per 64bit program header that is 100352 bytes
or 98KiB. A little bit expensive. A tuned data structure with 64bit base and size would only consume 1792*16 = 28672 or 28KiB.
If I prepare One elf header for each physical contiguous memory area (as obtained from /proc/iomem) instead of per zone, then number of elf headers will come down significantly. I don't have any idea on number of actual physically contiguous regions present per machine, but roughly assuming it to be 1 per node, it will lead to 256 + 1024 = 1280 program headers.At 56 bytes per 64 bit program header this will amount to 70KB.
This is worst case estimate and on lower end machines this will require much less a space. On machines as big as 1024 cpus, this should not be a concern, as big machines come with big RAMs.
Eric, do you still think that ELF headers are inappropriate to be passed across interface boundary.
ELF headers can be prepared by kexec-tools in advance and put into one of the data segments. This requires following information to be available to user space.
- Starting address of space reserved by kernel for notes section (crash_notes[]). Probably can be obtained from /proc/kallsysms?
- NR_CPUS. May be sysconf(_SC_NPROCESSORS_CONF) should be sufficient.
- Size of memory reserved per cpu. No clue how to get that? Any suggestions?
May be hard-coding like 1K area per cpu should be to address the future needs ?
Regarding Backup Region
-----------------------
- Kexec user space does the reservation for backup region segment.
- Purgatory copies the backup data to backup region. (Already implemented)
- A separate elf header is prepared to represent backed up memory region. And "offset" field of this program header can contain the actual physical address where backup contents are stored.
Thanks
Vivek
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 6 lines · Vivek Goyal
Well, trying to put the already discussed ideas together. I was planning to work on following design. Please comment.
Crashed Kernel <-->Capture Kernel(or User Space) Interface:
----------------------------------------------------------
Quoted 6 lines
The whole idea is that Crash image is represented in ELF Core format. These ELF Headers are prepared by kexec-tools user space and put in one segment. Address of start of image is passed to the capture kernel(or user space) using one command line (eg. crashimage=). Now either kernel space or user space can parse the elf headers and extract required information and export final kernel elf core image.
Sounds sane. We need to make certain there is a checksum of that region but putting it in a separate segment should ensure that.
I also think we need to look at the name crashimage= and see if we can find something more descriptive. But that is minor. Possibly elfcorehdr= We have a little while to think about that one before we are stuck.
Quoted 10 lines
Quoted 9 lines
[email protected] wrote:
If we were using an ELF header I would include onePT_NOTEprogram header per cpu (Giving each cpu it's own area to mess around in). And I would use onePT_LOADsegment per possible memory zone. So in the worst case (current sgi altix) (MAX_NUMNODES=256, MAX_NR_ZONES=3, MAX_NR_CPUS=1024) 256*3+1024 = 1792 program
headers. At 56 bytes per 64bit program header that is 100352 bytes
or 98KiB. A little bit expensive. A tuned data structure with 64bit base and size would only consume 1792*16 = 28672 or 28KiB.
Quoted 3 lines
If I prepare One elf header for each physical contiguous memory area (as obtained from
/proc/iomem) instead of per zone, then number of elf headers will come down significantly.
A clarification on terminology we are talking about struct Elf64_Phdr here. There is only one Elf header. That seems to be clear farther down.
Quoted 8 lines
I don't have any idea on number of
actual physically contiguous regions present per machine, but roughly assuming it to be 1 per node, it will lead to 256 + 1024 = 1280 program headers.At 56 bytes per 64 bit program header this will amount to 70KB.This is worst case estimate and on lower end machines this will require much less a space. On machines as big as 1024 cpus, this should not be a concern, as big machines come with big RAMs.
Agreed. Size is not the primary issue. There is some clear waste but that is a secondary concern. Not performing a 1-1 mapping to the kernel data structures also seems to be a win, as the concepts are noticeably different.
Quoted 2 lines
Eric, do you still think that ELF headers are inappropriate to be passed across interface boundary.
I have serious concerns about the kernel generating the ELF headers and only delivering them after the kernel has crashed. Because then we run into questions of what information can be trusted. If we avoid that issue I am not too concerned.
Quoted 3 lines
ELF headers can be prepared by kexec-tools in advance and put into one of the data segments. This requires following information to be available to user space.
For a first round doing it in user space sounds sane. Obtaining the information at the time of load is much more robust.
Quoted 2 lines
- Starting address of space reserved by kernel for notes section (crash_notes[]). Probably can be obtained from
/proc/kallsysms?
At least for a start.
Quoted 2 lines
-
NR_CPUS. May besysconf(_SC_NPROCESSORS_CONF)should be sufficient.
Either that or /proc/cpuinfo. But the sysconf approach looks more robust at this point.
Quoted 4 lines
- Size of memory reserved per cpu. No clue how to get that? Any suggestions?
May be hard-coding like 1K area per cpu should be to address the future needs ?
The nice thing about doing this in user space is that we can hack something together and get each side of the interface sorted out independently. i.e. We can hard code it for now. Sort out the users and then come back and make certain we have the information exported cleanly. 1K per cpu currently matches the kernel code so it is a good place to start :)
It does look like getting the size of each array element is a problem, so the current kernel code certainly needs to be revisited. And there are quite a few other things pieces of how we are obtaining the information that can be fixed as well.
Quoted 9 lines
Regarding Backup Region
------------------------ Kexec user space does the reservation for backup region segment.
- Purgatory copies the backup data to backup region. (Already implemented)
- A separate elf header is prepared to represent backed up memory region. And "offset" field of this program header can contain the actual physical address where backup contents are stored.
I like that. I was thinking a virtual versus physical address separation. But using the offset field is much more appropriate, and it leaves us the potential of doing something nice like specifying the kernels virtual address later on. Looking exclusively at the offset field to know which memory addresses to dump sounds good. For now we should have virtual==physical==offset except for the backup region.
This sounds like a good place to start.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Hi,
I can't understand why ELF format is necessary.
I think the only necessary information is "what physical address regions are valid to read". This information is necessary for any sort of dump tools. (and must get it while the system is normal.) The Eric's /proc/cpumem idea sounds nice to me.
Signature · 1 line
Itsuro ODA <[email protected]>
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 8 lines · Itsuro Oda
Hi,
I can't understand why ELF format is necessary.
I think the only necessary information is "what physical address regions are valid to read". This information is necessary for any sort of dump tools. (and must get it while the system is normal.)
The Eric's/proc/cpumemidea sounds nice to me.
I agree. Format conversion should be done in healthy system separately and we should restrict what to do while taking the dump as few as possible. Conversion from just memory image to crash/lcrash format will be very useful to use existing tools and experiences. I already have such tool and (if my administration allows) I can make such tool open. Let me do some paperwork.
Koichi Suzuki
NTT DATA Intellilink
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 8 lines · Koichi Suzuki
Quoted 6 lines · Itsuro Oda
Hi,
I can't understand why ELF format is necessary.
I think the only necessary information is "what physical address regions are valid to read". This information is necessary for any
sort of dump tools. (and must get it while the system is normal.) The Eric's/proc/cpumemidea sounds nice to me.
Quoted 5 lines
I agree. Format conversion should be done in healthy system separately and we should restrict what to do while taking the dump as few as possible. Conversion from just memory image to crash/lcrash format will be very useful to use
existing tools and experiences. I already have such tool and (if my
administration allows) I can make such tool open. Let me do some paperwork.
The big part of the conversation that is happening right now is how do we uncouple dependencies between the various parts as much as possible. There is nothing here about format conversions except as to convert weird kernel formats into a stable interface.
There are 3 pieces of code interacting.
1) The primary kernel that will call panic.
2) The kernel+initrd that takes over.
3) The user space that sets it all up (/sbin/kexec) while the primary kernel is still in a sane state.
The goal is to make those 3 pieces as independent of each other as reasonably possible.
So the kernel+initrd that captures a crash dump will live and execute in a reserved area of memory. It needs to know which memory regions are valid, and it needs to know small things like the final register state of each cpu. For the set of valid memory regions it is the intention to encode that as an array of ELF program headers. The information of what the final register contents were will be encoded as ELF notes. There will be one PT_NOTE segment per cpu that holds the notes needed to encode a given cpu's final state. It really does not matter to implementation that captures each cpu's final register state which format we record the data in so using a format designed not to change is not a problem. So all that needs to be communicated to the kernel+initrd that captures a crash dump is the location of an ELF header and it can figure out all of the rest.
For the primary kernel except for remembering it's final cpu register state as it dies it does nothing except jump to the crash recover kernel. All of the interesting information will be exported to user space.
/sbin/kexec is the glue that fills in the cracks. While the primary kernel is in a sane state it sets everything up including finding out which memory areas need to be looked at. And it stashes it all in a reserved area of memory, that has never been the target of DMA transfers.
The goal is to reduce the dependencies as much as possible. So an old stable kernel can take a crash dump of a new buggy kernel. And so that you don't have to be running the latest and greatest user space simply to set everything up. Although it is still better to require a user-space upgrade to cope with new kernels than to require the crash capture kernel+initrd to be upgraded.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Hi,
Quoted 5 lines · On 02 Feb 2005 08:24:03 -0700 [email protected] (Eric W. Biederman) wrote:
So the kernel+initrd that captures a crash dump will live and execute in a reserved area of memory. It needs to know which memory regions are valid, and it needs to know small things like the final register state of each cpu.
Exactly.
Please let me clarify what you are going to.
1) standard kernel: reserve a small contigous area for a dump kernel (this is not changed as the current code)
2) standard kernel: export the information of valid physical memory regions. (/proc/iomem or /proc/cpumem etc.)
3) kexec (system call?): store the information of valid physical memory regions as ELF program header to the reserved area (mentioned 1)).
4) standard kernel: when a panic occur, append (ex.) the register information as ELF note after the memory information (if necessary). and jump new kernel
5) dump kernel: export all valid physical memory (and saved register information) to the user. (as /dev/oldmem /proc/vmcore ?)
Is this correct ? one question: how the dump kernel know the saved area of ELF headers ?
one more question: I don't understand what the 640K backup area is. Please let me know why it is necessary.
Thanks.
Signature · 1 line
Itsuro ODA <[email protected]>
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 12 lines · Itsuro Oda
Hi,
Quoted 5 lines · On 02 Feb 2005 08:24:03 -0700 [email protected] (Eric W. Biederman) wrote:
So the kernel+initrd that captures a crash dump will live and execute in a reserved area of memory. It needs to know which memory regions are valid, and it needs to know small things like the final register state of each cpu.
Exactly.
Quoted 7 lines
Please let me clarify what you are going to.
1) standard kernel: reserve a small contigous area for a dump kernel (this is not changed as the current code)
2) standard kernel: export the information of valid physical memory regions. (/proc/iomemor/proc/cpumemetc.)
3) kexec (system call?): store the information of valid physical memory regions as ELF program header to the reserved area (mentioned 1)).
A better description is probably make a list of memory regions using an ELF header data structure in user space.
Use sys_kexec_load to put that list the dump kernel and a little big of glue code in the reserved area. The glue code includes a hash of all of everything so it can all be validated before use.
Quoted 3 lines
4) standard kernel: when a panic occur, append (ex.) the register information as ELF note after the memory information (if necessary). and jump new kernel
Record the register information as ELF notes in a per cpu data area. The per cpu data areas are known and enumerated in the list of memory regions. The kernel knows nothing about the ELF header etc.
Quoted 2 lines
5) dump kernel: export all valid physical memory (and saved register information) to the user. (as
/dev/oldmem/proc/vmcore?)
Or in user space, by just mmaping /dev/mem. That is part of the
current conversation. The only real point for putting that code in
the kernel (besides momentum) is it is a cheap way to get the exact data structures of the kernel you are using. But since:
(a) it does not look like any primary kernel data structures need to be examined.
(b) even simple compile options like SMP/NOSMP are enough to change the layout of the data structures.
I think there is a pretty good case for moving all of the work to user space. But you still need a kernel that loads and runs in the reserved area.
Quoted 2 lines
Is this correct ? one question: how the dump kernel know the saved area of ELF headers ?
A command line parameter will be passed. Probably elfcorehdr=xxx
Quoted 2 lines
one more question: I don't understand what the 640K backup area is. Please let me know why it is necessary.
In practice I think we can kill it on x86. It is necessary (at least a subset of it is) if we want to boot a SMP kernel. As cpu must start running code in the first 1M of the address space. In addition some architectures have exceptions vectors and or other data structures at fixed locations in memory so in the general case a backup area is required. So building the infrastructure to handle backup areas is needed even, even if we later stop using it on x86.
The other reason for the 640K backup area is the IBM guys were having
problems without it. The fact that you don't need it is a good
indication that it is unnecessary.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Hi,
Quoted 6 lines · On 03 Feb 2005 02:00:51 -0700 [email protected] (Eric W. Biederman) wrote:
A better description is probably make a list of memory regions using an ELF header data structure in user space.
Usesys_kexec_loadto put that list the dump kernel and a little big of glue code in the reserved area. The glue code includes a hash of all of everything so it can all be validated before use.
I see. The data structure is put on a part of loading kernel's data.
Quoted 5 lines
Record the register information as ELF notes in a per cpu data area. The per cpu data areas are known and enumerated in the list of memory regions. The kernel knows nothing about the ELF header etc.
I see.
Quoted 15 lines
Quoted 2 lines
5) dump kernel: export all valid physical memory (and saved register information) to the user. (as
/dev/oldmem/proc/vmcore?)Or in user space, by just mmaping
/dev/mem. That is part of the
current conversation. The only real point for putting that code in
the kernel (besides momentum) is it is a cheap way to get the exact data structures of the kernel you are using. But since:
(a) it does not look like any primary kernel data structures need to be examined.
(b) even simple compile options like SMP/NOSMP are enough to change the layout of the data structures.
I think there is a pretty good case for moving all of the work to user space. But you still need a kernel that loads and runs in the reserved area.
I don't make sense. what do you mean ?
What we want to do when the system is crashed is storing the whole physical memory (and saved register information for x86 arch) to some place (ex. a disk partition) for later analysis. So the basic requirments to the dump kernel is that:
* supply a method to access whole (valid) physical memory.
* supply a method to access the saved register information.
Does the kdump meet this requirment ?
(I am not interesting to /proc/vmcore. Constructing the vmcore
image is area of analysis tools. not kernel's task.)
Thanks.
Signature · 1 line
Itsuro ODA <[email protected]>
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 22 lines · Itsuro Oda
Hi,
Quoted 15 lines · On 03 Feb 2005 02:00:51 -0700 [email protected] (Eric W. Biederman) wrote:
Quoted 2 lines
5) dump kernel: export all valid physical memory (and saved register information) to the user. (as
/dev/oldmem/proc/vmcore?)Or in user space, by just mmaping
/dev/mem. That is part of the
current conversation. The only real point for putting that code in
the kernel (besides momentum) is it is a cheap way to get the exact data structures of the kernel you are using. But since:
(a) it does not look like any primary kernel data structures need to be examined.
(b) even simple compile options like SMP/NOSMP are enough to change the layout of the data structures.
I think there is a pretty good case for moving all of the work to user space. But you still need a kernel that loads and runs in the reserved area.I don't make sense. what do you mean ?
Quoted 8 lines
What we want to do when the system is crashed is storing the whole physical memory (and saved register information for x86 arch) to some place (ex. a disk partition) for later analysis. So the basic requirments to the dump kernel is that:
* supply a method to access whole (valid) physical memory.
* supply a method to access the saved register information.Does the kdump meet this requirment ?
Yes, the discussion in this area is what is the best way to implement
this requirement. How much should be in the kernel and how much
should be in user space.
At the moment things are broken but should be fixed shortly. So what has been implemented are /dev/oldmem which provides access to the old memory. And /proc/vmcore which provides both the old memory and the register information.
Quoted 2 lines
(I am not interesting to
/proc/vmcore. Constructing the vmcore
image is area of analysis tools. not kernel's task.)
There is a fine line there, as a simple ELF core dump has just enough information to describe discontiguous memory, and to have an out of band channel for register information. Adding anything extra like virtual addresses that match the kernel should be left for the crash dump analysis tools.
In code that is currently in the mainstream kernel /dev/mem can mmap any area of memory that is not used by the kernel as ram. So what I believe we will end up is that /sbin/kexec (user space) will prepare an ELF header (data) that describes the memory regions and details where to find the kernels register information. The address of that ELF header will be passed to the crash dump capture kernel and user space combination. The something (probably a user space program reading /dev/mem) will look at the ELF header and save the already prepared ELF core dump to disk. Possibly doing little things like merging the MAX_NR_CPUS note segments into one so it actually conforms to the ELF spec.
This thread started as the design discussion before finishing
that part of the implementation. The proof of concept
implementations have happened. We have all seen this kind of functionality implemented. Now is the time to come up with a good solid design that can be maintained and merged into the mainline kernel and distros.
So thank you for ask questions, it means we have a better chance of getting a solid design and a design that those people who care about this functionality can use. And with a little luck we can all wind up on agreeing on the general principles. You came in a little late to this conversation so a lot of details have been settled, but if you have a good argument for doing something another way we can certainly look at that.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
In reply to Itsuro OdaHi,
Quoted 16 lines · On Fri, 04 Feb 2005 08:18:56 +0900 Itsuro Oda <[email protected]> wrote:
Quoted 4 lines
Quoted 3 lines
5) dump kernel: export all valid physical memory (and saved register information) to the user. (as
/dev/oldmem/proc/vmcore?)Quoted 12 lines
Or in user space, by just mmaping
/dev/mem. That is part of the
current conversation. The only real point for putting that code in
the kernel (besides momentum) is it is a cheap way to get the exact data structures of the kernel you are using. But since:
(a) it does not look like any primary kernel data structures need to be examined.
(b) even simple compile options like SMP/NOSMP are enough to change the layout of the data structures.
I think there is a pretty good case for moving all of the work to user space. But you still need a kernel that loads and runs in the reserved area.
Quoted 2 lines
I don't make sense. what do you mean ?
"I don't make sense." should be "It does not make sense." sorry. I'm not familiar with English.
Signature · 1 line
Itsuro ODA <[email protected]>
PATCH 1/1 /proc/cpumem
In reply to Eric W. BiedermanHi, Eric and all
Attached is an implementation of /proc/cpumem. /proc/cpumem shows the valid physical memory ranges.
* i386 and x86_64
* implement valid_phys_addr_range() and use it.
(the first argument of the i386 version is little uncomfortable.)
* /dev/mem of the i386 version should be mofified. but not yet.
example: amd64 8GB Mem
# cat /proc/cpumem 0000000000000000 000000000009b800 0000000000100000 00000000fbe70000 0000000100000000 0000000100000000 # start address and size. hex digit.
Any comments, recomendations and suggestions are welcom.
BTW, does not kexec/kdump run on 2.6.11-rc3-mm2 ?
How do I get and examine the latest kexec/kdump ?
Thanks.
Signature · 266 lines
Itsuro ODA <[email protected]> --- --- linux-2.6.11-rc3-mm2/drivers/char/mem.c 2005-02-16 15:36:31.000000000 +0900 +++ linux-2.6.11-rc3-mm2-test/drivers/char/mem.c 2005-02-16 23:32:15.244876816 +0900 @@ -25,6 +25,9 @@ #include <linux/device.h> #include <linux/highmem.h> #include <linux/crash_dump.h> +#include <linux/bootmem.h> +#include <linux/proc_fs.h> +#include <linux/seq_file.h> #include <asm/uaccess.h> #include <asm/io.h> @@ -759,3 +762,125 @@ } fs_initcall(chr_dev_init); + +#ifdef CONFIG_PROC_FS +/* + * /proc/cpumem: show valid physical address range + */ +struct cpumem_info { + unsigned long long addr; + unsigned long long size; +}; + +static void *cpumem_next(struct seq_file *m, void *v, loff_t *pos) +{ + struct cpumem_info *p = m->private; + unsigned long long end = (unsigned long long)max_pfn << PAGE_SHIFT; + unsigned long long addr; + size_t size; + int found = 0; + + (*pos)++; + + if (p->addr >= end) { + return NULL; + } + + /* always start page boundary */ + addr = ((p->addr + p->size + PAGE_SIZE - 1) >> PAGE_SHIFT) << PAGE_SHIFT; + size = 0xf0000000; + + while (addr < end) { + if (valid_phys_addr_range(addr, &size)) { + if (!found) { + found = 1; + p->addr = addr; + p->size = size; + } else { + p->size += size; + } + addr += size; + size = 0xf0000000; + } else { + if (found) { + return p; + } + addr += PAGE_SIZE; + } + } + + return found ? p : NULL; +} + +static void *cpumem_start(struct seq_file *m, loff_t *pos) +{ + struct cpumem_info *p = m->private; + loff_t n = 0; + + p->addr = 0; + p->size = 0; + + while (n <= *pos) { + if (!cpumem_next(m, NULL, &n)) { + return NULL; + } + } + + return p; +} + +static void cpumem_stop(struct seq_file *m, void *v) +{ +} + +static int cpumem_show(struct seq_file *m, void *v) +{ + struct cpumem_info *p = m->private; + unsigned long long end = (unsigned long long)max_pfn << PAGE_SHIFT; + + if (p->addr < end) { + seq_printf(m, "%016llx %016llx\n", p->addr, p->size); + } + return 0; +} + +struct seq_operations cpumem_op = { + .start = cpumem_start, + .next = cpumem_next, + .stop = cpumem_stop, + .show = cpumem_show +}; + +static int cpumem_open(struct inode *inode, struct file *file) +{ + int res = seq_open(file, &cpumem_op); + if (!res) { + struct seq_file *m = file->private_data; + m->private = kmalloc(sizeof(struct cpumem_info), GFP_KERNEL); + if (!m->private) { + seq_release(inode, file); + return -ENOMEM; + } + } + return res; +} + +static struct file_operations proc_cpumem_operations = { + .open = cpumem_open, + .read = seq_read, + .llseek = seq_lseek, + .release = seq_release_private +}; + +static int __init cpumem_init(void) +{ + struct proc_dir_entry *entry; + + entry = create_proc_entry("cpumem", 0, NULL); + if (entry) { + entry->proc_fops = &proc_cpumem_operations; + } + return 0; +} +__initcall(cpumem_init); +#endif /* CONFIG_PROC_FS */ --- --- linux-2.6.11-rc3-mm2/arch/i386/mm/init.c 2005-02-16 15:36:29.000000000 +0900 +++ linux-2.6.11-rc3-mm2-test/arch/i386/mm/init.c 2005-02-16 23:32:29.499709752 +0900 @@ -248,6 +248,47 @@ return 0; } +int valid_phys_addr_range(unsigned long long phys_addr, size_t *size) +{ + int i; + unsigned long long addr, end; + efi_memory_desc_t *md; + + if (efi_enabled) { + for (i = 0; i < memmap.nr_map; i++) { + md = &memmap.map[i]; + if (!is_available_memory(md)) { + continue; + } + addr = md->phys_addr; + end = md->phys_addr + (md->num_pages << EFI_PAGE_SHIFT); + if ((phys_addr >= addr) && (phys_addr < end)) { + if (*size > end - phys_addr) { + *size = end - phys_addr; + } + return 1; + } + } + return 0; + } + + for (i = 0; i < e820.nr_map; i++) { + if (e820.map[i].type != E820_RAM) { + continue; + } + addr = e820.map[i].addr; + end = e820.map[i].addr + e820.map[i].size; + if ((phys_addr >= addr) && (phys_addr < end)) { + if (*size > end - phys_addr) { + *size = end - phys_addr; + } + return 1; + } + } + return 0; +} +EXPORT_SYMBOL(valid_phys_addr_range); + #ifdef CONFIG_HIGHMEM pte_t *kmap_pte; pgprot_t kmap_prot; --- --- linux-2.6.11-rc3-mm2/include/asm-i386/io.h 2004-12-25 06:35:40.000000000 +0900 +++ linux-2.6.11-rc3-mm2-test/include/asm-i386/io.h 2005-02-16 23:36:24.454991120 +0900 @@ -90,6 +90,12 @@ */ #define page_to_phys(page) ((dma_addr_t)page_to_pfn(page) << PAGE_SHIFT) +/* + * for /dev/mem + */ +#define ARCH_HAS_VALID_PHYS_ADDR_RANGE +extern int valid_phys_addr_range(unsigned long long, size_t *); + extern void __iomem * __ioremap(unsigned long offset, unsigned long size, unsigned long flags); /** --- --- linux-2.6.11-rc3-mm2/arch/x86_64/mm/init.c 2005-02-16 15:36:30.000000000 +0900 +++ linux-2.6.11-rc3-mm2-test/arch/x86_64/mm/init.c 2005-02-16 16:23:08.000000000 +0900 @@ -22,6 +22,7 @@ #include <linux/pagemap.h> #include <linux/bootmem.h> #include <linux/proc_fs.h> +#include <linux/module.h> #include <asm/processor.h> #include <asm/system.h> @@ -395,6 +396,27 @@ return 0; } +int valid_phys_addr_range(unsigned long phys_addr, size_t *size) +{ + int i; + unsigned long end; + + for (i = 0; i < e820.nr_map; i++) { + if (e820.map[i].type != E820_RAM) { + continue; + } + end = e820.map[i].addr + e820.map[i].size; + if (phys_addr >= e820.map[i].addr && phys_addr < end) { + if (*size > end - phys_addr) { + *size = end - phys_addr; + } + return 1; + } + } + return 0; +} +EXPORT_SYMBOL(valid_phys_addr_range); + extern int swiotlb_force; /* --- --- linux-2.6.11-rc3-mm2/include/asm-x86_64/io.h 2005-02-16 15:36:12.000000000 +0900 +++ linux-2.6.11-rc3-mm2-test/include/asm-x86_64/io.h 2005-02-16 16:23:59.000000000 +0900 @@ -123,6 +123,9 @@ { return __va(address); } + +#define ARCH_HAS_VALID_PHYS_ADDR_RANGE +extern int valid_phys_addr_range(unsigned long, size_t *); #endif /* ---
Re: [PATCH] /proc/cpumem
Quoted 4 lines · Itsuro Oda
Hi, Eric and all
Attached is an implementation of
/proc/cpumem./proc/cpumemshows the valid physical memory ranges.
Interesting. My imagination when I proposed this was something based on struct resource that works like /proc/iomem on x86 but can be meaningfully be used on systems with where ram lives in a separate address space from io device memory.
Quoted 7 lines
example: amd64 8GB Mem
# cat /proc/cpumem 0000000000000000 000000000009b800 0000000000100000 00000000fbe70000 0000000100000000 0000000100000000 # start address and size. hex digit.
The lack of a type field looses a fair amount of functionality compared to /proc/iomem. In particular you can't see where the ACPI data is.
The other direction something like this can go is to dump the data structures in linux/mmzone.h
Quoted 4 lines
Any comments, recomendations and suggestions are welcom.
BTW, does not kexec/kdump run on 2.6.11-rc3-mm2 ?
How do I get and examine the latest kexec/kdump ?
I'm not quite certain what is happening.
I have been playing with kexec user space a little bit and a new development release is at:
http://www.xmission.com/~ebiederm/files/kexec/kexec-tools-1.101.tar.gz
I have written a first pass at a user space core dump generator, using /dev/mem. /sbin/kexec still needs some work to prepare the ELF headers before a crash.
Eric
Re: [PATCH] /proc/cpumem
Hi Eric,
Quoted 2 lines
The lack of a type field looses a fair amount of functionality compared to
/proc/iomem. In particular you can't see where the ACPI data is.
Hmm, restricting System RAM only may be too pessimistic. (One of motivations of this work is for using /dev/mem safely.
"dd if=/dev/mem of=xxx" causes panic on my amd64(8GB mem) machine since reading from address around 0xfe000000 causes a machine check. hmm, this area is marked as "reserved". not ACPI area. ACPI area can be read.)
Ok, I will add a type field.
Quoted 2 lines
The other direction something like this can go is to dump the data structures in
linux/mmzone.h
Do you mean defining a data structure in linux/mmzone.h ?
I used to think a particular struct is not necessary for this work, but now I think it is better to define a struct for this. Let me consider.
Quoted 3 lines
I have written a first pass at a user space core dump generator, using
/dev/mem./sbin/kexecstill needs some work to prepare the ELF headers before a crash.
I am looking forward this :-)
And, you mentioned a couple of weeks ago:
Quoted 5 lines
Anyway one thing I want to do is actually drop the apic shutdown code altogether in this code path. I threw it in there to ease the transition from the old code base to the new, but if that code is causing issues.... So this is probably a good time to start testing that.
How about this ?
Thanks.
Signature · 1 line
Itsuro ODA <[email protected]>
Re: [Fastboot] Re: [PATCH] /proc/cpumem
Quoted 5 lines · Itsuro Oda
Hi Eric,
Quoted 2 lines
The lack of a type field looses a fair amount of functionality compared to
/proc/iomem. In particular you can't see where the ACPI data is.
Quoted 8 lines
Hmm, restricting System RAM only may be too pessimistic. (One of motivations of this work is for using
/dev/memsafely.
"dd if=/dev/mem of=xxx" causes panic on my amd64(8GB mem) machine since reading from address around0xfe000000causes a machine check. hmm, this area is marked as "reserved". not ACPI area. ACPI area can be read.)Ok, I will add a type field.
To be very clear. I do not believe is necessary for x86. The is already sufficient information elsewhere to handle this.
Quoted 8 lines
Quoted 2 lines
The other direction something like this can go is to dump the data structures in
linux/mmzone.hDo you mean defining a data structure in
linux/mmzone.h?I used to think a particular struct is not necessary for this work, but now I think it is better to define a struct for this. Let me consider.
To be clear there are two pieces of information that are needed.
1) The list of physical memory areas and what they are. /proc/iomem does a good job of this.
2) The list of which memory areas the kernel is using. It is the pgdat_t and related structures that define this. For the purposes of a core dump we want to capture this information before the kernel crashes and use it afterward.
Quoted 6 lines
Quoted 3 lines
I have written a first pass at a user space core dump generator, using
/dev/mem./sbin/kexecstill needs some work to prepare the ELF headers before a crash.I am looking forward this :-)
Quoted 8 lines
And, you mentioned a couple of weeks ago:
Quoted 5 lines
Anyway one thing I want to do is actually drop the apic shutdown code altogether in this code path. I threw it in there to ease the transition from the old code base to the new, but if that code is causing issues.... So this is probably a good time to start testing that.
How about this ?
My role in this is that of maintainer and architect. On a practical level I gain nothing from a working crash-dump/kexec-on-panic implementation except it stops being a gating factor for the rest of the kexec code. So while many times I can see what needs to be done it is hard for me to justify doing it. So a lot of times where I will weigh in with code is when I see a particular blind spot on the part of the implementors.
The parties I see actively working on the crash dump implementation are currently a group from IBM and you guys from valinux.co.jp. One of the primaries at IBM has been on vacation which is likely why we have not seen anything out of them for the last couple of weeks.
But also this is open source software it will be done when it is done.
Eric
Re: [Fastboot] Re: [PATCH] /proc/cpumem
Hi,
Quoted 7 lines · On 17 Feb 2005 02:55:31 -0700 [email protected] (Eric W. Biederman) wrote:
My role in this is that of maintainer and architect. On a practical level I gain nothing from a working crash-dump/kexec-on-panic implementation except it stops being a gating factor for the rest of the kexec code. So while many times I can see what needs to be done it is hard for me to justify doing it. So a lot of times where I will weigh in with code is when I see a particular blind spot on the part of the implementors.
I see. I would like to contribute as possible I can.
Thanks.
Signature · 1 line
Itsuro ODA <[email protected]>
Re: [Fastboot] Re: [PATCH] /proc/cpumem
Quoted 1 line · Itsuro Oda
I see. I would like to contribute as possible I can.
Pick some piece you that have an affinity for and work on it. Problems are best solved by those who see them and by those who care :)
I believe Vivek Goyal is currently working on the remaining user space piece, and expects to have something in a week or so.
Eric
Re: [PATCH] /proc/cpumem
In reply to Itsuro Odahi,
Quoted 8 lines
(quoted diff) +8
-0+ while (addr < end) { + if (valid_phys_addr_range(addr, &size)) { + if (!found) { + found = 1; + p->addr = addr; + p->size = size; + } else { + p->size += size;
Review
Quoted 10 lines
(quoted diff) +10
-0+ } + addr += size; + size = 0xf0000000; + } else { + if (found) { + return p; + } + addr += PAGE_SIZE; + } + }
doesn't this loop take very long time if you have a large hole?
i'd suggest to change valid_phys_addr_range to fill &size even when it returns false, so that caller can skip the hole efficiently.
YAMAMOTO Takashi
Re: [Fastboot] [PATCH] /proc/cpumem
In reply to Itsuro OdaHi,
Quoted 3 lines · Itsuro Oda
BTW, does not kexec/kdump run on 2.6.11-rc3-mm2 ?
How do I get and examine the latest kexec/kdump ?
Currently kdump is broken. I am working on Elf Header generation part in kexec-tools. Next week I should be able to post the initial patches.
---
Quoted 22 lines
---
linux-2.6.11-rc3-mm2/arch/i386/mm/init.c2005-02-16
15:36:29.000000000 +0900
+++linux-2.6.11-rc3-mm2-test/arch/i386/mm/init.c2005-02-16
23:32:29.499709752 +0900(quoted diff) +14
-0@@ -248,6 +248,47 @@ return 0; } +int valid_phys_addr_range(unsigned long long phys_addr, size_t *size) +{ + int i; + unsigned long long addr, end; + efi_memory_desc_t *md; + + if (efi_enabled) { + for (i = 0; i < memmap.nr_map; i++) { + md = &memmap.map[i]; + if (!is_available_memory(md)) { + continue; + } + addr = md->phys_addr; + end = md->phys_addr + (md->num_pages <<
Review line 274
Quoted 10 lines
(quoted diff) +9
-0EFI_PAGE_SHIFT); + if ((phys_addr >= addr) && (phys_addr < end)) { + if (*size > end - phys_addr) { + *size = end - phys_addr; + } + return 1; + } + } + return 0; + }
I thought efi related data structures are of type __initdata and will be gone after initilization. (efi.c)
Thanks
Vivek
Re: [Fastboot] [PATCH] /proc/cpumem
Hi,
Quoted 1 line
I thought efi related data structures are of type
__initdataand will be gone after initilization. (efi.c)
oops. certainly.
and, devmem_is_allowed does same mistake :-)
(I don't know who made it.)
Thanks.
Signature · 1 line
Itsuro ODA <[email protected]>
Re: [PATCH] /proc/cpumem
In reply to Itsuro OdaQuoted 11 lines · Itsuro Oda
Hi, Eric and all
Attached is an implementation of
/proc/cpumem./proc/cpumemshows the valid physical memory ranges.* i386 and
x86_64
* implementvalid_phys_addr_range()and use it.
(the first argument of the i386 version is little uncomfortable.)
*/dev/memof the i386 version should be mofified. but not yet.example: amd64 8GB Mem
Quoted 8 lines
# cat /proc/cpumem 0000000000000000 000000000009b800 0000000000100000 00000000fbe70000 0000000100000000 0000000100000000 # start address and size. hex digit.Any comments, recomendations and suggestions are welcom.
It may make more sense to export the entire e820 (or similar) bios memory tables. Probably better off in sysfs than adding more cruft to procfs too.
Dave
Re: [Fastboot] Re: [PATCH] /proc/cpumem
Quoted 19 lines · Dave Jones
Quoted 18 lines · Itsuro Oda
Hi, Eric and all
Attached is an implementation of
/proc/cpumem./proc/cpumemshows the valid physical memory ranges.* i386 and
x86_64
* implementvalid_phys_addr_range()and use it.
(the first argument of the i386 version is little uncomfortable.)
*/dev/memof the i386 version should be mofified. but not yet.example: amd64 8GB Mem
# cat /proc/cpumem 0000000000000000 000000000009b800 0000000000100000 00000000fbe70000 0000000100000000 0000000100000000 # start address and size. hex digit.
Quoted 5 lines
Quoted 1 line
Any comments, recomendations and suggestions are welcom.
It may make more sense to export the entire e820 (or similar) bios memory tables. Probably better off in sysfs than adding more cruft to procfs too.
Agreed. In practice we actually do this already with /proc/iomem. Except that we truncate everything above 4GB, and we allow the map to get mangled with mem=xxx options.
I brought up the idea of a /proc/cpumem by analogy because on platforms that have an iommu and memory is in a distinct address space there have been complaints that /proc/iomem just won't work. But it is simple enough to do something that is just for the cpu's memory.
As for how to do this cleanly this looks like the start of that discussion.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
In reply to Itsuro OdaQuoted 3 lines · Itsuro Oda
Hi,
I can't understand why ELF format is necessary.
ELF format is not. However essentially the information an ELF provides is. So using an ELF header to convey that information is a sane choice of data structure.
Quoted 4 lines
I think the only necessary information is "what physical address regions are valid to read". This information is necessary for any sort of dump tools. (and must get it while the system is normal.) The Eric's
/proc/cpumemidea sounds nice to me.
Patches welcome.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
In reply to Eric W. BiedermanQuoted 15 lines · Eric W. Biederman
Quoted 6 lines · Vivek Goyal
Well, trying to put the already discussed ideas together. I was planning to work on following design. Please comment.
Crashed Kernel <-->Capture Kernel(or User Space) Interface:
----------------------------------------------------------Quoted 6 lines
The whole idea is that Crash image is represented in ELF Core format. These ELF Headers are prepared by kexec-tools user space and put in one segment. Address of start of image is passed to the capture kernel(or user space) using one command line (eg. crashimage=). Now either kernel space or user space can parse the elf headers and extract required information and export final kernel elf core image.
Quoted 7 lines
Sounds sane. We need to make certain there is a checksum of that region but putting it in a separate segment should ensure that.
I also think we need to look at the name crashimage= and see if we can find something more descriptive. But that is minor. Possibly elfcorehdr= We have a little while to think about that one before we are stuck.
"elfcorehdr=" also looks good.
Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
[email protected] wrote:
If we were using an ELF header I would include onePT_NOTEprogram header per cpu (Giving each cpu it's own area to mess around in). And I would use onePT_LOADsegment per possible memory zone. So in the worst case (current sgi altix) (MAX_NUMNODES=256, MAX_NR_ZONES=3, MAX_NR_CPUS=1024) 256*3+1024 = 1792 program
headers. At 56 bytes per 64bit program header that is 100352 bytes
or 98KiB. A little bit expensive. A tuned data structure with 64bit base and size would only consume 1792*16 = 28672 or 28KiB.
Quoted 8 lines
Quoted 3 lines
If I prepare One elf header for each physical contiguous memory area (as obtained from
/proc/iomem) instead of per zone, then number of elf headers will come down significantly.A clarification on terminology we are talking about struct
Elf64_Phdrhere. There is only one Elf header. That seems to be clear farther down.
Exactly. There shall be one Elf header for whole of the image. In addition there will be one struct Elf64_Phdr, per contiguous physical memory area. One Elf64_Phdr of PT_NOTE type for notes section and one Elf64_Phdr for backup region.
Quoted 14 lines
Quoted 8 lines
I don't have any idea on number of
actual physically contiguous regions present per machine, but roughly assuming it to be 1 per node, it will lead to 256 + 1024 = 1280 program headers.At 56 bytes per 64 bit program header this will amount to 70KB.This is worst case estimate and on lower end machines this will require much less a space. On machines as big as 1024 cpus, this should not be a concern, as big machines come with big RAMs.
Agreed. Size is not the primary issue. There is some clear waste but that is a secondary concern. Not performing a 1-1 mapping to the kernel data structures also seems to be a win, as the concepts are noticeably different.
Quoted 7 lines
Quoted 2 lines
Eric, do you still think that ELF headers are inappropriate to be passed across interface boundary.
I have serious concerns about the kernel generating the ELF headers and only delivering them after the kernel has crashed. Because then we run into questions of what information can be trusted. If we avoid that issue I am not too concerned.
I hope, all elf headers once prepared by kexec-tools need not to change later (Cannot think of any piece of information which shall change later). These shall be put in separate segment. And SHA-256 shall take care of authenticity of information after crash.
Quoted 11 lines
Quoted 10 lines
Regarding Backup Region
------------------------ Kexec user space does the reservation for backup region segment.
- Purgatory copies the backup data to backup region. (Already implemented)
- A separate elf header is prepared to represent backed up memory region. And "offset" field of this program header can contain the actual physical address where backup contents are stored.
Quoted 7 lines
I like that. I was thinking a virtual versus physical address separation. But using the offset field is much more appropriate, and it leaves us the potential of doing something nice like specifying the kernels virtual address later on. Looking exclusively at the offset field to know which memory addresses to dump sounds good. For now we should have virtual==physical==offset except for the backup region.
For notes section program header, virtual = physical = 0 and "offset" shall point to crash_notes[], so that notes can directly be read by the capture kernel (or user space).
Thanks
Vivek
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 4 lines · Vivek Goyal
Quoted 1 line · Eric W. Biederman
Vivek Goyal <[email protected]> writes:
"elfcorehdr=" also looks good.
Then let's go with that for now. It is not perfect but it seems a little more self explanatory at first glance.
Quoted 10 lines
Quoted 4 lines
A clarification on terminology we are talking about struct
Elf64_Phdrhere. There is only one Elf header. That seems to be clear farther down.Exactly. There shall be one Elf header for whole of the image. In addition there will be one struct
Elf64_Phdr, per contiguous physical memory area. OneElf64_PhdrofPT_NOTEtype for notes section and oneElf64_Phdrfor backup region.
Actually if we are just pointing a kernel data structures we will need multiple Elf64_Phdr of PT_NOTE. Each cpu has it's own notes section and until the smoke clears we can't be confident about what is going to wind up there or how densely those will be packed. So collapsing everything into a single notes segment needs to happen after we have switched to the crash capture kernel.
Quoted 10 lines
Quoted 4 lines
I have serious concerns about the kernel generating the ELF headers and only delivering them after the kernel has crashed. Because then we run into questions of what information can be trusted. If we avoid that issue I am not too concerned.
I hope, all elf headers once prepared by kexec-tools need not to change later (Cannot think of any piece of information which shall change later). These shall be put in separate segment. And SHA-256 shall take care of authenticity of information after crash.
That should work fine. We need to consider through throwing in an extra note section with information like kernel version that we can capture while the system is running.
Quoted 3 lines
For notes section program header, virtual = physical = 0 and "offset" shall point to crash_notes[], so that notes can directly be read by the capture kernel (or user space).
I agree. But see my caveat. I think we should have one PT_NOTE segment point at each element of the crash_notes[] array. I know it is technically a violation of the ELF spec. But in this case
it makes sense. Since we can't guarantee that crash_notes will
be packed properly I don't know that we could reliably see more than one cpu if we pointed a PT_NOTE header at the whole thing.
If it turns out that we can reliably point a single PT_NOTE header at crash_notes so much the better but things are likely to be more robust if we don't start with that assumption. That at least allows us the freedom to capture some notes (like NT_UTSNAME) before the kernel crashes.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 20 lines · Eric W. Biederman
Quoted 4 lines · Vivek Goyal
Quoted 1 line · Eric W. Biederman
Vivek Goyal <[email protected]> writes:
"elfcorehdr=" also looks good.
Then let's go with that for now. It is not perfect but it seems a little more self explanatory at first glance.
Quoted 10 lines
Quoted 4 lines
A clarification on terminology we are talking about struct
Elf64_Phdrhere. There is only one Elf header. That seems to be clear farther down.Exactly. There shall be one Elf header for whole of the image. In addition there will be one struct
Elf64_Phdr, per contiguous physical memory area. OneElf64_PhdrofPT_NOTEtype for notes section and oneElf64_Phdrfor backup region.
Quoted 6 lines
Actually if we are just pointing a kernel data structures we will need multiple
Elf64_PhdrofPT_NOTE. Each cpu has it's own notes section and until the smoke clears we can't be confident about what is going to wind up there or how densely those will be packed. So collapsing everything into a single notes segment needs to happen after we have switched to the crash capture kernel.
Sounds good. So there shall be a PT_NOTE type program header per cpu. And these headers can be collapsed into one PT_NOTE type header later.
Quoted 35 lines
Quoted 11 lines
Quoted 5 lines
I have serious concerns about the kernel generating the ELF headers and only delivering them after the kernel has crashed. Because then we run into questions of what information can be trusted. If we avoid that issue I am not too concerned.
I hope, all elf headers once prepared by kexec-tools need not to change later (Cannot think of any piece of information which shall change later). These shall be put in separate segment. And SHA-256 shall take care of authenticity of information after crash.
That should work fine. We need to consider through throwing in an extra note section with information like kernel version that we can capture while the system is running.
Quoted 3 lines
For notes section program header, virtual = physical = 0 and "offset" shall point to crash_notes[], so that notes can directly be read by the capture kernel (or user space).
I agree. But see my caveat. I think we should have one
PT_NOTEsegment point at each element of the crash_notes[] array. I know it is technically a violation of the ELF spec. But in this case
it makes sense. Since we can't guarantee thatcrash_noteswill
be packed properly I don't know that we could reliably see more than one cpu if we pointed aPT_NOTEheader at the whole thing.If it turns out that we can reliably point a single
PT_NOTEheader atcrash_notesso much the better but things are likely to be more robust if we don't start with that assumption. That at least allows us the freedom to capture some notes (likeNT_UTSNAME) before the kernel crashes.Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
In reply to Eric W. BiedermanHook in panic code is very good idea and is useful in various scenes. It could be used to kick RAM dump code, obviously, and also kick the code to initiate failover, etc. Various use could be possible so I believe that this hook should be prepared for wider use.
--
Koichi Suzuki
NTT DATA Intellilink Corp.
Quoted 186 lines · [email protected] wrote:
For the guys on ppc, and other architectures that have all of their cpu memory behind an iommu. I propose we create a
/proc/cpumemwhich is the subset of/proc/iomemthat deals with RAM. In any event as something like that is straight forward to implement I will assume the existence of the functionality and we can attack the details when we do the merge the first of those architectures into the kernel.Quoted 5 lines · Vivek Goyal
Hi Eric,
It looks like we are looking at things a little differently. I see a portion of the picture in your mind, but obviously not entirely.
Quoted 7 lines
Perhaps, we need to step back and iron out in specific terms what the interface between the two kernels should be in the crash dump case, and the distribution of responsibility between kernel, user space
and the user.[BTW, the patch was intended as a step in development up for
comment early enough to be able to get agreement on the interface
and think issues through to more completeness before going too far. Sorry, if that wasn't apparent.]It wasn't quite, and the fact that Andrew picked it up added to the confusion.
Quoted 4 lines
When you say "evil intermingling", I'm guessing you mean the "crashbackup=" boot parameter ? If so, then yes, I agree it'd be nice to find a way around it that doesn't push hardcoding elsewhere.
I believe there are some alternatives to crashbackup= in the crashdump capture kernel. But as long as that code is running in the kernel we can't do a lot better.
However for the primary kernel it has no need to know that we even have a backup region, nor does it need to know about the size of the backup region. That can all be handled with the single reservation, we have now.
/sbin/kexecwhich makes the backup needs to know about it and it needs to pass that information on. But the primary kernel does not.The largest reason I am sensitive to this issue is that if you are not booting an SMP kernel I don't believe we need a backup region on x86 at all. If we can remove that dependency I want the freedom to do that without having to modify the primary kernel. Or if we discover we need to preserve other things like the ACPI, mp and pirq tables I don't want to require patching the kernel just so I can copy those and preserve them.
Quoted 5 lines
Let me explain the interface/approach I was looking at.
1.First kernel reserves some area of memory for crash/capture kernel as specified by crashkernel=X@Y boot time parameter.
Quoted 8 lines
2.First kernel marks the top 640K of this area as backup area. (If architecture needs it.) This is sort of a hardcoding and probably this space reservation can be managed from user space as well as mentioned by you in this mail below.
3. Location of backup region is exported through
/proc/iomemwhich can be read by user space utility to pass this information to purgatory code to determine where to copy the first 640K.And 1-3 can be done in
/sbin/kexec. And if it is done there we can increase our freedom of implementation in the crashdump capture process quite a bit.Quoted 5 lines
Note that we do not make any additional reservation for the backup region. We carve this out from the top of the already reserved region and export it through
/proc/iomemso that the user space code and the capture kernel code need not make any assumptions about where this region is located.4. Once the capture kernel boots, it needs to know the location of backup region for two purposes.
Quoted 8 lines
a. It should not overwrite the backup region.
b. There needs to be a way for the capture tool to access the original
contents of the backed up regionBoot time parameter crashbackup=A@B has been provided to pass this information to capture kernel. This parameter is valid only for capture kernel and becomes effective only if
CONFIG_CRASH_DUMPis enabled.But that is not what you implemented. crashbackup= was an alternative to the carving out of 640K in parts 1-3.
Quoted 5 lines
Quoted 2 lines
What is wrong with user space doing all of the extra space reservation?
Just for clarity, are you suggesting kexec-tools creating an additional segment for the backup region and pass the information to kernel.
Yes, having kexec create a bss segment for the backup region would be a good idea. It will keep us from stomping on the kernel trampoline (think the identity mapped
x86_64page tables here) by accident.Quoted 8 lines
There is no problem in doing reservation from user space except one. How does the user and in-turn capture kernel come to know the location of backup region, assuming that the user is going to provide the exactmap for capture kernel to boot into.
Just a thought, is it a good idea for kexec-tools to be creating and passing memmap parameters doing appropriate adjustment for backup region.
Exactly. Having
/sbin/kexecdo this instead of the user doing this manually is a much simpler solution than we have now.Quoted 1 line
I had another question. How is the starting location of elf headers communicated to capture tool? Is parameter segment a good idea? or some hardcoding?
I recognize the need for that information. But I do not recognize the need for it to be an ELF header (we do need something conceptually close). If we don't have regions of the memory map appearing and disappearing dynamically we can get this information from
/proc/iomem, before the crash and store it in one of the data segments that we checksum.Quoted 8 lines
Another approach can be that backup area information is encoded in elf headers and capture kernel is booted with modified memmap (User gets backup region information from
/proc/iomem) and capture tool can extract backup area information from elf headers as stored by first kernel.Could you please elaborate a little more on what aspect of your view differs from the above.
See above.
The direction I would take if I was to take to implementing this the crashdump functionality is something different.
Instead of patching crashdump functionality into the kernel, I would create a subdirectory in kexec-tools called crashdump and put in the source for a user-space program that could run as init. In addition I would but in the code to generate and initramfs cpio.gz archive of that program. And I would build the program against uclibc, klibc or one of the other libc variants that actually allows building static binaries. Unless something has changed recently glibc does not all for truly static binaries as it dynamically open
/lib/libnss*Given the pain of building against an external library that is not widely distributed I would probably take a snapshot of the code and place it in crashdump/libc in the kexec-tools source. Taking a snapshot of frequently used libraries is commonly done with the gnu toolchain and is wonderfully effective in resolving painful dependencies.
The crashdump /init would just mmap
/dev/memto read the raw memory.Quoted 1 line
From there it would generate the core file.
When kexec'ing a panic kernel I would simply have
/sbin/kexecunconditionally load that cpio.gz as the initrd and things would work.The large advantage of doing all of this in user space is that it moves all of the crashdump policy into user space and into one source tree, for simplified maintenance.
However as long as we gracefully handle the interface between the primary kernel and the capture kernel we can switch mechanisms for actually taking the crash dump, kernel based or user space as seems most sane.
Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 4 lines · Koichi Suzuki
Hook in panic code is very good idea and is useful in various scenes. It could be used to kick RAM dump code, obviously, and also kick the code to initiate
failover, etc. Various use could be possible so I believe that this hook
should be prepared for wider use.
It is. Basically it is the normal kexec interface that allows you to boot another kernel. With a few restrictions that should keep it as reliable as possible when the kernel has not shut itself down cleanly.
The hardest case is to do a useful system core dump. As that requires looking at what has gone before. For the rest if you can do it with a kernel and a initramfs you are in good shape.
There seems to be a significant amount of interest in the full system core dump case so that is what the work is concentrating on.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 17 lines · [email protected] wrote:
Quoted 4 lines · Koichi Suzuki
Hook in panic code is very good idea and is useful in various scenes. It could be used to kick RAM dump code, obviously, and also kick the code to initiate
failover, etc. Various use could be possible so I believe that this hook
should be prepared for wider use.It is. Basically it is the normal kexec interface that allows you to boot another kernel. With a few restrictions that should keep it as reliable as possible when the kernel has not shut itself down cleanly.
The hardest case is to do a useful system core dump. As that requires looking at what has gone before. For the rest if you can do it with a kernel and a initramfs you are in good shape.
Quoted 6 lines
There seems to be a significant amount of interest in the full system core dump case so that is what the work is concentrating on.
Eric
I meant with kexec and dump hook, there could be many more things can be done in addition to full core dump. Initiating failover to other node will be one example. Starting with this hook, there must be many good ideas. So my idea is to make this hook general purpose, not for specific core dump tool.
Koichi Suzuki
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 4 lines · Koichi Suzuki
I meant with kexec and dump hook, there could be many more things can be done in addition to full core dump. Initiating failover to other node will be one
example. Starting with this hook, there must be many good ideas. So my idea
is to make this hook general purpose, not for specific core dump tool.
Again that is what is has been implemented. A fully stand alone executable that lives in an independent and reserved address in memory is jumped to.
The goal in the generic kernel is to keep the code path to do that as small and as simple as possible to reduce the chances of it being mis-implemented, or the chances of attempting to use corrupted kernel functionality.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
In reply to Eric W. BiedermanHi,
I don't like calling crash_kexec() directly in (ex.) panic(). It should be call_dump_hook() (or something like this).
I think the necessary modifications of the kernel is only:
- insert the hooks that calls a dump function when crash occur
- binding interface that binds a dump function to the hook (like register_dump_hook())
- supply the information of valid physical address regions (- maybe some existent functions and variables need to be exported ?)
I think this makes any sort of dump functions can be implemented as a kernel module. I don't think it is best way that the "kexec based crashdump" is built in the kernel.
Thanks.
Quoted 25 lines · On 01 Feb 2005 02:06:42 -0700 [email protected] (Eric W. Biederman) wrote:
Quoted 4 lines · Koichi Suzuki
Hook in panic code is very good idea and is useful in various scenes. It could be used to kick RAM dump code, obviously, and also kick the code to initiate
failover, etc. Various use could be possible so I believe that this hook
should be prepared for wider use.It is. Basically it is the normal kexec interface that allows you to boot another kernel. With a few restrictions that should keep it as reliable as possible when the kernel has not shut itself down cleanly.
The hardest case is to do a useful system core dump. As that requires looking at what has gone before. For the rest if you can do it with a kernel and a initramfs you are in good shape.
There seems to be a significant amount of interest in the full system core dump case so that is what the work is concentrating on.
Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 1 line
Itsuro ODA <[email protected]>
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
And the feedback begins :)
Quoted 7 lines · Itsuro Oda
Hi,
I don't like calling
crash_kexec()directly in (ex.)panic(). It should becall_dump_hook()(or something like this).I think the necessary modifications of the kernel is only:
- insert the hooks that calls a dump function when crash occur
crash_kexec()
Quoted 2 lines
- binding interface that binds a dump function to the hook (like
register_dump_hook())
sys_kexec_load(...);
Quoted 1 line
- supply the information of valid physical address regions
/proc/iomem or possibly /proc/cpumem. At least until someone actually implements hot plug memory support.
Quoted 5 lines
(- maybe some existent functions and variables need to be exported ?)
I think this makes any sort of dump functions can be implemented as a kernel module. I don't think it is best way that the "kexec based crashdump" is built in the kernel.
For people developing code outside of the kernel I can see where this is a problem. Given the insane auditing requirements necessary to get a reliable code path I don't see how not putting the implementation in the kernel is sane. Anything that needs to be touched at that point is core kernel functionality GPL_ONLY if it is exported at all. Touching anything from a module at that point is not sane.
Basically the code path setup with crash_kexec is little more than a jump instruction. And it should be audited and reduced as much as possible. I don't see how you get simpler or what piece of functionality could possibly improve by having multiple implementations in kernel modules.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Hi,
Quoted 10 lines · On 02 Feb 2005 07:45:11 -0700 [email protected] (Eric W. Biederman) wrote:
And the feedback begins :)
Quoted 5 lines · Itsuro Oda
Hi,
I don't like calling
crash_kexec()directly in (ex.)panic(). It should becall_dump_hook()(or something like this).
Quoted 6 lines
Quoted 2 lines
I think the necessary modifications of the kernel is only:
- insert the hooks that calls a dump function when crash occur
crash_kexec()Quoted 2 lines
- binding interface that binds a dump function to the hook (like
register_dump_hook())
sys_kexec_load(...);
For example there are pepole who want to execute a built in kernel debugger when the system is crashed. or there are pepole who believe the diskdump is the best dump tool :-)
So I think a sort of hook is better than calling crash_kexec directly. (May I make a patch ?)
Thanks.
Signature · 1 line
Itsuro ODA <[email protected]>
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 20 lines · Itsuro Oda
Hi,
Quoted 15 lines · On 02 Feb 2005 07:45:11 -0700 [email protected] (Eric W. Biederman) wrote:
And the feedback begins :)
Quoted 7 lines · Itsuro Oda
Hi,
I don't like calling
crash_kexec()directly in (ex.)panic(). It should becall_dump_hook()(or something like this).I think the necessary modifications of the kernel is only:
- insert the hooks that calls a dump function when crash occur
crash_kexec()Quoted 2 lines
- binding interface that binds a dump function to the hook (like
register_dump_hook())
Quoted 8 lines
Quoted 1 line
sys_kexec_load(...);For example there are pepole who want to execute a built in kernel debugger when the system is crashed. or there are pepole who believe the diskdump is the best dump tool :-)
So I think a sort of hook is better than calling
crash_kexecdirectly. (May I make a patch ?)
The prevalent feeling I have heard from kernel developers and and my personal feeling as well is that after a kernel has called panic you can't trust it. Which means anything running in the kernel itself is suspect.
The crash_kexec() hooks enables everything that does not get linked into
the kernel. So I don't feel a hook in the panic path is necessary
nor do I feel that it is wise, especially with no in-kernel users.
Plus the worst part about a hook in the panic path is that it is inherently racy. Keeping the crash_kexec() code from blocking or being racy has been a challenge. And I still think that entire code path needs a review and some more code tweaks to remove races.
If someone else wants a hook in the panic path they can add their own hook, and make their own case for why it is needed.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
In reply to Vivek GoyalHi Vivek and Eric,
IMHO, why don't we swap not only the contents of the top 640K but also kernel working memory for kdump kernel?
I guess this approach has some good points.
1.Preallocating reserved area is not mandatory at boot time.
And the reserved area can be distributed in small pieces like original kexec does.
2.Special linking is not required for kdump kernel.
Each kdump kernel can be linked in the same way, where the original kernel exists.
Am I missing something?
physical memory
+-------+ | 640K ------------+ |.......| | | | copy +-------+ | | | | |original<-----+ | |kernel | | | | | | | |.......| | | | | | | | | | | | | swap | | | | | +-------+ | | |reserved<----------+ |area | | | | | |kdump |<-----+ |kernel | +-------+ | | | | | | +-------+
Quoted 79 lines
Hi Eric,
It looks like we are looking at things a little differently. I see a portion of the picture in your mind, but obviously not entirely.
Perhaps, we need to step back and iron out in specific terms what the interface between the two kernels should be in the crash dump case, and the distribution of responsibility between kernel, user space and the user.
[BTW, the patch was intended as a step in development up for comment early enough to be able to get agreement on the interface and think issues through to more completeness before going too far. Sorry, if that wasn't apparent.]
When you say "evil intermingling", I'm guessing you mean the "crashbackup=" boot parameter ? If so, then yes, I agree it'd be nice to find a way around it that doesn't push hardcoding elsewhere.
Let me explain the interface/approach I was looking at.
1.First kernel reserves some area of memory for crash/capture kernel as specified by crashkernel=X@Y boot time parameter.
2.First kernel marks the top 640K of this area as backup area. (If architecture needs it.) This is sort of a hardcoding and probably this space reservation can be managed from user space as well as mentioned by you in this mail below.
3. Location of backup region is exported through
/proc/iomemwhich can be read by user space utility to pass this information to purgatory code to determine where to copy the first 640K.Note that we do not make any additional reservation for the backup region. We carve this out from the top of the already reserved region and export it through
/proc/iomemso that the user space code and the capture kernel code need not make any assumptions about where this region is located.4. Once the capture kernel boots, it needs to know the location of backup region for two purposes.
a. It should not overwrite the backup region.
b. There needs to be a way for the capture tool to access the original
contents of the backed up regionBoot time parameter crashbackup=A@B has been provided to pass this information to capture kernel. This parameter is valid only for capture kernel and becomes effective only if
CONFIG_CRASH_DUMPis enabled.Quoted 2 lines
What is wrong with user space doing all of the extra space reservation?
Just for clarity, are you suggesting kexec-tools creating an additional segment for the backup region and pass the information to kernel.
There is no problem in doing reservation from user space except one. How does the user and in-turn capture kernel come to know the location of backup region, assuming that the user is going to provide the exactmap for capture kernel to boot into.
Just a thought, is it a good idea for kexec-tools to be creating and passing memmap parameters doing appropriate adjustment for backup region.
I had another question. How is the starting location of elf headers communicated to capture tool? Is parameter segment a good idea? or some hardcoding?
Another approach can be that backup area information is encoded in elf headers and capture kernel is booted with modified memmap (User gets backup region information from
/proc/iomem) and capture tool can extract backup area information from elf headers as stored by first kernel.
Quoted 5 lines
Could you please elaborate a little more on what aspect of your view differs from the above.
Thanks
Vivek
Thaks,
Hirokazu Takahashi.
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Hi,
Quoted 4 lines · Hirokazu Takahashi
Hi Vivek and Eric,
IMHO, why don't we swap not only the contents of the top 640K but also kernel working memory for kdump kernel?
Initial patches of kdump had adopted the same approach but given the fact devices are not stopped during transition to new kernel after a panic, it carried inherent risk of some DMA going on and corrupting the new kernel/data structures. Hence the idea of running the kernel from a reserved location came up. This should be DMA safe as long as DMA is not misdirected.
Thanks
Vivek
Quoted 40 lines
I guess this approach has some good points.
1.Preallocating reserved area is not mandatory at boot time.
And the reserved area can be distributed in small pieces like original kexec does.2.Special linking is not required for kdump kernel.
Each kdump kernel can be linked in the same way, where the original kernel exists.Am I missing something?
physical memory+-------+ | 640K ------------+ |.......| | | | copy +-------+ | | | | |original<-----+ | |kernel | | | | | | | |.......| | | | | | | | | | | | | swap | | | | | +-------+ | | |reserved<----------+ |area | | | | | |kdump |<-----+ |kernel | +-------+ | | | | | | +-------+
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Hi Vivek,
Quoted 6 lines
Quoted 4 lines
Hi Vivek and Eric,
IMHO, why don't we swap not only the contents of the top 640K but also kernel working memory for kdump kernel?
Quoted 6 lines
Initial patches of kdump had adopted the same approach but given the fact devices are not stopped during transition to new kernel after a panic, it carried inherent risk of some DMA going on and corrupting the new kernel/data structures. Hence the idea of running the kernel from a reserved location came up. This should be DMA safe as long as DMA is not misdirected.
I see, that makes sense.
But I'm not sure yet that it's safe to access the top of 640MB. I wonder how kmalloc(GFP_DMA) works in a kdump kernel.
Thanks,
Hirokazu Takahashi.
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 15 lines · Hirokazu Takahashi
Hi Vivek,
Quoted 6 lines
Quoted 4 lines
Hi Vivek and Eric,
IMHO, why don't we swap not only the contents of the top 640K but also kernel working memory for kdump kernel?
Quoted 6 lines
Initial patches of kdump had adopted the same approach but given the fact devices are not stopped during transition to new kernel after a panic, it carried inherent risk of some DMA going on and corrupting the new kernel/data structures. Hence the idea of running the kernel from a reserved location came up. This should be DMA safe as long as DMA is not misdirected.
Quoted 2 lines
I see, that makes sense.
But I'm not sure yet that it's safe to access the top of 640MB.
640K?
Quoted 1 line
I wonder how
kmalloc(GFP_DMA)works in a kdump kernel.
All that happens there is a one line change to vmlinux.lds.S that causes the kernel to live at a different physical and virtual address. So everything works as normal.
I do agree that it is risky to use the first 640K for normal work. But on the list of things to fix it is a minor war, and even if we back up that region of memory we don't need to use it.
There are still remain a lot of code reviews to ensure the code is generally safe.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
In reply to Hirokazu TakahashiQuoted 11 lines · Hirokazu Takahashi
Hi Vivek and Eric,
IMHO, why don't we swap not only the contents of the top 640K but also kernel working memory for kdump kernel?
I guess this approach has some good points.
1.Preallocating reserved area is not mandatory at boot time.
And the reserved area can be distributed in small pieces like original kexec does.
Quoted 5 lines
2.Special linking is not required for kdump kernel.
Each kdump kernel can be linked in the same way, where the original kernel exists.Am I missing something?
Preallocating the reserved area is largely to keep it from being the target of DMA accesses. Since we are not able to shutdown any of the drivers in the primary kernel running in a normal swath of memory sounds like a good way to get yourself stomped at the worst possible time.
In addition we get to avoid running a lot of code in the panic path if we are jumping to a contiguous region of memory with everything already setup.
To some extent this is a contest who has the better imagination for things that can go wrong. Real life on dying hardware and kernels, or the programmers writing the diagnostic code.
But if it is a gamble you are willing to take it is quite feasible to use the reserved region for what you are proposing and you could run a standard kernel.
The other reason for running out of the reserved region is that it actually requires less memory reserved. Every byte you backup needs to have a reserved area of memory to hold it. And if you are also going to fill that with meaningful content you need another byte to hold the data. So using a stock kernel probably requires 2/3 more memory.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Hi Eric,
Quoted 15 lines
Quoted 15 lines
Hi Vivek and Eric,
IMHO, why don't we swap not only the contents of the top 640K but also kernel working memory for kdump kernel?
I guess this approach has some good points.
1.Preallocating reserved area is not mandatory at boot time.
And the reserved area can be distributed in small pieces like original kexec does.2.Special linking is not required for kdump kernel.
Each kdump kernel can be linked in the same way, where the original kernel exists.
Quoted 7 lines
Quoted 1 line
Am I missing something?
Preallocating the reserved area is largely to keep it from being the target of DMA accesses. Since we are not able to shutdown any of the drivers in the primary kernel running in a normal swath of memory sounds like a good way to get yourself stomped at the worst possible time.
So what do you think my another idea?
I think we can always make a kdump kernel mapped to the same virtual address. So we will be free from caring about the physical address where the kdump kernel is loaded.
I believe the memsection functionality which LHMS project is working on would help this.
+
|
|
(user space)
|
|
physical | virtual
memory | space
+ ------------ +
| |
| |
| |
+ ------------.+
original | . | map kdump kernel here
kernel | . |
| . |
| . .+
+ . . |
| . . |
+ . |
kdump | . |
kernel | . |
| . |
+ |
| |
| |
| |
Thanks,
Hirokazu Takahashi.
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 19 lines · Hirokazu Takahashi
Hi Eric,
Quoted 17 lines
Quoted 16 lines
Hi Vivek and Eric,
IMHO, why don't we swap not only the contents of the top 640K but also kernel working memory for kdump kernel?
I guess this approach has some good points.
1.Preallocating reserved area is not mandatory at boot time.
And the reserved area can be distributed in small pieces like original kexec does.2.Special linking is not required for kdump kernel.
Each kdump kernel can be linked in the same way, where the original kernel exists.Am I missing something?
Quoted 7 lines
Quoted 5 lines
Preallocating the reserved area is largely to keep it from being the target of DMA accesses. Since we are not able to shutdown any of the drivers in the primary kernel running in a normal swath of memory sounds like a good way to get yourself stomped at the worst possible time.
So what do you think my another idea?
I have proposed it. I think ia64 already does that. It has been pointed that the PowerPC kernel occasionally runs with the mmu turned off. So it is not a technique the is 100% portable.
Quoted 6 lines
I think we can always make a kdump kernel mapped to the same virtual address. So we will be free from caring about the physical address where the kdump kernel is loaded.
I believe the memsection functionality which LHMS project is working on would help this.
You don't need anything fancy except to build the page tables during bootup. However there are a few potential gotchas with respect to using large pages, that can give 4MiB or greater alignment restrictions on the kernel. Code wise the gotcha is moving the kernel's .text section into what is essentially the vmalloc portion of the address space. For x86_64 the kernels virtual address is already decoupled from the physical addresses, so it is probably easier.
Most of this just results in easier management between the pieces. Which is a good thing. However at the moment I don't think it simplifies any of the core problems. I still need to reserve a large hunk of physical address space early on before any DMA transactions are setup to hold the new kernel.
So while I am happy to see patches that improve this I don't actually care right now.
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Hi,
Quoted 25 lines
Quoted 25 lines
Hi Eric,
Quoted 22 lines
Quoted 11 lines
Hi Vivek and Eric,
IMHO, why don't we swap not only the contents of the top 640K but also kernel working memory for kdump kernel?
I guess this approach has some good points.
1.Preallocating reserved area is not mandatory at boot time.
And the reserved area can be distributed in small pieces like original kexec does.Quoted 5 lines
2.Special linking is not required for kdump kernel.
Each kdump kernel can be linked in the same way, where the original kernel exists.Am I missing something?
Preallocating the reserved area is largely to keep it from being the target of DMA accesses. Since we are not able to shutdown any of the drivers in the primary kernel running in a normal swath of memory sounds like a good way to get yourself stomped at the worst possible time.
Quoted 6 lines
Quoted 1 line
So what do you think my another idea?
I have proposed it. I think ia64 already does that. It has been pointed that the PowerPC kernel occasionally runs with the mmu turned off. So it is not a technique the is 100% portable.
I see you have.
And MIPS CPUs doesn't allow kernel pages to be remapped either.
Quoted 7 lines
Quoted 6 lines
I think we can always make a kdump kernel mapped to the same virtual address. So we will be free from caring about the physical address where the kdump kernel is loaded.
I believe the memsection functionality which LHMS project is working on would help this.
Quoted 8 lines
You don't need anything fancy except to build the page tables during bootup. However there are a few potential gotchas with respect to using large pages, that can give 4MiB or greater alignment restrictions on the kernel. Code wise the gotcha is moving the kernel's .text section into what is essentially the vmalloc portion of the address space. For
x86_64the kernels virtual address is already decoupled from the physical addresses, so it is probably easier.
I know we can place the kernel in any address though there exist some exceptions.
I know mapping kernel pages to the same virtual address only helps to avoid caring about physical addresses or vmalloc'ed addresses when linking the kernel. I think it wouldn't be bad idea in many architectures. I prefer it rather than linking the kernel for each system.
Quoted 5 lines
Most of this just results in easier management between the pieces. Which is a good thing. However at the moment I don't think it simplifies any of the core problems. I still need to reserve a large hunk of physical address space early on before any DMA transactions are setup to hold the new kernel.
I agree that my idea is not essential at the moment.
Quoted 2 lines
So while I am happy to see patches that improve this I don't actually care right now.
ok.
Quoted 2 lines
Eric
Thanks,
Hirokazu Takahashi.
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 7 lines · Hirokazu Takahashi
Hi,
Quoted 2 lines
Quoted 2 lines
Hi Eric,
I see you have.
And MIPS CPUs doesn't allow kernel pages to be remapped either.
I guess I should add to be relocatable in the general case most likely requires running a PIC dynamic linker at kernel startup. If none of the rest of the kernel is built PIC and the relocation table is not too big we might be able to convince people to implement it generally.
At least that is one technique for generating a PIC kernel that I have not explored fully.
Quoted 9 lines
Quoted 8 lines
You don't need anything fancy except to build the page tables during bootup. However there are a few potential gotchas with respect to using large pages, that can give 4MiB or greater alignment restrictions on the kernel. Code wise the gotcha is moving the kernel's .text section into what is essentially the vmalloc portion of the address space. For
x86_64the kernels virtual address is already decoupled from the physical addresses, so it is probably easier.
Quoted 8 lines
I know we can place the kernel in any address though there exist some exceptions.
I know mapping kernel pages to the same virtual address only helps to avoid caring about physical addresses or vmalloc'ed addresses when linking the kernel. I think it wouldn't be bad idea in many architectures. I prefer it rather than linking the kernel for each system.
Agreed. Although I suspect most architectures will have a region that will work for most users.
Quoted 6 lines
Quoted 5 lines
Most of this just results in easier management between the pieces. Which is a good thing. However at the moment I don't think it simplifies any of the core problems. I still need to reserve a large hunk of physical address space early on before any DMA transactions are setup to hold the new kernel.
Quoted 6 lines
I agree that my idea is not essential at the moment.
Quoted 2 lines
So while I am happy to see patches that improve this I don't actually care right now.
ok.
The one part I do request is that if you build such a kernel that
you figure a way to get it's ELF header of type ET_DYN. So it
does not require a magic loader to load it.
I have recently patched both etherboot and /sbin/kexec to accept that kind of binary :)
Eric
Re: [Fastboot] [PATCH] Reserving backup region for kexec based crashdumps.
Quoted 7 lines · [email protected] (Eric W. Biederman) writes:
Quoted 6 lines · Hirokazu Takahashi
Quoted 5 lines
Most of this just results in easier management between the pieces. Which is a good thing. However at the moment I don't think it simplifies any of the core problems. I still need to reserve a large hunk of physical address space early on before any DMA transactions are setup to hold the new kernel.
Quoted 6 lines
Quoted 6 lines
I agree that my idea is not essential at the moment.
Quoted 2 lines
So while I am happy to see patches that improve this I don't actually care right now.
ok.
Thinking about this some more this does have a significant aspect on the design. For architectures that support this, on the primary kernel the command line option becomes:
crashkernel=size instead of crashkernel=size@location. Which means the kernel needs to call alloc_bootmem instead of reserve_bootmem. So it results in a primary kernel implementation difference.
In addition if we really can push all of the dump specific functionality into user space as it appears we can, this allows a generic kernel to be used for the crash dump process. It will probably still be a special hardened build where reliability is more important than performance. So that any micro hit we take in performance by modifying __pa() and __va() will be irrelevant.
I like it.
I have already demonstrated that there is a general technique that any architecture can use to build a kernel that runs at a non-default address. So for the architectures that cannot build a PIC kernel there is still a proven solution available, it simply will not be as nice to manage.
x86_64 should pretty straight forward. i386 will be a little more difficult but doable.
Patches are still welcome.
Eric