Handling spin table in kdump
Subsystems (2)
Hi
I came across the following issue while testing Kdump on an SMP board(Currituck) running a non-SMP kernel. Even though the kernel is UP, the device-tree has the nodes for second CPU and the related details.
The kexec tool adds the spin table area as a reserved section in the device tree for the dump capture kernel. This value is read from the 'cpu-release-addr'.
But now, if the spin table is not located within the 'Reserved region' for the crash kernel, the dump capture kernel would fail to boot, hitting a BUG in mm/bootmem.c as in [1].
This is because we try to reserve a region which is not available to the kernel.
So I am wondering how is this handled really on an SMP board (Fsl_bookE).
There are two possible solutions :
1) Do not reserve the regions for the spin-table, as we will use only the crashing CPU in the second kernel(maxcpus=1).
2) Add the spin-table region to the available memory regions passed to the kernel by kexec-tools.
I have tested (1) and it works fine for me. Yet to test (2).
Thoughts ?
Thanks
Suzuki
[1] Kernel Bug
----------------
Linux version 3.3.0-rc5 ([email protected]) (gcc version 4.3.4 [gcc-4_3-branch revision 152973] (GCC) ) #12 Tue May 22 18:03:01 IST2 Found legacy serial port 0 for /plb/opb/serial@10000000
mem=20010000000, taddr=20010000000, irq=0, clk=1851851, speed=115200
------------[ cut here ]------------
kernel BUG at mm/bootmem.c:351!
Vector: 700 (Program Check) at [c8a61e90]
pc: c847f91c: mark_bootmem+0xa0/0x14c
lr: c8472670: do_init_bootmem+0x1ac/0x218
sp: c8a61f40 msr: 21000
current = 0xc8a4a500
pid = 0, comm = swapper
kernel BUG at mm/bootmem.c:351!
enter ? for help
[c8a61f70] c8472670 do_init_bootmem+0x1ac/0x218
[c8a61f90] c847025c setup_arch+0x1bc/0x234
[c8a61fb0] c846b62c start_kernel+0x98/0x358
[c8a61ff0] c80000b4 _start+0xb4/0xf8
Quoted 25 lines · Suzuki K. Poulose
Hi
I came across the following issue while testing Kdump on an SMP board(Currituck) running a non-SMP kernel. Even though the kernel is UP, the device-tree has the nodes for second CPU and the related details.
The kexec tool adds the spin table area as a reserved section in the device tree for the dump capture kernel. This value is read from the 'cpu-release-addr'.
But now, if the spin table is not located within the 'Reserved region' for the crash kernel, the dump capture kernel would fail to boot, hitting a BUG in
mm/bootmem.cas in [1].This is because we try to reserve a region which is not available to the kernel.
So I am wondering how is this handled really on an SMP board (
Fsl_bookE).There are two possible solutions :
1) Do not reserve the regions for the spin-table, as we will use only the crashing CPU in the second kernel(maxcpus=1).
Quoted 7 lines
2) Add the spin-table region to the available memory regions passed to the kernel by kexec-tools.
I have tested (1) and it works fine for me. Yet to test (2).
Thoughts ?
I would think option 1 is acceptable. The crash kernel will never attempt to use the memory outside of the crash kernel region therefore it does not need to be reserved.
However, I thought we were able to boot SMP into the crash kernel on our parts... not sure how that effects things (although the maxcpus=1 here is a perfectly acceptable and safer thing to do anyways)
-M
Quoted 31 lines
Thanks
Suzuki[1] Kernel Bug
----------------Linux version 3.3.0-rc5 ([email protected]) (gcc version 4.3.4 [gcc-4_3-branch revision 152973] (GCC) ) #12 Tue May 22 18:03:01 IST2 Found legacy serial port 0 for /plb/opb/serial@10000000 mem=20010000000, taddr=20010000000, irq=0, clk=1851851, speed=115200
------------[ cut here ]------------ kernel BUG at mm/bootmem.c:351!Vector: 700 (Program Check) at [
c8a61e90]
pc:c847f91c:mark_bootmem+0xa0/0x14c
lr:c8472670:do_init_bootmem+0x1ac/0x218
sp:c8a61f40
msr: 21000
current =0xc8a4a500
pid = 0, comm = swapper
kernel BUG atmm/bootmem.c:351!
enter ? for help
[c8a61f70]c8472670do_init_bootmem+0x1ac/0x218
[c8a61f90]c847025csetup_arch+0x1bc/0x234
[c8a61fb0]c846b62cstart_kernel+0x98/0x358
[c8a61ff0]c80000b4_start+0xb4/0xf8
Quoted 4 lines
_______________________________________________
Linuxppc-dev mailing list
[email protected]
https://lists.ozlabs.org/listinfo/linuxppc-dev
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
PATCH 1/1 Do not reserve cpu spin-table for crash kernel
As of now, the kexec reserves the spin-table for all the CPUs on an SMP machine. The spin-table is pointed to by the cpu-release-addr property in the device-tree. Reserving the spin-table in the crash kernel will cause a BUG(), if the table lies outside the memory reserved for the crashkernel.
Disable reserving the spin-table regions and use maxcpus=1 to use only the crashing CPU to boot the crash kernel.
---
kexec/arch/ppc/crashdump-powerpc.c | 19 +++++++++++++------ kexec/arch/ppc/fixup_dtb.c | 4 ++++ 2 files changed, 17 insertions(+), 6 deletions(-)
kexec/arch/ppc/crashdump-powerpc.c +13 -6
@@ -262,10 +262,19 @@ static void ulltoa(unsigned long long i, char *str) } } +/* Append str to cmdline */ +static void add_cmdline (char *cmdline,char *str) +{ +int cmdlen =strlen (cmdline) +strlen (str); +if (cmdlen > (COMMAND_LINE_SIZE -1 )) +die ("Command line overflow\n" ); +strcat (cmdline, str); +} +static int add_cmdline_param (char *cmdline,unsigned long long addr,char *cmdstr,char *byte) { -int cmdlen, len, align =1024 ; +int align =1024 ;char str[COMMAND_LINE_SIZE ], *ptr;/* Passing in =xxxK / =xxxM format. Saves space required in cmdline.*/
@@ -284,11 +293,8 @@ static int add_cmdline_param(char *cmdline, unsigned long long addr, ptr +=strlen (str);ulltoa (addr, ptr);strcat (str, byte); - len =strlen (str); - cmdlen =strlen (cmdline) + len; -if (cmdlen > (COMMAND_LINE_SIZE -1 )) -die ("Command line overflow\n" ); -strcat (cmdline, str); + +add_cmdline (cmdline, str);dbgprintf ("Command line after adding elfcorehdr: %s\n" , cmdline);
@@ -365,6 +371,7 @@ int load_crashdump_segments(struct kexec_info *info, char *mod_cmdline,*/ add_cmdline_param (mod_cmdline, elfcorehdr," elfcorehdr=" ,"K" );add_cmdline_param (mod_cmdline, saved_max_mem," savemaxmem=" ,"M" ); +add_cmdline (mod_cmdline," maxcpus=1" );return 0 ; }
kexec/arch/ppc/fixup_dtb.c +4 -0
@@ -172,6 +172,9 @@ static void fixup_reserve_regions(struct kexec_info *info, char *blob_buf) } } +#if 0 +/* XXX: Do not reserve spin-table for CPUs. */ +/* Add reserve regions for cpu-release-addr */ nodeoffset =fdt_node_offset_by_prop_value (blob_buf,-1 ,"device_type" ,"cpu" ,4 );while (nodeoffset != -FDT_ERR_NOTFOUND ) {
@@ -201,6 +204,7 @@ static void fixup_reserve_regions(struct kexec_info *info, char *blob_buf) nodeoffset =fdt_node_offset_by_prop_value (blob_buf, nodeoffset, "device_type", "cpu", 4); } +#endif out:print_fdt_reserve_regions (blob_buf);
Re: [PATCH] [ppc] Do not reserve cpu spin-table for crash kernel
Quoted 10 lines · Suzuki K. Poulose
As of now, the kexec reserves the spin-table for all the CPUs on an SMP machine. The spin-table is pointed to by the cpu-release-addr property in the device-tree. Reserving the spin-table in the crash kernel will cause a
BUG(), if the table lies outside the memory reserved for the crashkernel.Disable reserving the spin-table regions and use maxcpus=1 to use only the crashing CPU to boot the crash kernel.
Signed-off-by Suzuki K. Poulose<[email protected]>
Simon,
Any response on this one ?
I have tested this on a Currituck board (476, SMP) with a UP kernel. Without this patch, the secondary kernel hits 'PANIC' in boot while trying to reserve a memory(the spin table), outside the memory range(crash reserve).
Thanks
Suzuki
Quoted 82 lines
---
kexec/arch/ppc/crashdump-powerpc.c | 19 +++++++++++++------ kexec/arch/ppc/fixup_dtb.c | 4 ++++ 2 files changed, 17 insertions(+), 6 deletions(-)kexec/arch/ppc/crashdump-powerpc.c +13
-6@@ -262,10 +262,19 @@ static void ulltoa(unsigned long long i, char *str) } } +/* Append str to cmdline */ +static void add_cmdline (char *cmdline,char *str) +{ +int cmdlen =strlen (cmdline) +strlen (str); +if (cmdlen> (COMMAND_LINE_SIZE -1 )) +die ("Command line overflow\n" ); +strcat (cmdline, str); +} +static int add_cmdline_param (char *cmdline,unsigned long long addr,char *cmdstr,char *byte) { -int cmdlen, len, align =1024 ; +int align =1024 ;char str[COMMAND_LINE_SIZE ], *ptr;/* Passing in =xxxK / =xxxM format. Saves space required in cmdline.*/ @@ -284,11 +293,8 @@ static int add_cmdline_param(char *cmdline, unsigned long long addr, ptr +=strlen (str);ulltoa (addr, ptr);strcat (str, byte); - len =strlen (str); - cmdlen =strlen (cmdline) + len; -if (cmdlen> (COMMAND_LINE_SIZE -1 )) -die ("Command line overflow\n" ); -strcat (cmdline, str); + +add_cmdline (cmdline, str);dbgprintf ("Command line after adding elfcorehdr: %s\n" , cmdline);@@ -365,6 +371,7 @@ int load_crashdump_segments(struct kexec_info *info, char *mod_cmdline,*/ add_cmdline_param (mod_cmdline, elfcorehdr," elfcorehdr=" ,"K" );add_cmdline_param (mod_cmdline, saved_max_mem," savemaxmem=" ,"M" ); +add_cmdline (mod_cmdline," maxcpus=1" );return 0 ; }kexec/arch/ppc/fixup_dtb.c +4
-0@@ -172,6 +172,9 @@ static void fixup_reserve_regions(struct kexec_info *info, char *blob_buf) } } +#if 0 +/* XXX: Do not reserve spin-table for CPUs. */ +/* Add reserve regions for cpu-release-addr */ nodeoffset =fdt_node_offset_by_prop_value (blob_buf,-1 ,"device_type" ,"cpu" ,4 );while (nodeoffset != -FDT_ERR_NOTFOUND ) {@@ -201,6 +204,7 @@ static void fixup_reserve_regions(struct kexec_info *info, char *blob_buf) nodeoffset =fdt_node_offset_by_prop_value (blob_buf, nodeoffset, "device_type", "cpu", 4); } +#endif out:print_fdt_reserve_regions (blob_buf);_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH] [ppc] Do not reserve cpu spin-table for crash kernel
Simon,
Please let me know your response.
Thanks
Suzuki
Quoted 108 lines · Suzuki K. Poulose
Quoted 10 lines · Suzuki K. Poulose
As of now, the kexec reserves the spin-table for all the CPUs on an SMP machine. The spin-table is pointed to by the cpu-release-addr property in the device-tree. Reserving the spin-table in the crash kernel will cause a
BUG(), if the table lies outside the memory reserved for the crashkernel.Disable reserving the spin-table regions and use maxcpus=1 to use only the crashing CPU to boot the crash kernel.
Signed-off-by Suzuki K. Poulose<[email protected]>Simon,
Any response on this one ?
I have tested this on a Currituck board (476, SMP) with a UP kernel. Without this patch, the secondary kernel hits 'PANIC' in boot while trying to reserve a memory(the spin table), outside the memory range(crash reserve).
Thanks
SuzukiQuoted 83 lines
---
kexec/arch/ppc/crashdump-powerpc.c| 19 +++++++++++++------kexec/arch/ppc/fixup_dtb.c| 4 ++++2 files changed, 17 insertions(+), 6 deletions(-)kexec/arch/ppc/crashdump-powerpc.c +13
-6@@ -262,10 +262,19 @@ static void ulltoa(unsigned long long i, char *str) } } +/* Append str to cmdline */ +static void add_cmdline (char *cmdline,char *str) +{ +int cmdlen =strlen (cmdline) +strlen (str); +if (cmdlen> (COMMAND_LINE_SIZE -1 )) +die ("Command line overflow\n" ); +strcat (cmdline, str); +} +static int add_cmdline_param (char *cmdline,unsigned long long addr,char *cmdstr,char *byte) { -int cmdlen, len, align =1024 ; +int align =1024 ;char str[COMMAND_LINE_SIZE ], *ptr;/* Passing in =xxxK / =xxxM format. Saves space required in cmdline.*/ @@ -284,11 +293,8 @@ static int add_cmdline_param(char *cmdline,unsigned long long addr, ptr += strlen(str);ulltoa (addr, ptr);strcat (str, byte); - len =strlen (str); - cmdlen =strlen (cmdline) + len; -if (cmdlen> (COMMAND_LINE_SIZE -1 )) -die ("Command line overflow\n" ); -strcat (cmdline, str); + +add_cmdline (cmdline, str);dbgprintf ("Command line after adding elfcorehdr: %s\n" , cmdline);@@ -365,6 +371,7 @@ int load_crashdump_segments(struct kexec_info*info, char *mod_cmdline, */ add_cmdline_param (mod_cmdline, elfcorehdr," elfcorehdr=" ,"K" );add_cmdline_param (mod_cmdline, saved_max_mem," savemaxmem=" ,"M" ); +add_cmdline (mod_cmdline," maxcpus=1" );return 0 ; }kexec/arch/ppc/fixup_dtb.c +3
-0@@ -172,6 +172,9 @@ static void fixup_reserve_regions(struct kexec_info *info, char *blob_buf) } } +#if 0 +/* XXX: Do not reserve spin-table for CPUs. */ +/* Add reserve regions for cpu-release-addr */ nodeoffset = fdt_node_offset_by_prop_value(blob_buf, -1,"
device_type", "cpu", 4);
while (nodeoffset != -FDT_ERR_NOTFOUND) {kexec/arch/ppc/fixup_dtb.c +1
-0@@ -201,6 +204,7 @@ static void fixup_reserve_regions(struct kexec_info *info, char *blob_buf) nodeoffset =fdt_node_offset_by_prop_value (blob_buf, nodeoffset, "device_type", "cpu", 4); } +#endif out:
print_fdt_reserve_regions(blob_buf);
Quoted 12 lines
Quoted 5 lines
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH] [ppc] Do not reserve cpu spin-table for crash kernel
Quoted 2 lines · Suzuki K. Poulose
Simon,
Ping.
Suzuki
Quoted 129 lines
Please let me know your response.
Thanks
SuzukiQuoted 109 lines · Suzuki K. Poulose
Quoted 10 lines · Suzuki K. Poulose
As of now, the kexec reserves the spin-table for all the CPUs on an SMP machine. The spin-table is pointed to by the cpu-release-addr property in the device-tree. Reserving the spin-table in the crash kernel will cause a
BUG(), if the table lies outside the memory reserved for the crashkernel.Disable reserving the spin-table regions and use maxcpus=1 to use only the crashing CPU to boot the crash kernel.
Signed-off-by Suzuki K. Poulose<[email protected]>Simon,
Any response on this one ?
I have tested this on a Currituck board (476, SMP) with a UP kernel. Without this patch, the secondary kernel hits 'PANIC' in boot while trying to reserve a memory(the spin table), outside the memory range(crash reserve).
Thanks
SuzukiQuoted 84 lines
---
kexec/arch/ppc/crashdump-powerpc.c| 19 +++++++++++++------kexec/arch/ppc/fixup_dtb.c| 4 ++++2 files changed, 17 insertions(+), 6 deletions(-)kexec/arch/ppc/crashdump-powerpc.c +10
-1@@ -262,10 +262,19 @@ static void ulltoa(unsigned long long i, char *str) } } +/* Append str to cmdline */ +static void add_cmdline (char *cmdline,char *str) +{ +int cmdlen =strlen (cmdline) +strlen (str); +if (cmdlen> (COMMAND_LINE_SIZE -1 )) +die ("Command line overflow\n" ); +strcat (cmdline, str); +} +static int add_cmdline_param (char *cmdline,unsigned long long addr,char *cmdstr,char *byte) { -int cmdlen, len, align =1024 ; +int align =1024 ;char str[COMMAND_LINE_SIZE ], *ptr;/* Passing in =xxxK / =xxxM format. Saves space required in cmdline.*/
kexec/arch/ppc/crashdump-powerpc.c +3
-5@@ -284,11 +293,8 @@ static int add_cmdline_param(char *cmdline,unsigned long long addr, ptr += strlen(str);ulltoa (addr, ptr);strcat (str, byte); - len =strlen (str); - cmdlen =strlen (cmdline) + len; -if (cmdlen> (COMMAND_LINE_SIZE -1 )) -die ("Command line overflow\n" ); -strcat (cmdline, str); + +add_cmdline (cmdline, str);dbgprintf ("Command line after adding elfcorehdr: %s\n" , cmdline);@@ -365,6 +371,7 @@ int load_crashdump_segments(struct kexec_info*info, char *mod_cmdline, */ add_cmdline_param (mod_cmdline, elfcorehdr," elfcorehdr=" ,"K" );add_cmdline_param (mod_cmdline, saved_max_mem," savemaxmem=" ,"M" ); +add_cmdline (mod_cmdline," maxcpus=1" );return 0 ; }kexec/arch/ppc/fixup_dtb.c +3
-0@@ -172,6 +172,9 @@ static void fixup_reserve_regions(struct kexec_info *info, char *blob_buf) } } +#if 0 +/* XXX: Do not reserve spin-table for CPUs. */ +/* Add reserve regions for cpu-release-addr */ nodeoffset = fdt_node_offset_by_prop_value(blob_buf, -1,"
device_type", "cpu", 4);
while (nodeoffset != -FDT_ERR_NOTFOUND) {kexec/arch/ppc/fixup_dtb.c +1
-0@@ -201,6 +204,7 @@ static void fixup_reserve_regions(struct kexec_info *info, char *blob_buf) nodeoffset =fdt_node_offset_by_prop_value (blob_buf, nodeoffset, "device_type", "cpu", 4); } +#endif out:
print_fdt_reserve_regions(blob_buf);Quoted 12 lines
Quoted 5 lines
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 5 lines
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH] [ppc] Do not reserve cpu spin-table for crash kernel
Quoted 5 lines · Suzuki K. Poulose
Quoted 2 lines · Suzuki K. Poulose
Simon,
Ping.
Sorry, I completely missed this!
I'll apply the change ASAP.
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Simon Horman
AppliedRe: [PATCH] [ppc] Do not reserve cpu spin-table for crash kernel
In reply to Suzuki K. PouloseQuoted 10 lines · Suzuki K. Poulose
As of now, the kexec reserves the spin-table for all the CPUs on an SMP machine. The spin-table is pointed to by the cpu-release-addr property in the device-tree. Reserving the spin-table in the crash kernel will cause a
BUG(), if the table lies outside the memory reserved for the crashkernel.Disable reserving the spin-table regions and use maxcpus=1 to use only the crashing CPU to boot the crash kernel.
Signed-off-by Suzuki K. Poulose <[email protected]>
Thanks, applied.
Sorry for not seeing this (much) earlier.
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec