arm64: Allocate elfcorehdr & crashkernel mem before any reservation
Subsystems (3)
PATCH v2 1/1 arm64: Allocate elfcorehdr & crashkernel mem before any reservation
Address for both crashkernel memory and elfcorehdr can be assigned statically. For crashkernel memory it is via crashkernel=SIZE at ADDRESS and elfcorehdr_addr via by kexec-utils in dump kernel device tree.
So memory should be reserved for both the above areas before any other memory reservations are done. Otherwise overlaps may happen and memory allocations will fail leading to failure in booting the dump capture kernel.
---
v2:
corrected the email address
reworked based on the discussions:(with Will Deacon and Takahiro)
https://patchwork.kernel.org/patch/10104249/
arch/arm64/mm/init.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
arch/arm64/mm/init.c +8 -4
@@ -453,6 +453,14 @@ void __init arm64_memblock_init(void)* Register the kernel text, kernel data, initrd, and initial * pagetables with memblock. */ + +/* make these the first reservations to avoid chances of +* overlap +*/ +reserve_elfcorehdr (); + +reserve_crashkernel (); +memblock_reserve (__pa_symbol (_text), _end - _text);#ifdef CONFIG_BLK_DEV_INITRD if (initrd_start) {
@@ -472,10 +480,6 @@ void __init arm64_memblock_init(void)else arm64_dma_phys_limit =PHYS_MASK +1 ; -reserve_crashkernel (); - -reserve_elfcorehdr (); - high_memory =__va (memblock_end_of_DRAM () -1 ) +1 ;dma_contiguous_reserve (arm64_dma_phys_limit);
Signature · 1 line
2.7.4
PATCH v2 1/1 arm64: Allocate elfcorehdr & crashkernel mem before any reservation
Hi Poonam,
Quoted 3 lines · Poonam Aggrwal
Address for both crashkernel memory and elfcorehdr can be assigned statically. For crashkernel memory it is via crashkernel=SIZE at ADDRESS and
elfcorehdr_addrvia by kexec-utils in dump kernel device tree.
There are some crashkernel=SIZE at ADDRESS values that the user can supply that we must reject. The obvious one is if it overlaps with the kernel text. (this patch won't spot this). We need to read the hardware's reserved regions from the DT before we allocate the crashkernel region, for example if the bootloader reserved a chunk of memory for a frame-buffer, I shouldn't be able to use that as crashkernel memory.
(you shouldn't need to specify an address, doing so prevents the kernel from picking memory it can use)
Quoted 6 lines
So memory should be reserved for both the above areas before any other memory reservations are done. Otherwise overlaps may happen and memory allocations will fail leading to failure in booting the dump capture kernel.
Signed-off-by Guanhua <[email protected]>
The first signed-off-by should be the patch author.
If you want to credit your colleagues you can use 'CC', or they can give reviewed-by/acked-by to the patch.
Quoted 2 lines
Signed-off-by Poonam Aggrwal <[email protected]>Signed-off-by Abhimanyu Saini <[email protected]>
And the last signed-of-by should be from the person posting to the mailing list.
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 455
Quoted 8 lines
arch/arm64/mm/init.c +0
-0@@ -453,6 +453,14 @@ void __init arm64_memblock_init(void)* Register the kernel text, kernel data, initrd, and initial * pagetables with memblock. */
Please don't insert this code between 'memblock_reserve(__pa_symbol(_text), _end
- _text);' and the comment that describes it.
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 459
Quoted 4 lines
arch/arm64/mm/init.c +4
-0+ +/* make these the first reservations to avoid chances of +* overlap +*/
Nit: comment style
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 460
Quoted 1 line
arch/arm64/mm/init.c +1
-0+reserve_elfcorehdr ();
(Moving reserve_elfcorehdr() makes sense, but..)
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 462
Quoted 1 line
arch/arm64/mm/init.c +1
-0+reserve_crashkernel ();
reserve_crashkernel() does the allocation for the crashkernels reserved memory. I expect this to always fail in the kdump kernel because there isn't enough memory. (fdt_enforce_memory_region() at the top of this function calls memblock_cap_memory_range()).
Moving this allocation above the early_init_fdt_scan_reserved_mem() below means we may allocate memory for the crashdump that is in use by firmware/hardware and described as reserved in the DT.
Quoted 5 lines
arch/arm64/mm/init.c +0
-0memblock_reserve (__pa_symbol (_text), _end - _text);#ifdef CONFIG_BLK_DEV_INITRD if (initrd_start) {
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 485
Quoted 10 lines
arch/arm64/mm/init.c +0
-4@@ -472,10 +480,6 @@ void __init arm64_memblock_init(void)else arm64_dma_phys_limit =PHYS_MASK +1 ; -reserve_crashkernel (); - -reserve_elfcorehdr (); - high_memory =__va (memblock_end_of_DRAM () -1 ) +1 ;dma_contiguous_reserve (arm64_dma_phys_limit);
Thanks,
James
Thanks James for the feedback.
Please find replies inline.
Regards
Poonam
Quoted 40 lines
-----Original Message-----
From: James Morse [mailto:[email protected]]
Sent: Friday, January 05, 2018 4:37 PM
To: Poonam Aggrwal <[email protected]>; linux-arm- [email protected]
Cc: [email protected]; [email protected];G.h. Gao <[email protected]>; Abhimanyu Saini <[email protected]>
Subject: Re: [PATCH][v2] arm64: Allocate elfcorehdr & crashkernel mem before any reservationHi Poonam,
Quoted 3 lines · Poonam Aggrwal
Address for both crashkernel memory and elfcorehdr can be assigned statically. For crashkernel memory it is via crashkernel=SIZE@ADDRESS and
elfcorehdr_addrvia by kexec-utils in dump kernel device tree.There are some crashkernel=SIZE@ADDRESS values that the user can supply that we must reject. The obvious one is if it overlaps with the kernel text. (this patch won't spot this). We need to read the hardware's reserved regions from the DT before we allocate the crashkernel region, for example if the bootloader reserved a chunk of memory for a frame-buffer, I shouldn't be able to use that as crashkernel memory.
(you shouldn't need to specify an address, doing so prevents the kernel from picking memory it can use)
Quoted 6 lines
So memory should be reserved for both the above areas before any other memory reservations are done. Otherwise overlaps may happen and memory allocations will fail leading to failure in booting the dump capture kernel.
Signed-off-by Guanhua <[email protected]>The first signed-off-by should be the patch author.
If you want to credit your colleagues you can use 'CC', or they can give reviewed-by/acked-by to the patch.
Quoted 5 lines
Quoted 2 lines
Signed-off-by Poonam Aggrwal <[email protected]>Signed-off-by Abhimanyu Saini <[email protected]>And the last signed-of-by should be from the person posting to the mailing list.
Sure will correct this.
Quoted 13 lines
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 455
Quoted 9 lines
arch/arm64/mm/init.c index renamed from arch/arm64/mm/init.c +0
-000e7b90..24ce15d 100644
arch/arm64/mm/init.c +0
-0@@ -453,6 +453,14 @@ void __init arm64_memblock_init(void)* Register the kernel text, kernel data, initrd, and initial * pagetables with memblock. */ Please don't insert this code between 'memblock_reserve(__pa_symbol(_text),
_end
-_text);' and the comment that describes it.
Thanks for catching this, will fix it.
Quoted 8 lines
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 459
Quoted 6 lines
arch/arm64/mm/init.c +4
-0+ +/* make these the first reservations to avoid chances of +* overlap +*/ Nit: comment style
Right, will fix it.
Quoted 14 lines
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 460
Quoted 3 lines
arch/arm64/mm/init.c +1
-0+reserve_elfcorehdr ();(Moving
reserve_elfcorehdr()makes sense, but..)arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 462
Quoted 1 line
arch/arm64/mm/init.c +1
-0+reserve_crashkernel ();
reserve_crashkernel()does the allocation for the crashkernels reserved memory. I expect this to always fail in the kdump kernel because there isn't enough memory. (fdt_enforce_memory_region()at the top of this function callsmemblock_cap_memory_range()).
Quoted 3 lines
Moving this allocation above the
early_init_fdt_scan_reserved_mem()below means we may allocate memory for the crashdump that is in use byfirmware/hardwareand described as reserved in the DT.
Yeah, this is a good point. So ideally the address of the crash kernel should be diligently provided by the user based on the system.
Quoted 15 lines
Quoted 15 lines
arch/arm64/mm/init.c +0
-0memblock_reserve(__pa_symbol(_text), _end - _text); #ifdef
CONFIG_BLK_DEV_INITRDarch/arm64/mm/init.c +0
-4if (initrd_start) {@@ -472,10 +480,6 @@ void __init arm64_memblock_init(void)else arm64_dma_phys_limit =PHYS_MASK +1 ; -reserve_crashkernel (); - -reserve_elfcorehdr (); - high_memory =__va (memblock_end_of_DRAM () -1 ) +1 ;
Quoted 7 lines
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 485
Quoted 2 lines
arch/arm64/mm/init.c +0
-0dma_contiguous_reserve (arm64_dma_phys_limit);Thanks,
James
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hi Poonam,
Quoted 40 lines · Poonam Aggrwal
Quoted 39 lines · James Morse
Quoted 3 lines · Poonam Aggrwal
Address for both crashkernel memory and elfcorehdr can be assigned statically. For crashkernel memory it is via crashkernel=SIZE@ADDRESS and
elfcorehdr_addrvia by kexec-utils in dump kernel device tree.There are some crashkernel=SIZE@ADDRESS values that the user can supply that we must reject. The obvious one is if it overlaps with the kernel text. (this patch won't spot this). We need to read the hardware's reserved regions from the DT before we allocate the crashkernel region, for example if the bootloader reserved a chunk of memory for a frame-buffer, I shouldn't be able to use that as crashkernel memory.
(you shouldn't need to specify an address, doing so prevents the kernel from picking memory it can use)
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 454
Quoted 12 lines
So memory should be reserved for both the above areas before any other memory reservations are done. Otherwise overlaps may happen and memory allocations will fail leading to failure in booting the dump capture kernel.
arch/arm64/mm/init.c index renamed from arch/arm64/mm/init.c +0
-000e7b90..24ce15d 100644
arch/arm64/mm/init.c +1
-0@@ -453,6 +453,14 @@ void __init arm64_memblock_init(void) +reserve_elfcorehdr ();(Moving
reserve_elfcorehdr()makes sense, but..)arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 462
Quoted 1 line
arch/arm64/mm/init.c +1
-0+reserve_crashkernel ();
reserve_crashkernel()does the allocation for the crashkernels reserved memory. I expect this to always fail in the kdump kernel because there isn't enough memory. (fdt_enforce_memory_region()at the top of this function callsmemblock_cap_memory_range()).
Quoted 6 lines
Quoted 3 lines
Moving this allocation above the
early_init_fdt_scan_reserved_mem()below means we may allocate memory for the crashdump that is in use byfirmware/hardwareand described as reserved in the DT.Yeah, this is a good point. So ideally the address of the crash kernel should be diligently provided by the user based on the system.
Even better: the region to store the crash kernel in should be chosen by the kernel. When using kdump I boot with 'crashkernel=1G', the kernel chooses where to place the reserved region. Even if I specified a reasonable physical address, the efistub may relocate the kernel over the top during boot as part of its KASLR work.
(Why does anyone ever need to specify an offset here?)
Thanks,
James
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hi James
Regards
Poonam
Quoted 60 lines
-----Original Message-----
From: James Morse [mailto:[email protected]]
Sent: Friday, January 12, 2018 11:29 PM
To: Poonam Aggrwal <[email protected]>
Cc: [email protected]; [email protected]; [email protected];G.h. Gao <[email protected]>; Abhimanyu Saini <[email protected]>; [email protected]
Subject: Re: [PATCH][v2] arm64: Allocate elfcorehdr & crashkernel mem before any reservationHi Poonam,
Quoted 5 lines · Poonam Aggrwal
Quoted 4 lines · James Morse
Quoted 3 lines · Poonam Aggrwal
Address for both crashkernel memory and elfcorehdr can be assigned statically. For crashkernel memory it is via crashkernel=SIZE@ADDRESS and
elfcorehdr_addrvia by kexec-utils in dumpkernel device tree.
Quoted 7 lines
Quoted 7 lines
There are some crashkernel=SIZE@ADDRESS values that the user can supply that we must reject. The obvious one is if it overlaps with the kernel text. (this patch won't spot this). We need to read the hardware's reserved regions from the DT before we allocate the crashkernel region, for example if the bootloader reserved a chunk of memory for a frame-buffer, I shouldn't be able to use that as crashkernel
memory.
Quoted 16 lines
Quoted 15 lines
(you shouldn't need to specify an address, doing so prevents the kernel from picking memory it can use)
Quoted 10 lines
So memory should be reserved for both the above areas before any other memory reservations are done. Otherwise overlaps may happen and memory allocations will fail leading to failure in booting the dump capture kernel.
arch/arm64/mm/init.c index renamed from arch/arm64/mm/init.c +0
-000e7b90..24ce15d 100644
arch/arm64/mm/init.c +0
-0@@ -453,6 +453,14 @@ void __init arm64_memblock_init(void)Quoted 8 lines
Quoted 8 lines
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 460
Quoted 1 line
arch/arm64/mm/init.c +1
-0+reserve_elfcorehdr ();(Moving
reserve_elfcorehdr()makes sense, but..)arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 462
Quoted 1 line
arch/arm64/mm/init.c +1
-0+reserve_crashkernel ();
reserve_crashkernel()does the allocation for the crashkernels reservedmemory.
Quoted 7 lines
Quoted 7 lines
I expect this to always fail in the kdump kernel because there isn't enough memory. (
fdt_enforce_memory_region()at the top of this function callsmemblock_cap_memory_range()).Moving this allocation above the
early_init_fdt_scan_reserved_mem()below means we may allocate memory for the crashdump that is in use byfirmware/hardwareand described as reserved in the DT.
Quoted 7 lines
Quoted 2 lines
Yeah, this is a good point. So ideally the address of the crash kernel should be diligently provided by the user based on the system.
Even better: the region to store the crash kernel in should be chosen by the kernel.
When using kdump I boot with 'crashkernel=1G', the kernel chooses where to place the reserved region.
Right. This is a commonly used.
Even if I specified a reasonable physical address, the
Quoted 2 lines
efistub may relocate the kernel over the top during boot as part of its KASLR work.
Agree
Quoted 2 lines
(Why does anyone ever need to specify an offset here?)
offset is normally an optional argument. Request Takahiro to provide his inputs. Does this imply any updates in kexec design/implementation/documentation?
Quoted 5 lines
Thanks,
James
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hi Poonam, James
Quoted 72 lines · Poonam Aggrwal
Hi James
Regards
PoonamQuoted 60 lines
-----Original Message-----
From: James Morse [mailto:[email protected]]
Sent: Friday, January 12, 2018 11:29 PM
To: Poonam Aggrwal <[email protected]>
Cc: [email protected]; [email protected]; [email protected];G.h. Gao <[email protected]>; Abhimanyu Saini <[email protected]>; [email protected]
Subject: Re: [PATCH][v2] arm64: Allocate elfcorehdr & crashkernel mem before any reservationHi Poonam,
Quoted 5 lines · Poonam Aggrwal
Quoted 4 lines · James Morse
Quoted 3 lines · Poonam Aggrwal
Address for both crashkernel memory and elfcorehdr can be assigned statically. For crashkernel memory it is via crashkernel=SIZE@ADDRESS and
elfcorehdr_addrvia by kexec-utils in dumpkernel device tree.
Quoted 7 lines
Quoted 7 lines
There are some crashkernel=SIZE@ADDRESS values that the user can supply that we must reject. The obvious one is if it overlaps with the kernel text. (this patch won't spot this). We need to read the hardware's reserved regions from the DT before we allocate the crashkernel region, for example if the bootloader reserved a chunk of memory for a frame-buffer, I shouldn't be able to use that as crashkernel
memory.
Quoted 16 lines
Quoted 15 lines
(you shouldn't need to specify an address, doing so prevents the kernel from picking memory it can use)
Quoted 10 lines
So memory should be reserved for both the above areas before any other memory reservations are done. Otherwise overlaps may happen and memory allocations will fail leading to failure in booting the dump capture kernel.
arch/arm64/mm/init.c index renamed from arch/arm64/mm/init.c +0
-000e7b90..24ce15d 100644
arch/arm64/mm/init.c +0
-0@@ -453,6 +453,14 @@ void __init arm64_memblock_init(void)Quoted 8 lines
Quoted 8 lines
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 460
Quoted 1 line
arch/arm64/mm/init.c +1
-0+reserve_elfcorehdr ();(Moving
reserve_elfcorehdr()makes sense, but..)arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 462
Quoted 1 line
arch/arm64/mm/init.c +1
-0+reserve_crashkernel ();
reserve_crashkernel()does the allocation for the crashkernels reservedmemory.
Quoted 7 lines
Quoted 7 lines
I expect this to always fail in the kdump kernel because there isn't enough memory. (
fdt_enforce_memory_region()at the top of this function callsmemblock_cap_memory_range()).Moving this allocation above the
early_init_fdt_scan_reserved_mem()below means we may allocate memory for the crashdump that is in use byfirmware/hardwareand described as reserved in the DT.Quoted 7 lines
Quoted 2 lines
Yeah, this is a good point. So ideally the address of the crash kernel should be diligently provided by the user based on the system.
Even better: the region to store the crash kernel in should be chosen by the kernel.
When using kdump I boot with 'crashkernel=1G', the kernel chooses where to place the reserved region.
Quoted 8 lines
Right. This is a commonly used.
Even if I specified a reasonable physical address, theQuoted 2 lines
efistub may relocate the kernel over the top during boot as part of its KASLR work.
Agree
Quoted 2 lines
(Why does anyone ever need to specify an offset here?)
offset is normally an optional argument. Request Takahiro to provide his inputs. Does this imply any updates in kexec design/implementation/documentation?
offset is a optional argument. For relocatable kernels (and kernels which support KASLR), specifying offset is normally not needed.
Please refer to the 'Extended crashkernel syntax' documentation
(<https://github.com/torvalds/linux/blob/master/Documentation/kdump/kdump.txt#L259>):
Extended crashkernel syntax
===========================
While the "crashkernel=size[@offset]" syntax is sufficient for most configurations, sometimes it's handy to have the reserved memory dependent on the value of System RAM -- that's mostly for distributors that pre-setup the kernel command line to avoid a unbootable system after some memory has been removed from the machine.
The syntax is:
crashkernel=<range1>:<size1>[,<range2>:<size2>,...][@offset]
range=start-[end]For example:
crashkernel=512M-2G:64M,2G-:128M
This would mean:
1) if the RAM is smaller than 512M, then don't reserve anything (this is the "rescue" case)
2) if the RAM size is between 512M and 2G (exclusive), then reserve 64M
3) if the RAM size is larger than 2G, then reserve 128M
As James mentioned for arm64, in case of relocatable/kaslr kernels, the efistub may relocate the kernel over the top during boot as part of its KASLR. So, the offset field may make more sense for non-relocatable/static kernels, but for newer kernels, its better to use the 'Extended crashkernel syntax' syntax which is also supported by newer distribution versions.
For e.g. see ubuntu trusty kdump-config man page -
<http://manpages.ubuntu.com/manpages/trusty/man8/kdump-config.8.html>:
kdump kernel relocation address does not match crashkernel= parameter:
For non-relocatable architectures, the kdump kernel must be
built with a predetermined start address. This message
indicates that the start address of the kdump kernel and the start address in the crashkernel= parameter do not match.
Regards,
Bhupesh
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Quoted 110 lines · Bhupesh SHARMA
Hi Poonam, James
Quoted 72 lines · Poonam Aggrwal
Hi James
Regards
PoonamQuoted 60 lines
-----Original Message-----
From: James Morse [mailto:james.morse at arm.com]
Sent: Friday, January 12, 2018 11:29 PM
To: Poonam Aggrwal <[email protected]>
Cc: linux-arm-kernel at lists.infradead.org; takahiro.akashi at linaro.org; will.deacon at arm.com;G.h. Gao <[email protected]>; Abhimanyu Saini <[email protected]>; kexec at lists.infradead.org
Subject: Re: [PATCH][v2] arm64: Allocate elfcorehdr & crashkernel mem before any reservationHi Poonam,
Quoted 5 lines · Poonam Aggrwal
Quoted 4 lines · James Morse
Quoted 3 lines · Poonam Aggrwal
Address for both crashkernel memory and elfcorehdr can be assigned statically. For crashkernel memory it is via
crashkernel=SIZE at ADDRESS andelfcorehdr_addrvia by kexec-utils in dumpkernel device tree.
Quoted 7 lines
Quoted 7 lines
There are some crashkernel=SIZE at ADDRESS values that the user can supply that we must reject. The obvious one is if it overlaps with the kernel text. (this patch won't spot this). We need to read the hardware's reserved regions from the DT before we allocate the crashkernel region, for example if the bootloader reserved a chunk of memory for a frame-buffer, I shouldn't be able to use that as crashkernel
memory.
Quoted 16 lines
Quoted 15 lines
(you shouldn't need to specify an address, doing so prevents the kernel from picking memory it can use)
Quoted 10 lines
So memory should be reserved for both the above areas before any other memory reservations are done. Otherwise overlaps may happen and memory allocations will fail leading to failure in booting the dump capture kernel.
arch/arm64/mm/init.c index renamed from arch/arm64/mm/init.c +0
-000e7b90..24ce15d 100644
arch/arm64/mm/init.c +0
-0@@ -453,6 +453,14 @@ void __init arm64_memblock_init(void)Quoted 8 lines
Quoted 8 lines
arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 460
Quoted 1 line
arch/arm64/mm/init.c +1
-0+reserve_elfcorehdr ();(Moving
reserve_elfcorehdr()makes sense, but..)arch/arm64/mm/init.c void __init arm64_memblock_init(void) · line 462
Quoted 1 line
arch/arm64/mm/init.c +1
-0+reserve_crashkernel ();
reserve_crashkernel()does the allocation for the crashkernels reservedmemory.
Quoted 7 lines
Quoted 7 lines
I expect this to always fail in the kdump kernel because there isn't enough memory. (
fdt_enforce_memory_region()at the top of this function callsmemblock_cap_memory_range()).Moving this allocation above the
early_init_fdt_scan_reserved_mem()below means we may allocate memory for the crashdump that is in use byfirmware/hardwareand described as reserved in the DT.Quoted 7 lines
Quoted 2 lines
Yeah, this is a good point. So ideally the address of the crash kernel should be diligently provided by the user based on the system.
Even better: the region to store the crash kernel in should be chosen by the kernel.
When using kdump I boot with 'crashkernel=1G', the kernel chooses where to place the reserved region.Quoted 8 lines
Right. This is a commonly used.
Even if I specified a reasonable physical address, theQuoted 2 lines
efistub may relocate the kernel over the top during boot as part of its KASLR work.
Agree
Quoted 2 lines
(Why does anyone ever need to specify an offset here?)
offset is normally an optional argument. Request Takahiro to provide his inputs. Does this imply any updates in kexec design/implementation/documentation?
offset is a optional argument. For relocatable kernels (and kernels which support KASLR), specifying offset is normally not needed.
Please refer to the 'Extended crashkernel syntax' documentation
(<https://github.com/torvalds/linux/blob/master/Documentation/kdump/kdump.txt#L259>):Extended crashkernel syntax
===========================While the "crashkernel=size[@offset]" syntax is sufficient for most configurations, sometimes it's handy to have the reserved memory dependent on the value of System RAM -- that's mostly for distributors that pre-setup the kernel command line to avoid a unbootable system after some memory has been removed from the machine.
The syntax is:
crashkernel=<range1>:<size1>[,<range2>:<size2>,...][@offset] range=start-[end]For example:
crashkernel=512M-2G:64M,2G-:128MThis would mean:
Quoted 8 lines
1) if the RAM is smaller than 512M, then don't reserve anything (this is the "rescue" case)
2) if the RAM size is between 512M and 2G (exclusive), then reserve 64M
3) if the RAM size is larger than 2G, then reserve 128MAs James mentioned for arm64, in case of relocatable/kaslr kernels, the efistub may relocate the kernel over the top during boot as part of its KASLR.
It would be sad if we couldn't specify kaslr and kdump at the same time. Since kaslr will skip any of memory regions whose attributes are not CONVENTIONAL_MEMORY for allocating a relocated kernel image, we will be able to have a dedicated range of memory reserved for kdump. In this case, using an "offset" in "crashkernel=" will be crucial.
(I don't know how we can notify uefi of the region though.)
Quoted 4 lines
So, the offset field may make more sense for non-relocatable/static kernels, but for newer kernels, its better to use the 'Extended crashkernel syntax' syntax which is also supported by newer distribution versions.
How better is it for the case?
I don't know exactly what you mean by "newer kernel/distribution", but kdump on arm64 supports this feature from the day one.
(It is basically independent from architectures.)
Thanks,
-Takahiro AKASHI
Quoted 11 lines
For e.g. see ubuntu trusty kdump-config man page -
<http://manpages.ubuntu.com/manpages/trusty/man8/kdump-config.8.html>:kdump kernel relocation address does not match crashkernel= parameter:
For non-relocatable architectures, the kdump kernel must be
built with a predetermined start address. This message
indicates that the start address of the kdump kernel and the start address in the crashkernel= parameter do not match.Regards,
Bhupesh
Hello,
Quoted 25 lines · On 16/01/18 07:07, [email protected] wrote:
Quoted 24 lines · Bhupesh SHARMA
Quoted 12 lines · Poonam Aggrwal
Quoted 11 lines · Poonam Aggrwal
Quoted 2 lines
Yeah, this is a good point. So ideally the address of the crash kernel should be diligently provided by the user based on the system.
Even better: the region to store the crash kernel in should be chosen by the kernel.
When using kdump I boot with 'crashkernel=1G', the kernel chooses where to place the reserved region.
Even if I specified a reasonable physical address, the
efistub may relocate the kernel over the top during boot as part of its KASLR work.Quoted 4 lines
Agree
Quoted 2 lines
(Why does anyone ever need to specify an offset here?)
offset is normally an optional argument. Request Takahiro to provide his inputs. Does this imply any updates in kexec design/implementation/documentation?
offset is a optional argument. For relocatable kernels (and kernels which support KASLR), specifying offset is normally not needed.
Please refer to the 'Extended crashkernel syntax' documentation
(<https://github.com/torvalds/linux/blob/master/Documentation/kdump/kdump.txt#L259>):
Quoted 8 lines
Quoted 8 lines
Extended crashkernel syntax
===========================While the "crashkernel=size[@offset]" syntax is sufficient for most configurations, sometimes it's handy to have the reserved memory dependent on the value of System RAM -- that's mostly for distributors that pre-setup the kernel command line to avoid a unbootable system after some memory has been removed from the machine.
This is to let the distribution provide a single value that works on machines with very little RAM, and machines that need gigabytes in order to boot.
Where does specifying the offset/absolute-address come in?
Quoted 5 lines
Quoted 3 lines
As James mentioned for arm64, in case of relocatable/kaslr kernels, the efistub may relocate the kernel over the top during boot as part of its KASLR.
It would be sad if we couldn't specify kaslr and kdump at the same time.
We can. The problem comes when you specify an absolute-address that should be reserved for user-space to eventually load the kdump kernel into. This is fragile for a number of reasons.
Quoted 6 lines
Since kaslr will skip any of memory regions whose attributes are not
CONVENTIONAL_MEMORYfor allocating a relocated kernel image, we will be able to have a dedicated range of memory reserved for kdump. In this case, using an "offset" in "crashkernel=" will be crucial.(I don't know how we can notify uefi of the region though.)
I'm confused. panic()->kdump:boot doesn't go via UEFI. It passes information in the DT:/chosen that may have been generated by the EFIStub, but it doesn't (and must not) change the EFI memory map.
We need to decide where the crashdump kernel region is when the first-kernel generates its page tables, as the protect/unprotect mechanism wants to be able to unmap them.
There is no reason for UEFI to know about the kdump region, BootServices are long-gone by the time its location has to be decided.
Quoted 4 lines
Quoted 4 lines
So, the offset field may make more sense for non-relocatable/static kernels, but for newer kernels, its better to use the 'Extended crashkernel syntax' syntax which is also supported by newer distribution versions.
(this extended crashkernel syntax looks like a tangent: its about specifying one-value string to specify a reservation-size on both small-memory and large-memory machines).
I don't think 'relocatable kernels' are relevant here. The KASLR series changed the kernel to no longer run from the linear map, so where in the linear map we allocate memory for the crash-kernel to boot from can't matter. These changes were merged before kexec/kdump support was added.
Even before this change, (I recall that:) the kernel would discard memory below its text. This isn't a problem as kexec-tools (typically) locates the kernel at the bottom of the region, and physical memory outside this range isn't accessible anyway because of the "linux,usable-memory" property. When we do want to access it, we remap it using the vmcore helpers.
I think this @offset must be for kernels that have to run from a physical/virtual address that is known at compile time. We don't have this problem on arm64, and specifying @offset makes kdump less reliable:
| cannot reserve crashkernel: region overlaps reserved memory
Quoted 5 lines
How better is it for the case?
I don't know exactly what you mean by "newer kernel/distribution", but kdump on arm64 supports this feature from the day one.
(It is basically independent from architectures.)
Support @offset? Yes, its core code allowing this.
Quoted 8 lines
Quoted 8 lines
For e.g. see ubuntu trusty kdump-config man page -
<http://manpages.ubuntu.com/manpages/trusty/man8/kdump-config.8.html>:kdump kernel relocation address does not match crashkernel= parameter:
For non-relocatable architectures, the kdump kernel must be
built with a predetermined start address. This message
indicates that the start address of the kdump kernel and the start address in the crashkernel= parameter do not match.
arm64 doesn't have this issue. The 'predetermined start address' is a relative value stored in the header. This is so the bootloader can place the kernel anywhere in memory and still have it boot.
I think we're in the weeds here: adding @offset to your 'crashkernel=' cmdline option tells the kernel you know this address will be free and not-reserved when it comes to reservation time. This isn't generally true.
Unless you wrote the DT and the bootloader, you can't know this.
Wasn't this patch moving the elfcorehdr reservation up to be before any dynamic reservations, to prevent them overlapping?
Thanks,
James
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hi,
Quoted 37 lines · James Morse
Hello,
Quoted 25 lines · On 16/01/18 07:07, [email protected] wrote:
Quoted 24 lines · Bhupesh SHARMA
Quoted 12 lines · Poonam Aggrwal
Quoted 11 lines · Poonam Aggrwal
Quoted 2 lines
Yeah, this is a good point. So ideally the address of the crash kernel should be diligently provided by the user based on the system.
Even better: the region to store the crash kernel in should be chosen by the kernel.
When using kdump I boot with 'crashkernel=1G', the kernel chooses where to place the reserved region.
Even if I specified a reasonable physical address, the
efistub may relocate the kernel over the top during boot as part of its KASLR work.Quoted 4 lines
Agree
Quoted 2 lines
(Why does anyone ever need to specify an offset here?)
offset is normally an optional argument. Request Takahiro to provide his inputs. Does this imply any updates in kexec design/implementation/documentation?
offset is a optional argument. For relocatable kernels (and kernels which support KASLR), specifying offset is normally not needed.
Please refer to the 'Extended crashkernel syntax' documentation
(<https://github.com/torvalds/linux/blob/master/Documentation/kdump/kdump.txt#L259>):Quoted 8 lines
Quoted 8 lines
Extended crashkernel syntax
===========================While the "crashkernel=size[@offset]" syntax is sufficient for most configurations, sometimes it's handy to have the reserved memory dependent on the value of System RAM -- that's mostly for distributors that pre-setup the kernel command line to avoid a unbootable system after some memory has been removed from the machine.
Quoted 4 lines
This is to let the distribution provide a single value that works on machines with very little RAM, and machines that need gigabytes in order to boot.
Where does specifying the offset/absolute-address come in?
It comes into picture where we know at compile time where we want to place the crashkernel at.
I remember using it on a few arm 32-bit machines but they had a static and well defined layout of the memory
map from the available system RAM, and required crashkernel to be placed at predefined offset to prevent 'undefined' behaviour.
I don't think this is relevant though now with the newer KASLR enabled kernels where we can RANDOMIZE the _text and also the module load area.
So using a static offset value in this case would be risky to say the least ...
Quoted 10 lines
Quoted 6 lines
Quoted 4 lines
As James mentioned for arm64, in case of relocatable/kaslr kernels, the efistub may relocate the kernel over the top during boot as part of its KASLR.
It would be sad if we couldn't specify kaslr and kdump at the same time.
We can. The problem comes when you specify an absolute-address that should be reserved for user-space to eventually load the kdump kernel into. This is fragile for a number of reasons.
... Fedora allows using kdump and KASLR both, so I am not sure that's a problem.
Quoted 22 lines
Quoted 6 lines
Since kaslr will skip any of memory regions whose attributes are not
CONVENTIONAL_MEMORYfor allocating a relocated kernel image, we will be able to have a dedicated range of memory reserved for kdump. In this case, using an "offset" in "crashkernel=" will be crucial.(I don't know how we can notify uefi of the region though.)
I'm confused. panic()->kdump:boot doesn't go via UEFI. It passes information in the DT:/chosen that may have been generated by the EFIStub, but it doesn't (and must not) change the EFI memory map.
We need to decide where the crashdump kernel region is when the first-kernel generates its page tables, as the protect/unprotect mechanism wants to be able to unmap them.
There is no reason for UEFI to know about the kdump region, BootServices are long-gone by the time its location has to be decided.Quoted 4 lines
Quoted 4 lines
So, the offset field may make more sense for non-relocatable/static kernels, but for newer kernels, its better to use the 'Extended crashkernel syntax' syntax which is also supported by newer distribution versions.
Quoted 8 lines
(this extended crashkernel syntax looks like a tangent: its about specifying one-value string to specify a reservation-size on both small-memory and large-memory machines).
I don't think 'relocatable kernels' are relevant here. The KASLR series changed the kernel to no longer run from the linear map, so where in the linear map we allocate memory for the crash-kernel to boot from can't matter. These changes were merged before kexec/kdump support was added.
Agreed.
Quoted 30 lines
Even before this change, (I recall that:) the kernel would discard memory below its text. This isn't a problem as kexec-tools (typically) locates the kernel at the bottom of the region, and physical memory outside this range isn't accessible anyway because of the "linux,usable-memory" property. When we do want to access it, we remap it using the vmcore helpers.
I think this @offset must be for kernels that have to run from a physical/virtual address that is known at compile time. We don't have this problem on arm64, and specifying @offset makes kdump less reliable:
| cannot reserve crashkernel: region overlaps reserved memoryQuoted 5 lines
How better is it for the case?
I don't know exactly what you mean by "newer kernel/distribution", but kdump on arm64 supports this feature from the day one.
(It is basically independent from architectures.)Support @offset? Yes, its core code allowing this.
Quoted 8 lines
Quoted 8 lines
For e.g. see ubuntu trusty kdump-config man page -
<http://manpages.ubuntu.com/manpages/trusty/man8/kdump-config.8.html>:kdump kernel relocation address does not match crashkernel= parameter:
For non-relocatable architectures, the kdump kernel must be
built with a predetermined start address. This message
indicates that the start address of the kdump kernel and the start address in the crashkernel= parameter do not match.
Quoted 8 lines
arm64 doesn't have this issue. The 'predetermined start address' is a relative value stored in the header. This is so the bootloader can place the kernel anywhere in memory and still have it boot.
I think we're in the weeds here: adding @offset to your 'crashkernel=' cmdline option tells the kernel you know this address will be free and not-reserved when it comes to reservation time. This isn't generally true.
Unless you wrote the DT and the bootloader, you can't know this.
Agreed. See my comments above. What I meant was that using @offset with newer kernels/distributions which support KASLR can lead to undefined behavior. What I tried to capture
from the ubuntu trusty man-page was that "kdump kernel must be built with a predetermined start address
for non-relocatable architectures".
Normally we use the same primary kernel as the kdump kernel, so if the primary kernels is Relocatable + KASLR enabled,
the kdump kernel would support the same as well. In that case assuming a fixed offset at which the kdump kernel can be loaded at or assuming a static
start address for the kdump kernel can lead to issues.
Regards,
Bhupesh
Quoted 7 lines
Wasn't this patch moving the elfcorehdr reservation up to be before any dynamic reservations, to prevent them overlapping?
Thanks,
James
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec