Skip to contentSkip to conversation

X86/kdump: move crashkernel=X to reserve under 4G by default

PATCH In review Dave Young 4 participants 9 messages Apr 21, 2019 – Apr 22, 2019

Subsystems (2)
lore
Share & tools
lore

Dave Young

author
#0 lore raw

PATCH 1/2 X86/kdump: move crashkernel=X to reserve under 4G by default

The kdump crashkernel low reservation is limited to under 896M even for X86_64. This obscure and miserable limitation exists for old kexec-tools compatibility, but the reason is not documented anywhere.

Some more tests/investigations about the background:
a) Previously old kexec-tools can only load purgatory to memory under 2G, Eric remove that limitation in 2012 in kexec-tools:
Commit b4f9f8599679 ("kexec x86_64: Make purgatory relocatable anywhere in the 64bit address space.")

b) back in 2013 Yinghai removed all the limitations in new kexec-tools, bzImage64 can be loaded to anywhere.
Commit 82c3dd2280d2 ("kexec, x86_64: Load bzImage64 above 4G")

c) test results with old kexec-tools with old and latest kernels.
1. old kexec-tools can not build with modern toolchain anymore, I built it in a RHEL6 vm
2. 2.0.0 kexec-tools does not work with latest kernel even with memory under 896M and give an error:
"ELF core (kcore) parse failed", it needs below kexec-tools fix Commit ed15ba1b9977 ("build_mem_phdrs(): check if p_paddr is invalid")
3. even with patched kexec-tools which fixes 2), it still needs some other fixes to work correctly for kaslr enabled kernels.

So the situation is:
* old kexec-tools is already broken with latest kernels
* we can not keep this limitations forever just for compatibility of very old kexec-tools.
* If one must use old tools then he/she can choose crashkernel=X@Y
* people have reported bugs crashkernel=384M failed because kaslr makes the 0-896M space sparse,
* crashkernel can reserve in low or high area, it is natural to understand low as memory under 4G

Hence drop the 896M limitation, and change crashkernel low reservation to reserve under 4G by default.

---
 arch/x86/kernel/setup.c |   10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
arch/x86/kernel/setup.c +5 -5
@@ -71,6 +71,7 @@
 #include <linux/tboot.h>
 #include <linux/jiffies.h>
 #include <linux/mem_encrypt.h>
+#include <linux/sizes.h>
 
 #include <linux/usb/xhci-dbgp.h>
 #include <video/edid.h>
@@ -448,18 +449,17 @@ static void __init memblock_x86_reserve_
 #ifdef CONFIG_KEXEC_CORE
 
 /* 16M alignment for crash kernel regions */
-#define CRASH_ALIGN		(16 << 20)
+#define CRASH_ALIGN		SZ_16M
 
 /*
  * Keep the crash kernel below this limit.  On 32 bits earlier kernels
  * would limit the kernel to the low 512 MiB due to mapping restrictions.
- * On 64bit, old kexec-tools need to under 896MiB.
  */
 #ifdef CONFIG_X86_32
-# define CRASH_ADDR_LOW_MAX	(512 << 20)
-# define CRASH_ADDR_HIGH_MAX	(512 << 20)
+# define CRASH_ADDR_LOW_MAX	SZ_512M
+# define CRASH_ADDR_HIGH_MAX	SZ_512M
 #else
-# define CRASH_ADDR_LOW_MAX	(896UL << 20)
+# define CRASH_ADDR_LOW_MAX	SZ_4G
 # define CRASH_ADDR_HIGH_MAX	MAXMEM
 #endif
 
author
#1 lore raw

PATCH 2/2 X86/kdump: fall back to reserve high crashkernel memory

crashkernel=xM tries to reserve crashkernel memory under 4G, which is enough for usual cases. But this could fail sometimes, for example one tries to reserve a big chunk like 2G, it is possible to fail.

So let the crashkernel=xM just fall back to use high memory in case it fails to find a suitable low range. Do not set the ,high as default because it allocs extra low memory for DMA buffers and swiotlb, this is not always necessary for all machines. Typically like crashkernel=128M usually work with low reservation under 4G, so still keep <4G as default.

---
 Documentation/admin-guide/kernel-parameters.txt |    7 +++++--
 arch/x86/kernel/setup.c                         |   22 ++++++++++++++--------
 2 files changed, 19 insertions(+), 10 deletions(-)
arch/x86/kernel/setup.c +14 -8
@@ -541,21 +541,27 @@ static void __init reserve_crashkernel(v
 	}
 
 	/* 0 means: find the address automatically */
-	if (crash_base <= 0) {
+	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * as old kexec-tools loads bzImage below that, unless
-		 * "crashkernel=size[KMG],high" is specified.
+		 * as crashkernel=x,high allocs memory over 4G, also allocs
+		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * but the extra memory is not required for all machines.
+		 * So prefer low memory first, and fallback to high memory
+		 * unless "crashkernel=size[KMG],high" is specified.
 		 */
-		crash_base = memblock_find_in_range(CRASH_ALIGN,
-						    high ? CRASH_ADDR_HIGH_MAX
-							 : CRASH_ADDR_LOW_MAX,
-						    crash_size, CRASH_ALIGN);
+		if (!high)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_LOW_MAX,
+						crash_size, CRASH_ALIGN);
+		if (!crash_base)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_HIGH_MAX,
+						crash_size, CRASH_ALIGN);
 		if (!crash_base) {
 			pr_info("crashkernel reservation failed - No suitable area found.\n");
 			return;
 		}
-
 	} else {
 		unsigned long long start;
 
Documentation/admin-guide/kernel-parameters.txt +5 -2
@@ -704,8 +704,11 @@
 			upon panic. This parameter reserves the physical
 			memory region [offset, offset + size] for that kernel
 			image. If '@offset' is omitted, then a suitable offset
-			is selected automatically. Check
-			Documentation/kdump/kdump.txt for further details.
+			is selected automatically.
+			[KNL, x86_64] select a region under 4G first, and
+			fallback to reserve region above 4G in case without
+			'@offset'.
+			See Documentation/kdump/kdump.txt for further details.
 
 	crashkernel=range1:size1[,range2:size2,...][@offset]
 			[KNL] Same as above, but depends on the memory
Reviewed-by on v1 1/2
#7 lore raw
In reply to Dave Young
Quoted 64 lines · Dave Young

The kdump crashkernel low reservation is limited to under 896M even for X86_64. This obscure and miserable limitation exists for old kexec-tools compatibility, but the reason is not documented anywhere.

Some more tests/investigations about the background:
a) Previously old kexec-tools can only load purgatory to memory under 2G, Eric remove that limitation in 2012 in kexec-tools:
Commit b4f9f8599679 ("kexec x86_64: Make purgatory relocatable anywhere in the 64bit address space.")

b) back in 2013 Yinghai removed all the limitations in new kexec-tools, bzImage64 can be loaded to anywhere.
Commit 82c3dd2280d2 ("kexec, x86_64: Load bzImage64 above 4G")

c) test results with old kexec-tools with old and latest kernels.
1. old kexec-tools can not build with modern toolchain anymore, I built it in a RHEL6 vm
2. 2.0.0 kexec-tools does not work with latest kernel even with memory under 896M and give an error:
"ELF core (kcore) parse failed", it needs below kexec-tools fix Commit ed15ba1b9977 ("build_mem_phdrs(): check if p_paddr is invalid")
3. even with patched kexec-tools which fixes 2), it still needs some other fixes to work correctly for kaslr enabled kernels.

So the situation is:
* old kexec-tools is already broken with latest kernels
* we can not keep this limitations forever just for compatibility of very old kexec-tools.
* If one must use old tools then he/she can choose crashkernel=X@Y
* people have reported bugs crashkernel=384M failed because kaslr makes the 0-896M space sparse,
* crashkernel can reserve in low or high area, it is natural to understand low as memory under 4G

Hence drop the 896M limitation, and change crashkernel low reservation to reserve under 4G by default.

---
 arch/x86/kernel/setup.c |   10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
arch/x86/kernel/setup.c +2 -2
@@ -71,6 +71,7 @@
 #include <linux/tboot.h>
 #include <linux/jiffies.h>
 #include <linux/mem_encrypt.h>
+#include <linux/sizes.h>
 
 #include <linux/usb/xhci-dbgp.h>
 #include <video/edid.h>
@@ -448,18 +449,17 @@ static void __init memblock_x86_reserve_
 #ifdef CONFIG_KEXEC_CORE
 
 /* 16M alignment for crash kernel regions */
-#define CRASH_ALIGN		(16 << 20)
+#define CRASH_ALIGN		SZ_16M
 
 /*
  * Keep the crash kernel below this limit.  On 32 bits earlier kernels
  * would limit the kernel to the low 512 MiB due to mapping restrictions.
- * On 64bit, old kexec-tools need to under 896MiB.
  */

arch/x86/kernel/setup.c static void __init memblock_x86_reserve_ · line 464

Quoted 10 lines
arch/x86/kernel/setup.c +3 -3
 #ifdef CONFIG_X86_32
-# define CRASH_ADDR_LOW_MAX	(512 << 20)
-# define CRASH_ADDR_HIGH_MAX	(512 << 20)
+# define CRASH_ADDR_LOW_MAX	SZ_512M
+# define CRASH_ADDR_HIGH_MAX	SZ_512M
 #else
-# define CRASH_ADDR_LOW_MAX	(896UL << 20)
+# define CRASH_ADDR_LOW_MAX	SZ_4G
 # define CRASH_ADDR_HIGH_MAX	MAXMEM
 #endif

Thanks,

Ingo

Reviewed-by on v1 2/2
#2 lore raw

Re: [PATCH 2/2] X86/kdump: fall back to reserve high crashkernel memory

In reply to Dave Young
Quoted 19 lines · Dave Young

crashkernel=xM tries to reserve crashkernel memory under 4G, which is enough for usual cases. But this could fail sometimes, for example one tries to reserve a big chunk like 2G, it is possible to fail.

So let the crashkernel=xM just fall back to use high memory in case it fails to find a suitable low range. Do not set the ,high as default because it allocs extra low memory for DMA buffers and swiotlb, this is not always necessary for all machines. Typically like crashkernel=128M usually work with low reservation under 4G, so still keep <4G as default.

---
 Documentation/admin-guide/kernel-parameters.txt |    7 +++++--
 arch/x86/kernel/setup.c                         |   22 ++++++++++++++--------
 2 files changed, 19 insertions(+), 10 deletions(-)

arch/x86/kernel/setup.c static void __init reserve_crashkernel(v · line 547

Quoted 10 lines
arch/x86/kernel/setup.c +2 -3
@@ -541,21 +541,27 @@ static void __init reserve_crashkernel(v
 	}
 
 	/* 0 means: find the address automatically */
-	if (crash_base <= 0) {
+	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * as old kexec-tools loads bzImage below that, unless
-		 * "crashkernel=size[KMG],high" is specified.
+		 * as crashkernel=x,high allocs memory over 4G, also allocs

s/allocs
/allocates

arch/x86/kernel/setup.c static void __init reserve_crashkernel(v · line 550

Quoted 3 lines
arch/x86/kernel/setup.c +3 -0
+		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * but the extra memory is not required for all machines.
+		 * So prefer low memory first, and fallback to high memory

s/fallback
/fall back

Quoted 25 lines
arch/x86/kernel/setup.c +9 -5
+		 * unless "crashkernel=size[KMG],high" is specified.
 		 */
-		crash_base = memblock_find_in_range(CRASH_ALIGN,
-						    high ? CRASH_ADDR_HIGH_MAX
-							 : CRASH_ADDR_LOW_MAX,
-						    crash_size, CRASH_ALIGN);
+		if (!high)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_LOW_MAX,
+						crash_size, CRASH_ALIGN);
+		if (!crash_base)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_HIGH_MAX,
+						crash_size, CRASH_ALIGN);
 		if (!crash_base) {
 			pr_info("crashkernel reservation failed - No suitable area found.\n");
 			return;
 		}
-
 	} else {
 		unsigned long long start;
 

Documentation/admin-guide/kernel-parameters.txt line 709

Quoted 8 lines
Documentation/admin-guide/kernel-parameters.txt +3 -2
@@ -704,8 +704,11 @@
 			upon panic. This parameter reserves the physical
 			memory region [offset, offset + size] for that kernel
 			image. If '@offset' is omitted, then a suitable offset
-			is selected automatically. Check
-			Documentation/kdump/kdump.txt for further details.
+			is selected automatically.
+			[KNL, x86_64] select a region under 4G first, and
+			fallback to reserve region above 4G in case without

s/fallback
/fall back

Documentation/admin-guide/kernel-parameters.txt line 714

Quoted 5 lines
Documentation/admin-guide/kernel-parameters.txt +2 -0
+			'@offset'.
+			See Documentation/kdump/kdump.txt for further details.
 
 	crashkernel=range1:size1[,range2:size2,...][@offset]
 			[KNL] Same as above, but depends on the memory

With the nits fixed:

Thanks,

Ingo

author
#3 lore raw

Re: [PATCH 2/2] X86/kdump: fall back to reserve high crashkernel memory

Quoted 83 lines · Ingo Molnar
Quoted 19 lines · Dave Young

crashkernel=xM tries to reserve crashkernel memory under 4G, which is enough for usual cases. But this could fail sometimes, for example one tries to reserve a big chunk like 2G, it is possible to fail.

So let the crashkernel=xM just fall back to use high memory in case it fails to find a suitable low range. Do not set the ,high as default because it allocs extra low memory for DMA buffers and swiotlb, this is not always necessary for all machines. Typically like crashkernel=128M usually work with low reservation under 4G, so still keep <4G as default.

---
 Documentation/admin-guide/kernel-parameters.txt |    7 +++++--
 arch/x86/kernel/setup.c                         |   22 ++++++++++++++--------
 2 files changed, 19 insertions(+), 10 deletions(-)

arch/x86/kernel/setup.c static void __init reserve_crashkernel(v · line 547

Quoted 10 lines
arch/x86/kernel/setup.c +2 -3
@@ -541,21 +541,27 @@ static void __init reserve_crashkernel(v
 	}
 
 	/* 0 means: find the address automatically */
-	if (crash_base <= 0) {
+	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * as old kexec-tools loads bzImage below that, unless
-		 * "crashkernel=size[KMG],high" is specified.
+		 * as crashkernel=x,high allocs memory over 4G, also allocs

s/allocs
/allocates

arch/x86/kernel/setup.c static void __init reserve_crashkernel(v · line 550

Quoted 3 lines
arch/x86/kernel/setup.c +3 -0
+		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * but the extra memory is not required for all machines.
+		 * So prefer low memory first, and fallback to high memory

s/fallback
/fall back

Quoted 25 lines
arch/x86/kernel/setup.c +9 -5
+		 * unless "crashkernel=size[KMG],high" is specified.
 		 */
-		crash_base = memblock_find_in_range(CRASH_ALIGN,
-						    high ? CRASH_ADDR_HIGH_MAX
-							 : CRASH_ADDR_LOW_MAX,
-						    crash_size, CRASH_ALIGN);
+		if (!high)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_LOW_MAX,
+						crash_size, CRASH_ALIGN);
+		if (!crash_base)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_HIGH_MAX,
+						crash_size, CRASH_ALIGN);
 		if (!crash_base) {
 			pr_info("crashkernel reservation failed - No suitable area found.\n");
 			return;
 		}
-
 	} else {
 		unsigned long long start;
 

Documentation/admin-guide/kernel-parameters.txt line 709

Quoted 8 lines
Documentation/admin-guide/kernel-parameters.txt +3 -2
@@ -704,8 +704,11 @@
 			upon panic. This parameter reserves the physical
 			memory region [offset, offset + size] for that kernel
 			image. If '@offset' is omitted, then a suitable offset
-			is selected automatically. Check
-			Documentation/kdump/kdump.txt for further details.
+			is selected automatically.
+			[KNL, x86_64] select a region under 4G first, and
+			fallback to reserve region above 4G in case without

s/fallback
/fall back

Quoted 3 lines
Documentation/admin-guide/kernel-parameters.txt +2 -0
+			'@offset'.
+			See Documentation/kdump/kdump.txt for further details.
 
Quoted 6 lines

Documentation/admin-guide/kernel-parameters.txt line 714

Quoted 2 lines
Documentation/admin-guide/kernel-parameters.txt +0 -0
 	crashkernel=range1:size1[,range2:size2,...][@offset]
 			[KNL] Same as above, but depends on the memory

With the nits fixed:

Thanks for review, will reply to 2/2 with an update of those spelling issues.

Dave

author
#4 lore raw

PATCH 2/2 X86/kdump: fall back to reserve high crashkernel memory

In reply to Dave Young

crashkernel=xM tries to reserve crashkernel memory under 4G, which is enough for usual cases. But this could fail sometimes, for example one tries to reserve a big chunk like 2G, it is possible to fail.

So let the crashkernel=xM just fall back to use high memory in case it fails to find a suitable low range. Do not set the ,high as default because it allocates extra low memory for DMA buffers and swiotlb, this is not always necessary for all machines. Typically like crashkernel=128M usually work with low reservation under 4G, so still keep <4G as default.

---
 Documentation/admin-guide/kernel-parameters.txt |    7 +++++--
 arch/x86/kernel/setup.c                         |   22 ++++++++++++++--------
 2 files changed, 19 insertions(+), 10 deletions(-)
arch/x86/kernel/setup.c +14 -8
@@ -541,21 +541,27 @@ static void __init reserve_crashkernel(v
 	}
 
 	/* 0 means: find the address automatically */
-	if (crash_base <= 0) {
+	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * as old kexec-tools loads bzImage below that, unless
-		 * "crashkernel=size[KMG],high" is specified.
+		 * crashkernel=x,high reserves memory over 4G, also allocates
+		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * but the extra memory is not required for all machines.
+		 * So prefer low memory first, and fall back to high memory
+		 * unless "crashkernel=size[KMG],high" is specified.
 		 */
-		crash_base = memblock_find_in_range(CRASH_ALIGN,
-						    high ? CRASH_ADDR_HIGH_MAX
-							 : CRASH_ADDR_LOW_MAX,
-						    crash_size, CRASH_ALIGN);
+		if (!high)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_LOW_MAX,
+						crash_size, CRASH_ALIGN);
+		if (!crash_base)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_HIGH_MAX,
+						crash_size, CRASH_ALIGN);
 		if (!crash_base) {
 			pr_info("crashkernel reservation failed - No suitable area found.\n");
 			return;
 		}
-
 	} else {
 		unsigned long long start;
 
Documentation/admin-guide/kernel-parameters.txt +5 -2
@@ -704,8 +704,11 @@
 			upon panic. This parameter reserves the physical
 			memory region [offset, offset + size] for that kernel
 			image. If '@offset' is omitted, then a suitable offset
-			is selected automatically. Check
-			Documentation/kdump/kdump.txt for further details.
+			is selected automatically.
+			[KNL, x86_64] select a region under 4G first, and
+			fall back to reserve region above 4G in case without
+			'@offset'.
+			See Documentation/kdump/kdump.txt for further details.
 
 	crashkernel=range1:size1[,range2:size2,...][@offset]
 			[KNL] Same as above, but depends on the memory
Acked-by on v1 2/2
#5 lore raw

Re: [PATCH 2/2 update] X86/kdump: fall back to reserve high crashkernel memory

Quoted 10 lines · Dave Young

crashkernel=xM tries to reserve crashkernel memory under 4G, which is enough for usual cases. But this could fail sometimes, for example one tries to reserve a big chunk like 2G, it is possible to fail.

So let the crashkernel=xM just fall back to use high memory in case it fails to find a suitable low range. Do not set the ,high as default because it allocates extra low memory for DMA buffers and swiotlb, this is not always necessary for all machines. Typically like crashkernel=128M usually work with low reservation under 4G, so still keep <4G as default.

Quoted 3 lines

---

Ack the whole series, thanks for the effort.

Quoted 58 lines
 Documentation/admin-guide/kernel-parameters.txt |    7 +++++--
 arch/x86/kernel/setup.c                         |   22 ++++++++++++++--------
 2 files changed, 19 insertions(+), 10 deletions(-)
arch/x86/kernel/setup.c +14 -8
@@ -541,21 +541,27 @@ static void __init reserve_crashkernel(v
 	}
 
 	/* 0 means: find the address automatically */
-	if (crash_base <= 0) {
+	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * as old kexec-tools loads bzImage below that, unless
-		 * "crashkernel=size[KMG],high" is specified.
+		 * crashkernel=x,high reserves memory over 4G, also allocates
+		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * but the extra memory is not required for all machines.
+		 * So prefer low memory first, and fall back to high memory
+		 * unless "crashkernel=size[KMG],high" is specified.
 		 */
-		crash_base = memblock_find_in_range(CRASH_ALIGN,
-						    high ? CRASH_ADDR_HIGH_MAX
-							 : CRASH_ADDR_LOW_MAX,
-						    crash_size, CRASH_ALIGN);
+		if (!high)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_LOW_MAX,
+						crash_size, CRASH_ALIGN);
+		if (!crash_base)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_HIGH_MAX,
+						crash_size, CRASH_ALIGN);
 		if (!crash_base) {
 			pr_info("crashkernel reservation failed - No suitable area found.\n");
 			return;
 		}
-
 	} else {
 		unsigned long long start;
 
Documentation/admin-guide/kernel-parameters.txt +5 -2
@@ -704,8 +704,11 @@
 			upon panic. This parameter reserves the physical
 			memory region [offset, offset + size] for that kernel
 			image. If '@offset' is omitted, then a suitable offset
-			is selected automatically. Check
-			Documentation/kdump/kdump.txt for further details.
+			is selected automatically.
+			[KNL, x86_64] select a region under 4G first, and
+			fall back to reserve region above 4G in case without
+			'@offset'.
+			See Documentation/kdump/kdump.txt for further details.
 
 	crashkernel=range1:size1[,range2:size2,...][@offset]
 			[KNL] Same as above, but depends on the memory
#6 lore raw

[tip:x86/kdump] x86/kdump: Fall back to reserve high crashkernel memory

In reply to Dave Young
x86/kdump: Fall back to reserve high crashkernel memory
Commit-ID:  b9ac3849af412fd3887d7652bdbabf29d2aecc16
Gitweb:     https://git.kernel.org/tip/b9ac3849af412fd3887d7652bdbabf29d2aecc16
Author:     Dave Young <[email protected]>
AuthorDate: Mon, 22 Apr 2019 11:19:05 +0800
Committer:  Borislav Petkov <[email protected]>
CommitDate: Mon, 22 Apr 2019 10:23:05 +0200

x86/kdump: Fall back to reserve high crashkernel memory

crashkernel=xM tries to reserve memory for the crash kernel under 4G, which is enough, usually. But this could fail sometimes, for example when one tries to reserve a big chunk like 2G, for example.

So let the crashkernel=xM just fall back to use high memory in case it fails to find a suitable low range. Do not set the ,high as default because it allocates extra low memory for DMA buffers and swiotlb, and this is not always necessary for all machines.

Typically, crashkernel=128M usually works with low reservation under 4G, so keep <4G as default.

[ bp: Massage. ]

Cc Dave Young <[email protected]>
Cc David Howells <[email protected]>
Cc Eric Biederman <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc "H. Peter Anvin" <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Jiri Kosina <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Juergen Gross <[email protected]>
Cc Kees Cook <[email protected]>
Cc Konrad Rzeszutek Wilk <[email protected]>
Cc "Paul E. McKenney" <[email protected]>
Cc Petr Tesarik <[email protected]>
Cc Ram Pai <[email protected]>
Cc Sinan Kaya <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Thymo van Beers <[email protected]>
Cc x86-ml <[email protected]>
Cc Yinghai Lu <[email protected]>
Cc Zhimin Gu <[email protected]>
---
 Documentation/admin-guide/kernel-parameters.txt |  7 +++++--
 arch/x86/kernel/setup.c                         | 22 ++++++++++++++--------
 2 files changed, 19 insertions(+), 10 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +5 -2
index 2b8ee90bb644..24d01648edeb 100644
@@ -704,8 +704,11 @@
 			upon panic. This parameter reserves the physical
 			memory region [offset, offset + size] for that kernel
 			image. If '@offset' is omitted, then a suitable offset
-			is selected automatically. Check
-			Documentation/kdump/kdump.txt for further details.
+			is selected automatically.
+			[KNL, x86_64] select a region under 4G first, and
+			fall back to reserve region above 4G when '@offset'
+			hasn't been specified.
+			See Documentation/kdump/kdump.txt for further details.
 
 	crashkernel=range1:size1[,range2:size2,...][@offset]
 			[KNL] Same as above, but depends on the memory
arch/x86/kernel/setup.c +14 -8
index daf7c5650c18..c15f362a2516 100644
@@ -541,21 +541,27 @@ static void __init reserve_crashkernel(void)
 	}
 
 	/* 0 means: find the address automatically */
-	if (crash_base <= 0) {
+	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * as old kexec-tools loads bzImage below that, unless
-		 * "crashkernel=size[KMG],high" is specified.
+		 * crashkernel=x,high reserves memory over 4G, also allocates
+		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * But the extra memory is not required for all machines.
+		 * So try low memory first and fall back to high memory
+		 * unless "crashkernel=size[KMG],high" is specified.
 		 */
-		crash_base = memblock_find_in_range(CRASH_ALIGN,
-						    high ? CRASH_ADDR_HIGH_MAX
-							 : CRASH_ADDR_LOW_MAX,
-						    crash_size, CRASH_ALIGN);
+		if (!high)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_LOW_MAX,
+						crash_size, CRASH_ALIGN);
+		if (!crash_base)
+			crash_base = memblock_find_in_range(CRASH_ALIGN,
+						CRASH_ADDR_HIGH_MAX,
+						crash_size, CRASH_ALIGN);
 		if (!crash_base) {
 			pr_info("crashkernel reservation failed - No suitable area found.\n");
 			return;
 		}
-
 	} else {
 		unsigned long long start;
 
#8 lore raw

[tip:x86/kdump] x86/kdump: Have crashkernel=X reserve under 4G by default

In reply to Dave Young
x86/kdump: Have crashkernel=X reserve under 4G by default
Commit-ID:  9ca5c8e632ce8f144ec6d00da2dc5e16b41d593c
Gitweb:     https://git.kernel.org/tip/9ca5c8e632ce8f144ec6d00da2dc5e16b41d593c
Author:     Dave Young <[email protected]>
AuthorDate: Sun, 21 Apr 2019 11:50:59 +0800
Committer:  Borislav Petkov <[email protected]>
CommitDate: Mon, 22 Apr 2019 10:15:16 +0200

x86/kdump: Have crashkernel=X reserve under 4G by default

The kdump crashkernel low reservation is limited to under 896M even for X86_64. This obscure and miserable limitation exists for compatibility with old kexec-tools but the reason is not documented anywhere.

Some more tests/investigations about the background:

a) Previously, old kexec-tools could only load purgatory to memory under 2G. Eric removed that limitation in 2012 in kexec-tools:

b4f9f8599679 ("kexec x86_64: Make purgatory relocatable anywhere
in the 64bit address space.")

b) Back in 2013 Yinghai removed all the limitations in new kexec-tools, bzImage64 can be loaded anywhere:

     82c3dd2280d2 ("kexec, x86_64: Load bzImage64 above 4G")

c) Test results with old kexec-tools with old and latest kernels:

1. Old kexec-tools can not build with modern toolchain anymore, I built it in a RHEL6 vm.

2. 2.0.0 kexec-tools does not work with the latest kernel even with memory under 896M and gives an error:

"ELF core (kcore) parse failed"

For that it needs below kexec-tools fix:

ed15ba1b9977 ("build_mem_phdrs(): check if p_paddr is invalid")

3. Even with patched kexec-tools which fixes 2), it still needs some other fixes to work correctly for KASLR-enabled kernels.

So the situation is:

* Old kexec-tools is already broken with latest kernels.

* We can not keep these limitations forever just for compatibility with very old kexec-tools.

* If one must use old tools then he/she can choose crashkernel=X@Y.

* People have reported bugs where crashkernel=384M failed because KASLR makes the 0-896M space sparse.

* Crashkernel can reserve in low or high area, it is natural to understand low as memory under 4G.

Hence drop the 896M limitation and change crashkernel low reservation to reserve under 4G by default.

---
 arch/x86/kernel/setup.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
arch/x86/kernel/setup.c +5 -5
index 3d872a527cd9..daf7c5650c18 100644
@@ -71,6 +71,7 @@
 #include <linux/tboot.h>
 #include <linux/jiffies.h>
 #include <linux/mem_encrypt.h>
+#include <linux/sizes.h>
 
 #include <linux/usb/xhci-dbgp.h>
 #include <video/edid.h>
@@ -448,18 +449,17 @@ static void __init memblock_x86_reserve_range_setup_data(void)
 #ifdef CONFIG_KEXEC_CORE
 
 /* 16M alignment for crash kernel regions */
-#define CRASH_ALIGN		(16 << 20)
+#define CRASH_ALIGN		SZ_16M
 
 /*
  * Keep the crash kernel below this limit.  On 32 bits earlier kernels
  * would limit the kernel to the low 512 MiB due to mapping restrictions.
- * On 64bit, old kexec-tools need to under 896MiB.
  */
 #ifdef CONFIG_X86_32
-# define CRASH_ADDR_LOW_MAX	(512 << 20)
-# define CRASH_ADDR_HIGH_MAX	(512 << 20)
+# define CRASH_ADDR_LOW_MAX	SZ_512M
+# define CRASH_ADDR_HIGH_MAX	SZ_512M
 #else
-# define CRASH_ADDR_LOW_MAX	(896UL << 20)
+# define CRASH_ADDR_LOW_MAX	SZ_4G
 # define CRASH_ADDR_HIGH_MAX	MAXMEM
 #endif