Skip to contentSkip to conversation

support reserving crashkernel above 4G on arm64 kdump

PATCH Superseded Chen Zhou 6 participants 36 messages Sep 7, 2020 – Oct 19, 2020

Subsystems (7)
Documentation x86 arm64 Live update +3 more subsystems

v12 of 14 Superseded Latest is v14 →

lore
Share & tools
lore

Chen Zhou

author
#0 lore raw

v12 cover support reserving crashkernel above 4G on arm64 kdump

There are following issues in arm64 kdump:
1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory.
2. If reserving crashkernel above 4G, in this case, crash dump kernel will boot failure because there is no low memory available for allocation.
3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"), if the memory reserved for crash dump kernel falled in ZONE_DMA32, the devices in crash dump kernel need to use ZONE_DMA will alloc fail.

To solve these issues, change the behavior of crashkernel=X. crashkernel=X tries low allocation in DMA zone, and fall back to high allocation if it fails.
If requized size X is too large and leads to very little low memory in DMA zone after low allocation, the system may not work normally. So add a threshold and go for high allocation directly if the required size is too large. The value of threshold is set as the half of the low memory.

We can also use "crashkernel=X,high" to select a high region above DMA zone, which also tries to allocate at least 256M low memory in DMA zone automatically.
"crashkernel=Y,low" can be used to allocate specified size low memory. For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.

When reserving crashkernel in high memory, some low memory is reserved for crash dump kernel devices. So there may be two regions reserved for crash dump kernel.
In order to distinct from the high region and make no effect to the use of existing kexec-tools, rename the low region as "Crash kernel (low)", and pass the low region by reusing DT property "linux,usable-memory-range". We made the low memory region as the last range of "linux,usable-memory-range" to keep compatibility with existing user-space and older kdump kernels.

Besides, we need to modify kexec-tools:
arm64: support more than one crash kernel regions(see [1])

Another update is document about DT property 'linux,usable-memory-range':
schemas: update 'linux,usable-memory-range' node schema(see [2])

This patchset contains the following nine patches:
0001-x86-kdump-move-CRASH_ALIGN-to-2M.patch 0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch 0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch 0004-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch 0005-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch 0006-arm64-kdump-reimplement-crashkernel-X.patch 0007-kdump-add-threshold-for-the-required-memory.patch 0008-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch 0009-kdump-update-Documentation-about-crashkernel.patch

0001-0003 are some x86 cleanups which prepares for making functionsreserve_crashkernel[_low]() generic.

0004 makes functions reserve_crashkernel[_low]() generic. 0005-0006 reimplements crashkernel=X.
0007 adds threshold for the required memory.
0008 adds memory for devices by DT property linux,usable-memory-range. 0009 updates the doc.

Changes since [v11]
- Rebased on top of 5.9-rc4.
- Make the function reserve_crashkernel() of x86 generic. Suggested by Catalin, make the function reserve_crashkernel() of x86 generic and arm64 use the generic version to reimplement crashkernel=X.

Changes since [v10]
- Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin.

Changes since [v9]
- Patch 1 add Acked-by from Dave.
- Update patch 5 according to Dave's comments.
- Update chosen schema.

Changes since [v8]
- Reuse DT property "linux,usable-memory-range".
Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the low memory region.
- Fix kdump broken with ZONE_DMA reintroduced.
- Update chosen schema.

Changes since [v7]
- Move x86 CRASH_ALIGN to 2M
Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
- Update Documentation/devicetree/bindings/chosen.txt.
Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt suggested by Arnd.
- Add Tested-by from Jhon and pk.

Changes since [v6]
- Fix build errors reported by kbuild test robot.

Changes since [v5]
- Move reserve_crashkernel_low() into kernel/crash_core.c.
- Delete crashkernel=X,high.
- Modify crashkernel=X,low.
If crashkernel=X,low is specified simultaneously, reserve spcified size low memory for crash kdump kernel devices firstly and then reserve memory above 4G. In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then pass to crash dump kernel by DT property "linux,low-memory-range".
- Update Documentation/admin-guide/kdump/kdump.rst.

Changes since [v4]
- Reimplement memblock_cap_memory_ranges for multiple ranges by Mike.

Changes since [v3]
- Add memblock_cap_memory_ranges back for multiple ranges.
- Fix some compiling warnings.

Changes since [v2]
- Split patch "arm64: kdump: support reserving crashkernel above 4G" as two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate patch.

Changes since [v1]:
- Move common reserve_crashkernel_low() code into kernel/kexec_core.c.
- Remove memblock_cap_memory_ranges() i added in v1 and implement that in fdt_enforce_memory_region().
There are at most two crash kernel regions, for two crash kernel regions case, we cap the memory range [min(regs[*].start), max(regs[*].end)] and then remove the memory range in the middle.

[1]: http://lists.infradead.org/pipermail/kexec/2020-June/020737.html
[2]: https://github.com/robherring/dt-schema/pull/19
[v1]: https://lkml.org/lkml/2019/4/2/1174
[v2]: https://lkml.org/lkml/2019/4/9/86
[v3]: https://lkml.org/lkml/2019/4/9/306
[v4]: https://lkml.org/lkml/2019/4/15/273
[v5]: https://lkml.org/lkml/2019/5/6/1360
[v6]: https://lkml.org/lkml/2019/8/30/142
[v7]: https://lkml.org/lkml/2019/12/23/411
[v8]: https://lkml.org/lkml/2020/5/21/213
[v9]: https://lkml.org/lkml/2020/6/28/73
[v10]: https://lkml.org/lkml/2020/7/2/1443
[v11]: https://lkml.org/lkml/2020/8/1/150

Chen Zhou (9):
x86: kdump: move CRASH_ALIGN to 2M
x86: kdump: make the lower bound of crash kernel reservation
consistent
x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions
reserve_crashkernel[_low]()
x86: kdump: move reserve_crashkernel[_low]() into crash_core.c
arm64: kdump: introduce some macroes for crash kernel reservation
arm64: kdump: reimplement crashkernel=X
kdump: add threshold for the required memory
arm64: kdump: add memory for devices by DT property
linux,usable-memory-range
kdump: update Documentation about crashkernel

 Documentation/admin-guide/kdump/kdump.rst     |  25 ++-
 .../admin-guide/kernel-parameters.txt         |  13 +-
 arch/arm64/include/asm/kexec.h                |  15 ++
 arch/arm64/include/asm/processor.h            |   1 +
 arch/arm64/kernel/setup.c                     |  13 +-
 arch/arm64/mm/init.c                          | 105 ++++------
 arch/arm64/mm/mmu.c                           |   4 +
 arch/x86/include/asm/kexec.h                  |  28 +++
 arch/x86/kernel/setup.c                       | 165 +--------------
 include/linux/crash_core.h                    |   4 +
 include/linux/kexec.h                         |   2 -
 kernel/crash_core.c                           | 192 ++++++++++++++++++
 kernel/kexec_core.c                           |  17 --
 13 files changed, 328 insertions(+), 256 deletions(-)
Signature · 7 lines
2.20.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#1 lore raw

PATCH v12 1/9 x86: kdump: move CRASH_ALIGN to 2M

CONFIG_PHYSICAL_ALIGN can be selected from 2M to 16M and default value is 2M, so move CRASH_ALIGN to 2M, with smaller value reservation can have more chance to succeed.
And replace the hard-coded alignment with macro CRASH_ALIGN in function reserve_crashkernel().

---
 arch/x86/include/asm/kexec.h | 3 +++
 arch/x86/kernel/setup.c      | 5 +----
 2 files changed, 4 insertions(+), 4 deletions(-)
arch/x86/include/asm/kexec.h +3 -0
index 6802c59e8252..83f200dd54a1 100644
@@ -18,6 +18,9 @@
 
 # define KEXEC_CONTROL_CODE_MAX_SIZE	2048
 
+/* 2M alignment for crash kernel regions */
+#define CRASH_ALIGN		SZ_2M
+
 #ifndef __ASSEMBLY__
 
 #include <linux/string.h>
arch/x86/kernel/setup.c +1 -4
index 3511736fbc74..296294ad0dd8 100644
@@ -402,9 +402,6 @@ static void __init memblock_x86_reserve_range_setup_data(void)
 
 #ifdef CONFIG_KEXEC_CORE
 
-/* 16M alignment for crash kernel regions */
-#define CRASH_ALIGN		SZ_16M
-
 /*
  * Keep the crash kernel below this limit.
  *
@@ -530,7 +527,7 @@ static void __init reserve_crashkernel(void)
 
 		start = memblock_find_in_range(crash_base,
 					       crash_base + crash_size,
-					       crash_size, 1 << 20);
+					       crash_size, CRASH_ALIGN);
 		if (start != crash_base) {
 			pr_info("crashkernel reservation failed - memory is in use.\n");
 			return;
Signature · 7 lines
2.20.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#2 lore raw

Re: [PATCH v12 1/9] x86: kdump: move CRASH_ALIGN to 2M

Hi,

Quoted 3 lines · Chen Zhou

CONFIG_PHYSICAL_ALIGN can be selected from 2M to 16M and default value is 2M, so move CRASH_ALIGN to 2M, with smaller value reservation can have more chance to succeed.

Seems still some misunderstanding about the change :( I'm sorry if I did not explain it clearly.

Previously I missed the PHYSICAL_ALIGN can change according to .config I mean we should change the value to CONFIG_PHYSICAL_ALIGN for X86 And I suggest to move back to keep using 16M. And do not change it in this series.

Quoted 50 lines

And replace the hard-coded alignment with macro CRASH_ALIGN in function reserve_crashkernel().

---
 arch/x86/include/asm/kexec.h | 3 +++
 arch/x86/kernel/setup.c      | 5 +----
 2 files changed, 4 insertions(+), 4 deletions(-)
arch/x86/include/asm/kexec.h +3 -0
index 6802c59e8252..83f200dd54a1 100644
@@ -18,6 +18,9 @@
 
 # define KEXEC_CONTROL_CODE_MAX_SIZE	2048
 
+/* 2M alignment for crash kernel regions */
+#define CRASH_ALIGN		SZ_2M
+
 #ifndef __ASSEMBLY__
 
 #include <linux/string.h>
arch/x86/kernel/setup.c +1 -4
index 3511736fbc74..296294ad0dd8 100644
@@ -402,9 +402,6 @@ static void __init memblock_x86_reserve_range_setup_data(void)
 
 #ifdef CONFIG_KEXEC_CORE
 
-/* 16M alignment for crash kernel regions */
-#define CRASH_ALIGN		SZ_16M
-
 /*
  * Keep the crash kernel below this limit.
  *
@@ -530,7 +527,7 @@ static void __init reserve_crashkernel(void)
 
 		start = memblock_find_in_range(crash_base,
 					       crash_base + crash_size,
-					       crash_size, 1 << 20);
+					       crash_size, CRASH_ALIGN);
 		if (start != crash_base) {
 			pr_info("crashkernel reservation failed - memory is in use.\n");
 			return;
Signature · 1 line
2.20.1

Thanks
Dave

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#3 lore raw

Re: [PATCH v12 1/9] x86: kdump: move CRASH_ALIGN to 2M

Quoted 6 lines · Dave Young

Hi,

Quoted 3 lines · Chen Zhou

CONFIG_PHYSICAL_ALIGN can be selected from 2M to 16M and default value is 2M, so move CRASH_ALIGN to 2M, with smaller value reservation can have more chance to succeed.

Quoted 7 lines

Seems still some misunderstanding about the change :( I'm sorry if I did not explain it clearly.

Previously I missed the PHYSICAL_ALIGN can change according to .config I mean we should change the value to CONFIG_PHYSICAL_ALIGN for X86 And I suggest to move back to keep using 16M. And do not change it in this series.

Hi Dave,

Sorry, i misunderstood about this.

Ok, this patch will keep the value of CRASH_ALIGN as it is,
just move CRASH_ALIGN to header asm/kexec.h and replace the hard-coded alignment with macro CRASH_ALIGN in function reserve_crashkernel().

Thanks,
Chen Zhou

Quoted 55 lines
Quoted 51 lines

And replace the hard-coded alignment with macro CRASH_ALIGN in function reserve_crashkernel().

---
 arch/x86/include/asm/kexec.h | 3 +++
 arch/x86/kernel/setup.c      | 5 +----
 2 files changed, 4 insertions(+), 4 deletions(-)
arch/x86/include/asm/kexec.h +3 -0
index 6802c59e8252..83f200dd54a1 100644
@@ -18,6 +18,9 @@
 
 # define KEXEC_CONTROL_CODE_MAX_SIZE	2048
 
+/* 2M alignment for crash kernel regions */
+#define CRASH_ALIGN		SZ_2M
+
 #ifndef __ASSEMBLY__
 
 #include <linux/string.h>
arch/x86/kernel/setup.c +1 -4
index 3511736fbc74..296294ad0dd8 100644
@@ -402,9 +402,6 @@ static void __init memblock_x86_reserve_range_setup_data(void)
 
 #ifdef CONFIG_KEXEC_CORE
 
-/* 16M alignment for crash kernel regions */
-#define CRASH_ALIGN		SZ_16M
-
 /*
  * Keep the crash kernel below this limit.
  *
@@ -530,7 +527,7 @@ static void __init reserve_crashkernel(void)
 
 		start = memblock_find_in_range(crash_base,
 					       crash_base + crash_size,
-					       crash_size, 1 << 20);
+					       crash_size, CRASH_ALIGN);
 		if (start != crash_base) {
 			pr_info("crashkernel reservation failed - memory is in use.\n");
 			return;
Signature · 1 line
2.20.1

Thanks
Dave

Quoted 2 lines

.

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#4 lore raw

PATCH v12 2/9 x86: kdump: make the lower bound of crash kernel reservation consistent

In reply to Chen Zhou

The lower bounds of crash kernel reservation and crash kernel low reservation are different, use the consistent value CRASH_ALIGN.

---
 arch/x86/kernel/setup.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kernel/setup.c +1 -1
index 296294ad0dd8..d7fd90c52dae 100644
@@ -451,7 +451,7 @@ static int __init reserve_crashkernel_low(void)
 			return 0;
 	}
 
-	low_base = memblock_find_in_range(0, 1ULL << 32, low_size, CRASH_ALIGN);
+	low_base = memblock_find_in_range(CRASH_ALIGN, 1ULL << 32, low_size, CRASH_ALIGN);
 	if (!low_base) {
 		pr_err("Cannot reserve %ldMB crashkernel low memory, please try smaller size.\n",
 		       (unsigned long)(low_size >> 20));
Signature · 7 lines
2.20.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#5 lore raw

PATCH v12 3/9 x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]()

In reply to Chen Zhou

To make the functions reserve_crashkernel[_low]() as generic, replace some hard-coded numbers with macro CRASH_ADDR_LOW_MAX.

---
 arch/x86/kernel/setup.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
arch/x86/kernel/setup.c +6 -5
index d7fd90c52dae..71a6a6e7ca5b 100644
@@ -430,7 +430,7 @@ static int __init reserve_crashkernel_low(void)
 	unsigned long total_low_mem;
 	int ret;
 
-	total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
+	total_low_mem = memblock_mem_size(CRASH_ADDR_LOW_MAX >> PAGE_SHIFT);
 
 	/* crashkernel=Y,low */
 	ret = parse_crashkernel_low(boot_command_line, total_low_mem, &low_size, &base);
@@ -451,7 +451,7 @@ static int __init reserve_crashkernel_low(void)
 			return 0;
 	}
 
-	low_base = memblock_find_in_range(CRASH_ALIGN, 1ULL << 32, low_size, CRASH_ALIGN);
+	low_base = memblock_find_in_range(CRASH_ALIGN, CRASH_ADDR_LOW_MAX, low_size, CRASH_ALIGN);
 	if (!low_base) {
 		pr_err("Cannot reserve %ldMB crashkernel low memory, please try smaller size.\n",
 		       (unsigned long)(low_size >> 20));
@@ -504,8 +504,9 @@ static void __init reserve_crashkernel(void)
 	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * crashkernel=x,high reserves memory over 4G, also allocates
-		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * crashkernel=x,high reserves memory over CRASH_ADDR_LOW_MAX,
+		 * also allocates 256M extra low memory for DMA buffers
+		 * and swiotlb.
 		 * But the extra memory is not required for all machines.
 		 * So try low memory first and fall back to high memory
 		 * unless "crashkernel=size[KMG],high" is specified.
@@ -539,7 +540,7 @@ static void __init reserve_crashkernel(void)
 		return;
 	}
 
-	if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
+	if (crash_base >= CRASH_ADDR_LOW_MAX && reserve_crashkernel_low()) {
 		memblock_free(crash_base, crash_size);
 		return;
 	}
Signature · 7 lines
2.20.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#6 lore raw

Re: [PATCH v12 3/9] x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]()

Quoted 13 lines · Chen Zhou

To make the functions reserve_crashkernel[_low]() as generic, replace some hard-coded numbers with macro CRASH_ADDR_LOW_MAX.

---
 arch/x86/kernel/setup.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
arch/x86/kernel/setup.c +0 -0
index d7fd90c52dae..71a6a6e7ca5b 100644

arch/x86/kernel/setup.c static int __init reserve_crashkernel_low(void) · line 433

Quoted 5 lines
arch/x86/kernel/setup.c +1 -1
@@ -430,7 +430,7 @@ static int __init reserve_crashkernel_low(void)
 	unsigned long total_low_mem;
 	int ret;
 
-	total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
+	total_low_mem = memblock_mem_size(CRASH_ADDR_LOW_MAX >> PAGE_SHIFT);

total_low_mem != CRASH_ADDR_LOW_MAX

arch/x86/kernel/setup.c static int __init reserve_crashkernel_low(void) · line 436

Quoted 3 lines
arch/x86/kernel/setup.c +0 -0
 	/* crashkernel=Y,low */
 	ret = parse_crashkernel_low(boot_command_line, total_low_mem, &low_size, &base);

The param total_low_mem is for dynamically change crash_size according to system ram size.

Is above change a must for your arm64 patches?

Quoted 33 lines
arch/x86/kernel/setup.c +5 -4
@@ -451,7 +451,7 @@ static int __init reserve_crashkernel_low(void)
 			return 0;
 	}
 
-	low_base = memblock_find_in_range(CRASH_ALIGN, 1ULL << 32, low_size, CRASH_ALIGN);
+	low_base = memblock_find_in_range(CRASH_ALIGN, CRASH_ADDR_LOW_MAX, low_size, CRASH_ALIGN);
 	if (!low_base) {
 		pr_err("Cannot reserve %ldMB crashkernel low memory, please try smaller size.\n",
 		       (unsigned long)(low_size >> 20));
@@ -504,8 +504,9 @@ static void __init reserve_crashkernel(void)
 	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * crashkernel=x,high reserves memory over 4G, also allocates
-		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * crashkernel=x,high reserves memory over CRASH_ADDR_LOW_MAX,
+		 * also allocates 256M extra low memory for DMA buffers
+		 * and swiotlb.
 		 * But the extra memory is not required for all machines.
 		 * So try low memory first and fall back to high memory
 		 * unless "crashkernel=size[KMG],high" is specified.
@@ -539,7 +540,7 @@ static void __init reserve_crashkernel(void)
 		return;
 	}
 
-	if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
+	if (crash_base >= CRASH_ADDR_LOW_MAX && reserve_crashkernel_low()) {
 		memblock_free(crash_base, crash_size);
 		return;
 	}
Signature · 1 line
2.20.1

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#7 lore raw

Re: [PATCH v12 3/9] x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]()

Hi Dave,

Quoted 17 lines · Dave Young
Quoted 16 lines · Chen Zhou

To make the functions reserve_crashkernel[_low]() as generic, replace some hard-coded numbers with macro CRASH_ADDR_LOW_MAX.

---
 arch/x86/kernel/setup.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
arch/x86/kernel/setup.c +0 -0
index d7fd90c52dae..71a6a6e7ca5b 100644
@@ -430,7 +430,7 @@ static int __init reserve_crashkernel_low(void)
 	unsigned long total_low_mem;
 	int ret;
 
Quoted 3 lines

arch/x86/kernel/setup.c static int __init reserve_crashkernel_low(void) · line 433

Quoted 2 lines
arch/x86/kernel/setup.c +1 -1
-	total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
+	total_low_mem = memblock_mem_size(CRASH_ADDR_LOW_MAX >> PAGE_SHIFT);

total_low_mem != CRASH_ADDR_LOW_MAX

I just replace the magic number with macro, no other change. Besides, function memblock_mem_size(limit_pfn) will compute the memory size according to the actual system ram.

Thanks,
Chen Zhou

Quoted 8 lines

arch/x86/kernel/setup.c static int __init reserve_crashkernel_low(void) · line 436

Quoted 4 lines
arch/x86/kernel/setup.c +0 -0
 	/* crashkernel=Y,low */
 	ret = parse_crashkernel_low(boot_command_line, total_low_mem, &low_size, &base);

The param total_low_mem is for dynamically change crash_size according to system ram size.

Is above change a must for your arm64 patches?

See above.

Quoted 26 lines
Quoted 26 lines
arch/x86/kernel/setup.c +4 -3
@@ -451,7 +451,7 @@ static int __init reserve_crashkernel_low(void)
 			return 0;
 	}
 
-	low_base = memblock_find_in_range(CRASH_ALIGN, 1ULL << 32, low_size, CRASH_ALIGN);
+	low_base = memblock_find_in_range(CRASH_ALIGN, CRASH_ADDR_LOW_MAX, low_size, CRASH_ALIGN);
 	if (!low_base) {
 		pr_err("Cannot reserve %ldMB crashkernel low memory, please try smaller size.\n",
 		       (unsigned long)(low_size >> 20));
@@ -504,8 +504,9 @@ static void __init reserve_crashkernel(void)
 	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * crashkernel=x,high reserves memory over 4G, also allocates
-		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * crashkernel=x,high reserves memory over CRASH_ADDR_LOW_MAX,
+		 * also allocates 256M extra low memory for DMA buffers
+		 * and swiotlb.
 		 * But the extra memory is not required for all machines.
 		 * So try low memory first and fall back to high memory
 		 * unless "crashkernel=size[KMG],high" is specified.
@@ -539,7 +540,7 @@ static void __init reserve_crashkernel(void)
 		return;
 	}
 
Quoted 10 lines
Quoted 8 lines
arch/x86/kernel/setup.c +1 -1
-	if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
+	if (crash_base >= CRASH_ADDR_LOW_MAX && reserve_crashkernel_low()) {
 		memblock_free(crash_base, crash_size);
 		return;
 	}
Signature · 1 line
2.20.1

.

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#8 lore raw

Re: [PATCH v12 3/9] x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]()

Quoted 21 lines · chenzhou

Hi Dave,

Quoted 17 lines · Dave Young
Quoted 16 lines · Chen Zhou

To make the functions reserve_crashkernel[_low]() as generic, replace some hard-coded numbers with macro CRASH_ADDR_LOW_MAX.

---
 arch/x86/kernel/setup.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
arch/x86/kernel/setup.c +0 -0
index d7fd90c52dae..71a6a6e7ca5b 100644
@@ -430,7 +430,7 @@ static int __init reserve_crashkernel_low(void)
 	unsigned long total_low_mem;
 	int ret;
 
Quoted 7 lines
Quoted 3 lines

arch/x86/kernel/setup.c static int __init reserve_crashkernel_low(void) · line 433

Quoted 2 lines
arch/x86/kernel/setup.c +1 -1
-	total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
+	total_low_mem = memblock_mem_size(CRASH_ADDR_LOW_MAX >> PAGE_SHIFT);

total_low_mem != CRASH_ADDR_LOW_MAX

I just replace the magic number with macro, no other change. Besides, function memblock_mem_size(limit_pfn) will compute the memory size according to the actual system ram.

Ok, it is not obvious in patch this is 64bit only, I'm fine with this then.

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#9 lore raw

Re: [PATCH v12 3/9] x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]()

In reply to Chen Zhou

Hi,

Quoted 13 lines · Chen Zhou

To make the functions reserve_crashkernel[_low]() as generic, replace some hard-coded numbers with macro CRASH_ADDR_LOW_MAX.

---
 arch/x86/kernel/setup.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
arch/x86/kernel/setup.c +0 -0
index d7fd90c52dae..71a6a6e7ca5b 100644

arch/x86/kernel/setup.c static int __init reserve_crashkernel_low(void) · line 433

Quoted 5 lines
arch/x86/kernel/setup.c +1 -1
@@ -430,7 +430,7 @@ static int __init reserve_crashkernel_low(void)
 	unsigned long total_low_mem;
 	int ret;
 
-	total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
+	total_low_mem = memblock_mem_size(CRASH_ADDR_LOW_MAX >> PAGE_SHIFT);

Just note that the replacement has been done in another patch from Mike Rapoport, partially. He seems to have done reserve_crashkernel_low() part, there's one left in reserve_crashkernel(), you might want to check that.

Mike's patch which is from a patchset has been merged into Andrew's next tree.

commit 6e50f7672ffa362e9bd4bc0c0d2524ed872828c5

Author: Mike Rapoport <[email protected]>
Date:   Wed Aug 26 15:22:32 2020 +1000

x86/setup: simplify reserve_crashkernel()

Quoted 36 lines
arch/x86/kernel/setup.c +5 -4
 	/* crashkernel=Y,low */
 	ret = parse_crashkernel_low(boot_command_line, total_low_mem, &low_size, &base);
@@ -451,7 +451,7 @@ static int __init reserve_crashkernel_low(void)
 			return 0;
 	}
 
-	low_base = memblock_find_in_range(CRASH_ALIGN, 1ULL << 32, low_size, CRASH_ALIGN);
+	low_base = memblock_find_in_range(CRASH_ALIGN, CRASH_ADDR_LOW_MAX, low_size, CRASH_ALIGN);
 	if (!low_base) {
 		pr_err("Cannot reserve %ldMB crashkernel low memory, please try smaller size.\n",
 		       (unsigned long)(low_size >> 20));
@@ -504,8 +504,9 @@ static void __init reserve_crashkernel(void)
 	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * crashkernel=x,high reserves memory over 4G, also allocates
-		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * crashkernel=x,high reserves memory over CRASH_ADDR_LOW_MAX,
+		 * also allocates 256M extra low memory for DMA buffers
+		 * and swiotlb.
 		 * But the extra memory is not required for all machines.
 		 * So try low memory first and fall back to high memory
 		 * unless "crashkernel=size[KMG],high" is specified.
@@ -539,7 +540,7 @@ static void __init reserve_crashkernel(void)
 		return;
 	}
 
-	if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
+	if (crash_base >= CRASH_ADDR_LOW_MAX && reserve_crashkernel_low()) {
 		memblock_free(crash_base, crash_size);
 		return;
 	}
Signature · 1 line
2.20.1

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#10 lore raw

Re: [PATCH v12 3/9] x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]()

Hi Baoquan,

Quoted 26 lines · Baoquan He

Hi,

Quoted 13 lines · Chen Zhou

To make the functions reserve_crashkernel[_low]() as generic, replace some hard-coded numbers with macro CRASH_ADDR_LOW_MAX.

---
 arch/x86/kernel/setup.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
arch/x86/kernel/setup.c +0 -0
index d7fd90c52dae..71a6a6e7ca5b 100644

arch/x86/kernel/setup.c static int __init reserve_crashkernel_low(void) · line 433

Quoted 5 lines
arch/x86/kernel/setup.c +1 -1
@@ -430,7 +430,7 @@ static int __init reserve_crashkernel_low(void)
 	unsigned long total_low_mem;
 	int ret;
 
-	total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
+	total_low_mem = memblock_mem_size(CRASH_ADDR_LOW_MAX >> PAGE_SHIFT);

Just note that the replacement has been done in another patch from Mike Rapoport, partially. He seems to have done reserve_crashkernel_low() part, there's one left in reserve_crashkernel(), you might want to check that.

Quoted 8 lines

Mike's patch which is from a patchset has been merged into Andrew's next tree.

commit 6e50f7672ffa362e9bd4bc0c0d2524ed872828c5

Author: Mike Rapoport <[email protected]>
Date:   Wed Aug 26 15:22:32 2020 +1000

x86/setup: simplify reserve_crashkernel()

Yeah, the function reserve_crashkernel() has been changed in the next tree. Thanks for your review and reminder.

Thanks,
Chen Zhou

Quoted 29 lines
Quoted 29 lines
arch/x86/kernel/setup.c +4 -3
 	/* crashkernel=Y,low */
 	ret = parse_crashkernel_low(boot_command_line, total_low_mem, &low_size, &base);
@@ -451,7 +451,7 @@ static int __init reserve_crashkernel_low(void)
 			return 0;
 	}
 
-	low_base = memblock_find_in_range(CRASH_ALIGN, 1ULL << 32, low_size, CRASH_ALIGN);
+	low_base = memblock_find_in_range(CRASH_ALIGN, CRASH_ADDR_LOW_MAX, low_size, CRASH_ALIGN);
 	if (!low_base) {
 		pr_err("Cannot reserve %ldMB crashkernel low memory, please try smaller size.\n",
 		       (unsigned long)(low_size >> 20));
@@ -504,8 +504,9 @@ static void __init reserve_crashkernel(void)
 	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * crashkernel=x,high reserves memory over 4G, also allocates
-		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * crashkernel=x,high reserves memory over CRASH_ADDR_LOW_MAX,
+		 * also allocates 256M extra low memory for DMA buffers
+		 * and swiotlb.
 		 * But the extra memory is not required for all machines.
 		 * So try low memory first and fall back to high memory
 		 * unless "crashkernel=size[KMG],high" is specified.
@@ -539,7 +540,7 @@ static void __init reserve_crashkernel(void)
 		return;
 	}
 
Quoted 10 lines
Quoted 8 lines
arch/x86/kernel/setup.c +1 -1
-	if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
+	if (crash_base >= CRASH_ADDR_LOW_MAX && reserve_crashkernel_low()) {
 		memblock_free(crash_base, crash_size);
 		return;
 	}
Signature · 1 line
2.20.1

.

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#11 lore raw

Re: [PATCH v12 3/9] x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]()

Hi Catalin,

Quoted 37 lines · chenzhou

Hi Baoquan,

Quoted 34 lines · Baoquan He

Hi,

Quoted 13 lines · Chen Zhou

To make the functions reserve_crashkernel[_low]() as generic, replace some hard-coded numbers with macro CRASH_ADDR_LOW_MAX.

---
 arch/x86/kernel/setup.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
arch/x86/kernel/setup.c +0 -0
index d7fd90c52dae..71a6a6e7ca5b 100644

arch/x86/kernel/setup.c static int __init reserve_crashkernel_low(void) · line 433

Quoted 5 lines
arch/x86/kernel/setup.c +1 -1
@@ -430,7 +430,7 @@ static int __init reserve_crashkernel_low(void)
 	unsigned long total_low_mem;
 	int ret;
 
-	total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
+	total_low_mem = memblock_mem_size(CRASH_ADDR_LOW_MAX >> PAGE_SHIFT);

Just note that the replacement has been done in another patch from Mike Rapoport, partially. He seems to have done reserve_crashkernel_low() part, there's one left in reserve_crashkernel(), you might want to check that.

Mike's patch which is from a patchset has been merged into Andrew's next tree.

commit 6e50f7672ffa362e9bd4bc0c0d2524ed872828c5

Author: Mike Rapoport <[email protected]>
Date:   Wed Aug 26 15:22:32 2020 +1000

x86/setup: simplify reserve_crashkernel()

As Baoquan said, some functions have been changed in the next tree, if i need to rebase on top of the next tree.

Thanks,
Chen Zhou

Quoted 43 lines

Yeah, the function reserve_crashkernel() has been changed in the next tree. Thanks for your review and reminder.

Thanks,
Chen Zhou

Quoted 38 lines
Quoted 36 lines
arch/x86/kernel/setup.c +5 -4
 	/* crashkernel=Y,low */
 	ret = parse_crashkernel_low(boot_command_line, total_low_mem, &low_size, &base);
@@ -451,7 +451,7 @@ static int __init reserve_crashkernel_low(void)
 			return 0;
 	}
 
-	low_base = memblock_find_in_range(CRASH_ALIGN, 1ULL << 32, low_size, CRASH_ALIGN);
+	low_base = memblock_find_in_range(CRASH_ALIGN, CRASH_ADDR_LOW_MAX, low_size, CRASH_ALIGN);
 	if (!low_base) {
 		pr_err("Cannot reserve %ldMB crashkernel low memory, please try smaller size.\n",
 		       (unsigned long)(low_size >> 20));
@@ -504,8 +504,9 @@ static void __init reserve_crashkernel(void)
 	if (!crash_base) {
 		/*
 		 * Set CRASH_ADDR_LOW_MAX upper bound for crash memory,
-		 * crashkernel=x,high reserves memory over 4G, also allocates
-		 * 256M extra low memory for DMA buffers and swiotlb.
+		 * crashkernel=x,high reserves memory over CRASH_ADDR_LOW_MAX,
+		 * also allocates 256M extra low memory for DMA buffers
+		 * and swiotlb.
 		 * But the extra memory is not required for all machines.
 		 * So try low memory first and fall back to high memory
 		 * unless "crashkernel=size[KMG],high" is specified.
@@ -539,7 +540,7 @@ static void __init reserve_crashkernel(void)
 		return;
 	}
 
-	if (crash_base >= (1ULL << 32) && reserve_crashkernel_low()) {
+	if (crash_base >= CRASH_ADDR_LOW_MAX && reserve_crashkernel_low()) {
 		memblock_free(crash_base, crash_size);
 		return;
 	}
Signature · 1 line
2.20.1

.

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

maintainer
#12 lore raw

Re: [PATCH v12 3/9] x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions reserve_crashkernel[_low]()

Quoted 29 lines · chenzhou
Quoted 28 lines · chenzhou
Quoted 27 lines · Baoquan He
Quoted 13 lines · Chen Zhou

To make the functions reserve_crashkernel[_low]() as generic, replace some hard-coded numbers with macro CRASH_ADDR_LOW_MAX.

---
 arch/x86/kernel/setup.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
arch/x86/kernel/setup.c +0 -0
index d7fd90c52dae..71a6a6e7ca5b 100644

arch/x86/kernel/setup.c static int __init reserve_crashkernel_low(void) · line 433

Quoted 5 lines
arch/x86/kernel/setup.c +1 -1
@@ -430,7 +430,7 @@ static int __init reserve_crashkernel_low(void)
 	unsigned long total_low_mem;
 	int ret;
 
-	total_low_mem = memblock_mem_size(1UL << (32 - PAGE_SHIFT));
+	total_low_mem = memblock_mem_size(CRASH_ADDR_LOW_MAX >> PAGE_SHIFT);

Just note that the replacement has been done in another patch from Mike Rapoport, partially. He seems to have done reserve_crashkernel_low() part, there's one left in reserve_crashkernel(), you might want to check that.

Mike's patch which is from a patchset has been merged into Andrew's next tree.

Quoted 7 lines
Quoted 5 lines
Quoted 5 lines

commit 6e50f7672ffa362e9bd4bc0c0d2524ed872828c5

Author: Mike Rapoport <[email protected]>
Date:   Wed Aug 26 15:22:32 2020 +1000

x86/setup: simplify reserve_crashkernel()

As Baoquan said, some functions have been changed in the next tree, if i need to rebase on top of the next tree.

Please rebase at 5.10-rc1 when the x86 change will probably be in and aim to queue this series for 5.11.

Thanks.

Signature · 6 lines
Catalin

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#13 lore raw

PATCH v12 4/9 x86: kdump: move reserve_crashkernel[_low]() into crash_core.c

In reply to Chen Zhou

Make the functions reserve_crashkernel[_low]() as generic. Arm64 will use these to reimplement crashkernel=X.

There is no functional change.

---
 arch/x86/include/asm/kexec.h |  25 +++++
 arch/x86/kernel/setup.c      | 163 +------------------------------
 include/linux/crash_core.h   |   4 +
 include/linux/kexec.h        |   2 -
 kernel/crash_core.c          | 179 +++++++++++++++++++++++++++++++++++
 kernel/kexec_core.c          |  17 ----
 6 files changed, 210 insertions(+), 180 deletions(-)
arch/x86/include/asm/kexec.h +25 -0
index 83f200dd54a1..adf5e9a016bd 100644
@@ -21,6 +21,27 @@
 /* 2M alignment for crash kernel regions */
 #define CRASH_ALIGN		SZ_2M
 
+/*
+ * Keep the crash kernel below this limit.
+ *
+ * Earlier 32-bits kernels would limit the kernel to the low 512 MB range
+ * due to mapping restrictions.
+ *
+ * 64-bit kdump kernels need to be restricted to be under 64 TB, which is
+ * the upper limit of system RAM in 4-level paging mode. Since the kdump
+ * jump could be from 5-level paging to 4-level paging, the jump will fail if
+ * the kernel is put above 64 TB, and during the 1st kernel bootup there's
+ * no good way to detect the paging mode of the target kernel which will be
+ * loaded for dumping.
+ */
+#ifdef CONFIG_X86_32
+# define CRASH_ADDR_LOW_MAX	SZ_512M
+# define CRASH_ADDR_HIGH_MAX	SZ_512M
+#else
+# define CRASH_ADDR_LOW_MAX	SZ_4G
+# define CRASH_ADDR_HIGH_MAX	SZ_64T
+#endif
+
 #ifndef __ASSEMBLY__
 
 #include <linux/string.h>
@@ -200,6 +221,10 @@ typedef void crash_vmclear_fn(void);
 extern crash_vmclear_fn __rcu *crash_vmclear_loaded_vmcss;
 extern void kdump_nmi_shootdown_cpus(void);
 
+#ifdef CONFIG_KEXEC_CORE
+extern void __init reserve_crashkernel(void);
+#endif
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* _ASM_X86_KEXEC_H */
arch/x86/kernel/setup.c +2 -161

181 lines not on this page · raw message

include/linux/crash_core.h +4 -0
index 6594dbc34a37..eabbb703d2a3 100644
@@ -66,6 +66,9 @@ extern unsigned char *vmcoreinfo_data;
 extern size_t vmcoreinfo_size;
 extern u32 *vmcoreinfo_note;
 
+extern struct resource crashk_res;
+extern struct resource crashk_low_res;
+
 /* raw contents of kernel .notes section */
 extern const void __start_notes __weak;
 extern const void __stop_notes __weak;
@@ -80,5 +83,6 @@ int parse_crashkernel_high(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base);
 int parse_crashkernel_low(char *cmdline, unsigned long long system_ram,
 		unsigned long long *crash_size, unsigned long long *crash_base);
+int __init reserve_crashkernel_low(void);
 
 #endif /* LINUX_CRASH_CORE_H */
include/linux/kexec.h +0 -2
index 9e93bef52968..f301f2f5cfc4 100644
@@ -337,8 +337,6 @@ extern int kexec_load_disabled;
 
 /* Location of a reserved region to hold the crash kernel.
  */
-extern struct resource crashk_res;
-extern struct resource crashk_low_res;
 extern note_buf_t __percpu *crash_notes;
 
 /* flag to track if kexec reboot is in progress */
kernel/crash_core.c +179 -0

198 lines not on this page · raw message

kernel/kexec_core.c +0 -17
index c19c0dad1ebe..db66bbabfff3 100644
@@ -53,23 +53,6 @@ note_buf_t __percpu *crash_notes;
 /* Flag to indicate we are going to kexec a new kernel */
 bool kexec_in_progress = false;
 
-
-/* Location of the reserved area for the crash kernel */
-struct resource crashk_res = {
-	.name  = "Crash kernel",
-	.start = 0,
-	.end   = 0,
-	.flags = IORESOURCE_BUSY | IORESOURCE_SYSTEM_RAM,
-	.desc  = IORES_DESC_CRASH_KERNEL
-};
-struct resource crashk_low_res = {
-	.name  = "Crash kernel",
-	.start = 0,
-	.end   = 0,
-	.flags = IORESOURCE_BUSY | IORESOURCE_SYSTEM_RAM,
-	.desc  = IORES_DESC_CRASH_KERNEL
-};
-
 int kexec_should_crash(struct task_struct *p)
 {
 	/*
Signature · 7 lines
2.20.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#14 lore raw

PATCH v12 5/9 arm64: kdump: introduce some macroes for crash kernel reservation

In reply to Chen Zhou

Introduce macro CRASH_ALIGN for alignment, macro CRASH_ADDR_LOW_MAX for upper bound of low crash memory, macro CRASH_ADDR_HIGH_MAX for upper bound of high crash memory, use macroes instead.

Besides, keep consistent with x86, use CRASH_ALIGN as the lower bound of crash kernel reservation.

---
 arch/arm64/include/asm/kexec.h     | 6 ++++++
 arch/arm64/include/asm/processor.h | 1 +
 arch/arm64/mm/init.c               | 8 ++++----
 3 files changed, 11 insertions(+), 4 deletions(-)
arch/arm64/include/asm/kexec.h +6 -0
index d24b527e8c00..402d208265a3 100644
@@ -25,6 +25,12 @@
 
 #define KEXEC_ARCH KEXEC_ARCH_AARCH64
 
+/* 2M alignment for crash kernel regions */
+#define CRASH_ALIGN	SZ_2M
+
+#define CRASH_ADDR_LOW_MAX	arm64_dma32_phys_limit
+#define CRASH_ADDR_HIGH_MAX	MEMBLOCK_ALLOC_ACCESSIBLE
+
 #ifndef __ASSEMBLY__
 
 /**
arch/arm64/include/asm/processor.h +1 -0
index 240fe5e5b720..af71063f352c 100644
@@ -95,6 +95,7 @@
 #endif /* CONFIG_ARM64_FORCE_52BIT */
 
 extern phys_addr_t arm64_dma_phys_limit;
+extern phys_addr_t arm64_dma32_phys_limit;
 #define ARCH_LOW_ADDRESS_LIMIT	(arm64_dma_phys_limit - 1)
 
 struct debug_info {
arch/arm64/mm/init.c +4 -4
index 481d22c32a2e..ad27dc4cc55e 100644
@@ -67,7 +67,7 @@ EXPORT_SYMBOL(vmemmap);
  * bit addressable memory area.
  */
 phys_addr_t arm64_dma_phys_limit __ro_after_init;
-static phys_addr_t arm64_dma32_phys_limit __ro_after_init;
+phys_addr_t arm64_dma32_phys_limit __ro_after_init;
 
 #ifdef CONFIG_KEXEC_CORE
 /*
@@ -92,8 +92,8 @@ static void __init reserve_crashkernel(void)
 
 	if (crash_base == 0) {
 		/* Current arm64 boot protocol requires 2MB alignment */
-		crash_base = memblock_find_in_range(0, arm64_dma32_phys_limit,
-				crash_size, SZ_2M);
+		crash_base = memblock_find_in_range(CRASH_ALIGN, CRASH_ADDR_LOW_MAX,
+				crash_size, CRASH_ALIGN);
 		if (crash_base == 0) {
 			pr_warn("cannot allocate crashkernel (size:0x%llx)\n",
 				crash_size);
@@ -111,7 +111,7 @@ static void __init reserve_crashkernel(void)
 			return;
 		}
 
-		if (!IS_ALIGNED(crash_base, SZ_2M)) {
+		if (!IS_ALIGNED(crash_base, CRASH_ALIGN)) {
 			pr_warn("cannot reserve crashkernel: base address is not 2MB aligned\n");
 			return;
 		}
Signature · 7 lines
2.20.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#15 lore raw

PATCH v12 6/9 arm64: kdump: reimplement crashkernel=X

In reply to Chen Zhou

There are following issues in arm64 kdump:
1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory.
2. If reserving crashkernel above 4G, in this case, crash dump kernel will boot failure because there is no low memory available for allocation.
3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"), if the memory reserved for crash dump kernel falled in ZONE_DMA32, the devices in crash dump kernel need to use ZONE_DMA will alloc fail.

To solve these issues, change the behavior of crashkernel=X and introduce crashkernel=X,[high,low].
crashkernel=X tries low allocation in DMA zone, and fall back to high allocation if it fails.
We can also use "crashkernel=X,high" to select a region above DMA zone, which also tries to allocate at least 256M in DMA zone automatically. "crashkernel=Y,low" can be used to allocate specified size low memory.

For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.

Another minor change, there may be two regions reserved for crash dump kernel, in order to distinct from the high region and make no effect to the use of existing kexec-tools, rename the low region as "Crash kernel (low)".

---
 arch/arm64/include/asm/kexec.h |  9 +++++
 arch/arm64/kernel/setup.c      | 13 +++++++-
 arch/arm64/mm/init.c           | 60 ++--------------------------------
 arch/arm64/mm/mmu.c            |  4 +++
 kernel/crash_core.c            |  8 +++--
 5 files changed, 34 insertions(+), 60 deletions(-)
arch/arm64/include/asm/kexec.h +9 -0
index 402d208265a3..79909ae5e22e 100644
@@ -28,7 +28,12 @@
 /* 2M alignment for crash kernel regions */
 #define CRASH_ALIGN	SZ_2M
 
+#ifdef CONFIG_ZONE_DMA
+#define CRASH_ADDR_LOW_MAX	arm64_dma_phys_limit
+#else
 #define CRASH_ADDR_LOW_MAX	arm64_dma32_phys_limit
+#endif
+
 #define CRASH_ADDR_HIGH_MAX	MEMBLOCK_ALLOC_ACCESSIBLE
 
 #ifndef __ASSEMBLY__
@@ -96,6 +101,10 @@ static inline void crash_prepare_suspend(void) {}
 static inline void crash_post_resume(void) {}
 #endif
 
+#ifdef CONFIG_KEXEC_CORE
+extern void __init reserve_crashkernel(void);
+#endif
+
 #ifdef CONFIG_KEXEC_FILE
 #define ARCH_HAS_KIMAGE_ARCH
 
arch/arm64/kernel/setup.c +12 -1
index 53acbeca4f57..1b24072f2bae 100644
@@ -238,7 +238,18 @@ static void __init request_standard_resources(void)
 		    kernel_data.end <= res->end)
 			request_resource(res, &kernel_data);
 #ifdef CONFIG_KEXEC_CORE
-		/* Userspace will find "Crash kernel" region in /proc/iomem. */
+		/*
+		 * Userspace will find "Crash kernel" or "Crash kernel (low)"
+		 * region in /proc/iomem.
+		 * In order to distinct from the high region and make no effect
+		 * to the use of existing kexec-tools, rename the low region as
+		 * "Crash kernel (low)".
+		 */
+		if (crashk_low_res.end && crashk_low_res.start >= res->start &&
+				crashk_low_res.end <= res->end) {
+			crashk_low_res.name = "Crash kernel (low)";
+			request_resource(res, &crashk_low_res);
+		}
 		if (crashk_res.end && crashk_res.start >= res->start &&
 		    crashk_res.end <= res->end)
 			request_resource(res, &crashk_res);
arch/arm64/mm/init.c +3 -57

75 lines not on this page · raw message

arch/arm64/mm/mmu.c +4 -0
index 75df62fea1b6..0b1b3ec18785 100644
@@ -476,6 +476,10 @@ static void __init map_mem(pgd_t *pgdp)
 	 */
 	memblock_mark_nomap(kernel_start, kernel_end - kernel_start);
 #ifdef CONFIG_KEXEC_CORE
+	if (crashk_low_res.end)
+		memblock_mark_nomap(crashk_low_res.start,
+				    resource_size(&crashk_low_res));
+
 	if (crashk_res.end)
 		memblock_mark_nomap(crashk_res.start,
 				    resource_size(&crashk_res));
kernel/crash_core.c +6 -2
index d968ef268fb4..3f735cb37ace 100644
@@ -321,7 +321,7 @@ int __init parse_crashkernel_low(char *cmdline,
 
 int __init reserve_crashkernel_low(void)
 {
-#if defined(CONFIG_X86_64)
+#if defined(CONFIG_X86_64) || defined(CONFIG_ARM64)
 	unsigned long long base, low_base = 0, low_size = 0;
 	unsigned long total_low_mem;
 	int ret;
@@ -370,12 +370,14 @@ int __init reserve_crashkernel_low(void)
 
 	crashk_low_res.start = low_base;
 	crashk_low_res.end   = low_base + low_size - 1;
+#ifdef CONFIG_X86
 	insert_resource(&iomem_resource, &crashk_low_res);
+#endif
 #endif
 	return 0;
 }
 
-#if defined(CONFIG_X86)
+#if defined(CONFIG_X86) || defined(CONFIG_ARM64)
 #ifdef CONFIG_KEXEC_CORE
 /*
  * reserve_crashkernel() - reserves memory for crash kernel
@@ -468,7 +470,9 @@ void __init reserve_crashkernel(void)
 
 	crashk_res.start = crash_base;
 	crashk_res.end   = crash_base + crash_size - 1;
+#ifdef CONFIG_X86
 	insert_resource(&iomem_resource, &crashk_res);
+#endif
 }
 #endif /* CONFIG_KEXEC_CORE */
 #endif
Signature · 7 lines
2.20.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
maintainer
#16 lore raw

Re: [PATCH v12 6/9] arm64: kdump: reimplement crashkernel=X

Quoted 11 lines · Chen Zhou
arch/arm64/kernel/setup.c +2 -1
index 53acbeca4f57..1b24072f2bae 100644
@@ -238,7 +238,18 @@ static void __init request_standard_resources(void)
 		    kernel_data.end <= res->end)
 			request_resource(res, &kernel_data);
 #ifdef CONFIG_KEXEC_CORE
-		/* Userspace will find "Crash kernel" region in /proc/iomem. */
+		/*
+		 * Userspace will find "Crash kernel" or "Crash kernel (low)"

arch/arm64/kernel/setup.c static void __init request_standard_resources(void) · line 252

Quoted 10 lines
arch/arm64/kernel/setup.c +10 -0
+		 * region in /proc/iomem.
+		 * In order to distinct from the high region and make no effect
+		 * to the use of existing kexec-tools, rename the low region as
+		 * "Crash kernel (low)".
+		 */
+		if (crashk_low_res.end && crashk_low_res.start >= res->start &&
+				crashk_low_res.end <= res->end) {
+			crashk_low_res.name = "Crash kernel (low)";
+			request_resource(res, &crashk_low_res);
+		}

With the changes in this series (including the above), how do the current kexec-tools behave? Do they pick just the high region and the loaded kernel will subsequently fail to boot?

Signature · 6 lines
Catalin

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#17 lore raw

Re: [PATCH v12 6/9] arm64: kdump: reimplement crashkernel=X

Quoted 17 lines · Catalin Marinas
Quoted 16 lines · Chen Zhou
arch/arm64/kernel/setup.c +7 -1
index 53acbeca4f57..1b24072f2bae 100644
@@ -238,7 +238,18 @@ static void __init request_standard_resources(void)
 		    kernel_data.end <= res->end)
 			request_resource(res, &kernel_data);
 #ifdef CONFIG_KEXEC_CORE
-		/* Userspace will find "Crash kernel" region in /proc/iomem. */
+		/*
+		 * Userspace will find "Crash kernel" or "Crash kernel (low)"
+		 * region in /proc/iomem.
+		 * In order to distinct from the high region and make no effect
+		 * to the use of existing kexec-tools, rename the low region as
+		 * "Crash kernel (low)".
+		 */
Quoted 8 lines

arch/arm64/kernel/setup.c static void __init request_standard_resources(void) · line 252

Quoted 5 lines
arch/arm64/kernel/setup.c +5 -0
+		if (crashk_low_res.end && crashk_low_res.start >= res->start &&
+				crashk_low_res.end <= res->end) {
+			crashk_low_res.name = "Crash kernel (low)";
+			request_resource(res, &crashk_low_res);
+		}

With the changes in this series (including the above), how do the current kexec-tools behave? Do they pick just the high region and the loaded kernel will subsequently fail to boot?

Yes,just pick the high region and will boot fail if low memory is needed.

Thanks,
Chen Zhou

Quoted 1 line

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#18 lore raw

PATCH v12 7/9 kdump: add threshold for the required memory

In reply to Chen Zhou

For crashkernel=X, if required size X is too large and leads to very little free low memory after low allocation, the system may not work normally.
So add a threshold and go for high allocation directly if the required size is too large. The value of threshold is set as the half of the low memory.

---
 kernel/crash_core.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)
kernel/crash_core.c +10 -1
index 3f735cb37ace..d11d597a470d 100644
@@ -378,6 +378,15 @@ int __init reserve_crashkernel_low(void)
 }
 
 #if defined(CONFIG_X86) || defined(CONFIG_ARM64)
+
+/*
+ * Add a threshold for required memory size of crashkernel. If required memory
+ * size is greater than threshold, just go for high allocation directly. The
+ * value of threshold is set as half of the total low memory.
+ */
+#define REQUIRED_MEMORY_THRESHOLD	(memblock_mem_size(CRASH_ADDR_LOW_MAX >> \
+			PAGE_SHIFT) >> 1)
+
 #ifdef CONFIG_KEXEC_CORE
 /*
  * reserve_crashkernel() - reserves memory for crash kernel
@@ -422,7 +431,7 @@ void __init reserve_crashkernel(void)
 		 * So try low memory first and fall back to high memory
 		 * unless "crashkernel=size[KMG],high" is specified.
 		 */
-		if (!high)
+		if (!high && crash_size <= REQUIRED_MEMORY_THRESHOLD)
 			crash_base = memblock_find_in_range(CRASH_ALIGN,
 						CRASH_ADDR_LOW_MAX,
 						crash_size, CRASH_ALIGN);
Signature · 7 lines
2.20.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
maintainer
#19 lore raw

Re: [PATCH v12 7/9] kdump: add threshold for the required memory

Quoted 21 lines · Chen Zhou
kernel/crash_core.c +9 -0
index 3f735cb37ace..d11d597a470d 100644
@@ -378,6 +378,15 @@ int __init reserve_crashkernel_low(void)
 }
 
 #if defined(CONFIG_X86) || defined(CONFIG_ARM64)
+
+/*
+ * Add a threshold for required memory size of crashkernel. If required memory
+ * size is greater than threshold, just go for high allocation directly. The
+ * value of threshold is set as half of the total low memory.
+ */
+#define REQUIRED_MEMORY_THRESHOLD	(memblock_mem_size(CRASH_ADDR_LOW_MAX >> \
+			PAGE_SHIFT) >> 1)
+
 #ifdef CONFIG_KEXEC_CORE
 /*
  * reserve_crashkernel() - reserves memory for crash kernel

kernel/crash_core.c void __init reserve_crashkernel(void) · line 437

Quoted 8 lines
kernel/crash_core.c +1 -1
@@ -422,7 +431,7 @@ void __init reserve_crashkernel(void)
 		 * So try low memory first and fall back to high memory
 		 * unless "crashkernel=size[KMG],high" is specified.
 		 */
-		if (!high)
+		if (!high && crash_size <= REQUIRED_MEMORY_THRESHOLD)
 			crash_base = memblock_find_in_range(CRASH_ALIGN,
 						CRASH_ADDR_LOW_MAX,
 						crash_size, CRASH_ALIGN);

Since any change now is affecting the x86 semantics slightly, I'd suggest you drop this patch. We can add it later if needed, once the core changes are in.

Thinking about this, if one requires a crashkernel reservation that allocates all of the ZONE_DMA, it would probably be noticed and explicit ,high/,low options can be used.

Note that we are also trying to make ZONE_DMA full 32-bit on non-RPi4 hardware.

Signature · 6 lines
Catalin

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#20 lore raw

Re: [PATCH v12 7/9] kdump: add threshold for the required memory

Quoted 25 lines · Catalin Marinas
Quoted 24 lines · Chen Zhou
kernel/crash_core.c +9 -0
index 3f735cb37ace..d11d597a470d 100644
@@ -378,6 +378,15 @@ int __init reserve_crashkernel_low(void)
 }
 
 #if defined(CONFIG_X86) || defined(CONFIG_ARM64)
+
+/*
+ * Add a threshold for required memory size of crashkernel. If required memory
+ * size is greater than threshold, just go for high allocation directly. The
+ * value of threshold is set as half of the total low memory.
+ */
+#define REQUIRED_MEMORY_THRESHOLD	(memblock_mem_size(CRASH_ADDR_LOW_MAX >> \
+			PAGE_SHIFT) >> 1)
+
 #ifdef CONFIG_KEXEC_CORE
 /*
  * reserve_crashkernel() - reserves memory for crash kernel
@@ -422,7 +431,7 @@ void __init reserve_crashkernel(void)
 		 * So try low memory first and fall back to high memory
 		 * unless "crashkernel=size[KMG],high" is specified.
 		 */
Quoted 8 lines

kernel/crash_core.c void __init reserve_crashkernel(void) · line 437

Quoted 5 lines
kernel/crash_core.c +1 -1
-		if (!high)
+		if (!high && crash_size <= REQUIRED_MEMORY_THRESHOLD)
 			crash_base = memblock_find_in_range(CRASH_ALIGN,
 						CRASH_ADDR_LOW_MAX,
 						crash_size, CRASH_ALIGN);

Since any change now is affecting the x86 semantics slightly, I'd suggest you drop this patch. We can add it later if needed, once the core changes are in.

Ok, i will drop this patch in next version.

Thanks,
Chen Zhou

Quoted 8 lines

Thinking about this, if one requires a crashkernel reservation that allocates all of the ZONE_DMA, it would probably be noticed and explicit ,high/,low options can be used.

Note that we are also trying to make ZONE_DMA full 32-bit on non-RPi4 hardware.

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#21 lore raw

PATCH v12 8/9 arm64: kdump: add memory for devices by DT property linux,usable-memory-range

In reply to Chen Zhou

When reserving crashkernel in high memory, some low memory is reserved for crash dump kernel devices and never mapped by the first kernel. This memory range is advertised to crash dump kernel via DT property under /chosen,
linux,usable-memory-range = <BASE1 SIZE1 [BASE2 SIZE2]>

We reused the DT property linux,usable-memory-range and made the low memory region as the second range "BASE2 SIZE2", which keeps compatibility with existing user-space and older kdump kernels.

Crash dump kernel reads this property at boot time and call memblock_add() to add the low memory region after memblock_cap_memory_range() has been called.

---
 arch/arm64/mm/init.c | 43 +++++++++++++++++++++++++++++++++----------
 1 file changed, 33 insertions(+), 10 deletions(-)
arch/arm64/mm/init.c +33 -10
index e56a0e5d5b77..2af8c38279d9 100644
@@ -76,6 +76,15 @@ static void __init reserve_crashkernel(void)
 }
 #endif
 
+/*
+ * The main usage of linux,usable-memory-range is for crash dump kernel.
+ * Originally, the number of usable-memory regions is one. Now there may
+ * be two regions, low region and high region.
+ * To make compatibility with existing user-space and older kdump, the low
+ * region is always the last range of linux,usable-memory-range if exist.
+ */
+#define MAX_USABLE_RANGES	2
+
 #ifdef CONFIG_CRASH_DUMP
 static int __init early_init_dt_scan_elfcorehdr(unsigned long node,
 		const char *uname, int depth, void *data)
@@ -191,9 +200,9 @@ early_param("mem", early_mem);
 static int __init early_init_dt_scan_usablemem(unsigned long node,
 		const char *uname, int depth, void *data)
 {
-	struct memblock_region *usablemem = data;
-	const __be32 *reg;
-	int len;
+	struct memblock_region *usable_rgns = data;
+	const __be32 *reg, *endp;
+	int len, nr = 0;
 
 	if (depth != 1 || strcmp(uname, "chosen") != 0)
 		return 0;
@@ -202,22 +211,36 @@ static int __init early_init_dt_scan_usablemem(unsigned long node,
 	if (!reg || (len < (dt_root_addr_cells + dt_root_size_cells)))
 		return 1;
 
-	usablemem->base = dt_mem_next_cell(dt_root_addr_cells, &reg);
-	usablemem->size = dt_mem_next_cell(dt_root_size_cells, &reg);
+	endp = reg + (len / sizeof(__be32));
+	while ((endp - reg) >= (dt_root_addr_cells + dt_root_size_cells)) {
+		usable_rgns[nr].base = dt_mem_next_cell(dt_root_addr_cells, &reg);
+		usable_rgns[nr].size = dt_mem_next_cell(dt_root_size_cells, &reg);
+
+		if (++nr >= MAX_USABLE_RANGES)
+			break;
+	}
 
 	return 1;
 }
 
 static void __init fdt_enforce_memory_region(void)
 {
-	struct memblock_region reg = {
-		.size = 0,
+	struct memblock_region usable_rgns[MAX_USABLE_RANGES] = {
+		{ .size = 0 },
+		{ .size = 0 }
 	};
 
-	of_scan_flat_dt(early_init_dt_scan_usablemem, &reg);
+	of_scan_flat_dt(early_init_dt_scan_usablemem, &usable_rgns);
 
-	if (reg.size)
-		memblock_cap_memory_range(reg.base, reg.size);
+	/*
+	 * The first range of usable-memory regions is for crash dump
+	 * kernel with only one region or for high region with two regions,
+	 * the second range is dedicated for low region if exist.
+	 */
+	if (usable_rgns[0].size)
+		memblock_cap_memory_range(usable_rgns[0].base, usable_rgns[0].size);
+	if (usable_rgns[1].size)
+		memblock_add(usable_rgns[1].base, usable_rgns[1].size);
 }
 
 void __init arm64_memblock_init(void)
Signature · 7 lines
2.20.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#22 lore raw

PATCH v12 9/9 kdump: update Documentation about crashkernel

In reply to Chen Zhou

For arm64, the behavior of crashkernel=X has been changed, which tries low allocation in DMA zone, and fall back to high allocation if it fails.
We can also use "crashkernel=X,high" to select a high region above DMA zone, which also tries to allocate at least 256M low memory in DMA zone automatically.
"crashkernel=Y,low" can be used to allocate specified size low memory in DMA zone.
For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.

For x86 and arm64, we introduce threshold for the required memory. if required size X is too large and leads to very little free low memory after low allocation, the system may not work well. So add a threshold and go for high allocation directly if the required size is too large. The threshold is set as the half of low memory.

So update the Documentation.

---
 Documentation/admin-guide/kdump/kdump.rst     | 25 ++++++++++++++++---
 .../admin-guide/kernel-parameters.txt         | 13 ++++++++--
 2 files changed, 33 insertions(+), 5 deletions(-)
Documentation/admin-guide/kdump/kdump.rst +22 -3
index 2da65fef2a1c..549611abc581 100644
@@ -299,7 +299,16 @@ Boot into System Kernel
    "crashkernel=64M@16M" tells the system kernel to reserve 64 MB of memory
    starting at physical address 0x01000000 (16MB) for the dump-capture kernel.
 
-   On x86 and x86_64, use "crashkernel=64M@16M".
+   On x86 use "crashkernel=64M@16M".
+
+   On x86_64, use "crashkernel=X" to select a region under 4G first, and
+   fall back to reserve region above 4G. And go for high allocation
+   directly if the required size is too large.
+   We can also use "crashkernel=X,high" to select a region above 4G, which
+   also tries to allocate at least 256M below 4G automatically and
+   "crashkernel=Y,low" can be used to allocate specified size low memory.
+   Use "crashkernel=Y@X" if you really have to reserve memory from specified
+   start address X.
 
    On ppc64, use "crashkernel=128M@32M".
 
@@ -316,8 +325,18 @@ Boot into System Kernel
    kernel will automatically locate the crash kernel image within the
    first 512MB of RAM if X is not given.
 
-   On arm64, use "crashkernel=Y[@X]".  Note that the start address of
-   the kernel, X if explicitly specified, must be aligned to 2MiB (0x200000).
+   On arm64, use "crashkernel=X" to try low allocation in DMA zone, and
+   fall back to high allocation if it fails. And go for high allocation
+   directly if the required size is too large.
+   We can also use "crashkernel=X,high" to select a high region above
+   DMA zone, which also tries to allocate at least 256M low memory in
+   DMA zone automatically.
+   "crashkernel=Y,low" can be used to allocate specified size low memory
+   in DMA zone.
+   For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.
+   Use "crashkernel=Y@X" if you really have to reserve memory from
+   specified start address X. Note that the start address of the kernel,
+   X if explicitly specified, must be aligned to 2MiB (0x200000).
 
 Load the Dump-capture Kernel
 ============================
Documentation/admin-guide/kernel-parameters.txt +11 -2
index a1068742a6df..f7df572d8f64 100644
@@ -727,6 +727,10 @@
 			[KNL, X86-64] Select a region under 4G first, and
 			fall back to reserve region above 4G when '@offset'
 			hasn't been specified.
+			[KNL, arm64] Try low allocation in DMA zone, fall back
+			to high allocation if it fails when '@offset' hasn't been
+			specified. For non-RPi4 platforms, change DMA zone to
+			DMA32 zone.
 			See Documentation/admin-guide/kdump/kdump.rst for further details.
 
 	crashkernel=range1:size1[,range2:size2,...][@offset]
@@ -743,6 +747,8 @@
 			Otherwise memory region will be allocated below 4G, if
 			available.
 			It will be ignored if crashkernel=X is specified.
+			[KNL, arm64] range in high memory.
+			Allow kernel to allocate physical memory region from top.
 	crashkernel=size[KMG],low
 			[KNL, X86-64] range under 4G. When crashkernel=X,high
 			is passed, kernel could allocate physical memory region
@@ -751,13 +757,16 @@
 			requires at least 64M+32K low memory, also enough extra
 			low memory is needed to make sure DMA buffers for 32-bit
 			devices won't run out. Kernel would try to allocate at
-			at least 256M below 4G automatically.
+			least 256M below 4G automatically.
 			This one let user to specify own low range under 4G
 			for second kernel instead.
 			0: to disable low allocation.
 			It will be ignored when crashkernel=X,high is not used
 			or memory reserved is below 4G.
-
+			[KNL, arm64] range in low memory.
+			This one let user to specify a low range in DMA zone for
+			crash dump kernel. For non-RPi4 platforms, change DMA zone
+			to DMA32 zone.
 	cryptomgr.notests
 			[KNL] Disable crypto self-tests
 
Signature · 7 lines
2.20.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
maintainer
#23 lore raw

Re: [PATCH v12 9/9] kdump: update Documentation about crashkernel

Quoted 4 lines · Chen Zhou
Documentation/admin-guide/kdump/kdump.rst +0 -0
index 2da65fef2a1c..549611abc581 100644

[...]

Quoted 5 lines
Documentation/admin-guide/kdump/kdump.rst +0 -1
@@ -316,8 +325,18 @@ Boot into System Kernel
    kernel will automatically locate the crash kernel image within the
    first 512MB of RAM if X is not given.
 
-   On arm64, use "crashkernel=Y[@X]".  Note that the start address of

Documentation/admin-guide/kdump/kdump.rst Boot into System Kernel · line 336

Quoted 10 lines
Documentation/admin-guide/kdump/kdump.rst +9 -1
-   the kernel, X if explicitly specified, must be aligned to 2MiB (0x200000).
+   On arm64, use "crashkernel=X" to try low allocation in DMA zone, and
+   fall back to high allocation if it fails. And go for high allocation
+   directly if the required size is too large.
+   We can also use "crashkernel=X,high" to select a high region above
+   DMA zone, which also tries to allocate at least 256M low memory in
+   DMA zone automatically.
+   "crashkernel=Y,low" can be used to allocate specified size low memory
+   in DMA zone.
+   For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.

I don't think we should mention non-RPi4 explicitly here. I don't even understand what the suggestion is since the only way is to disable ZONE_DMA in the kernel config. I'd just stick to ZONE_DMA description here.

Quoted 11 lines
Documentation/admin-guide/kdump/kdump.rst +3 -0
+   Use "crashkernel=Y@X" if you really have to reserve memory from
+   specified start address X. Note that the start address of the kernel,
+   X if explicitly specified, must be aligned to 2MiB (0x200000).
 
 Load the Dump-capture Kernel
 ============================
Documentation/admin-guide/kernel-parameters.txt +0 -0
index a1068742a6df..f7df572d8f64 100644

Documentation/admin-guide/kernel-parameters.txt line 733

Quoted 7 lines
Documentation/admin-guide/kernel-parameters.txt +4 -0
@@ -727,6 +727,10 @@
 			[KNL, X86-64] Select a region under 4G first, and
 			fall back to reserve region above 4G when '@offset'
 			hasn't been specified.
+			[KNL, arm64] Try low allocation in DMA zone, fall back
+			to high allocation if it fails when '@offset' hasn't been
+			specified. For non-RPi4 platforms, change DMA zone to
+			DMA32 zone.

Same here, unclear what "change DMA zone to DMA32 zone" means.

Quoted 18 lines
Documentation/admin-guide/kernel-parameters.txt +3 -1
 			See Documentation/admin-guide/kdump/kdump.rst for further details.
 
 	crashkernel=range1:size1[,range2:size2,...][@offset]
@@ -743,6 +747,8 @@
 			Otherwise memory region will be allocated below 4G, if
 			available.
 			It will be ignored if crashkernel=X is specified.
+			[KNL, arm64] range in high memory.
+			Allow kernel to allocate physical memory region from top.
 	crashkernel=size[KMG],low
 			[KNL, X86-64] range under 4G. When crashkernel=X,high
 			is passed, kernel could allocate physical memory region
@@ -751,13 +757,16 @@
 			requires at least 64M+32K low memory, also enough extra
 			low memory is needed to make sure DMA buffers for 32-bit
 			devices won't run out. Kernel would try to allocate at
-			at least 256M below 4G automatically.
+			least 256M below 4G automatically.

Documentation/admin-guide/kernel-parameters.txt line 769

Quoted 10 lines
Documentation/admin-guide/kernel-parameters.txt +4 -1
 			This one let user to specify own low range under 4G
 			for second kernel instead.
 			0: to disable low allocation.
 			It will be ignored when crashkernel=X,high is not used
 			or memory reserved is below 4G.
-
+			[KNL, arm64] range in low memory.
+			This one let user to specify a low range in DMA zone for
+			crash dump kernel. For non-RPi4 platforms, change DMA zone
+			to DMA32 zone.

And again here.

Signature · 6 lines
Catalin

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#24 lore raw

Re: [PATCH v12 9/9] kdump: update Documentation about crashkernel

Hi Catalin,

Quoted 10 lines · Catalin Marinas
Quoted 4 lines · Chen Zhou
Documentation/admin-guide/kdump/kdump.rst +0 -0
index 2da65fef2a1c..549611abc581 100644

[...]

Quoted 4 lines
Documentation/admin-guide/kdump/kdump.rst +0 -0
@@ -316,8 +325,18 @@ Boot into System Kernel
    kernel will automatically locate the crash kernel image within the
    first 512MB of RAM if X is not given.
 
Quoted 15 lines

Documentation/admin-guide/kdump/kdump.rst Boot into System Kernel · line 336

Quoted 11 lines
Documentation/admin-guide/kdump/kdump.rst +9 -2
-   On arm64, use "crashkernel=Y[@X]".  Note that the start address of
-   the kernel, X if explicitly specified, must be aligned to 2MiB (0x200000).
+   On arm64, use "crashkernel=X" to try low allocation in DMA zone, and
+   fall back to high allocation if it fails. And go for high allocation
+   directly if the required size is too large.
+   We can also use "crashkernel=X,high" to select a high region above
+   DMA zone, which also tries to allocate at least 256M low memory in
+   DMA zone automatically.
+   "crashkernel=Y,low" can be used to allocate specified size low memory
+   in DMA zone.
+   For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.

I don't think we should mention non-RPi4 explicitly here. I don't even understand what the suggestion is since the only way is to disable ZONE_DMA in the kernel config. I'd just stick to ZONE_DMA description here.

How about like this:
If the kernel config ZONE_DMA is disabled, just try low allocation in DMA32 zone and high allocation above DMA32 zone.

Thanks,
Chen Zhou

Quoted 42 lines
Quoted 12 lines
Documentation/admin-guide/kdump/kdump.rst +3 -0
+   Use "crashkernel=Y@X" if you really have to reserve memory from
+   specified start address X. Note that the start address of the kernel,
+   X if explicitly specified, must be aligned to 2MiB (0x200000).
 
 Load the Dump-capture Kernel
 ============================
Documentation/admin-guide/kernel-parameters.txt +0 -0
index a1068742a6df..f7df572d8f64 100644

Documentation/admin-guide/kernel-parameters.txt line 733

Quoted 7 lines
Documentation/admin-guide/kernel-parameters.txt +4 -0
@@ -727,6 +727,10 @@
 			[KNL, X86-64] Select a region under 4G first, and
 			fall back to reserve region above 4G when '@offset'
 			hasn't been specified.
+			[KNL, arm64] Try low allocation in DMA zone, fall back
+			to high allocation if it fails when '@offset' hasn't been
+			specified. For non-RPi4 platforms, change DMA zone to
+			DMA32 zone.

Same here, unclear what "change DMA zone to DMA32 zone" means.

Quoted 21 lines
Documentation/admin-guide/kernel-parameters.txt +3 -1
 			See Documentation/admin-guide/kdump/kdump.rst for further details.
 
 	crashkernel=range1:size1[,range2:size2,...][@offset]
@@ -743,6 +747,8 @@
 			Otherwise memory region will be allocated below 4G, if
 			available.
 			It will be ignored if crashkernel=X is specified.
+			[KNL, arm64] range in high memory.
+			Allow kernel to allocate physical memory region from top.
 	crashkernel=size[KMG],low
 			[KNL, X86-64] range under 4G. When crashkernel=X,high
 			is passed, kernel could allocate physical memory region
@@ -751,13 +757,16 @@
 			requires at least 64M+32K low memory, also enough extra
 			low memory is needed to make sure DMA buffers for 32-bit
 			devices won't run out. Kernel would try to allocate at
-			at least 256M below 4G automatically.
+			least 256M below 4G automatically.
 			This one let user to specify own low range under 4G
 			for second kernel instead.
 			0: to disable low allocation.
Quoted 9 lines

Documentation/admin-guide/kernel-parameters.txt line 769

Quoted 7 lines
Documentation/admin-guide/kernel-parameters.txt +4 -1
 			It will be ignored when crashkernel=X,high is not used
 			or memory reserved is below 4G.
-
+			[KNL, arm64] range in low memory.
+			This one let user to specify a low range in DMA zone for
+			crash dump kernel. For non-RPi4 platforms, change DMA zone
+			to DMA32 zone.

And again here.

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

maintainer
#25 lore raw

Re: [PATCH v12 9/9] kdump: update Documentation about crashkernel

Quoted 26 lines · chenzhou
Quoted 10 lines · Catalin Marinas
Quoted 4 lines · Chen Zhou
Documentation/admin-guide/kdump/kdump.rst +0 -0
index 2da65fef2a1c..549611abc581 100644

[...]

Quoted 4 lines
Documentation/admin-guide/kdump/kdump.rst +0 -0
@@ -316,8 +325,18 @@ Boot into System Kernel
    kernel will automatically locate the crash kernel image within the
    first 512MB of RAM if X is not given.
 
Quoted 15 lines

Documentation/admin-guide/kdump/kdump.rst Boot into System Kernel · line 336

Quoted 11 lines
Documentation/admin-guide/kdump/kdump.rst +9 -2
-   On arm64, use "crashkernel=Y[@X]".  Note that the start address of
-   the kernel, X if explicitly specified, must be aligned to 2MiB (0x200000).
+   On arm64, use "crashkernel=X" to try low allocation in DMA zone, and
+   fall back to high allocation if it fails. And go for high allocation
+   directly if the required size is too large.
+   We can also use "crashkernel=X,high" to select a high region above
+   DMA zone, which also tries to allocate at least 256M low memory in
+   DMA zone automatically.
+   "crashkernel=Y,low" can be used to allocate specified size low memory
+   in DMA zone.
+   For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.

I don't think we should mention non-RPi4 explicitly here. I don't even understand what the suggestion is since the only way is to disable ZONE_DMA in the kernel config. I'd just stick to ZONE_DMA description here.

Quoted 3 lines

How about like this:
If the kernel config ZONE_DMA is disabled, just try low allocation in DMA32 zone and high allocation above DMA32 zone.

Something like: "allocate 256M low memory in the DMA zone automatically (or the DMA32 zone if CONFIG_ZONE_DMA is disabled)".

I'd keep it short.

Signature · 6 lines
Catalin

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Tested-by on the v12 series
#26 lore raw
In reply to Chen Zhou
Quoted 152 lines · Chen Zhou

There are following issues in arm64 kdump:
1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory.
2. If reserving crashkernel above 4G, in this case, crash dump kernel will boot failure because there is no low memory available for allocation.
3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"), if the memory reserved for crash dump kernel falled in ZONE_DMA32, the devices in crash dump kernel need to use ZONE_DMA will alloc fail.

To solve these issues, change the behavior of crashkernel=X. crashkernel=X tries low allocation in DMA zone, and fall back to high allocation if it fails.
If requized size X is too large and leads to very little low memory in DMA zone after low allocation, the system may not work normally. So add a threshold and go for high allocation directly if the required size is too large. The value of threshold is set as the half of the low memory.

We can also use "crashkernel=X,high" to select a high region above DMA zone, which also tries to allocate at least 256M low memory in DMA zone automatically.
"crashkernel=Y,low" can be used to allocate specified size low memory. For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.

When reserving crashkernel in high memory, some low memory is reserved for crash dump kernel devices. So there may be two regions reserved for crash dump kernel.
In order to distinct from the high region and make no effect to the use of existing kexec-tools, rename the low region as "Crash kernel (low)", and pass the low region by reusing DT property "linux,usable-memory-range". We made the low memory region as the last range of "linux,usable-memory-range" to keep compatibility with existing user-space and older kdump kernels.

Besides, we need to modify kexec-tools:
arm64: support more than one crash kernel regions(see [1])

Another update is document about DT property 'linux,usable-memory-range':
schemas: update 'linux,usable-memory-range' node schema(see [2])

This patchset contains the following nine patches:
0001-x86-kdump-move-CRASH_ALIGN-to-2M.patch 0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch 0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch 0004-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch 0005-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch 0006-arm64-kdump-reimplement-crashkernel-X.patch 0007-kdump-add-threshold-for-the-required-memory.patch 0008-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch 0009-kdump-update-Documentation-about-crashkernel.patch

0001-0003 are some x86 cleanups which prepares for making functionsreserve_crashkernel[_low]() generic.

0004 makes functions reserve_crashkernel[_low]() generic. 0005-0006 reimplements crashkernel=X.
0007 adds threshold for the required memory.
0008 adds memory for devices by DT property linux,usable-memory-range. 0009 updates the doc.

Changes since [v11]
- Rebased on top of 5.9-rc4.
- Make the function reserve_crashkernel() of x86 generic. Suggested by Catalin, make the function reserve_crashkernel() of x86 generic and arm64 use the generic version to reimplement crashkernel=X.

Changes since [v10]
- Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin.

Changes since [v9]
- Patch 1 add Acked-by from Dave.
- Update patch 5 according to Dave's comments.
- Update chosen schema.

Changes since [v8]
- Reuse DT property "linux,usable-memory-range".
Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the low memory region.
- Fix kdump broken with ZONE_DMA reintroduced.
- Update chosen schema.

Changes since [v7]
- Move x86 CRASH_ALIGN to 2M
Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
- Update Documentation/devicetree/bindings/chosen.txt.
Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt suggested by Arnd.
- Add Tested-by from Jhon and pk.

Changes since [v6]
- Fix build errors reported by kbuild test robot.

Changes since [v5]
- Move reserve_crashkernel_low() into kernel/crash_core.c.
- Delete crashkernel=X,high.
- Modify crashkernel=X,low.
If crashkernel=X,low is specified simultaneously, reserve spcified size low memory for crash kdump kernel devices firstly and then reserve memory above 4G. In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then pass to crash dump kernel by DT property "linux,low-memory-range".
- Update Documentation/admin-guide/kdump/kdump.rst.

Changes since [v4]
- Reimplement memblock_cap_memory_ranges for multiple ranges by Mike.

Changes since [v3]
- Add memblock_cap_memory_ranges back for multiple ranges.
- Fix some compiling warnings.

Changes since [v2]
- Split patch "arm64: kdump: support reserving crashkernel above 4G" as two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate patch.

Changes since [v1]:
- Move common reserve_crashkernel_low() code into kernel/kexec_core.c.
- Remove memblock_cap_memory_ranges() i added in v1 and implement that in fdt_enforce_memory_region().
There are at most two crash kernel regions, for two crash kernel regions case, we cap the memory range [min(regs[*].start), max(regs[*].end)] and then remove the memory range in the middle.

[1]: https://urldefense.com/v3/__http://lists.infradead.org/pipermail/kexec/2020-June/020737.html__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6BsfMsIet$
[2]: https://urldefense.com/v3/__https://github.com/robherring/dt-schema/pull/19__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6Bv1JxB2D$
[v1]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/2/1174__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6BgTzrgKq$
[v2]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/86__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6Btz3iM8F$
[v3]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/306__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6BuqcVDab$
[v4]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/15/273__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6Bgdlc1Y7$
[v5]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/5/6/1360__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6BsuuZ6C_$
[v6]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/8/30/142__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6Bo4IxHqi$
[v7]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/12/23/411__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6BjlqN_6I$
[v8]: https://urldefense.com/v3/__https://lkml.org/lkml/2020/5/21/213__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6BlBSztwY$
[v9]: https://urldefense.com/v3/__https://lkml.org/lkml/2020/6/28/73__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6BoNFCNt9$
[v10]: https://urldefense.com/v3/__https://lkml.org/lkml/2020/7/2/1443__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6BvfD2Ihf$
[v11]: https://urldefense.com/v3/__https://lkml.org/lkml/2020/8/1/150__;!!GqivPVa7Brio!IzjRTihkWj0uY8lqf60OD7rbqIAhyGD20C4EZpBaPsNfWxuPgeU1Av-fzig6BohKxmce$

Chen Zhou (9):
x86: kdump: move CRASH_ALIGN to 2M
x86: kdump: make the lower bound of crash kernel reservation
consistent
x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions
reserve_crashkernel[_low]()
x86: kdump: move reserve_crashkernel[_low]() into crash_core.c
arm64: kdump: introduce some macroes for crash kernel reservation
arm64: kdump: reimplement crashkernel=X
kdump: add threshold for the required memory
arm64: kdump: add memory for devices by DT property
linux,usable-memory-range
kdump: update Documentation about crashkernel

Quoted 15 lines
  Documentation/admin-guide/kdump/kdump.rst     |  25 ++-
  .../admin-guide/kernel-parameters.txt         |  13 +-
  arch/arm64/include/asm/kexec.h                |  15 ++
  arch/arm64/include/asm/processor.h            |   1 +
  arch/arm64/kernel/setup.c                     |  13 +-
  arch/arm64/mm/init.c                          | 105 ++++------
  arch/arm64/mm/mmu.c                           |   4 +
  arch/x86/include/asm/kexec.h                  |  28 +++
  arch/x86/kernel/setup.c                       | 165 +--------------
  include/linux/crash_core.h                    |   4 +
  include/linux/kexec.h                         |   2 -
  kernel/crash_core.c                           | 192 ++++++++++++++++++
  kernel/kexec_core.c                           |  17 --
  13 files changed, 328 insertions(+), 256 deletions(-)

I did a brief unit-test on 5.9-rc4.

Please add:

This activity is over a year old. It needs accepted.

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

maintainer
#27 lore raw
Quoted 14 lines · John Donnelly
Quoted 13 lines · Chen Zhou

Chen Zhou (9):
x86: kdump: move CRASH_ALIGN to 2M
x86: kdump: make the lower bound of crash kernel reservation
consistent
x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions
reserve_crashkernel[_low]()
x86: kdump: move reserve_crashkernel[_low]() into crash_core.c
arm64: kdump: introduce some macroes for crash kernel reservation
arm64: kdump: reimplement crashkernel=X
kdump: add threshold for the required memory
arm64: kdump: add memory for devices by DT property
linux,usable-memory-range
kdump: update Documentation about crashkernel

[...]

Quoted 5 lines

I did a brief unit-test on 5.9-rc4.

Please add:

Thanks for testing.

Quoted 1 line

This activity is over a year old. It needs accepted.

It's getting there, hopefully in 5.11. There are some minor tweaks to address.

Signature · 6 lines
Catalin

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#28 lore raw

Hi Catalin, Chen,

Quoted 21 lines · Catalin Marinas
Quoted 14 lines · John Donnelly
Quoted 13 lines · Chen Zhou

Chen Zhou (9):
x86: kdump: move CRASH_ALIGN to 2M
x86: kdump: make the lower bound of crash kernel reservation
consistent
x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions
reserve_crashkernel[_low]()
x86: kdump: move reserve_crashkernel[_low]() into crash_core.c
arm64: kdump: introduce some macroes for crash kernel reservation
arm64: kdump: reimplement crashkernel=X
kdump: add threshold for the required memory
arm64: kdump: add memory for devices by DT property
linux,usable-memory-range
kdump: update Documentation about crashkernel

[...]

Quoted 4 lines

I did a brief unit-test on 5.9-rc4.

Please add:

Quoted 8 lines
Quoted 1 line

Thanks for testing.

Quoted 1 line

This activity is over a year old. It needs accepted.

It's getting there, hopefully in 5.11. There are some minor tweaks to address.

I think my earlier email with the test results on this series bounced off the mailing list server (for some weird reason), but I still see several issues with this patchset. I will add specific issues in the review comments for each patch again, but overall, with a crashkernel size of say 786M, I see the following issue:

# cat /proc/cmdline
BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..>
rd.lvm.lv=<..snip..> crashkernel=786M

I see two regions of size 786M and 256M reserved in low and high regions respectively, So we reserve a total of 1042M of memory, which is an incorrect behaviour:

# dmesg | grep -i crash
[    0.000000] Reserving 256MB of low memory at 2816MB for crashkernel
(System low RAM: 768MB)
[    0.000000] Reserving 786MB of memory at 654158MB for crashkernel
(System RAM: 130816MB)
[    0.000000] Kernel command line:
BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+
root=/dev/mapper/rhel_ampere--hr330a--03-root ro
rd.lvm.lv=rhel_ampere-hr330a-03/root
rd.lvm.lv=rhel_ampere-hr330a-03/swap crashkernel=786M cma=1024M
# cat /proc/iomem | grep -i crash
  b0000000-bfffffff : Crash kernel (low)
  bfcbe00000-bffcffffff : Crash kernel

IMO, we should test this feature more before including this in 5.11

Thanks,
Bhupesh

author
#29 lore raw

Hi Bhupesh,

Quoted 54 lines · Bhupesh Sharma

Hi Catalin, Chen,

Quoted 18 lines · Catalin Marinas
Quoted 14 lines · John Donnelly
Quoted 13 lines · Chen Zhou

Chen Zhou (9):
x86: kdump: move CRASH_ALIGN to 2M
x86: kdump: make the lower bound of crash kernel reservation
consistent
x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions
reserve_crashkernel[_low]()
x86: kdump: move reserve_crashkernel[_low]() into crash_core.c
arm64: kdump: introduce some macroes for crash kernel reservation
arm64: kdump: reimplement crashkernel=X
kdump: add threshold for the required memory
arm64: kdump: add memory for devices by DT property
linux,usable-memory-range
kdump: update Documentation about crashkernel

[...]

Quoted 2 lines

I did a brief unit-test on 5.9-rc4.

Quoted 8 lines
Quoted 3 lines

Please add:

Thanks for testing.

Quoted 1 line

This activity is over a year old. It needs accepted.

It's getting there, hopefully in 5.11. There are some minor tweaks to address.

I think my earlier email with the test results on this series bounced off the mailing list server (for some weird reason), but I still see several issues with this patchset. I will add specific issues in the review comments for each patch again, but overall, with a crashkernel size of say 786M, I see the following issue:

# cat /proc/cmdline
BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..>
rd.lvm.lv=<..snip..> crashkernel=786M

I see two regions of size 786M and 256M reserved in low and high regions respectively, So we reserve a total of 1042M of memory, which is an incorrect behaviour:

# dmesg | grep -i crash
[    0.000000] Reserving 256MB of low memory at 2816MB for crashkernel
(System low RAM: 768MB)
[    0.000000] Reserving 786MB of memory at 654158MB for crashkernel
(System RAM: 130816MB)
[    0.000000] Kernel command line:
BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+
root=/dev/mapper/rhel_ampere--hr330a--03-root ro
rd.lvm.lv=rhel_ampere-hr330a-03/root
rd.lvm.lv=rhel_ampere-hr330a-03/swap crashkernel=786M cma=1024M
Quoted 5 lines
# cat /proc/iomem | grep -i crash
  b0000000-bfffffff : Crash kernel (low)
  bfcbe00000-bffcffffff : Crash kernel

IMO, we should test this feature more before including this in 5.11

Thanks for you test. This behavior is what we what. What is the correct behavior you think?

Besides, this feature is been tested by John and PK, and i test for various parameters. We may miss something, any comments are welcome.

Thanks,
Chen Zhou

Quoted 6 lines

Thanks,
Bhupesh

.

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

maintainer
#30 lore raw
In reply to Bhupesh Sharma
Quoted 13 lines · Bhupesh Sharma

I think my earlier email with the test results on this series bounced off the mailing list server (for some weird reason), but I still see several issues with this patchset. I will add specific issues in the review comments for each patch again, but overall, with a crashkernel size of say 786M, I see the following issue:

# cat /proc/cmdline
BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> rd.lvm.lv=<..snip..> crashkernel=786M

I see two regions of size 786M and 256M reserved in low and high regions respectively, So we reserve a total of 1042M of memory, which is an incorrect behaviour:

Quoted 8 lines
# dmesg | grep -i crash
[    0.000000] Reserving 256MB of low memory at 2816MB for crashkernel (System low RAM: 768MB)
[    0.000000] Reserving 786MB of memory at 654158MB for crashkernel (System RAM: 130816MB)
[    0.000000] Kernel command line: BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ root=/dev/mapper/rhel_ampere--hr330a--03-root ro rd.lvm.lv=rhel_ampere-hr330a-03/root rd.lvm.lv=rhel_ampere-hr330a-03/swap crashkernel=786M cma=1024M
# cat /proc/iomem | grep -i crash
  b0000000-bfffffff : Crash kernel (low)
  bfcbe00000-bffcffffff : Crash kernel

As Chen said, that's the intended behaviour and how x86 works. The requested 768M goes in the high range if there's not enough low memory and an additional buffer for swiotlb is allocated, hence the low 256M.

We could (as an additional patch), subtract the 256M from the high allocation so that you'd get a low 256M and a high 512M, not sure it's worth it. Note that with a "crashkernel=768M,high" option, you still get the additional low 256M, otherwise the crashkernel won't be able to boot as there's no memory in ZONE_DMA. In the explicit ",high" request case, I'm not sure subtracted the 256M is more intuitive.

In 5.11, we also hope to fix the ZONE_DMA layout for non-RPi4 platforms to cover the entire 32-bit address space (i.e. identical to the current ZONE_DMA32).

Quoted 1 line

IMO, we should test this feature more before including this in 5.11

Definitely. That's one of the reasons we haven't queued it yet. So any help with testing here is appreciated.

Thanks.

Signature · 6 lines
Catalin

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#31 lore raw

Hi Catalin,

Quoted 20 lines · Catalin Marinas
Quoted 18 lines · Bhupesh Sharma

I think my earlier email with the test results on this series bounced off the mailing list server (for some weird reason), but I still see several issues with this patchset. I will add specific issues in the review comments for each patch again, but overall, with a crashkernel size of say 786M, I see the following issue:

# cat /proc/cmdline
BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> rd.lvm.lv=<..snip..> crashkernel=786M

I see two regions of size 786M and 256M reserved in low and high regions respectively, So we reserve a total of 1042M of memory, which is an incorrect behaviour:

# dmesg | grep -i crash
[    0.000000] Reserving 256MB of low memory at 2816MB for crashkernel (System low RAM: 768MB)
[    0.000000] Reserving 786MB of memory at 654158MB for crashkernel (System RAM: 130816MB)
[    0.000000] Kernel command line: BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ root=/dev/mapper/rhel_ampere--hr330a--03-root ro rd.lvm.lv=rhel_ampere-hr330a-03/root rd.lvm.lv=rhel_ampere-hr330a-03/swap crashkernel=786M cma=1024M
Quoted 7 lines
Quoted 3 lines
# cat /proc/iomem | grep -i crash
  b0000000-bfffffff : Crash kernel (low)
  bfcbe00000-bffcffffff : Crash kernel

As Chen said, that's the intended behaviour and how x86 works. The requested 768M goes in the high range if there's not enough low memory and an additional buffer for swiotlb is allocated, hence the low 256M.

I understand, but why 256M (as low) for arm64? x86_64 setups usually have more system memory available as compared to several commercially available arm64 setups. So is the intent, just to keep the behavior similar between arm64 and x86_64?

Should we have a CONFIG option / bootarg to help one select the max 'low_size'? Currently the ' low_size' value is calculated as:

/*
* two parts from kernel/dma/swiotlb.c:
* -swiotlb size: user-specified with swiotlb= or default. *
* -swiotlb overflow buffer: now hardcoded to 32k. We round it
* to 8M for other buffers that may need to stay low too. Also
* make sure we allocate enough extra low memory so that we
* don't run out of DMA buffers for 32-bit devices.
*/
low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20);

Since many arm64 boards ship with swiotlb=0 (turned off) via kernel bootargs, the low_size, still ends up being 256M in such cases, whereas this 256M can be used for some other purposes - so should we be limiting this to 64M and failing the crash kernel allocation request (gracefully) otherwise?

Quoted 7 lines

We could (as an additional patch), subtract the 256M from the high allocation so that you'd get a low 256M and a high 512M, not sure it's worth it. Note that with a "crashkernel=768M,high" option, you still get the additional low 256M, otherwise the crashkernel won't be able to boot as there's no memory in ZONE_DMA. In the explicit ",high" request case, I'm not sure subtracted the 256M is more intuitive.

Quoted 8 lines

In 5.11, we also hope to fix the ZONE_DMA layout for non-RPi4 platforms to cover the entire 32-bit address space (i.e. identical to the current ZONE_DMA32).

Quoted 1 line

IMO, we should test this feature more before including this in 5.11

Definitely. That's one of the reasons we haven't queued it yet. So any help with testing here is appreciated.

Sure, I am running more checks on this series. I will be soon back with more updates.

Regards,
Bhupesh

maintainer
#32 lore raw
Quoted 24 lines · Bhupesh Sharma
Quoted 23 lines · Catalin Marinas
Quoted 21 lines · Bhupesh Sharma

I think my earlier email with the test results on this series bounced off the mailing list server (for some weird reason), but I still see several issues with this patchset. I will add specific issues in the review comments for each patch again, but overall, with a crashkernel size of say 786M, I see the following issue:

# cat /proc/cmdline
BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> rd.lvm.lv=<..snip..> crashkernel=786M

I see two regions of size 786M and 256M reserved in low and high regions respectively, So we reserve a total of 1042M of memory, which is an incorrect behaviour:

# dmesg | grep -i crash
[    0.000000] Reserving 256MB of low memory at 2816MB for crashkernel (System low RAM: 768MB)
[    0.000000] Reserving 786MB of memory at 654158MB for crashkernel (System RAM: 130816MB)
[    0.000000] Kernel command line: BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ root=/dev/mapper/rhel_ampere--hr330a--03-root ro rd.lvm.lv=rhel_ampere-hr330a-03/root rd.lvm.lv=rhel_ampere-hr330a-03/swap crashkernel=786M cma=1024M
# cat /proc/iomem | grep -i crash
  b0000000-bfffffff : Crash kernel (low)
  bfcbe00000-bffcffffff : Crash kernel
Quoted 8 lines
Quoted 3 lines

As Chen said, that's the intended behaviour and how x86 works. The requested 768M goes in the high range if there's not enough low memory and an additional buffer for swiotlb is allocated, hence the low 256M.

I understand, but why 256M (as low) for arm64? x86_64 setups usually have more system memory available as compared to several commercially available arm64 setups. So is the intent, just to keep the behavior similar between arm64 and x86_64?

Similar in the sense of the fallback to high memory and some low memory allocation but the amounts can vary per architecture.

Quoted 14 lines

Should we have a CONFIG option / bootarg to help one select the max 'low_size'? Currently the ' low_size' value is calculated as:

/*
* two parts from kernel/dma/swiotlb.c:
* -swiotlb size: user-specified with swiotlb= or default. *
* -swiotlb overflow buffer: now hardcoded to 32k. We round it
* to 8M for other buffers that may need to stay low too. Also
* make sure we allocate enough extra low memory so that we
* don't run out of DMA buffers for 32-bit devices.
*/
low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20);

Quoted 5 lines

Since many arm64 boards ship with swiotlb=0 (turned off) via kernel bootargs, the low_size, still ends up being 256M in such cases, whereas this 256M can be used for some other purposes - so should we be limiting this to 64M and failing the crash kernel allocation request (gracefully) otherwise?

I think it makes sense to set a low_size = 0 if swiotlb_size_or_default() is 0. The assumption would be that if the main kernel doesn't need an swiotlb, the crashdump one wouldn't need it either. But this probably needs the ZONE_DMA for non-RPi4 platforms addressed as well (expanded to the whole ZONE_DMA32).

Signature · 6 lines
Catalin

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#33 lore raw
In reply to Bhupesh Sharma

Hi Bhupesh,

Quoted 63 lines · Bhupesh Sharma

Hi Catalin,

Quoted 19 lines · Catalin Marinas
Quoted 18 lines · Bhupesh Sharma

I think my earlier email with the test results on this series bounced off the mailing list server (for some weird reason), but I still see several issues with this patchset. I will add specific issues in the review comments for each patch again, but overall, with a crashkernel size of say 786M, I see the following issue:

# cat /proc/cmdline
BOOT_IMAGE=(hd7,gpt2)/vmlinuz-5.9.0-rc7+ root=<..snip..> rd.lvm.lv=<..snip..> crashkernel=786M

I see two regions of size 786M and 256M reserved in low and high regions respectively, So we reserve a total of 1042M of memory, which is an incorrect behaviour:

# dmesg | grep -i crash
[    0.000000] Reserving 256MB of low memory at 2816MB for crashkernel (System low RAM: 768MB)
[    0.000000] Reserving 786MB of memory at 654158MB for crashkernel (System RAM: 130816MB)
[    0.000000] Kernel command line: BOOT_IMAGE=(hd2,gpt2)/vmlinuz-5.9.0-rc7+ root=/dev/mapper/rhel_ampere--hr330a--03-root ro rd.lvm.lv=rhel_ampere-hr330a-03/root rd.lvm.lv=rhel_ampere-hr330a-03/swap crashkernel=786M cma=1024M
Quoted 6 lines
Quoted 3 lines
# cat /proc/iomem | grep -i crash
  b0000000-bfffffff : Crash kernel (low)
  bfcbe00000-bffcffffff : Crash kernel

As Chen said, that's the intended behaviour and how x86 works. The requested 768M goes in the high range if there's not enough low memory and an additional buffer for swiotlb is allocated, hence the low 256M.

I understand, but why 256M (as low) for arm64? x86_64 setups usually have more system memory available as compared to several commercially available arm64 setups. So is the intent, just to keep the behavior similar between arm64 and x86_64?

Should we have a CONFIG option / bootarg to help one select the max 'low_size'? Currently the ' low_size' value is calculated as:

/*
* two parts from kernel/dma/swiotlb.c:
* -swiotlb size: user-specified with swiotlb= or default. *
* -swiotlb overflow buffer: now hardcoded to 32k. We round it
* to 8M for other buffers that may need to stay low too. Also
* make sure we allocate enough extra low memory so that we
* don't run out of DMA buffers for 32-bit devices.
*/
low_size = max(swiotlb_size_or_default() + (8UL << 20), 256UL << 20);

Since many arm64 boards ship with swiotlb=0 (turned off) via kernel bootargs, the low_size, still ends up being 256M in such cases, whereas this 256M can be used for some other purposes - so should we be limiting this to 64M and failing the crash kernel allocation request (gracefully) otherwise?

Quoted 10 lines

We could (as an additional patch), subtract the 256M from the high allocation so that you'd get a low 256M and a high 512M, not sure it's worth it. Note that with a "crashkernel=768M,high" option, you still get the additional low 256M, otherwise the crashkernel won't be able to boot as there's no memory in ZONE_DMA. In the explicit ",high" request case, I'm not sure subtracted the 256M is more intuitive.
In 5.11, we also hope to fix the ZONE_DMA layout for non-RPi4 platforms to cover the entire 32-bit address space (i.e. identical to the current ZONE_DMA32).

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

IMO, we should test this feature more before including this in 5.11

Definitely. That's one of the reasons we haven't queued it yet. So any help with testing here is appreciated.

Sure, I am running more checks on this series. I will be soon back with more updates.

Sorry to bother you. I am looking forward to your review comments.

Thanks,
Chen Zhou

Quoted 6 lines

Regards,
Bhupesh

.

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#34 lore raw
In reply to Chen Zhou
Quoted 53 lines · Chen Zhou

There are following issues in arm64 kdump:
1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory.
2. If reserving crashkernel above 4G, in this case, crash dump kernel will boot failure because there is no low memory available for allocation.
3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"), if the memory reserved for crash dump kernel falled in ZONE_DMA32, the devices in crash dump kernel need to use ZONE_DMA will alloc fail.

To solve these issues, change the behavior of crashkernel=X. crashkernel=X tries low allocation in DMA zone, and fall back to high allocation if it fails.
If requized size X is too large and leads to very little low memory in DMA zone after low allocation, the system may not work normally. So add a threshold and go for high allocation directly if the required size is too large. The value of threshold is set as the half of the low memory.

We can also use "crashkernel=X,high" to select a high region above DMA zone, which also tries to allocate at least 256M low memory in DMA zone automatically.
"crashkernel=Y,low" can be used to allocate specified size low memory. For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.

When reserving crashkernel in high memory, some low memory is reserved for crash dump kernel devices. So there may be two regions reserved for crash dump kernel.
In order to distinct from the high region and make no effect to the use of existing kexec-tools, rename the low region as "Crash kernel (low)", and pass the low region by reusing DT property "linux,usable-memory-range". We made the low memory region as the last range of "linux,usable-memory-range" to keep compatibility with existing user-space and older kdump kernels.

Besides, we need to modify kexec-tools:
arm64: support more than one crash kernel regions(see [1])

Another update is document about DT property 'linux,usable-memory-range':
schemas: update 'linux,usable-memory-range' node schema(see [2])

This patchset contains the following nine patches:
0001-x86-kdump-move-CRASH_ALIGN-to-2M.patch 0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch 0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch 0004-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch 0005-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch 0006-arm64-kdump-reimplement-crashkernel-X.patch 0007-kdump-add-threshold-for-the-required-memory.patch 0008-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch 0009-kdump-update-Documentation-about-crashkernel.patch

Quoted 8 lines

0001-0003 are some x86 cleanups which prepares for making functionsreserve_crashkernel[_low]() generic.

0004 makes functions reserve_crashkernel[_low]() generic. 0005-0006 reimplements crashkernel=X.
0007 adds threshold for the required memory.
0008 adds memory for devices by DT property linux,usable-memory-range. 0009 updates the doc.

Hi Catalin and Dave,

Any other suggestions about this patchset? Let me know if you have any questions.

Thanks,
Chen Zhou

Quoted 91 lines

Changes since [v11]
- Rebased on top of 5.9-rc4.
- Make the function reserve_crashkernel() of x86 generic. Suggested by Catalin, make the function reserve_crashkernel() of x86 generic and arm64 use the generic version to reimplement crashkernel=X.

Changes since [v10]
- Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin.

Changes since [v9]
- Patch 1 add Acked-by from Dave.
- Update patch 5 according to Dave's comments.
- Update chosen schema.

Changes since [v8]
- Reuse DT property "linux,usable-memory-range".
Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the low memory region.
- Fix kdump broken with ZONE_DMA reintroduced.
- Update chosen schema.

Changes since [v7]
- Move x86 CRASH_ALIGN to 2M
Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
- Update Documentation/devicetree/bindings/chosen.txt.
Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt suggested by Arnd.
- Add Tested-by from Jhon and pk.

Changes since [v6]
- Fix build errors reported by kbuild test robot.

Changes since [v5]
- Move reserve_crashkernel_low() into kernel/crash_core.c.
- Delete crashkernel=X,high.
- Modify crashkernel=X,low.
If crashkernel=X,low is specified simultaneously, reserve spcified size low memory for crash kdump kernel devices firstly and then reserve memory above 4G. In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then pass to crash dump kernel by DT property "linux,low-memory-range".
- Update Documentation/admin-guide/kdump/kdump.rst.

Changes since [v4]
- Reimplement memblock_cap_memory_ranges for multiple ranges by Mike.

Changes since [v3]
- Add memblock_cap_memory_ranges back for multiple ranges.
- Fix some compiling warnings.

Changes since [v2]
- Split patch "arm64: kdump: support reserving crashkernel above 4G" as two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate patch.

Changes since [v1]:
- Move common reserve_crashkernel_low() code into kernel/kexec_core.c.
- Remove memblock_cap_memory_ranges() i added in v1 and implement that in fdt_enforce_memory_region().
There are at most two crash kernel regions, for two crash kernel regions case, we cap the memory range [min(regs[*].start), max(regs[*].end)] and then remove the memory range in the middle.

[1]: http://lists.infradead.org/pipermail/kexec/2020-June/020737.html
[2]: https://github.com/robherring/dt-schema/pull/19
[v1]: https://lkml.org/lkml/2019/4/2/1174
[v2]: https://lkml.org/lkml/2019/4/9/86
[v3]: https://lkml.org/lkml/2019/4/9/306
[v4]: https://lkml.org/lkml/2019/4/15/273
[v5]: https://lkml.org/lkml/2019/5/6/1360
[v6]: https://lkml.org/lkml/2019/8/30/142
[v7]: https://lkml.org/lkml/2019/12/23/411
[v8]: https://lkml.org/lkml/2020/5/21/213
[v9]: https://lkml.org/lkml/2020/6/28/73
[v10]: https://lkml.org/lkml/2020/7/2/1443
[v11]: https://lkml.org/lkml/2020/8/1/150

Chen Zhou (9):
x86: kdump: move CRASH_ALIGN to 2M
x86: kdump: make the lower bound of crash kernel reservation
consistent
x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions
reserve_crashkernel[_low]()
x86: kdump: move reserve_crashkernel[_low]() into crash_core.c
arm64: kdump: introduce some macroes for crash kernel reservation
arm64: kdump: reimplement crashkernel=X
kdump: add threshold for the required memory
arm64: kdump: add memory for devices by DT property
linux,usable-memory-range
kdump: update Documentation about crashkernel

Quoted 15 lines
 Documentation/admin-guide/kdump/kdump.rst     |  25 ++-
 .../admin-guide/kernel-parameters.txt         |  13 +-
 arch/arm64/include/asm/kexec.h                |  15 ++
 arch/arm64/include/asm/processor.h            |   1 +
 arch/arm64/kernel/setup.c                     |  13 +-
 arch/arm64/mm/init.c                          | 105 ++++------
 arch/arm64/mm/mmu.c                           |   4 +
 arch/x86/include/asm/kexec.h                  |  28 +++
 arch/x86/kernel/setup.c                       | 165 +--------------
 include/linux/crash_core.h                    |   4 +
 include/linux/kexec.h                         |   2 -
 kernel/crash_core.c                           | 192 ++++++++++++++++++
 kernel/kexec_core.c                           |  17 --
 13 files changed, 328 insertions(+), 256 deletions(-)

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#35 lore raw
Quoted 63 lines · chenzhou
Quoted 56 lines · Chen Zhou

There are following issues in arm64 kdump:
1. We use crashkernel=X to reserve crashkernel below 4G, which will fail when there is no enough low memory.
2. If reserving crashkernel above 4G, in this case, crash dump kernel will boot failure because there is no low memory available for allocation.
3. Since commit 1a8e1cef7603 ("arm64: use both ZONE_DMA and ZONE_DMA32"), if the memory reserved for crash dump kernel falled in ZONE_DMA32, the devices in crash dump kernel need to use ZONE_DMA will alloc fail.

To solve these issues, change the behavior of crashkernel=X. crashkernel=X tries low allocation in DMA zone, and fall back to high allocation if it fails.
If requized size X is too large and leads to very little low memory in DMA zone after low allocation, the system may not work normally. So add a threshold and go for high allocation directly if the required size is too large. The value of threshold is set as the half of the low memory.

We can also use "crashkernel=X,high" to select a high region above DMA zone, which also tries to allocate at least 256M low memory in DMA zone automatically.
"crashkernel=Y,low" can be used to allocate specified size low memory. For non-RPi4 platforms, change DMA zone memtioned above to DMA32 zone.

When reserving crashkernel in high memory, some low memory is reserved for crash dump kernel devices. So there may be two regions reserved for crash dump kernel.
In order to distinct from the high region and make no effect to the use of existing kexec-tools, rename the low region as "Crash kernel (low)", and pass the low region by reusing DT property "linux,usable-memory-range". We made the low memory region as the last range of "linux,usable-memory-range" to keep compatibility with existing user-space and older kdump kernels.

Besides, we need to modify kexec-tools:
arm64: support more than one crash kernel regions(see [1])

Another update is document about DT property 'linux,usable-memory-range':
schemas: update 'linux,usable-memory-range' node schema(see [2])

This patchset contains the following nine patches:
0001-x86-kdump-move-CRASH_ALIGN-to-2M.patch 0002-x86-kdump-make-the-lower-bound-of-crash-kernel-reser.patch 0003-x86-kdump-use-macro-CRASH_ADDR_LOW_MAX-in-functions-.patch 0004-x86-kdump-move-reserve_crashkernel-_low-into-crash_c.patch 0005-arm64-kdump-introduce-some-macroes-for-crash-kernel-.patch 0006-arm64-kdump-reimplement-crashkernel-X.patch 0007-kdump-add-threshold-for-the-required-memory.patch 0008-arm64-kdump-add-memory-for-devices-by-DT-property-li.patch 0009-kdump-update-Documentation-about-crashkernel.patch

0001-0003 are some x86 cleanups which prepares for making functionsreserve_crashkernel[_low]() generic.

Quoted 6 lines
Quoted 5 lines

0004 makes functions reserve_crashkernel[_low]() generic. 0005-0006 reimplements crashkernel=X.
0007 adds threshold for the required memory.
0008 adds memory for devices by DT property linux,usable-memory-range. 0009 updates the doc.

Hi Catalin and Dave,

Hi,

This patch set has been going on since May, 2019. When will this be accepted and integrated into a rc build ?

Quoted 110 lines

Any other suggestions about this patchset? Let me know if you have any questions.

Thanks,
Chen Zhou

Quoted 105 lines

Changes since [v11]
- Rebased on top of 5.9-rc4.
- Make the function reserve_crashkernel() of x86 generic. Suggested by Catalin, make the function reserve_crashkernel() of x86 generic and arm64 use the generic version to reimplement crashkernel=X.

Changes since [v10]
- Reimplement crashkernel=X suggested by Catalin, Many thanks to Catalin.

Changes since [v9]
- Patch 1 add Acked-by from Dave.
- Update patch 5 according to Dave's comments.
- Update chosen schema.

Changes since [v8]
- Reuse DT property "linux,usable-memory-range".
Suggested by Rob, reuse DT property "linux,usable-memory-range" to pass the low memory region.
- Fix kdump broken with ZONE_DMA reintroduced.
- Update chosen schema.

Changes since [v7]
- Move x86 CRASH_ALIGN to 2M
Suggested by Dave and do some test, move x86 CRASH_ALIGN to 2M.
- Update Documentation/devicetree/bindings/chosen.txt.
Add corresponding documentation to Documentation/devicetree/bindings/chosen.txt suggested by Arnd.
- Add Tested-by from Jhon and pk.

Changes since [v6]
- Fix build errors reported by kbuild test robot.

Changes since [v5]
- Move reserve_crashkernel_low() into kernel/crash_core.c.
- Delete crashkernel=X,high.
- Modify crashkernel=X,low.
If crashkernel=X,low is specified simultaneously, reserve spcified size low memory for crash kdump kernel devices firstly and then reserve memory above 4G. In addition, rename crashk_low_res as "Crash kernel (low)" for arm64, and then pass to crash dump kernel by DT property "linux,low-memory-range".
- Update Documentation/admin-guide/kdump/kdump.rst.

Changes since [v4]
- Reimplement memblock_cap_memory_ranges for multiple ranges by Mike.

Changes since [v3]
- Add memblock_cap_memory_ranges back for multiple ranges.
- Fix some compiling warnings.

Changes since [v2]
- Split patch "arm64: kdump: support reserving crashkernel above 4G" as two. Put "move reserve_crashkernel_low() into kexec_core.c" in a separate patch.

Changes since [v1]:
- Move common reserve_crashkernel_low() code into kernel/kexec_core.c.
- Remove memblock_cap_memory_ranges() i added in v1 and implement that in fdt_enforce_memory_region().
There are at most two crash kernel regions, for two crash kernel regions case, we cap the memory range [min(regs[*].start), max(regs[*].end)] and then remove the memory range in the middle.

[1]: https://urldefense.com/v3/__http://lists.infradead.org/pipermail/kexec/2020-June/020737.html__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_zMCI6U-$
[2]: https://urldefense.com/v3/__https://github.com/robherring/dt-schema/pull/19__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_5c9NEUf$
[v1]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/2/1174__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_1bFn-eN$
[v2]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/86__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_wVqWygD$
[v3]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/9/306__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_8fQ7uBl$
[v4]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/4/15/273__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_ztbOBKM$
[v5]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/5/6/1360__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_9TAk7Oj$
[v6]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/8/30/142__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_9IFx5Hx$
[v7]: https://urldefense.com/v3/__https://lkml.org/lkml/2019/12/23/411__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_0x8im8q$
[v8]: https://urldefense.com/v3/__https://lkml.org/lkml/2020/5/21/213__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_yVVP42e$
[v9]: https://urldefense.com/v3/__https://lkml.org/lkml/2020/6/28/73__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_y2-BLN1$
[v10]: https://urldefense.com/v3/__https://lkml.org/lkml/2020/7/2/1443__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_0qJHLGR$
[v11]: https://urldefense.com/v3/__https://lkml.org/lkml/2020/8/1/150__;!!GqivPVa7Brio!JI57eED82U9Uq1k8V_Kus7azGGPSDqfaSZPHM0WkR6OxQ0trzzeR2zyIkUM8_3QitPUY$

Chen Zhou (9):
x86: kdump: move CRASH_ALIGN to 2M
x86: kdump: make the lower bound of crash kernel reservation
consistent
x86: kdump: use macro CRASH_ADDR_LOW_MAX in functions
reserve_crashkernel[_low]()
x86: kdump: move reserve_crashkernel[_low]() into crash_core.c
arm64: kdump: introduce some macroes for crash kernel reservation
arm64: kdump: reimplement crashkernel=X
kdump: add threshold for the required memory
arm64: kdump: add memory for devices by DT property
linux,usable-memory-range
kdump: update Documentation about crashkernel

Documentation/admin-guide/kdump/kdump.rst     |  25 ++-
.../admin-guide/kernel-parameters.txt         |  13 +-
arch/arm64/include/asm/kexec.h                |  15 ++
arch/arm64/include/asm/processor.h            |   1 +
arch/arm64/kernel/setup.c                     |  13 +-
arch/arm64/mm/init.c                          | 105 ++++------
arch/arm64/mm/mmu.c                           |   4 +
arch/x86/include/asm/kexec.h                  |  28 +++
arch/x86/kernel/setup.c                       | 165 +--------------
include/linux/crash_core.h                    |   4 +
include/linux/kexec.h                         |   2 -
kernel/crash_core.c                           | 192 ++++++++++++++++++
kernel/kexec_core.c                           |  17 --
13 files changed, 328 insertions(+), 256 deletions(-)

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec