Skip to contentSkip to conversation

kho: do not reserve scratch memory in the kdump kernel

PATCH Superseded Maxi Saparov 2 participants 6 messages Aug 3 – Aug 24

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kho: do not reserve scratch memory in the kdump kernel

From: Maxi Saparov <[email protected]>

If a kernel with CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y (or kho=on) does not receive a handover FDT, it reserves KHO scratch memory in kho_reserve_scratch(). The function computes the sizes from the early memblock reservations of the kernel. The default scratch_scale is 200%. The lowmem and global scratch areas each get twice the size of the early reservations. Each NUMA node gets one more scratch area at the same scale.

The kdump kernel always takes this path. Since commit a6715d7ec472 ("kho: skip KHO for crash kernel"), the kernel does not add KHO metadata to the crash kimage. As a result, the kdump kernel never receives a handover FDT and always reserves scratch memory.

The kdump kernel boots in the small crashkernel= memory reservation. On our x86 hosts, the kernel and initramfs make approximately 176 MB of early reservations. The scratch areas then use approximately 650 MB more of the crashkernel memory. When a large early allocation fails, the kdump kernel panics:

bio: can't create integrity buf pool

A kdump kernel has nothing to hand over: its only task is to dump the memory of the old kernel and reboot. Disable KHO in the kdump kernel so that it does not reserve scratch memory.

Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")
---
 kernel/liveupdate/kexec_handover.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/liveupdate/kexec_handover.c +4 -0
index 6fad9152387a..53b59edb5b36 100644
@@ -12,6 +12,7 @@
 
 #include <linux/cleanup.h>
 #include <linux/cma.h>
+#include <linux/crash_dump.h>
 #include <linux/kmemleak.h>
 #include <linux/count_zeros.h>
 #include <linux/kasan.h>
@@ -1636,6 +1637,9 @@ void __init kho_memory_init(void)
 
 		if (kho_mem_retrieve(kho_get_fdt()))
 			kho_in.fdt_phys = 0;
+	} else if (is_kdump_kernel()) {
+		kho_enable = false;
+		pr_info("disabled in the kdump kernel\n");
 	} else {
 		kho_reserve_scratch();
 	}

base-commit: 3a0b8fa2eb36afc88b62a95f33f0c77c71fa5ded

Signature · 1 line
2.55.0
maintainer
#1 lore raw

Hi,

Quoted 47 lines · Maxi Saparov

From: Maxi Saparov <[email protected]>

If a kernel with CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y (or kho=on) does not receive a handover FDT, it reserves KHO scratch memory in kho_reserve_scratch(). The function computes the sizes from the early memblock reservations of the kernel. The default scratch_scale is 200%. The lowmem and global scratch areas each get twice the size of the early reservations. Each NUMA node gets one more scratch area at the same scale.

The kdump kernel always takes this path. Since commit a6715d7ec472 ("kho: skip KHO for crash kernel"), the kernel does not add KHO metadata to the crash kimage. As a result, the kdump kernel never receives a handover FDT and always reserves scratch memory.

The kdump kernel boots in the small crashkernel= memory reservation. On our x86 hosts, the kernel and initramfs make approximately 176 MB of early reservations. The scratch areas then use approximately 650 MB more of the crashkernel memory. When a large early allocation fails, the kdump kernel panics:

bio: can't create integrity buf pool

A kdump kernel has nothing to hand over: its only task is to dump the memory of the old kernel and reboot. Disable KHO in the kdump kernel so that it does not reserve scratch memory.

Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")
---
 kernel/liveupdate/kexec_handover.c | 4 ++++
 1 file changed, 4 insertions(+)
kernel/liveupdate/kexec_handover.c +1 -0
index 6fad9152387a..53b59edb5b36 100644
@@ -12,6 +12,7 @@
 
 #include <linux/cleanup.h>
 #include <linux/cma.h>
+#include <linux/crash_dump.h>
 #include <linux/kmemleak.h>
 #include <linux/count_zeros.h>
 #include <linux/kasan.h>

kernel/liveupdate/kexec_handover.c void __init kho_memory_init(void) · line 1642

Quoted 6 lines
kernel/liveupdate/kexec_handover.c +3 -0
@@ -1636,6 +1637,9 @@ void __init kho_memory_init(void)
 
 		if (kho_mem_retrieve(kho_get_fdt()))
 			kho_in.fdt_phys = 0;
+	} else if (is_kdump_kernel()) {
+		kho_enable = false;
+		pr_info("disabled in the kdump kernel\n");

Can we put this on top as a separate if, like

	if (is_kdump_kernel()) {
		/* disable KHO */
		return;
	}
Quoted 8 lines
kernel/liveupdate/kexec_handover.c +0 -0
 	} else {
 		kho_reserve_scratch();
 	}

base-commit: 3a0b8fa2eb36afc88b62a95f33f0c77c71fa5ded

Signature · 1 line
2.55.0
Signature · 2 lines
Sincerely yours,
Mike.
#2 lore raw

Hi,

Quoted 8 lines · Mike Rapoport

Can we put this on top as a separate if, like

	if (is_kdump_kernel()) {
		/* disable KHO */
		return;
	}

Sure, I agree early return is nicer here. Sending V2.

Thanks,
Maxi

#3 lore raw

PATCH v2 1/1 kho: do not reserve scratch memory in the kdump kernel

In reply to Maxi Saparov

From: Maxi Saparov <[email protected]>

If a kernel with CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y (or kho=on) does not receive a handover FDT, it reserves KHO scratch memory in kho_reserve_scratch().

The kdump kernel always takes this path. Since commit a6715d7ec472 ("kho: skip KHO for crash kernel"), the kernel does not add KHO metadata to the crash kimage. As a result, the kdump kernel never receives a handover FDT and always reserves scratch memory.

The kdump kernel boots in the small crashkernel= memory reservation. On our x86 hosts, the kernel and initramfs make approximately 176 MB of early reservations. The scratch areas then use approximately 650 MB more of the crashkernel memory. When a large early allocation fails, the kdump kernel panics:

bio: can't create integrity buf pool

A kdump kernel has nothing to hand over: its only task is to dump the memory of the old kernel and reboot. Fix is to disable KHO at the top of kho_memory_init() so that the kdump kernel does not reserve scratch memory.

Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")

---
v2: Move the kdump check to the top of kho_memory_init() as an early return.
v1: https://lore.kernel.org/all/[email protected]/

 kernel/liveupdate/kexec_handover.c | 7 +++++++
 1 file changed, 7 insertions(+)
kernel/liveupdate/kexec_handover.c +7 -0
index 6fad9152387a..97729dec6359 100644
@@ -12,6 +12,7 @@
 
 #include <linux/cleanup.h>
 #include <linux/cma.h>
+#include <linux/crash_dump.h>
 #include <linux/kmemleak.h>
 #include <linux/count_zeros.h>
 #include <linux/kasan.h>
@@ -1631,6 +1632,12 @@ fs_initcall(kho_init);
 
 void __init kho_memory_init(void)
 {
+	if (is_kdump_kernel()) {
+		kho_enable = false;
+		pr_info("disabled in the kdump kernel\n");
+		return;
+	}
+
 	if (kho_in.scratch_phys) {
 		kho_scratch = phys_to_virt(kho_in.scratch_phys);
 

base-commit: 3a0b8fa2eb36afc88b62a95f33f0c77c71fa5ded

Signature · 1 line
2.55.0
maintainer
#4 lore raw

Hi,

Please don't send vN+1 as a reply to vN next time.

Quoted 19 lines · Maxi Saparov

From: Maxi Saparov <[email protected]>

If a kernel with CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y (or kho=on) does not receive a handover FDT, it reserves KHO scratch memory in kho_reserve_scratch().

The kdump kernel always takes this path. Since commit a6715d7ec472 ("kho: skip KHO for crash kernel"), the kernel does not add KHO metadata to the crash kimage. As a result, the kdump kernel never receives a handover FDT and always reserves scratch memory.

The kdump kernel boots in the small crashkernel= memory reservation. On our x86 hosts, the kernel and initramfs make approximately 176 MB of early reservations. The scratch areas then use approximately 650 MB more of the crashkernel memory. When a large early allocation fails, the kdump kernel panics:

bio: can't create integrity buf pool

Quoted 8 lines

A kdump kernel has nothing to hand over: its only task is to dump the memory of the old kernel and reboot. Fix is to disable KHO at the top of kho_memory_init() so that the kdump kernel does not reserve scratch memory.

Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")

I queued this patch for v7.4, it'll appear in the liveupdate tree after the merge window.

Quoted 6 lines

---
v2: Move the kdump check to the top of kho_memory_init() as an early return.
v1: https://lore.kernel.org/all/[email protected]/

 kernel/liveupdate/kexec_handover.c | 7 +++++++
 1 file changed, 7 insertions(+)
Signature · 2 lines
Sincerely yours,
Mike.
#5 lore raw

Hi,

Quoted 4 lines · Mike Rapoport

Hi,

Please don't send vN+1 as a reply to vN next time.

Ack, sorry about that! Thank you for the review.

Maxi