kho: do not reserve scratch memory in the kdump kernel
Subsystems (4)
PATCH 1/1 kho: do not reserve scratch memory in the kdump kernel
From: Maxi Saparov <[email protected]>
If a kernel with CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y (or kho=on) does not receive a handover FDT, it reserves KHO scratch memory in kho_reserve_scratch(). The function computes the sizes from the early memblock reservations of the kernel. The default scratch_scale is 200%. The lowmem and global scratch areas each get twice the size of the early reservations. Each NUMA node gets one more scratch area at the same scale.
The kdump kernel always takes this path. Since commit a6715d7ec472 ("kho: skip KHO for crash kernel"), the kernel does not add KHO metadata to the crash kimage. As a result, the kdump kernel never receives a handover FDT and always reserves scratch memory.
The kdump kernel boots in the small crashkernel= memory reservation. On our x86 hosts, the kernel and initramfs make approximately 176 MB of early reservations. The scratch areas then use approximately 650 MB more of the crashkernel memory. When a large early allocation fails, the kdump kernel panics:
bio: can't create integrity buf pool
A kdump kernel has nothing to hand over: its only task is to dump the memory of the old kernel and reboot. Disable KHO in the kdump kernel so that it does not reserve scratch memory.
--- kernel/liveupdate/kexec_handover.c | 4 ++++ 1 file changed, 4 insertions(+)
kernel/liveupdate/kexec_handover.c +4 -0
@@ -12,6 +12,7 @@#include <linux/cleanup.h> #include <linux/cma.h> +#include <linux/crash_dump.h> #include <linux/kmemleak.h> #include <linux/count_zeros.h> #include <linux/kasan.h>
@@ -1636,6 +1637,9 @@ void __init kho_memory_init(void)if (kho_mem_retrieve (kho_get_fdt ())) kho_in.fdt_phys =0 ; + }else if (is_kdump_kernel ()) { + kho_enable = false; +pr_info ("disabled in the kdump kernel\n" ); }else {kho_reserve_scratch (); }
base-commit: 3a0b8fa2eb36afc88b62a95f33f0c77c71fa5ded
Signature · 1 line
2.55.0
Mike Rapoport
maintainerHi,
Quoted 47 lines · Maxi Saparov
From: Maxi Saparov <[email protected]>
If a kernel with
CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y(or kho=on) does not receive a handover FDT, it reserves KHO scratch memory inkho_reserve_scratch(). The function computes the sizes from the early memblock reservations of the kernel. The defaultscratch_scaleis 200%. The lowmem and global scratch areas each get twice the size of the early reservations. Each NUMA node gets one more scratch area at the same scale.The kdump kernel always takes this path. Since commit
a6715d7ec472("kho: skip KHO for crash kernel"), the kernel does not add KHO metadata to the crash kimage. As a result, the kdump kernel never receives a handover FDT and always reserves scratch memory.The kdump kernel boots in the small crashkernel= memory reservation. On our x86 hosts, the kernel and initramfs make approximately 176 MB of early reservations. The scratch areas then use approximately 650 MB more of the crashkernel memory. When a large early allocation fails, the kdump kernel panics:
bio: can't create integrity buf pool
A kdump kernel has nothing to hand over: its only task is to dump the memory of the old kernel and reboot. Disable KHO in the kdump kernel so that it does not reserve scratch memory.
Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")Signed-off-by Maxi Saparov <[email protected]>--- kernel/liveupdate/kexec_handover.c | 4 ++++ 1 file changed, 4 insertions(+)kernel/liveupdate/kexec_handover.c +1
-0@@ -12,6 +12,7 @@#include <linux/cleanup.h> #include <linux/cma.h> +#include <linux/crash_dump.h> #include <linux/kmemleak.h> #include <linux/count_zeros.h> #include <linux/kasan.h>
kernel/liveupdate/kexec_handover.c void __init kho_memory_init(void) · line 1642
Quoted 6 lines
kernel/liveupdate/kexec_handover.c +3
-0@@ -1636,6 +1637,9 @@ void __init kho_memory_init(void)if (kho_mem_retrieve (kho_get_fdt ())) kho_in.fdt_phys =0 ; + }else if (is_kdump_kernel ()) { + kho_enable = false; +pr_info ("disabled in the kdump kernel\n" );
Can we put this on top as a separate if, like
if (is_kdump_kernel()) {
/* disable KHO */
return;
}Quoted 8 lines
kernel/liveupdate/kexec_handover.c +0
-0}else {kho_reserve_scratch (); }base-commit:
3a0b8fa2eb36afc88b62a95f33f0c77c71fa5dedSignature · 1 line
2.55.0
Signature · 2 lines
Sincerely yours, Mike.
Hi,
Quoted 8 lines · Mike Rapoport
Can we put this on top as a separate if, like
if (is_kdump_kernel()) { /* disable KHO */ return; }
Sure, I agree early return is nicer here. Sending V2.
Thanks,
Maxi
PATCH v2 1/1 kho: do not reserve scratch memory in the kdump kernel
In reply to Maxi SaparovFrom: Maxi Saparov <[email protected]>
If a kernel with CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y (or kho=on) does not receive a handover FDT, it reserves KHO scratch memory in kho_reserve_scratch().
The kdump kernel always takes this path. Since commit a6715d7ec472 ("kho: skip KHO for crash kernel"), the kernel does not add KHO metadata to the crash kimage. As a result, the kdump kernel never receives a handover FDT and always reserves scratch memory.
The kdump kernel boots in the small crashkernel= memory reservation. On our x86 hosts, the kernel and initramfs make approximately 176 MB of early reservations. The scratch areas then use approximately 650 MB more of the crashkernel memory. When a large early allocation fails, the kdump kernel panics:
bio: can't create integrity buf pool
A kdump kernel has nothing to hand over: its only task is to dump the memory of the old kernel and reboot. Fix is to disable KHO at the top of kho_memory_init() so that the kdump kernel does not reserve scratch memory.
---
v2: Move the kdump check to the top of kho_memory_init() as an early return.
v1: https://lore.kernel.org/all/[email protected]/
kernel/liveupdate/kexec_handover.c | 7 +++++++ 1 file changed, 7 insertions(+)
kernel/liveupdate/kexec_handover.c +7 -0
@@ -12,6 +12,7 @@#include <linux/cleanup.h> #include <linux/cma.h> +#include <linux/crash_dump.h> #include <linux/kmemleak.h> #include <linux/count_zeros.h> #include <linux/kasan.h>
@@ -1631,6 +1632,12 @@ fs_initcall(kho_init);void __init kho_memory_init (void ) { +if (is_kdump_kernel ()) { + kho_enable = false; +pr_info ("disabled in the kdump kernel\n" ); +return ; + } +if (kho_in.scratch_phys) { kho_scratch =phys_to_virt (kho_in.scratch_phys);
base-commit: 3a0b8fa2eb36afc88b62a95f33f0c77c71fa5ded
Signature · 1 line
2.55.0
Mike Rapoport
maintainerHi,
Please don't send vN+1 as a reply to vN next time.
Quoted 19 lines · Maxi Saparov
From: Maxi Saparov <[email protected]>
If a kernel with
CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y(or kho=on) does not receive a handover FDT, it reserves KHO scratch memory inkho_reserve_scratch().The kdump kernel always takes this path. Since commit
a6715d7ec472("kho: skip KHO for crash kernel"), the kernel does not add KHO metadata to the crash kimage. As a result, the kdump kernel never receives a handover FDT and always reserves scratch memory.The kdump kernel boots in the small crashkernel= memory reservation. On our x86 hosts, the kernel and initramfs make approximately 176 MB of early reservations. The scratch areas then use approximately 650 MB more of the crashkernel memory. When a large early allocation fails, the kdump kernel panics:
bio: can't create integrity buf pool
Quoted 8 lines
A kdump kernel has nothing to hand over: its only task is to dump the memory of the old kernel and reboot. Fix is to disable KHO at the top of
kho_memory_init()so that the kdump kernel does not reserve scratch memory.Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")Signed-off-by Maxi Saparov <[email protected]>
I queued this patch for v7.4, it'll appear in the liveupdate tree after the merge window.
Quoted 6 lines
---
v2: Move the kdump check to the top ofkho_memory_init()as an early return.
v1: https://lore.kernel.org/all/[email protected]/kernel/liveupdate/kexec_handover.c | 7 +++++++ 1 file changed, 7 insertions(+)
Signature · 2 lines
Sincerely yours, Mike.
Hi,
Quoted 4 lines · Mike Rapoport
Hi,
Please don't send vN+1 as a reply to vN next time.
Ack, sorry about that! Thank you for the review.
Maxi