Skip to contentSkip to conversation

efi: Support booting with kexec handover (KHO)

PATCH In review Evangelos Petrongonas 2 participants 4 messages Aug 8, 2025 – Aug 14, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 efi: Support booting with kexec handover (KHO)

When KHO (Kexec HandOver) is enabled, it sets up scratch memory regions early during device tree scanning. After kexec, the new kernel exclusively uses this region for memory allocations during boot up to the initialization of the page allocator

However, when booting with EFI, EFI's reserve_regions() uses memblock_remove(0, PHYS_ADDR_MAX) to clear all memory regions before rebuilding them from EFI data. This destroys KHO scratch regions and their flags, thus causing a kernel panic, as there are no scratch memory regions.

Instead of wholesale removal, iterate through memory regions and only remove non-KHO ones. This preserves KHO scratch regions while still allowing EFI to rebuild its memory map.

---

Reproduction/Verification Steps
The issue and the fix can be reproduced/verified by booting a VM with EFI and attempting to perform a KHO enabled kexec. The fix was developed/tested on arm64.

 drivers/firmware/efi/efi-init.c | 31 +++++++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 4 deletions(-)
drivers/firmware/efi/efi-init.c +27 -4
index a00e07b853f22..2f08b1ab764f6 100644
@@ -164,12 +164,35 @@ static __init void reserve_regions(void)
 		pr_info("Processing EFI memory map:\n");
 
 	/*
-	 * Discard memblocks discovered so far: if there are any at this
-	 * point, they originate from memory nodes in the DT, and UEFI
-	 * uses its own memory map instead.
+	 * Discard memblocks discovered so far except for KHO scratch regions.
+	 * Most memblocks at this point originate from memory nodes in the DT,
+	 * and UEFI uses its own memory map instead. However, if KHO is enabled,
+	 * scratch regions must be preserved.
 	 */
 	memblock_dump_all();
-	memblock_remove(0, PHYS_ADDR_MAX);
+
+	if (IS_ENABLED(CONFIG_MEMBLOCK_KHO_SCRATCH)) {
+		struct memblock_region *reg;
+		phys_addr_t start, size;
+		int i;
+
+		/* Remove all non-KHO regions */
+		for (i = memblock.memory.cnt - 1; i >= 0; i--) {
+			reg = &memblock.memory.regions[i];
+			if (!memblock_is_kho_scratch(reg)) {
+				start = reg->base;
+				size = reg->size;
+				memblock_remove(start, size);
+			}
+		}
+	} else {
+	/*
+	 * KHO is disabled. Discard memblocks discovered so far: if there
+	 * are any at this point, they originate from memory nodes in the
+	 * DT, and UEFI uses its own memory map instead.
+	 */
+		memblock_remove(0, PHYS_ADDR_MAX);
+	}
 
 	for_each_efi_memory_desc(md) {
 		paddr = md->phys_addr;
Signature · 12 lines
2.43.0




Amazon Web Services Development Center Germany GmbH
Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597
maintainer
#1 lore raw
Quoted 11 lines · Evangelos Petrongonas

When KHO (Kexec HandOver) is enabled, it sets up scratch memory regions early during device tree scanning. After kexec, the new kernel exclusively uses this region for memory allocations during boot up to the initialization of the page allocator

However, when booting with EFI, EFI's reserve_regions() uses memblock_remove(0, PHYS_ADDR_MAX) to clear all memory regions before rebuilding them from EFI data. This destroys KHO scratch regions and their flags, thus causing a kernel panic, as there are no scratch memory regions.

Quoted 3 lines

Instead of wholesale removal, iterate through memory regions and only remove non-KHO ones. This preserves KHO scratch regions while still allowing EFI to rebuild its memory map.

It's worth mentioning that scratch areas are "good known memory" :)

Quoted 21 lines

---

Reproduction/Verification Steps
The issue and the fix can be reproduced/verified by booting a VM with EFI and attempting to perform a KHO enabled kexec. The fix was developed/tested on arm64.

 drivers/firmware/efi/efi-init.c | 31 +++++++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 4 deletions(-)
drivers/firmware/efi/efi-init.c +0 -2
index a00e07b853f22..2f08b1ab764f6 100644
@@ -164,12 +164,35 @@ static __init void reserve_regions(void)
 		pr_info("Processing EFI memory map:\n");
 
 	/*
-	 * Discard memblocks discovered so far: if there are any at this
-	 * point, they originate from memory nodes in the DT, and UEFI

drivers/firmware/efi/efi-init.c static __init void reserve_regions(void) · line 174

Quoted 10 lines
drivers/firmware/efi/efi-init.c +6 -2
-	 * uses its own memory map instead.
+	 * Discard memblocks discovered so far except for KHO scratch regions.
+	 * Most memblocks at this point originate from memory nodes in the DT,
+	 * and UEFI uses its own memory map instead. However, if KHO is enabled,
+	 * scratch regions must be preserved.
 	 */
 	memblock_dump_all();
-	memblock_remove(0, PHYS_ADDR_MAX);
+
+	if (IS_ENABLED(CONFIG_MEMBLOCK_KHO_SCRATCH)) {

It's better to condition this on kho_get_fdt() that means that we are actually doing a handover.

drivers/firmware/efi/efi-init.c static __init void reserve_regions(void) · line 180

Quoted 6 lines
drivers/firmware/efi/efi-init.c +6 -0
+		struct memblock_region *reg;
+		phys_addr_t start, size;
+		int i;
+
+		/* Remove all non-KHO regions */
+		for (i = memblock.memory.cnt - 1; i >= 0; i--) {

Please use for_each_mem_region()

Quoted 20 lines
drivers/firmware/efi/efi-init.c +15 -0
+			reg = &memblock.memory.regions[i];
+			if (!memblock_is_kho_scratch(reg)) {
+				start = reg->base;
+				size = reg->size;
+				memblock_remove(start, size);
+			}
+		}
+	} else {
+	/*
+	 * KHO is disabled. Discard memblocks discovered so far: if there
+	 * are any at this point, they originate from memory nodes in the
+	 * DT, and UEFI uses its own memory map instead.
+	 */
+		memblock_remove(0, PHYS_ADDR_MAX);
+	}
 
 	for_each_efi_memory_desc(md) {
 		paddr = md->phys_addr;
Signature · 1 line
2.43.0
Signature · 2 lines
Sincerely yours,
Mike.
#2 lore raw

Hey Mike, thanks for your review,

Quoted 12 lines · Mike Rapoport
Quoted 11 lines · Evangelos Petrongonas

When KHO (Kexec HandOver) is enabled, it sets up scratch memory regions early during device tree scanning. After kexec, the new kernel exclusively uses this region for memory allocations during boot up to the initialization of the page allocator

However, when booting with EFI, EFI's reserve_regions() uses memblock_remove(0, PHYS_ADDR_MAX) to clear all memory regions before rebuilding them from EFI data. This destroys KHO scratch regions and their flags, thus causing a kernel panic, as there are no scratch memory regions.

Quoted 6 lines
Quoted 3 lines

Instead of wholesale removal, iterate through memory regions and only remove non-KHO ones. This preserves KHO scratch regions while still allowing EFI to rebuild its memory map.

It's worth mentioning that scratch areas are "good known memory" :)

I Will do so on Rev2.

Quoted 18 lines
Quoted 18 lines

---

Reproduction/Verification Steps
The issue and the fix can be reproduced/verified by booting a VM with EFI and attempting to perform a KHO enabled kexec. The fix was developed/tested on arm64.

 drivers/firmware/efi/efi-init.c | 31 +++++++++++++++++++++++++++----
 1 file changed, 27 insertions(+), 4 deletions(-)
drivers/firmware/efi/efi-init.c +0 -0
index a00e07b853f22..2f08b1ab764f6 100644
@@ -164,12 +164,35 @@ static __init void reserve_regions(void)
 		pr_info("Processing EFI memory map:\n");
 
Quoted 17 lines

drivers/firmware/efi/efi-init.c static __init void reserve_regions(void) · line 174

Quoted 13 lines
drivers/firmware/efi/efi-init.c +6 -4
 	/*
-	 * Discard memblocks discovered so far: if there are any at this
-	 * point, they originate from memory nodes in the DT, and UEFI
-	 * uses its own memory map instead.
+	 * Discard memblocks discovered so far except for KHO scratch regions.
+	 * Most memblocks at this point originate from memory nodes in the DT,
+	 * and UEFI uses its own memory map instead. However, if KHO is enabled,
+	 * scratch regions must be preserved.
 	 */
 	memblock_dump_all();
-	memblock_remove(0, PHYS_ADDR_MAX);
+
+	if (IS_ENABLED(CONFIG_MEMBLOCK_KHO_SCRATCH)) {

It's better to condition this on kho_get_fdt() that means that we are actually doing a handover.

Hmm, I see that kho_get_fdt() is static. My first instinct was to use kho_enable() instead. Diving a bit more into the initialisation flow, during the setup_arch()->efi_init(), kho_enable() will return true if kho is enabled in the cmdline, but not if we are actually doing a KHO enabled kexec. However, in this case, the parsing of memory regions is going to be a noop in terms of functionality, but will contribute, negatively —though the overhead would likely be unmeasurable to the (cold) boot time. If we want to avoid that, we might consider adding another function to the KHO API, like is_booting_with_kho(), that practically wraps the kho_get_fdt(). IMO, it feels a bit cleaner this way, as other components don't necessarily (need to) know the internal FDT based implementation of KHO. That being said, I am definitely not the most experienced person when it comes to API design, so there is a high chance that I am way off :)

So to sum it up, I see three paths forward:
1. Condition with kho_is_enabled() instead of the CONFIG (accepting the minor cold boot overhead)
2. Post another patch that extends the KHO API, adding a wrapper for the kho_get_fdt(), like is_booting_with_kho() indicating that we are booting with KHO enabled
3. Post another patch that exports the kho_get_fdt() directly.

I am happy to implement any of the three, or any other suggestion you might have.

Quoted 9 lines

drivers/firmware/efi/efi-init.c static __init void reserve_regions(void) · line 180

Quoted 6 lines
drivers/firmware/efi/efi-init.c +6 -0
+		struct memblock_region *reg;
+		phys_addr_t start, size;
+		int i;
+
+		/* Remove all non-KHO regions */
+		for (i = memblock.memory.cnt - 1; i >= 0; i--) {

Please use for_each_mem_region()

Todo in Rev2.

Quoted 16 lines
Quoted 16 lines
drivers/firmware/efi/efi-init.c +15 -0
+			reg = &memblock.memory.regions[i];
+			if (!memblock_is_kho_scratch(reg)) {
+				start = reg->base;
+				size = reg->size;
+				memblock_remove(start, size);
+			}
+		}
+	} else {
+	/*
+	 * KHO is disabled. Discard memblocks discovered so far: if there
+	 * are any at this point, they originate from memory nodes in the
+	 * DT, and UEFI uses its own memory map instead.
+	 */
+		memblock_remove(0, PHYS_ADDR_MAX);
+	}
 
Quoted 10 lines
Quoted 4 lines
drivers/firmware/efi/efi-init.c +0 -1
 	for_each_efi_memory_desc(md) {
 		paddr = md->phys_addr;
--

2.43.0

--
Sincerely yours,
Mike.

--
Kind Regards,
Evangelos.

Amazon Web Services Development Center Germany GmbH Tamara-Danz-Str. 13
10243 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B
Sitz: Berlin
Ust-ID: DE 365 538 597

maintainer
#3 lore raw
Quoted 53 lines · Evangelos Petrongonas

Hey Mike, thanks for your review,

Quoted 12 lines · Mike Rapoport
Quoted 11 lines · Evangelos Petrongonas

When KHO (Kexec HandOver) is enabled, it sets up scratch memory regions early during device tree scanning. After kexec, the new kernel exclusively uses this region for memory allocations during boot up to the initialization of the page allocator

However, when booting with EFI, EFI's reserve_regions() uses memblock_remove(0, PHYS_ADDR_MAX) to clear all memory regions before rebuilding them from EFI data. This destroys KHO scratch regions and their flags, thus causing a kernel panic, as there are no scratch memory regions.

Quoted 6 lines
Quoted 3 lines

Instead of wholesale removal, iterate through memory regions and only remove non-KHO ones. This preserves KHO scratch regions while still allowing EFI to rebuild its memory map.

It's worth mentioning that scratch areas are "good known memory" :)

I Will do so on Rev2.

Quoted 12 lines

drivers/firmware/efi/efi-init.c static __init void reserve_regions(void) · line 174

Quoted 8 lines

---

drivers/firmware/efi/efi-init.c +2 -1
 	 */
 	memblock_dump_all();
-	memblock_remove(0, PHYS_ADDR_MAX);
+
+	if (IS_ENABLED(CONFIG_MEMBLOCK_KHO_SCRATCH)) {

It's better to condition this on kho_get_fdt() that means that we are actually doing a handover.

Hmm, I see that kho_get_fdt() is static. My first instinct was to use kho_enable() instead. Diving a bit more into the initialisation flow, during the setup_arch()->efi_init(), kho_enable() will return true if kho is enabled in the cmdline, but not if we are actually doing a KHO enabled kexec. However, in this case, the parsing of memory regions is going to be a noop in terms of functionality, but will contribute, negatively —though the overhead would likely be unmeasurable to the (cold) boot time. If we want to avoid that, we might consider adding another function to the KHO API, like is_booting_with_kho(), that practically wraps the kho_get_fdt(). IMO, it feels a bit cleaner this way, as other components don't necessarily (need to) know the internal FDT based implementation of KHO. That being said, I am definitely not the most experienced person when it comes to API design, so there is a high chance that I am way off :)

Quoted 7 lines

So to sum it up, I see three paths forward:
1. Condition with kho_is_enabled() instead of the CONFIG (accepting the minor cold boot overhead)
2. Post another patch that extends the KHO API, adding a wrapper for the kho_get_fdt(), like is_booting_with_kho() indicating that we are booting with KHO enabled
3. Post another patch that exports the kho_get_fdt() directly.

My preference is for the second option, I'd just name it is_kho_boot()

Quoted 18 lines

I am happy to implement any of the three, or any other suggestion you might have.

Quoted 9 lines

drivers/firmware/efi/efi-init.c static __init void reserve_regions(void) · line 180

Quoted 6 lines
drivers/firmware/efi/efi-init.c +6 -0
+		struct memblock_region *reg;
+		phys_addr_t start, size;
+		int i;
+
+		/* Remove all non-KHO regions */
+		for (i = memblock.memory.cnt - 1; i >= 0; i--) {

Please use for_each_mem_region()

Todo in Rev2.

--
Kind Regards,
Evangelos.

Signature · 2 lines
Sincerely yours,
Mike.