Skip to contentSkip to conversation

kho: do not reserve empty per-node bootmem areas

PATCH Applied Pratyush Yadav 3 participants 6 messages Oct 14 – Oct 15

Subsystems (3)
lore
Share & tools
lore

Pratyush Yadav

author maintainer
#0 lore raw

PATCH 1/1 kho: do not reserve empty per-node bootmem areas

From: "Pratyush Yadav (Google)" <[email protected]>

Some nodes do not allocate any early boot memory. An example is hotpluggable nodes with movable_node enabled. So the scratch size for them ends up being zero. Zero size allocations normally don't play well with memblock. Memblock returns an address, but then doesn't actually reserve anything. This address ends up being recorded in kho_bootmem with a size of 0. In case of hotpluggable nodes with movable_nodes, the allocation on the node actually fails because memblock refuses to allocate there, and so KHO fails to initialize.

Skip bootmem reservation for such nodes entirely.

To keep the code more robust, avoid overloading kho_bootmem_cnt. use max_nr_bootmem to store the size of kho_bootmem array. This ensures that the location of kho_bootmem_cnt assignment does not affect the cleanup paths.

Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")
---
 kernel/liveupdate/kexec_handover.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
kernel/liveupdate/kexec_handover.c +13 -4
index 7018e25e4eda..6f1f55fe3181 100644
@@ -831,8 +831,8 @@ bool kho_bootmem_overlap(phys_addr_t phys, size_t size)
  */
 static void __init kho_reserve_bootmem(void)
 {
+	int nid, i = 0, max_nr_bootmem;
 	phys_addr_t addr, size;
-	int nid, i = 0;
 
 	if (!kho_enable)
 		return;
@@ -840,8 +840,8 @@ static void __init kho_reserve_bootmem(void)
 	bootmem_size_update();
 
 	/* FIXME: deal with node hot-plug/remove */
-	kho_bootmem_cnt = nodes_weight(node_states[N_MEMORY]) + 2;
-	size = kho_bootmem_cnt * sizeof(*kho_bootmem);
+	max_nr_bootmem = nodes_weight(node_states[N_MEMORY]) + 2;
+	size = max_nr_bootmem * sizeof(*kho_bootmem);
 	kho_bootmem = memblock_alloc(size, PAGE_SIZE);
 	if (!kho_bootmem) {
 		pr_err("Failed to reserve bootmem array\n");
@@ -882,6 +882,9 @@ static void __init kho_reserve_bootmem(void)
 	 */
 	for_each_node_state(nid, N_MEMORY) {
 		size = bootmem_size_node(nid);
+		if (!size)
+			continue;
+
 		addr = memblock_alloc_range_nid(size, BOOTMEM_ALIGNMENT_BYTES,
 						0, MEMBLOCK_ALLOC_ACCESSIBLE,
 						nid, true);
@@ -895,13 +898,19 @@ static void __init kho_reserve_bootmem(void)
 		i++;
 	}
 
+	/*
+	 * Nodes that needed no bootmem were skipped, so fewer areas than
+	 * initially estimated may have been reserved.
+	 */
+	kho_bootmem_cnt = i;
+
 	return;
 
 err_free_bootmem_areas:
 	for (i--; i >= 0; i--)
 		memblock_phys_free(kho_bootmem[i].addr, kho_bootmem[i].size);
 err_free_bootmem_desc:
-	memblock_free(kho_bootmem, kho_bootmem_cnt * sizeof(*kho_bootmem));
+	memblock_free(kho_bootmem, max_nr_bootmem * sizeof(*kho_bootmem));
 err_disable_kho:
 	pr_warn("Failed to reserve bootmem, disabling kexec handover\n");
 	kho_enable = false;

base-commit: e1fe7daedae7a4a23aca6dbd58e1746750b632dc

Signature · 1 line
2.56.0.385.gd3acb90ef8-goog
#1 lore raw
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] kho: incomplete fix for zero-size memblock allocations in bootmem --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] kho: incomplete fix for zero-size memblock allocations in bootmem --

commit 67ab500bd948f5a3b52f1c8170626cb8f1dfc6c3
Author: Pratyush Yadav (Google) <[email protected]>

kho: do not reserve empty per-node bootmem areas

Some nodes do not allocate any early boot memory, causing memblock to return an address without reserving anything when requested with zero size. This address is recorded in kho_bootmem with a size of 0, and can cause KHO to fail to initialize. This patch skips bootmem reservation for memoryless nodes entirely.

Quoted 4 lines
kernel/liveupdate/kexec_handover.c +0 -0
index 7018e25e4eda1..6f1f55fe31819 100644

[ ... ]

kernel/liveupdate/kexec_handover.c static void __init kho_reserve_bootmem(void) · line 888

Quoted 8 lines
kernel/liveupdate/kexec_handover.c +3 -0
@@ -882,6 +882,9 @@ static void __init kho_reserve_bootmem(void)
 	 */
 	for_each_node_state(nid, N_MEMORY) {
 		size = bootmem_size_node(nid);
+		if (!size)
+			continue;
+
 		addr = memblock_alloc_range_nid(size, BOOTMEM_ALIGNMENT_BYTES,

[Severity: High]
This adds a zero-size check for per-node bootmem allocations, but does the same issue exist for bootmem_size_lowmem and bootmem_size_global?

Looking at kho_reserve_bootmem() just before this hunk, they are allocated without a similar check:

kernel/liveupdate/kexec_handover.c:kho_reserve_bootmem() {
    ...
	size = bootmem_size_lowmem;
	addr = memblock_phys_alloc_range(size, BOOTMEM_ALIGNMENT_BYTES, 0,
					 ARCH_LOW_ADDRESS_LIMIT);
	if (!addr) {
    ...
	size = bootmem_size_global;
	addr = memblock_phys_alloc(size, BOOTMEM_ALIGNMENT_BYTES);
	if (!addr) {
    ...
}

If bootmem_size_lowmem or bootmem_size_global evaluates to 0 (for example via boot parameters or on systems lacking lowmem), will memblock return an unreserved address and add it to kho_bootmem with size 0? Alternatively, if the region has no free memory, could the 0-size allocation fail and halt KHO initialization entirely?

Later, if KHO attempts to preserve a contiguous range of pages that spans across this unreserved boundary, could kho_bootmem_overlap() incorrectly evaluate the overlap and fail the preservation request?

Quoted 3 lines
kernel/liveupdate/kexec_handover.c +0 -0
 						0, MEMBLOCK_ALLOC_ACCESSIBLE,
 						nid, true);

if (!addr) {

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author maintainer
#2 lore raw
Quoted 22 lines · On Wed, Oct 14 2026, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] kho: incomplete fix for zero-size memblock allocations in bootmem --

commit 67ab500bd948f5a3b52f1c8170626cb8f1dfc6c3
Author: Pratyush Yadav (Google) <[email protected]>

kho: do not reserve empty per-node bootmem areas

Some nodes do not allocate any early boot memory, causing memblock to return an address without reserving anything when requested with zero size. This address is recorded in kho_bootmem with a size of 0, and can cause KHO to fail to initialize. This patch skips bootmem reservation for memoryless nodes entirely.

Quoted 4 lines
kernel/liveupdate/kexec_handover.c +0 -0
index 7018e25e4eda1..6f1f55fe31819 100644

[ ... ]

Quoted 12 lines

kernel/liveupdate/kexec_handover.c static void __init kho_reserve_bootmem(void) · line 888

Quoted 8 lines
kernel/liveupdate/kexec_handover.c +3 -0
@@ -882,6 +882,9 @@ static void __init kho_reserve_bootmem(void)
 	 */
 	for_each_node_state(nid, N_MEMORY) {
 		size = bootmem_size_node(nid);
+		if (!size)
+			continue;
+
 		addr = memblock_alloc_range_nid(size, BOOTMEM_ALIGNMENT_BYTES,

[Severity: High]
This adds a zero-size check for per-node bootmem allocations, but does the same issue exist for bootmem_size_lowmem and bootmem_size_global?

Valid point, but that should be solved by the general bootmem sizing patches being developed separately.

[...]

Signature · 2 lines
Regards,
Pratyush Yadav
maintainer
#3 lore raw
In reply to Pratyush Yadav
Quoted 4 lines · Pratyush Yadav

From: "Pratyush Yadav (Google)" <[email protected]>

Some nodes do not allocate any early boot memory. An example is hotpluggable nodes with movable_node enabled. So the scratch size for

Let's use the new naming please ^ ;-)

I'll fix it up when applying.

Quoted 82 lines

them ends up being zero. Zero size allocations normally don't play well with memblock. Memblock returns an address, but then doesn't actually reserve anything. This address ends up being recorded in kho_bootmem with a size of 0. In case of hotpluggable nodes with movable_nodes, the allocation on the node actually fails because memblock refuses to allocate there, and so KHO fails to initialize.

Skip bootmem reservation for such nodes entirely.

To keep the code more robust, avoid overloading kho_bootmem_cnt. use max_nr_bootmem to store the size of kho_bootmem array. This ensures that the location of kho_bootmem_cnt assignment does not affect the cleanup paths.

Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")
---
 kernel/liveupdate/kexec_handover.c | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)
kernel/liveupdate/kexec_handover.c +13 -4
index 7018e25e4eda..6f1f55fe3181 100644
@@ -831,8 +831,8 @@ bool kho_bootmem_overlap(phys_addr_t phys, size_t size)
  */
 static void __init kho_reserve_bootmem(void)
 {
+	int nid, i = 0, max_nr_bootmem;
 	phys_addr_t addr, size;
-	int nid, i = 0;
 
 	if (!kho_enable)
 		return;
@@ -840,8 +840,8 @@ static void __init kho_reserve_bootmem(void)
 	bootmem_size_update();
 
 	/* FIXME: deal with node hot-plug/remove */
-	kho_bootmem_cnt = nodes_weight(node_states[N_MEMORY]) + 2;
-	size = kho_bootmem_cnt * sizeof(*kho_bootmem);
+	max_nr_bootmem = nodes_weight(node_states[N_MEMORY]) + 2;
+	size = max_nr_bootmem * sizeof(*kho_bootmem);
 	kho_bootmem = memblock_alloc(size, PAGE_SIZE);
 	if (!kho_bootmem) {
 		pr_err("Failed to reserve bootmem array\n");
@@ -882,6 +882,9 @@ static void __init kho_reserve_bootmem(void)
 	 */
 	for_each_node_state(nid, N_MEMORY) {
 		size = bootmem_size_node(nid);
+		if (!size)
+			continue;
+
 		addr = memblock_alloc_range_nid(size, BOOTMEM_ALIGNMENT_BYTES,
 						0, MEMBLOCK_ALLOC_ACCESSIBLE,
 						nid, true);
@@ -895,13 +898,19 @@ static void __init kho_reserve_bootmem(void)
 		i++;
 	}
 
+	/*
+	 * Nodes that needed no bootmem were skipped, so fewer areas than
+	 * initially estimated may have been reserved.
+	 */
+	kho_bootmem_cnt = i;
+
 	return;
 
 err_free_bootmem_areas:
 	for (i--; i >= 0; i--)
 		memblock_phys_free(kho_bootmem[i].addr, kho_bootmem[i].size);
 err_free_bootmem_desc:
-	memblock_free(kho_bootmem, kho_bootmem_cnt * sizeof(*kho_bootmem));
+	memblock_free(kho_bootmem, max_nr_bootmem * sizeof(*kho_bootmem));
 err_disable_kho:
 	pr_warn("Failed to reserve bootmem, disabling kexec handover\n");
 	kho_enable = false;

base-commit: e1fe7daedae7a4a23aca6dbd58e1746750b632dc

Signature · 1 line
2.56.0.385.gd3acb90ef8-goog
Signature · 2 lines
Sincerely yours,
Mike.
author maintainer
#4 lore raw
Quoted 7 lines · Mike Rapoport
Quoted 4 lines · Pratyush Yadav

From: "Pratyush Yadav (Google)" <[email protected]>

Some nodes do not allocate any early boot memory. An example is hotpluggable nodes with movable_node enabled. So the scratch size for

Let's use the new naming please ^ ;-)

Argh, muscle memory...

Quoted 2 lines

I'll fix it up when applying.

Thanks!

Quoted 14 lines
Quoted 14 lines

them ends up being zero. Zero size allocations normally don't play well with memblock. Memblock returns an address, but then doesn't actually reserve anything. This address ends up being recorded in kho_bootmem with a size of 0. In case of hotpluggable nodes with movable_nodes, the allocation on the node actually fails because memblock refuses to allocate there, and so KHO fails to initialize.

Skip bootmem reservation for such nodes entirely.

To keep the code more robust, avoid overloading kho_bootmem_cnt. use max_nr_bootmem to store the size of kho_bootmem array. This ensures that the location of kho_bootmem_cnt assignment does not affect the cleanup paths.

[...]

Signature · 2 lines
Regards,
Pratyush Yadav
maintainer Applied
#5 lore raw
In reply to Pratyush Yadav

From: Mike Rapoport (Microsoft) <[email protected]>

Quoted 10 lines · Pratyush Yadav

Some nodes do not allocate any early boot memory. An example is hotpluggable nodes with movable_node enabled. So the scratch size for them ends up being zero. Zero size allocations normally don't play well with memblock. Memblock returns an address, but then doesn't actually reserve anything. This address ends up being recorded in kho_bootmem with a size of 0. In case of hotpluggable nodes with movable_nodes, the allocation on the node actually fails because memblock refuses to allocate there, and so KHO fails to initialize.

[...]

Applied to next branch of liveupdate/linux.git tree, thanks!

[1/1] kho: do not reserve empty per-node bootmem areas

      commit: 9a6a70aefe77f6fb9ff2b3037dcf87f6dff65c8c

tree: https://git.kernel.org/pub/scm/linux/kernel/git/liveupdate/linux
branch: next

--
Sincerely yours,
Mike.