kho: skip memoryless NUMA nodes when reserving scratch areas
Subsystems (3)
PATCH v2 1/1 kho: skip memoryless NUMA nodes when reserving scratch areas
kho_reserve_scratch() iterates over all online NUMA nodes to allocate per-node scratch memory. On systems with memoryless NUMA nodes (nodes that have CPUs but no memory), memblock_alloc_range_nid() fails because there is no memory available on that node. This causes KHO initialization to fail and kho_enable to be set to false.
Some ARM64 systems have NUMA topologies where certain nodes contain only CPUs without any associated memory. These configurations are valid and should not prevent KHO from functioning.
Fix this by only counting nodes that have memory (N_MEMORY state) and skip memoryless nodes in the per-node scratch allocation loop.
---
v2:
- Removed kho_mem_nodes_count in favour of nodes_weight(nodes_state[N_MEMORY])
- Use for_each_node_state(nid, N_MEMORY) to loop over nodes that are both online and have memory.
TIL: Nodes in N_MEMORY are a subset of those that are online. Thanks Mike :)
kernel/liveupdate/kexec_handover.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
kernel/liveupdate/kexec_handover.c +6 -2
@@ -643,7 +643,7 @@ static void __init kho_reserve_scratch(void)scratch_size_update ();/* FIXME: deal with node hot-plug/remove */ - kho_scratch_cnt =num_online_nodes () +2 ; + kho_scratch_cnt =nodes_weight (node_states[N_MEMORY ]) +2 ; size = kho_scratch_cnt *sizeof (*kho_scratch); kho_scratch =memblock_alloc (size,PAGE_SIZE );if (!kho_scratch)
@@ -673,7 +673,11 @@ static void __init kho_reserve_scratch(void) kho_scratch[i].size = size; i++; -for_each_online_node (nid) { +/* +* Loop over nodes that have both memory and are online. Skip +* memoryless nodes, as we can not allocate scratch areas there. +*/ +for_each_node_state (nid,N_MEMORY ) { size =scratch_size_node (nid); addr = memblock_alloc_range_nid(size, CMA_MIN_ALIGNMENT_BYTES, 0, MEMBLOCK_ALLOC_ACCESSIBLE,
Signature · 12 lines
2.43.0 Amazon Web Services Development Center Germany GmbH Tamara-Danz-Str. 13 10243 Berlin Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B Sitz: Berlin Ust-ID: DE 365 538 597
Quoted 6 lines · Evangelos Petrongonas
kho_reserve_scratch()iterates over all online NUMA nodes to allocate per-node scratch memory. On systems with memoryless NUMA nodes (nodes that have CPUs but no memory),memblock_alloc_range_nid()fails because there is no memory available on that node. This causes KHO initialization to fail andkho_enableto be set to false.
Quoted 8 lines
Some ARM64 systems have NUMA topologies where certain nodes contain only CPUs without any associated memory. These configurations are valid and should not prevent KHO from functioning.
Fix this by only counting nodes that have memory (
N_MEMORYstate) and skip memoryless nodes in the per-node scratch allocation loop.Signed-off-by Evangelos Petrongonas <[email protected]>
[...]
Signature · 2 lines
Regards, Pratyush Yadav
Quoted 6 lines · Evangelos Petrongonas
kho_reserve_scratch()iterates over all online NUMA nodes to allocate per-node scratch memory. On systems with memoryless NUMA nodes (nodes that have CPUs but no memory),memblock_alloc_range_nid()fails because there is no memory available on that node. This causes KHO initialization to fail andkho_enableto be set to false.
Quoted 8 lines
Some ARM64 systems have NUMA topologies where certain nodes contain only CPUs without any associated memory. These configurations are valid and should not prevent KHO from functioning.
Fix this by only counting nodes that have memory (
N_MEMORYstate) and skip memoryless nodes in the per-node scratch allocation loop.Signed-off-by Evangelos Petrongonas <[email protected]>
Quoted 50 lines
---
v2:
- Removedkho_mem_nodes_countin favour ofnodes_weight(nodes_state[N_MEMORY])
- Usefor_each_node_state(nid, N_MEMORY)to loop over nodes that are both online and have memory.
TIL: Nodes inN_MEMORYare a subset of those that are online. Thanks Mike :)kernel/liveupdate/kexec_handover.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)kernel/liveupdate/kexec_handover.c +6
-2@@ -643,7 +643,7 @@ static void __init kho_reserve_scratch(void)scratch_size_update ();/* FIXME: deal with node hot-plug/remove */ - kho_scratch_cnt =num_online_nodes () +2 ; + kho_scratch_cnt =nodes_weight (node_states[N_MEMORY ]) +2 ; size = kho_scratch_cnt *sizeof (*kho_scratch); kho_scratch =memblock_alloc (size,PAGE_SIZE );if (!kho_scratch)@@ -673,7 +673,11 @@ static void __init kho_reserve_scratch(void) kho_scratch[i].size = size; i++; -for_each_online_node (nid) { +/* +* Loop over nodes that have both memory and are online. Skip +* memoryless nodes, as we can not allocate scratch areas there. +*/ +for_each_node_state (nid,N_MEMORY ) { size =scratch_size_node (nid); addr = memblock_alloc_range_nid(size, CMA_MIN_ALIGNMENT_BYTES, 0, MEMBLOCK_ALLOC_ACCESSIBLE,Signature · 12 lines
2.43.0 Amazon Web Services Development Center Germany GmbH Tamara-Danz-Str. 13 10243 Berlin Geschaeftsfuehrung: Christof Hellmis, Andreas Stieger Eingetragen am Amtsgericht Charlottenburg unter HRB 257764 B Sitz: Berlin Ust-ID: DE 365 538 597
Signature · 2 lines
Sincerely yours, Mike.
Quoted 41 lines · Evangelos Petrongonas
kho_reserve_scratch()iterates over all online NUMA nodes to allocate per-node scratch memory. On systems with memoryless NUMA nodes (nodes that have CPUs but no memory),memblock_alloc_range_nid()fails because there is no memory available on that node. This causes KHO initialization to fail andkho_enableto be set to false.Some ARM64 systems have NUMA topologies where certain nodes contain only CPUs without any associated memory. These configurations are valid and should not prevent KHO from functioning.
Fix this by only counting nodes that have memory (
N_MEMORYstate) and skip memoryless nodes in the per-node scratch allocation loop.Signed-off-by Evangelos Petrongonas <[email protected]>---
v2:
- Removedkho_mem_nodes_countin favour ofnodes_weight(nodes_state[N_MEMORY])
- Usefor_each_node_state(nid, N_MEMORY)to loop over nodes that are both online and have memory.
TIL: Nodes inN_MEMORYare a subset of those that are online. Thanks Mike :)kernel/liveupdate/kexec_handover.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)kernel/liveupdate/kexec_handover.c +1
-1@@ -643,7 +643,7 @@ static void __init kho_reserve_scratch(void)scratch_size_update ();/* FIXME: deal with node hot-plug/remove */ - kho_scratch_cnt =num_online_nodes () +2 ; + kho_scratch_cnt =nodes_weight (node_states[N_MEMORY ]) +2 ; size = kho_scratch_cnt *sizeof (*kho_scratch); kho_scratch =memblock_alloc (size,PAGE_SIZE );if (!kho_scratch)@@ -673,7 +673,11 @@ static void __init kho_reserve_scratch(void) kho_scratch[i].size = size; i++;
kernel/liveupdate/kexec_handover.c static void __init kho_reserve_scratch(void) · line 683
Quoted 10 lines
kernel/liveupdate/kexec_handover.c +5
-1-for_each_online_node (nid) { +/* +* Loop over nodes that have both memory and are online. Skip +* memoryless nodes, as we can not allocate scratch areas there. +*/ +for_each_node_state (nid,N_MEMORY ) { size =scratch_size_node (nid); addr = memblock_alloc_range_nid(size, CMA_MIN_ALIGNMENT_BYTES, 0, MEMBLOCK_ALLOC_ACCESSIBLE,
Andrew Morton
maintainerQuoted 6 lines · On Tue, 20 Jan 2026 17:59:11 +0000 Evangelos Petrongonas <[email protected]>…
kho_reserve_scratch()iterates over all online NUMA nodes to allocate per-node scratch memory. On systems with memoryless NUMA nodes (nodes that have CPUs but no memory),memblock_alloc_range_nid()fails because there is no memory available on that node. This causes KHO initialization to fail andkho_enableto be set to false.
Quoted 7 lines
Some ARM64 systems have NUMA topologies where certain nodes contain only CPUs without any associated memory. These configurations are valid and should not prevent KHO from functioning.
Fix this by only counting nodes that have memory (
N_MEMORYstate) and skip memoryless nodes in the per-node scratch allocation loop.
So kho is unusable on such machines.
Should we backport this? I'm thinking
Mike Rapoport
maintainerQuoted 16 lines · Andrew Morton
Quoted 6 lines · On Tue, 20 Jan 2026 17:59:11 +0000 Evangelos Petrongonas <[email protected]>…
kho_reserve_scratch()iterates over all online NUMA nodes to allocate per-node scratch memory. On systems with memoryless NUMA nodes (nodes that have CPUs but no memory),memblock_alloc_range_nid()fails because there is no memory available on that node. This causes KHO initialization to fail andkho_enableto be set to false.Quoted 7 lines
Some ARM64 systems have NUMA topologies where certain nodes contain only CPUs without any associated memory. These configurations are valid and should not prevent KHO from functioning.
Fix this by only counting nodes that have memory (
N_MEMORYstate) and skip memoryless nodes in the per-node scratch allocation loop.
Quoted 5 lines
So kho is unusable on such machines.
Should we backport this? I'm thinking
Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers").
It's only for v6.18, but sure, why not.
Signature · 2 lines
Sincerely yours, Mike.