liveupdate: kho: calculate per-node scratch sizes before allocation
Subsystems (4)
PATCH 1/1 liveupdate: kho: calculate per-node scratch sizes before allocation
From: George Guo <[email protected]>
The default percentage-based policy sizes scratch areas from the current kernel's MEMBLOCK_RSRV_KERN footprint. This is a reasonable heuristic for predicting the early memory demand of the next kernel.
scratch_size_update() calculates the lowmem and global sizes before either area is allocated. However, kho_reserve_scratch() calculates each per-node size only after allocating the lowmem and global areas. Since memblock allocations are marked MEMBLOCK_RSRV_KERN, the per-node calculation includes those newly allocated scratch areas and scales them again.
This feedback substantially inflates the per-node request. On a 1 GiB LoongArch QEMU guest, the relevant reservation baseline is about 98.45 MiB. With the default 200% scale and 32 MiB alignment, the old ordering allocates 224 MiB of lowmem scratch, then requests 672 MiB for node 0, for a total of 896 MiB. The node allocation fails and KHO is disabled. The same incorrect calculation is hidden on the tested 1 GiB x86 guest because its baseline is smaller and its memory map can satisfy the inflated request.
Calculate and save all per-node sizes in the scratch descriptors before reserving any scratch areas, then use the saved sizes during allocation. This keeps the percentage heuristic while preventing scratch memory from becoming input to the sizing of more scratch memory. On the LoongArch guest, the aligned total is reduced from 896 MiB to 448 MiB.
The KHO vmtest passes on 1 GiB LoongArch and x86 QEMU guests with this change.
--- kernel/liveupdate/kexec_handover.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
kernel/liveupdate/kexec_handover.c +12 -1
@@ -847,6 +847,17 @@ static void __init kho_reserve_scratch(void) goto err_disable_kho; } +/* +* Calculate the per-node sizes before reserving any scratch areas. +* memblock allocations are marked MEMBLOCK_RSRV_KERN, so calculating +* them later would count the lowmem and global scratch areas as kernel +* allocations and scale them again. +*/ + i =2 ; +for_each_node_state (nid,N_MEMORY ) + kho_scratch[i++].size =scratch_size_node (nid); + i =0 ; +/* * reserve scratch area in low memory for lowmem allocations in the * next kernel
@@ -880,7 +891,7 @@ static void __init kho_reserve_scratch(void)* memoryless nodes, as we can not allocate scratch areas there. */ for_each_node_state (nid,N_MEMORY ) { - size =scratch_size_node (nid); + size = kho_scratch[i].size; addr =memblock_alloc_range_nid (size,SCRATCH_ALIGNMENT_BYTES ,0 ,MEMBLOCK_ALLOC_ACCESSIBLE , nid, true);
Signature · 1 line
2.53.0
Mike Rapoport
maintainerHi George,
Quoted 21 lines · George Guo
From: George Guo <[email protected]>
The default percentage-based policy sizes scratch areas from the current kernel's
MEMBLOCK_RSRV_KERNfootprint. This is a reasonable heuristic for predicting the early memory demand of the next kernel.
scratch_size_update()calculates the lowmem and global sizes before either area is allocated. However,kho_reserve_scratch()calculates each per-node size only after allocating the lowmem and global areas. Since memblock allocations are markedMEMBLOCK_RSRV_KERN, the per-node calculation includes those newly allocated scratch areas and scales them again.This feedback substantially inflates the per-node request. On a 1 GiB LoongArch QEMU guest, the relevant reservation baseline is about 98.45 MiB. With the default 200% scale and 32 MiB alignment, the old ordering allocates 224 MiB of lowmem scratch, then requests 672 MiB for node 0, for a total of 896 MiB. The node allocation fails and KHO is disabled. The same incorrect calculation is hidden on the tested 1 GiB x86 guest because its baseline is smaller and its memory map can satisfy the inflated request.
Quoted 8 lines
Calculate and save all per-node sizes in the scratch descriptors before reserving any scratch areas, then use the saved sizes during allocation. This keeps the percentage heuristic while preventing scratch memory from becoming input to the sizing of more scratch memory. On the LoongArch guest, the aligned total is reduced from 896 MiB to 448 MiB.
The KHO vmtest passes on 1 GiB LoongArch and x86 QEMU guests with this change.
This reads as LLM-generated text. Please add LLM attribution as per
https://docs.kernel.org/process/coding-assistants.html#attribution
Quoted 17 lines
Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")Reported-by Kexin Liu <[email protected]>Co-developed-by Kexin Liu <[email protected]>Signed-off-by Kexin Liu <[email protected]>Signed-off-by George Guo <[email protected]>--- kernel/liveupdate/kexec_handover.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)kernel/liveupdate/kexec_handover.c +0
-0@@ -847,6 +847,17 @@ static void __init kho_reserve_scratch(void) goto err_disable_kho; }
kernel/liveupdate/kexec_handover.c static void __init kho_reserve_scratch(void) · line 859
Quoted 10 lines
kernel/liveupdate/kexec_handover.c +10
-0+/* +* Calculate the per-node sizes before reserving any scratch areas. +* memblock allocations are marked MEMBLOCK_RSRV_KERN, so calculating +* them later would count the lowmem and global scratch areas as kernel +* allocations and scale them again. +*/ + i =2 ; +for_each_node_state (nid,N_MEMORY ) + kho_scratch[i++].size =scratch_size_node (nid); + i =0 ;
Ugh, this really does not look nice.
Can't we just calculated all the sizes first and than do the allocations?
Quoted 16 lines
kernel/liveupdate/kexec_handover.c +2
-1+/* * reserve scratch area in low memory for lowmem allocations in the * next kernel @@ -880,7 +891,7 @@ static void __init kho_reserve_scratch(void)* memoryless nodes, as we can not allocate scratch areas there. */ for_each_node_state (nid,N_MEMORY ) { - size =scratch_size_node (nid); + size = kho_scratch[i].size; addr =memblock_alloc_range_nid (size,SCRATCH_ALIGNMENT_BYTES ,0 ,MEMBLOCK_ALLOC_ACCESSIBLE , nid, true);Signature · 1 line
2.53.0
Signature · 2 lines
Sincerely yours, Mike.
Hi Mike,
Quoted 3 lines
This reads as LLM-generated text. Please add LLM attribution as per
https://docs.kernel.org/process/coding-assistants.html#attribution
I'll add the Assisted-by tag.
Quoted 1 line
Can't we just calculated all the sizes first and than do the allocations?
Yes, I'll rework this to calculate all scratch sizes together before allocating the scratch areas.
Adding Sourabh to Cc, since this relates to the scratch-sizing discussion in the PowerPC KHO thread.
Sourabh, this patch prevents newly allocated scratch areas from being counted in the per-node baseline. It does not address the overlap between global and per-node accounting. Could you review this approach as well?
Thanks,
George
Quoted 11 lines · George Guo
Hi Mike,
Quoted 3 lines
This reads as LLM-generated text. Please add LLM attribution as per
https://docs.kernel.org/process/coding-assistants.html#attribution
I'll add the Assisted-by tag.
Quoted 1 line
Can't we just calculated all the sizes first and than do the allocations?
Yes, I'll rework this to calculate all scratch sizes together before allocating the scratch areas.
Quoted 6 lines
Adding Sourabh to Cc, since this relates to the scratch-sizing discussion in the PowerPC KHO thread.
Sourabh, this patch prevents newly allocated scratch areas from being counted in the per-node baseline. It does not address the overlap between global and per-node accounting. Could you review this approach as well?
Sure I will review your patch.
- Sourabh Jain
Hello George,
Quoted 11 lines · George Guo
From: George Guo <[email protected]>
The default percentage-based policy sizes scratch areas from the current kernel's
MEMBLOCK_RSRV_KERNfootprint. This is a reasonable heuristic for predicting the early memory demand of the next kernel.
scratch_size_update()calculates the lowmem and global sizes before either area is allocated. However,kho_reserve_scratch()calculates each per-node size only after allocating the lowmem and global areas. Since memblock allocations are markedMEMBLOCK_RSRV_KERN, the per-node calculation includes those newly allocated scratch areas and scales them again.
I may be missing something here, but doesn't memblock_reserved_kern_size() check the node ID of the region before checking the region flag (MEMBLOCK_RSRV_KERN)?
Code snippet from memblock_reserved_kern_size()
``` if (nid == memblock_get_region_node(r) || !numa_valid_node(nid)) if (r->flags & MEMBLOCK_RSRV_KERN) total += size; ```
For a valid nid, my understanding is that the global and lowmem scratch areas should not be counted because they are allocated with NUMA_NO_NODE (-1). So, ideally, these regions should be excluded when calculating the reserved memory for a specific node ID.
Based on this, I am not sure that marking the lowmem and global areas as MEMBLOCK_RSRV_KERN is what causes the per-node size calculation to be inflated. I am looking into the code further to better understand the actual cause of the issue that this patch is trying to address.
With that said, I wonder if this fix might be more of a stop-gap solution. As mentioned above, since NUMA_NO_NODE (-1) is used for the lowmem and global allocations, my understanding is that these areas ideally should not be included when calculating the size for a specific node ID.
I could be missing something in my understanding, so I would appreciate your thoughts on these observations.
- Sourabh Jain
Quoted 40 lines
Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")Reported-by Kexin Liu <[email protected]>Co-developed-by Kexin Liu <[email protected]>Signed-off-by Kexin Liu <[email protected]>Signed-off-by George Guo <[email protected]>--- kernel/liveupdate/kexec_handover.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)kernel/liveupdate/kexec_handover.c +12
-1@@ -847,6 +847,17 @@ static void __init kho_reserve_scratch(void) goto err_disable_kho; } +/* +* Calculate the per-node sizes before reserving any scratch areas. +* memblock allocations are marked MEMBLOCK_RSRV_KERN, so calculating +* them later would count the lowmem and global scratch areas as kernel +* allocations and scale them again. +*/ + i =2 ; +for_each_node_state (nid,N_MEMORY ) + kho_scratch[i++].size =scratch_size_node (nid); + i =0 ; +/* * reserve scratch area in low memory for lowmem allocations in the * next kernel @@ -880,7 +891,7 @@ static void __init kho_reserve_scratch(void)* memoryless nodes, as we can not allocate scratch areas there. */ for_each_node_state (nid,N_MEMORY ) { - size =scratch_size_node (nid); + size = kho_scratch[i].size; addr =memblock_alloc_range_nid (size,SCRATCH_ALIGNMENT_BYTES ,0 ,MEMBLOCK_ALLOC_ACCESSIBLE , nid, true);
Hello George,
Quoted 30 lines · Sourabh Jain
Hello George,
Quoted 11 lines · George Guo
From: George Guo <[email protected]>
The default percentage-based policy sizes scratch areas from the current kernel's
MEMBLOCK_RSRV_KERNfootprint. This is a reasonable heuristic for predicting the early memory demand of the next kernel.
scratch_size_update()calculates the lowmem and global sizes before either area is allocated. However,kho_reserve_scratch()calculates each per-node size only after allocating the lowmem and global areas. Since memblock allocations are markedMEMBLOCK_RSRV_KERN, the per-node calculation includes those newly allocated scratch areas and scales them again.I may be missing something here, but doesn't
memblock_reserved_kern_size()check the node ID of the region before checking the region flag (MEMBLOCK_RSRV_KERN)?Code snippet from
memblock_reserved_kern_size()``` if (nid == memblock_get_region_node(r) || !numa_valid_node(nid)) if (r->flags & MEMBLOCK_RSRV_KERN) total += size; ```For a valid nid, my understanding is that the global and lowmem scratch areas should not be counted because they are allocated with
NUMA_NO_NODE(-1). So, ideally, these regions should be excluded when calculating the reserved memory for a specific node ID.
Quoted 4 lines
Based on this, I am not sure that marking the lowmem and global areas as
MEMBLOCK_RSRV_KERNis what causes the per-node size calculation to be inflated. I am looking into the code further to better understand the actual cause of the issue that this patch is trying to address.
I added some prints in kho_reserve_scratch() and found that the per-node size calculation is not impacted by the lowmem and global scratch memory allocations.
KHO: Before low and global scratch allocations
KHO: low size = 899 KB
KHO: global size = 137 MB
KHO: Per node 2 = 80 MB
KHO: After low and global scratch allocations
KHO: low size = 312195 KB
KHO: global size = 441 MB
KHO: Per node 2 = 80 MB
KHO: After per node allocation
KHO: low size = 394115 KB
KHO: global size = 521 MB
KHO: Per node 2 = 240 MB
I only had one NUMA node (nid=2), and the per-NUMA allocation before and after the lowmem and global scratch memory allocations remained the same at 80 MB.
The experiment was done on the PowerPC architecture.
I am wondering how the per-NUMA allocation in your setup is getting inflated due to the lowmem and global scratch memory reservations.
Feel free to use the change below to print the similar states in your setup.
index 9260e601c..b5bd9617b 100644
kernel/liveupdate/kexec_handover.c +0 -0
@@ -670,6 +670,24 @@ static phys_addr_t __init scratch_size_node(int nid)
return round_up(size, SCRATCH_ALIGNMENT_BYTES); }
+static void __init scratch_size_print(char *s)
+{
+ int nid;
+ phys_addr_t size;
+
+ pr_info("%s\n", s);
+ size = memblock_reserved_kern_size(ARCH_LOW_ADDRESS_LIMIT, NUMA_NO_NODE); + pr_info("low size = %llu KB\n", (unsigned long long)(size >> 10)); +
+ size = memblock_reserved_kern_size(MEMBLOCK_ALLOC_ANYWHERE, NUMA_NO_NODE); + pr_info("global size = %llu MB\n", (unsigned long long)(size >> 20)); +
+ for_each_node_state(nid, N_MEMORY) {
+ size = scratch_size_node(nid);
+ pr_info("Per node %d = %llu MB\n", nid, (unsigned long long)(size >> 20)); + }
+}
+
/**
* kho_reserve_scratch - Reserve a contiguous chunk of memory for kexec
*
@@ -698,6 +716,7 @@ static void __init kho_reserve_scratch(void)
goto err_disable_kho;
}
+ scratch_size_print("Before low and global scratch allocations"); /*
* reserve scratch area in low memory for lowmem allocations in the
* next kernel
@@ -730,6 +749,8 @@ static void __init kho_reserve_scratch(void)
* Loop over nodes that have both memory and are online. Skip
* memoryless nodes, as we can not allocate scratch areas there. */
+
+ scratch_size_print("After low and global scratch allocations"); for_each_node_state(nid, N_MEMORY) {
size = scratch_size_node(nid);
addr = memblock_alloc_range_nid(size, SCRATCH_ALIGNMENT_BYTES, @@ -744,6 +765,7 @@ static void __init kho_reserve_scratch(void) kho_scratch[i].size = size;
i++;
}
+ scratch_size_print("After per node allocation");
return;
Quoted 53 lines
With that said, I wonder if this fix might be more of a stop-gap solution. As mentioned above, since
NUMA_NO_NODE(-1) is used for the lowmem and global allocations, my understanding is that these areas ideally should not be included when calculating the size for a specific node ID.I could be missing something in my understanding, so I would appreciate your thoughts on these observations.
- Sourabh Jain
Quoted 41 lines
Fixes 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")Reported-by Kexin Liu <[email protected]>Co-developed-by Kexin Liu <[email protected]>Signed-off-by Kexin Liu <[email protected]>Signed-off-by George Guo <[email protected]>---
kernel/liveupdate/kexec_handover.c| 13 ++++++++++++-1 file changed, 12 insertions(+), 1 deletion(-)kernel/liveupdate/kexec_handover.c +0
-0@@ -847,6 +847,17 @@ static void __init kho_reserve_scratch(void) goto err_disable_kho; }+ /*
+ * Calculate the per-node sizes before reserving any scratch areas. + * memblock allocations are markedMEMBLOCK_RSRV_KERN, so calculating
+ * them later would count the lowmem and global scratch areas as kernel
+ * allocations and scale them again.
+ */
+ i = 2;
+for_each_node_state(nid, N_MEMORY)
+ kho_scratch[i++].size =scratch_size_node(nid);
+ i = 0;
+
/*
* reserve scratch area in low memory for lowmem allocations in the
* next kernelkernel/liveupdate/kexec_handover.c +1
-1@@ -880,7 +891,7 @@ static void __init kho_reserve_scratch(void)* memoryless nodes, as we can not allocate scratch areas there. */ for_each_node_state(nid, N_MEMORY) { - size = scratch_size_node(nid); + size = kho_scratch[i].size; addr = memblock_alloc_range_nid(size, SCRATCH_ALIGNMENT_BYTES, 0, MEMBLOCK_ALLOC_ACCESSIBLE, nid, true);
Pratyush Yadav
maintainerQuoted 61 lines · Sourabh Jain
Hello George,
Quoted 31 lines · Sourabh Jain
Hello George,
Quoted 11 lines · George Guo
From: George Guo <[email protected]>
The default percentage-based policy sizes scratch areas from the current kernel's
MEMBLOCK_RSRV_KERNfootprint. This is a reasonable heuristic for predicting the early memory demand of the next kernel.
scratch_size_update()calculates the lowmem and global sizes before either area is allocated. However,kho_reserve_scratch()calculates each per-node size only after allocating the lowmem and global areas. Since memblock allocations are markedMEMBLOCK_RSRV_KERN, the per-node calculation includes those newly allocated scratch areas and scales them again.I may be missing something here, but doesn't
memblock_reserved_kern_size()check the node ID of the region before checking the region flag (MEMBLOCK_RSRV_KERN)?Code snippet from
memblock_reserved_kern_size()``` if (nid == memblock_get_region_node(r) || !numa_valid_node(nid)) if (r->flags & MEMBLOCK_RSRV_KERN) total += size; ```For a valid nid, my understanding is that the global and lowmem scratch areas should not be counted because they are allocated with
NUMA_NO_NODE(-1). So, ideally, these regions should be excluded when calculating the reserved memory for a specific node ID.Quoted 4 lines
Based on this, I am not sure that marking the lowmem and global areas as
MEMBLOCK_RSRV_KERNis what causes the per-node size calculation to be inflated. I am looking into the code further to better understand the actual cause of the issue that this patch is trying to address.I added some prints in
kho_reserve_scratch()and found that the per-node size calculation is not impacted by the lowmem and global scratch memory allocations.KHO: Before low and global scratch allocations
KHO: low size = 899 KB
KHO: global size = 137 MB
KHO: Per node 2 = 80 MBKHO: After low and global scratch allocations
KHO: low size = 312195 KB
KHO: global size = 441 MB
KHO: Per node 2 = 80 MBKHO: After per node allocation
KHO: low size = 394115 KB
KHO: global size = 521 MB
KHO: Per node 2 = 240 MBI only had one NUMA node (nid=2), and the per-NUMA allocation before and after the lowmem and global scratch memory allocations remained the same at 80 MB.
Quoted 5 lines
The experiment was done on the PowerPC architecture.
I am wondering how the per-NUMA allocation in your setup is getting inflated due to the lowmem and global scratch memory reservations.
I had the same question, so I asked AI. Here's what it says:
--- 8< ---
The bug was reported and tested using the KHO self-test runner (tools/testing/selftests/kho/vmtest.sh), which builds a test kernel using make olddefconfig with only a minimal set of CONFIG_* options. Crucially, CONFIG_NUMA is not enabled.
When CONFIG_NUMA is disabled:
#ifndef CONFIG_NUMA
static inline void memblock_set_region_node(struct memblock_region *r, int nid)
{
}
static inline int memblock_get_region_node(const struct memblock_region *r)
{
return 0;
}
#endifstruct memblock_region does not even contain an nid member.
memblock_set_region_node() is a no-op (the NUMA_NO_NODE argument is simply discarded), and memblock_get_region_node() is hardcoded to always return 0.
There is only one node (nid = 0), so for_each_node_state(nid, N_MEMORY) loops once for nid = 0.
Both memblock_phys_alloc_range() and memblock_phys_alloc() mark their allocations with MEMBLOCK_RSRV_KERN. Therefore, when scratch_size_node(0) runs after allocating the lowmem scratch buffer, memblock_get_region_node(r) returns 0 for that lowmem scratch buffer, and r->flags & MEMBLOCK_RSRV_KERN is true. As a result, scratch_size_node(0) counts the lowmem scratch area as part of Node 0's kernel footprint and scales it by scratch_scale (200%) again.
--- >8 ---
I didn't look closer, but it does seem to make sense.
But in practice, this problem is only on CONFIG_NUMA=n and I don't think in practice KHO or LUO is being used in non-NUMA systems. So while I think it is worth fixing, I think we should also have a test where we enable CONFIG_NUMA.
Your system probably has CONFIG_NUMA=y and that's why you aren't able to reproduce this bug.
I think on NUMA systems the problem is the other way round. The calculation for the global scratch also counts per-node allocations.
So I think the proper fix for scratch sizing is what this patch does and then a fixup for the global scratch calculation as well.
[...]
Signature · 2 lines
Regards, Pratyush Yadav
Hi Sourabh, Pratyush,
Quoted 4 lines
But in practice, this problem is only on
CONFIG_NUMA=nand I don't think in practice KHO or LUO is being used in non-NUMA systems. So while I think it is worth fixing, I think we should also have a test where we enableCONFIG_NUMA.
Confirmed. My vmtest kernel has CONFIG_NUMA unset, and your analysis matches the data.
I reran the vmtest with Sourabh's debug prints on both configurations:
same kernel, same QEMU command, and without my this fix patch, so the default percentage policy is exercised. The only difference is CONFIG_NUMA.
Without CONFIG_NUMA:
KHO: Before low and global scratch allocations
KHO: low size = 330185 KB
KHO: global size = 322 MB
KHO: Per node 0 = 672 MB
KHO: After low and global scratch allocations
KHO: low size = 330185 KB
KHO: global size = 322 MB
KHO: Per node 0 = 672 MB
KHO: Failed to reserve nid 0 scratch buffer
KHO: Failed to reserve scratch area, disabling kexec handover
With CONFIG_NUMA=y:
KHO: Before low and global scratch allocations
KHO: low size = 330197 KB
KHO: global size = 322 MB
KHO: Per node 0 = 96 MB
KHO: After low and global scratch allocations
KHO: low size = 330197 KB
KHO: global size = 322 MB
KHO: Per node 0 = 96 MB
KHO: After per node allocation
KHO: low size = 428501 KB
KHO: global size = 418 MB
KHO: Per node 0 = 288 MB
In the run without CONFIG_NUMA, the reserved-kern sum the sizing sees is 330185 KB, which is the 98.45 MiB baseline plus the 224 MiB lowmem scratch area: with memblock_get_region_node() hardcoded to return 0, the NUMA_NO_NODE lowmem area is counted as node 0's kernel reservation. Node 0 therefore requests 200% of (98.45 MiB + 224 MiB), rounded up to 32 MiB alignment: 672 MiB. That no longer fits next to the other areas in the 1 GiB guest, and KHO disables itself.
In the run with CONFIG_NUMA=y, the real node ID excludes the NUMA_NO_NODE regions, so node 0 requests 96 MiB. The allocation succeeds and is visible in the sums printed afterwards (330197 KB -> 428501 KB), and the KHO selftest passes end to end ("KHO: found kexec handover data", restore succeeds).
Sourabh, this also answers your question. Your PowerPC system runs CONFIG_NUMA=y, so the node filter excludes the lowmem and global areas and your numbers stay flat. Your experiment and mine are the two halves of the same mechanism.
Quoted 5 lines
I think on NUMA systems the problem is the other way round. The calculation for the global scratch also counts per-node allocations.
So I think the proper fix for scratch sizing is what this patch does and then a fixup for the global scratch calculation as well.
Agreed. For v2 I plan to:
- Compute all scratch sizes (lowmem, global, per-node) before any scratch area is allocated, per Mike's comment. The sizes are a function of the pre-allocation state, so this seals both feedback directions at once.
- State the !CONFIG_NUMA condition in the commit message. The feedback described there is not unconditional, which is what triggered the question.
- Add the LLM attribution Mike asked for.
- Include the CONFIG_NUMA=y vmtest result as coverage.
- Look at the global scratch calculation on NUMA systems as a follow-up.
Thanks,
George
Quoted 41 lines · George Guo
Hi Sourabh, Pratyush,
Quoted 4 lines
But in practice, this problem is only on
CONFIG_NUMA=nand I don't think in practice KHO or LUO is being used in non-NUMA systems. So while I think it is worth fixing, I think we should also have a test where we enableCONFIG_NUMA.Confirmed. My vmtest kernel has
CONFIG_NUMAunset, and your analysis matches the data.I reran the vmtest with Sourabh's debug prints on both configurations:
same kernel, same QEMU command, and without my this fix patch, so the default percentage policy is exercised. The only difference isCONFIG_NUMA.Without
CONFIG_NUMA:KHO: Before low and global scratch allocations
KHO: low size = 330185 KB
KHO: global size = 322 MB
KHO: Per node 0 = 672 MB
KHO: After low and global scratch allocations
KHO: low size = 330185 KB
KHO: global size = 322 MB
KHO: Per node 0 = 672 MB
KHO: Failed to reserve nid 0 scratch buffer
KHO: Failed to reserve scratch area, disabling kexec handoverWith
CONFIG_NUMA=y:KHO: Before low and global scratch allocations
KHO: low size = 330197 KB
KHO: global size = 322 MB
KHO: Per node 0 = 96 MB
KHO: After low and global scratch allocations
KHO: low size = 330197 KB
KHO: global size = 322 MB
KHO: Per node 0 = 96 MB
KHO: After per node allocation
KHO: low size = 428501 KB
KHO: global size = 418 MB
KHO: Per node 0 = 288 MB
[...]
Quoted 26 lines
In the run without
CONFIG_NUMA, the reserved-kern sum the sizing sees is 330185 KB, which is the 98.45 MiB baseline plus the 224 MiB lowmem scratch area: withmemblock_get_region_node()hardcoded to return 0, theNUMA_NO_NODElowmem area is counted as node 0's kernel reservation. Node 0 therefore requests 200% of (98.45 MiB + 224 MiB), rounded up to 32 MiB alignment: 672 MiB. That no longer fits next to the other areas in the 1 GiB guest, and KHO disables itself.In the run with
CONFIG_NUMA=y, the real node ID excludes theNUMA_NO_NODEregions, so node 0 requests 96 MiB. The allocation succeeds and is visible in the sums printed afterwards (330197 KB -> 428501 KB), and the KHO selftest passes end to end ("KHO: found kexec handover data", restore succeeds).Sourabh, this also answers your question. Your PowerPC system runs
CONFIG_NUMA=y, so the node filter excludes the lowmem and global areas and your numbers stay flat. Your experiment and mine are the two halves of the same mechanism.Quoted 4 lines
I think on NUMA systems the problem is the other way round. The calculation for the global scratch also counts per-node allocations. So I think the proper fix for scratch sizing is what this patch does and then a fixup for the global scratch calculation as well.
Agreed. For v2 I plan to:
Quoted 11 lines
- Compute all scratch sizes (lowmem, global, per-node) before any scratch area is allocated, per Mike's comment. The sizes are a function of the pre-allocation state, so this seals both feedback directions at once.
- State the !CONFIG_NUMA condition in the commit message. The feedback described there is not unconditional, which is what triggered the question.
- Add the LLM attribution Mike asked for.
- Include theCONFIG_NUMA=yvmtest result as coverage.
- Look at the global scratch calculation on NUMA systems as a follow-up.
I have sent a fix to handle global scratch calculation on NUMA systems.
https://lore.kernel.org/all/[email protected]/
Please review the changes and share your comments.
- Sourabh Jain
Quoted 39 lines · Pratyush Yadav
Quoted 37 lines · Sourabh Jain
Hello George,
Quoted 34 lines · Sourabh Jain
Hello George,
Quoted 11 lines · George Guo
From: George Guo <[email protected]>
The default percentage-based policy sizes scratch areas from the current kernel's
MEMBLOCK_RSRV_KERNfootprint. This is a reasonable heuristic for predicting the early memory demand of the next kernel.
scratch_size_update()calculates the lowmem and global sizes before either area is allocated. However,kho_reserve_scratch()calculates each per-node size only after allocating the lowmem and global areas. Since memblock allocations are markedMEMBLOCK_RSRV_KERN, the per-node calculation includes those newly allocated scratch areas and scales them again.I may be missing something here, but doesn't
memblock_reserved_kern_size()check the node ID of the region before checking the region flag (MEMBLOCK_RSRV_KERN)?Code snippet from
memblock_reserved_kern_size()``` if (nid == memblock_get_region_node(r) || !numa_valid_node(nid)) if (r->flags & MEMBLOCK_RSRV_KERN) total += size; ```For a valid nid, my understanding is that the global and lowmem scratch areas should not be counted because they are allocated with
NUMA_NO_NODE(-1). So, ideally, these regions should be excluded when calculating the reserved memory for a specific node ID.Based on this, I am not sure that marking the lowmem and global areas as
MEMBLOCK_RSRV_KERNis what causes the per-node size calculation to be inflated. I am looking into the code further to better understand the actual cause of the issue that this patch is trying to address.
[...]
Quoted 59 lines
Quoted 22 lines
I added some prints in
kho_reserve_scratch()and found that the per-node size calculation is not impacted by the lowmem and global scratch memory allocations.KHO: Before low and global scratch allocations
KHO: low size = 899 KB
KHO: global size = 137 MB
KHO: Per node 2 = 80 MBKHO: After low and global scratch allocations
KHO: low size = 312195 KB
KHO: global size = 441 MB
KHO: Per node 2 = 80 MBKHO: After per node allocation
KHO: low size = 394115 KB
KHO: global size = 521 MB
KHO: Per node 2 = 240 MBI only had one NUMA node (nid=2), and the per-NUMA allocation before and after the lowmem and global scratch memory allocations remained the same at 80 MB.
Quoted 5 lines
The experiment was done on the PowerPC architecture.
I am wondering how the per-NUMA allocation in your setup is getting inflated due to the lowmem and global scratch memory reservations.
I had the same question, so I asked AI. Here's what it says:
--- 8< ---
The bug was reported and tested using the KHO self-test runner (
tools/testing/selftests/kho/vmtest.sh), which builds a test kernel using make olddefconfig with only a minimal set of CONFIG_* options. Crucially,CONFIG_NUMAis not enabled.When
CONFIG_NUMAis disabled:#ifndef CONFIG_NUMA static inline void memblock_set_region_node(struct memblock_region *r, int nid) { } static inline int memblock_get_region_node(const struct memblock_region *r) { return 0; } #endifstruct
memblock_regiondoes not even contain an nid member.
memblock_set_region_node()is a no-op (theNUMA_NO_NODEargument is simply discarded), andmemblock_get_region_node()is hardcoded to always return 0.There is only one node (nid = 0), so
for_each_node_state(nid, N_MEMORY)loops once for nid = 0.
Quoted 7 lines
Both
memblock_phys_alloc_range()andmemblock_phys_alloc()mark their allocations withMEMBLOCK_RSRV_KERN. Therefore, whenscratch_size_node(0)runs after allocating the lowmem scratch buffer,memblock_get_region_node(r)returns 0 for that lowmem scratch buffer, andr->flags&MEMBLOCK_RSRV_KERNis true. As a result,scratch_size_node(0)counts the lowmem scratch area as part of Node 0's kernel footprint and scales it byscratch_scale(200%) again.
Yes, CONFIG_NUMA is the reason I didn't observe this in my setup. With CONFIG_NUMA disabled, I can see the per-node allocation getting inflated due to the low and global allocations.
This raises the question: do we really need per-node allocation when CONFIG_NUMA is disabled?
For example, if a VM boots with 4G RAM, the low, global, and per-node allocations are all the same. With KHO enabled, the kernel would then allocate 6 times the total memblock reserved memory: twice for each of the three components (low, global, and per-node).
Quoted 9 lines
--- >8 ---
I didn't look closer, but it does seem to make sense.
But in practice, this problem is only on
CONFIG_NUMA=nand I don't think in practice KHO or LUO is being used in non-NUMA systems. So while I think it is worth fixing, I think we should also have a test where we enableCONFIG_NUMA.
Quoted 8 lines
Your system probably has
CONFIG_NUMA=yand that's why you aren't able to reproduce this bug.I think on NUMA systems the problem is the other way round. The calculation for the global scratch also counts per-node allocations.
So I think the proper fix for scratch sizing is what this patch does and then a fixup for the global scratch calculation as well.
Agree.
- Sourabh Jain