kho: initialize tail pages for higher order folios properly
Subsystems (4)
Pratyush Yadav
maintainerPATCH 1/1 kho: initialize tail pages for higher order folios properly
From: Pratyush Yadav <[email protected]>
Currently, when restoring higher order folios, kho_restore_folio() only calls prep_compound_page() on all the pages. That is not enough to properly initialize the folios. The managed page count does not get updated, the reserved flag does not get dropped, and page count does not get initialized properly.
Restoring a higher order folio with it results in the following BUG with CONFIG_DEBUG_VM when attempting to free the folio:
BUG: Bad page state in process test pfn:104e2b
page: refcount:1 mapcount:0 mapping:0000000000000000 index:0xffffffffffffffff pfn:0x104e2b
flags: 0x2fffff80000000(node=0|zone=2|lastcpupid=0x1fffff)
raw: 002fffff80000000 0000000000000000 00000000ffffffff 0000000000000000
raw: ffffffffffffffff 0000000000000000 00000001ffffffff 0000000000000000
page dumped because: nonzero _refcount
[...]
Call Trace:
<TASK>
dump_stack_lvl+0x4b/0x70
bad_page.cold+0x97/0xb2
__free_frozen_pages+0x616/0x850
[...]Combine the path for 0-order and higher order folios, initialize the tail pages with a count of zero, and call adjust_managed_page_count() to account for all the pages instead of just missing them.
In addition, since all the KHO-preserved pages get marked with MEMBLOCK_RSRV_NOINIT by deserialize_bitmap(), the reserved flag is not actually set (as can also be seen from the flags of the dumped page in the logs above). So drop the ClearPageReserved() calls.
---
Side note: get_maintainers.pl for KHO only lists kexec@ as the mailing list. Since KHO has a bunch of MM bits as well, should we also add linux-mm@ to its MAINTAINERS entry?
Adding linux-mm@ to this patch at least, in case MM people have an opinion on this.
kernel/kexec_handover.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-)
kernel/kexec_handover.c +17 -12
@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, }/* almost as free_reserved_page(), just don't free the page */ -static void kho_restore_page (struct page *page) +static void kho_restore_page (struct page *page,unsigned int order) { -ClearPageReserved (page); -init_page_count (page); -adjust_managed_page_count (page,1 ); +unsigned int i, nr_pages = (1 << order); + +/* Head page gets refcount of 1. */ +set_page_count (page,1 ); + +/* For higher order folios, tail pages get a page count of zero. */ +for (i =1 ; i < nr_pages; i++) +set_page_count (page + i,0 ); + +if (order >0 ) +prep_compound_page (page, order); + +adjust_managed_page_count (page, nr_pages); }/**
@@ -179,15 +189,10 @@ struct folio *kho_restore_folio(phys_addr_t phys)return NULL ; order = page->private; -if (order) { -if (order >MAX_PAGE_ORDER ) -return NULL ; - -prep_compound_page (page, order); - }else { -kho_restore_page (page); - } +if (order >MAX_PAGE_ORDER ) +return NULL ; +kho_restore_page (page, order);return page_folio (page); }EXPORT_SYMBOL_GPL (kho_restore_folio);
--
2.47.1
Andrew Morton
maintainerQuoted 3 lines · On Thu, 5 Jun 2025 19:11:41 +0200 Pratyush Yadav <[email protected]> wrote:
Side note: get_maintainers.pl for KHO only lists kexec@ as the mailing list. Since KHO has a bunch of MM bits as well, should we also add linux-mm@ to its
MAINTAINERSentry?
I think so. Please send a patch?
Mike Rapoport
maintainerQuoted 53 lines · Pratyush Yadav
From: Pratyush Yadav <[email protected]>
Currently, when restoring higher order folios,
kho_restore_folio()only callsprep_compound_page()on all the pages. That is not enough to properly initialize the folios. The managed page count does not get updated, the reserved flag does not get dropped, and page count does not get initialized properly.Restoring a higher order folio with it results in the following BUG with
CONFIG_DEBUG_VMwhen attempting to free the folio:BUG: Bad page state in process test pfn:104e2b page: refcount:1 mapcount:0 mapping:0000000000000000 index:0xffffffffffffffff pfn:0x104e2b flags: 0x2fffff80000000(node=0|zone=2|lastcpupid=0x1fffff) raw: 002fffff80000000 0000000000000000 00000000ffffffff 0000000000000000 raw: ffffffffffffffff 0000000000000000 00000001ffffffff 0000000000000000 page dumped because: nonzero _refcount [...] Call Trace: <TASK> dump_stack_lvl+0x4b/0x70 bad_page.cold+0x97/0xb2 __free_frozen_pages+0x616/0x850 [...]Combine the path for 0-order and higher order folios, initialize the tail pages with a count of zero, and call
adjust_managed_page_count()to account for all the pages instead of just missing them.In addition, since all the KHO-preserved pages get marked with
MEMBLOCK_RSRV_NOINITbydeserialize_bitmap(), the reserved flag is not actually set (as can also be seen from the flags of the dumped page in the logs above). So drop theClearPageReserved()calls.Fixes fc33e4b44b271 ("kexec: enable KHO support for memory preservation")Signed-off-by Pratyush Yadav <[email protected]>---
Side note: get_maintainers.pl for KHO only lists kexec@ as the mailing list. Since KHO has a bunch of MM bits as well, should we also add linux-mm@ to its
MAINTAINERSentry?Adding linux-mm@ to this patch at least, in case MM people have an opinion on this.
kernel/kexec_handover.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-)kernel/kexec_handover.c +0
-0
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162
Quoted 7 lines
kernel/kexec_handover.c +1
-2@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, }/* almost as free_reserved_page(), just don't free the page */ -static void kho_restore_page (struct page *page) +static void kho_restore_page (struct page *page,unsigned int order) { -ClearPageReserved (page);
So now we don't clear PG_Reserved even on order-0 pages? ;-)
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162
Quoted 3 lines
kernel/kexec_handover.c +1
-2-init_page_count (page); -adjust_managed_page_count (page,1 ); +unsigned int i, nr_pages = (1 << order);
Can you please declare 'i' inside the loop, looks nicer IMHO.
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 165
Quoted 3 lines
kernel/kexec_handover.c +3
-0+ +/* Head page gets refcount of 1. */ +set_page_count (page,1 );
ClearPageReserved(page) here?
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 169
Quoted 4 lines
kernel/kexec_handover.c +4
-0+ +/* For higher order folios, tail pages get a page count of zero. */ +for (i =1 ; i < nr_pages; i++) +set_page_count (page + i,0 );
and here?
Quoted 30 lines
kernel/kexec_handover.c +8
-9+ +if (order >0 ) +prep_compound_page (page, order); + +adjust_managed_page_count (page, nr_pages); }/** @@ -179,15 +189,10 @@ struct folio *kho_restore_folio(phys_addr_t phys)return NULL ; order = page->private; -if (order) { -if (order >MAX_PAGE_ORDER ) -return NULL ; - -prep_compound_page (page, order); - }else { -kho_restore_page (page); - } +if (order >MAX_PAGE_ORDER ) +return NULL ; +kho_restore_page (page, order);return page_folio (page); }EXPORT_SYMBOL_GPL (kho_restore_folio); --2.47.1
Signature · 2 lines
Sincerely yours, Mike.
Pratyush Yadav
maintainerHi Mike,
Quoted 56 lines · Mike Rapoport
Quoted 55 lines · Pratyush Yadav
From: Pratyush Yadav <[email protected]>
Currently, when restoring higher order folios,
kho_restore_folio()only callsprep_compound_page()on all the pages. That is not enough to properly initialize the folios. The managed page count does not get updated, the reserved flag does not get dropped, and page count does not get initialized properly.Restoring a higher order folio with it results in the following BUG with
CONFIG_DEBUG_VMwhen attempting to free the folio:BUG: Bad page state in process test pfn:104e2b page: refcount:1 mapcount:0 mapping:0000000000000000 index:0xffffffffffffffff pfn:0x104e2b flags: 0x2fffff80000000(node=0|zone=2|lastcpupid=0x1fffff) raw: 002fffff80000000 0000000000000000 00000000ffffffff 0000000000000000 raw: ffffffffffffffff 0000000000000000 00000001ffffffff 0000000000000000 page dumped because: nonzero _refcount [...] Call Trace: <TASK> dump_stack_lvl+0x4b/0x70 bad_page.cold+0x97/0xb2 __free_frozen_pages+0x616/0x850 [...]Combine the path for 0-order and higher order folios, initialize the tail pages with a count of zero, and call
adjust_managed_page_count()to account for all the pages instead of just missing them.In addition, since all the KHO-preserved pages get marked with
MEMBLOCK_RSRV_NOINITbydeserialize_bitmap(), the reserved flag is not actually set (as can also be seen from the flags of the dumped page in the logs above). So drop theClearPageReserved()calls.Fixes fc33e4b44b271 ("kexec: enable KHO support for memory preservation")Signed-off-by Pratyush Yadav <[email protected]>---
Side note: get_maintainers.pl for KHO only lists kexec@ as the mailing list. Since KHO has a bunch of MM bits as well, should we also add linux-mm@ to its
MAINTAINERSentry?Adding linux-mm@ to this patch at least, in case MM people have an opinion on this.
kernel/kexec_handover.c | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-)kernel/kexec_handover.c +0
-0@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, }
Quoted 7 lines
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162
Quoted 5 lines
kernel/kexec_handover.c +1
-2/* almost as free_reserved_page(), just don't free the page */ -static void kho_restore_page (struct page *page) +static void kho_restore_page (struct page *page,unsigned int order) { -ClearPageReserved (page);So now we don't clear
PG_Reservedeven on order-0 pages? ;-)
We don't need to. As I mentioned in the commit message as well, PG_Reserved is never set for KHO pages since they are reserved with MEMBLOCK_RSRV_NOINIT, so memmap_init_reserved_pages() skips over them.
To double-check, I added some quick prints to kho_restore_page():
/* Head page gets refcount of 1. */
set_page_count(page, 1);
printk("Head page flags: 0x%lx reserved: %d\n",
page->flags, PageReserved(page));
/* For higher order folios, tail pages get a page count of zero. */
for (i = 1; i < nr_pages; i++) {
printk("Tail page %u flags: 0x%lx reserved: %d\n",
i, (page+i)->flags, PageReserved(page+i));
set_page_count(page + i, 0);
}And this is what I get:
[ 9.003187] Head page flags: 0x2fffff80000000 reserved: 0
[ 9.003730] Tail page 1 flags: 0x2fffff80000000 reserved: 0
[ 9.004229] Tail page 2 flags: 0x2fffff80000000 reserved: 0
[ 9.004740] Tail page 3 flags: 0x2fffff80000000 reserved: 0
[ 9.005265] Head page flags: 0x2fffff80000000 reserved: 0
[ 9.005759] Head page flags: 0x2fffff80000000 reserved: 0
[...]So PG_Reserved is never set.
That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, with CONFIG_DEFERRED_STRUCT_PAGE_INIT == n, all the pages get initialized when memmap_init_range() is called from setup_arch (paging_init() on x86). This happens before kho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.
With deferred page init, some pages might not get initialized early, and get initialized after kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So, deferred_init_maxorder() will skip over those pages and leave them uninitialized.
And sure enough, doing the same with CONFIG_DEFERRED_STRUCT_PAGE_INIT == y results in:
[ 10.060842] Head page flags: 0x2fffff80000000 reserved: 0
[ 10.061387] Tail page 1 flags: 0x2fffff80000000 reserved: 0
[ 10.061902] Tail page 2 flags: 0x2fffff80000000 reserved: 0
[ 10.062400] Tail page 3 flags: 0x2fffff80000000 reserved: 0
[ 10.062924] page:00000000fb3dca54 is uninitialized and poisoned
[ 10.063494] page dumped because: VM_BUG_ON_PAGE(PagePoisoned(page))
[ 10.064190] ------------[ cut here ]------------
[ 10.064636] kernel BUG at ./include/linux/page-flags.h:571!
[ 10.065194] Oops: invalid opcode: 0000 [#1] SMP PTI
[ 10.065661] CPU: 2 UID: 0 PID: 1954 Comm: test Not tainted 6.15.0+ #297 PREEMPT(undef)
[ 10.066449] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.3-1-1 04/01/2014
[ 10.067353] RIP: 0010:kho_restore_folio+0x4e/0x70
[...]So we need to either also call init_deferred_page(), or remove the memblock_reserved_mark_noinit() call in deserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.
What do you think? Should we drop noinit or call init_deferred_page()? FWIW, my preference is to drop noinit, since init_deferred_page() is __meminit and we would have to make sure it doesn't go away after boot.
Quoted 6 lines
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162
Quoted 4 lines
kernel/kexec_handover.c +1
-2-init_page_count (page); -adjust_managed_page_count (page,1 ); +unsigned int i, nr_pages = (1 << order);Can you please declare 'i' inside the loop, looks nicer IMHO.
Ok, will do.
Quoted 22 lines
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 165
Quoted 4 lines
kernel/kexec_handover.c +3
-0+ +/* Head page gets refcount of 1. */ +set_page_count (page,1 );ClearPageReserved(page) here?
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 169
Quoted 4 lines
kernel/kexec_handover.c +4
-0+ +/* For higher order folios, tail pages get a page count of zero. */ +for (i =1 ; i < nr_pages; i++) +set_page_count (page + i,0 );and here?
Quoted 8 lines
kernel/kexec_handover.c +5
-0+ +if (order >0 ) +prep_compound_page (page, order); + +adjust_managed_page_count (page, nr_pages); }/**
[...]
Signature · 2 lines
Regards, Pratyush Yadav
Mike Rapoport
maintainerHi Pratyush,
Quoted 20 lines · Pratyush Yadav
Hi Mike,
Quoted 8 lines · Mike Rapoport
Quoted 7 lines · Pratyush Yadav
From: Pratyush Yadav <[email protected]>
kernel/kexec_handover.c +0
-0@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, }Quoted 7 lines
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162
Quoted 5 lines
kernel/kexec_handover.c +1
-2/* almost as free_reserved_page(), just don't free the page */ -static void kho_restore_page (struct page *page) +static void kho_restore_page (struct page *page,unsigned int order) { -ClearPageReserved (page);So now we don't clear
PG_Reservedeven on order-0 pages? ;-)
Quoted 3 lines
We don't need to. As I mentioned in the commit message as well,
PG_Reservedis never set for KHO pages since they are reserved withMEMBLOCK_RSRV_NOINIT, somemmap_init_reserved_pages()skips over them.
You are right, I missed it.
Quoted 12 lines
That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, withCONFIG_DEFERRED_STRUCT_PAGE_INIT== n, all the pages get initialized whenmemmap_init_range()is called fromsetup_arch(paging_init()on x86). This happens beforekho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.With deferred page init, some pages might not get initialized early, and get initialized after
kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So,deferred_init_maxorder()will skip over those pages and leave them uninitialized.
Quoted 6 lines
So we need to either also call
init_deferred_page(), or remove thememblock_reserved_mark_noinit()call indeserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.
One benefit is performance indeed, because in not deferred case the initialization of reserved pages in memmap_init_reserved_pages() is really excessive.
But more importantly, if we remove memblock_reserved_mark_noinit(), with CONFIG_DEFERRED_STRUCT_PAGE_INIT we'd loose page->private because the struct page will be cleared after kho_mem_deserialize().
Quoted 3 lines
What do you think? Should we drop noinit or call
init_deferred_page()? FWIW, my preference is to drop noinit, sinceinit_deferred_page()is__meminitand we would have to make sure it doesn't go away after boot.
We can't drop noinit and calling init_deferred_page() after boot just won't work because it uses memblock to find the page's node and memblock is gone after init.
The simplest short-term solution is to disable KHO when CONFIG_DEFERRED_STRUCT_PAGE_INIT is set and then find an efficient way to make it all work together.
Signature · 2 lines
Sincerely yours, Mike.
Pasha Tatashin
maintainerQuoted 61 lines · Mike Rapoport
Hi Pratyush,
Quoted 20 lines · Pratyush Yadav
Hi Mike,
Quoted 8 lines · Mike Rapoport
Quoted 7 lines · Pratyush Yadav
From: Pratyush Yadav <[email protected]>
kernel/kexec_handover.c +0
-0@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, }Quoted 7 lines
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162
Quoted 5 lines
kernel/kexec_handover.c +1
-2/* almost as free_reserved_page(), just don't free the page */ -static void kho_restore_page (struct page *page) +static void kho_restore_page (struct page *page,unsigned int order) { -ClearPageReserved (page);So now we don't clear
PG_Reservedeven on order-0 pages? ;-)Quoted 3 lines
We don't need to. As I mentioned in the commit message as well,
PG_Reservedis never set for KHO pages since they are reserved withMEMBLOCK_RSRV_NOINIT, somemmap_init_reserved_pages()skips over them.You are right, I missed it.
Quoted 12 lines
That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, withCONFIG_DEFERRED_STRUCT_PAGE_INIT== n, all the pages get initialized whenmemmap_init_range()is called fromsetup_arch(paging_init()on x86). This happens beforekho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.With deferred page init, some pages might not get initialized early, and get initialized after
kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So,deferred_init_maxorder()will skip over those pages and leave them uninitialized.Quoted 6 lines
So we need to either also call
init_deferred_page(), or remove thememblock_reserved_mark_noinit()call indeserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.One benefit is performance indeed, because in not deferred case the initialization of reserved pages in
memmap_init_reserved_pages()is really excessive.But more importantly, if we remove
memblock_reserved_mark_noinit(), withCONFIG_DEFERRED_STRUCT_PAGE_INITwe'd loosepage->privatebecause the struct page will be cleared afterkho_mem_deserialize().Quoted 3 lines
What do you think? Should we drop noinit or call
init_deferred_page()? FWIW, my preference is to drop noinit, sinceinit_deferred_page()is__meminitand we would have to make sure it doesn't go away after boot.
Quoted 7 lines
We can't drop noinit and calling
init_deferred_page()after boot just won't work because it uses memblock to find the page's node and memblock is gone after init.The simplest short-term solution is to disable KHO when
CONFIG_DEFERRED_STRUCT_PAGE_INITis set and then find an efficient way to make it all work together.
This is what I've done in LUOv3 WIP:
https://github.com/soleen/linux/commit/3059f38ac0a39a397873759fb429bd5d1f8ea681
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
Pasha
Quoted 4 lines
--
Sincerely yours,
Mike.
Mike Rapoport
maintainerQuoted 66 lines · Pasha Tatashin
Quoted 65 lines · Mike Rapoport
Hi Pratyush,
Quoted 20 lines · Pratyush Yadav
Hi Mike,
Quoted 8 lines · Mike Rapoport
Quoted 7 lines · Pratyush Yadav
From: Pratyush Yadav <[email protected]>
kernel/kexec_handover.c +0
-0@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, }Quoted 7 lines
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162
Quoted 5 lines
kernel/kexec_handover.c +1
-2/* almost as free_reserved_page(), just don't free the page */ -static void kho_restore_page (struct page *page) +static void kho_restore_page (struct page *page,unsigned int order) { -ClearPageReserved (page);So now we don't clear
PG_Reservedeven on order-0 pages? ;-)Quoted 3 lines
We don't need to. As I mentioned in the commit message as well,
PG_Reservedis never set for KHO pages since they are reserved withMEMBLOCK_RSRV_NOINIT, somemmap_init_reserved_pages()skips over them.You are right, I missed it.
Quoted 12 lines
That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, withCONFIG_DEFERRED_STRUCT_PAGE_INIT== n, all the pages get initialized whenmemmap_init_range()is called fromsetup_arch(paging_init()on x86). This happens beforekho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.With deferred page init, some pages might not get initialized early, and get initialized after
kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So,deferred_init_maxorder()will skip over those pages and leave them uninitialized.Quoted 6 lines
So we need to either also call
init_deferred_page(), or remove thememblock_reserved_mark_noinit()call indeserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.One benefit is performance indeed, because in not deferred case the initialization of reserved pages in
memmap_init_reserved_pages()is really excessive.But more importantly, if we remove
memblock_reserved_mark_noinit(), withCONFIG_DEFERRED_STRUCT_PAGE_INITwe'd loosepage->privatebecause the struct page will be cleared afterkho_mem_deserialize().Quoted 3 lines
What do you think? Should we drop noinit or call
init_deferred_page()? FWIW, my preference is to drop noinit, sinceinit_deferred_page()is__meminitand we would have to make sure it doesn't go away after boot.We can't drop noinit and calling
init_deferred_page()after boot just won't work because it uses memblock to find the page's node and memblock is gone after init.
Quoted 6 lines
Quoted 3 lines
The simplest short-term solution is to disable KHO when
CONFIG_DEFERRED_STRUCT_PAGE_INITis set and then find an efficient way to make it all work together.This is what I've done in LUOv3 WIP:
https://github.com/soleen/linux/commit/3059f38ac0a39a397873759fb429bd5d1f8ea681
I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.
Quoted 3 lines
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires finding its node with early_pfn_to_nid() that's also suboptimal.
Quoted 6 lines
Pasha
Quoted 4 lines
--
Sincerely yours,
Mike.
Signature · 2 lines
Sincerely yours, Mike.
Pasha Tatashin
maintainerQuoted 72 lines · Mike Rapoport
Quoted 70 lines · Pasha Tatashin
Quoted 68 lines · Mike Rapoport
Hi Pratyush,
Quoted 20 lines · Pratyush Yadav
Hi Mike,
Quoted 8 lines · Mike Rapoport
Quoted 7 lines · Pratyush Yadav
From: Pratyush Yadav <[email protected]>
kernel/kexec_handover.c +0
-0@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, }Quoted 7 lines
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162
Quoted 5 lines
kernel/kexec_handover.c +1
-2/* almost as free_reserved_page(), just don't free the page */ -static void kho_restore_page (struct page *page) +static void kho_restore_page (struct page *page,unsigned int order) { -ClearPageReserved (page);So now we don't clear
PG_Reservedeven on order-0 pages? ;-)Quoted 3 lines
We don't need to. As I mentioned in the commit message as well,
PG_Reservedis never set for KHO pages since they are reserved withMEMBLOCK_RSRV_NOINIT, somemmap_init_reserved_pages()skips over them.You are right, I missed it.
Quoted 12 lines
That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, withCONFIG_DEFERRED_STRUCT_PAGE_INIT== n, all the pages get initialized whenmemmap_init_range()is called fromsetup_arch(paging_init()on x86). This happens beforekho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.With deferred page init, some pages might not get initialized early, and get initialized after
kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So,deferred_init_maxorder()will skip over those pages and leave them uninitialized.Quoted 6 lines
So we need to either also call
init_deferred_page(), or remove thememblock_reserved_mark_noinit()call indeserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.One benefit is performance indeed, because in not deferred case the initialization of reserved pages in
memmap_init_reserved_pages()is really excessive.But more importantly, if we remove
memblock_reserved_mark_noinit(), withCONFIG_DEFERRED_STRUCT_PAGE_INITwe'd loosepage->privatebecause the struct page will be cleared afterkho_mem_deserialize().Quoted 3 lines
What do you think? Should we drop noinit or call
init_deferred_page()? FWIW, my preference is to drop noinit, sinceinit_deferred_page()is__meminitand we would have to make sure it doesn't go away after boot.We can't drop noinit and calling
init_deferred_page()after boot just won't work because it uses memblock to find the page's node and memblock is gone after init.The simplest short-term solution is to disable KHO when
CONFIG_DEFERRED_STRUCT_PAGE_INITis set and then find an efficient way to make it all work together.
Quoted 5 lines
Quoted 2 lines
This is what I've done in LUOv3 WIP:
https://github.com/soleen/linux/commit/3059f38ac0a39a397873759fb429bd5d1f8ea681I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.
Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)
Quoted 6 lines
Quoted 3 lines
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
We could, but with that would mean we'll run this before SMP and it's not desirable. Also,
init_deferred_page()for a random page requires
We already run KHO init before smp_init:start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!
While deferred struct pages are initialized:start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()
If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()
Pasha
Mike Rapoport
maintainerQuoted 76 lines · Pasha Tatashin
Quoted 75 lines · Mike Rapoport
Quoted 72 lines · Pasha Tatashin
Quoted 61 lines · Mike Rapoport
Hi Pratyush,
Quoted 20 lines · Pratyush Yadav
Hi Mike,
Quoted 8 lines · Mike Rapoport
Quoted 7 lines · Pratyush Yadav
From: Pratyush Yadav <[email protected]>
kernel/kexec_handover.c +0
-0@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, }Quoted 7 lines
kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162
Quoted 5 lines
kernel/kexec_handover.c +1
-2/* almost as free_reserved_page(), just don't free the page */ -static void kho_restore_page (struct page *page) +static void kho_restore_page (struct page *page,unsigned int order) { -ClearPageReserved (page);So now we don't clear
PG_Reservedeven on order-0 pages? ;-)Quoted 3 lines
We don't need to. As I mentioned in the commit message as well,
PG_Reservedis never set for KHO pages since they are reserved withMEMBLOCK_RSRV_NOINIT, somemmap_init_reserved_pages()skips over them.You are right, I missed it.
Quoted 12 lines
That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, withCONFIG_DEFERRED_STRUCT_PAGE_INIT== n, all the pages get initialized whenmemmap_init_range()is called fromsetup_arch(paging_init()on x86). This happens beforekho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.With deferred page init, some pages might not get initialized early, and get initialized after
kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So,deferred_init_maxorder()will skip over those pages and leave them uninitialized.Quoted 6 lines
So we need to either also call
init_deferred_page(), or remove thememblock_reserved_mark_noinit()call indeserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.One benefit is performance indeed, because in not deferred case the initialization of reserved pages in
memmap_init_reserved_pages()is really excessive.But more importantly, if we remove
memblock_reserved_mark_noinit(), withCONFIG_DEFERRED_STRUCT_PAGE_INITwe'd loosepage->privatebecause the struct page will be cleared afterkho_mem_deserialize().Quoted 3 lines
What do you think? Should we drop noinit or call
init_deferred_page()? FWIW, my preference is to drop noinit, sinceinit_deferred_page()is__meminitand we would have to make sure it doesn't go away after boot.Quoted 7 lines
We can't drop noinit and calling
init_deferred_page()after boot just won't work because it uses memblock to find the page's node and memblock is gone after init.The simplest short-term solution is to disable KHO when
CONFIG_DEFERRED_STRUCT_PAGE_INITis set and then find an efficient way to make it all work together.This is what I've done in LUOv3 WIP:
https://github.com/soleen/linux/commit/3059f38ac0a39a397873759fb429bd5d1f8ea681
Quoted 6 lines
Quoted 2 lines
I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.
Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)
Hmm, weird, worked for me :/
Quoted 16 lines
Quoted 6 lines
Quoted 3 lines
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
We could, but with that would mean we'll run this before SMP and it's not desirable. Also,
init_deferred_page()for a random page requiresWe already run KHO init before
smp_init:start_kernel()->mm_core_init()->kho_memory_init()->kho_restore_folio()-> struct pages must be already initialized here!While deferred struct pages are initialized:
start_kernel()->rest_init()->kernel_init()->kernel_init_freeable()->page_alloc_init_late()->deferred_init_memmap()
Quoted 5 lines
If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see
deferred_grow_zone()
deferred_grow_zone() takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early, deferred_init_memmap() will overwrite them.
Anyway, I'm going to look into it, hopefully I'll have something Really Soon (tm).
Quoted 1 line
Pasha
Signature · 2 lines
Sincerely yours, Mike.
Pasha Tatashin
maintainerQuoted 8 lines
Quoted 6 lines
Quoted 2 lines
I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.
Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)
Hmm, weird, worked for me :/
I am super confused, it did not work for me over weekend, and now it is working. Even make menuconfig would not work. Anyways, I will put it in the appropriate place.
Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
We could, but with that would mean we'll run this before SMP and it's not desirable. Also,
init_deferred_page()for a random page requiresWe already run KHO init before
smp_init:start_kernel()->mm_core_init()->kho_memory_init()->kho_restore_folio()-> struct pages must be already initialized here!While deferred struct pages are initialized:
start_kernel()->rest_init()->kernel_init()->kernel_init_freeable()->page_alloc_init_late()->deferred_init_memmap()Quoted 5 lines
If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see
deferred_grow_zone()
Quoted 3 lines
deferred_grow_zone()takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early,deferred_init_memmap()will overwrite them.
Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.
Quoted 2 lines
Anyway, I'm going to look into it, hopefully I'll have something Really Soon (tm).
Pasha
Pratyush Yadav
maintainerQuoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.
Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)
Hmm, weird, worked for me :/
Worked for me as well.
Quoted 32 lines
I am super confused, it did not work for me over weekend, and now it is working. Even
make menuconfigwould not work. Anyways, I will put it in the appropriate place.Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
We could, but with that would mean we'll run this before SMP and it's not desirable. Also,
init_deferred_page()for a random page requiresWe already run KHO init before
smp_init:start_kernel()->mm_core_init()->kho_memory_init()->kho_restore_folio()-> struct pages must be already initialized here!While deferred struct pages are initialized:
start_kernel()->rest_init()->kernel_init()->kernel_init_freeable()->page_alloc_init_late()->deferred_init_memmap()Quoted 5 lines
If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see
deferred_grow_zone()Quoted 3 lines
deferred_grow_zone()takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early,deferred_init_memmap()will overwrite them.
Quoted 5 lines
Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.
Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.
Currently the only user of kho_restore_folio() during init is kho_memory_init(). The FDT is accessed by doing phys_to_virt(kho_in.fdt_phys) anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.
Signature · 2 lines
Regards, Pratyush Yadav
Pasha Tatashin
maintainerQuoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.
Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)
Hmm, weird, worked for me :/
Worked for me as well.
Quoted 32 lines
I am super confused, it did not work for me over weekend, and now it is working. Even
make menuconfigwould not work. Anyways, I will put it in the appropriate place.Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
We could, but with that would mean we'll run this before SMP and it's not desirable. Also,
init_deferred_page()for a random page requiresWe already run KHO init before
smp_init:start_kernel()->mm_core_init()->kho_memory_init()->kho_restore_folio()-> struct pages must be already initialized here!While deferred struct pages are initialized:
start_kernel()->rest_init()->kernel_init()->kernel_init_freeable()->page_alloc_init_late()->deferred_init_memmap()Quoted 5 lines
If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see
deferred_grow_zone()Quoted 3 lines
deferred_grow_zone()takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early,deferred_init_memmap()will overwrite them.Quoted 5 lines
Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.
Quoted 8 lines
Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.
Currently the only user of
kho_restore_folio()during init iskho_memory_init(). The FDT is accessed by doingphys_to_virt(kho_in.fdt_phys)anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.
Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.
Pasha
Quoted 4 lines
--
Regards,
Pratyush Yadav
Mike Rapoport
maintainerQuoted 62 lines · Pasha Tatashin
Quoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.
Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)
Hmm, weird, worked for me :/
Worked for me as well.
Quoted 32 lines
I am super confused, it did not work for me over weekend, and now it is working. Even
make menuconfigwould not work. Anyways, I will put it in the appropriate place.Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
We could, but with that would mean we'll run this before SMP and it's not desirable. Also,
init_deferred_page()for a random page requiresWe already run KHO init before
smp_init:start_kernel()->mm_core_init()->kho_memory_init()->kho_restore_folio()-> struct pages must be already initialized here!While deferred struct pages are initialized:
start_kernel()->rest_init()->kernel_init()->kernel_init_freeable()->page_alloc_init_late()->deferred_init_memmap()Quoted 5 lines
If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see
deferred_grow_zone()Quoted 3 lines
deferred_grow_zone()takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early,deferred_init_memmap()will overwrite them.Quoted 5 lines
Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.
Quoted 8 lines
Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.
Currently the only user of
kho_restore_folio()during init iskho_memory_init(). The FDT is accessed by doingphys_to_virt(kho_in.fdt_phys)anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.
Quoted 6 lines
Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.
KHO already accesses stuct page early and uses page->private for order. Since preserved memory is reserved in memblock, deferred init of struct pages won't touch those pages, we just need to make sure they are properly initialized at some point. If we don't expect many kho_restore_folio() before page_alloc_init_late() we can use init_deferred_page() for early accesses.
Quoted 6 lines
Pasha
Quoted 4 lines
--
Regards,
Pratyush Yadav
Signature · 2 lines
Sincerely yours, Mike.
Pratyush Yadav
maintainerQuoted 70 lines · Mike Rapoport
Quoted 62 lines · Pasha Tatashin
Quoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.
Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)
Hmm, weird, worked for me :/
Worked for me as well.
Quoted 32 lines
I am super confused, it did not work for me over weekend, and now it is working. Even
make menuconfigwould not work. Anyways, I will put it in the appropriate place.Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
We could, but with that would mean we'll run this before SMP and it's not desirable. Also,
init_deferred_page()for a random page requiresWe already run KHO init before
smp_init:start_kernel()->mm_core_init()->kho_memory_init()->kho_restore_folio()-> struct pages must be already initialized here!While deferred struct pages are initialized:
start_kernel()->rest_init()->kernel_init()->kernel_init_freeable()->page_alloc_init_late()->deferred_init_memmap()Quoted 5 lines
If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see
deferred_grow_zone()Quoted 3 lines
deferred_grow_zone()takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early,deferred_init_memmap()will overwrite them.Quoted 5 lines
Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.
Quoted 8 lines
Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.
Currently the only user of
kho_restore_folio()during init iskho_memory_init(). The FDT is accessed by doingphys_to_virt(kho_in.fdt_phys)anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.Quoted 6 lines
Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.
Quoted 3 lines
KHO already accesses stuct page early and uses
page->privatefor order. Since preserved memory is reserved in memblock, deferred init of struct pages won't touch those pages, we just need to make sure they are properly
Not strictly true. Some of them might have been initialized from free_area_init() -> memmap_init() (the ones not eligible for deferred init), which happens before KHO makes its memblock reservations.
Quoted 3 lines
initialized at some point. If we don't expect many
kho_restore_folio()beforepage_alloc_init_late()we can useinit_deferred_page()for early accesses.
I tried doing this when looking into this initially, but it doesn't work for some reason.
static void kho_restore_page(struct page *page, unsigned int order)
{
unsigned int i, nr_pages = (1 << order);
/* Head page gets refcount of 1. */
init_deferred_page(page_to_pfn(page), NUMA_NO_NODE);
set_page_count(page, 1);
/* For higher order folios, tail pages get a page count of zero. */
for (i = 1; i < nr_pages; i++) {
init_deferred_page(page_to_pfn(page + i), NUMA_NO_NODE);
set_page_count(page + i, 0);
}
[...]results in:
[ 0.644032] page:(____ptrval____) is uninitialized and poisoned
[ 0.644679] page dumped because: VM_BUG_ON_PAGE(PagePoisoned(page))
[ 0.645376] ------------[ cut here ]------------
[ 0.645883] kernel BUG at ./include/linux/mm.h:1512!
[...]
[ 0.647924] RIP: 0010:__pageblock_pfn_to_page+0x166/0x180
[...]
[ 0.647924] <TASK>
[ 0.647924] set_zone_contiguous+0x6b/0x90
[ 0.647924] page_alloc_init_late+0x356/0x370
[ 0.647924] kernel_init_freeable+0x12d/0x190
[ 0.647924] ? __pfx_kernel_init+0x10/0x10
[ 0.647924] kernel_init+0x1a/0x130didn't dig any deeper on why it happens...
Signature · 2 lines
Regards, Pratyush Yadav
Mike Rapoport
maintainerQuoted 87 lines · Pratyush Yadav
Quoted 70 lines · Mike Rapoport
Quoted 62 lines · Pasha Tatashin
Quoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.
Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)
Hmm, weird, worked for me :/
Worked for me as well.
Quoted 32 lines
I am super confused, it did not work for me over weekend, and now it is working. Even
make menuconfigwould not work. Anyways, I will put it in the appropriate place.Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
We could, but with that would mean we'll run this before SMP and it's not desirable. Also,
init_deferred_page()for a random page requiresWe already run KHO init before
smp_init:start_kernel()->mm_core_init()->kho_memory_init()->kho_restore_folio()-> struct pages must be already initialized here!While deferred struct pages are initialized:
start_kernel()->rest_init()->kernel_init()->kernel_init_freeable()->page_alloc_init_late()->deferred_init_memmap()Quoted 5 lines
If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see
deferred_grow_zone()Quoted 3 lines
deferred_grow_zone()takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early,deferred_init_memmap()will overwrite them.Quoted 5 lines
Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.
Quoted 8 lines
Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.
Currently the only user of
kho_restore_folio()during init iskho_memory_init(). The FDT is accessed by doingphys_to_virt(kho_in.fdt_phys)anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.Quoted 6 lines
Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.Quoted 3 lines
KHO already accesses stuct page early and uses
page->privatefor order. Since preserved memory is reserved in memblock, deferred init of struct pages won't touch those pages, we just need to make sure they are properlyNot strictly true. Some of them might have been initialized from
free_area_init()->memmap_init()(the ones not eligible for deferred init), which happens before KHO makes its memblock reservations.Quoted 3 lines
initialized at some point. If we don't expect many
kho_restore_folio()beforepage_alloc_init_late()we can useinit_deferred_page()for early accesses.I tried doing this when looking into this initially, but it doesn't work for some reason.
Quoted 6 lines
static void kho_restore_page(struct page *page, unsigned int order)
{
unsigned int i, nr_pages = (1 << order);
/* Head page gets refcount of 1. */
init_deferred_page(page_to_pfn(page), NUMA_NO_NODE);This would do
if (early_page_initialised(pfn, nid)) return; __init_page_from_nid(pfn, nid);
and I'm really surprised it didn't crash in early_page_initialised() because of NUMA_NO_NODE :)
What might work here is
pfn = page_to_pfn(page); __init_page_from_nid(pfn, early_pfn_to_nid(pfn));
Quoted 31 lines
set_page_count(page, 1); /* For higher order folios, tail pages get a page count of zero. */ for (i = 1; i < nr_pages; i++) { init_deferred_page(page_to_pfn(page + i), NUMA_NO_NODE); set_page_count(page + i, 0); } [...]results in:
[ 0.644032] page:(____ptrval____) is uninitialized and poisoned [ 0.644679] page dumped because: VM_BUG_ON_PAGE(PagePoisoned(page)) [ 0.645376] ------------[ cut here ]------------ [ 0.645883] kernel BUG at ./include/linux/mm.h:1512! [...] [ 0.647924] RIP: 0010:__pageblock_pfn_to_page+0x166/0x180 [...] [ 0.647924] <TASK> [ 0.647924] set_zone_contiguous+0x6b/0x90 [ 0.647924] page_alloc_init_late+0x356/0x370 [ 0.647924] kernel_init_freeable+0x12d/0x190 [ 0.647924] ? __pfx_kernel_init+0x10/0x10 [ 0.647924] kernel_init+0x1a/0x130didn't dig any deeper on why it happens...
Signature · 2 lines
Regards, Pratyush Yadav
Signature · 2 lines
Sincerely yours, Mike.
Pratyush Yadav
maintainerQuoted 98 lines · Mike Rapoport
Quoted 87 lines · Pratyush Yadav
Quoted 70 lines · Mike Rapoport
Quoted 62 lines · Pasha Tatashin
Quoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.
Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)
Hmm, weird, worked for me :/
Worked for me as well.
Quoted 32 lines
I am super confused, it did not work for me over weekend, and now it is working. Even
make menuconfigwould not work. Anyways, I will put it in the appropriate place.Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
We could, but with that would mean we'll run this before SMP and it's not desirable. Also,
init_deferred_page()for a random page requiresWe already run KHO init before
smp_init:start_kernel()->mm_core_init()->kho_memory_init()->kho_restore_folio()-> struct pages must be already initialized here!While deferred struct pages are initialized:
start_kernel()->rest_init()->kernel_init()->kernel_init_freeable()->page_alloc_init_late()->deferred_init_memmap()Quoted 5 lines
If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see
deferred_grow_zone()Quoted 3 lines
deferred_grow_zone()takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early,deferred_init_memmap()will overwrite them.Quoted 5 lines
Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.
Quoted 8 lines
Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.
Currently the only user of
kho_restore_folio()during init iskho_memory_init(). The FDT is accessed by doingphys_to_virt(kho_in.fdt_phys)anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.Quoted 6 lines
Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.Quoted 3 lines
KHO already accesses stuct page early and uses
page->privatefor order. Since preserved memory is reserved in memblock, deferred init of struct pages won't touch those pages, we just need to make sure they are properlyNot strictly true. Some of them might have been initialized from
free_area_init()->memmap_init()(the ones not eligible for deferred init), which happens before KHO makes its memblock reservations.Quoted 3 lines
initialized at some point. If we don't expect many
kho_restore_folio()beforepage_alloc_init_late()we can useinit_deferred_page()for early accesses.I tried doing this when looking into this initially, but it doesn't work for some reason.
Quoted 6 lines
static void kho_restore_page(struct page *page, unsigned int order) { unsigned int i, nr_pages = (1 << order); /* Head page gets refcount of 1. */ init_deferred_page(page_to_pfn(page), NUMA_NO_NODE);This would do
Quoted 7 lines
if (early_page_initialised(pfn, nid)) return; __init_page_from_nid(pfn, nid);and I'm really surprised it didn't crash in
early_page_initialised()because ofNUMA_NO_NODE:)
Oh, right. Using the wrong node completely throwsearly_page_initialised() off.
Quoted 5 lines
What might work here is
pfn = page_to_pfn(page); __init_page_from_nid(pfn, early_pfn_to_nid(pfn));
Yep, that works. Although this would do early_pfn_to_nid() for each page so it isn't very efficient. And we also need to make sure memblock does not go away.
Quoted 11 lines
Quoted 11 lines
set_page_count(page, 1); /* For higher order folios, tail pages get a page count of zero. */ for (i = 1; i < nr_pages; i++) { init_deferred_page(page_to_pfn(page + i), NUMA_NO_NODE); set_page_count(page + i, 0); } [...]
[...]
Signature · 2 lines
Regards, Pratyush Yadav
Mike Rapoport
maintainerQuoted 112 lines · Pratyush Yadav
Quoted 98 lines · Mike Rapoport
Quoted 87 lines · Pratyush Yadav
Quoted 70 lines · Mike Rapoport
Quoted 62 lines · Pasha Tatashin
Quoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.
Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)
Hmm, weird, worked for me :/
Worked for me as well.
Quoted 32 lines
I am super confused, it did not work for me over weekend, and now it is working. Even
make menuconfigwould not work. Anyways, I will put it in the appropriate place.Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines
We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.
We could, but with that would mean we'll run this before SMP and it's not desirable. Also,
init_deferred_page()for a random page requiresWe already run KHO init before
smp_init:start_kernel()->mm_core_init()->kho_memory_init()->kho_restore_folio()-> struct pages must be already initialized here!While deferred struct pages are initialized:
start_kernel()->rest_init()->kernel_init()->kernel_init_freeable()->page_alloc_init_late()->deferred_init_memmap()Quoted 5 lines
If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see
deferred_grow_zone()Quoted 3 lines
deferred_grow_zone()takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early,deferred_init_memmap()will overwrite them.Quoted 5 lines
Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.
Quoted 8 lines
Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.
Currently the only user of
kho_restore_folio()during init iskho_memory_init(). The FDT is accessed by doingphys_to_virt(kho_in.fdt_phys)anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.Quoted 6 lines
Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.Quoted 3 lines
KHO already accesses stuct page early and uses
page->privatefor order. Since preserved memory is reserved in memblock, deferred init of struct pages won't touch those pages, we just need to make sure they are properlyNot strictly true. Some of them might have been initialized from
free_area_init()->memmap_init()(the ones not eligible for deferred init), which happens before KHO makes its memblock reservations.Quoted 3 lines
initialized at some point. If we don't expect many
kho_restore_folio()beforepage_alloc_init_late()we can useinit_deferred_page()for early accesses.I tried doing this when looking into this initially, but it doesn't work for some reason.
Quoted 6 lines
static void kho_restore_page(struct page *page, unsigned int order) { unsigned int i, nr_pages = (1 << order); /* Head page gets refcount of 1. */ init_deferred_page(page_to_pfn(page), NUMA_NO_NODE);This would do
Quoted 7 lines
if (early_page_initialised(pfn, nid)) return; __init_page_from_nid(pfn, nid);and I'm really surprised it didn't crash in
early_page_initialised()because ofNUMA_NO_NODE:)Oh, right. Using the wrong node completely throws
early_page_initialised()off.Quoted 1 line
Quoted 8 lines
Quoted 4 lines
What might work here is
pfn = page_to_pfn(page); __init_page_from_nid(pfn, early_pfn_to_nid(pfn));Yep, that works. Although this would do
early_pfn_to_nid()for each page so it isn't very efficient. And we also need to make sure memblock does not go away.
Yeah, that's far from ideal. Let's just gate KHO on !CONFIG_DEFERRED_PAGE_STRUCT_INIT for now.
Quoted 15 lines
Quoted 10 lines
Quoted 10 lines
set_page_count(page, 1); /* For higher order folios, tail pages get a page count of zero. */ for (i = 1; i < nr_pages; i++) { init_deferred_page(page_to_pfn(page + i), NUMA_NO_NODE); set_page_count(page + i, 0); } [...][...]
Signature · 2 lines
Regards, Pratyush Yadav
Signature · 2 lines
Sincerely yours, Mike.
Pratyush Yadav
maintainerQuoted 3 lines · Pasha Tatashin
Quoted 2 lines · Pratyush Yadav
On Tue, Jun 10 2025, Pasha Tatashin wrote:
[...]
Quoted 37 lines
Quoted 28 lines
Quoted 22 lines
Quoted 18 lines
Quoted 12 lines
Quoted 2 lines
We could, but with that would mean we'll run this before SMP and it's not desirable. Also,
init_deferred_page()for a random page requiresWe already run KHO init before
smp_init:start_kernel()->mm_core_init()->kho_memory_init()->kho_restore_folio()-> struct pages must be already initialized here!While deferred struct pages are initialized:
start_kernel()->rest_init()->kernel_init()->kernel_init_freeable()->page_alloc_init_late()->deferred_init_memmap()Quoted 5 lines
If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see
deferred_grow_zone()Quoted 3 lines
deferred_grow_zone()takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early,deferred_init_memmap()will overwrite them.Quoted 5 lines
Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.
Quoted 8 lines
Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.
Currently the only user of
kho_restore_folio()during init iskho_memory_init(). The FDT is accessed by doingphys_to_virt(kho_in.fdt_phys)anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.
Quoted 6 lines
Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.
Right. We might end up needing it down the line. But from a quick look, it doesn't seem to be trivial to solve, so IMO we should solve it when those use cases actually show up, and keep things simple for now.
Signature · 2 lines
Regards, Pratyush Yadav