Skip to contentSkip to conversation

kho: initialize tail pages for higher order folios properly

PATCH Superseded Pratyush Yadav 4 participants 18 messages Jun 5, 2025 – Jun 13, 2025

Subsystems (4)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Pratyush Yadav

author maintainer
#0 lore raw

PATCH 1/1 kho: initialize tail pages for higher order folios properly

From: Pratyush Yadav <[email protected]>

Currently, when restoring higher order folios, kho_restore_folio() only calls prep_compound_page() on all the pages. That is not enough to properly initialize the folios. The managed page count does not get updated, the reserved flag does not get dropped, and page count does not get initialized properly.

Restoring a higher order folio with it results in the following BUG with CONFIG_DEBUG_VM when attempting to free the folio:

    BUG: Bad page state in process test  pfn:104e2b
    page: refcount:1 mapcount:0 mapping:0000000000000000 index:0xffffffffffffffff pfn:0x104e2b
    flags: 0x2fffff80000000(node=0|zone=2|lastcpupid=0x1fffff)
    raw: 002fffff80000000 0000000000000000 00000000ffffffff 0000000000000000
    raw: ffffffffffffffff 0000000000000000 00000001ffffffff 0000000000000000
    page dumped because: nonzero _refcount
    [...]
    Call Trace:
    <TASK>
    dump_stack_lvl+0x4b/0x70
    bad_page.cold+0x97/0xb2
    __free_frozen_pages+0x616/0x850
    [...]

Combine the path for 0-order and higher order folios, initialize the tail pages with a count of zero, and call adjust_managed_page_count() to account for all the pages instead of just missing them.

In addition, since all the KHO-preserved pages get marked with MEMBLOCK_RSRV_NOINIT by deserialize_bitmap(), the reserved flag is not actually set (as can also be seen from the flags of the dumped page in the logs above). So drop the ClearPageReserved() calls.

Fixes fc33e4b44b271 ("kexec: enable KHO support for memory preservation")

---

Side note: get_maintainers.pl for KHO only lists kexec@ as the mailing list. Since KHO has a bunch of MM bits as well, should we also add linux-mm@ to its MAINTAINERS entry?

Adding linux-mm@ to this patch at least, in case MM people have an opinion on this.

 kernel/kexec_handover.c | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
kernel/kexec_handover.c +17 -12
index eb305e7e61296..5214ab27d1f8d 100644
@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
 }
 
 /* almost as free_reserved_page(), just don't free the page */
-static void kho_restore_page(struct page *page)
+static void kho_restore_page(struct page *page, unsigned int order)
 {
-	ClearPageReserved(page);
-	init_page_count(page);
-	adjust_managed_page_count(page, 1);
+	unsigned int i, nr_pages = (1 << order);
+
+	/* Head page gets refcount of 1. */
+	set_page_count(page, 1);
+
+	/* For higher order folios, tail pages get a page count of zero. */
+	for (i = 1; i < nr_pages; i++)
+		set_page_count(page + i, 0);
+
+	if (order > 0)
+		prep_compound_page(page, order);
+
+	adjust_managed_page_count(page, nr_pages);
 }
 
 /**
@@ -179,15 +189,10 @@ struct folio *kho_restore_folio(phys_addr_t phys)
 		return NULL;
 
 	order = page->private;
-	if (order) {
-		if (order > MAX_PAGE_ORDER)
-			return NULL;
-
-		prep_compound_page(page, order);
-	} else {
-		kho_restore_page(page);
-	}
+	if (order > MAX_PAGE_ORDER)
+		return NULL;
 
+	kho_restore_page(page, order);
 	return page_folio(page);
 }
 EXPORT_SYMBOL_GPL(kho_restore_folio);

--
2.47.1

maintainer
#1 lore raw
Quoted 3 lines · On Thu, 5 Jun 2025 19:11:41 +0200 Pratyush Yadav <[email protected]> wrote:

Side note: get_maintainers.pl for KHO only lists kexec@ as the mailing list. Since KHO has a bunch of MM bits as well, should we also add linux-mm@ to its MAINTAINERS entry?

I think so. Please send a patch?

maintainer
#2 lore raw
In reply to Pratyush Yadav
Quoted 53 lines · Pratyush Yadav

From: Pratyush Yadav <[email protected]>

Currently, when restoring higher order folios, kho_restore_folio() only calls prep_compound_page() on all the pages. That is not enough to properly initialize the folios. The managed page count does not get updated, the reserved flag does not get dropped, and page count does not get initialized properly.

Restoring a higher order folio with it results in the following BUG with CONFIG_DEBUG_VM when attempting to free the folio:

    BUG: Bad page state in process test  pfn:104e2b
    page: refcount:1 mapcount:0 mapping:0000000000000000 index:0xffffffffffffffff pfn:0x104e2b
    flags: 0x2fffff80000000(node=0|zone=2|lastcpupid=0x1fffff)
    raw: 002fffff80000000 0000000000000000 00000000ffffffff 0000000000000000
    raw: ffffffffffffffff 0000000000000000 00000001ffffffff 0000000000000000
    page dumped because: nonzero _refcount
    [...]
    Call Trace:
    <TASK>
    dump_stack_lvl+0x4b/0x70
    bad_page.cold+0x97/0xb2
    __free_frozen_pages+0x616/0x850
    [...]

Combine the path for 0-order and higher order folios, initialize the tail pages with a count of zero, and call adjust_managed_page_count() to account for all the pages instead of just missing them.

In addition, since all the KHO-preserved pages get marked with MEMBLOCK_RSRV_NOINIT by deserialize_bitmap(), the reserved flag is not actually set (as can also be seen from the flags of the dumped page in the logs above). So drop the ClearPageReserved() calls.

Fixes fc33e4b44b271 ("kexec: enable KHO support for memory preservation")

---

Side note: get_maintainers.pl for KHO only lists kexec@ as the mailing list. Since KHO has a bunch of MM bits as well, should we also add linux-mm@ to its MAINTAINERS entry?

Adding linux-mm@ to this patch at least, in case MM people have an opinion on this.

 kernel/kexec_handover.c | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
kernel/kexec_handover.c +0 -0
index eb305e7e61296..5214ab27d1f8d 100644

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162

Quoted 7 lines
kernel/kexec_handover.c +1 -2
@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
 }
 
 /* almost as free_reserved_page(), just don't free the page */
-static void kho_restore_page(struct page *page)
+static void kho_restore_page(struct page *page, unsigned int order)
 {
-	ClearPageReserved(page);

So now we don't clear PG_Reserved even on order-0 pages? ;-)

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162

Quoted 3 lines
kernel/kexec_handover.c +1 -2
-	init_page_count(page);
-	adjust_managed_page_count(page, 1);
+	unsigned int i, nr_pages = (1 << order);

Can you please declare 'i' inside the loop, looks nicer IMHO.

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 165

Quoted 3 lines
kernel/kexec_handover.c +3 -0
+
+	/* Head page gets refcount of 1. */
+	set_page_count(page, 1);

ClearPageReserved(page) here?

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 169

Quoted 4 lines
kernel/kexec_handover.c +4 -0
+
+	/* For higher order folios, tail pages get a page count of zero. */
+	for (i = 1; i < nr_pages; i++)
+		set_page_count(page + i, 0);

and here?

Quoted 30 lines
kernel/kexec_handover.c +8 -9
+
+	if (order > 0)
+		prep_compound_page(page, order);
+
+	adjust_managed_page_count(page, nr_pages);
 }
 
 /**
@@ -179,15 +189,10 @@ struct folio *kho_restore_folio(phys_addr_t phys)
 		return NULL;
 
 	order = page->private;
-	if (order) {
-		if (order > MAX_PAGE_ORDER)
-			return NULL;
-
-		prep_compound_page(page, order);
-	} else {
-		kho_restore_page(page);
-	}
+	if (order > MAX_PAGE_ORDER)
+		return NULL;
 
+	kho_restore_page(page, order);
 	return page_folio(page);
 }
 EXPORT_SYMBOL_GPL(kho_restore_folio);
--

2.47.1

Signature · 2 lines
Sincerely yours,
Mike.
author maintainer
#3 lore raw

Hi Mike,

Quoted 56 lines · Mike Rapoport
Quoted 55 lines · Pratyush Yadav

From: Pratyush Yadav <[email protected]>

Currently, when restoring higher order folios, kho_restore_folio() only calls prep_compound_page() on all the pages. That is not enough to properly initialize the folios. The managed page count does not get updated, the reserved flag does not get dropped, and page count does not get initialized properly.

Restoring a higher order folio with it results in the following BUG with CONFIG_DEBUG_VM when attempting to free the folio:

    BUG: Bad page state in process test  pfn:104e2b
    page: refcount:1 mapcount:0 mapping:0000000000000000 index:0xffffffffffffffff pfn:0x104e2b
    flags: 0x2fffff80000000(node=0|zone=2|lastcpupid=0x1fffff)
    raw: 002fffff80000000 0000000000000000 00000000ffffffff 0000000000000000
    raw: ffffffffffffffff 0000000000000000 00000001ffffffff 0000000000000000
    page dumped because: nonzero _refcount
    [...]
    Call Trace:
    <TASK>
    dump_stack_lvl+0x4b/0x70
    bad_page.cold+0x97/0xb2
    __free_frozen_pages+0x616/0x850
    [...]

Combine the path for 0-order and higher order folios, initialize the tail pages with a count of zero, and call adjust_managed_page_count() to account for all the pages instead of just missing them.

In addition, since all the KHO-preserved pages get marked with MEMBLOCK_RSRV_NOINIT by deserialize_bitmap(), the reserved flag is not actually set (as can also be seen from the flags of the dumped page in the logs above). So drop the ClearPageReserved() calls.

Fixes fc33e4b44b271 ("kexec: enable KHO support for memory preservation")

---

Side note: get_maintainers.pl for KHO only lists kexec@ as the mailing list. Since KHO has a bunch of MM bits as well, should we also add linux-mm@ to its MAINTAINERS entry?

Adding linux-mm@ to this patch at least, in case MM people have an opinion on this.

 kernel/kexec_handover.c | 29 +++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)
kernel/kexec_handover.c +0 -0
index eb305e7e61296..5214ab27d1f8d 100644
@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
 }
 
Quoted 7 lines

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162

Quoted 5 lines
kernel/kexec_handover.c +1 -2
 /* almost as free_reserved_page(), just don't free the page */
-static void kho_restore_page(struct page *page)
+static void kho_restore_page(struct page *page, unsigned int order)
 {
-	ClearPageReserved(page);

So now we don't clear PG_Reserved even on order-0 pages? ;-)

We don't need to. As I mentioned in the commit message as well, PG_Reserved is never set for KHO pages since they are reserved with MEMBLOCK_RSRV_NOINIT, so memmap_init_reserved_pages() skips over them.

To double-check, I added some quick prints to kho_restore_page():

    /* Head page gets refcount of 1. */
    set_page_count(page, 1);
    printk("Head page flags: 0x%lx reserved: %d\n",
           page->flags, PageReserved(page));
    
    /* For higher order folios, tail pages get a page count of zero. */
    for (i = 1; i < nr_pages; i++) {
    	printk("Tail page %u flags: 0x%lx reserved: %d\n",
    		i, (page+i)->flags, PageReserved(page+i));
    	set_page_count(page + i, 0);
    }

And this is what I get:

    [    9.003187] Head page flags: 0x2fffff80000000 reserved: 0
    [    9.003730] Tail page 1 flags: 0x2fffff80000000 reserved: 0
    [    9.004229] Tail page 2 flags: 0x2fffff80000000 reserved: 0
    [    9.004740] Tail page 3 flags: 0x2fffff80000000 reserved: 0
    [    9.005265] Head page flags: 0x2fffff80000000 reserved: 0
    [    9.005759] Head page flags: 0x2fffff80000000 reserved: 0
    [...]

So PG_Reserved is never set.

That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, with CONFIG_DEFERRED_STRUCT_PAGE_INIT == n, all the pages get initialized when memmap_init_range() is called from setup_arch (paging_init() on x86). This happens before kho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.

With deferred page init, some pages might not get initialized early, and get initialized after kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So, deferred_init_maxorder() will skip over those pages and leave them uninitialized.

And sure enough, doing the same with CONFIG_DEFERRED_STRUCT_PAGE_INIT == y results in:

    [   10.060842] Head page flags: 0x2fffff80000000 reserved: 0
    [   10.061387] Tail page 1 flags: 0x2fffff80000000 reserved: 0
    [   10.061902] Tail page 2 flags: 0x2fffff80000000 reserved: 0
    [   10.062400] Tail page 3 flags: 0x2fffff80000000 reserved: 0
    [   10.062924] page:00000000fb3dca54 is uninitialized and poisoned
    [   10.063494] page dumped because: VM_BUG_ON_PAGE(PagePoisoned(page))
    [   10.064190] ------------[ cut here ]------------
    [   10.064636] kernel BUG at ./include/linux/page-flags.h:571!
    [   10.065194] Oops: invalid opcode: 0000 [#1] SMP PTI
    [   10.065661] CPU: 2 UID: 0 PID: 1954 Comm: test Not tainted 6.15.0+ #297 PREEMPT(undef)
    [   10.066449] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.3-1-1 04/01/2014
    [   10.067353] RIP: 0010:kho_restore_folio+0x4e/0x70
    [...]

So we need to either also call init_deferred_page(), or remove the memblock_reserved_mark_noinit() call in deserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.

What do you think? Should we drop noinit or call init_deferred_page()? FWIW, my preference is to drop noinit, since init_deferred_page() is __meminit and we would have to make sure it doesn't go away after boot.

Quoted 6 lines

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162

Quoted 4 lines
kernel/kexec_handover.c +1 -2
-	init_page_count(page);
-	adjust_managed_page_count(page, 1);
+	unsigned int i, nr_pages = (1 << order);

Can you please declare 'i' inside the loop, looks nicer IMHO.

Ok, will do.

Quoted 22 lines

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 165

Quoted 4 lines
kernel/kexec_handover.c +3 -0
+
+	/* Head page gets refcount of 1. */
+	set_page_count(page, 1);

ClearPageReserved(page) here?

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 169

Quoted 4 lines
kernel/kexec_handover.c +4 -0
+
+	/* For higher order folios, tail pages get a page count of zero. */
+	for (i = 1; i < nr_pages; i++)
+		set_page_count(page + i, 0);

and here?

Quoted 8 lines
kernel/kexec_handover.c +5 -0
+
+	if (order > 0)
+		prep_compound_page(page, order);
+
+	adjust_managed_page_count(page, nr_pages);
 }
 
 /**

[...]

Signature · 2 lines
Regards,
Pratyush Yadav
maintainer
#4 lore raw

Hi Pratyush,

Quoted 20 lines · Pratyush Yadav

Hi Mike,

Quoted 8 lines · Mike Rapoport
Quoted 7 lines · Pratyush Yadav

From: Pratyush Yadav <[email protected]>

kernel/kexec_handover.c +0 -0
@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
 }
 
Quoted 7 lines

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162

Quoted 5 lines
kernel/kexec_handover.c +1 -2
 /* almost as free_reserved_page(), just don't free the page */
-static void kho_restore_page(struct page *page)
+static void kho_restore_page(struct page *page, unsigned int order)
 {
-	ClearPageReserved(page);

So now we don't clear PG_Reserved even on order-0 pages? ;-)

Quoted 3 lines

We don't need to. As I mentioned in the commit message as well, PG_Reserved is never set for KHO pages since they are reserved with MEMBLOCK_RSRV_NOINIT, so memmap_init_reserved_pages() skips over them.

You are right, I missed it.

Quoted 12 lines

That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, with CONFIG_DEFERRED_STRUCT_PAGE_INIT == n, all the pages get initialized when memmap_init_range() is called from setup_arch (paging_init() on x86). This happens before kho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.

With deferred page init, some pages might not get initialized early, and get initialized after kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So, deferred_init_maxorder() will skip over those pages and leave them uninitialized.

Quoted 6 lines

So we need to either also call init_deferred_page(), or remove the memblock_reserved_mark_noinit() call in deserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.

One benefit is performance indeed, because in not deferred case the initialization of reserved pages in memmap_init_reserved_pages() is really excessive.

But more importantly, if we remove memblock_reserved_mark_noinit(), with CONFIG_DEFERRED_STRUCT_PAGE_INIT we'd loose page->private because the struct page will be cleared after kho_mem_deserialize().

Quoted 3 lines

What do you think? Should we drop noinit or call init_deferred_page()? FWIW, my preference is to drop noinit, since init_deferred_page() is __meminit and we would have to make sure it doesn't go away after boot.

We can't drop noinit and calling init_deferred_page() after boot just won't work because it uses memblock to find the page's node and memblock is gone after init.

The simplest short-term solution is to disable KHO when CONFIG_DEFERRED_STRUCT_PAGE_INIT is set and then find an efficient way to make it all work together.

Signature · 2 lines
Sincerely yours,
Mike.
maintainer
#5 lore raw
Quoted 61 lines · Mike Rapoport

Hi Pratyush,

Quoted 20 lines · Pratyush Yadav

Hi Mike,

Quoted 8 lines · Mike Rapoport
Quoted 7 lines · Pratyush Yadav

From: Pratyush Yadav <[email protected]>

kernel/kexec_handover.c +0 -0
@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
 }
 
Quoted 7 lines

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162

Quoted 5 lines
kernel/kexec_handover.c +1 -2
 /* almost as free_reserved_page(), just don't free the page */
-static void kho_restore_page(struct page *page)
+static void kho_restore_page(struct page *page, unsigned int order)
 {
-  ClearPageReserved(page);

So now we don't clear PG_Reserved even on order-0 pages? ;-)

Quoted 3 lines

We don't need to. As I mentioned in the commit message as well, PG_Reserved is never set for KHO pages since they are reserved with MEMBLOCK_RSRV_NOINIT, so memmap_init_reserved_pages() skips over them.

You are right, I missed it.

Quoted 12 lines

That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, with CONFIG_DEFERRED_STRUCT_PAGE_INIT == n, all the pages get initialized when memmap_init_range() is called from setup_arch (paging_init() on x86). This happens before kho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.

With deferred page init, some pages might not get initialized early, and get initialized after kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So, deferred_init_maxorder() will skip over those pages and leave them uninitialized.

Quoted 6 lines

So we need to either also call init_deferred_page(), or remove the memblock_reserved_mark_noinit() call in deserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.

One benefit is performance indeed, because in not deferred case the initialization of reserved pages in memmap_init_reserved_pages() is really excessive.

But more importantly, if we remove memblock_reserved_mark_noinit(), with CONFIG_DEFERRED_STRUCT_PAGE_INIT we'd loose page->private because the struct page will be cleared after kho_mem_deserialize().

Quoted 3 lines

What do you think? Should we drop noinit or call init_deferred_page()? FWIW, my preference is to drop noinit, since init_deferred_page() is __meminit and we would have to make sure it doesn't go away after boot.

Quoted 7 lines

We can't drop noinit and calling init_deferred_page() after boot just won't work because it uses memblock to find the page's node and memblock is gone after init.

The simplest short-term solution is to disable KHO when CONFIG_DEFERRED_STRUCT_PAGE_INIT is set and then find an efficient way to make it all work together.

This is what I've done in LUOv3 WIP:
https://github.com/soleen/linux/commit/3059f38ac0a39a397873759fb429bd5d1f8ea681

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

Pasha

Quoted 4 lines

--
Sincerely yours,
Mike.

maintainer
#6 lore raw
Quoted 66 lines · Pasha Tatashin
Quoted 65 lines · Mike Rapoport

Hi Pratyush,

Quoted 20 lines · Pratyush Yadav

Hi Mike,

Quoted 8 lines · Mike Rapoport
Quoted 7 lines · Pratyush Yadav

From: Pratyush Yadav <[email protected]>

kernel/kexec_handover.c +0 -0
@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
 }
 
Quoted 7 lines

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162

Quoted 5 lines
kernel/kexec_handover.c +1 -2
 /* almost as free_reserved_page(), just don't free the page */
-static void kho_restore_page(struct page *page)
+static void kho_restore_page(struct page *page, unsigned int order)
 {
-  ClearPageReserved(page);

So now we don't clear PG_Reserved even on order-0 pages? ;-)

Quoted 3 lines

We don't need to. As I mentioned in the commit message as well, PG_Reserved is never set for KHO pages since they are reserved with MEMBLOCK_RSRV_NOINIT, so memmap_init_reserved_pages() skips over them.

You are right, I missed it.

Quoted 12 lines

That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, with CONFIG_DEFERRED_STRUCT_PAGE_INIT == n, all the pages get initialized when memmap_init_range() is called from setup_arch (paging_init() on x86). This happens before kho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.

With deferred page init, some pages might not get initialized early, and get initialized after kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So, deferred_init_maxorder() will skip over those pages and leave them uninitialized.

Quoted 6 lines

So we need to either also call init_deferred_page(), or remove the memblock_reserved_mark_noinit() call in deserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.

One benefit is performance indeed, because in not deferred case the initialization of reserved pages in memmap_init_reserved_pages() is really excessive.

But more importantly, if we remove memblock_reserved_mark_noinit(), with CONFIG_DEFERRED_STRUCT_PAGE_INIT we'd loose page->private because the struct page will be cleared after kho_mem_deserialize().

Quoted 3 lines

What do you think? Should we drop noinit or call init_deferred_page()? FWIW, my preference is to drop noinit, since init_deferred_page() is __meminit and we would have to make sure it doesn't go away after boot.

We can't drop noinit and calling init_deferred_page() after boot just won't work because it uses memblock to find the page's node and memblock is gone after init.

Quoted 6 lines
Quoted 3 lines

The simplest short-term solution is to disable KHO when CONFIG_DEFERRED_STRUCT_PAGE_INIT is set and then find an efficient way to make it all work together.

This is what I've done in LUOv3 WIP:
https://github.com/soleen/linux/commit/3059f38ac0a39a397873759fb429bd5d1f8ea681

I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.

Quoted 3 lines

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires finding its node with early_pfn_to_nid() that's also suboptimal.

Quoted 6 lines

Pasha

Quoted 4 lines

--
Sincerely yours,
Mike.

Signature · 2 lines
Sincerely yours,
Mike.
maintainer
#7 lore raw
Quoted 72 lines · Mike Rapoport
Quoted 70 lines · Pasha Tatashin
Quoted 68 lines · Mike Rapoport

Hi Pratyush,

Quoted 20 lines · Pratyush Yadav

Hi Mike,

Quoted 8 lines · Mike Rapoport
Quoted 7 lines · Pratyush Yadav

From: Pratyush Yadav <[email protected]>

kernel/kexec_handover.c +0 -0
@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
 }
 
Quoted 7 lines

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162

Quoted 5 lines
kernel/kexec_handover.c +1 -2
 /* almost as free_reserved_page(), just don't free the page */
-static void kho_restore_page(struct page *page)
+static void kho_restore_page(struct page *page, unsigned int order)
 {
-  ClearPageReserved(page);

So now we don't clear PG_Reserved even on order-0 pages? ;-)

Quoted 3 lines

We don't need to. As I mentioned in the commit message as well, PG_Reserved is never set for KHO pages since they are reserved with MEMBLOCK_RSRV_NOINIT, so memmap_init_reserved_pages() skips over them.

You are right, I missed it.

Quoted 12 lines

That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, with CONFIG_DEFERRED_STRUCT_PAGE_INIT == n, all the pages get initialized when memmap_init_range() is called from setup_arch (paging_init() on x86). This happens before kho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.

With deferred page init, some pages might not get initialized early, and get initialized after kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So, deferred_init_maxorder() will skip over those pages and leave them uninitialized.

Quoted 6 lines

So we need to either also call init_deferred_page(), or remove the memblock_reserved_mark_noinit() call in deserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.

One benefit is performance indeed, because in not deferred case the initialization of reserved pages in memmap_init_reserved_pages() is really excessive.

But more importantly, if we remove memblock_reserved_mark_noinit(), with CONFIG_DEFERRED_STRUCT_PAGE_INIT we'd loose page->private because the struct page will be cleared after kho_mem_deserialize().

Quoted 3 lines

What do you think? Should we drop noinit or call init_deferred_page()? FWIW, my preference is to drop noinit, since init_deferred_page() is __meminit and we would have to make sure it doesn't go away after boot.

We can't drop noinit and calling init_deferred_page() after boot just won't work because it uses memblock to find the page's node and memblock is gone after init.

The simplest short-term solution is to disable KHO when CONFIG_DEFERRED_STRUCT_PAGE_INIT is set and then find an efficient way to make it all work together.

Quoted 5 lines
Quoted 2 lines

This is what I've done in LUOv3 WIP:
https://github.com/soleen/linux/commit/3059f38ac0a39a397873759fb429bd5d1f8ea681

I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.

Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)

Quoted 6 lines
Quoted 3 lines

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires

We already run KHO init before smp_init:
start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!

While deferred struct pages are initialized:
start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()

If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()

Pasha

maintainer
#8 lore raw
Quoted 76 lines · Pasha Tatashin
Quoted 75 lines · Mike Rapoport
Quoted 72 lines · Pasha Tatashin
Quoted 61 lines · Mike Rapoport

Hi Pratyush,

Quoted 20 lines · Pratyush Yadav

Hi Mike,

Quoted 8 lines · Mike Rapoport
Quoted 7 lines · Pratyush Yadav

From: Pratyush Yadav <[email protected]>

kernel/kexec_handover.c +0 -0
@@ -157,11 +157,21 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,
 }
 
Quoted 7 lines

kernel/kexec_handover.c static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn, · line 162

Quoted 5 lines
kernel/kexec_handover.c +1 -2
 /* almost as free_reserved_page(), just don't free the page */
-static void kho_restore_page(struct page *page)
+static void kho_restore_page(struct page *page, unsigned int order)
 {
-  ClearPageReserved(page);

So now we don't clear PG_Reserved even on order-0 pages? ;-)

Quoted 3 lines

We don't need to. As I mentioned in the commit message as well, PG_Reserved is never set for KHO pages since they are reserved with MEMBLOCK_RSRV_NOINIT, so memmap_init_reserved_pages() skips over them.

You are right, I missed it.

Quoted 12 lines

That said, while reading through some of the code, I noticed another
bug: because KHO reserves the preserved pages as NOINIT, with CONFIG_DEFERRED_STRUCT_PAGE_INIT == n, all the pages get initialized when memmap_init_range() is called from setup_arch (paging_init() on x86). This happens before kho_memory_init(), so the KHO-preserved pages are not marked as reserved to memblock yet.

With deferred page init, some pages might not get initialized early, and get initialized after kho_memory_init(), by which time the KHO-preserved pages are marked as reserved. So, deferred_init_maxorder() will skip over those pages and leave them uninitialized.

Quoted 6 lines

So we need to either also call init_deferred_page(), or remove the memblock_reserved_mark_noinit() call in deserialize_bitmap(). And TBH, I am not sure why KHO pages even need to be marked noinit in the first place. Probably the only benefit would be if a large chunk of memory is KHO-preserved, the pages can be initialized later on-demand, reducing bootup time a bit.

One benefit is performance indeed, because in not deferred case the initialization of reserved pages in memmap_init_reserved_pages() is really excessive.

But more importantly, if we remove memblock_reserved_mark_noinit(), with CONFIG_DEFERRED_STRUCT_PAGE_INIT we'd loose page->private because the struct page will be cleared after kho_mem_deserialize().

Quoted 3 lines

What do you think? Should we drop noinit or call init_deferred_page()? FWIW, my preference is to drop noinit, since init_deferred_page() is __meminit and we would have to make sure it doesn't go away after boot.

Quoted 7 lines

We can't drop noinit and calling init_deferred_page() after boot just won't work because it uses memblock to find the page's node and memblock is gone after init.

The simplest short-term solution is to disable KHO when CONFIG_DEFERRED_STRUCT_PAGE_INIT is set and then find an efficient way to make it all work together.

This is what I've done in LUOv3 WIP:
https://github.com/soleen/linux/commit/3059f38ac0a39a397873759fb429bd5d1f8ea681

Quoted 6 lines
Quoted 2 lines

I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.

Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)

Hmm, weird, worked for me :/

Quoted 16 lines
Quoted 6 lines
Quoted 3 lines

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires

We already run KHO init before smp_init:
start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!

While deferred struct pages are initialized:
start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()

Quoted 5 lines

If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()

deferred_grow_zone() takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early, deferred_init_memmap() will overwrite them.

Anyway, I'm going to look into it, hopefully I'll have something Really Soon (tm).

Quoted 1 line

Pasha

Signature · 2 lines
Sincerely yours,
Mike.
maintainer
#9 lore raw
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.

Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)

Hmm, weird, worked for me :/

I am super confused, it did not work for me over weekend, and now it is working. Even make menuconfig would not work. Anyways, I will put it in the appropriate place.

Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires

We already run KHO init before smp_init:
start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!

While deferred struct pages are initialized:
start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()

Quoted 5 lines

If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()

Quoted 3 lines

deferred_grow_zone() takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early, deferred_init_memmap() will overwrite them.

Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.

Quoted 2 lines

Anyway, I'm going to look into it, hopefully I'll have something Really Soon (tm).

CC Michal Clapinski, who was also planning to look into this problem.

Pasha

author maintainer
#10 lore raw
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.

Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)

Hmm, weird, worked for me :/

Worked for me as well.

Quoted 32 lines

I am super confused, it did not work for me over weekend, and now it is working. Even make menuconfig would not work. Anyways, I will put it in the appropriate place.

Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires

We already run KHO init before smp_init:
start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!

While deferred struct pages are initialized:
start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()

Quoted 5 lines

If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()

Quoted 3 lines

deferred_grow_zone() takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early, deferred_init_memmap() will overwrite them.

Quoted 5 lines

Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.

Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.

Currently the only user of kho_restore_folio() during init is kho_memory_init(). The FDT is accessed by doing phys_to_virt(kho_in.fdt_phys) anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.

Signature · 2 lines
Regards,
Pratyush Yadav
maintainer
#11 lore raw
Quoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.

Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)

Hmm, weird, worked for me :/

Worked for me as well.

Quoted 32 lines

I am super confused, it did not work for me over weekend, and now it is working. Even make menuconfig would not work. Anyways, I will put it in the appropriate place.

Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires

We already run KHO init before smp_init:
start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!

While deferred struct pages are initialized:
start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()

Quoted 5 lines

If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()

Quoted 3 lines

deferred_grow_zone() takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early, deferred_init_memmap() will overwrite them.

Quoted 5 lines

Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.

Quoted 8 lines

Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.

Currently the only user of kho_restore_folio() during init is kho_memory_init(). The FDT is accessed by doing phys_to_virt(kho_in.fdt_phys) anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.

Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.

Pasha

Quoted 4 lines

--
Regards,
Pratyush Yadav

maintainer
#12 lore raw
Quoted 62 lines · Pasha Tatashin
Quoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.

Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)

Hmm, weird, worked for me :/

Worked for me as well.

Quoted 32 lines

I am super confused, it did not work for me over weekend, and now it is working. Even make menuconfig would not work. Anyways, I will put it in the appropriate place.

Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires

We already run KHO init before smp_init:
start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!

While deferred struct pages are initialized:
start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()

Quoted 5 lines

If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()

Quoted 3 lines

deferred_grow_zone() takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early, deferred_init_memmap() will overwrite them.

Quoted 5 lines

Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.

Quoted 8 lines

Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.

Currently the only user of kho_restore_folio() during init is kho_memory_init(). The FDT is accessed by doing phys_to_virt(kho_in.fdt_phys) anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.

Quoted 6 lines

Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.

KHO already accesses stuct page early and uses page->private for order. Since preserved memory is reserved in memblock, deferred init of struct pages won't touch those pages, we just need to make sure they are properly initialized at some point. If we don't expect many kho_restore_folio() before page_alloc_init_late() we can use init_deferred_page() for early accesses.

Quoted 6 lines

Pasha

Quoted 4 lines

--
Regards,
Pratyush Yadav

Signature · 2 lines
Sincerely yours,
Mike.
author maintainer
#13 lore raw
Quoted 70 lines · Mike Rapoport
Quoted 62 lines · Pasha Tatashin
Quoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.

Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)

Hmm, weird, worked for me :/

Worked for me as well.

Quoted 32 lines

I am super confused, it did not work for me over weekend, and now it is working. Even make menuconfig would not work. Anyways, I will put it in the appropriate place.

Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires

We already run KHO init before smp_init:
start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!

While deferred struct pages are initialized:
start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()

Quoted 5 lines

If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()

Quoted 3 lines

deferred_grow_zone() takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early, deferred_init_memmap() will overwrite them.

Quoted 5 lines

Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.

Quoted 8 lines

Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.

Currently the only user of kho_restore_folio() during init is kho_memory_init(). The FDT is accessed by doing phys_to_virt(kho_in.fdt_phys) anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.

Quoted 6 lines

Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.

Quoted 3 lines

KHO already accesses stuct page early and uses page->private for order. Since preserved memory is reserved in memblock, deferred init of struct pages won't touch those pages, we just need to make sure they are properly

Not strictly true. Some of them might have been initialized from free_area_init() -> memmap_init() (the ones not eligible for deferred init), which happens before KHO makes its memblock reservations.

Quoted 3 lines

initialized at some point. If we don't expect many kho_restore_folio() before page_alloc_init_late() we can use init_deferred_page() for early accesses.

I tried doing this when looking into this initially, but it doesn't work for some reason.

    static void kho_restore_page(struct page *page, unsigned int order)
    {
    	unsigned int i, nr_pages = (1 << order);
    
    	/* Head page gets refcount of 1. */
    	init_deferred_page(page_to_pfn(page), NUMA_NO_NODE);
    	set_page_count(page, 1);
    
    	/* For higher order folios, tail pages get a page count of zero. */
    	for (i = 1; i < nr_pages; i++) {
    		init_deferred_page(page_to_pfn(page + i), NUMA_NO_NODE);
    		set_page_count(page + i, 0);
    	}
    
    [...]

results in:

    [    0.644032] page:(____ptrval____) is uninitialized and poisoned
    [    0.644679] page dumped because: VM_BUG_ON_PAGE(PagePoisoned(page))
    [    0.645376] ------------[ cut here ]------------
    [    0.645883] kernel BUG at ./include/linux/mm.h:1512!
    [...]
    [    0.647924] RIP: 0010:__pageblock_pfn_to_page+0x166/0x180
    [...]
    [    0.647924]  <TASK>
    [    0.647924]  set_zone_contiguous+0x6b/0x90
    [    0.647924]  page_alloc_init_late+0x356/0x370
    [    0.647924]  kernel_init_freeable+0x12d/0x190
    [    0.647924]  ? __pfx_kernel_init+0x10/0x10
    [    0.647924]  kernel_init+0x1a/0x130

didn't dig any deeper on why it happens...

Signature · 2 lines
Regards,
Pratyush Yadav
maintainer
#14 lore raw
Quoted 87 lines · Pratyush Yadav
Quoted 70 lines · Mike Rapoport
Quoted 62 lines · Pasha Tatashin
Quoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.

Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)

Hmm, weird, worked for me :/

Worked for me as well.

Quoted 32 lines

I am super confused, it did not work for me over weekend, and now it is working. Even make menuconfig would not work. Anyways, I will put it in the appropriate place.

Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires

We already run KHO init before smp_init:
start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!

While deferred struct pages are initialized:
start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()

Quoted 5 lines

If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()

Quoted 3 lines

deferred_grow_zone() takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early, deferred_init_memmap() will overwrite them.

Quoted 5 lines

Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.

Quoted 8 lines

Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.

Currently the only user of kho_restore_folio() during init is kho_memory_init(). The FDT is accessed by doing phys_to_virt(kho_in.fdt_phys) anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.

Quoted 6 lines

Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.

Quoted 3 lines

KHO already accesses stuct page early and uses page->private for order. Since preserved memory is reserved in memblock, deferred init of struct pages won't touch those pages, we just need to make sure they are properly

Not strictly true. Some of them might have been initialized from free_area_init() -> memmap_init() (the ones not eligible for deferred init), which happens before KHO makes its memblock reservations.

Quoted 3 lines

initialized at some point. If we don't expect many kho_restore_folio() before page_alloc_init_late() we can use init_deferred_page() for early accesses.

I tried doing this when looking into this initially, but it doesn't work for some reason.

Quoted 6 lines
    static void kho_restore_page(struct page *page, unsigned int order)
    {
    	unsigned int i, nr_pages = (1 << order);
    
    	/* Head page gets refcount of 1. */
    	init_deferred_page(page_to_pfn(page), NUMA_NO_NODE);

This would do

	if (early_page_initialised(pfn, nid))
		return;

	__init_page_from_nid(pfn, nid);

and I'm really surprised it didn't crash in early_page_initialised() because of NUMA_NO_NODE :)

What might work here is

	pfn = page_to_pfn(page);
	__init_page_from_nid(pfn, early_pfn_to_nid(pfn));
Quoted 31 lines
    	set_page_count(page, 1);
    
    	/* For higher order folios, tail pages get a page count of zero. */
    	for (i = 1; i < nr_pages; i++) {
    		init_deferred_page(page_to_pfn(page + i), NUMA_NO_NODE);
    		set_page_count(page + i, 0);
    	}
    
    [...]

results in:

    [    0.644032] page:(____ptrval____) is uninitialized and poisoned
    [    0.644679] page dumped because: VM_BUG_ON_PAGE(PagePoisoned(page))
    [    0.645376] ------------[ cut here ]------------
    [    0.645883] kernel BUG at ./include/linux/mm.h:1512!
    [...]
    [    0.647924] RIP: 0010:__pageblock_pfn_to_page+0x166/0x180
    [...]
    [    0.647924]  <TASK>
    [    0.647924]  set_zone_contiguous+0x6b/0x90
    [    0.647924]  page_alloc_init_late+0x356/0x370
    [    0.647924]  kernel_init_freeable+0x12d/0x190
    [    0.647924]  ? __pfx_kernel_init+0x10/0x10
    [    0.647924]  kernel_init+0x1a/0x130

didn't dig any deeper on why it happens...

Signature · 2 lines
Regards,
Pratyush Yadav
Signature · 2 lines
Sincerely yours,
Mike.
author maintainer
#15 lore raw
Quoted 98 lines · Mike Rapoport
Quoted 87 lines · Pratyush Yadav
Quoted 70 lines · Mike Rapoport
Quoted 62 lines · Pasha Tatashin
Quoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.

Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)

Hmm, weird, worked for me :/

Worked for me as well.

Quoted 32 lines

I am super confused, it did not work for me over weekend, and now it is working. Even make menuconfig would not work. Anyways, I will put it in the appropriate place.

Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires

We already run KHO init before smp_init:
start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!

While deferred struct pages are initialized:
start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()

Quoted 5 lines

If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()

Quoted 3 lines

deferred_grow_zone() takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early, deferred_init_memmap() will overwrite them.

Quoted 5 lines

Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.

Quoted 8 lines

Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.

Currently the only user of kho_restore_folio() during init is kho_memory_init(). The FDT is accessed by doing phys_to_virt(kho_in.fdt_phys) anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.

Quoted 6 lines

Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.

Quoted 3 lines

KHO already accesses stuct page early and uses page->private for order. Since preserved memory is reserved in memblock, deferred init of struct pages won't touch those pages, we just need to make sure they are properly

Not strictly true. Some of them might have been initialized from free_area_init() -> memmap_init() (the ones not eligible for deferred init), which happens before KHO makes its memblock reservations.

Quoted 3 lines

initialized at some point. If we don't expect many kho_restore_folio() before page_alloc_init_late() we can use init_deferred_page() for early accesses.

I tried doing this when looking into this initially, but it doesn't work for some reason.

Quoted 6 lines
    static void kho_restore_page(struct page *page, unsigned int order)
    {
    	unsigned int i, nr_pages = (1 << order);
    
    	/* Head page gets refcount of 1. */
    	init_deferred_page(page_to_pfn(page), NUMA_NO_NODE);

This would do

Quoted 7 lines
	if (early_page_initialised(pfn, nid))
		return;

	__init_page_from_nid(pfn, nid);

and I'm really surprised it didn't crash in early_page_initialised() because of NUMA_NO_NODE :)

Oh, right. Using the wrong node completely throws
early_page_initialised() off.

Quoted 5 lines

What might work here is

	pfn = page_to_pfn(page);
	__init_page_from_nid(pfn, early_pfn_to_nid(pfn));

Yep, that works. Although this would do early_pfn_to_nid() for each page so it isn't very efficient. And we also need to make sure memblock does not go away.

Quoted 11 lines
Quoted 11 lines
    	set_page_count(page, 1);
    
    	/* For higher order folios, tail pages get a page count of zero. */
    	for (i = 1; i < nr_pages; i++) {
    		init_deferred_page(page_to_pfn(page + i), NUMA_NO_NODE);
    		set_page_count(page + i, 0);
    	}
    
    [...]

[...]

Signature · 2 lines
Regards,
Pratyush Yadav
maintainer
#16 lore raw
Quoted 112 lines · Pratyush Yadav
Quoted 98 lines · Mike Rapoport
Quoted 87 lines · Pratyush Yadav
Quoted 70 lines · Mike Rapoport
Quoted 62 lines · Pasha Tatashin
Quoted 52 lines · Pratyush Yadav
Quoted 8 lines · Pasha Tatashin
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

I think it should be the other way around, KHO should depend on !DEFERRED_STRUCT_PAGE_INIT.

Agreed, and this is what I first tried, but that does not work, there is some circular dependency breaking the build. If you feel adventurous you can try that :-)

Hmm, weird, worked for me :/

Worked for me as well.

Quoted 32 lines

I am super confused, it did not work for me over weekend, and now it is working. Even make menuconfig would not work. Anyways, I will put it in the appropriate place.

Quoted 23 lines
Quoted 17 lines
Quoted 7 lines
Quoted 4 lines

We will need to teah KHO to work with deferred struct page init. I suspect, we could init preserved struct pages and then skip over them during deferred init.

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires

We already run KHO init before smp_init:
start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!

While deferred struct pages are initialized:
start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()

Quoted 5 lines

If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()

Quoted 3 lines

deferred_grow_zone() takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early, deferred_init_memmap() will overwrite them.

Quoted 5 lines

Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.

Quoted 8 lines

Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.

Currently the only user of kho_restore_folio() during init is kho_memory_init(). The FDT is accessed by doing phys_to_virt(kho_in.fdt_phys) anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.

Quoted 6 lines

Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.

Quoted 3 lines

KHO already accesses stuct page early and uses page->private for order. Since preserved memory is reserved in memblock, deferred init of struct pages won't touch those pages, we just need to make sure they are properly

Not strictly true. Some of them might have been initialized from free_area_init() -> memmap_init() (the ones not eligible for deferred init), which happens before KHO makes its memblock reservations.

Quoted 3 lines

initialized at some point. If we don't expect many kho_restore_folio() before page_alloc_init_late() we can use init_deferred_page() for early accesses.

I tried doing this when looking into this initially, but it doesn't work for some reason.

Quoted 6 lines
    static void kho_restore_page(struct page *page, unsigned int order)
    {
    	unsigned int i, nr_pages = (1 << order);
    
    	/* Head page gets refcount of 1. */
    	init_deferred_page(page_to_pfn(page), NUMA_NO_NODE);

This would do

Quoted 7 lines
	if (early_page_initialised(pfn, nid))
		return;

	__init_page_from_nid(pfn, nid);

and I'm really surprised it didn't crash in early_page_initialised() because of NUMA_NO_NODE :)

Oh, right. Using the wrong node completely throws
early_page_initialised() off.

Quoted 1 line
Quoted 8 lines
Quoted 4 lines

What might work here is

	pfn = page_to_pfn(page);
	__init_page_from_nid(pfn, early_pfn_to_nid(pfn));

Yep, that works. Although this would do early_pfn_to_nid() for each page so it isn't very efficient. And we also need to make sure memblock does not go away.

Yeah, that's far from ideal. Let's just gate KHO on !CONFIG_DEFERRED_PAGE_STRUCT_INIT for now.

Quoted 15 lines
Quoted 10 lines
Quoted 10 lines
    	set_page_count(page, 1);
    
    	/* For higher order folios, tail pages get a page count of zero. */
    	for (i = 1; i < nr_pages; i++) {
    		init_deferred_page(page_to_pfn(page + i), NUMA_NO_NODE);
    		set_page_count(page + i, 0);
    	}
    
    [...]

[...]

Signature · 2 lines
Regards,
Pratyush Yadav
Signature · 2 lines
Sincerely yours,
Mike.
author maintainer
#17 lore raw
In reply to Pasha Tatashin
Quoted 3 lines · Pasha Tatashin
Quoted 2 lines · Pratyush Yadav

On Tue, Jun 10 2025, Pasha Tatashin wrote:

[...]

Quoted 37 lines
Quoted 28 lines
Quoted 22 lines
Quoted 18 lines
Quoted 12 lines
Quoted 2 lines

We could, but with that would mean we'll run this before SMP and it's not desirable. Also, init_deferred_page() for a random page requires

We already run KHO init before smp_init:
start_kernel() -> mm_core_init() -> kho_memory_init() -> kho_restore_folio() -> struct pages must be already initialized here!

While deferred struct pages are initialized:
start_kernel() -> rest_init() -> kernel_init() -> kernel_init_freeable() -> page_alloc_init_late() -> deferred_init_memmap()

Quoted 5 lines

If the number of preserved pages that is needed during early boot is relatively small, that it should not be an issue to pre-initialize struct pages for them before deferred struct pages are initialized. We already pre-initialize some "struct pages" that are needed during early boot before the reset are initialized, see deferred_grow_zone()

Quoted 3 lines

deferred_grow_zone() takes a chunk in the beginning of uninitialized range, with kho we are talking about some random pages. If we preinit them early, deferred_init_memmap() will overwrite them.

Quoted 5 lines

Yes, this is why I am saying that we would need to skip the KHO initialized "struct pages" somehow during deferred initialization. If we create an ordered by PFN list of early-initialized KHO struct pages, skipping during deferred initialization could be done efficiently.

Quoted 8 lines

Or keep things simple and don't use any KHO struct pages during early init. You can access the page itself, just don't use its struct page.

Currently the only user of kho_restore_folio() during init is kho_memory_init(). The FDT is accessed by doing phys_to_virt(kho_in.fdt_phys) anyway, so there is really no need for restoring the folio so early. It can be done later, for example when LUO does the finish event, to clean up and free the folio.

Quoted 6 lines

Good suggestion, however, KHO does not have any sophisticated users that we are going to be adding as part of the live update work in the
future: IR, KVM, early VCPU threads, and so on. So, while today, this might work, in the future, I am not sure if we should expect struct pages are not accessed until after deferred initialization or simply fix it once and for all.

Right. We might end up needing it down the line. But from a quick look, it doesn't seem to be trivial to solve, so IMO we should solve it when those use cases actually show up, and keep things simple for now.

Signature · 2 lines
Regards,
Pratyush Yadav