kho: support preserving unsplit high-order pages

RFC PATCH Superseded Pranjal Shrivastava 3 participants 8 messages Jul 13 – Jul 22

Subsystems (3)

Memory management Live update kexec handover (KHO)

lore

Share & tools

lore

Pranjal Shrivastava

author

2026-07-13 20:49 UTC #0 lore raw

v2 cover kho: support preserving unsplit high-order pages

Introduction
============
This series is required for the ongoing effort to preserve DMA allocations across KHO [1]. It addresses a fundamental mismatch between the current KHO restoration logic and the physical reality of high-order buddy allocations.

The Problem
===========
The current KHO restore implementation treats all multi-page blocks as split pages during restoration. Specifically, kho_restore_pages() initializes every 4KB sub-page with a refcount of 1.

However, many kernel subsystems, most notably the DMA allocator (via dma_alloc_coherent), frequently return high-order non-compound pages. In this unsplit state, only the head page carries a refcount of 1, while all tail pages have a refcount of 0.

Consequently, when these contiguous but unsplit blocks are restored by KHO in the new kernel, the forced reference count of 1 on tail pages causes some trouble with the buddy allocator. Downstream of the eventual free path, __free_pages_prepare() [2] ends up calling page_expected_state() [3] when is_check_pages_enabled() returns true (triggered when CONFIG_DEBUG_VM is enabled or debug_pagealloc=on).

This detects the unexpected non-zero reference counts on tail pages [4] and incorrectly taints the kernel while leaking the physical pages in question.

Proposed Solution
=================
Following feedback on the v1 RFC, this series moves away from auto type detection and instead introduces explicit preserve / restore APIs for high-order unsplit pages.

Callers are now explicitly specify if a block is unsplit & preserve it by using kho_preserve_page() and kho_restore_page(). These functions apply a refcount of 1 to the head page while leaving tail pages at 0.

The existing APIs (kho_preserve_pages / kho_restore_pages) remain as is for ranges of independent 4KB pages, continuing to use the split refcount.

The internal initialization logic is refactored to provide a helper:
kho_init_unsplit_pages(), which is shared between folios and unsplit page restore APIs.

[v2]
- Dropped automatic type detection via higher bits in Radix key.
- Introduced explicit kho_preserve_page and kho_restore_page helpers.
- Refactored internal init logic to share code between folios & unsplit pages.

[v1] https://lore.kernel.org/all/[email protected]/

Thanks,
Praan

[1] https://lore.kernel.org/all/[email protected]/
[2] https://elixir.bootlin.com/linux/v7.1.1/source/mm/page_alloc.c#L1370
[3] https://elixir.bootlin.com/linux/v7.1.1/source/mm/page_alloc.c#L1027
[4] https://elixir.bootlin.com/linux/v7.1.1/source/mm/page_alloc.c#L1034

Pranjal Shrivastava (2):
kho: Introduce a helper to init unsplit pages
kho: Introduce preserve/restore APIs for unsplit pages

 include/linux/kexec_handover.h     |  10 +++
 kernel/liveupdate/kexec_handover.c | 115 +++++++++++++++++++++++++----
 2 files changed, 111 insertions(+), 14 deletions(-)

base-commit: 0f26556c5eeea62cc934fa8938b148aa5844a6b6

Signature · 1 line

2.55.0.141.g00534a21ce-goog

Pranjal Shrivastava

author

2026-07-13 20:49 UTC #1 lore raw

RFC PATCH v2 1/2 kho: Introduce a helper to init unsplit pages

The current KHO restoration logic assumes all multi-page blocks are split into independent 4KB pages. Break out a helper to prepare for supporting high-order non-compound pages.

Extract kho_init_unsplit_pages() to handle the refcount pattern where only the head page is refcounted. Use the helper for folio restoration that requires a similar refcount logic.

Signed-off-by Pranjal Shrivastava <[email protected]>

---
 kernel/liveupdate/kexec_handover.c | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

kernel/liveupdate/kexec_handover.c +19 ~~-10~~

index 4834a809985a..07f6f453cd69 100644
@@ -357,6 +357,24 @@ int kho_radix_walk_tree(struct kho_radix_tree *tree,
 }
 EXPORT_SYMBOL_GPL(kho_radix_walk_tree);
 
+/* For physically contiguous pages. */
+static void kho_init_unsplit_pages(struct page *page, unsigned int order)
+{
+	unsigned long nr_pages = (1UL << order);
+
+	/* Head page gets refcount of 1. */
+	set_page_count(page, 1);
+	/* Clear head page's codetag to avoid accounting mismatch. */
+	clear_page_tag_ref(page);
+
+	/* For high-order blocks, tail pages get a page count of zero. */
+	for (unsigned long i = 1; i < nr_pages; i++) {
+		set_page_count(page + i, 0);
+		/* Clear each page's codetag to avoid accounting mismatch. */
+		clear_page_tag_ref(page + i);
+	}
+}
+
 /* For physically contiguous 0-order pages. */
 static void kho_init_pages(struct page *page, unsigned long nr_pages)
 {

@@ -369,16 +387,7 @@ static void kho_init_pages(struct page *page, unsigned long nr_pages)
 
 static void kho_init_folio(struct page *page, unsigned int order)
 {
-	unsigned long nr_pages = (1 << order);
-
-	/* Head page gets refcount of 1. */
-	set_page_count(page, 1);
-	/* Clear head page's codetag to avoid accounting mismatch. */
-	clear_page_tag_ref(page);
-
-	/* For higher order folios, tail pages get a page count of zero. */
-	for (unsigned long i = 1; i < nr_pages; i++)
-		set_page_count(page + i, 0);
+	kho_init_unsplit_pages(page, order);
 
 	if (order > 0)
 		prep_compound_page(page, order);

Signature · 1 line

2.55.0.141.g00534a21ce-goog

Samiullah Khawaja

Reviewed-by on v2 1/2

2026-07-13 21:26 UTC #2 lore raw

Re: [RFC PATCH v2 1/2] kho: Introduce a helper to init unsplit pages

Quoted 4 lines · Pranjal Shrivastava

The current KHO restoration logic assumes all multi-page blocks are split into independent 4KB pages. Break out a helper to prepare for supporting high-order non-compound pages.

Quoted 9 lines

Extract kho_init_unsplit_pages() to handle the refcount pattern where only the head page is refcounted. Use the helper for folio restoration that requires a similar refcount logic.
Signed-off-by Pranjal Shrivastava <[email protected]>
---
kernel/liveupdate/kexec_handover.c | 29 +++++++++++++++++++----------
1 file changed, 19 insertions(+), 10 deletions(-)

Reviewed-by Samiullah Khawaja <[email protected]>

Mike Rapoport

maintainer

2026-07-20 16:57 UTC #3 lore raw

Re: [RFC PATCH v2 1/2] kho: Introduce a helper to init unsplit pages

In reply to Pranjal Shrivastava

Quoted 15 lines

The current KHO restoration logic assumes all multi-page blocks are split into independent 4KB pages. Break out a helper to prepare for supporting high-order non-compound pages.
Extract kho_init_unsplit_pages() to handle the refcount pattern where only the head page is refcounted. Use the helper for folio restoration that requires a similar refcount logic.
Signed-off-by Pranjal Shrivastava <[email protected]>
kernel/liveupdate/kexec_handover.c +0 -0
index 4834a809985ab..07f6f453cd69d 100644

kernel/liveupdate/kexec_handover.c int kho_radix_walk_tree(struct kho_radix_tree *tree, · line 362

Quoted 6 lines

kernel/liveupdate/kexec_handover.c +3 -0

@@ -357,6 +357,24 @@ int kho_radix_walk_tree(struct kho_radix_tree *tree,
 }
 EXPORT_SYMBOL_GPL(kho_radix_walk_tree);
 
+/* For physically contiguous pages. */
+static void kho_init_unsplit_pages(struct page *page, unsigned int order)
+{

Can't say I like "unsplit". How about kho_init_high_order_page?

And whatever name we pick here, it's surely must end with "page" rather than "pages".

Signature · 2 lines

Sincerely yours,
Mike.

Pranjal Shrivastava

author

2026-07-22 21:18 UTC #4 lore raw

Re: [RFC PATCH v2 1/2] kho: Introduce a helper to init unsplit pages

Quoted 18 lines · Mike Rapoport

Quoted 18 lines
The current KHO restoration logic assumes all multi-page blocks are split into independent 4KB pages. Break out a helper to prepare for supporting high-order non-compound pages.
Extract kho_init_unsplit_pages() to handle the refcount pattern where only the head page is refcounted. Use the helper for folio restoration that requires a similar refcount logic.
Signed-off-by Pranjal Shrivastava <[email protected]>
kernel/liveupdate/kexec_handover.c +0 -0
index 4834a809985ab..07f6f453cd69d 100644
@@ -357,6 +357,24 @@ int kho_radix_walk_tree(struct kho_radix_tree *tree,
 }
 EXPORT_SYMBOL_GPL(kho_radix_walk_tree);
 

Quoted 9 lines

kernel/liveupdate/kexec_handover.c int kho_radix_walk_tree(struct kho_radix_tree *tree, · line 362
Quoted 3 lines
kernel/liveupdate/kexec_handover.c +3 -0
+/* For physically contiguous pages. */
+static void kho_init_unsplit_pages(struct page *page, unsigned int order)
+{
Can't say I like "unsplit". How about kho_init_high_order_page?
And whatever name we pick here, it's surely must end with "page" rather than "pages".

Ack. I'll rename this to kho_init_high_order_page().

Thanks,
Praan

Pranjal Shrivastava

author

2026-07-13 20:49 UTC #5 lore raw

RFC PATCH v2 2/2 kho: Introduce preserve/restore APIs for unsplit pages

In reply to Pranjal Shrivastava

The current KHO page preservation APIs (e.g. kho_preserve_pages) assume that multi-page blocks are split into independent 4KB pages during restoration. This is incompatible with high-order non-compound pages, such as DMA buffers, which must be restored with tail pages having a zero reference count.

Introduce explicit preserve and restore APIs for unsplit pages, which preserve and restore a high-order page block as a single unit, applying a refcount of 1 to the head page while leaving tail pages at 0. Rename the existing internal helper to __kho_restore_page().

Signed-off-by Pranjal Shrivastava <[email protected]>

---
 include/linux/kexec_handover.h     | 10 ++++
 kernel/liveupdate/kexec_handover.c | 86 ++++++++++++++++++++++++++++--
 2 files changed, 92 insertions(+), 4 deletions(-)

include/linux/kexec_handover.h +10 -0

index 8968c56d2d73..6e2c75b16209 100644
@@ -22,6 +22,8 @@ bool is_kho_boot(void);
 
 int kho_preserve_folio(struct folio *folio);
 void kho_unpreserve_folio(struct folio *folio);
+int kho_preserve_page(struct page *page, unsigned int order);
+void kho_unpreserve_page(struct page *page, unsigned int order);
 int kho_preserve_pages(struct page *page, unsigned long nr_pages);
 void kho_unpreserve_pages(struct page *page, unsigned long nr_pages);
 int kho_preserve_vmalloc(void *ptr, struct kho_vmalloc *preservation);

@@ -30,6 +32,7 @@ void *kho_alloc_preserve(size_t size);
 void kho_unpreserve_free(void *mem);
 void kho_restore_free(void *mem);
 struct folio *kho_restore_folio(phys_addr_t phys);
+struct page *kho_restore_page(phys_addr_t phys);
 struct page *kho_restore_pages(phys_addr_t phys, unsigned long nr_pages);
 void *kho_restore_vmalloc(const struct kho_vmalloc *preservation);
 int kho_add_subtree(const char *name, void *blob, size_t size);

@@ -65,6 +68,13 @@ static inline int kho_preserve_pages(struct page *page, unsigned int nr_pages)
 
 static inline void kho_unpreserve_pages(struct page *page, unsigned int nr_pages) { }
 
+static inline int kho_preserve_page(struct page *page, unsigned int order)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline void kho_unpreserve_page(struct page *page, unsigned int order) { }
+
 static inline int kho_preserve_vmalloc(void *ptr,
 				       struct kho_vmalloc *preservation)
 {

kernel/liveupdate/kexec_handover.c +82 -4

121 lines not on this page · raw message

Signature · 1 line

2.55.0.141.g00534a21ce-goog

Mike Rapoport

maintainer

2026-07-20 16:57 UTC #6 lore raw

Re: [RFC PATCH v2 2/2] kho: Introduce preserve/restore APIs for unsplit pages

Quoted 7 lines

The current KHO page preservation APIs (e.g. kho_preserve_pages) assume that multi-page blocks are split into independent 4KB pages during restoration. This is incompatible with high-order non-compound pages, such as DMA buffers, which must be restored with tail pages having a zero reference count.
Introduce explicit preserve and restore APIs for unsplit pages,

high-order non-compound ^

Quoted 84 lines

which preserve and restore a high-order page block as a single unit, applying a refcount of 1 to the head page while leaving tail pages at 0. Rename the existing internal helper to __kho_restore_page().

Signed-off-by Pranjal Shrivastava <[email protected]>

include/linux/kexec_handover.h +10 -0

index 8968c56d2d73e..6e2c75b16209c 100644
@@ -22,6 +22,8 @@ bool is_kho_boot(void);
 
 int kho_preserve_folio(struct folio *folio);
 void kho_unpreserve_folio(struct folio *folio);
+int kho_preserve_page(struct page *page, unsigned int order);
+void kho_unpreserve_page(struct page *page, unsigned int order);
 int kho_preserve_pages(struct page *page, unsigned long nr_pages);
 void kho_unpreserve_pages(struct page *page, unsigned long nr_pages);
 int kho_preserve_vmalloc(void *ptr, struct kho_vmalloc *preservation);

@@ -30,6 +32,7 @@ void *kho_alloc_preserve(size_t size);
 void kho_unpreserve_free(void *mem);
 void kho_restore_free(void *mem);
 struct folio *kho_restore_folio(phys_addr_t phys);
+struct page *kho_restore_page(phys_addr_t phys);
 struct page *kho_restore_pages(phys_addr_t phys, unsigned long nr_pages);
 void *kho_restore_vmalloc(const struct kho_vmalloc *preservation);
 int kho_add_subtree(const char *name, void *blob, size_t size);

@@ -65,6 +68,13 @@ static inline int kho_preserve_pages(struct page *page, unsigned int nr_pages)
 
 static inline void kho_unpreserve_pages(struct page *page, unsigned int nr_pages) { }
 
+static inline int kho_preserve_page(struct page *page, unsigned int order)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline void kho_unpreserve_page(struct page *page, unsigned int order) { }
+
 static inline int kho_preserve_vmalloc(void *ptr,
 				       struct kho_vmalloc *preservation)
 {

kernel/liveupdate/kexec_handover.c +21 -2

index 07f6f453cd69d..43ae3e2de5da2 100644
@@ -393,7 +393,7 @@ static void kho_init_folio(struct page *page, unsigned int order)
 		prep_compound_page(page, order);
 }
 
-static struct page *kho_restore_page(phys_addr_t phys, bool is_folio)
+static struct page *__kho_restore_page(phys_addr_t phys, bool is_folio)
 {
 	struct page *page = pfn_to_online_page(PHYS_PFN(phys));
 	unsigned long nr_pages;

@@ -432,12 +432,44 @@ static struct page *kho_restore_page(phys_addr_t phys, bool is_folio)
  */
 struct folio *kho_restore_folio(phys_addr_t phys)
 {
-	struct page *page = kho_restore_page(phys, true);
+	struct page *page = __kho_restore_page(phys, true);
 
 	return page ? page_folio(page) : NULL;
 }
 EXPORT_SYMBOL_GPL(kho_restore_folio);
 
+/**
+ * kho_restore_page - restore a higher-order unsplit page block.
+ * @phys: physical address of the first page.
+ *
+ * Restore a higher-order unsplit page block that was preserved with
+ * kho_preserve_page().
+ *
+ * Return: the head page on success, NULL on failure.
+ */
+struct page *kho_restore_page(phys_addr_t phys)
+{
+	struct page *page = pfn_to_online_page(PHYS_PFN(phys));
+	unsigned long nr_pages;
+	union kho_page_info info;
+
+	if (!page)
+		return NULL;
+
+	info.page_private = page->private;

kernel/liveupdate/kexec_handover.c static struct page *kho_restore_page(phys_addr_t phys, bool is_folio) · line 469

Quoted 10 lines

kernel/liveupdate/kexec_handover.c +10 -0

+	if (WARN_ON_ONCE(info.magic != KHO_PAGE_MAGIC))
+		return NULL;
+
+	nr_pages = (1UL << info.order);
+	page->private = 0;
+
+	kho_init_unsplit_pages(page, info.order);
+	adjust_managed_page_count(page, nr_pages);
+
+	return page;

Hmm, it copies a lot from __kho_init_page().
We should keep the common part of order-0 page/folio/high-order page initialization in a single place.

Signature · 2 lines

Sincerely yours,
Mike.

Pranjal Shrivastava

author

2026-07-22 21:20 UTC #7 lore raw

Re: [RFC PATCH v2 2/2] kho: Introduce preserve/restore APIs for unsplit pages

Quoted 10 lines · Mike Rapoport

Quoted 7 lines
The current KHO page preservation APIs (e.g. kho_preserve_pages) assume that multi-page blocks are split into independent 4KB pages during restoration. This is incompatible with high-order non-compound pages, such as DMA buffers, which must be restored with tail pages having a zero reference count.
Introduce explicit preserve and restore APIs for unsplit pages,
high-order non-compound ^

Ack, will update the commit msg.

Quoted 12 lines

Quoted 12 lines
which preserve and restore a high-order page block as a single unit, applying a refcount of 1 to the head page while leaving tail pages at 0. Rename the existing internal helper to __kho_restore_page().
Signed-off-by Pranjal Shrivastava <[email protected]>
include/linux/kexec_handover.h +0 -0
index 8968c56d2d73e..6e2c75b16209c 100644
@@ -22,6 +22,8 @@ bool is_kho_boot(void);
 

[...]

Quoted 16 lines

include/linux/kexec_handover.h +16 -0

+struct page *kho_restore_page(phys_addr_t phys)
+{
+	struct page *page = pfn_to_online_page(PHYS_PFN(phys));
+	unsigned long nr_pages;
+	union kho_page_info info;
+
+	if (!page)
+		return NULL;
+
+	info.page_private = page->private;
+	if (WARN_ON_ONCE(info.magic != KHO_PAGE_MAGIC))
+		return NULL;
+
+	nr_pages = (1UL << info.order);
+	page->private = 0;
+

Quoted 9 lines

include/linux/kexec_handover.h
Quoted 4 lines
include/linux/kexec_handover.h +4 -0
+	kho_init_unsplit_pages(page, info.order);
+	adjust_managed_page_count(page, nr_pages);
+
+	return page;
Hmm, it copies a lot from __kho_init_page().
We should keep the common part of order-0 page/folio/high-order page initialization in a single place.

Alright, I'll break out a helper.

Thanks,
Praan