kho: fixes for vmalloc restoration
Subsystems (3)
Mike Rapoport
maintainercover kho: fixes for vmalloc restoration
From: "Mike Rapoport (Microsoft)" <[email protected]>
Hi,
Pratyush reported off-list that when kho_restore_vmalloc() is used to restore a vmalloc_huge() allocation it hits VM_BUG_ON() when we reconstruct the struct pages in kho_restore_pages().
These patches fix the issue.
Mike Rapoport (Microsoft) (2):
kho: kho_restore_vmalloc: fix initialization of pages array
kho: fix restoring of contiguous ranges of order-0 pages
kernel/liveupdate/kexec_handover.c | 22 +++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-)
Signature · 1 line
2.50.1
Mike Rapoport
maintainerPATCH 1/2 kho: kho_restore_vmalloc: fix initialization of pages array
From: "Mike Rapoport (Microsoft)" <[email protected]>
In case a preserved vmalloc allocation was using huge pages, all pages in the array of pages added to vm_struct during kho_restore_vmalloc() are wrongly set to the same page.
Fix the indexing when assigning pages to that array.
--- kernel/liveupdate/kexec_handover.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/liveupdate/kexec_handover.c +1 -1
@@ -1096,7 +1096,7 @@ void *kho_restore_vmalloc(const struct kho_vmalloc *preservation) goto err_free_pages_array;for (int j =0 ; j < contig_pages; j++) - pages[idx++] = page; + pages[idx++] = page + j; phys += contig_pages *PAGE_SIZE ; }
Signature · 1 line
2.50.1
Re: [PATCH 1/2] kho: kho_restore_vmalloc: fix initialization of pages array
Quoted 10 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
In case a preserved vmalloc allocation was using huge pages, all pages in the array of pages added to
vm_structduringkho_restore_vmalloc()are wrongly set to the same page.Fix the indexing when assigning pages to that array.
Fixes a667300bd53f ("kho: add support for preserving vmalloc allocations")Signed-off-by Mike Rapoport (Microsoft) <[email protected]>
[...]
Signature · 2 lines
Regards, Pratyush Yadav
Mike Rapoport
maintainerPATCH 2/2 kho: fix restoring of contiguous ranges of order-0 pages
In reply to Mike RapoportFrom: "Mike Rapoport (Microsoft)" <[email protected]>
When contiguous ranges of order-0 pages are restored, kho_restore_page() calls prep_compound_page() with the first page in the range and order as parameters and then kho_restore_pages() calls split_page() to make sure all pages in the range are order-0.
However, since split_page() is not intended to split compound pages and with VM_DEBUG enabled it will trigger a VM_BUG_ON_PAGE().
Update kho_restore_page() so that it will use prep_compound_page() when it restores a folio and make sure it properly sets page count for both large folios and ranges of order-0 pages.
--- kernel/liveupdate/kexec_handover.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
kernel/liveupdate/kexec_handover.c +12 -8
@@ -219,11 +219,11 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,return 0 ; } -static struct page *kho_restore_page (phys_addr_t phys) +static struct page *kho_restore_page (phys_addr_t phys,bool is_folio) {struct page *page =pfn_to_online_page (PHYS_PFN (phys)); +unsigned int nr_pages, ref_cnt;union kho_page_info info; -unsigned int nr_pages;if (!page)return NULL ;
@@ -243,11 +243,16 @@ static struct page *kho_restore_page(phys_addr_t phys)/* Head page gets refcount of 1. */ set_page_count (page,1 ); -/* For higher order folios, tail pages get a page count of zero. */ +/* +* For higher order folios, tail pages get a page count of zero. +* For physically contiguous order-0 pages every pages gets a page +* count of 1 +*/ + ref_cnt = is_folio ?0 :1 ;for (unsigned int i =1 ; i < nr_pages; i++) -set_page_count (page + i,0 ); +set_page_count (page + i, ref_cnt); -if (info.order >0 ) +if (is_folio && info.order)prep_compound_page (page, info.order);adjust_managed_page_count (page, nr_pages);
@@ -262,7 +267,7 @@ static struct page *kho_restore_page(phys_addr_t phys)*/ struct folio *kho_restore_folio (phys_addr_t phys) { -struct page *page =kho_restore_page (phys); +struct page *page =kho_restore_page (phys, true);return page ?page_folio (page) :NULL ; }
@@ -287,11 +292,10 @@ struct page *kho_restore_pages(phys_addr_t phys, unsigned int nr_pages)while (pfn < end_pfn) {const unsigned int order =min (count_trailing_zeros (pfn),ilog2 (end_pfn - pfn)); -struct page *page =kho_restore_page (PFN_PHYS (pfn)); +struct page *page =kho_restore_page (PFN_PHYS (pfn), false);if (!page)return NULL ; -split_page (page, order); pfn +=1 << order; }
Signature · 1 line
2.50.1
Pratyush Yadav
maintainerRe: [PATCH 2/2] kho: fix restoring of contiguous ranges of order-0 pages
Quoted 46 lines · Mike Rapoport
From: "Mike Rapoport (Microsoft)" <[email protected]>
When contiguous ranges of order-0 pages are restored,
kho_restore_page()callsprep_compound_page()with the first page in the range and order as parameters and thenkho_restore_pages()callssplit_page()to make sure all pages in the range are order-0.However, since
split_page()is not intended to split compound pages and withVM_DEBUGenabled it will trigger aVM_BUG_ON_PAGE().Update
kho_restore_page()so that it will useprep_compound_page()when it restores a folio and make sure it properly sets page count for both large folios and ranges of order-0 pages.Reported-by Pratyush Yadav <[email protected]>Fixes a667300bd53f ("kho: add support for preserving vmalloc allocations")Signed-off-by Mike Rapoport (Microsoft) <[email protected]>--- kernel/liveupdate/kexec_handover.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)kernel/liveupdate/kexec_handover.c +4
-3@@ -219,11 +219,11 @@ static int __kho_preserve_order(struct kho_mem_track *track, unsigned long pfn,return 0 ; } -static struct page *kho_restore_page (phys_addr_t phys) +static struct page *kho_restore_page (phys_addr_t phys,bool is_folio) {struct page *page =pfn_to_online_page (PHYS_PFN (phys)); +unsigned int nr_pages, ref_cnt;union kho_page_info info; -unsigned int nr_pages;if (!page)return NULL ;@@ -243,11 +243,16 @@ static struct page *kho_restore_page(phys_addr_t phys)/* Head page gets refcount of 1. */ set_page_count (page,1 ); -/* For higher order folios, tail pages get a page count of zero. */ +/* +* For higher order folios, tail pages get a page count of zero.
kernel/liveupdate/kexec_handover.c static struct page *kho_restore_page(phys_addr_t phys) · line 255
Quoted 10 lines
kernel/liveupdate/kexec_handover.c +6
-2+* For physically contiguous order-0 pages every pages gets a page +* count of 1 +*/ + ref_cnt = is_folio ?0 :1 ;for (unsigned int i =1 ; i < nr_pages; i++) -set_page_count (page + i,0 ); +set_page_count (page + i, ref_cnt); -if (info.order >0 ) +if (is_folio && info.order)
This is getting a bit difficult to parse. Let's separate out folio and page initialization to separate helpers:
/* Initalize 0-order KHO pages */
static void kho_init_page(struct page *page, unsigned int nr_pages)
{
for (unsigned int i = 0; i < nr_pages; i++)
set_page_count(page + i, 1);
}
static void kho_init_folio(struct page *page, unsigned int order)
{
unsigned int nr_pages = (1 << order);
/* Head page gets refcount of 1. */
set_page_count(page, 1);
/* For higher order folios, tail pages get a page count of zero. */
for (unsigned int i = 1; i < nr_pages; i++)
set_page_count(page + i, 0);
if (order > 0)
prep_compound_page(page, order);
}Quoted 24 lines
kernel/liveupdate/kexec_handover.c +2
-3prep_compound_page (page, info.order);adjust_managed_page_count (page, nr_pages);@@ -262,7 +267,7 @@ static struct page *kho_restore_page(phys_addr_t phys)*/ struct folio *kho_restore_folio (phys_addr_t phys) { -struct page *page =kho_restore_page (phys); +struct page *page =kho_restore_page (phys, true);return page ?page_folio (page) :NULL ; }@@ -287,11 +292,10 @@ struct page *kho_restore_pages(phys_addr_t phys, unsigned int nr_pages)while (pfn < end_pfn) {const unsigned int order =min (count_trailing_zeros (pfn),ilog2 (end_pfn - pfn)); -struct page *page =kho_restore_page (PFN_PHYS (pfn)); +struct page *page =kho_restore_page (PFN_PHYS (pfn), false);if (!page)return NULL ; -split_page (page, order); pfn +=1 << order; }
Signature · 2 lines
Regards, Pratyush Yadav
Mike Rapoport
maintainerRe: [PATCH 2/2] kho: fix restoring of contiguous ranges of order-0 pages
Hi Pratyush,
Quoted 1 line · Pratyush Yadav
On Tue, Nov 25 2025, Mike Rapoport wrote:
...
Quoted 15 lines
Quoted 15 lines
kernel/liveupdate/kexec_handover.c +7
-2@@ -243,11 +243,16 @@ static struct page *kho_restore_page(phys_addr_t phys)/* Head page gets refcount of 1. */ set_page_count (page,1 ); -/* For higher order folios, tail pages get a page count of zero. */ +/* +* For higher order folios, tail pages get a page count of zero. +* For physically contiguous order-0 pages every pages gets a page +* count of 1 +*/ + ref_cnt = is_folio ?0 :1 ;for (unsigned int i =1 ; i < nr_pages; i++) -set_page_count (page + i,0 ); +set_page_count (page + i, ref_cnt);
Quoted 5 lines
kernel/liveupdate/kexec_handover.c static struct page *kho_restore_page(phys_addr_t phys) · line 255
Quoted 2 lines
kernel/liveupdate/kexec_handover.c +1
-1-if (info.order >0 ) +if (is_folio && info.order)This is getting a bit difficult to parse. Let's separate out folio and page initialization to separate helpers:
Sorry, I've missed this earlier and now the patches are in akpm's -stable branch.
Let's postpone these changes for the next cycle, maybe along with support for deferred initialization of struct page.
Quoted 21 lines
/* Initalize 0-order KHO pages */
static void kho_init_page(struct page *page, unsigned int nr_pages)
{
for (unsigned int i = 0; i < nr_pages; i++)
set_page_count(page + i, 1);
}
static void kho_init_folio(struct page *page, unsigned int order)
{
unsigned int nr_pages = (1 << order);
/* Head page gets refcount of 1. */
set_page_count(page, 1);
/* For higher order folios, tail pages get a page count of zero. */
for (unsigned int i = 1; i < nr_pages; i++)
set_page_count(page + i, 0);
if (order > 0)
prep_compound_page(page, order);
}Signature · 2 lines
Sincerely yours, Mike.
Pratyush Yadav
maintainerRe: [PATCH 2/2] kho: fix restoring of contiguous ranges of order-0 pages
Quoted 28 lines · Mike Rapoport
Hi Pratyush,
Quoted 1 line · Pratyush Yadav
On Tue, Nov 25 2025, Mike Rapoport wrote:
...
Quoted 15 lines
Quoted 15 lines
kernel/liveupdate/kexec_handover.c +7
-2@@ -243,11 +243,16 @@ static struct page *kho_restore_page(phys_addr_t phys)/* Head page gets refcount of 1. */ set_page_count (page,1 ); -/* For higher order folios, tail pages get a page count of zero. */ +/* +* For higher order folios, tail pages get a page count of zero. +* For physically contiguous order-0 pages every pages gets a page +* count of 1 +*/ + ref_cnt = is_folio ?0 :1 ;for (unsigned int i =1 ; i < nr_pages; i++) -set_page_count (page + i,0 ); +set_page_count (page + i, ref_cnt);Quoted 5 lines
kernel/liveupdate/kexec_handover.c static struct page *kho_restore_page(phys_addr_t phys) · line 255
Quoted 2 lines
kernel/liveupdate/kexec_handover.c +1
-1-if (info.order >0 ) +if (is_folio && info.order)This is getting a bit difficult to parse. Let's separate out folio and page initialization to separate helpers:
Quoted 4 lines
Sorry, I've missed this earlier and now the patches are in akpm's -stable branch.
Let's postpone these changes for the next cycle, maybe along with support for deferred initialization of struct page.
Sure, no problem.
[...]
Signature · 2 lines
Regards, Pratyush Yadav