Skip to contentSkip to conversation

mm/hugetlb: init tails before init_migratetype

PATCH In review Michal Clapinski 3 participants 3 messages Jun 22

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm/hugetlb: init tails before init_migratetype

Currently, if you enable HVO, DEFERRED_STRUCT_PAGE_INIT and VM_DEBUG the kernel will crash with the following stack trace

get_pfnblock_bitmap_bitidx
__set_pfnblock_flags_mask
hugetlb_bootmem_init_migratetype
prep_and_add_bootmem_folios
gather_bootmem_prealloc_node
gather_bootmem_prealloc_parallel
padata_do_multithreaded
gather_bootmem_prealloc
hugetlb_init

on this code

VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page);

This code looks inside the struct page which will be uninitialized for hugetlb tail pages, which will cause a false positive.

So let's initialize the tail pages before this happens.

Fixes 622026e87c40 ("mm/hugetlb: remove fake head pages")

---
I think this one check is the only thing that fails, so I can probably just remove it instead.

---
 mm/hugetlb.c         |  1 +
 mm/hugetlb_vmemmap.c | 14 +++++++++-----
 mm/hugetlb_vmemmap.h |  5 +++++
 3 files changed, 15 insertions(+), 5 deletions(-)
mm/hugetlb.c +1 -0
index 571212b80835..de5e7c104c10 100644
@@ -4127,6 +4127,7 @@ static int __init hugetlb_init(void)
 	}
 
 	hugetlb_init_hstates();
+	hugetlb_vmemmap_init_tails();
 	gather_bootmem_prealloc();
 	report_hugepages();
 
mm/hugetlb_vmemmap.c +9 -5
index 133b46dfb09f..cf485c101926 100644
@@ -867,14 +867,10 @@ static const struct ctl_table hugetlb_vmemmap_sysctls[] = {
 	},
 };
 
-static int __init hugetlb_vmemmap_init(void)
+void __init hugetlb_vmemmap_init_tails(void)
 {
-	const struct hstate *h;
 	struct zone *zone;
 
-	/* HUGETLB_VMEMMAP_RESERVE_SIZE should cover all used struct pages */
-	BUILD_BUG_ON(__NR_USED_SUBPAGE > HUGETLB_VMEMMAP_RESERVE_PAGES);
-
 	for_each_zone(zone) {
 		for (int i = 0; i < NR_VMEMMAP_TAILS; i++) {
 			struct page *tail, *p;
@@ -890,6 +886,14 @@ static int __init hugetlb_vmemmap_init(void)
 				init_compound_tail(p + j, NULL, order, zone);
 		}
 	}
+}
+
+static int __init hugetlb_vmemmap_init(void)
+{
+	const struct hstate *h;
+
+	/* HUGETLB_VMEMMAP_RESERVE_SIZE should cover all used struct pages */
+	BUILD_BUG_ON(__NR_USED_SUBPAGE > HUGETLB_VMEMMAP_RESERVE_PAGES);
 
 	for_each_hstate(h) {
 		if (hugetlb_vmemmap_optimizable(h)) {
mm/hugetlb_vmemmap.h +5 -0
index 18b490825215..91cdb28969fa 100644
@@ -20,6 +20,7 @@
 #define HUGETLB_VMEMMAP_RESERVE_PAGES	(HUGETLB_VMEMMAP_RESERVE_SIZE / sizeof(struct page))
 
 #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP
+void hugetlb_vmemmap_init_tails(void);
 int hugetlb_vmemmap_restore_folio(const struct hstate *h, struct folio *folio);
 long hugetlb_vmemmap_restore_folios(const struct hstate *h,
 					struct list_head *folio_list,
@@ -72,6 +73,10 @@ static inline void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list
 {
 }
 
+static inline void hugetlb_vmemmap_init_tails(void)
+{
+}
+
 static inline void hugetlb_vmemmap_optimize_bootmem_folios(struct hstate *h,
 						struct list_head *folio_list)
 {
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
Reviewed-by Tested-by on v1 1/1
#1 lore raw
Quoted 17 lines · Michal Clapinski

Currently, if you enable HVO, DEFERRED_STRUCT_PAGE_INIT and VM_DEBUG the kernel will crash with the following stack trace

get_pfnblock_bitmap_bitidx
__set_pfnblock_flags_mask
hugetlb_bootmem_init_migratetype
prep_and_add_bootmem_folios
gather_bootmem_prealloc_node
gather_bootmem_prealloc_parallel
padata_do_multithreaded
gather_bootmem_prealloc
hugetlb_init

on this code

VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page);

Quoted 7 lines

This code looks inside the struct page which will be uninitialized for hugetlb tail pages, which will cause a false positive.

So let's initialize the tail pages before this happens.

Fixes 622026e87c40 ("mm/hugetlb: remove fake head pages")

One nit below.

Quoted 3 lines

---
I think this one check is the only thing that fails, so I can probably just remove it instead.

Please don't. It caught a legitimate issue.

Quoted 23 lines
---
 mm/hugetlb.c         |  1 +
 mm/hugetlb_vmemmap.c | 14 +++++++++-----
 mm/hugetlb_vmemmap.h |  5 +++++
 3 files changed, 15 insertions(+), 5 deletions(-)
mm/hugetlb.c +1 -0
index 571212b80835..de5e7c104c10 100644
@@ -4127,6 +4127,7 @@ static int __init hugetlb_init(void)
 	}
 
 	hugetlb_init_hstates();
+	hugetlb_vmemmap_init_tails();
 	gather_bootmem_prealloc();
 	report_hugepages();
 
mm/hugetlb_vmemmap.c +0 -0
index 133b46dfb09f..cf485c101926 100644

mm/hugetlb_vmemmap.c static const struct ctl_table hugetlb_vmemmap_sysctls[] = { · line 870

Quoted 5 lines
mm/hugetlb_vmemmap.c +1 -1
@@ -867,14 +867,10 @@ static const struct ctl_table hugetlb_vmemmap_sysctls[] = {
 	},
 };
 
-static int __init hugetlb_vmemmap_init(void)
+void __init hugetlb_vmemmap_init_tails(void)

While you're touching this, please also fix up the now-stale comment in vmemmap_get_tail() in mm/sparse-vmemmap.c, which still points at the old function.

Signature · 1 line
  Kiryl Shutsemau / Kirill A. Shutemov
maintainer
#2 lore raw
In reply to Michal Clapinski
Quoted 21 lines · Michal Clapinski

Currently, if you enable HVO, DEFERRED_STRUCT_PAGE_INIT and VM_DEBUG the kernel will crash with the following stack trace

get_pfnblock_bitmap_bitidx
__set_pfnblock_flags_mask
hugetlb_bootmem_init_migratetype
prep_and_add_bootmem_folios
gather_bootmem_prealloc_node
gather_bootmem_prealloc_parallel
padata_do_multithreaded
gather_bootmem_prealloc
hugetlb_init

on this code

VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page);

Quoted 7 lines

This code looks inside the struct page which will be uninitialized for hugetlb tail pages, which will cause a false positive.

So let's initialize the tail pages before this happens.

Fixes 622026e87c40 ("mm/hugetlb: remove fake head pages")

Thanks for catching this! Your fix is perfectly correct and addresses a real issue. I just wanted to let you know that a similar fix was actually submitted a few days ago in patch [1].

[1] https://lore.kernel.org/linux-mm/[email protected]/

Muchun,
Thanks.