mm/hugetlb: init tails before init_migratetype
Subsystems (2)
PATCH 1/1 mm/hugetlb: init tails before init_migratetype
Currently, if you enable HVO, DEFERRED_STRUCT_PAGE_INIT and VM_DEBUG the kernel will crash with the following stack trace
get_pfnblock_bitmap_bitidx__set_pfnblock_flags_maskhugetlb_bootmem_init_migratetypeprep_and_add_bootmem_foliosgather_bootmem_prealloc_nodegather_bootmem_prealloc_parallelpadata_do_multithreadedgather_bootmem_preallochugetlb_init
on this code
VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page);
This code looks inside the struct page which will be uninitialized for hugetlb tail pages, which will cause a false positive.
So let's initialize the tail pages before this happens.
---
I think this one check is the only thing that fails, so I can probably just remove it instead.
--- mm/hugetlb.c | 1 + mm/hugetlb_vmemmap.c | 14 +++++++++----- mm/hugetlb_vmemmap.h | 5 +++++ 3 files changed, 15 insertions(+), 5 deletions(-)
mm/hugetlb.c +1 -0
@@ -4127,6 +4127,7 @@ static int __init hugetlb_init(void) }hugetlb_init_hstates (); +hugetlb_vmemmap_init_tails ();gather_bootmem_prealloc ();report_hugepages ();
mm/hugetlb_vmemmap.c +9 -5
@@ -867,14 +867,10 @@ static const struct ctl_table hugetlb_vmemmap_sysctls[] = { }, }; -static int __init hugetlb_vmemmap_init (void ) +void __init hugetlb_vmemmap_init_tails (void ) { -const struct hstate *h;struct zone *zone; -/* HUGETLB_VMEMMAP_RESERVE_SIZE should cover all used struct pages */ -BUILD_BUG_ON (__NR_USED_SUBPAGE >HUGETLB_VMEMMAP_RESERVE_PAGES ); -for_each_zone (zone) {for (int i =0 ; i <NR_VMEMMAP_TAILS ; i++) {struct page *tail, *p;
@@ -890,6 +886,14 @@ static int __init hugetlb_vmemmap_init(void)init_compound_tail (p + j,NULL , order, zone); } } +} + +static int __init hugetlb_vmemmap_init (void ) +{ +const struct hstate *h; + +/* HUGETLB_VMEMMAP_RESERVE_SIZE should cover all used struct pages */ +BUILD_BUG_ON (__NR_USED_SUBPAGE >HUGETLB_VMEMMAP_RESERVE_PAGES );for_each_hstate (h) {if (hugetlb_vmemmap_optimizable (h)) {
mm/hugetlb_vmemmap.h +5 -0
@@ -20,6 +20,7 @@#define HUGETLB_VMEMMAP_RESERVE_PAGES (HUGETLB_VMEMMAP_RESERVE_SIZE / sizeof(struct page))#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP +void hugetlb_vmemmap_init_tails (void );int hugetlb_vmemmap_restore_folio (const struct hstate *h,struct folio *folio); long hugetlb_vmemmap_restore_folios(const struct hstate *h, struct list_head *folio_list,
@@ -72,6 +73,10 @@ static inline void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list { } +static inline void hugetlb_vmemmap_init_tails (void ) +{ +} +static inline void hugetlb_vmemmap_optimize_bootmem_folios (struct hstate *h,struct list_head *folio_list) {
Signature · 1 line
2.55.0.rc0.799.gd6f94ed593-goog
Quoted 17 lines · Michal Clapinski
Currently, if you enable HVO,
DEFERRED_STRUCT_PAGE_INITandVM_DEBUGthe kernel will crash with the following stack trace
get_pfnblock_bitmap_bitidx__set_pfnblock_flags_maskhugetlb_bootmem_init_migratetypeprep_and_add_bootmem_foliosgather_bootmem_prealloc_nodegather_bootmem_prealloc_parallelpadata_do_multithreadedgather_bootmem_preallochugetlb_initon this code
VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page);
Quoted 7 lines
This code looks inside the struct page which will be uninitialized for hugetlb tail pages, which will cause a false positive.
So let's initialize the tail pages before this happens.
Fixes 622026e87c40 ("mm/hugetlb: remove fake head pages")Signed-off-by Michal Clapinski <[email protected]>
One nit below.
Quoted 3 lines
---
I think this one check is the only thing that fails, so I can probably just remove it instead.
Please don't. It caught a legitimate issue.
Quoted 23 lines
--- mm/hugetlb.c | 1 + mm/hugetlb_vmemmap.c | 14 +++++++++----- mm/hugetlb_vmemmap.h | 5 +++++ 3 files changed, 15 insertions(+), 5 deletions(-)mm/hugetlb.c +1
-0@@ -4127,6 +4127,7 @@ static int __init hugetlb_init(void) }hugetlb_init_hstates (); +hugetlb_vmemmap_init_tails ();gather_bootmem_prealloc ();report_hugepages ();mm/hugetlb_vmemmap.c +0
-0
mm/hugetlb_vmemmap.c static const struct ctl_table hugetlb_vmemmap_sysctls[] = { · line 870
Quoted 5 lines
mm/hugetlb_vmemmap.c +1
-1@@ -867,14 +867,10 @@ static const struct ctl_table hugetlb_vmemmap_sysctls[] = { }, }; -static int __init hugetlb_vmemmap_init (void ) +void __init hugetlb_vmemmap_init_tails (void )
While you're touching this, please also fix up the now-stale comment in vmemmap_get_tail() in mm/sparse-vmemmap.c, which still points at the old function.
Signature · 1 line
Kiryl Shutsemau / Kirill A. Shutemov
Muchun Song
maintainerQuoted 21 lines · Michal Clapinski
Currently, if you enable HVO,
DEFERRED_STRUCT_PAGE_INITandVM_DEBUGthe kernel will crash with the following stack trace
get_pfnblock_bitmap_bitidx__set_pfnblock_flags_maskhugetlb_bootmem_init_migratetypeprep_and_add_bootmem_foliosgather_bootmem_prealloc_nodegather_bootmem_prealloc_parallelpadata_do_multithreadedgather_bootmem_preallochugetlb_initon this code
VM_BUG_ON_PAGE(!zone_spans_pfn(page_zone(page), pfn), page);
Quoted 7 lines
This code looks inside the struct page which will be uninitialized for hugetlb tail pages, which will cause a false positive.
So let's initialize the tail pages before this happens.
Fixes 622026e87c40 ("mm/hugetlb: remove fake head pages")Signed-off-by Michal Clapinski <[email protected]>
Thanks for catching this! Your fix is perfectly correct and addresses a real issue. I just wanted to let you know that a similar fix was actually submitted a few days ago in patch [1].
[1] https://lore.kernel.org/linux-mm/[email protected]/
Muchun,
Thanks.