FAILED: patch "[PATCH] mm/sparse-vmemmap: fix DAX vmemmap accounting with" failed to apply to 7.1-stable tree
Subsystems (3)
[email protected]
maintainerThe patch below does not apply to the 7.1-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.
To reproduce the conflict and resubmit, you may use the following commands:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-7.1.y
git checkout FETCH_HEAD
git cherry-pick -x 721a73e30c9e3e8fcffe1725bcede1bbd20b4918
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<[email protected]>' --in-reply-to '2026072010-unadorned-lubricate-bf28@gregkh' --subject-prefix 'PATCH 7.1.y' 'HEAD^..'
Possible dependencies:
thanks,
greg k-h
------------------ original commit in Linus's tree ------------------
From 721a73e30c9e3e8fcffe1725bcede1bbd20b4918 Mon Sep 17 00:00:00 2001
From: Muchun Song <[email protected]>
Date: Tue, 28 Apr 2026 16:18:53 +0800
Subject: [PATCH] mm/sparse-vmemmap: fix DAX vmemmap accounting with
optimization
When vmemmap optimization is enabled for DAX, the nr_memmap_pages counter in /proc/vmstat is incorrect. The current code always accounts for the full, non-optimized vmemmap size, but vmemmap optimization reduces the actual number of vmemmap pages by reusing tail pages. This causes the system to overcount vmemmap usage, leading to inaccurate page statistics in /proc/vmstat.
Fix this by introducing section_nr_vmemmap_pages(), which returns the exact vmemmap page count for a given pfn range based on whether optimization is in effect.
mm/sparse-vmemmap.c +30 -4
@@ -647,6 +647,31 @@ void offline_mem_sections(unsigned long start_pfn, unsigned long end_pfn) } } +static int __meminit section_nr_vmemmap_pages (unsigned long pfn,unsigned long nr_pages, +struct vmem_altmap *altmap,struct dev_pagemap *pgmap) +{ +const unsigned int order = pgmap ? pgmap->vmemmap_shift :0 ; +const unsigned long pages_per_compound =1UL << order; + +VM_WARN_ON_ONCE (!IS_ALIGNED (pfn | nr_pages,PAGES_PER_SUBSECTION )); +VM_WARN_ON_ONCE (nr_pages >PAGES_PER_SECTION ); + +if (!vmemmap_can_optimize (altmap, pgmap)) +return DIV_ROUND_UP (nr_pages *sizeof (struct page ),PAGE_SIZE ); + +if (order <PFN_SECTION_SHIFT ) { +VM_WARN_ON_ONCE (!IS_ALIGNED (pfn | nr_pages, pages_per_compound)); +return VMEMMAP_RESERVE_NR * nr_pages / pages_per_compound; + } + +VM_WARN_ON_ONCE (!IS_ALIGNED (pfn | nr_pages,PAGES_PER_SECTION )); + +if (IS_ALIGNED (pfn, pages_per_compound)) +return VMEMMAP_RESERVE_NR ; + +return 0 ; +} +static struct page *__meminit populate_section_memmap (unsigned long pfn,unsigned long nr_pages,int nid,struct vmem_altmap *altmap,struct dev_pagemap *pgmap)
@@ -654,7 +679,7 @@ static struct page * __meminit populate_section_memmap(unsigned long pfn,struct page *page =__populate_section_memmap (pfn, nr_pages, nid, altmap, pgmap); - memmap_pages_add(DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE)); +memmap_pages_add (section_nr_vmemmap_pages (pfn, nr_pages, altmap, pgmap));return page; }
@@ -665,7 +690,7 @@ static void depopulate_section_memmap(unsigned long pfn, unsigned long nr_pages,unsigned long start = (unsigned long )pfn_to_page (pfn);unsigned long end = start + nr_pages *sizeof (struct page ); -memmap_pages_add (-1L * (DIV_ROUND_UP (nr_pages *sizeof (struct page ),PAGE_SIZE ))); +memmap_pages_add (-section_nr_vmemmap_pages (pfn, nr_pages, altmap, pgmap));vmemmap_free (start, end, altmap); }
@@ -673,9 +698,10 @@ static void free_map_bootmem(struct page *memmap) {unsigned long start = (unsigned long )memmap;unsigned long end = (unsigned long )(memmap +PAGES_PER_SECTION ); +unsigned long pfn =page_to_pfn (memmap); -memmap_boot_pages_add (-1L * (DIV_ROUND_UP (PAGES_PER_SECTION *sizeof (struct page ), -PAGE_SIZE ))); +memmap_boot_pages_add (-section_nr_vmemmap_pages (pfn,PAGES_PER_SECTION , +NULL ,NULL ));vmemmap_free (start, end,NULL ); }
Sasha Levin
maintainerPATCH 1/4 mm/sparse-vmemmap: fix vmemmap accounting underflow
From: Muchun Song <[email protected]>
[ Upstream commit c373f7f98e6ad591c85d40548cf8b6443be69311 ]
Patch series "mm: Fix vmemmap optimization accounting and initialization", v8.
The series fixes several bugs in vmemmap optimization, mainly around incorrect page accounting and memmap initialization in DAX and memory hotplug paths. It also fixes pageblock migratetype initialization and struct page initialization for ZONE_DEVICE compound pages.
Patches 1-4 fix vmemmap accounting issues. Patch 1 fixes an accounting underflow in the section activation failure path by moving vmemmap page accounting into the lower-level allocation and freeing helpers. Patch 2 fixes incorrect altmap passing in the memory hotplug error path. Patch 3 passes pgmap through memory deactivation paths so the teardown side can determine whether vmemmap optimization was in effect. Patch 4 uses that information to account the optimized DAX vmemmap size correctly.
Patches 5-6 fix initialization issues in mm/mm_init. One makes sure all pageblocks in ZONE_DEVICE compound pages get their migratetype initialized. The other fixes a case where DAX memory hotplug reuses an unoptimized early-section memmap while compound_nr_pages() still assumes vmemmap optimization, leaving tail struct pages uninitialized.
This patch (of 6):
In section_activate(), if populate_section_memmap() fails, the error handling path calls section_deactivate() to roll back the state. This causes a vmemmap accounting imbalance.
Since commit c3576889d87b ("mm: fix accounting of memmap pages"), memmap pages are accounted for only after populate_section_memmap() succeeds. However, the failure path unconditionally calls section_deactivate(), which decreases the vmemmap count. Consequently, a failure in populate_section_memmap() leads to an accounting underflow, incorrectly reducing the system's tracked vmemmap usage.
Fix this more thoroughly by moving all accounting calls into the lower level functions that actually perform the vmemmap allocation and freeing:
- populate_section_memmap() accounts for newly allocated vmemmap pages -depopulate_section_memmap() unaccounts when vmemmap is freed
This ensures proper accounting in all code paths, including error handling and early section cases.
Stable-dep-of: 721a73e30c9e ("mm/sparse-vmemmap: fix DAX vmemmap accounting with optimization")
--- mm/sparse-vmemmap.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
mm/sparse-vmemmap.c +12 -8
@@ -656,7 +656,12 @@ static struct page * __meminit populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap, struct dev_pagemap *pgmap) { - return __populate_section_memmap(pfn, nr_pages, nid, altmap, pgmap); + struct page *page = __populate_section_memmap(pfn, nr_pages, nid, altmap, + pgmap); + + memmap_pages_add(DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE)); + +return page; } static void depopulate_section_memmap(unsigned long pfn, unsigned long nr_pages,
@@ -665,13 +670,17 @@ static void depopulate_section_memmap(unsigned long pfn, unsigned long nr_pages,unsigned long start = (unsigned long )pfn_to_page (pfn);unsigned long end = start + nr_pages *sizeof (struct page ); +memmap_pages_add (-1L * (DIV_ROUND_UP (nr_pages *sizeof (struct page ),PAGE_SIZE )));vmemmap_free (start, end, altmap); } +static void free_map_bootmem (struct page *memmap) {unsigned long start = (unsigned long )memmap;unsigned long end = (unsigned long )(memmap +PAGES_PER_SECTION ); +memmap_boot_pages_add (-1L * (DIV_ROUND_UP (PAGES_PER_SECTION *sizeof (struct page ), +PAGE_SIZE )));vmemmap_free (start, end,NULL ); }
@@ -774,14 +783,10 @@ static void section_deactivate(unsigned long pfn, unsigned long nr_pages,* The memmap of early sections is always fully populated. See * section_activate() and pfn_valid() . */ -if (!section_is_early) { -memmap_pages_add (-1L * (DIV_ROUND_UP (nr_pages *sizeof (struct page ),PAGE_SIZE ))); +if (!section_is_early)depopulate_section_memmap (pfn, nr_pages, altmap); - }else if (memmap) { -memmap_boot_pages_add (-1L * (DIV_ROUND_UP (nr_pages *sizeof (struct page ), -PAGE_SIZE ))); +else if (memmap)free_map_bootmem (memmap); - }if (empty) ms->section_mem_map = (unsigned long )NULL ;
@@ -826,7 +831,6 @@ static struct page * __meminit section_activate(int nid, unsigned long pfn,section_deactivate (pfn, nr_pages, altmap);return ERR_PTR (-ENOMEM ); } - memmap_pages_add(DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE));return memmap; }
Signature · 1 line
2.53.0
Sasha Levin
maintainerPATCH 2/4 mm/memory_hotplug: fix incorrect altmap passing in error path
From: Muchun Song <[email protected]>
[ Upstream commit 2fac4afa0e2e68841334c78c1821e49f74fbc66a ]
In create_altmaps_and_memory_blocks(), when arch_add_memory() succeeds with memmap_on_memory enabled, the vmemmap pages are allocated from params.altmap. If create_memory_block_devices() subsequently fails, the error path calls arch_remove_memory() with a NULL altmap instead of params.altmap.
This is a bug that could lead to memory corruption. Since altmap is NULL, vmemmap_free() falls back to freeing the vmemmap pages into the system buddy allocator via free_pages() instead of the altmap. arch_remove_memory() then immediately destroys the physical linear mapping for this memory. This injects unowned pages into the buddy allocator, causing machine checks or memory corruption if the system later attempts to allocate and use those freed pages.
Fix this by passing params.altmap to arch_remove_memory() in the error path.
Stable-dep-of: 721a73e30c9e ("mm/sparse-vmemmap: fix DAX vmemmap accounting with optimization")
--- mm/memory_hotplug.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/memory_hotplug.c +1 -1
@@ -1470,7 +1470,7 @@ static int create_altmaps_and_memory_blocks(int nid, struct memory_group *group, ret =create_memory_block_devices (cur_start, memblock_size, nid, params.altmap, group);if (ret) { -arch_remove_memory (cur_start, memblock_size,NULL ); +arch_remove_memory (cur_start, memblock_size, params.altmap);kfree (params.altmap); goto out; }
Signature · 1 line
2.53.0
Sasha Levin
maintainerPATCH 3/4 mm/sparse-vmemmap: pass @pgmap argument to memory deactivation paths
In reply to Sasha LevinFrom: Muchun Song <[email protected]>
[ Upstream commit 3bbc54dd1b62f1a4b218c70aafbeceeba7c90c5d ]
Currently, the memory hot-remove call chain -- arch_remove_memory(), __remove_pages(), sparse_remove_section() and section_deactivate() -- does not carry the struct dev_pagemap pointer. This prevents the lower levels from knowing whether the section was originally populated with vmemmap optimizations (e.g., DAX with vmemmap optimization enabled).
Without this information, we cannot call vmemmap_can_optimize() to determine if the vmemmap pages were optimized. As a result, the vmemmap page accounting during teardown will mistakenly assume a non-optimized allocation, leading to incorrect memmap statistics.
To lay the groundwork for fixing the vmemmap page accounting, we need to pass the @pgmap pointer down to the deactivation location. Plumb the @pgmap argument through the APIs of arch_remove_memory(), __remove_pages() and sparse_remove_section(), mirroring the corresponding *_activate() paths.
Stable-dep-of: 721a73e30c9e ("mm/sparse-vmemmap: fix DAX vmemmap accounting with optimization")
--- arch/arm64/mm/mmu.c | 5 +++-- arch/loongarch/mm/init.c | 5 +++-- arch/powerpc/mm/mem.c | 5 +++-- arch/riscv/mm/init.c | 5 +++-- arch/s390/mm/init.c | 5 +++-- arch/x86/mm/init_64.c | 5 +++-- include/linux/memory_hotplug.h | 8 +++++--- mm/memory_hotplug.c | 13 +++++++------ mm/memremap.c | 4 ++-- mm/sparse-vmemmap.c | 12 ++++++------ 10 files changed, 38 insertions(+), 29 deletions(-)
arch/arm64/mm/mmu.c +3 -2
@@ -2025,12 +2025,13 @@ int arch_add_memory(int nid, u64 start, u64 size,return ret; } -void arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap) +void arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap, +struct dev_pagemap *pgmap) {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -__remove_pages (start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap, pgmap);__remove_pgd_mapping (swapper_pg_dir,__phys_to_virt (start), size); }
arch/loongarch/mm/init.c +3 -2
@@ -119,12 +119,13 @@ int arch_add_memory(int nid, u64 start, u64 size, struct mhp_params *params) return ret; } -void arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap) +void arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap, +struct dev_pagemap *pgmap) {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -__remove_pages (start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap, pgmap); }#endif
arch/powerpc/mm/mem.c +3 -2
@@ -158,12 +158,13 @@ int __ref arch_add_memory(int nid, u64 start, u64 size, return rc; } -void __ref arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap) +void __ref arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap, +struct dev_pagemap *pgmap) {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -__remove_pages (start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap, pgmap);arch_remove_linear_mapping (start, size); }#endif
arch/riscv/mm/init.c +3 -2
@@ -1742,9 +1742,10 @@ int __ref arch_add_memory(int nid, u64 start, u64 size, struct mhp_params *paramreturn ret; } -void __ref arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap) +void __ref arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap, +struct dev_pagemap *pgmap) { -__remove_pages (start >>PAGE_SHIFT , size >>PAGE_SHIFT , altmap); +__remove_pages (start >>PAGE_SHIFT , size >>PAGE_SHIFT , altmap, pgmap);remove_linear_mapping (start, size);flush_tlb_all (); }
arch/s390/mm/init.c +3 -2
@@ -276,12 +276,13 @@ int arch_add_memory(int nid, u64 start, u64 size, return rc; } -void arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap) +void arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap, +struct dev_pagemap *pgmap) {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -__remove_pages (start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap, pgmap);vmem_remove_mapping (start, size); }#endif /* CONFIG_MEMORY_HOTPLUG */
arch/x86/mm/init_64.c +3 -2
@@ -1300,12 +1300,13 @@ kernel_physical_mapping_remove(unsigned long start, unsigned long end) remove_pagetable(start, end, true, NULL); } -void __ref arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap) +void __ref arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap, +struct dev_pagemap *pgmap) {unsigned long start_pfn = start >>PAGE_SHIFT ;unsigned long nr_pages = size >>PAGE_SHIFT ; -__remove_pages (start_pfn, nr_pages, altmap); +__remove_pages (start_pfn, nr_pages, altmap, pgmap);kernel_physical_mapping_remove (start, start + size); }#endif /* CONFIG_MEMORY_HOTPLUG */
include/linux/memory_hotplug.h +5 -3
@@ -135,9 +135,10 @@ static inline bool movable_node_is_enabled(void)return movable_node_enabled; } -extern void arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap); +extern void arch_remove_memory (u64 start,u64 size,struct vmem_altmap *altmap, +struct dev_pagemap *pgmap);extern void __remove_pages (unsigned long start_pfn,unsigned long nr_pages, -struct vmem_altmap *altmap); +struct vmem_altmap *altmap,struct dev_pagemap *pgmap);/* reasonably generic interface to expand the physical pages */ extern int __add_pages(int nid, unsigned long start_pfn, unsigned long nr_pages,
@@ -307,7 +308,8 @@ extern int sparse_add_section(int nid, unsigned long pfn, unsigned long nr_pages, struct vmem_altmap *altmap, struct dev_pagemap *pgmap);extern void sparse_remove_section (unsigned long pfn,unsigned long nr_pages, -struct vmem_altmap *altmap); +struct vmem_altmap *altmap, +struct dev_pagemap *pgmap);extern struct zone *zone_for_pfn_range (enum mmop online_type,int nid,struct memory_group *group,unsigned long start_pfn,unsigned long nr_pages);
mm/memory_hotplug.c +7 -6
@@ -576,6 +576,7 @@ void remove_pfn_range_from_zone(struct zone *zone,* @pfn: starting pageframe (must be aligned to start of a section) * @nr_pages: number of pages to remove (must be multiple of section size) * @altmap: alternative device page map or %NULL if default memmap is used +* @pgmap: device page map or %NULL if not ZONE_DEVICE * * Generic helper function to remove section mappings and sysfs entries * for the section of the memory we are removing. Caller needs to make
@@ -583,7 +584,7 @@ void remove_pfn_range_from_zone(struct zone *zone,* calling offline_pages(). */ void __remove_pages (unsigned long pfn,unsigned long nr_pages, -struct vmem_altmap *altmap) +struct vmem_altmap *altmap,struct dev_pagemap *pgmap) {const unsigned long end_pfn = pfn + nr_pages;unsigned long cur_nr_pages;
@@ -598,7 +599,7 @@ void __remove_pages(unsigned long pfn, unsigned long nr_pages,/* Select all remaining pages up to the next section boundary */ cur_nr_pages =min (end_pfn - pfn,SECTION_ALIGN_UP (pfn +1 ) - pfn); -sparse_remove_section (pfn, cur_nr_pages, altmap); +sparse_remove_section (pfn, cur_nr_pages, altmap, pgmap); } }
@@ -1427,7 +1428,7 @@ static void remove_memory_blocks_and_altmaps(u64 start, u64 size)remove_memory_block_devices (cur_start, memblock_size); -arch_remove_memory (cur_start, memblock_size, altmap); +arch_remove_memory (cur_start, memblock_size, altmap,NULL );/* Verify that all vmemmap pages have actually been freed. */ WARN (altmap->alloc,"Altmap not fully unmapped" );
@@ -1470,7 +1471,7 @@ static int create_altmaps_and_memory_blocks(int nid, struct memory_group *group, ret =create_memory_block_devices (cur_start, memblock_size, nid, params.altmap, group);if (ret) { -arch_remove_memory (cur_start, memblock_size, params.altmap); +arch_remove_memory (cur_start, memblock_size, params.altmap,NULL );kfree (params.altmap); goto out; }
@@ -1556,7 +1557,7 @@ int add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags)/* create memory block devices after memory was added */ ret =create_memory_block_devices (start, size, nid,NULL , group);if (ret) { -arch_remove_memory (start, size, params.altmap); +arch_remove_memory (start, size, params.altmap,NULL ); goto error; } }
@@ -2268,7 +2269,7 @@ static int try_remove_memory(u64 start, u64 size)* No altmaps present, do the removal directly */ remove_memory_block_devices (start, size); -arch_remove_memory (start, size,NULL ); +arch_remove_memory (start, size,NULL ,NULL ); }else {/* all memblocks in the range have altmaps */ remove_memory_blocks_and_altmaps (start, size);
mm/memremap.c +2 -2
@@ -97,10 +97,10 @@ static void pageunmap_range(struct dev_pagemap *pgmap, int range_id) PHYS_PFN(range_len(range)));if (pgmap->type ==MEMORY_DEVICE_PRIVATE ) {__remove_pages (PHYS_PFN (range->start), -PHYS_PFN (range_len (range)),NULL ); +PHYS_PFN (range_len (range)),NULL , pgmap); }else {arch_remove_memory (range->start,range_len (range), -pgmap_altmap (pgmap)); +pgmap_altmap (pgmap), pgmap);kasan_remove_zero_shadow (__va (range->start),range_len (range)); }mem_hotplug_done ();
mm/sparse-vmemmap.c +6 -6
@@ -665,7 +665,7 @@ static struct page * __meminit populate_section_memmap(unsigned long pfn, }static void depopulate_section_memmap (unsigned long pfn,unsigned long nr_pages, -struct vmem_altmap *altmap) +struct vmem_altmap *altmap,struct dev_pagemap *pgmap) {unsigned long start = (unsigned long )pfn_to_page (pfn);unsigned long end = start + nr_pages *sizeof (struct page );
@@ -746,7 +746,7 @@ static int fill_subsection_map(unsigned long pfn, unsigned long nr_pages)* usage map, but still need to free the vmemmap range. */ static void section_deactivate (unsigned long pfn,unsigned long nr_pages, -struct vmem_altmap *altmap) +struct vmem_altmap *altmap,struct dev_pagemap *pgmap) {struct mem_section *ms =__pfn_to_section (pfn);bool section_is_early =early_section (ms);
@@ -784,7 +784,7 @@ static void section_deactivate(unsigned long pfn, unsigned long nr_pages,* section_activate() and pfn_valid() . */ if (!section_is_early) -depopulate_section_memmap (pfn, nr_pages, altmap); +depopulate_section_memmap (pfn, nr_pages, altmap, pgmap);else if (memmap)free_map_bootmem (memmap);
@@ -828,7 +828,7 @@ static struct page * __meminit section_activate(int nid, unsigned long pfn, memmap =populate_section_memmap (pfn, nr_pages, nid, altmap, pgmap);if (!memmap) { -section_deactivate (pfn, nr_pages, altmap); +section_deactivate (pfn, nr_pages, altmap, pgmap);return ERR_PTR (-ENOMEM ); }
@@ -889,13 +889,13 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn, }void sparse_remove_section (unsigned long pfn,unsigned long nr_pages, -struct vmem_altmap *altmap) +struct vmem_altmap *altmap,struct dev_pagemap *pgmap) {struct mem_section *ms =__pfn_to_section (pfn);if (WARN_ON_ONCE (!valid_section (ms)))return ; -section_deactivate (pfn, nr_pages, altmap); +section_deactivate (pfn, nr_pages, altmap, pgmap); }#endif /* CONFIG_MEMORY_HOTPLUG */
Signature · 1 line
2.53.0
Sasha Levin
maintainerPATCH 4/4 mm/sparse-vmemmap: fix DAX vmemmap accounting with optimization
In reply to Sasha LevinFrom: Muchun Song <[email protected]>
[ Upstream commit 721a73e30c9e3e8fcffe1725bcede1bbd20b4918 ]
When vmemmap optimization is enabled for DAX, the nr_memmap_pages counter in /proc/vmstat is incorrect. The current code always accounts for the full, non-optimized vmemmap size, but vmemmap optimization reduces the actual number of vmemmap pages by reusing tail pages. This causes the system to overcount vmemmap usage, leading to inaccurate page statistics in /proc/vmstat.
Fix this by introducing section_nr_vmemmap_pages(), which returns the exact vmemmap page count for a given pfn range based on whether optimization is in effect.
--- mm/sparse-vmemmap.c | 34 ++++++++++++++++++++++++++++++---- 1 file changed, 30 insertions(+), 4 deletions(-)
mm/sparse-vmemmap.c +30 -4
@@ -652,6 +652,31 @@ void offline_mem_sections(unsigned long start_pfn, unsigned long end_pfn) } } +static int __meminit section_nr_vmemmap_pages (unsigned long pfn,unsigned long nr_pages, +struct vmem_altmap *altmap,struct dev_pagemap *pgmap) +{ +const unsigned int order = pgmap ? pgmap->vmemmap_shift :0 ; +const unsigned long pages_per_compound =1UL << order; + +VM_WARN_ON_ONCE (!IS_ALIGNED (pfn | nr_pages,PAGES_PER_SUBSECTION )); +VM_WARN_ON_ONCE (nr_pages >PAGES_PER_SECTION ); + +if (!vmemmap_can_optimize (altmap, pgmap)) +return DIV_ROUND_UP (nr_pages *sizeof (struct page ),PAGE_SIZE ); + +if (order <PFN_SECTION_SHIFT ) { +VM_WARN_ON_ONCE (!IS_ALIGNED (pfn | nr_pages, pages_per_compound)); +return VMEMMAP_RESERVE_NR * nr_pages / pages_per_compound; + } + +VM_WARN_ON_ONCE (!IS_ALIGNED (pfn | nr_pages,PAGES_PER_SECTION )); + +if (IS_ALIGNED (pfn, pages_per_compound)) +return VMEMMAP_RESERVE_NR ; + +return 0 ; +} +static struct page *__meminit populate_section_memmap (unsigned long pfn,unsigned long nr_pages,int nid,struct vmem_altmap *altmap,struct dev_pagemap *pgmap)
@@ -659,7 +684,7 @@ static struct page * __meminit populate_section_memmap(unsigned long pfn,struct page *page =__populate_section_memmap (pfn, nr_pages, nid, altmap, pgmap); - memmap_pages_add(DIV_ROUND_UP(nr_pages * sizeof(struct page), PAGE_SIZE)); +memmap_pages_add (section_nr_vmemmap_pages (pfn, nr_pages, altmap, pgmap));return page; }
@@ -670,7 +695,7 @@ static void depopulate_section_memmap(unsigned long pfn, unsigned long nr_pages,unsigned long start = (unsigned long )pfn_to_page (pfn);unsigned long end = start + nr_pages *sizeof (struct page ); -memmap_pages_add (-1L * (DIV_ROUND_UP (nr_pages *sizeof (struct page ),PAGE_SIZE ))); +memmap_pages_add (-section_nr_vmemmap_pages (pfn, nr_pages, altmap, pgmap));vmemmap_free (start, end, altmap); }
@@ -678,9 +703,10 @@ static void free_map_bootmem(struct page *memmap) {unsigned long start = (unsigned long )memmap;unsigned long end = (unsigned long )(memmap +PAGES_PER_SECTION ); +unsigned long pfn =page_to_pfn (memmap); -memmap_boot_pages_add (-1L * (DIV_ROUND_UP (PAGES_PER_SECTION *sizeof (struct page ), -PAGE_SIZE ))); +memmap_boot_pages_add (-section_nr_vmemmap_pages (pfn,PAGES_PER_SECTION , +NULL ,NULL ));vmemmap_free (start, end,NULL ); }
Signature · 1 line
2.53.0