mm/memory_hotplug: use PageOffline() instead of PageReserved() for !ZONE_DEVICE
Subsystems (11)
cover mm/memory_hotplug: use PageOffline() instead of PageReserved() for !ZONE_DEVICE
This can be a considered a long-overdue follow-up to some parts of [1]. The patches are based on [2], but they are not strictly required -- just makes it clearer why we can use adjust_managed_page_count() for memory hotplug without going into details about highmem.
We stop initializing pages with PageReserved() in memory hotplug code -- except when dealing with ZONE_DEVICE for now. Instead, we use PageOffline(): all pages are initialized to PageOffline() when onlining a memory section, and only the ones actually getting exposed to the system/page allocator will get PageOffline cleared.
This way, we enlighten memory hotplug more about PageOffline() pages and can cleanup some hacks we have in virtio-mem code.
What about ZONE_DEVICE? PageOffline() is wrong, but we might just stop using PageReserved() for them later by simply checking for is_zone_device_page() at suitable places. That will be a separate patch set / proposal.
This primarily affects virtio-mem, HV-balloon and XEN balloon. I only briefly tested with virtio-mem, which benefits most from these cleanups.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lkml.kernel.org/r/[email protected]
David Hildenbrand (3):
mm: pass meminit_context to __free_pages_core() mm/memory_hotplug: initialize memmap of !ZONE_DEVICE with
PageOffline() instead of PageReserved()
mm/memory_hotplug: skip adjust_managed_page_count() for PageOffline()
pages when offlining
drivers/hv/hv_balloon.c | 5 ++-- drivers/virtio/virtio_mem.c | 29 +++++++++--------- drivers/xen/balloon.c | 9 ++++-- include/linux/memory_hotplug.h | 4 +-- include/linux/page-flags.h | 20 +++++++------ mm/internal.h | 3 +- mm/kmsan/init.c | 2 +- mm/memory_hotplug.c | 31 +++++++++---------- mm/mm_init.c | 14 ++++++--- mm/page_alloc.c | 55 +++++++++++++++++++++++++++------- 10 files changed, 108 insertions(+), 64 deletions(-)
base-commit: 19b8422c5bd56fb5e7085995801c6543a98bda1f
prerequisite-patch-id: ca280eafd2732d7912e0c5249dc0df9ecbef19ca
prerequisite-patch-id: 8f43ebc81fdf7b9b665b57614e9e569535094758
Signature · 1 line
2.45.1
PATCH 1/3 mm: pass meminit_context to __free_pages_core()
In preparation for further changes, let's teach __free_pages_core() about the differences of memory hotplug handling.
Move the memory hotplug specific handling from generic_online_page() to __free_pages_core(), use adjust_managed_page_count() on the memory hotplug path, and spell out why memory freed via memblock cannot currently use adjust_managed_page_count().
--- mm/internal.h | 3 ++- mm/kmsan/init.c | 2 +- mm/memory_hotplug.c | 9 +-------- mm/mm_init.c | 4 ++-- mm/page_alloc.c | 17 +++++++++++++++-- 5 files changed, 21 insertions(+), 14 deletions(-)
mm/internal.h +2 -1
@@ -604,7 +604,8 @@ extern void __putback_isolated_page(struct page *page, unsigned int order, int mt);extern void memblock_free_pages (struct page *page,unsigned long pfn,unsigned int order); -extern void __free_pages_core (struct page *page,unsigned int order); +extern void __free_pages_core (struct page *page,unsigned int order, +enum meminit_context );/* * This will have no effect, other than possibly generating a warning, if the
mm/kmsan/init.c +1 -1
@@ -172,7 +172,7 @@ static void do_collection(void) shadow =smallstack_pop (&collect); origin =smallstack_pop (&collect);kmsan_setup_meta (page, shadow, origin, collect.order); -__free_pages_core (page, collect.order); +__free_pages_core (page, collect.order,MEMINIT_EARLY ); } }
mm/memory_hotplug.c +1 -8
@@ -630,14 +630,7 @@ EXPORT_SYMBOL_GPL(restore_online_page_callback);void generic_online_page (struct page *page,unsigned int order) { -/* -* Freeing the page with debug_pagealloc enabled will try to unmap it, -* so we should map it first. This is better than introducing a special -* case in page freeing fast path. -*/ -debug_pagealloc_map_pages (page,1 << order); -__free_pages_core (page, order); -totalram_pages_add (1UL << order); +__free_pages_core (page, order,MEMINIT_HOTPLUG ); }EXPORT_SYMBOL_GPL (generic_online_page);
mm/mm_init.c +2 -2
@@ -1938,7 +1938,7 @@ static void __init deferred_free_range(unsigned long pfn,for (i =0 ; i < nr_pages; i++, page++, pfn++) {if (pageblock_aligned (pfn))set_pageblock_migratetype (page,MIGRATE_MOVABLE ); -__free_pages_core (page,0 ); +__free_pages_core (page,0 ,MEMINIT_EARLY ); } }
@@ -2513,7 +2513,7 @@ void __init memblock_free_pages(struct page *page, unsigned long pfn, } } -__free_pages_core (page, order); +__free_pages_core (page, order,MEMINIT_EARLY ); }DEFINE_STATIC_KEY_MAYBE (CONFIG_INIT_ON_ALLOC_DEFAULT_ON , init_on_alloc);
mm/page_alloc.c +15 -2
@@ -1214,7 +1214,8 @@ static void __free_pages_ok(struct page *page, unsigned int order,__count_vm_events (PGFREE ,1 << order); } -void __free_pages_core (struct page *page,unsigned int order) +void __free_pages_core (struct page *page,unsigned int order, +enum meminit_context context) {unsigned int nr_pages =1 << order;struct page *p = page;
@@ -1234,7 +1235,19 @@ void __free_pages_core(struct page *page, unsigned int order)__ClearPageReserved (p);set_page_count (p,0 ); -atomic_long_add (nr_pages, &page_zone (page)->managed_pages); +if (IS_ENABLED (CONFIG_MEMORY_HOTPLUG ) && +unlikely (context ==MEMINIT_HOTPLUG )) { +/* +* Freeing the page with debug_pagealloc enabled will try to +* unmap it; some archs don't like double-unmappings, so +* map it first. +*/ +debug_pagealloc_map_pages (page, nr_pages); +adjust_managed_page_count (page, nr_pages); + }else { +/* memblock adjusts totalram_pages() ahead of time. */ +atomic_long_add (nr_pages, &page_zone (page)->managed_pages); + }if (page_contains_unaccepted (page, order)) {if (order ==MAX_PAGE_ORDER &&__free_unaccepted (page))
Signature · 1 line
2.45.1
Re: [PATCH v1 1/3] mm: pass meminit_context to __free_pages_core()
Quoted 69 lines · David Hildenbrand
In preparation for further changes, let's teach
__free_pages_core()about the differences of memory hotplug handling.Move the memory hotplug specific handling from
generic_online_page()to__free_pages_core(), useadjust_managed_page_count()on the memory hotplug path, and spell out why memory freed via memblock cannot currently useadjust_managed_page_count().Signed-off-by David Hildenbrand <[email protected]>--- mm/internal.h | 3 ++- mm/kmsan/init.c | 2 +- mm/memory_hotplug.c | 9 +-------- mm/mm_init.c | 4 ++-- mm/page_alloc.c | 17 +++++++++++++++-- 5 files changed, 21 insertions(+), 14 deletions(-)mm/internal.h +2
-1@@ -604,7 +604,8 @@ extern void __putback_isolated_page(struct page *page, unsigned int order, int mt);extern void memblock_free_pages (struct page *page,unsigned long pfn,unsigned int order); -extern void __free_pages_core (struct page *page,unsigned int order); +extern void __free_pages_core (struct page *page,unsigned int order, +enum meminit_context );/* * This will have no effect, other than possibly generating a warning, if the mm/kmsan/init.c +1
-1@@ -172,7 +172,7 @@ static void do_collection(void) shadow =smallstack_pop (&collect); origin =smallstack_pop (&collect);kmsan_setup_meta (page, shadow, origin, collect.order); -__free_pages_core (page, collect.order); +__free_pages_core (page, collect.order,MEMINIT_EARLY ); } }mm/memory_hotplug.c +1
-8@@ -630,14 +630,7 @@ EXPORT_SYMBOL_GPL(restore_online_page_callback);void generic_online_page (struct page *page,unsigned int order) { -/* -* Freeing the page with debug_pagealloc enabled will try to unmap it, -* so we should map it first. This is better than introducing a special -* case in page freeing fast path. -*/ -debug_pagealloc_map_pages (page,1 << order); -__free_pages_core (page, order); -totalram_pages_add (1UL << order); +__free_pages_core (page, order,MEMINIT_HOTPLUG ); }EXPORT_SYMBOL_GPL (generic_online_page);mm/mm_init.c +0
-0
mm/mm_init.c static void __init deferred_free_range(unsigned long pfn, · line 1943
Quoted 7 lines
mm/mm_init.c +1
-1@@ -1938,7 +1938,7 @@ static void __init deferred_free_range(unsigned long pfn,for (i =0 ; i < nr_pages; i++, page++, pfn++) {if (pageblock_aligned (pfn))set_pageblock_migratetype (page,MIGRATE_MOVABLE ); -__free_pages_core (page,0 ); +__free_pages_core (page,0 ,MEMINIT_EARLY ); } }
The build bot just reminded me that I missed another case in this function:
(CONFIG_DEFERRED_STRUCT_PAGE_INIT)
mm/mm_init.c +1 -1
@@ -1928,7 +1928,7 @@ static void __init deferred_free_range(unsigned long pfn,if (nr_pages ==MAX_ORDER_NR_PAGES &&IS_MAX_ORDER_ALIGNED (pfn)) {for (i =0 ; i < nr_pages; i += pageblock_nr_pages)set_pageblock_migratetype (page + i,MIGRATE_MOVABLE ); -__free_pages_core (page,MAX_PAGE_ORDER ); +__free_pages_core (page,MAX_PAGE_ORDER ,MEMINIT_EARLY );return ; }
Signature · 3 lines
Cheers, David / dhildenb
Oscar Salvador
maintainerRe: [PATCH v1 1/3] mm: pass meminit_context to __free_pages_core()
In reply to David HildenbrandQuoted 9 lines · David Hildenbrand
In preparation for further changes, let's teach
__free_pages_core()about the differences of memory hotplug handling.Move the memory hotplug specific handling from
generic_online_page()to__free_pages_core(), useadjust_managed_page_count()on the memory hotplug path, and spell out why memory freed via memblock cannot currently useadjust_managed_page_count().Signed-off-by David Hildenbrand <[email protected]>
All looks good but I am puzzled with something.
mm/page_alloc.c void __free_pages_core(struct page *page, unsigned int order) · line 1250
Quoted 4 lines
mm/page_alloc.c +4
-0+ }else { +/* memblock adjusts totalram_pages() ahead of time. */ +atomic_long_add (nr_pages, &page_zone (page)->managed_pages); + }
You say that memblock adjusts totalram_pages ahead of time, and I guess you mean in memblock_free_all()
pages = free_low_memory_core_early()
totalram_pages_add(pages);
but that is not ahead, it looks like it is upading __after__ sending them to buddy?
Signature · 2 lines
Oscar Salvador SUSE Labs
Re: [PATCH v1 1/3] mm: pass meminit_context to __free_pages_core()
Quoted 13 lines · Oscar Salvador
Quoted 9 lines · David Hildenbrand
In preparation for further changes, let's teach
__free_pages_core()about the differences of memory hotplug handling.Move the memory hotplug specific handling from
generic_online_page()to__free_pages_core(), useadjust_managed_page_count()on the memory hotplug path, and spell out why memory freed via memblock cannot currently useadjust_managed_page_count().Signed-off-by David Hildenbrand <[email protected]>All looks good but I am puzzled with something.
Quoted 7 lines
mm/page_alloc.c void __free_pages_core(struct page *page, unsigned int order) · line 1250
Quoted 4 lines
mm/page_alloc.c +4
-0+ }else { +/* memblock adjusts totalram_pages() ahead of time. */ +atomic_long_add (nr_pages, &page_zone (page)->managed_pages); + }You say that memblock adjusts
totalram_pagesahead of time, and I guess you mean inmemblock_free_all()
And memblock_free_late(), which uses atomic_long_inc().
Quoted 6 lines
pages = free_low_memory_core_early() totalram_pages_add(pages);but that is not ahead, it looks like it is upading
__after__sending them to buddy?
Right (it's suboptimal, but not really problematic so far. Hopefully Wei can clean it up and move it in here as well)
For the time being
"/* memblock adjusts totalram_pages() manually. */"
?
Thanks!
Signature · 3 lines
Cheers, David / dhildenb
Oscar Salvador
maintainerRe: [PATCH v1 1/3] mm: pass meminit_context to __free_pages_core()
Quoted 22 lines · David Hildenbrand
Quoted 13 lines · Oscar Salvador
Quoted 9 lines · David Hildenbrand
In preparation for further changes, let's teach
__free_pages_core()about the differences of memory hotplug handling.Move the memory hotplug specific handling from
generic_online_page()to__free_pages_core(), useadjust_managed_page_count()on the memory hotplug path, and spell out why memory freed via memblock cannot currently useadjust_managed_page_count().Signed-off-by David Hildenbrand <[email protected]>All looks good but I am puzzled with something.
Quoted 7 lines
mm/page_alloc.c void __free_pages_core(struct page *page, unsigned int order) · line 1250
Quoted 4 lines
mm/page_alloc.c +4
-0+ }else { +/* memblock adjusts totalram_pages() ahead of time. */ +atomic_long_add (nr_pages, &page_zone (page)->managed_pages); + }You say that memblock adjusts
totalram_pagesahead of time, and I guess you mean inmemblock_free_all()
Quoted 1 line
And
memblock_free_late(), which usesatomic_long_inc().
Ah yes.
Quoted 2 lines
Right (it's suboptimal, but not really problematic so far. Hopefully Wei can clean it up and move it in here as well)
That would be great.
Quoted 3 lines
For the time being
"/* memblock adjusts
totalram_pages()manually. */"
Yes, I think that is better ;-)
Thanks!
Signature · 2 lines
Oscar Salvador SUSE Labs
Re: [PATCH v1 1/3] mm: pass meminit_context to __free_pages_core()
In reply to David HildenbrandQuoted 10 lines · David Hildenbrand
In preparation for further changes, let's teach
__free_pages_core()about the differences of memory hotplug handling.Move the memory hotplug specific handling from
generic_online_page()to__free_pages_core(), useadjust_managed_page_count()on the memory hotplug path, and spell out why memory freed via memblock cannot currently useadjust_managed_page_count().Signed-off-by David Hildenbrand <[email protected]>---
@Andrew, can you squash the following?
From 0a7921cf21cacf178ca7485da0138fc38a97a28e Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Tue, 11 Jun 2024 12:05:09 +0200
Subject: [PATCH] fixup: mm/highmem: make nr_free_highpages() return "unsigned
long"
Fixup the memblock comment.
--- mm/page_alloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/page_alloc.c +1 -1
@@ -1245,7 +1245,7 @@ void __free_pages_core(struct page *page, unsigned int order,debug_pagealloc_map_pages (page, nr_pages);adjust_managed_page_count (page, nr_pages); }else { -/* memblock adjusts totalram_pages() ahead of time. */ +/* memblock adjusts totalram_pages() manually. */ atomic_long_add (nr_pages, &page_zone (page)->managed_pages); }
Signature · 8 lines
2.45.2 -- Cheers, David / dhildenb
Re: [PATCH v1 1/3] mm: pass meminit_context to __free_pages_core()
Quoted 9 lines · On Tue, 11 Jun 2024 12:06:56 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 8 lines · David Hildenbrand
In preparation for further changes, let's teach
__free_pages_core()about the differences of memory hotplug handling.Move the memory hotplug specific handling from
generic_online_page()to__free_pages_core(), useadjust_managed_page_count()on the memory hotplug path, and spell out why memory freed via memblock cannot currently useadjust_managed_page_count().
Quoted 4 lines
Quoted 2 lines
Signed-off-by David Hildenbrand <[email protected]>---
@Andrew, can you squash the following?
Sure.
I queued it against "mm: pass meminit_context to __free_pages_core()", not against
Quoted 2 lines
Subject: [PATCH] fixup: mm/highmem: make
nr_free_highpages()return "unsigned
long"
Re: [PATCH v1 1/3] mm: pass meminit_context to __free_pages_core()
Quoted 14 lines · Andrew Morton
Quoted 12 lines · On Tue, 11 Jun 2024 12:06:56 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 10 lines · David Hildenbrand
In preparation for further changes, let's teach
__free_pages_core()about the differences of memory hotplug handling.Move the memory hotplug specific handling from
generic_online_page()to__free_pages_core(), useadjust_managed_page_count()on the memory hotplug path, and spell out why memory freed via memblock cannot currently useadjust_managed_page_count().Signed-off-by David Hildenbrand <[email protected]>---
Quoted 6 lines
Quoted 1 line
@Andrew, can you squash the following?
Sure.
I queued it against "mm: pass
meminit_contextto__free_pages_core()", not against
Ah yes, sorry. Thanks!
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v1 1/3] mm: pass meminit_context to __free_pages_core()
In reply to Andrew MortonQuoted 14 lines · Andrew Morton
Quoted 12 lines · On Tue, 11 Jun 2024 12:06:56 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 10 lines · David Hildenbrand
In preparation for further changes, let's teach
__free_pages_core()about the differences of memory hotplug handling.Move the memory hotplug specific handling from
generic_online_page()to__free_pages_core(), useadjust_managed_page_count()on the memory hotplug path, and spell out why memory freed via memblock cannot currently useadjust_managed_page_count().Signed-off-by David Hildenbrand <[email protected]>---
Quoted 10 lines
Quoted 1 line
@Andrew, can you squash the following?
Sure.
I queued it against "mm: pass
meminit_contextto__free_pages_core()", not againstQuoted 2 lines
Subject: [PATCH] fixup: mm/highmem: make
nr_free_highpages()return "unsigned
long"
Can you squash the following as well? (hopefully the last fixup, otherwise I might just resend a v2)
From 53c8c5834e638b2ae5e2a34fa7d49ce0dcf25192 Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Wed, 12 Jun 2024 20:31:07 +0200
Subject: [PATCH] fixup: mm: pass meminit_context to __free_pages_core()
Let's add the parameter name also in the declaration.
--- mm/internal.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/internal.h +1 -1
@@ -605,7 +605,7 @@ extern void __putback_isolated_page(struct page *page, unsigned int order,extern void memblock_free_pages (struct page *page,unsigned long pfn,unsigned int order);extern void __free_pages_core (struct page *page,unsigned int order, -enum meminit_context ); +enum meminit_context context);/* * This will have no effect, other than possibly generating a warning, if the
Signature · 7 lines
2.45.2 -- Cheers, David / dhildenb
Re: [PATCH v1 1/3] mm: pass meminit_context to __free_pages_core()
In reply to David HildenbrandQuoted 22 lines · David Hildenbrand
In preparation for further changes, let's teach
__free_pages_core()about the differences of memory hotplug handling.Move the memory hotplug specific handling from
generic_online_page()to__free_pages_core(), useadjust_managed_page_count()on the memory hotplug path, and spell out why memory freed via memblock cannot currently useadjust_managed_page_count().Signed-off-by David Hildenbrand <[email protected]>--- mm/internal.h | 3 ++- mm/kmsan/init.c | 2 +- mm/memory_hotplug.c | 9 +-------- mm/mm_init.c | 4 ++-- mm/page_alloc.c | 17 +++++++++++++++-- 5 files changed, 21 insertions(+), 14 deletions(-)mm/internal.h +0
-0
mm/internal.h extern void __putback_isolated_page(struct page *page, unsigned int order, · line 608
Quoted 6 lines
mm/internal.h +2
-1@@ -604,7 +604,8 @@ extern void __putback_isolated_page(struct page *page, unsigned int order, int mt);extern void memblock_free_pages (struct page *page,unsigned long pfn,unsigned int order); -extern void __free_pages_core (struct page *page,unsigned int order); +extern void __free_pages_core (struct page *page,unsigned int order, +enum meminit_context );
Shouldn't the above be
enum meminit_context context);
mm/internal.h extern void __putback_isolated_page(struct page *page, unsigned int order, · line 611
Quoted 3 lines
mm/internal.h +0
-0/* * This will have no effect, other than possibly generating a warning, if the
Thanks.
Tim
Re: [PATCH v1 1/3] mm: pass meminit_context to __free_pages_core()
Quoted 31 lines · Tim Chen
Quoted 22 lines · David Hildenbrand
In preparation for further changes, let's teach
__free_pages_core()about the differences of memory hotplug handling.Move the memory hotplug specific handling from
generic_online_page()to__free_pages_core(), useadjust_managed_page_count()on the memory hotplug path, and spell out why memory freed via memblock cannot currently useadjust_managed_page_count().Signed-off-by David Hildenbrand <[email protected]>--- mm/internal.h | 3 ++- mm/kmsan/init.c | 2 +- mm/memory_hotplug.c | 9 +-------- mm/mm_init.c | 4 ++-- mm/page_alloc.c | 17 +++++++++++++++-- 5 files changed, 21 insertions(+), 14 deletions(-)mm/internal.h +0
-0mm/internal.h extern void __putback_isolated_page(struct page *page, unsigned int order, · line 608
Quoted 6 lines
mm/internal.h +2
-1@@ -604,7 +604,8 @@ extern void __putback_isolated_page(struct page *page, unsigned int order, int mt);extern void memblock_free_pages (struct page *page,unsigned long pfn,unsigned int order); -extern void __free_pages_core (struct page *page,unsigned int order); +extern void __free_pages_core (struct page *page,unsigned int order, +enum meminit_context );Shouldn't the above be
Quoted 1 line
enum meminit_context context);
Although C allows parameters without names in declarations, this was unintended.
Thanks!
Signature · 3 lines
Cheers, David / dhildenb
PATCH 2/3 mm/memory_hotplug: initialize memmap of !ZONE_DEVICE with PageOffline() instead of PageReserved()
In reply to David HildenbrandWe currently initialize the memmap such that PG_reserved is set and the refcount of the page is 1. In virtio-mem code, we have to manually clear that PG_reserved flag to make memory offlining with partially hotplugged memory blocks possible: has_unmovable_pages() would otherwise bail out on such pages.
We want to avoid PG_reserved where possible and move to typed pages instead. Further, we want to further enlighten memory offlining code aboutPG_offline: offline pages in an online memory section. One example is handling managed page count adjustments in a cleaner way during memory offlining.
So let's initialize the pages with PG_offline instead of PG_reserved. generic_online_page()->__free_pages_core() will now clear that flag before handing that memory to the buddy.
Note that the page refcount is still 1 and would forbid offlining of such memory except when special care is take during GOING_OFFLINE as currently only implemented by virtio-mem.
With this change, we can now get non-PageReserved() pages in the XEN balloon list. From what I can tell, that can already happen via decrease_reservation(), so that should be fine.
HV-balloon should not really observe a change: partial online memory blocks still cannot get surprise-offlined, because the refcount of these PageOffline() pages is 1.
Update virtio-mem, HV-balloon and XEN-balloon code to be aware that hotplugged pages are now PageOffline() instead of PageReserved() before they are handed over to the buddy.
We'll leave the ZONE_DEVICE case alone for now.
--- drivers/hv/hv_balloon.c | 5 ++--- drivers/virtio/virtio_mem.c | 18 ++++++++++++------ drivers/xen/balloon.c | 9 +++++++-- include/linux/page-flags.h | 12 +++++------- mm/memory_hotplug.c | 16 ++++++++++------ mm/mm_init.c | 10 ++++++++-- mm/page_alloc.c | 32 +++++++++++++++++++++++--------- 7 files changed, 67 insertions(+), 35 deletions(-)
drivers/hv/hv_balloon.c +2 -3
@@ -693,9 +693,8 @@ static void hv_page_online_one(struct hv_hotadd_state *has, struct page *pg)if (!PageOffline (pg))__SetPageOffline (pg);return ; - } -if (PageOffline (pg)) -__ClearPageOffline (pg); + }else if (!PageOffline (pg)) +return ;/* This frame is currently backed; online the page. */ generic_online_page (pg,0 );
drivers/virtio/virtio_mem.c +12 -6
@@ -1146,12 +1146,16 @@ static void virtio_mem_set_fake_offline(unsigned long pfn,for (; nr_pages--; pfn++) {struct page *page =pfn_to_page (pfn); -__SetPageOffline (page); -if (!onlined) { +if (!onlined) +/* +* Pages that have not been onlined yet were initialized +* to PageOffline(). Remember that we have to route them +* through generic_online_page(). +*/ SetPageDirty (page); -/* FIXME: remove after cleanups */ -ClearPageReserved (page); - } +else +__SetPageOffline (page); +VM_WARN_ON_ONCE (!PageOffline (page)); }page_offline_end (); }
@@ -1166,9 +1170,11 @@ static void virtio_mem_clear_fake_offline(unsigned long pfn,for (; nr_pages--; pfn++) {struct page *page =pfn_to_page (pfn); -__ClearPageOffline (page);if (!onlined) +/* generic_online_page() will clear PageOffline(). */ ClearPageDirty (page); +else +__ClearPageOffline (page); } }
drivers/xen/balloon.c +7 -2
@@ -146,7 +146,8 @@ static DECLARE_WAIT_QUEUE_HEAD(balloon_wq);/* balloon_append: add the given page to the balloon. */ static void balloon_append (struct page *page) { -__SetPageOffline (page); +if (!PageOffline (page)) +__SetPageOffline (page);/* Lowmem is re-populated first, so highmem pages go at list tail. */ if (PageHighMem (page)) {
@@ -412,7 +413,11 @@ static enum bp_state increase_reservation(unsigned long nr_pages)xenmem_reservation_va_mapping_update (1 , &page, &frame_list[i]); -/* Relinquish the page back to the allocator. */ +/* +* Relinquish the page back to the allocator. Note that +* some pages, including ones added via xen_online_page(), might +* not be marked reserved; free_reserved_page() will handle that. +*/ free_reserved_page (page); }
include/linux/page-flags.h +5 -7
@@ -30,16 +30,11 @@* - Pages falling into physical memory gaps - not IORESOURCE_SYSRAM. Trying * to read/write these pages might end badly. Don't touch! * - The zero page(s) -* - Pages not added to the page allocator when onlining a section because -* they were excluded via the online_page_callback() or because they are -* PG_hwpoison. * - Pages allocated in the context of kexec/kdump (loaded kernel image, * control pages, vmcoreinfo) * - MMIO/DMA pages. Some architectures don't allow to ioremap pages that are * not marked PG_reserved (as they might be in use by somebody else who does * not respect the caching strategy). -* - Pages part of an offline section (struct pages of offline sections should -* not be trusted as they will be initialized when first onlined). * - MCA pages on ia64 * - Pages holding CPU notes for POWER Firmware Assisted Dump * - Device memory (e.g. PMEM, DAX, HMM)
@@ -1021,6 +1016,10 @@ PAGE_TYPE_OPS(Buddy, buddy, buddy)* The content of these pages is effectively stale. Such pages should not * be touched (read/write/dump/save) except by their owner. * +* When a memory block gets onlined, all pages are initialized with a +* refcount of 1 and PageOffline(). generic_online_page() will +* take care of clearing PageOffline(). +* * If a driver wants to allow to offline unmovable PageOffline() pages without * putting them back to the buddy, it can do so via the memory notifier by * decrementing the reference count in MEM_GOING_OFFLINE and incrementing the
@@ -1028,8 +1027,7 @@ PAGE_TYPE_OPS(Buddy, buddy, buddy)* pages (now with a reference count of zero) are treated like free pages, * allowing the containing memory block to get offlined. A driver that * relies on this feature is aware that re-onlining the memory block will -* require to re-set the pages PageOffline() and not giving them to the -* buddy via online_page_callback_t. +* require not giving them to the buddy via generic_online_page(). * * There are drivers that mark a page PageOffline() and expect there won't be * any further access to page content. PFN walkers that read content of random
mm/memory_hotplug.c +10 -6
@@ -734,7 +734,7 @@ static inline void section_taint_zone_device(unsigned long pfn)/* * Associate the pfn range with the given zone, initializing the memmaps * and resizing the pgdat/zone data to span the added pages. After this -* call, all affected pages are PG_reserved. +* call, all affected pages are PageOffline(). * * All aligned pageblocks are initialized to the specified migratetype * (usually MIGRATE_MOVABLE). Besides setting the migratetype, no related
@@ -1100,8 +1100,12 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,move_pfn_range_to_zone (zone, pfn, nr_pages,NULL ,MIGRATE_UNMOVABLE ); -for (i =0 ; i < nr_pages; i++) -SetPageVmemmapSelfHosted (pfn_to_page (pfn + i)); +for (i =0 ; i < nr_pages; i++) { +struct page *page =pfn_to_page (pfn + i); + +__ClearPageOffline (page); +SetPageVmemmapSelfHosted (page); + }/* * It might be that the vmemmap_pages fully span sections. If that is
@@ -1959,9 +1963,9 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages,* Don't allow to offline memory blocks that contain holes. * Consequently, memory blocks with holes can never get onlined * via the hotplug path - online_pages() - as hotplugged memory has -* no holes. This way, we e.g., don't have to worry about marking -* memory holes PG_reserved, don't need pfn_valid() checks, and can -* avoid using walk_system_ram_range() later. +* no holes. This way, we don't have to worry about memory holes, +* don't need pfn_valid() checks, and can avoid using +* walk_system_ram_range() later. */ walk_system_ram_range (start_pfn, nr_pages, &system_ram_pages, count_system_ram_pages_cb);
mm/mm_init.c +8 -2
@@ -892,8 +892,14 @@ void __meminit memmap_init_range(unsigned long size, int nid, unsigned long zone page =pfn_to_page (pfn);__init_single_page (page, pfn, zone, nid); -if (context ==MEMINIT_HOTPLUG ) -__SetPageReserved (page); +if (context ==MEMINIT_HOTPLUG ) { +#ifdef CONFIG_ZONE_DEVICE +if (zone ==ZONE_DEVICE ) +__SetPageReserved (page); + else +#endif +__SetPageOffline (page); + }/* * Usually, we want to mark the pageblock MIGRATE_MOVABLE,
mm/page_alloc.c +23 -9
@@ -1225,18 +1225,23 @@ void __free_pages_core(struct page *page, unsigned int order,* When initializing the memmap, __init_single_page() sets the refcount * of all pages to 1 ("allocated"/"not free"). We have to set the * refcount of all involved pages to 0. +* +* Note that hotplugged memory pages are initialized to PageOffline(). +* Pages freed from memblock might be marked as reserved. */ -prefetchw (p); -for (loop =0 ; loop < (nr_pages -1 ); loop++, p++) { -prefetchw (p +1 ); -__ClearPageReserved (p); -set_page_count (p,0 ); - } -__ClearPageReserved (p); -set_page_count (p,0 ); -if (IS_ENABLED (CONFIG_MEMORY_HOTPLUG ) &&unlikely (context ==MEMINIT_HOTPLUG )) { +prefetchw (p); +for (loop =0 ; loop < (nr_pages -1 ); loop++, p++) { +prefetchw (p +1 ); +VM_WARN_ON_ONCE (PageReserved (p)); +__ClearPageOffline (p); +set_page_count (p,0 ); + } +VM_WARN_ON_ONCE (PageReserved (p)); +__ClearPageOffline (p); +set_page_count (p,0 ); +/* * Freeing the page with debug_pagealloc enabled will try to * unmap it; some archs don't like double-unmappings, so
@@ -1245,6 +1250,15 @@ void __free_pages_core(struct page *page, unsigned int order,debug_pagealloc_map_pages (page, nr_pages);adjust_managed_page_count (page, nr_pages); }else { +prefetchw (p); +for (loop =0 ; loop < (nr_pages -1 ); loop++, p++) { +prefetchw (p +1 ); +__ClearPageReserved (p); +set_page_count (p,0 ); + } +__ClearPageReserved (p); +set_page_count (p,0 ); +/* memblock adjusts totalram_pages() ahead of time. */ atomic_long_add (nr_pages, &page_zone (page)->managed_pages); }
Signature · 1 line
2.45.1
Oscar Salvador
maintainerRe: [PATCH v1 2/3] mm/memory_hotplug: initialize memmap of !ZONE_DEVICE with PageOffline() instead of PageReserved()
Quoted 50 lines · David Hildenbrand
We currently initialize the memmap such that
PG_reservedis set and the refcount of the page is 1. In virtio-mem code, we have to manually clear thatPG_reservedflag to make memory offlining with partially hotplugged memory blocks possible:has_unmovable_pages()would otherwise bail out on such pages.We want to avoid
PG_reservedwhere possible and move to typed pages instead. Further, we want to further enlighten memory offlining code aboutPG_offline: offline pages in an online memory section. One example is handling managed page count adjustments in a cleaner way during memory offlining.So let's initialize the pages with
PG_offlineinstead ofPG_reserved. generic_online_page()->__free_pages_core() will now clear that flag before handing that memory to the buddy.Note that the page refcount is still 1 and would forbid offlining of such memory except when special care is take during
GOING_OFFLINEas currently only implemented by virtio-mem.With this change, we can now get non-PageReserved() pages in the XEN balloon list. From what I can tell, that can already happen via
decrease_reservation(), so that should be fine.HV-balloon should not really observe a change: partial online memory blocks still cannot get surprise-offlined, because the refcount of these
PageOffline()pages is 1.Update virtio-mem, HV-balloon and XEN-balloon code to be aware that hotplugged pages are now
PageOffline()instead ofPageReserved()before they are handed over to the buddy.We'll leave the
ZONE_DEVICEcase alone for now.Signed-off-by David Hildenbrand <[email protected]>mm/memory_hotplug.c +1
-1@@ -734,7 +734,7 @@ static inline void section_taint_zone_device(unsigned long pfn)/* * Associate the pfn range with the given zone, initializing the memmaps * and resizing the pgdat/zone data to span the added pages. After this -* call, all affected pages are PG_reserved. +* call, all affected pages are PageOffline(). * * All aligned pageblocks are initialized to the specified migratetype * (usually MIGRATE_MOVABLE). Besides setting the migratetype, no related
mm/memory_hotplug.c int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages, · line 1107
Quoted 10 lines
mm/memory_hotplug.c +5
-2@@ -1100,8 +1100,12 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,move_pfn_range_to_zone (zone, pfn, nr_pages,NULL ,MIGRATE_UNMOVABLE ); -for (i =0 ; i < nr_pages; i++) -SetPageVmemmapSelfHosted (pfn_to_page (pfn + i)); +for (i =0 ; i < nr_pages; i++) { +struct page *page =pfn_to_page (pfn + i); + +__ClearPageOffline (page); +SetPageVmemmapSelfHosted (page);
So, refresh my memory here please.
AFAIR, those VmemmapSelfHosted pages were marked Reserved before, but now, memmap_init_range() will not mark them reserved anymore.
I do not think that is ok? I am worried about walkers getting this wrong.
We usually skip PageReserved pages in walkers because are pages we cannot deal with for those purposes, but with this change, we will leak PageVmemmapSelfHosted, and I am not sure whether are ready for that.
Moreover, boot memmap pages are marked as PageReserved, which would be now inconsistent with those added during hotplug operations.
All in all, I feel uneasy about this change.
Signature · 2 lines
Oscar Salvador SUSE Labs
Re: [PATCH v1 2/3] mm/memory_hotplug: initialize memmap of !ZONE_DEVICE with PageOffline() instead of PageReserved()
Quoted 54 lines · Oscar Salvador
Quoted 53 lines · David Hildenbrand
We currently initialize the memmap such that
PG_reservedis set and the refcount of the page is 1. In virtio-mem code, we have to manually clear thatPG_reservedflag to make memory offlining with partially hotplugged memory blocks possible:has_unmovable_pages()would otherwise bail out on such pages.We want to avoid
PG_reservedwhere possible and move to typed pages instead. Further, we want to further enlighten memory offlining code aboutPG_offline: offline pages in an online memory section. One example is handling managed page count adjustments in a cleaner way during memory offlining.So let's initialize the pages with
PG_offlineinstead ofPG_reserved. generic_online_page()->__free_pages_core() will now clear that flag before handing that memory to the buddy.Note that the page refcount is still 1 and would forbid offlining of such memory except when special care is take during
GOING_OFFLINEas currently only implemented by virtio-mem.With this change, we can now get non-PageReserved() pages in the XEN balloon list. From what I can tell, that can already happen via
decrease_reservation(), so that should be fine.HV-balloon should not really observe a change: partial online memory blocks still cannot get surprise-offlined, because the refcount of these
PageOffline()pages is 1.Update virtio-mem, HV-balloon and XEN-balloon code to be aware that hotplugged pages are now
PageOffline()instead ofPageReserved()before they are handed over to the buddy.We'll leave the
ZONE_DEVICEcase alone for now.Signed-off-by David Hildenbrand <[email protected]>mm/memory_hotplug.c +1
-1@@ -734,7 +734,7 @@ static inline void section_taint_zone_device(unsigned long pfn)/* * Associate the pfn range with the given zone, initializing the memmaps * and resizing the pgdat/zone data to span the added pages. After this -* call, all affected pages are PG_reserved. +* call, all affected pages are PageOffline(). * * All aligned pageblocks are initialized to the specified migratetype * (usually MIGRATE_MOVABLE). Besides setting the migratetype, no related @@ -1100,8 +1100,12 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,move_pfn_range_to_zone (zone, pfn, nr_pages,NULL ,MIGRATE_UNMOVABLE );
Quoted 11 lines
mm/memory_hotplug.c int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages, · line 1107
Quoted 7 lines
mm/memory_hotplug.c +5
-2-for (i =0 ; i < nr_pages; i++) -SetPageVmemmapSelfHosted (pfn_to_page (pfn + i)); +for (i =0 ; i < nr_pages; i++) { +struct page *page =pfn_to_page (pfn + i); + +__ClearPageOffline (page); +SetPageVmemmapSelfHosted (page);So, refresh my memory here please.
AFAIR, those VmemmapSelfHosted pages were marked Reserved before, but now,memmap_init_range()will not mark them reserved anymore.
Correct.
Quoted 5 lines
I do not think that is ok? I am worried about walkers getting this wrong.
We usually skip PageReserved pages in walkers because are pages we cannot deal with for those purposes, but with this change, we will leak PageVmemmapSelfHosted, and I am not sure whether are ready for that.
There are fortunately not that many left.
I'd even say marking them (vmemmap) reserved is more wrong than right: note that ordinary vmemmap pages after memory hotplug are not reserved! Only bootmem should be reserved.
Let's take at the relevant core-mm ones (arch stuff is mostly just for MMIO remapping)
fs/proc/task_mmu.c: if (PageReserved(page)) fs/proc/task_mmu.c: if (PageReserved(page))
-> If we find vmemmap pages mapped into user space we already messed up
seriously
kernel/power/snapshot.c: if (PageReserved(page) || kernel/power/snapshot.c: if (PageReserved(page)
-> There should be no change (saveable_page() would still allow saving
them, highmem does not apply)
mm/hugetlb_vmemmap.c: if (!PageReserved(head)) mm/hugetlb_vmemmap.c: if (PageReserved(page))
-> Wants to identify bootmem, but we exclude these
PageVmemmapSelfHosted() on the splitting part already properly
mm/page_alloc.c: VM_WARN_ON_ONCE(PageReserved(p)); mm/page_alloc.c: if (PageReserved(page))
-> pfn_range_valid_contig() would scan them, just like for ordinary
vmemmap pages during hotplug. We'll simply fail isolating/migrating them similarly (like any unmovable allocations) later
mm/page_ext.c: BUG_ON(PageReserved(page));
-> free_page_ext handling, does not apply
mm/page_isolation.c: if (PageReserved(page))
-> has_unmovable_pages() should still detect them as unmovable (e.g.,
neither movable nor LRU).
mm/page_owner.c: if (PageReserved(page)) mm/page_owner.c: if (PageReserved(page))
-> Simply page_ext_get() will return NULL instead and we'll similarly
skip them
mm/sparse.c: if (!PageReserved(virt_to_page(ms->usage))) {
-> Detecting boot memory for ms->usage allocation, does not apply to
vmemmap.
virt/kvm/kvm_main.c: if (!PageReserved(page)) virt/kvm/kvm_main.c: return !PageReserved(page);
-> For MMIO remapping purposes, does not apply to vmemmap
Quoted 2 lines
Moreover, boot memmap pages are marked as PageReserved, which would be now inconsistent with those added during hotplug operations.
Just like vmemmap pages allocated dynamically during memory hotplug. Now, really only bootmem-ones are PageReserved.
Quoted 1 line
All in all, I feel uneasy about this change.
I really don't want to mark these pages here PageReserved for the sake of it.
Any PageReserved user that I am missing, or why we should handle these vmemmap pages differently than the ones allocated during ordinary memory hotplug?
In the future, we might want to consider using a dedicated page type for them, so we can stop using a bit that doesn't allow to reliably identify them. (we should mark all vmemmap with that type then)
Thanks!
Signature · 3 lines
Cheers, David / dhildenb
Oscar Salvador
maintainerRe: [PATCH v1 2/3] mm/memory_hotplug: initialize memmap of !ZONE_DEVICE with PageOffline() instead of PageReserved()
Quoted 5 lines · David Hildenbrand
There are fortunately not that many left.
I'd even say marking them (vmemmap) reserved is more wrong than right: note that ordinary vmemmap pages after memory hotplug are not reserved! Only bootmem should be reserved.
Ok, that is a very good point that I missed.
I thought that hotplugged-vmemmap pages (not selfhosted) were marked as Reserved, that is why I thought this would be inconsistent. But then, if that is the case, I think we are safe as kernel can already encounter vmemmap pages that are not reserved and it deals with them somehow.
Quoted 3 lines
Let's take at the relevant core-mm ones (arch stuff is mostly just for MMIO remapping)
...
Quoted 3 lines
Any PageReserved user that I am missing, or why we should handle these vmemmap pages differently than the ones allocated during ordinary memory hotplug?
No, I cannot think of a reason why normal vmemmap pages should behave different than self-hosted.
I was also confused because I thought that after this change pfn_to_online_page() would be different for self-hosted vmemmap pages, because I thought that somehow we relied on PageOffline(), but it is not the case.
Quoted 3 lines
In the future, we might want to consider using a dedicated page type for them, so we can stop using a bit that doesn't allow to reliably identify them. (we should mark all vmemmap with that type then)
Yes, a all-vmemmap pages type would be a good thing, so we do not have to special case.
Just one last thing.
Now self-hosted vmemmap pages will have the PageOffline cleared, and that will still remain after the memory-block they belong to has gone offline, which is ok because those vmemmap pages lay around until the chunk of memory gets removed.
Ok, just wanted to convince myself that there will no be surprises.
Thanks David for claryfing.
Signature · 2 lines
Oscar Salvador SUSE Labs
Re: [PATCH v1 2/3] mm/memory_hotplug: initialize memmap of !ZONE_DEVICE with PageOffline() instead of PageReserved()
Quoted 22 lines · Oscar Salvador
Quoted 5 lines · David Hildenbrand
There are fortunately not that many left.
I'd even say marking them (vmemmap) reserved is more wrong than right: note that ordinary vmemmap pages after memory hotplug are not reserved! Only bootmem should be reserved.
Ok, that is a very good point that I missed.
I thought that hotplugged-vmemmap pages (not selfhosted) were marked as Reserved, that is why I thought this would be inconsistent. But then, if that is the case, I think we are safe as kernel can already encounter vmemmap pages that are not reserved and it deals with them somehow.Quoted 3 lines
Let's take at the relevant core-mm ones (arch stuff is mostly just for MMIO remapping)
...
Quoted 3 lines
Any PageReserved user that I am missing, or why we should handle these vmemmap pages differently than the ones allocated during ordinary memory hotplug?
Quoted 7 lines
No, I cannot think of a reason why normal vmemmap pages should behave different than self-hosted.
I was also confused because I thought that after this change
pfn_to_online_page()would be different for self-hosted vmemmap pages, because I thought that somehow we relied onPageOffline(), but it is not the case.
Fortunately not :) PageFakeOffline() or PageLogicallyOffline() might be clearer, but I don't quite like these names. If you have a good idea, please let me know.
Quoted 5 lines
Quoted 4 lines
In the future, we might want to consider using a dedicated page type for them, so we can stop using a bit that doesn't allow to reliably identify them. (we should mark all vmemmap with that type then)
Quoted 8 lines
Yes, a all-vmemmap pages type would be a good thing, so we do not have to special case.
Just one last thing.
Now self-hosted vmemmap pages will have the PageOffline cleared, and that will still remain after the memory-block they belong to has gone offline, which is ok because those vmemmap pages lay around until the chunk of memory gets removed.
Yes, and that memmap might even get poisoned in debug kernels to catch any wrong access.
Quoted 4 lines
Ok, just wanted to convince myself that there will no be surprises.
Thanks David for claryfing.
Thanks for the review and raising that. I'll add more details to the patch description!
Signature · 3 lines
Cheers, David / dhildenb
Re: [PATCH v1 2/3] mm/memory_hotplug: initialize memmap of !ZONE_DEVICE with PageOffline() instead of PageReserved()
In reply to David HildenbrandQuoted 28 lines · David Hildenbrand
We currently initialize the memmap such that
PG_reservedis set and the refcount of the page is 1. In virtio-mem code, we have to manually clear thatPG_reservedflag to make memory offlining with partially hotplugged memory blocks possible:has_unmovable_pages()would otherwise bail out on such pages.We want to avoid
PG_reservedwhere possible and move to typed pages instead. Further, we want to further enlighten memory offlining code aboutPG_offline: offline pages in an online memory section. One example is handling managed page count adjustments in a cleaner way during memory offlining.So let's initialize the pages with
PG_offlineinstead ofPG_reserved. generic_online_page()->__free_pages_core() will now clear that flag before handing that memory to the buddy.Note that the page refcount is still 1 and would forbid offlining of such memory except when special care is take during
GOING_OFFLINEas currently only implemented by virtio-mem.With this change, we can now get non-PageReserved() pages in the XEN balloon list. From what I can tell, that can already happen via
decrease_reservation(), so that should be fine.HV-balloon should not really observe a change: partial online memory blocks still cannot get surprise-offlined, because the refcount of these
PageOffline()pages is 1.
Quoted 7 lines
Update virtio-mem, HV-balloon and XEN-balloon code to be aware that hotplugged pages are now
PageOffline()instead ofPageReserved()before they are handed over to the buddy.We'll leave the
ZONE_DEVICEcase alone for now.Signed-off-by David Hildenbrand <[email protected]>
memory-hotplug bits
Signature · 2 lines
Oscar Salvador SUSE Labs
Re: [PATCH v1 2/3] mm/memory_hotplug: initialize memmap of !ZONE_DEVICE with PageOffline() instead of PageReserved()
In reply to David HildenbrandQuoted 28 lines · David Hildenbrand
We currently initialize the memmap such that
PG_reservedis set and the refcount of the page is 1. In virtio-mem code, we have to manually clear thatPG_reservedflag to make memory offlining with partially hotplugged memory blocks possible:has_unmovable_pages()would otherwise bail out on such pages.We want to avoid
PG_reservedwhere possible and move to typed pages instead. Further, we want to further enlighten memory offlining code aboutPG_offline: offline pages in an online memory section. One example is handling managed page count adjustments in a cleaner way during memory offlining.So let's initialize the pages with
PG_offlineinstead ofPG_reserved. generic_online_page()->__free_pages_core() will now clear that flag before handing that memory to the buddy.Note that the page refcount is still 1 and would forbid offlining of such memory except when special care is take during
GOING_OFFLINEas currently only implemented by virtio-mem.With this change, we can now get non-PageReserved() pages in the XEN balloon list. From what I can tell, that can already happen via
decrease_reservation(), so that should be fine.HV-balloon should not really observe a change: partial online memory blocks still cannot get surprise-offlined, because the refcount of these
PageOffline()pages is 1.
Quoted 6 lines
Update virtio-mem, HV-balloon and XEN-balloon code to be aware that hotplugged pages are now
PageOffline()instead ofPageReserved()before they are handed over to the buddy.We'll leave the
ZONE_DEVICEcase alone for now.
@Andrew, can we add here:
"Note that self-hosted vmemmap pages will no longer be marked as reserved. This matches ordinary vmemmap pages allocated from the buddy during memory hotplug. Now, really only vmemmap pages allocated from memblock during early boot will be marked reserved. Existing PageReserved() checks seem to be handling all relevant cases correctly even after this change."
Signature · 3 lines
Cheers, David / dhildenb
Andrew Morton
maintainerRe: [PATCH v1 2/3] mm/memory_hotplug: initialize memmap of !ZONE_DEVICE with PageOffline() instead of PageReserved()
Quoted 11 lines · On Tue, 11 Jun 2024 11:42:56 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 2 lines
We'll leave the
ZONE_DEVICEcase alone for now.@Andrew, can we add here:
"Note that self-hosted vmemmap pages will no longer be marked as reserved. This matches ordinary vmemmap pages allocated from the buddy during memory hotplug. Now, really only vmemmap pages allocated from memblock during early boot will be marked reserved. Existing
PageReserved()checks seem to be handling all relevant cases correctly even after this change."
Done, thanks.
PATCH 3/3 mm/memory_hotplug: skip adjust_managed_page_count() for PageOffline() pages when offlining
In reply to David HildenbrandWe currently have a hack for virtio-mem in place to handle memory offlining with PageOffline pages for which we already adjusted the managed page count.
Let's enlighten memory offlining code so we can get rid of that hack, and document the situation.
--- drivers/virtio/virtio_mem.c | 11 ++--------- include/linux/memory_hotplug.h | 4 ++-- include/linux/page-flags.h | 8 ++++++-- mm/memory_hotplug.c | 6 +++--- mm/page_alloc.c | 12 ++++++++++-- 5 files changed, 23 insertions(+), 18 deletions(-)
drivers/virtio/virtio_mem.c +2 -9
@@ -1269,12 +1269,6 @@ static void virtio_mem_fake_offline_going_offline(unsigned long pfn,struct page *page;unsigned long i; -/* -* Drop our reference to the pages so the memory can get offlined -* and add the unplugged pages to the managed page counters (so -* offlining code can correctly subtract them again). -*/ -adjust_managed_page_count (pfn_to_page (pfn), nr_pages);/* Drop our reference to the pages so the memory can get offlined. */ for (i =0 ; i < nr_pages; i++) { page =pfn_to_page (pfn + i);
@@ -1293,10 +1287,9 @@ static void virtio_mem_fake_offline_cancel_offline(unsigned long pfn,unsigned long i;/* -* Get the reference we dropped when going offline and subtract the -* unplugged pages from the managed page counters. +* Get the reference again that we dropped via page_ref_dec_and_test() +* when going offline. */ -adjust_managed_page_count (pfn_to_page (pfn), -nr_pages);for (i =0 ; i < nr_pages; i++)page_ref_inc (pfn_to_page (pfn + i)); }
include/linux/memory_hotplug.h +2 -2
@@ -175,8 +175,8 @@ extern int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,extern void mhp_deinit_memmap_on_memory (unsigned long pfn,unsigned long nr_pages);extern int online_pages (unsigned long pfn,unsigned long nr_pages,struct zone *zone,struct memory_group *group); -extern void __offline_isolated_pages (unsigned long start_pfn, -unsigned long end_pfn); +extern unsigned long __offline_isolated_pages (unsigned long start_pfn, +unsigned long end_pfn);typedef void (*online_page_callback_t )(struct page *page,unsigned int order);
include/linux/page-flags.h +6 -2
@@ -1024,11 +1024,15 @@ PAGE_TYPE_OPS(Buddy, buddy, buddy)* putting them back to the buddy, it can do so via the memory notifier by * decrementing the reference count in MEM_GOING_OFFLINE and incrementing the * reference count in MEM_CANCEL_OFFLINE. When offlining, the PageOffline() -* pages (now with a reference count of zero) are treated like free pages, -* allowing the containing memory block to get offlined. A driver that +* pages (now with a reference count of zero) are treated like free (unmanaged) +* pages, allowing the containing memory block to get offlined. A driver that * relies on this feature is aware that re-onlining the memory block will * require not giving them to the buddy via generic_online_page(). * +* Memory offlining code will not adjust the managed page count for any +* PageOffline() pages, treating them like they were never exposed to the +* buddy using generic_online_page(). +* * There are drivers that mark a page PageOffline() and expect there won't be * any further access to page content. PFN walkers that read content of random * pages should check PageOffline() and synchronize with such drivers using
mm/memory_hotplug.c +3 -3
@@ -1941,7 +1941,7 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages, struct zone *zone, struct memory_group *group) {const unsigned long end_pfn = start_pfn + nr_pages; -unsigned long pfn, system_ram_pages =0 ; +unsigned long pfn, managed_pages, system_ram_pages =0 ;const int node =zone_to_nid (zone);unsigned long flags;struct memory_notify arg;
@@ -2062,7 +2062,7 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages, }while (ret);/* Mark all sections offline and remove free pages from the buddy. */ -__offline_isolated_pages (start_pfn, end_pfn); + managed_pages =__offline_isolated_pages (start_pfn, end_pfn);pr_debug ("Offlined Pages %ld\n" , nr_pages);/*
@@ -2078,7 +2078,7 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages,zone_pcp_enable (zone);/* removal success */ -adjust_managed_page_count (pfn_to_page (start_pfn), -nr_pages); +adjust_managed_page_count (pfn_to_page (start_pfn), -managed_pages);adjust_present_page_count (pfn_to_page (start_pfn), group, -nr_pages);/* reinitialise watermarks and update pcp limits */
mm/page_alloc.c +10 -2
@@ -6745,14 +6745,19 @@ void zone_pcp_reset(struct zone *zone)/* * All pages in the range must be in a single zone, must not contain holes, * must span full sections, and must be isolated before calling this function. +* +* Returns the number of managed (non-PageOffline()) pages in the range: the +* number of pages for which memory offlining code must adjust managed page +* counters using adjust_managed_page_count(). */ -void __offline_isolated_pages (unsigned long start_pfn,unsigned long end_pfn) +unsigned long __offline_isolated_pages (unsigned long start_pfn, +unsigned long end_pfn) { +unsigned long already_offline =0 , flags;unsigned long pfn = start_pfn;struct page *page;struct zone *zone;unsigned int order; -unsigned long flags;offline_mem_sections (pfn, end_pfn); zone =page_zone (pfn_to_page (pfn));
@@ -6774,6 +6779,7 @@ void __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn)if (PageOffline (page)) {BUG_ON (page_count (page));BUG_ON (PageBuddy (page)); + already_offline++; pfn++;continue ; }
@@ -6786,6 +6792,8 @@ void __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn) pfn += (1 << order); }spin_unlock_irqrestore (&zone->lock, flags); + +return end_pfn - start_pfn - already_offline; }#endif
Signature · 1 line
2.45.1
Re: [PATCH v1 3/3] mm/memory_hotplug: skip adjust_managed_page_count() for PageOffline() pages when offlining
Quoted 8 lines · David Hildenbrand
We currently have a hack for virtio-mem in place to handle memory offlining with PageOffline pages for which we already adjusted the managed page count.
Let's enlighten memory offlining code so we can get rid of that hack, and document the situation.
Signed-off-by David Hildenbrand <[email protected]>
Signature · 2 lines
Oscar Salvador SUSE Labs
Re: [PATCH v1 3/3] mm/memory_hotplug: skip adjust_managed_page_count() for PageOffline() pages when offlining
Quoted 5 lines · Oscar Salvador
Quoted 4 lines · David Hildenbrand
We currently have a hack for virtio-mem in place to handle memory offlining with PageOffline pages for which we already adjusted the managed page count.
Quoted 7 lines
Quoted 4 lines
Let's enlighten memory offlining code so we can get rid of that hack, and document the situation.
Signed-off-by David Hildenbrand <[email protected]>Acked-by Oscar Salvador <[email protected]>
Thanks for the review!
Signature · 3 lines
Cheers, David / dhildenb
Andrew Morton
maintainerafaict we're in decent state to move this series into mm-stable. I've tagged the following issues:
https://lkml.kernel.org/r/[email protected]
https://lkml.kernel.org/r/[email protected]
Have these been addressed and are we ready to send this series into the world?
Thanks.
Quoted 7 lines · Andrew Morton
afaict we're in decent state to move this series into mm-stable. I've tagged the following issues:
https://lkml.kernel.org/r/[email protected]
https://lkml.kernel.org/r/[email protected]Have these been addressed and are we ready to send this series into the world?
Yes, should all be addressed and this should be good to go.
Signature · 3 lines
Cheers, David / dhildenb