mm/kdump: allow to exclude pages that are logically offline
Subsystems (15)
cover mm/kdump: allow to exclude pages that are logically offline
Right now, pages inflated as part of a balloon driver will be dumped by dump tools like makedumpfile. While XEN is able to check in the crash kernel whether a certain pfn is actuall backed by memory in the hypervisor (see xen_oldmem_pfn_is_ram) and optimize this case, dumps of virtio-balloon, hv-balloon and VMWare balloon inflated memory will essentially result in zero pages getting allocated by the hypervisor and the dump getting filled with this data.
The allocation and reading of zero pages can directly be avoided if a dumping tool could know which pages only contain stale information not to be dumped.
Also for XEN, calling into the kernel and asking the hypervisor if a pfn is backed can be avoided if the duming tool would skip such pages right from the beginning.
Dumping tools have no idea whether a given page is part of a balloon driver and shall not be dumped. Esp. PG_reserved cannot be used for that purpose as all memory allocated during early boot is also PG_reserved, see discussion at [1]. So some other way of indication is required and a new page flag is frowned upon.
We have PG_balloon (MAPCOUNT value), which is essentially unused now. I suggest renaming it to something more generic (PG_offline) to mark pages as logically offline. This flag can than e.g. also be used by virtio-mem in the future to mark subsections as offline. Or by other code that wants to put pages logically offline (e.g. later maybe poisoned pages that shall no longer be used).
This series converts PG_balloon to PG_offline, allows dumping tools to query the value to detect such pages and marks pages in the hv-balloon and XEN balloon properly as PG_offline. Note that virtio-balloon already set pages to PG_balloon (and now PG_offline).
Please note that this is also helpful for a problem we were seeing under
Hyper-V: Dumping logically offline memory (pages kept fake offline while onlining a section via online_page_callback) would under some condicions result in a kernel panic when dumping them.
As I don't have access to neither XEN nor Hyper-V nor VMWare installations, this was only tested with the virtio-balloon and pages were properly skipped when dumping. I'll also attach the makedumpfile patch to this series.
[1] https://lkml.org/lkml/2018/7/20/566
RFC -> v1:
- Add "PM / Hibernate: use pfn_to_online_page()"
- Add "vmw_balloon: mark inflated pages PG_offline"
- "mm: convert PG_balloon to PG_offline"
-- After discussions, also rename the UAPI bit name (KPF_BALLOON -> KPF_OFFLINE)
David Hildenbrand (8):
mm: balloon: update comment about isolation/migration/compaction
mm: convert PG_balloon to PG_offline
kexec: export PG_offline to VMCOREINFO
xen/balloon: mark inflated pages PG_offline
hv_balloon: mark inflated pages PG_offline
vmw_balloon: mark inflated pages PG_offline
PM / Hibernate: use pfn_to_online_page()
PM / Hibernate: exclude all PageOffline() pages
Documentation/admin-guide/mm/pagemap.rst | 9 ++++--- drivers/hv/hv_balloon.c | 14 ++++++++-- drivers/misc/vmw_balloon.c | 32 ++++++++++++++++++++++ drivers/xen/balloon.c | 3 +++ fs/proc/page.c | 4 +-- include/linux/balloon_compaction.h | 34 +++++++++--------------- include/linux/page-flags.h | 11 +++++--- include/uapi/linux/kernel-page-flags.h | 2 +- kernel/crash_core.c | 2 ++ kernel/power/snapshot.c | 13 +++++---- tools/vm/page-types.c | 2 +- 11 files changed, 87 insertions(+), 39 deletions(-)
Signature · 1 line
2.17.2
PATCH 1/8 mm: balloon: update comment about isolation/migration/compaction
Commit b1123ea6d3b3 ("mm: balloon: use general non-lru movable page feature") reworked balloon handling to make use of the general non-lru movable page feature. The big comment block in balloon_compaction.h contains quite some outdated information. Let's fix this.
--- include/linux/balloon_compaction.h | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-)
include/linux/balloon_compaction.h +9 -17
@@ -4,15 +4,18 @@* * Common interface definitions for making balloon pages movable by compaction. * -* Despite being perfectly possible to perform ballooned pages migration, they -* make a special corner case to compaction scans because balloon pages are not -* enlisted at any LRU list like the other pages we do compact / migrate. +* Balloon page migration makes use of the general non-lru movable page +* feature. +* +* page->private is used to reference the responsible balloon device. +* page->mapping is used in context of non-lru page migration to reference +* the address space operations for page isolation/migration/compaction. * * As the page isolation scanning step a compaction thread does is a lockless * procedure (from a page standpoint), it might bring some racy situations while * performing balloon page compaction. In order to sort out these racy scenarios * and safely perform balloon's page compaction and migration we must, always, -* ensure following these three simple rules: +* ensure following these simple rules: * * i. when updating a balloon's page ->mapping element, strictly do it under * the following lock order, independently of the far superior
@@ -21,19 +24,8 @@* +--spin_lock_irq(&b_dev_info->pages_lock); * ... page->mapping updates here ... * -* ii. before isolating or dequeueing a balloon page from the balloon device -* pages list, the page reference counter must be raised by one and the -* extra refcount must be dropped when the page is enqueued back into -* the balloon device page list, thus a balloon page keeps its reference -* counter raised only while it is under our special handling; -* -* iii. after the lockless scan step have selected a potential balloon page for -* isolation, re-test the PageBalloon mark and the PagePrivate flag -* under the proper page lock, to ensure isolating a valid balloon page -* (not yet isolated, nor under release procedure) -* -* iv. isolation or dequeueing procedure must clear PagePrivate flag under -* page lock together with removing page from balloon device page list. +* ii. isolation or dequeueing procedure must remove the page from balloon +* device page list under b_dev_info->pages_lock. * * The functions provided by this interface are placed to help on coping with * the aforementioned balloon page corner case, as well as to ensure the simple
Signature · 1 line
2.17.2
Re: [PATCH v1 1/8] mm: balloon: update comment about isolation/migration/compaction
Quoted 11 lines · David Hildenbrand
Commit
b1123ea6d3b3("mm: balloon: use general non-lru movable page feature") reworked balloon handling to make use of the general non-lru movable page feature. The big comment block inballoon_compaction.hcontains quite some outdated information. Let's fix this.Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 55 lines
--- include/linux/balloon_compaction.h | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-)include/linux/balloon_compaction.h +9
-17@@ -4,15 +4,18 @@* * Common interface definitions for making balloon pages movable by compaction. * -* Despite being perfectly possible to perform ballooned pages migration, they -* make a special corner case to compaction scans because balloon pages are not -* enlisted at any LRU list like the other pages we do compact / migrate. +* Balloon page migration makes use of the general non-lru movable page +* feature. +* +* page->private is used to reference the responsible balloon device. +* page->mapping is used in context of non-lru page migration to reference +* the address space operations for page isolation/migration/compaction. * * As the page isolation scanning step a compaction thread does is a lockless * procedure (from a page standpoint), it might bring some racy situations while * performing balloon page compaction. In order to sort out these racy scenarios * and safely perform balloon's page compaction and migration we must, always, -* ensure following these three simple rules: +* ensure following these simple rules: * * i. when updating a balloon's page ->mapping element, strictly do it under * the following lock order, independently of the far superior @@ -21,19 +24,8 @@* +--spin_lock_irq(&b_dev_info->pages_lock); * ... page->mapping updates here ... * -* ii. before isolating or dequeueing a balloon page from the balloon device -* pages list, the page reference counter must be raised by one and the -* extra refcount must be dropped when the page is enqueued back into -* the balloon device page list, thus a balloon page keeps its reference -* counter raised only while it is under our special handling; -* -* iii. after the lockless scan step have selected a potential balloon page for -* isolation, re-test the PageBalloon mark and the PagePrivate flag -* under the proper page lock, to ensure isolating a valid balloon page -* (not yet isolated, nor under release procedure) -* -* iv. isolation or dequeueing procedure must clear PagePrivate flag under -* page lock together with removing page from balloon device page list. +* ii. isolation or dequeueing procedure must remove the page from balloon +* device page list under b_dev_info->pages_lock. * * The functions provided by this interface are placed to help on coping with * the aforementioned balloon page corner case, as well as to ensure the simple Signature · 1 line
2.17.2
Re: [PATCH v1 1/8] mm: balloon: update comment about isolation/migration/compaction
In reply to David HildenbrandQuoted 11 lines · David Hildenbrand
Commit
b1123ea6d3b3("mm: balloon: use general non-lru movable page feature") reworked balloon handling to make use of the general non-lru movable page feature. The big comment block inballoon_compaction.hcontains quite some outdated information. Let's fix this.Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 55 lines
--- include/linux/balloon_compaction.h | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-)include/linux/balloon_compaction.h +9
-17@@ -4,15 +4,18 @@* * Common interface definitions for making balloon pages movable by compaction. * -* Despite being perfectly possible to perform ballooned pages migration, they -* make a special corner case to compaction scans because balloon pages are not -* enlisted at any LRU list like the other pages we do compact / migrate. +* Balloon page migration makes use of the general non-lru movable page +* feature. +* +* page->private is used to reference the responsible balloon device. +* page->mapping is used in context of non-lru page migration to reference +* the address space operations for page isolation/migration/compaction. * * As the page isolation scanning step a compaction thread does is a lockless * procedure (from a page standpoint), it might bring some racy situations while * performing balloon page compaction. In order to sort out these racy scenarios * and safely perform balloon's page compaction and migration we must, always, -* ensure following these three simple rules: +* ensure following these simple rules: * * i. when updating a balloon's page ->mapping element, strictly do it under * the following lock order, independently of the far superior @@ -21,19 +24,8 @@* +--spin_lock_irq(&b_dev_info->pages_lock); * ... page->mapping updates here ... * -* ii. before isolating or dequeueing a balloon page from the balloon device -* pages list, the page reference counter must be raised by one and the -* extra refcount must be dropped when the page is enqueued back into -* the balloon device page list, thus a balloon page keeps its reference -* counter raised only while it is under our special handling; -* -* iii. after the lockless scan step have selected a potential balloon page for -* isolation, re-test the PageBalloon mark and the PagePrivate flag -* under the proper page lock, to ensure isolating a valid balloon page -* (not yet isolated, nor under release procedure) -* -* iv. isolation or dequeueing procedure must clear PagePrivate flag under -* page lock together with removing page from balloon device page list. +* ii. isolation or dequeueing procedure must remove the page from balloon +* device page list under b_dev_info->pages_lock. * * The functions provided by this interface are placed to help on coping with * the aforementioned balloon page corner case, as well as to ensure the simple Signature · 1 line
2.17.2
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 1/8 mm: balloon: update comment about isolation/migration/compaction
In reply to David HildenbrandCommit b1123ea6d3b3 ("mm: balloon: use general non-lru movable page feature") reworked balloon handling to make use of the general non-lru movable page feature. The big comment block in balloon_compaction.h contains quite some outdated information. Let's fix this.
--- include/linux/balloon_compaction.h | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-)
include/linux/balloon_compaction.h +9 -17
@@ -4,15 +4,18 @@* * Common interface definitions for making balloon pages movable by compaction. * -* Despite being perfectly possible to perform ballooned pages migration, they -* make a special corner case to compaction scans because balloon pages are not -* enlisted at any LRU list like the other pages we do compact / migrate. +* Balloon page migration makes use of the general non-lru movable page +* feature. +* +* page->private is used to reference the responsible balloon device. +* page->mapping is used in context of non-lru page migration to reference +* the address space operations for page isolation/migration/compaction. * * As the page isolation scanning step a compaction thread does is a lockless * procedure (from a page standpoint), it might bring some racy situations while * performing balloon page compaction. In order to sort out these racy scenarios * and safely perform balloon's page compaction and migration we must, always, -* ensure following these three simple rules: +* ensure following these simple rules: * * i. when updating a balloon's page ->mapping element, strictly do it under * the following lock order, independently of the far superior
@@ -21,19 +24,8 @@* +--spin_lock_irq(&b_dev_info->pages_lock); * ... page->mapping updates here ... * -* ii. before isolating or dequeueing a balloon page from the balloon device -* pages list, the page reference counter must be raised by one and the -* extra refcount must be dropped when the page is enqueued back into -* the balloon device page list, thus a balloon page keeps its reference -* counter raised only while it is under our special handling; -* -* iii. after the lockless scan step have selected a potential balloon page for -* isolation, re-test the PageBalloon mark and the PagePrivate flag -* under the proper page lock, to ensure isolating a valid balloon page -* (not yet isolated, nor under release procedure) -* -* iv. isolation or dequeueing procedure must clear PagePrivate flag under -* page lock together with removing page from balloon device page list. +* ii. isolation or dequeueing procedure must remove the page from balloon +* device page list under b_dev_info->pages_lock. * * The functions provided by this interface are placed to help on coping with * the aforementioned balloon page corner case, as well as to ensure the simple
Signature · 7 lines
2.17.2 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 2/8 mm: convert PG_balloon to PG_offline
In reply to David HildenbrandPG_balloon was introduced to implement page migration/compaction for pages inflated in virtio-balloon. Nowadays, it is only a marker that a page is part of virtio-balloon and therefore logically offline.
We also want to make use of this flag in other balloon drivers - for inflated pages or when onlining a section but keeping some pages offline (e.g. used right now by XEN and Hyper-V via set_online_page_callback()).
We are going to expose this flag to dump tools like makedumpfile. But instead of exposing PG_balloon, let's generalize the concept of marking pages as logically offline, so it can be reused for other purposes later on.
Rename PG_balloon to PG_offline. This is an indicator that the page is logically offline, the content stale and that it should not be touched (e.g. a hypervisor would have to allocate backing storage in order for the guest to dump an unused page). We can then e.g. exclude such pages from dumps.
We replace and reuse KPF_BALLOON (23), as this shouldn't really harm (and for now the semantics stay the same). In following patches, we will make use of this bit also in other balloon drivers. While at it, document PGTABLE.
--- Documentation/admin-guide/mm/pagemap.rst | 9 ++++++--- fs/proc/page.c | 4 ++-- include/linux/balloon_compaction.h | 8 ++++---- include/linux/page-flags.h | 11 +++++++---- include/uapi/linux/kernel-page-flags.h | 2 +- tools/vm/page-types.c | 2 +- 6 files changed, 21 insertions(+), 15 deletions(-)
Documentation/admin-guide/mm/pagemap.rst +6 -3
@@ -75,9 +75,10 @@ number of times a page is mapped. 20. NOPAGE 21. KSM 22. THP - 23. BALLOON + 23. OFFLINE 24. ZERO_PAGE 25. IDLE + 26. PGTABLE * ``/proc/kpagecgroup``. This file contains a 64-bit inode number of the memory cgroup each page is charged to, indexed by PFN. Only available when
@@ -118,8 +119,8 @@ Short descriptions to the page flags identical memory pages dynamically shared between one or more processes 22 - THP contiguous pages which construct transparent hugepages -23 - BALLOON - balloon compaction page +23 - OFFLINE + page is logically offline 24 - ZERO_PAGE zero page for pfn_zero or huge_zero page 25 - IDLE
@@ -128,6 +129,8 @@ Short descriptions to the page flags Note that this flag may be stale in case the page was accessed via a PTE. To make sure the flag is up-to-date one has to read ``/sys/kernel/mm/page_idle/bitmap`` first. +26 - PGTABLE + page is in use as a page table IO related page flags ---------------------
fs/proc/page.c +2 -2
@@ -152,8 +152,8 @@ u64 stable_page_flags(struct page *page)else if (page_count (page) ==0 &&is_free_buddy_page (page)) u |=1 <<KPF_BUDDY ; -if (PageBalloon (page)) - u |=1 <<KPF_BALLOON ; +if (PageOffline (page)) + u |=1 <<KPF_OFFLINE ;if (PageTable (page)) u |=1 <<KPF_PGTABLE ;
include/linux/balloon_compaction.h +4 -4
@@ -95,7 +95,7 @@ extern int balloon_page_migrate(struct address_space *mapping,static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);__SetPageMovable (page, balloon->inode->i_mapping);set_page_private (page, (unsigned long )balloon);list_add (&page->lru, &balloon->pages);
@@ -111,7 +111,7 @@ static inline void balloon_page_insert(struct balloon_dev_info *balloon,*/ static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);__ClearPageMovable (page);set_page_private (page,0 );/*
@@ -141,13 +141,13 @@ static inline gfp_t balloon_mapping_gfp_mask(void)static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);list_add (&page->lru, &balloon->pages); }static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);list_del (&page->lru); }
include/linux/page-flags.h +7 -4
@@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap)#define PAGE_TYPE_BASE 0xf0000000/* Reserve 0x0000007f to catch underflows of page_mapcount */ #define PG_buddy 0x00000080 -#define PG_balloon 0x00000100 +#define PG_offline 0x00000100#define PG_kmemcg 0x00000200#define PG_table 0x00000400
@@ -700,10 +700,13 @@ static __always_inline void __ClearPage##uname(struct page *page) \PAGE_TYPE_OPS (Buddy, buddy)/* -* PageBalloon() is true for pages that are on the balloon page list -* (see mm/balloon_compaction.c). +* PageOffline() indicates that the pages is logically offline although the +* containing section is online. (e.g. inflated in a balloon driver or +* not onlined when onlining the section). +* The content of these pages is effectively stale. Such pages should not +* be touched (read/write/dump/save) except by their owner. */ -PAGE_TYPE_OPS (Balloon, balloon) +PAGE_TYPE_OPS (Offline, offline)/* * If kmemcg is enabled, the buddy allocator will set PageKmemcg() on
include/uapi/linux/kernel-page-flags.h +1 -1
@@ -32,7 +32,7 @@#define KPF_KSM 21#define KPF_THP 22 -#define KPF_BALLOON 23 +#define KPF_OFFLINE 23#define KPF_ZERO_PAGE 24#define KPF_IDLE 25#define KPF_PGTABLE 26
tools/vm/page-types.c +1 -1
@@ -133,7 +133,7 @@ static const char * const page_flag_names[] = { [KPF_NOPAGE] = "n:nopage", [KPF_KSM] = "x:ksm", [KPF_THP] = "t:thp", - [KPF_BALLOON] = "o:balloon", + [KPF_OFFLINE] = "o:offline", [KPF_PGTABLE] = "g:pgtable", [KPF_ZERO_PAGE] = "z:zero_page", [KPF_IDLE] = "i:idle_page",
Signature · 1 line
2.17.2
Re: [PATCH v1 2/8] mm: convert PG_balloon to PG_offline
Quoted 20 lines · David Hildenbrand
PG_balloonwas introduced to implement page migration/compaction for pages inflated in virtio-balloon. Nowadays, it is only a marker that a page is part of virtio-balloon and therefore logically offline.We also want to make use of this flag in other balloon drivers - for inflated pages or when onlining a section but keeping some pages offline (e.g. used right now by XEN and Hyper-V via
set_online_page_callback()).We are going to expose this flag to dump tools like makedumpfile. But instead of exposing
PG_balloon, let's generalize the concept of marking pages as logically offline, so it can be reused for other purposes later on.Rename
PG_balloontoPG_offline. This is an indicator that the page is logically offline, the content stale and that it should not be touched (e.g. a hypervisor would have to allocate backing storage in order for the guest to dump an unused page). We can then e.g. exclude such pages from dumps.
Quoted 4 lines
We replace and reuse
KPF_BALLOON(23), as this shouldn't really harm (and for now the semantics stay the same). In following patches, we will make use of this bit also in other balloon drivers. While at it, document PGTABLE.
Ok, makes sense.
Quoted 166 lines
Cc Jonathan Corbet <[email protected]>Cc Alexey Dobriyan <[email protected]>Cc Mike Rapoport <[email protected]>Cc Andrew Morton <[email protected]>Cc Christian Hansen <[email protected]>Cc Vlastimil Babka <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Stephen Rothwell <[email protected]>Cc Matthew Wilcox <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Alexander Duyck <[email protected]>Cc Naoya Horiguchi <[email protected]>Cc Miles Chen <[email protected]>Cc David Rientjes <[email protected]>Cc Konstantin Khlebnikov <[email protected]>Cc Kazuhito Hagio <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- Documentation/admin-guide/mm/pagemap.rst | 9 ++++++--- fs/proc/page.c | 4 ++-- include/linux/balloon_compaction.h | 8 ++++---- include/linux/page-flags.h | 11 +++++++---- include/uapi/linux/kernel-page-flags.h | 2 +- tools/vm/page-types.c | 2 +- 6 files changed, 21 insertions(+), 15 deletions(-)Documentation/admin-guide/mm/pagemap.rst +6
-3@@ -75,9 +75,10 @@ number of times a page is mapped. 20. NOPAGE 21. KSM 22. THP - 23. BALLOON + 23. OFFLINE 24. ZERO_PAGE 25. IDLE + 26. PGTABLE * ``/proc/kpagecgroup``. This file contains a 64-bit inode number of the memory cgroup each page is charged to, indexed by PFN. Only available when@@ -118,8 +119,8 @@ Short descriptions to the page flags identical memory pages dynamically shared between one or more processes 22 - THP contiguous pages which construct transparent hugepages -23 - BALLOON - balloon compaction page +23 - OFFLINE + page is logically offline 24 - ZERO_PAGE zero page for pfn_zero or huge_zero page 25 - IDLE@@ -128,6 +129,8 @@ Short descriptions to the page flags Note that this flag may be stale in case the page was accessed via a PTE. To make sure the flag is up-to-date one has to read ``/sys/kernel/mm/page_idle/bitmap`` first. +26 - PGTABLE + page is in use as a page table IO related page flags ---------------------fs/proc/page.c +2
-2@@ -152,8 +152,8 @@ u64 stable_page_flags(struct page *page)else if (page_count (page) ==0 &&is_free_buddy_page (page)) u |=1 <<KPF_BUDDY ; -if (PageBalloon (page)) - u |=1 <<KPF_BALLOON ; +if (PageOffline (page)) + u |=1 <<KPF_OFFLINE ;if (PageTable (page)) u |=1 <<KPF_PGTABLE ;include/linux/balloon_compaction.h +4
-4@@ -95,7 +95,7 @@ extern int balloon_page_migrate(struct address_space *mapping,static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);__SetPageMovable (page, balloon->inode->i_mapping);set_page_private (page, (unsigned long )balloon);list_add (&page->lru, &balloon->pages);@@ -111,7 +111,7 @@ static inline void balloon_page_insert(struct balloon_dev_info *balloon,*/ static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);__ClearPageMovable (page);set_page_private (page,0 );/* @@ -141,13 +141,13 @@ static inline gfp_t balloon_mapping_gfp_mask(void)static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);list_add (&page->lru, &balloon->pages); }static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);list_del (&page->lru); }include/linux/page-flags.h +7
-4@@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap)#define PAGE_TYPE_BASE 0xf0000000/* Reserve 0x0000007f to catch underflows of page_mapcount */ #define PG_buddy 0x00000080 -#define PG_balloon 0x00000100 +#define PG_offline 0x00000100#define PG_kmemcg 0x00000200#define PG_table 0x00000400@@ -700,10 +700,13 @@ static __always_inline void __ClearPage##uname(struct page *page) \PAGE_TYPE_OPS (Buddy, buddy)/* -* PageBalloon() is true for pages that are on the balloon page list -* (see mm/balloon_compaction.c). +* PageOffline() indicates that the pages is logically offline although the +* containing section is online. (e.g. inflated in a balloon driver or +* not onlined when onlining the section). +* The content of these pages is effectively stale. Such pages should not +* be touched (read/write/dump/save) except by their owner. */ -PAGE_TYPE_OPS (Balloon, balloon) +PAGE_TYPE_OPS (Offline, offline)/* * If kmemcg is enabled, the buddy allocator will set PageKmemcg() on include/uapi/linux/kernel-page-flags.h +1
-1@@ -32,7 +32,7 @@#define KPF_KSM 21#define KPF_THP 22 -#define KPF_BALLOON 23 +#define KPF_OFFLINE 23#define KPF_ZERO_PAGE 24#define KPF_IDLE 25#define KPF_PGTABLE 26tools/vm/page-types.c +0
-0
tools/vm/page-types.c static const char * const page_flag_names[] = { · line 136
Quoted 5 lines
tools/vm/page-types.c +1
-1@@ -133,7 +133,7 @@ static const char * const page_flag_names[] = { [KPF_NOPAGE] = "n:nopage", [KPF_KSM] = "x:ksm", [KPF_THP] = "t:thp", - [KPF_BALLOON] = "o:balloon", + [KPF_OFFLINE] = "o:offline",
Even 'o' keyword is better suits for Offline than for ballOon =)
Quoted 6 lines
tools/vm/page-types.c +0
-1[KPF_PGTABLE] = "g:pgtable", [KPF_ZERO_PAGE] = "z:zero_page", [KPF_IDLE] = "i:idle_page", --2.17.2
Re: [PATCH v1 2/8] mm: convert PG_balloon to PG_offline
In reply to David HildenbrandQuoted 20 lines · David Hildenbrand
PG_balloonwas introduced to implement page migration/compaction for pages inflated in virtio-balloon. Nowadays, it is only a marker that a page is part of virtio-balloon and therefore logically offline.We also want to make use of this flag in other balloon drivers - for inflated pages or when onlining a section but keeping some pages offline (e.g. used right now by XEN and Hyper-V via
set_online_page_callback()).We are going to expose this flag to dump tools like makedumpfile. But instead of exposing
PG_balloon, let's generalize the concept of marking pages as logically offline, so it can be reused for other purposes later on.Rename
PG_balloontoPG_offline. This is an indicator that the page is logically offline, the content stale and that it should not be touched (e.g. a hypervisor would have to allocate backing storage in order for the guest to dump an unused page). We can then e.g. exclude such pages from dumps.
Quoted 4 lines
We replace and reuse
KPF_BALLOON(23), as this shouldn't really harm (and for now the semantics stay the same). In following patches, we will make use of this bit also in other balloon drivers. While at it, document PGTABLE.
Ok, makes sense.
Quoted 166 lines
Cc Jonathan Corbet <[email protected]>Cc Alexey Dobriyan <[email protected]>Cc Mike Rapoport <[email protected]>Cc Andrew Morton <[email protected]>Cc Christian Hansen <[email protected]>Cc Vlastimil Babka <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Stephen Rothwell <[email protected]>Cc Matthew Wilcox <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Alexander Duyck <[email protected]>Cc Naoya Horiguchi <[email protected]>Cc Miles Chen <[email protected]>Cc David Rientjes <[email protected]>Cc Konstantin Khlebnikov <[email protected]>Cc Kazuhito Hagio <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- Documentation/admin-guide/mm/pagemap.rst | 9 ++++++--- fs/proc/page.c | 4 ++-- include/linux/balloon_compaction.h | 8 ++++---- include/linux/page-flags.h | 11 +++++++---- include/uapi/linux/kernel-page-flags.h | 2 +- tools/vm/page-types.c | 2 +- 6 files changed, 21 insertions(+), 15 deletions(-)Documentation/admin-guide/mm/pagemap.rst +6
-3@@ -75,9 +75,10 @@ number of times a page is mapped. 20. NOPAGE 21. KSM 22. THP - 23. BALLOON + 23. OFFLINE 24. ZERO_PAGE 25. IDLE + 26. PGTABLE * ``/proc/kpagecgroup``. This file contains a 64-bit inode number of the memory cgroup each page is charged to, indexed by PFN. Only available when@@ -118,8 +119,8 @@ Short descriptions to the page flags identical memory pages dynamically shared between one or more processes 22 - THP contiguous pages which construct transparent hugepages -23 - BALLOON - balloon compaction page +23 - OFFLINE + page is logically offline 24 - ZERO_PAGE zero page for pfn_zero or huge_zero page 25 - IDLE@@ -128,6 +129,8 @@ Short descriptions to the page flags Note that this flag may be stale in case the page was accessed via a PTE. To make sure the flag is up-to-date one has to read ``/sys/kernel/mm/page_idle/bitmap`` first. +26 - PGTABLE + page is in use as a page table IO related page flags ---------------------fs/proc/page.c +2
-2@@ -152,8 +152,8 @@ u64 stable_page_flags(struct page *page)else if (page_count (page) ==0 &&is_free_buddy_page (page)) u |=1 <<KPF_BUDDY ; -if (PageBalloon (page)) - u |=1 <<KPF_BALLOON ; +if (PageOffline (page)) + u |=1 <<KPF_OFFLINE ;if (PageTable (page)) u |=1 <<KPF_PGTABLE ;include/linux/balloon_compaction.h +4
-4@@ -95,7 +95,7 @@ extern int balloon_page_migrate(struct address_space *mapping,static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);__SetPageMovable (page, balloon->inode->i_mapping);set_page_private (page, (unsigned long )balloon);list_add (&page->lru, &balloon->pages);@@ -111,7 +111,7 @@ static inline void balloon_page_insert(struct balloon_dev_info *balloon,*/ static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);__ClearPageMovable (page);set_page_private (page,0 );/* @@ -141,13 +141,13 @@ static inline gfp_t balloon_mapping_gfp_mask(void)static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);list_add (&page->lru, &balloon->pages); }static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);list_del (&page->lru); }include/linux/page-flags.h +7
-4@@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap)#define PAGE_TYPE_BASE 0xf0000000/* Reserve 0x0000007f to catch underflows of page_mapcount */ #define PG_buddy 0x00000080 -#define PG_balloon 0x00000100 +#define PG_offline 0x00000100#define PG_kmemcg 0x00000200#define PG_table 0x00000400@@ -700,10 +700,13 @@ static __always_inline void __ClearPage##uname(struct page *page) \PAGE_TYPE_OPS (Buddy, buddy)/* -* PageBalloon() is true for pages that are on the balloon page list -* (see mm/balloon_compaction.c). +* PageOffline() indicates that the pages is logically offline although the +* containing section is online. (e.g. inflated in a balloon driver or +* not onlined when onlining the section). +* The content of these pages is effectively stale. Such pages should not +* be touched (read/write/dump/save) except by their owner. */ -PAGE_TYPE_OPS (Balloon, balloon) +PAGE_TYPE_OPS (Offline, offline)/* * If kmemcg is enabled, the buddy allocator will set PageKmemcg() on include/uapi/linux/kernel-page-flags.h +1
-1@@ -32,7 +32,7 @@#define KPF_KSM 21#define KPF_THP 22 -#define KPF_BALLOON 23 +#define KPF_OFFLINE 23#define KPF_ZERO_PAGE 24#define KPF_IDLE 25#define KPF_PGTABLE 26tools/vm/page-types.c +0
-0
tools/vm/page-types.c static const char * const page_flag_names[] = { · line 136
Quoted 5 lines
tools/vm/page-types.c +1
-1@@ -133,7 +133,7 @@ static const char * const page_flag_names[] = { [KPF_NOPAGE] = "n:nopage", [KPF_KSM] = "x:ksm", [KPF_THP] = "t:thp", - [KPF_BALLOON] = "o:balloon", + [KPF_OFFLINE] = "o:offline",
Even 'o' keyword is better suits for Offline than for ballOon =)
Quoted 6 lines
tools/vm/page-types.c +0
-1[KPF_PGTABLE] = "g:pgtable", [KPF_ZERO_PAGE] = "z:zero_page", [KPF_IDLE] = "i:idle_page", --2.17.2
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 2/8] mm: convert PG_balloon to PG_offline
In reply to David HildenbrandQuoted 15 lines
include/linux/page-flags.h +1
-1@@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap)#define PAGE_TYPE_BASE 0xf0000000/* Reserve 0x0000007f to catch underflows of page_mapcount */ #define PG_buddy 0x00000080 -#define PG_balloon 0x00000100 +#define PG_offline 0x00000100#define PG_kmemcg 0x00000200#define PG_table 0x00000400
include/linux/page-flags.h static __always_inline void __ClearPage##uname(struct page *page) \ · line 703
Quoted 6 lines
include/linux/page-flags.h +1
-2@@ -700,10 +700,13 @@ static __always_inline void __ClearPage##uname(struct page *page) \PAGE_TYPE_OPS (Buddy, buddy)/* -* PageBalloon() is true for pages that are on the balloon page list -* (see mm/balloon_compaction.c). +* PageOffline() indicates that the pages is logically offline although the
s/pages/page/
:)
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v1 2/8] mm: convert PG_balloon to PG_offline
In reply to David HildenbrandQuoted 15 lines
include/linux/page-flags.h +1
-1@@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap)#define PAGE_TYPE_BASE 0xf0000000/* Reserve 0x0000007f to catch underflows of page_mapcount */ #define PG_buddy 0x00000080 -#define PG_balloon 0x00000100 +#define PG_offline 0x00000100#define PG_kmemcg 0x00000200#define PG_table 0x00000400
include/linux/page-flags.h static __always_inline void __ClearPage##uname(struct page *page) \ · line 703
Quoted 6 lines
include/linux/page-flags.h +1
-2@@ -700,10 +700,13 @@ static __always_inline void __ClearPage##uname(struct page *page) \PAGE_TYPE_OPS (Buddy, buddy)/* -* PageBalloon() is true for pages that are on the balloon page list -* (see mm/balloon_compaction.c). +* PageOffline() indicates that the pages is logically offline although the
s/pages/page/
:)
Signature · 8 lines
Thanks, David / dhildenb _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 2/8] mm: convert PG_balloon to PG_offline
In reply to David HildenbrandQuoted 155 lines · David Hildenbrand
PG_balloonwas introduced to implement page migration/compaction for pages inflated in virtio-balloon. Nowadays, it is only a marker that a page is part of virtio-balloon and therefore logically offline.We also want to make use of this flag in other balloon drivers - for inflated pages or when onlining a section but keeping some pages offline (e.g. used right now by XEN and Hyper-V via
set_online_page_callback()).We are going to expose this flag to dump tools like makedumpfile. But instead of exposing
PG_balloon, let's generalize the concept of marking pages as logically offline, so it can be reused for other purposes later on.Rename
PG_balloontoPG_offline. This is an indicator that the page is logically offline, the content stale and that it should not be touched (e.g. a hypervisor would have to allocate backing storage in order for the guest to dump an unused page). We can then e.g. exclude such pages from dumps.We replace and reuse
KPF_BALLOON(23), as this shouldn't really harm (and for now the semantics stay the same). In following patches, we will make use of this bit also in other balloon drivers. While at it, document PGTABLE.Cc Jonathan Corbet <[email protected]>Cc Alexey Dobriyan <[email protected]>Cc Mike Rapoport <[email protected]>Cc Andrew Morton <[email protected]>Cc Christian Hansen <[email protected]>Cc Vlastimil Babka <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Stephen Rothwell <[email protected]>Cc Matthew Wilcox <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Alexander Duyck <[email protected]>Cc Naoya Horiguchi <[email protected]>Cc Miles Chen <[email protected]>Cc David Rientjes <[email protected]>Cc Konstantin Khlebnikov <[email protected]>Cc Kazuhito Hagio <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- Documentation/admin-guide/mm/pagemap.rst | 9 ++++++--- fs/proc/page.c | 4 ++-- include/linux/balloon_compaction.h | 8 ++++---- include/linux/page-flags.h | 11 +++++++---- include/uapi/linux/kernel-page-flags.h | 2 +- tools/vm/page-types.c | 2 +- 6 files changed, 21 insertions(+), 15 deletions(-)Documentation/admin-guide/mm/pagemap.rst +6
-3@@ -75,9 +75,10 @@ number of times a page is mapped. 20. NOPAGE 21. KSM 22. THP - 23. BALLOON + 23. OFFLINE 24. ZERO_PAGE 25. IDLE + 26. PGTABLE * ``/proc/kpagecgroup``. This file contains a 64-bit inode number of the memory cgroup each page is charged to, indexed by PFN. Only available when@@ -118,8 +119,8 @@ Short descriptions to the page flags identical memory pages dynamically shared between one or more processes 22 - THP contiguous pages which construct transparent hugepages -23 - BALLOON - balloon compaction page +23 - OFFLINE + page is logically offline 24 - ZERO_PAGE zero page for pfn_zero or huge_zero page 25 - IDLE@@ -128,6 +129,8 @@ Short descriptions to the page flags Note that this flag may be stale in case the page was accessed via a PTE. To make sure the flag is up-to-date one has to read ``/sys/kernel/mm/page_idle/bitmap`` first. +26 - PGTABLE + page is in use as a page table IO related page flags ---------------------fs/proc/page.c +2
-2@@ -152,8 +152,8 @@ u64 stable_page_flags(struct page *page)else if (page_count (page) ==0 &&is_free_buddy_page (page)) u |=1 <<KPF_BUDDY ; -if (PageBalloon (page)) - u |=1 <<KPF_BALLOON ; +if (PageOffline (page)) + u |=1 <<KPF_OFFLINE ;if (PageTable (page)) u |=1 <<KPF_PGTABLE ;include/linux/balloon_compaction.h +4
-4@@ -95,7 +95,7 @@ extern int balloon_page_migrate(struct address_space *mapping,static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);__SetPageMovable (page, balloon->inode->i_mapping);set_page_private (page, (unsigned long )balloon);list_add (&page->lru, &balloon->pages);@@ -111,7 +111,7 @@ static inline void balloon_page_insert(struct balloon_dev_info *balloon,*/ static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);__ClearPageMovable (page);set_page_private (page,0 );/* @@ -141,13 +141,13 @@ static inline gfp_t balloon_mapping_gfp_mask(void)static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);list_add (&page->lru, &balloon->pages); }static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);list_del (&page->lru); }include/linux/page-flags.h +1
-1@@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap)#define PAGE_TYPE_BASE 0xf0000000/* Reserve 0x0000007f to catch underflows of page_mapcount */ #define PG_buddy 0x00000080 -#define PG_balloon 0x00000100 +#define PG_offline 0x00000100#define PG_kmemcg 0x00000200#define PG_table 0x00000400
include/linux/page-flags.h static __always_inline void __ClearPage##uname(struct page *page) \ · line 703
Quoted 6 lines
include/linux/page-flags.h +1
-2@@ -700,10 +700,13 @@ static __always_inline void __ClearPage##uname(struct page *page) \PAGE_TYPE_OPS (Buddy, buddy)/* -* PageBalloon() is true for pages that are on the balloon page list -* (see mm/balloon_compaction.c). +* PageOffline() indicates that the pages is logically offline although the
with this typo fixed ..
Quoted 28 lines
include/linux/page-flags.h +5
-1+* containing section is online. (e.g. inflated in a balloon driver or +* not onlined when onlining the section). +* The content of these pages is effectively stale. Such pages should not +* be touched (read/write/dump/save) except by their owner. */ -PAGE_TYPE_OPS (Balloon, balloon) +PAGE_TYPE_OPS (Offline, offline)/* * If kmemcg is enabled, the buddy allocator will set PageKmemcg() on include/uapi/linux/kernel-page-flags.h +1
-1@@ -32,7 +32,7 @@#define KPF_KSM 21#define KPF_THP 22 -#define KPF_BALLOON 23 +#define KPF_OFFLINE 23#define KPF_ZERO_PAGE 24#define KPF_IDLE 25#define KPF_PGTABLE 26tools/vm/page-types.c +0
-0
tools/vm/page-types.c static const char * const page_flag_names[] = { · line 139
Quoted 8 lines
tools/vm/page-types.c +1
-1@@ -133,7 +133,7 @@ static const char * const page_flag_names[] = { [KPF_NOPAGE] = "n:nopage", [KPF_KSM] = "x:ksm", [KPF_THP] = "t:thp", - [KPF_BALLOON] = "o:balloon", + [KPF_OFFLINE] = "o:offline", [KPF_PGTABLE] = "g:pgtable", [KPF_ZERO_PAGE] = "z:zero_page", [KPF_IDLE] = "i:idle_page",
for balloon changes:
Quoted 2 lines
Signature · 1 line
2.17.2
Re: [PATCH v1 2/8] mm: convert PG_balloon to PG_offline
In reply to David HildenbrandQuoted 155 lines · David Hildenbrand
PG_balloonwas introduced to implement page migration/compaction for pages inflated in virtio-balloon. Nowadays, it is only a marker that a page is part of virtio-balloon and therefore logically offline.We also want to make use of this flag in other balloon drivers - for inflated pages or when onlining a section but keeping some pages offline (e.g. used right now by XEN and Hyper-V via
set_online_page_callback()).We are going to expose this flag to dump tools like makedumpfile. But instead of exposing
PG_balloon, let's generalize the concept of marking pages as logically offline, so it can be reused for other purposes later on.Rename
PG_balloontoPG_offline. This is an indicator that the page is logically offline, the content stale and that it should not be touched (e.g. a hypervisor would have to allocate backing storage in order for the guest to dump an unused page). We can then e.g. exclude such pages from dumps.We replace and reuse
KPF_BALLOON(23), as this shouldn't really harm (and for now the semantics stay the same). In following patches, we will make use of this bit also in other balloon drivers. While at it, document PGTABLE.Cc Jonathan Corbet <[email protected]>Cc Alexey Dobriyan <[email protected]>Cc Mike Rapoport <[email protected]>Cc Andrew Morton <[email protected]>Cc Christian Hansen <[email protected]>Cc Vlastimil Babka <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Stephen Rothwell <[email protected]>Cc Matthew Wilcox <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Alexander Duyck <[email protected]>Cc Naoya Horiguchi <[email protected]>Cc Miles Chen <[email protected]>Cc David Rientjes <[email protected]>Cc Konstantin Khlebnikov <[email protected]>Cc Kazuhito Hagio <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- Documentation/admin-guide/mm/pagemap.rst | 9 ++++++--- fs/proc/page.c | 4 ++-- include/linux/balloon_compaction.h | 8 ++++---- include/linux/page-flags.h | 11 +++++++---- include/uapi/linux/kernel-page-flags.h | 2 +- tools/vm/page-types.c | 2 +- 6 files changed, 21 insertions(+), 15 deletions(-)Documentation/admin-guide/mm/pagemap.rst +6
-3@@ -75,9 +75,10 @@ number of times a page is mapped. 20. NOPAGE 21. KSM 22. THP - 23. BALLOON + 23. OFFLINE 24. ZERO_PAGE 25. IDLE + 26. PGTABLE * ``/proc/kpagecgroup``. This file contains a 64-bit inode number of the memory cgroup each page is charged to, indexed by PFN. Only available when@@ -118,8 +119,8 @@ Short descriptions to the page flags identical memory pages dynamically shared between one or more processes 22 - THP contiguous pages which construct transparent hugepages -23 - BALLOON - balloon compaction page +23 - OFFLINE + page is logically offline 24 - ZERO_PAGE zero page for pfn_zero or huge_zero page 25 - IDLE@@ -128,6 +129,8 @@ Short descriptions to the page flags Note that this flag may be stale in case the page was accessed via a PTE. To make sure the flag is up-to-date one has to read ``/sys/kernel/mm/page_idle/bitmap`` first. +26 - PGTABLE + page is in use as a page table IO related page flags ---------------------fs/proc/page.c +2
-2@@ -152,8 +152,8 @@ u64 stable_page_flags(struct page *page)else if (page_count (page) ==0 &&is_free_buddy_page (page)) u |=1 <<KPF_BUDDY ; -if (PageBalloon (page)) - u |=1 <<KPF_BALLOON ; +if (PageOffline (page)) + u |=1 <<KPF_OFFLINE ;if (PageTable (page)) u |=1 <<KPF_PGTABLE ;include/linux/balloon_compaction.h +4
-4@@ -95,7 +95,7 @@ extern int balloon_page_migrate(struct address_space *mapping,static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);__SetPageMovable (page, balloon->inode->i_mapping);set_page_private (page, (unsigned long )balloon);list_add (&page->lru, &balloon->pages);@@ -111,7 +111,7 @@ static inline void balloon_page_insert(struct balloon_dev_info *balloon,*/ static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);__ClearPageMovable (page);set_page_private (page,0 );/* @@ -141,13 +141,13 @@ static inline gfp_t balloon_mapping_gfp_mask(void)static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);list_add (&page->lru, &balloon->pages); }static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);list_del (&page->lru); }include/linux/page-flags.h +1
-1@@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap)#define PAGE_TYPE_BASE 0xf0000000/* Reserve 0x0000007f to catch underflows of page_mapcount */ #define PG_buddy 0x00000080 -#define PG_balloon 0x00000100 +#define PG_offline 0x00000100#define PG_kmemcg 0x00000200#define PG_table 0x00000400
include/linux/page-flags.h static __always_inline void __ClearPage##uname(struct page *page) \ · line 703
Quoted 6 lines
include/linux/page-flags.h +1
-2@@ -700,10 +700,13 @@ static __always_inline void __ClearPage##uname(struct page *page) \PAGE_TYPE_OPS (Buddy, buddy)/* -* PageBalloon() is true for pages that are on the balloon page list -* (see mm/balloon_compaction.c). +* PageOffline() indicates that the pages is logically offline although the
with this typo fixed ..
Quoted 28 lines
include/linux/page-flags.h +5
-1+* containing section is online. (e.g. inflated in a balloon driver or +* not onlined when onlining the section). +* The content of these pages is effectively stale. Such pages should not +* be touched (read/write/dump/save) except by their owner. */ -PAGE_TYPE_OPS (Balloon, balloon) +PAGE_TYPE_OPS (Offline, offline)/* * If kmemcg is enabled, the buddy allocator will set PageKmemcg() on include/uapi/linux/kernel-page-flags.h +1
-1@@ -32,7 +32,7 @@#define KPF_KSM 21#define KPF_THP 22 -#define KPF_BALLOON 23 +#define KPF_OFFLINE 23#define KPF_ZERO_PAGE 24#define KPF_IDLE 25#define KPF_PGTABLE 26tools/vm/page-types.c +0
-0
tools/vm/page-types.c static const char * const page_flag_names[] = { · line 139
Quoted 8 lines
tools/vm/page-types.c +1
-1@@ -133,7 +133,7 @@ static const char * const page_flag_names[] = { [KPF_NOPAGE] = "n:nopage", [KPF_KSM] = "x:ksm", [KPF_THP] = "t:thp", - [KPF_BALLOON] = "o:balloon", + [KPF_OFFLINE] = "o:offline", [KPF_PGTABLE] = "g:pgtable", [KPF_ZERO_PAGE] = "z:zero_page", [KPF_IDLE] = "i:idle_page",
for balloon changes:
Quoted 2 lines
Signature · 1 line
2.17.2
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 2/8] mm: convert PG_balloon to PG_offline
In reply to David HildenbrandQuoted 207 lines
PG_balloonwas introduced to implement page migration/compaction for pages inflated in virtio-balloon. Nowadays, it is only a marker that a page is part of virtio-balloon and therefore logically offline.We also want to make use of this flag in other balloon drivers - for inflated pages or when onlining a section but keeping some pages offline (e.g. used right now by XEN and Hyper-V via
set_online_page_callback()).We are going to expose this flag to dump tools like makedumpfile. But instead of exposing
PG_balloon, let's generalize the concept of marking pages as logically offline, so it can be reused for other purposes later on.Rename
PG_balloontoPG_offline. This is an indicator that the page is logically offline, the content stale and that it should not be touched (e.g. a hypervisor would have to allocate backing storage in order for the guest to dump an unused page). We can then e.g. exclude such pages from dumps.We replace and reuse
KPF_BALLOON(23), as this shouldn't really harm (and for now the semantics stay the same). In following patches, we will make use of this bit also in other balloon drivers. While at it, document PGTABLE.Cc Jonathan Corbet <[email protected]>Cc Alexey Dobriyan <[email protected]>Cc Mike Rapoport <[email protected]>Cc Andrew Morton <[email protected]>Cc Christian Hansen <[email protected]>Cc Vlastimil Babka <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Stephen Rothwell <[email protected]>Cc Matthew Wilcox <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Alexander Duyck <[email protected]>Cc Naoya Horiguchi <[email protected]>Cc Miles Chen <[email protected]>Cc David Rientjes <[email protected]>Cc Konstantin Khlebnikov <[email protected]>Cc Kazuhito Hagio <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- Documentation/admin-guide/mm/pagemap.rst | 9 ++++++--- fs/proc/page.c | 4 ++-- include/linux/balloon_compaction.h | 8 ++++---- include/linux/page-flags.h | 11 +++++++---- include/uapi/linux/kernel-page-flags.h | 2 +- tools/vm/page-types.c | 2 +- 6 files changed, 21 insertions(+), 15 deletions(-)Documentation/admin-guide/mm/pagemap.rst +2
-1@@ -75,9 +75,10 @@ number of times a page is mapped. 20. NOPAGE 21. KSM 22. THP - 23. BALLOON + 23. OFFLINE 24. ZERO_PAGE 25. IDLE + 26. PGTABLE * ``/proc/kpagecgroup``. This file contains a 64-bit inode number of the memory cgroup each page is charged to, indexed by PFN. Only availablewhen
Documentation/admin-guide/mm/pagemap.rst +4
-2@@ -118,8 +119,8 @@ Short descriptions to the page flags identical memory pages dynamically shared between one or more processes 22 - THP contiguous pages which construct transparent hugepages -23 - BALLOON - balloon compaction page +23 - OFFLINE + page is logically offline 24 - ZERO_PAGE zero page for pfn_zero or huge_zero page 25 - IDLE@@ -128,6 +129,8 @@ Short descriptions to the page flags Note that this flag may be stale in case the page was accessed via a PTE. To make sure the flag is up-to-date one has to read ``/sys/kernel/mm/page_idle/bitmap`` first. +26 - PGTABLE + page is in use as a page table IO related page flags ---------------------fs/proc/page.c +2
-2@@ -152,8 +152,8 @@ u64 stable_page_flags(struct page *page)else if (page_count (page) ==0 &&is_free_buddy_page (page)) u |=1 <<KPF_BUDDY ; -if (PageBalloon (page)) - u |=1 <<KPF_BALLOON ; +if (PageOffline (page)) + u |=1 <<KPF_OFFLINE ;if (PageTable (page)) u |=1 <<KPF_PGTABLE ;include/linux/balloon_compaction.h +1
-1@@ -95,7 +95,7 @@ extern int balloon_page_migrate(struct address_space *mapping, static inline void balloon_page_insert(struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);__SetPageMovable (page, balloon->inode->i_mapping);set_page_private (page, (unsigned long )balloon);include/linux/balloon_compaction.h +1
-1list_add (&page->lru, &balloon->pages);@@ -111,7 +111,7 @@ static inline void balloon_page_insert(structballoon_dev_info *balloon, */ static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);__ClearPageMovable (page);set_page_private (page,0 );include/linux/balloon_compaction.h +2
-2/* @@ -141,13 +141,13 @@ static inline gfp_t balloon_mapping_gfp_mask(void)static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);list_add (&page->lru, &balloon->pages); }static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);list_del (&page->lru); }include/linux/page-flags.h +7
-4@@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap)#define PAGE_TYPE_BASE 0xf0000000/* Reserve 0x0000007f to catch underflows of page_mapcount */ #define PG_buddy 0x00000080 -#define PG_balloon 0x00000100 +#define PG_offline 0x00000100#define PG_kmemcg 0x00000200#define PG_table 0x00000400@@ -700,10 +700,13 @@ static __always_inline void __ClearPage##uname(struct page *page) \ PAGE_TYPE_OPS(Buddy, buddy) /* - * PageBalloon() is true for pages that are on the balloon page list - * (see mm/balloon_compaction.c). + * PageOffline() indicates that the pages is logically offline although the + * containing section is online. (e.g. inflated in a balloon driver or + * not onlined when onlining the section). + * The content of these pages is effectively stale. Such pages should not + * be touched (read/write/dump/save) except by their owner. */ -PAGE_TYPE_OPS(Balloon, balloon) +PAGE_TYPE_OPS(Offline, offline)/* * If kmemcg is enabled, the buddy allocator will set
PageKmemcg()oninclude/uapi/linux/kernel-page-flags.h +1
-1@@ -32,7 +32,7 @@#define KPF_KSM 21#define KPF_THP 22 -#define KPF_BALLOON 23 +#define KPF_OFFLINE 23#define KPF_ZERO_PAGE 24#define KPF_IDLE 25#define KPF_PGTABLE 26tools/vm/page-types.c +1
-2@@ -133,7 +133,7 @@ static const char * const page_flag_names[] = { [KPF_NOPAGE] = "n:nopage", [KPF_KSM] = "x:ksm", [KPF_THP] = "t:thp", - [KPF_BALLOON] = "o:balloon", + [KPF_OFFLINE] = "o:offline", [KPF_PGTABLE] = "g:pgtable", [KPF_ZERO_PAGE] = "z:zero_page", [KPF_IDLE] = "i:idle_page", --
Quoted 1 line
2.17.2
Quoted 2 lines
Re: [PATCH v1 2/8] mm: convert PG_balloon to PG_offline
In reply to David HildenbrandQuoted 207 lines
PG_balloonwas introduced to implement page migration/compaction for pages inflated in virtio-balloon. Nowadays, it is only a marker that a page is part of virtio-balloon and therefore logically offline.We also want to make use of this flag in other balloon drivers - for inflated pages or when onlining a section but keeping some pages offline (e.g. used right now by XEN and Hyper-V via
set_online_page_callback()).We are going to expose this flag to dump tools like makedumpfile. But instead of exposing
PG_balloon, let's generalize the concept of marking pages as logically offline, so it can be reused for other purposes later on.Rename
PG_balloontoPG_offline. This is an indicator that the page is logically offline, the content stale and that it should not be touched (e.g. a hypervisor would have to allocate backing storage in order for the guest to dump an unused page). We can then e.g. exclude such pages from dumps.We replace and reuse
KPF_BALLOON(23), as this shouldn't really harm (and for now the semantics stay the same). In following patches, we will make use of this bit also in other balloon drivers. While at it, document PGTABLE.Cc Jonathan Corbet <[email protected]>Cc Alexey Dobriyan <[email protected]>Cc Mike Rapoport <[email protected]>Cc Andrew Morton <[email protected]>Cc Christian Hansen <[email protected]>Cc Vlastimil Babka <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Stephen Rothwell <[email protected]>Cc Matthew Wilcox <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Michal Hocko <[email protected]>Cc Pavel Tatashin <[email protected]>Cc Alexander Duyck <[email protected]>Cc Naoya Horiguchi <[email protected]>Cc Miles Chen <[email protected]>Cc David Rientjes <[email protected]>Cc Konstantin Khlebnikov <[email protected]>Cc Kazuhito Hagio <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- Documentation/admin-guide/mm/pagemap.rst | 9 ++++++--- fs/proc/page.c | 4 ++-- include/linux/balloon_compaction.h | 8 ++++---- include/linux/page-flags.h | 11 +++++++---- include/uapi/linux/kernel-page-flags.h | 2 +- tools/vm/page-types.c | 2 +- 6 files changed, 21 insertions(+), 15 deletions(-)Documentation/admin-guide/mm/pagemap.rst +2
-1@@ -75,9 +75,10 @@ number of times a page is mapped. 20. NOPAGE 21. KSM 22. THP - 23. BALLOON + 23. OFFLINE 24. ZERO_PAGE 25. IDLE + 26. PGTABLE * ``/proc/kpagecgroup``. This file contains a 64-bit inode number of the memory cgroup each page is charged to, indexed by PFN. Only availablewhen
Documentation/admin-guide/mm/pagemap.rst +4
-2@@ -118,8 +119,8 @@ Short descriptions to the page flags identical memory pages dynamically shared between one or more processes 22 - THP contiguous pages which construct transparent hugepages -23 - BALLOON - balloon compaction page +23 - OFFLINE + page is logically offline 24 - ZERO_PAGE zero page for pfn_zero or huge_zero page 25 - IDLE@@ -128,6 +129,8 @@ Short descriptions to the page flags Note that this flag may be stale in case the page was accessed via a PTE. To make sure the flag is up-to-date one has to read ``/sys/kernel/mm/page_idle/bitmap`` first. +26 - PGTABLE + page is in use as a page table IO related page flags ---------------------fs/proc/page.c +2
-2@@ -152,8 +152,8 @@ u64 stable_page_flags(struct page *page)else if (page_count (page) ==0 &&is_free_buddy_page (page)) u |=1 <<KPF_BUDDY ; -if (PageBalloon (page)) - u |=1 <<KPF_BALLOON ; +if (PageOffline (page)) + u |=1 <<KPF_OFFLINE ;if (PageTable (page)) u |=1 <<KPF_PGTABLE ;include/linux/balloon_compaction.h +1
-1@@ -95,7 +95,7 @@ extern int balloon_page_migrate(struct address_space *mapping, static inline void balloon_page_insert(struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);__SetPageMovable (page, balloon->inode->i_mapping);set_page_private (page, (unsigned long )balloon);include/linux/balloon_compaction.h +1
-1list_add (&page->lru, &balloon->pages);@@ -111,7 +111,7 @@ static inline void balloon_page_insert(structballoon_dev_info *balloon, */ static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);__ClearPageMovable (page);set_page_private (page,0 );include/linux/balloon_compaction.h +2
-2/* @@ -141,13 +141,13 @@ static inline gfp_t balloon_mapping_gfp_mask(void)static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);list_add (&page->lru, &balloon->pages); }static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);list_del (&page->lru); }include/linux/page-flags.h +7
-4@@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap)#define PAGE_TYPE_BASE 0xf0000000/* Reserve 0x0000007f to catch underflows of page_mapcount */ #define PG_buddy 0x00000080 -#define PG_balloon 0x00000100 +#define PG_offline 0x00000100#define PG_kmemcg 0x00000200#define PG_table 0x00000400@@ -700,10 +700,13 @@ static __always_inline void __ClearPage##uname(struct page *page) \ PAGE_TYPE_OPS(Buddy, buddy) /* - * PageBalloon() is true for pages that are on the balloon page list - * (see mm/balloon_compaction.c). + * PageOffline() indicates that the pages is logically offline although the + * containing section is online. (e.g. inflated in a balloon driver or + * not onlined when onlining the section). + * The content of these pages is effectively stale. Such pages should not + * be touched (read/write/dump/save) except by their owner. */ -PAGE_TYPE_OPS(Balloon, balloon) +PAGE_TYPE_OPS(Offline, offline)/* * If kmemcg is enabled, the buddy allocator will set
PageKmemcg()oninclude/uapi/linux/kernel-page-flags.h +1
-1@@ -32,7 +32,7 @@#define KPF_KSM 21#define KPF_THP 22 -#define KPF_BALLOON 23 +#define KPF_OFFLINE 23#define KPF_ZERO_PAGE 24#define KPF_IDLE 25#define KPF_PGTABLE 26tools/vm/page-types.c +1
-2@@ -133,7 +133,7 @@ static const char * const page_flag_names[] = { [KPF_NOPAGE] = "n:nopage", [KPF_KSM] = "x:ksm", [KPF_THP] = "t:thp", - [KPF_BALLOON] = "o:balloon", + [KPF_OFFLINE] = "o:offline", [KPF_PGTABLE] = "g:pgtable", [KPF_ZERO_PAGE] = "z:zero_page", [KPF_IDLE] = "i:idle_page", --
Quoted 1 line
2.17.2
Quoted 2 lines
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 2/8 mm: convert PG_balloon to PG_offline
In reply to David HildenbrandPG_balloon was introduced to implement page migration/compaction for pages inflated in virtio-balloon. Nowadays, it is only a marker that a page is part of virtio-balloon and therefore logically offline.
We also want to make use of this flag in other balloon drivers - for inflated pages or when onlining a section but keeping some pages offline (e.g. used right now by XEN and Hyper-V via set_online_page_callback()).
We are going to expose this flag to dump tools like makedumpfile. But instead of exposing PG_balloon, let's generalize the concept of marking pages as logically offline, so it can be reused for other purposes later on.
Rename PG_balloon to PG_offline. This is an indicator that the page is logically offline, the content stale and that it should not be touched (e.g. a hypervisor would have to allocate backing storage in order for the guest to dump an unused page). We can then e.g. exclude such pages from dumps.
We replace and reuse KPF_BALLOON (23), as this shouldn't really harm (and for now the semantics stay the same). In following patches, we will make use of this bit also in other balloon drivers. While at it, document PGTABLE.
--- Documentation/admin-guide/mm/pagemap.rst | 9 ++++++--- fs/proc/page.c | 4 ++-- include/linux/balloon_compaction.h | 8 ++++---- include/linux/page-flags.h | 11 +++++++---- include/uapi/linux/kernel-page-flags.h | 2 +- tools/vm/page-types.c | 2 +- 6 files changed, 21 insertions(+), 15 deletions(-)
Documentation/admin-guide/mm/pagemap.rst +6 -3
@@ -75,9 +75,10 @@ number of times a page is mapped. 20. NOPAGE 21. KSM 22. THP - 23. BALLOON + 23. OFFLINE 24. ZERO_PAGE 25. IDLE + 26. PGTABLE * ``/proc/kpagecgroup``. This file contains a 64-bit inode number of the memory cgroup each page is charged to, indexed by PFN. Only available when
@@ -118,8 +119,8 @@ Short descriptions to the page flags identical memory pages dynamically shared between one or more processes 22 - THP contiguous pages which construct transparent hugepages -23 - BALLOON - balloon compaction page +23 - OFFLINE + page is logically offline 24 - ZERO_PAGE zero page for pfn_zero or huge_zero page 25 - IDLE
@@ -128,6 +129,8 @@ Short descriptions to the page flags Note that this flag may be stale in case the page was accessed via a PTE. To make sure the flag is up-to-date one has to read ``/sys/kernel/mm/page_idle/bitmap`` first. +26 - PGTABLE + page is in use as a page table IO related page flags ---------------------
fs/proc/page.c +2 -2
@@ -152,8 +152,8 @@ u64 stable_page_flags(struct page *page)else if (page_count (page) ==0 &&is_free_buddy_page (page)) u |=1 <<KPF_BUDDY ; -if (PageBalloon (page)) - u |=1 <<KPF_BALLOON ; +if (PageOffline (page)) + u |=1 <<KPF_OFFLINE ;if (PageTable (page)) u |=1 <<KPF_PGTABLE ;
include/linux/balloon_compaction.h +4 -4
@@ -95,7 +95,7 @@ extern int balloon_page_migrate(struct address_space *mapping,static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);__SetPageMovable (page, balloon->inode->i_mapping);set_page_private (page, (unsigned long )balloon);list_add (&page->lru, &balloon->pages);
@@ -111,7 +111,7 @@ static inline void balloon_page_insert(struct balloon_dev_info *balloon,*/ static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);__ClearPageMovable (page);set_page_private (page,0 );/*
@@ -141,13 +141,13 @@ static inline gfp_t balloon_mapping_gfp_mask(void)static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { -__SetPageBalloon (page); +__SetPageOffline (page);list_add (&page->lru, &balloon->pages); }static inline void balloon_page_delete (struct page *page) { -__ClearPageBalloon (page); +__ClearPageOffline (page);list_del (&page->lru); }
include/linux/page-flags.h +7 -4
@@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap)#define PAGE_TYPE_BASE 0xf0000000/* Reserve 0x0000007f to catch underflows of page_mapcount */ #define PG_buddy 0x00000080 -#define PG_balloon 0x00000100 +#define PG_offline 0x00000100#define PG_kmemcg 0x00000200#define PG_table 0x00000400
@@ -700,10 +700,13 @@ static __always_inline void __ClearPage##uname(struct page *page) \PAGE_TYPE_OPS (Buddy, buddy)/* -* PageBalloon() is true for pages that are on the balloon page list -* (see mm/balloon_compaction.c). +* PageOffline() indicates that the pages is logically offline although the +* containing section is online. (e.g. inflated in a balloon driver or +* not onlined when onlining the section). +* The content of these pages is effectively stale. Such pages should not +* be touched (read/write/dump/save) except by their owner. */ -PAGE_TYPE_OPS (Balloon, balloon) +PAGE_TYPE_OPS (Offline, offline)/* * If kmemcg is enabled, the buddy allocator will set PageKmemcg() on
include/uapi/linux/kernel-page-flags.h +1 -1
@@ -32,7 +32,7 @@#define KPF_KSM 21#define KPF_THP 22 -#define KPF_BALLOON 23 +#define KPF_OFFLINE 23#define KPF_ZERO_PAGE 24#define KPF_IDLE 25#define KPF_PGTABLE 26
tools/vm/page-types.c +1 -1
@@ -133,7 +133,7 @@ static const char * const page_flag_names[] = { [KPF_NOPAGE] = "n:nopage", [KPF_KSM] = "x:ksm", [KPF_THP] = "t:thp", - [KPF_BALLOON] = "o:balloon", + [KPF_OFFLINE] = "o:offline", [KPF_PGTABLE] = "g:pgtable", [KPF_ZERO_PAGE] = "z:zero_page", [KPF_IDLE] = "i:idle_page",
Signature · 7 lines
2.17.2 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 3/8 kexec: export PG_offline to VMCOREINFO
In reply to David HildenbrandRight now, pages inflated as part of a balloon driver will be dumped by dump tools like makedumpfile. While XEN is able to check in the crash kernel whether a certain pfn is actuall backed by memory in the hypervisor (see xen_oldmem_pfn_is_ram) and optimize this case, dumps of other balloon inflated memory will essentially result in zero pages getting allocated by the hypervisor and the dump getting filled with this data.
The allocation and reading of zero pages can directly be avoided if a dumping tool could know which pages only contain stale information not to be dumped.
We now have PG_offline which can be (and already is by virtio-balloon) used for marking pages as logically offline. Follow up patches will make use of this flag also in other balloon implementations.
Let's export PG_offline via PAGE_OFFLINE_MAPCOUNT_VALUE, so makedumpfile can directly skip pages that are logically offline and the content therefore stale.
Please note that this is also helpful for a problem we were seeing under
Hyper-V: Dumping logically offline memory (pages kept fake offline while onlining a section via online_page_callback) would under some condicions result in a kernel panic when dumping them.
--- kernel/crash_core.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/crash_core.c +2 -0
@@ -464,6 +464,8 @@ static int __init crash_save_vmcoreinfo_init(void)VMCOREINFO_NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE );#ifdef CONFIG_HUGETLB_PAGE VMCOREINFO_NUMBER (HUGETLB_PAGE_DTOR ); +#define PAGE_OFFLINE_MAPCOUNT_VALUE (~PG_offline) +VMCOREINFO_NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE );#endif arch_crash_save_vmcoreinfo ();
Signature · 1 line
2.17.2
Re: [PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO
Quoted 24 lines · David Hildenbrand
Right now, pages inflated as part of a balloon driver will be dumped by dump tools like makedumpfile. While XEN is able to check in the crash kernel whether a certain pfn is actuall backed by memory in the hypervisor (see
xen_oldmem_pfn_is_ram) and optimize this case, dumps of other balloon inflated memory will essentially result in zero pages getting allocated by the hypervisor and the dump getting filled with this data.The allocation and reading of zero pages can directly be avoided if a dumping tool could know which pages only contain stale information not to be dumped.
We now have
PG_offlinewhich can be (and already is by virtio-balloon) used for marking pages as logically offline. Follow up patches will make use of this flag also in other balloon implementations.Let's export
PG_offlineviaPAGE_OFFLINE_MAPCOUNT_VALUE, so makedumpfile can directly skip pages that are logically offline and the content therefore stale.Please note that this is also helpful for a problem we were seeing under
Hyper-V: Dumping logically offline memory (pages kept fake offline while onlining a section viaonline_page_callback) would under some condicions result in a kernel panic when dumping them.
Quoted 13 lines
Cc Andrew Morton <[email protected]>Cc Dave Young <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Baoquan He <[email protected]>Cc Omar Sandoval <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Lianbo Jiang <[email protected]>Cc Borislav Petkov <[email protected]>Cc Kazuhito Hagio <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 19 lines
--- kernel/crash_core.c | 2 ++ 1 file changed, 2 insertions(+)kernel/crash_core.c +2
-0@@ -464,6 +464,8 @@ static int __init crash_save_vmcoreinfo_init(void)VMCOREINFO_NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE );#ifdef CONFIG_HUGETLB_PAGE VMCOREINFO_NUMBER (HUGETLB_PAGE_DTOR ); +#define PAGE_OFFLINE_MAPCOUNT_VALUE (~PG_offline) +VMCOREINFO_NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE );#endif arch_crash_save_vmcoreinfo ();Signature · 1 line
2.17.2
Re: [PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO
In reply to David HildenbrandQuoted 24 lines · David Hildenbrand
Right now, pages inflated as part of a balloon driver will be dumped by dump tools like makedumpfile. While XEN is able to check in the crash kernel whether a certain pfn is actuall backed by memory in the hypervisor (see
xen_oldmem_pfn_is_ram) and optimize this case, dumps of other balloon inflated memory will essentially result in zero pages getting allocated by the hypervisor and the dump getting filled with this data.The allocation and reading of zero pages can directly be avoided if a dumping tool could know which pages only contain stale information not to be dumped.
We now have
PG_offlinewhich can be (and already is by virtio-balloon) used for marking pages as logically offline. Follow up patches will make use of this flag also in other balloon implementations.Let's export
PG_offlineviaPAGE_OFFLINE_MAPCOUNT_VALUE, so makedumpfile can directly skip pages that are logically offline and the content therefore stale.Please note that this is also helpful for a problem we were seeing under
Hyper-V: Dumping logically offline memory (pages kept fake offline while onlining a section viaonline_page_callback) would under some condicions result in a kernel panic when dumping them.
Quoted 13 lines
Cc Andrew Morton <[email protected]>Cc Dave Young <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Baoquan He <[email protected]>Cc Omar Sandoval <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Lianbo Jiang <[email protected]>Cc Borislav Petkov <[email protected]>Cc Kazuhito Hagio <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 19 lines
--- kernel/crash_core.c | 2 ++ 1 file changed, 2 insertions(+)kernel/crash_core.c +2
-0@@ -464,6 +464,8 @@ static int __init crash_save_vmcoreinfo_init(void)VMCOREINFO_NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE );#ifdef CONFIG_HUGETLB_PAGE VMCOREINFO_NUMBER (HUGETLB_PAGE_DTOR ); +#define PAGE_OFFLINE_MAPCOUNT_VALUE (~PG_offline) +VMCOREINFO_NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE );#endif arch_crash_save_vmcoreinfo ();Signature · 1 line
2.17.2
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO
In reply to David HildenbrandQuoted 57 lines · David Hildenbrand
Right now, pages inflated as part of a balloon driver will be dumped by dump tools like makedumpfile. While XEN is able to check in the crash kernel whether a certain pfn is actuall backed by memory in the hypervisor (see
xen_oldmem_pfn_is_ram) and optimize this case, dumps of other balloon inflated memory will essentially result in zero pages getting allocated by the hypervisor and the dump getting filled with this data.The allocation and reading of zero pages can directly be avoided if a dumping tool could know which pages only contain stale information not to be dumped.
We now have
PG_offlinewhich can be (and already is by virtio-balloon) used for marking pages as logically offline. Follow up patches will make use of this flag also in other balloon implementations.Let's export
PG_offlineviaPAGE_OFFLINE_MAPCOUNT_VALUE, so makedumpfile can directly skip pages that are logically offline and the content therefore stale.Please note that this is also helpful for a problem we were seeing under
Hyper-V: Dumping logically offline memory (pages kept fake offline while onlining a section viaonline_page_callback) would under some condicions result in a kernel panic when dumping them.Cc Andrew Morton <[email protected]>Cc Dave Young <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Baoquan He <[email protected]>Cc Omar Sandoval <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Lianbo Jiang <[email protected]>Cc Borislav Petkov <[email protected]>Cc Kazuhito Hagio <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- kernel/crash_core.c | 2 ++ 1 file changed, 2 insertions(+)kernel/crash_core.c +2
-0@@ -464,6 +464,8 @@ static int __init crash_save_vmcoreinfo_init(void)VMCOREINFO_NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE );#ifdef CONFIG_HUGETLB_PAGE VMCOREINFO_NUMBER (HUGETLB_PAGE_DTOR ); +#define PAGE_OFFLINE_MAPCOUNT_VALUE (~PG_offline) +VMCOREINFO_NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE );#endif arch_crash_save_vmcoreinfo ();Signature · 1 line
2.17.2
Thanks
Dave
Re: [PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO
In reply to David HildenbrandQuoted 57 lines · David Hildenbrand
Right now, pages inflated as part of a balloon driver will be dumped by dump tools like makedumpfile. While XEN is able to check in the crash kernel whether a certain pfn is actuall backed by memory in the hypervisor (see
xen_oldmem_pfn_is_ram) and optimize this case, dumps of other balloon inflated memory will essentially result in zero pages getting allocated by the hypervisor and the dump getting filled with this data.The allocation and reading of zero pages can directly be avoided if a dumping tool could know which pages only contain stale information not to be dumped.
We now have
PG_offlinewhich can be (and already is by virtio-balloon) used for marking pages as logically offline. Follow up patches will make use of this flag also in other balloon implementations.Let's export
PG_offlineviaPAGE_OFFLINE_MAPCOUNT_VALUE, so makedumpfile can directly skip pages that are logically offline and the content therefore stale.Please note that this is also helpful for a problem we were seeing under
Hyper-V: Dumping logically offline memory (pages kept fake offline while onlining a section viaonline_page_callback) would under some condicions result in a kernel panic when dumping them.Cc Andrew Morton <[email protected]>Cc Dave Young <[email protected]>Cc "Kirill A. Shutemov" <[email protected]>Cc Baoquan He <[email protected]>Cc Omar Sandoval <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Cc Lianbo Jiang <[email protected]>Cc Borislav Petkov <[email protected]>Cc Kazuhito Hagio <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- kernel/crash_core.c | 2 ++ 1 file changed, 2 insertions(+)kernel/crash_core.c +2
-0@@ -464,6 +464,8 @@ static int __init crash_save_vmcoreinfo_init(void)VMCOREINFO_NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE );#ifdef CONFIG_HUGETLB_PAGE VMCOREINFO_NUMBER (HUGETLB_PAGE_DTOR ); +#define PAGE_OFFLINE_MAPCOUNT_VALUE (~PG_offline) +VMCOREINFO_NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE );#endif arch_crash_save_vmcoreinfo ();Signature · 1 line
2.17.2
Thanks
Dave
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO
In reply to David Hildenbrandkernel/crash_core.c static int __init crash_save_vmcoreinfo_init(void) · line 469
Quoted 11 lines · David Hildenbrand
kernel/crash_core.c +2
-0@@ -464,6 +464,8 @@ static int __init crash_save_vmcoreinfo_init(void)VMCOREINFO_NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE );#ifdef CONFIG_HUGETLB_PAGE VMCOREINFO_NUMBER (HUGETLB_PAGE_DTOR ); +#define PAGE_OFFLINE_MAPCOUNT_VALUE (~PG_offline) +VMCOREINFO_NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE );#endif
This solution looks good to me. One small concern is why we don't export PG_offline to vmcoreinfo directly, then define PAGE_OFFLINE_MAPCOUNT_VALUE in makedumpfile. We have been exporting kernel data/MACRO directly, why this one is exceptional.
Thanks
Baoquan
Re: [PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO
Quoted 18 lines · Baoquan He
kernel/crash_core.c static int __init crash_save_vmcoreinfo_init(void) · line 469
Quoted 11 lines · David Hildenbrand
kernel/crash_core.c +2
-0@@ -464,6 +464,8 @@ static int __init crash_save_vmcoreinfo_init(void)VMCOREINFO_NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE );#ifdef CONFIG_HUGETLB_PAGE VMCOREINFO_NUMBER (HUGETLB_PAGE_DTOR ); +#define PAGE_OFFLINE_MAPCOUNT_VALUE (~PG_offline) +VMCOREINFO_NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE );#endif This solution looks good to me. One small concern is why we don't export
PG_offlineto vmcoreinfo directly, then definePAGE_OFFLINE_MAPCOUNT_VALUEin makedumpfile. We have been exporting kernel data/MACRO directly, why this one is exceptional.
1. We are much more similar to PG_buddy (in contrast to actual page flags), and for PG_buddy it is historically handled like this (and I think it makes sense to expose these as actual MAPCOUNT_VALUEs).
2. Right now only one page type per page is supported. Therefore only exactly one value in mapcount indicates e.g. PageBuddy()/PageOffline().
Now, if we ever decide to change this (e.g. treat them like real flags), it is much easier to switch to PG_offline/PG_buddy then. We can directly see in makedumpfile that .*_MAPCOUNT_VALUE is no longer available but instead e.g. PG_offline and PG_buddy. Instead we would no see a change in makedumpfile and would have to rely on other properties.
If there are no strong opinions I will leave it like this.
Thanks!
Quoted 3 lines
Thanks
Baoquan
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO
In reply to Baoquan HeQuoted 18 lines · Baoquan He
kernel/crash_core.c static int __init crash_save_vmcoreinfo_init(void) · line 469
Quoted 11 lines · David Hildenbrand
kernel/crash_core.c +2
-0@@ -464,6 +464,8 @@ static int __init crash_save_vmcoreinfo_init(void)VMCOREINFO_NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE );#ifdef CONFIG_HUGETLB_PAGE VMCOREINFO_NUMBER (HUGETLB_PAGE_DTOR ); +#define PAGE_OFFLINE_MAPCOUNT_VALUE (~PG_offline) +VMCOREINFO_NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE );#endif This solution looks good to me. One small concern is why we don't export
PG_offlineto vmcoreinfo directly, then definePAGE_OFFLINE_MAPCOUNT_VALUEin makedumpfile. We have been exporting kernel data/MACRO directly, why this one is exceptional.
1. We are much more similar to PG_buddy (in contrast to actual page flags), and for PG_buddy it is historically handled like this (and I think it makes sense to expose these as actual MAPCOUNT_VALUEs).
2. Right now only one page type per page is supported. Therefore only exactly one value in mapcount indicates e.g. PageBuddy()/PageOffline().
Now, if we ever decide to change this (e.g. treat them like real flags), it is much easier to switch to PG_offline/PG_buddy then. We can directly see in makedumpfile that .*_MAPCOUNT_VALUE is no longer available but instead e.g. PG_offline and PG_buddy. Instead we would no see a change in makedumpfile and would have to rely on other properties.
If there are no strong opinions I will leave it like this.
Thanks!
Quoted 3 lines
Thanks
Baoquan
Signature · 8 lines
Thanks, David / dhildenb _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 3/8] kexec: export PG_offline to VMCOREINFO
In reply to David Hildenbrandkernel/crash_core.c static int __init crash_save_vmcoreinfo_init(void) · line 469
Quoted 11 lines · David Hildenbrand
kernel/crash_core.c +2
-0@@ -464,6 +464,8 @@ static int __init crash_save_vmcoreinfo_init(void)VMCOREINFO_NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE );#ifdef CONFIG_HUGETLB_PAGE VMCOREINFO_NUMBER (HUGETLB_PAGE_DTOR ); +#define PAGE_OFFLINE_MAPCOUNT_VALUE (~PG_offline) +VMCOREINFO_NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE );#endif
This solution looks good to me. One small concern is why we don't export PG_offline to vmcoreinfo directly, then define PAGE_OFFLINE_MAPCOUNT_VALUE in makedumpfile. We have been exporting kernel data/MACRO directly, why this one is exceptional.
Thanks
Baoquan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 3/8 kexec: export PG_offline to VMCOREINFO
In reply to David HildenbrandRight now, pages inflated as part of a balloon driver will be dumped by dump tools like makedumpfile. While XEN is able to check in the crash kernel whether a certain pfn is actuall backed by memory in the hypervisor (see xen_oldmem_pfn_is_ram) and optimize this case, dumps of other balloon inflated memory will essentially result in zero pages getting allocated by the hypervisor and the dump getting filled with this data.
The allocation and reading of zero pages can directly be avoided if a dumping tool could know which pages only contain stale information not to be dumped.
We now have PG_offline which can be (and already is by virtio-balloon) used for marking pages as logically offline. Follow up patches will make use of this flag also in other balloon implementations.
Let's export PG_offline via PAGE_OFFLINE_MAPCOUNT_VALUE, so makedumpfile can directly skip pages that are logically offline and the content therefore stale.
Please note that this is also helpful for a problem we were seeing under
Hyper-V: Dumping logically offline memory (pages kept fake offline while onlining a section via online_page_callback) would under some condicions result in a kernel panic when dumping them.
--- kernel/crash_core.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/crash_core.c +2 -0
@@ -464,6 +464,8 @@ static int __init crash_save_vmcoreinfo_init(void)VMCOREINFO_NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE );#ifdef CONFIG_HUGETLB_PAGE VMCOREINFO_NUMBER (HUGETLB_PAGE_DTOR ); +#define PAGE_OFFLINE_MAPCOUNT_VALUE (~PG_offline) +VMCOREINFO_NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE );#endif arch_crash_save_vmcoreinfo ();
Signature · 7 lines
2.17.2 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 4/8 xen/balloon: mark inflated pages PG_offline
In reply to David HildenbrandMark inflated and never onlined pages PG_offline, to tell the world that the content is stale and should not be dumped.
--- drivers/xen/balloon.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/xen/balloon.c +3 -0
@@ -425,6 +425,7 @@ static int xen_bring_pgs_online(struct page *pg, unsigned int order)for (i =0 ; i < size; i++) { p =pfn_to_page (start_pfn + i);__online_page_set_limits (p); +__SetPageOffline (p);__balloon_append (p); }mutex_unlock (&balloon_mutex);
@@ -493,6 +494,7 @@ static enum bp_state increase_reservation(unsigned long nr_pages)xenmem_reservation_va_mapping_update (1 , &page, &frame_list[i]);/* Relinquish the page back to the allocator. */ +__ClearPageOffline (page);free_reserved_page (page); }
@@ -519,6 +521,7 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp) state =BP_EAGAIN ;break ; } +__SetPageOffline (page);adjust_managed_page_count (page,-1 );xenmem_reservation_scrub_page (page);list_add (&page->lru, &pages);
Signature · 1 line
2.17.2
Juergen Gross
maintainerRe: [PATCH v1 4/8] xen/balloon: mark inflated pages PG_offline
Quoted 20 lines · David Hildenbrand
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc Boris Ostrovsky <[email protected]>Cc Juergen Gross <[email protected]>Cc Stefano Stabellini <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- drivers/xen/balloon.c | 3 +++ 1 file changed, 3 insertions(+)drivers/xen/balloon.c +0
-0
drivers/xen/balloon.c static int xen_bring_pgs_online(struct page *pg, unsigned int order) · line 430
Quoted 6 lines
drivers/xen/balloon.c +1
-0@@ -425,6 +425,7 @@ static int xen_bring_pgs_online(struct page *pg, unsigned int order)for (i =0 ; i < size; i++) { p =pfn_to_page (start_pfn + i);__online_page_set_limits (p); +__SetPageOffline (p);__balloon_append (p); }
This seems not to be based on current master. Could you please tell against which tree this should be reviewed?
Juergen
Re: [PATCH v1 4/8] xen/balloon: mark inflated pages PG_offline
Quoted 16 lines · Juergen Gross
Quoted 15 lines · David Hildenbrand
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc Boris Ostrovsky <[email protected]>Cc Juergen Gross <[email protected]>Cc Stefano Stabellini <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- drivers/xen/balloon.c | 3 +++ 1 file changed, 3 insertions(+)
Quoted 15 lines
drivers/xen/balloon.c static int xen_bring_pgs_online(struct page *pg, unsigned int order) · line 430
Quoted 11 lines
drivers/xen/balloon.c +1
-0@@ -425,6 +425,7 @@ static int xen_bring_pgs_online(struct page *pg, unsigned int order)for (i =0 ; i < size; i++) { p =pfn_to_page (start_pfn + i);__online_page_set_limits (p); +__SetPageOffline (p);__balloon_append (p); }This seems not to be based on current master. Could you please tell against which tree this should be reviewed?
Hi Juergen,
this is based on linux-next/master.
Thanks!
Quoted 3 lines
Juergen
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v1 4/8] xen/balloon: mark inflated pages PG_offline
In reply to Juergen GrossQuoted 16 lines · Juergen Gross
Quoted 15 lines · David Hildenbrand
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc Boris Ostrovsky <[email protected]>Cc Juergen Gross <[email protected]>Cc Stefano Stabellini <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- drivers/xen/balloon.c | 3 +++ 1 file changed, 3 insertions(+)
Quoted 15 lines
drivers/xen/balloon.c static int xen_bring_pgs_online(struct page *pg, unsigned int order) · line 430
Quoted 11 lines
drivers/xen/balloon.c +1
-0@@ -425,6 +425,7 @@ static int xen_bring_pgs_online(struct page *pg, unsigned int order)for (i =0 ; i < size; i++) { p =pfn_to_page (start_pfn + i);__online_page_set_limits (p); +__SetPageOffline (p);__balloon_append (p); }This seems not to be based on current master. Could you please tell against which tree this should be reviewed?
Hi Juergen,
this is based on linux-next/master.
Thanks!
Quoted 3 lines
Juergen
Signature · 8 lines
Thanks, David / dhildenb _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Juergen Gross
maintainerRe: [PATCH v1 4/8] xen/balloon: mark inflated pages PG_offline
In reply to David HildenbrandQuoted 20 lines · David Hildenbrand
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc Boris Ostrovsky <[email protected]>Cc Juergen Gross <[email protected]>Cc Stefano Stabellini <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- drivers/xen/balloon.c | 3 +++ 1 file changed, 3 insertions(+)drivers/xen/balloon.c +0
-0
drivers/xen/balloon.c static int xen_bring_pgs_online(struct page *pg, unsigned int order) · line 430
Quoted 6 lines
drivers/xen/balloon.c +1
-0@@ -425,6 +425,7 @@ static int xen_bring_pgs_online(struct page *pg, unsigned int order)for (i =0 ; i < size; i++) { p =pfn_to_page (start_pfn + i);__online_page_set_limits (p); +__SetPageOffline (p);__balloon_append (p); }
This seems not to be based on current master. Could you please tell against which tree this should be reviewed?
Juergen
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 4/8 xen/balloon: mark inflated pages PG_offline
In reply to David HildenbrandMark inflated and never onlined pages PG_offline, to tell the world that the content is stale and should not be dumped.
--- drivers/xen/balloon.c | 3 +++ 1 file changed, 3 insertions(+)
drivers/xen/balloon.c +3 -0
@@ -425,6 +425,7 @@ static int xen_bring_pgs_online(struct page *pg, unsigned int order)for (i =0 ; i < size; i++) { p =pfn_to_page (start_pfn + i);__online_page_set_limits (p); +__SetPageOffline (p);__balloon_append (p); }mutex_unlock (&balloon_mutex);
@@ -493,6 +494,7 @@ static enum bp_state increase_reservation(unsigned long nr_pages)xenmem_reservation_va_mapping_update (1 , &page, &frame_list[i]);/* Relinquish the page back to the allocator. */ +__ClearPageOffline (page);free_reserved_page (page); }
@@ -519,6 +521,7 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp) state =BP_EAGAIN ;break ; } +__SetPageOffline (page);adjust_managed_page_count (page,-1 );xenmem_reservation_scrub_page (page);list_add (&page->lru, &pages);
Signature · 7 lines
2.17.2 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 5/8 hv_balloon: mark inflated pages PG_offline
In reply to David HildenbrandMark inflated and never onlined pages PG_offline, to tell the world that the content is stale and should not be dumped.
--- drivers/hv/hv_balloon.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/hv/hv_balloon.c +12 -2
@@ -681,8 +681,13 @@ static struct notifier_block hv_memory_nb = {/* Check if the particular page is backed and can be onlined and online it. */ static void hv_page_online_one (struct hv_hotadd_state *has,struct page *pg) { -if (!has_pfn_is_backed (has,page_to_pfn (pg))) +if (!has_pfn_is_backed (has,page_to_pfn (pg))) { +if (!PageOffline (pg)) +__SetPageOffline (pg);return ; + } +if (PageOffline (pg)) +__ClearPageOffline (pg);/* This frame is currently backed; online the page. */ __online_page_set_limits (pg);
@@ -1201,6 +1206,7 @@ static void free_balloon_pages(struct hv_dynmem_device *dm,for (i =0 ; i < num_pages; i++) { pg =pfn_to_page (i + start_frame); +__ClearPageOffline (pg);__free_page (pg); dm->num_pages_ballooned--; }
@@ -1213,7 +1219,7 @@ static unsigned int alloc_balloon_pages(struct hv_dynmem_device *dm, struct dm_balloon_response *bl_resp, int alloc_unit) { -unsigned int i =0 ; +unsigned int i, j;struct page *pg;if (num_pages < alloc_unit)
@@ -1245,6 +1251,10 @@ static unsigned int alloc_balloon_pages(struct hv_dynmem_device *dm,if (alloc_unit !=1 )split_page (pg,get_order (alloc_unit <<PAGE_SHIFT )); +/* mark all pages offline */ +for (j =0 ; j < (1 <<get_order (alloc_unit <<PAGE_SHIFT )); j++) +__SetPageOffline (pg + j); + bl_resp->range_count++; bl_resp->range_array[i].finfo.start_page =page_to_pfn (pg);
Signature · 1 line
2.17.2
Re: [PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline
Hi David,
Quoted 39 lines
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc "K. Y. Srinivasan" <[email protected]>Cc Haiyang Zhang <[email protected]>Cc Stephen Hemminger <[email protected]>Cc Kairui Song <[email protected]>Cc Vitaly Kuznetsov <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- drivers/hv/hv_balloon.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)drivers/hv/hv_balloon.c +6
-1@@ -681,8 +681,13 @@ static struct notifier_block hv_memory_nb = {/* Check if the particular page is backed and can be onlined and online it. */ static void hv_page_online_one (struct hv_hotadd_state *has,struct page *pg) { -if (!has_pfn_is_backed (has,page_to_pfn (pg))) +if (!has_pfn_is_backed (has,page_to_pfn (pg))) { +if (!PageOffline (pg)) +__SetPageOffline (pg);return ; + } +if (PageOffline (pg)) +__ClearPageOffline (pg);/* This frame is currently backed; online the page. */ drivers/hv/hv_balloon.c +0
-0__online_page_set_limits (pg);
drivers/hv/hv_balloon.c static void free_balloon_pages(struct hv_dynmem_device · line 1210
Quoted 5 lines
drivers/hv/hv_balloon.c +1
-0@@ -1201,6 +1206,7 @@ static void free_balloon_pages(struct hv_dynmem_device *dm, for (i = 0; i < num_pages; i++) { pg =pfn_to_page (i + start_frame); +__ClearPageOffline (pg);
Just thinking, do we need to care for clearing PageOffline flag before freeing a balloon'd page?
Thanks,
Pankaj
Quoted 29 lines
drivers/hv/hv_balloon.c +0
-0__free_page (pg); dm->num_pages_ballooned--;drivers/hv/hv_balloon.c +1
-1}@@ -1213,7 +1219,7 @@ static unsigned int alloc_balloon_pages(structhv_dynmem_device *dm, struct dm_balloon_response *bl_resp, int alloc_unit) { - unsigned int i = 0; + unsigned int i, j;struct page *pg;drivers/hv/hv_balloon.c +4
-0if (num_pages < alloc_unit)@@ -1245,6 +1251,10 @@ static unsigned int alloc_balloon_pages(struct hv_dynmem_device *dm, if (alloc_unit != 1)split_page (pg,get_order (alloc_unit <<PAGE_SHIFT )); +/* mark all pages offline */ +for (j =0 ; j < (1 <<get_order (alloc_unit <<PAGE_SHIFT )); j++) +__SetPageOffline (pg + j); + bl_resp->range_count++; bl_resp->range_array[i].finfo.start_page =
page_to_pfn(pg);
--
2.17.2
Re: [PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline
Quoted 44 lines · Pankaj Gupta
Hi David,
Quoted 41 lines
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc "K. Y. Srinivasan" <[email protected]>Cc Haiyang Zhang <[email protected]>Cc Stephen Hemminger <[email protected]>Cc Kairui Song <[email protected]>Cc Vitaly Kuznetsov <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- drivers/hv/hv_balloon.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)drivers/hv/hv_balloon.c +6
-1@@ -681,8 +681,13 @@ static struct notifier_block hv_memory_nb = {/* Check if the particular page is backed and can be onlined and online it. */ static void hv_page_online_one (struct hv_hotadd_state *has,struct page *pg) { -if (!has_pfn_is_backed (has,page_to_pfn (pg))) +if (!has_pfn_is_backed (has,page_to_pfn (pg))) { +if (!PageOffline (pg)) +__SetPageOffline (pg);return ; + } +if (PageOffline (pg)) +__ClearPageOffline (pg);/* This frame is currently backed; online the page. */ drivers/hv/hv_balloon.c +0
-0__online_page_set_limits (pg);@@ -1201,6 +1206,7 @@ static void free_balloon_pages(struct hv_dynmem_device *dm,
Quoted 6 lines
drivers/hv/hv_balloon.c static void free_balloon_pages(struct hv_dynmem_device *dm, · line 1209
Quoted 3 lines
drivers/hv/hv_balloon.c +1
-0for (i =0 ; i < num_pages; i++) { pg =pfn_to_page (i + start_frame); +__ClearPageOffline (pg);Just thinking, do we need to care for clearing PageOffline flag before freeing a balloon'd page?
Yes we have to otherwise the code will crash when trying to set PageBuddy.
(only one page type at a time may be set right now, and it makes sense. A page that is offline cannot e.g. be a buddy page)
So PageOffline is completely managed by the page owner.
Quoted 3 lines
Thanks,
Pankaj
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline
Quoted 39 lines
Quoted 31 lines
Quoted 31 lines
--- drivers/hv/hv_balloon.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)drivers/hv/hv_balloon.c +6
-1@@ -681,8 +681,13 @@ static struct notifier_block hv_memory_nb = {/* Check if the particular page is backed and can be onlined and online it. */ static void hv_page_online_one (struct hv_hotadd_state *has,struct page *pg) { -if (!has_pfn_is_backed (has,page_to_pfn (pg))) +if (!has_pfn_is_backed (has,page_to_pfn (pg))) { +if (!PageOffline (pg)) +__SetPageOffline (pg);return ; + } +if (PageOffline (pg)) +__ClearPageOffline (pg);drivers/hv/hv_balloon.c +0
-0/* This frame is currently backed; online the page. */ __online_page_set_limits (pg);@@ -1201,6 +1206,7 @@ static void free_balloon_pages(struct
hv_dynmem_device
*dm,Quoted 7 lines
drivers/hv/hv_balloon.c static void free_balloon_pages(struct hv_dynmem_device *dm, · line 1209
Quoted 3 lines
drivers/hv/hv_balloon.c +1
-0for (i =0 ; i < num_pages; i++) { pg =pfn_to_page (i + start_frame); +__ClearPageOffline (pg);Just thinking, do we need to care for clearing PageOffline flag before freeing
a balloon'd page?
Quoted 4 lines
Yes we have to otherwise the code will crash when trying to set PageBuddy.
(only one page type at a time may be set right now, and it makes sense. A page that is offline cannot e.g. be a buddy page)
o.k
Quoted 2 lines
So PageOffline is completely managed by the page owner.
Makes sense. Thanks for explaining.
Pankaj
Quoted 12 lines
Quoted 4 lines
Thanks,
Pankaj--
Thanks,
David / dhildenb
Pankaj Gupta
Acked-byRe: [PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline
Quoted 41 lines
Quoted 41 lines
Quoted 31 lines
Quoted 31 lines
--- drivers/hv/hv_balloon.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)drivers/hv/hv_balloon.c +6
-1@@ -681,8 +681,13 @@ static struct notifier_block hv_memory_nb = {/* Check if the particular page is backed and can be onlined and online it. */ static void hv_page_online_one (struct hv_hotadd_state *has,struct page *pg) { -if (!has_pfn_is_backed (has,page_to_pfn (pg))) +if (!has_pfn_is_backed (has,page_to_pfn (pg))) { +if (!PageOffline (pg)) +__SetPageOffline (pg);return ; + } +if (PageOffline (pg)) +__ClearPageOffline (pg);drivers/hv/hv_balloon.c +0
-0/* This frame is currently backed; online the page. */ __online_page_set_limits (pg);@@ -1201,6 +1206,7 @@ static void free_balloon_pages(struct
hv_dynmem_device
*dm,Quoted 7 lines
drivers/hv/hv_balloon.c static void free_balloon_pages(struct hv_dynmem_device *dm, · line 1209
Quoted 3 lines
drivers/hv/hv_balloon.c +1
-0for (i =0 ; i < num_pages; i++) { pg =pfn_to_page (i + start_frame); +__ClearPageOffline (pg);Just thinking, do we need to care for clearing PageOffline flag before freeing
a balloon'd page?Yes we have to otherwise the code will crash when trying to set PageBuddy.
Quoted 8 lines
Quoted 2 lines
(only one page type at a time may be set right now, and it makes sense. A page that is offline cannot e.g. be a buddy page)
o.k
Quoted 2 lines
So PageOffline is completely managed by the page owner.
Makes sense. Thanks for explaining.
Looks good to me.
Pankaj Gupta
Acked-byRe: [PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline
In reply to Pankaj GuptaQuoted 41 lines
Quoted 41 lines
Quoted 31 lines
Quoted 31 lines
--- drivers/hv/hv_balloon.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)drivers/hv/hv_balloon.c +6
-1@@ -681,8 +681,13 @@ static struct notifier_block hv_memory_nb = {/* Check if the particular page is backed and can be onlined and online it. */ static void hv_page_online_one (struct hv_hotadd_state *has,struct page *pg) { -if (!has_pfn_is_backed (has,page_to_pfn (pg))) +if (!has_pfn_is_backed (has,page_to_pfn (pg))) { +if (!PageOffline (pg)) +__SetPageOffline (pg);return ; + } +if (PageOffline (pg)) +__ClearPageOffline (pg);drivers/hv/hv_balloon.c +0
-0/* This frame is currently backed; online the page. */ __online_page_set_limits (pg);@@ -1201,6 +1206,7 @@ static void free_balloon_pages(struct
hv_dynmem_device
*dm,Quoted 7 lines
drivers/hv/hv_balloon.c static void free_balloon_pages(struct hv_dynmem_device *dm, · line 1209
Quoted 3 lines
drivers/hv/hv_balloon.c +1
-0for (i =0 ; i < num_pages; i++) { pg =pfn_to_page (i + start_frame); +__ClearPageOffline (pg);Just thinking, do we need to care for clearing PageOffline flag before freeing
a balloon'd page?Yes we have to otherwise the code will crash when trying to set PageBuddy.
Quoted 8 lines
Quoted 2 lines
(only one page type at a time may be set right now, and it makes sense. A page that is offline cannot e.g. be a buddy page)
o.k
Quoted 2 lines
So PageOffline is completely managed by the page owner.
Makes sense. Thanks for explaining.
Looks good to me.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline
In reply to David HildenbrandQuoted 39 lines
Quoted 31 lines
Quoted 31 lines
--- drivers/hv/hv_balloon.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)drivers/hv/hv_balloon.c +6
-1@@ -681,8 +681,13 @@ static struct notifier_block hv_memory_nb = {/* Check if the particular page is backed and can be onlined and online it. */ static void hv_page_online_one (struct hv_hotadd_state *has,struct page *pg) { -if (!has_pfn_is_backed (has,page_to_pfn (pg))) +if (!has_pfn_is_backed (has,page_to_pfn (pg))) { +if (!PageOffline (pg)) +__SetPageOffline (pg);return ; + } +if (PageOffline (pg)) +__ClearPageOffline (pg);drivers/hv/hv_balloon.c +0
-0/* This frame is currently backed; online the page. */ __online_page_set_limits (pg);@@ -1201,6 +1206,7 @@ static void free_balloon_pages(struct
hv_dynmem_device
*dm,Quoted 7 lines
drivers/hv/hv_balloon.c static void free_balloon_pages(struct hv_dynmem_device *dm, · line 1209
Quoted 3 lines
drivers/hv/hv_balloon.c +1
-0for (i =0 ; i < num_pages; i++) { pg =pfn_to_page (i + start_frame); +__ClearPageOffline (pg);Just thinking, do we need to care for clearing PageOffline flag before freeing
a balloon'd page?
Quoted 4 lines
Yes we have to otherwise the code will crash when trying to set PageBuddy.
(only one page type at a time may be set right now, and it makes sense. A page that is offline cannot e.g. be a buddy page)
o.k
Quoted 2 lines
So PageOffline is completely managed by the page owner.
Makes sense. Thanks for explaining.
Pankaj
Quoted 6 lines
Quoted 4 lines
Thanks,
Pankaj
Quoted 6 lines
--
Thanks,
David / dhildenb
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline
In reply to Pankaj GuptaQuoted 44 lines · Pankaj Gupta
Hi David,
Quoted 41 lines
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc "K. Y. Srinivasan" <[email protected]>Cc Haiyang Zhang <[email protected]>Cc Stephen Hemminger <[email protected]>Cc Kairui Song <[email protected]>Cc Vitaly Kuznetsov <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- drivers/hv/hv_balloon.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)drivers/hv/hv_balloon.c +6
-1@@ -681,8 +681,13 @@ static struct notifier_block hv_memory_nb = {/* Check if the particular page is backed and can be onlined and online it. */ static void hv_page_online_one (struct hv_hotadd_state *has,struct page *pg) { -if (!has_pfn_is_backed (has,page_to_pfn (pg))) +if (!has_pfn_is_backed (has,page_to_pfn (pg))) { +if (!PageOffline (pg)) +__SetPageOffline (pg);return ; + } +if (PageOffline (pg)) +__ClearPageOffline (pg);/* This frame is currently backed; online the page. */ drivers/hv/hv_balloon.c +0
-0__online_page_set_limits (pg);@@ -1201,6 +1206,7 @@ static void free_balloon_pages(struct hv_dynmem_device *dm,
Quoted 6 lines
drivers/hv/hv_balloon.c static void free_balloon_pages(struct hv_dynmem_device *dm, · line 1209
Quoted 3 lines
drivers/hv/hv_balloon.c +1
-0for (i =0 ; i < num_pages; i++) { pg =pfn_to_page (i + start_frame); +__ClearPageOffline (pg);Just thinking, do we need to care for clearing PageOffline flag before freeing a balloon'd page?
Yes we have to otherwise the code will crash when trying to set PageBuddy.
(only one page type at a time may be set right now, and it makes sense. A page that is offline cannot e.g. be a buddy page)
So PageOffline is completely managed by the page owner.
Quoted 3 lines
Thanks,
Pankaj
Signature · 8 lines
Thanks, David / dhildenb _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 5/8] hv_balloon: mark inflated pages PG_offline
In reply to David HildenbrandHi David,
Quoted 39 lines
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc "K. Y. Srinivasan" <[email protected]>Cc Haiyang Zhang <[email protected]>Cc Stephen Hemminger <[email protected]>Cc Kairui Song <[email protected]>Cc Vitaly Kuznetsov <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>--- drivers/hv/hv_balloon.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)drivers/hv/hv_balloon.c +6
-1@@ -681,8 +681,13 @@ static struct notifier_block hv_memory_nb = {/* Check if the particular page is backed and can be onlined and online it. */ static void hv_page_online_one (struct hv_hotadd_state *has,struct page *pg) { -if (!has_pfn_is_backed (has,page_to_pfn (pg))) +if (!has_pfn_is_backed (has,page_to_pfn (pg))) { +if (!PageOffline (pg)) +__SetPageOffline (pg);return ; + } +if (PageOffline (pg)) +__ClearPageOffline (pg);/* This frame is currently backed; online the page. */ drivers/hv/hv_balloon.c +0
-0__online_page_set_limits (pg);
drivers/hv/hv_balloon.c static void free_balloon_pages(struct hv_dynmem_device · line 1210
Quoted 5 lines
drivers/hv/hv_balloon.c +1
-0@@ -1201,6 +1206,7 @@ static void free_balloon_pages(struct hv_dynmem_device *dm, for (i = 0; i < num_pages; i++) { pg =pfn_to_page (i + start_frame); +__ClearPageOffline (pg);
Just thinking, do we need to care for clearing PageOffline flag before freeing a balloon'd page?
Thanks,
Pankaj
Quoted 24 lines
drivers/hv/hv_balloon.c +0
-0__free_page (pg); dm->num_pages_ballooned--;drivers/hv/hv_balloon.c +1
-1}@@ -1213,7 +1219,7 @@ static unsigned int alloc_balloon_pages(structhv_dynmem_device *dm, struct dm_balloon_response *bl_resp, int alloc_unit) { - unsigned int i = 0; + unsigned int i, j;struct page *pg;drivers/hv/hv_balloon.c +4
-0if (num_pages < alloc_unit)@@ -1245,6 +1251,10 @@ static unsigned int alloc_balloon_pages(struct hv_dynmem_device *dm, if (alloc_unit != 1)split_page (pg,get_order (alloc_unit <<PAGE_SHIFT )); +/* mark all pages offline */ +for (j =0 ; j < (1 <<get_order (alloc_unit <<PAGE_SHIFT )); j++) +__SetPageOffline (pg + j); + bl_resp->range_count++; bl_resp->range_array[i].finfo.start_page =
Quoted 5 lines
page_to_pfn(pg);
--
2.17.2
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 5/8 hv_balloon: mark inflated pages PG_offline
In reply to David HildenbrandMark inflated and never onlined pages PG_offline, to tell the world that the content is stale and should not be dumped.
--- drivers/hv/hv_balloon.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
drivers/hv/hv_balloon.c +12 -2
@@ -681,8 +681,13 @@ static struct notifier_block hv_memory_nb = {/* Check if the particular page is backed and can be onlined and online it. */ static void hv_page_online_one (struct hv_hotadd_state *has,struct page *pg) { -if (!has_pfn_is_backed (has,page_to_pfn (pg))) +if (!has_pfn_is_backed (has,page_to_pfn (pg))) { +if (!PageOffline (pg)) +__SetPageOffline (pg);return ; + } +if (PageOffline (pg)) +__ClearPageOffline (pg);/* This frame is currently backed; online the page. */ __online_page_set_limits (pg);
@@ -1201,6 +1206,7 @@ static void free_balloon_pages(struct hv_dynmem_device *dm,for (i =0 ; i < num_pages; i++) { pg =pfn_to_page (i + start_frame); +__ClearPageOffline (pg);__free_page (pg); dm->num_pages_ballooned--; }
@@ -1213,7 +1219,7 @@ static unsigned int alloc_balloon_pages(struct hv_dynmem_device *dm, struct dm_balloon_response *bl_resp, int alloc_unit) { -unsigned int i =0 ; +unsigned int i, j;struct page *pg;if (num_pages < alloc_unit)
@@ -1245,6 +1251,10 @@ static unsigned int alloc_balloon_pages(struct hv_dynmem_device *dm,if (alloc_unit !=1 )split_page (pg,get_order (alloc_unit <<PAGE_SHIFT )); +/* mark all pages offline */ +for (j =0 ; j < (1 <<get_order (alloc_unit <<PAGE_SHIFT )); j++) +__SetPageOffline (pg + j); + bl_resp->range_count++; bl_resp->range_array[i].finfo.start_page =page_to_pfn (pg);
Signature · 7 lines
2.17.2 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 6/8 vmw_balloon: mark inflated pages PG_offline
In reply to David HildenbrandMark inflated and never onlined pages PG_offline, to tell the world that the content is stale and should not be dumped.
--- drivers/misc/vmw_balloon.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
drivers/misc/vmw_balloon.c +32 -0
@@ -544,6 +544,36 @@ unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size)return page_size ==VMW_BALLOON_2M_PAGE ?VMW_BALLOON_2M_ORDER :0 ; } +/** +* vmballoon_mark_page_offline() - mark a page as offline +* @page: pointer for the page +* @page_size: the size of the page. +*/ +static void +vmballoon_mark_page_offline (struct page *page, +enum vmballoon_page_size_type page_size) +{ +int i; + +for (i =0 ; i <1ULL <<vmballoon_page_order (page_size); i++) +__SetPageOffline (page + i); +} + +/** +* vmballoon_mark_page_online() - mark a page as online +* @page: pointer for the page +* @page_size: the size of the page. +*/ +static void +vmballoon_mark_page_online (struct page *page, +enum vmballoon_page_size_type page_size) +{ +int i; + +for (i =0 ; i <1ULL <<vmballoon_page_order (page_size); i++) +__ClearPageOffline (page + i); +} +/** * vmballoon_page_in_frames() - returns the number of frames in a page. * @page_size: the size of the page.
@@ -612,6 +642,7 @@ static int vmballoon_alloc_page_list(struct vmballoon *b, ctl->page_size);if (page) { +vmballoon_mark_page_offline (page, ctl->page_size);/* Success. Add the page to the list and continue. */ list_add (&page->lru, &ctl->pages);continue ;
@@ -850,6 +881,7 @@ static void vmballoon_release_page_list(struct list_head *page_list,list_for_each_entry_safe (page, tmp, page_list, lru) {list_del (&page->lru); +vmballoon_mark_page_online (page, page_size);__free_pages (page,vmballoon_page_order (page_size)); }
Signature · 1 line
2.17.2
Re: [PATCH v1 6/8] vmw_balloon: mark inflated pages PG_offline
Thanks for this patch!
Quoted 24 lines · David Hildenbrand
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc Xavier Deguillard <[email protected]>Cc Nadav Amit <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Julien Freche <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
drivers/misc/vmw_balloon.c| 32 ++++++++++++++++++++++++++++++++1 file changed, 32 insertions(+)drivers/misc/vmw_balloon.c +0
-0
drivers/misc/vmw_balloon.c unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size) · line 549
Quoted 6 lines
drivers/misc/vmw_balloon.c +3
-0@@ -544,6 +544,36 @@ unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size)return page_size ==VMW_BALLOON_2M_PAGE ?VMW_BALLOON_2M_ORDER :0 ; } +/** +* vmballoon_mark_page_offline() - mark a page as offline +* @page: pointer for the page
If possible, please add a period at the end of the sentence (yes, I know I got it wrong in some places too).
drivers/misc/vmw_balloon.c unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size) · line 558
Quoted 9 lines
drivers/misc/vmw_balloon.c +9
-0+* @page_size: the size of the page. +*/ +static void +vmballoon_mark_page_offline (struct page *page, +enum vmballoon_page_size_type page_size) +{ +int i; + +for (i =0 ; i <1ULL <<vmballoon_page_order (page_size); i++)
Can you please do instead:
unsigned int; for (i = 0; i < vmballoon_page_in_frames(page_size); i++)
Quoted 6 lines
drivers/misc/vmw_balloon.c +6
-0+__SetPageOffline (page + i); +} + +/** +* vmballoon_mark_page_online() - mark a page as online +* @page: pointer for the page
drivers/misc/vmw_balloon.c unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size) · line 574
Quoted 10 lines
drivers/misc/vmw_balloon.c +10
-0+* @page_size: the size of the page. +*/ +static void +vmballoon_mark_page_online (struct page *page, +enum vmballoon_page_size_type page_size) +{ +int i; + +for (i =0 ; i <1ULL <<vmballoon_page_order (page_size); i++) +__ClearPageOffline (page + i);
Same here (use vmballoon_page_in_frames).
Quoted 14 lines
drivers/misc/vmw_balloon.c +3
-0+} +/** * vmballoon_page_in_frames() - returns the number of frames in a page. * @page_size: the size of the page. @@ -612,6 +642,7 @@ static int vmballoon_alloc_page_list(struct vmballoon *b, ctl->page_size);if (page) { +vmballoon_mark_page_offline (page, ctl->page_size);/* Success. Add the page to the list and continue. */ list_add (&page->lru, &ctl->pages);continue ;
drivers/misc/vmw_balloon.c static void vmballoon_release_page_list(struct list_head *page_list, · line 886
Quoted 6 lines
drivers/misc/vmw_balloon.c +1
-0@@ -850,6 +881,7 @@ static void vmballoon_release_page_list(struct list_head *page_list,list_for_each_entry_safe (page, tmp, page_list, lru) {list_del (&page->lru); +vmballoon_mark_page_online (page, page_size);__free_pages (page,vmballoon_page_order (page_size)); }
We would like to test it in the next few days, but in the meanwhile, after you address these minor issues:
Thanks again,
Nadav
Re: [PATCH v1 6/8] vmw_balloon: mark inflated pages PG_offline
Quoted 29 lines · Nadav Amit
Thanks for this patch!
Quoted 27 lines · David Hildenbrand
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc Xavier Deguillard <[email protected]>Cc Nadav Amit <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Julien Freche <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
drivers/misc/vmw_balloon.c| 32 ++++++++++++++++++++++++++++++++1 file changed, 32 insertions(+)drivers/misc/vmw_balloon.c +0
-0@@ -544,6 +544,36 @@ unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size)return page_size ==VMW_BALLOON_2M_PAGE ?VMW_BALLOON_2M_ORDER :0 ; }
Quoted 6 lines
drivers/misc/vmw_balloon.c unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size) · line 549
Quoted 3 lines
drivers/misc/vmw_balloon.c +3
-0+/** +* vmballoon_mark_page_offline() - mark a page as offline +* @page: pointer for the page If possible, please add a period at the end of the sentence (yes, I know I got it wrong in some places too).
Sure :)
Quoted 13 lines
drivers/misc/vmw_balloon.c unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size) · line 558
Quoted 10 lines
drivers/misc/vmw_balloon.c +9
-0+* @page_size: the size of the page. +*/ +static void +vmballoon_mark_page_offline (struct page *page, +enum vmballoon_page_size_type page_size) +{ +int i; + +for (i =0 ; i <1ULL <<vmballoon_page_order (page_size); i++)Can you please do instead:
Quoted 4 lines
unsigned int; for (i = 0; i < vmballoon_page_in_frames(page_size); i++)
Will do, will have to move both functions a little bit down in the file (exactly one function).
Quoted 4 lines
We would like to test it in the next few days, but in the meanwhile, after you address these minor issues:
Acked-by Nadav Amit <[email protected]>
Thanks!
Quoted 4 lines
Thanks again,
Nadav
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v1 6/8] vmw_balloon: mark inflated pages PG_offline
In reply to Nadav AmitQuoted 29 lines · Nadav Amit
Thanks for this patch!
Quoted 27 lines · David Hildenbrand
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc Xavier Deguillard <[email protected]>Cc Nadav Amit <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Julien Freche <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
drivers/misc/vmw_balloon.c| 32 ++++++++++++++++++++++++++++++++1 file changed, 32 insertions(+)drivers/misc/vmw_balloon.c +0
-0@@ -544,6 +544,36 @@ unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size)return page_size ==VMW_BALLOON_2M_PAGE ?VMW_BALLOON_2M_ORDER :0 ; }
Quoted 6 lines
drivers/misc/vmw_balloon.c unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size) · line 549
Quoted 3 lines
drivers/misc/vmw_balloon.c +3
-0+/** +* vmballoon_mark_page_offline() - mark a page as offline +* @page: pointer for the page If possible, please add a period at the end of the sentence (yes, I know I got it wrong in some places too).
Sure :)
Quoted 13 lines
drivers/misc/vmw_balloon.c unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size) · line 558
Quoted 10 lines
drivers/misc/vmw_balloon.c +9
-0+* @page_size: the size of the page. +*/ +static void +vmballoon_mark_page_offline (struct page *page, +enum vmballoon_page_size_type page_size) +{ +int i; + +for (i =0 ; i <1ULL <<vmballoon_page_order (page_size); i++)Can you please do instead:
Quoted 4 lines
unsigned int; for (i = 0; i < vmballoon_page_in_frames(page_size); i++)
Will do, will have to move both functions a little bit down in the file (exactly one function).
Quoted 4 lines
We would like to test it in the next few days, but in the meanwhile, after you address these minor issues:
Acked-by Nadav Amit <[email protected]>
Thanks!
Quoted 4 lines
Thanks again,
Nadav
Signature · 8 lines
Thanks, David / dhildenb _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 6/8] vmw_balloon: mark inflated pages PG_offline
In reply to David HildenbrandThanks for this patch!
Quoted 24 lines · David Hildenbrand
Mark inflated and never onlined pages
PG_offline, to tell the world that the content is stale and should not be dumped.Cc Xavier Deguillard <[email protected]>Cc Nadav Amit <[email protected]>Cc Arnd Bergmann <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Julien Freche <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Signed-off-by David Hildenbrand <[email protected]>---
drivers/misc/vmw_balloon.c| 32 ++++++++++++++++++++++++++++++++1 file changed, 32 insertions(+)drivers/misc/vmw_balloon.c +0
-0
drivers/misc/vmw_balloon.c unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size) · line 549
Quoted 6 lines
drivers/misc/vmw_balloon.c +3
-0@@ -544,6 +544,36 @@ unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size)return page_size ==VMW_BALLOON_2M_PAGE ?VMW_BALLOON_2M_ORDER :0 ; } +/** +* vmballoon_mark_page_offline() - mark a page as offline +* @page: pointer for the page
If possible, please add a period at the end of the sentence (yes, I know I got it wrong in some places too).
drivers/misc/vmw_balloon.c unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size) · line 558
Quoted 9 lines
drivers/misc/vmw_balloon.c +9
-0+* @page_size: the size of the page. +*/ +static void +vmballoon_mark_page_offline (struct page *page, +enum vmballoon_page_size_type page_size) +{ +int i; + +for (i =0 ; i <1ULL <<vmballoon_page_order (page_size); i++)
Can you please do instead:
unsigned int; for (i = 0; i < vmballoon_page_in_frames(page_size); i++)
Quoted 6 lines
drivers/misc/vmw_balloon.c +6
-0+__SetPageOffline (page + i); +} + +/** +* vmballoon_mark_page_online() - mark a page as online +* @page: pointer for the page
drivers/misc/vmw_balloon.c unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size) · line 574
Quoted 10 lines
drivers/misc/vmw_balloon.c +10
-0+* @page_size: the size of the page. +*/ +static void +vmballoon_mark_page_online (struct page *page, +enum vmballoon_page_size_type page_size) +{ +int i; + +for (i =0 ; i <1ULL <<vmballoon_page_order (page_size); i++) +__ClearPageOffline (page + i);
Same here (use vmballoon_page_in_frames).
Quoted 14 lines
drivers/misc/vmw_balloon.c +3
-0+} +/** * vmballoon_page_in_frames() - returns the number of frames in a page. * @page_size: the size of the page. @@ -612,6 +642,7 @@ static int vmballoon_alloc_page_list(struct vmballoon *b, ctl->page_size);if (page) { +vmballoon_mark_page_offline (page, ctl->page_size);/* Success. Add the page to the list and continue. */ list_add (&page->lru, &ctl->pages);continue ;
drivers/misc/vmw_balloon.c static void vmballoon_release_page_list(struct list_head *page_list, · line 886
Quoted 6 lines
drivers/misc/vmw_balloon.c +1
-0@@ -850,6 +881,7 @@ static void vmballoon_release_page_list(struct list_head *page_list,list_for_each_entry_safe (page, tmp, page_list, lru) {list_del (&page->lru); +vmballoon_mark_page_online (page, page_size);__free_pages (page,vmballoon_page_order (page_size)); }
We would like to test it in the next few days, but in the meanwhile, after you address these minor issues:
Thanks again,
Nadav
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 6/8 vmw_balloon: mark inflated pages PG_offline
In reply to David HildenbrandMark inflated and never onlined pages PG_offline, to tell the world that the content is stale and should not be dumped.
--- drivers/misc/vmw_balloon.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
drivers/misc/vmw_balloon.c +32 -0
@@ -544,6 +544,36 @@ unsigned int vmballoon_page_order(enum vmballoon_page_size_type page_size)return page_size ==VMW_BALLOON_2M_PAGE ?VMW_BALLOON_2M_ORDER :0 ; } +/** +* vmballoon_mark_page_offline() - mark a page as offline +* @page: pointer for the page +* @page_size: the size of the page. +*/ +static void +vmballoon_mark_page_offline (struct page *page, +enum vmballoon_page_size_type page_size) +{ +int i; + +for (i =0 ; i <1ULL <<vmballoon_page_order (page_size); i++) +__SetPageOffline (page + i); +} + +/** +* vmballoon_mark_page_online() - mark a page as online +* @page: pointer for the page +* @page_size: the size of the page. +*/ +static void +vmballoon_mark_page_online (struct page *page, +enum vmballoon_page_size_type page_size) +{ +int i; + +for (i =0 ; i <1ULL <<vmballoon_page_order (page_size); i++) +__ClearPageOffline (page + i); +} +/** * vmballoon_page_in_frames() - returns the number of frames in a page. * @page_size: the size of the page.
@@ -612,6 +642,7 @@ static int vmballoon_alloc_page_list(struct vmballoon *b, ctl->page_size);if (page) { +vmballoon_mark_page_offline (page, ctl->page_size);/* Success. Add the page to the list and continue. */ list_add (&page->lru, &ctl->pages);continue ;
@@ -850,6 +881,7 @@ static void vmballoon_release_page_list(struct list_head *page_list,list_for_each_entry_safe (page, tmp, page_list, lru) {list_del (&page->lru); +vmballoon_mark_page_online (page, page_size);__free_pages (page,vmballoon_page_order (page_size)); }
Signature · 7 lines
2.17.2 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 7/8 PM / Hibernate: use pfn_to_online_page()
In reply to David HildenbrandLet's use pfn_to_online_page() instead of pfn_to_page() when checking for saveable pages to not save/restore offline memory sections.
--- kernel/power/snapshot.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/power/snapshot.c +4 -4
@@ -1215,8 +1215,8 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (!PageHighMem (page));
@@ -1277,8 +1277,8 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (PageHighMem (page));
Signature · 1 line
2.17.2
Re: [PATCH v1 7/8] PM / Hibernate: use pfn_to_online_page()
Quoted 12 lines · David Hildenbrand
Let's use
pfn_to_online_page()instead ofpfn_to_page()when checking for saveable pages to not save/restore offline memory sections.Cc "Rafael J. Wysocki" <[email protected]>Cc Pavel Machek <[email protected]>Cc Len Brown <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Suggested-by Michal Hocko <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
I have only a very vague understanding of this specific code but I do not really see any real reason for checking offlined ranges. Also offline pfn ranges might have uninitialized struct pages so making any decisions of the struct page is basically undefined behavior.
Quoted 32 lines
--- kernel/power/snapshot.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)kernel/power/snapshot.c +4
-4@@ -1215,8 +1215,8 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (!PageHighMem (page));@@ -1277,8 +1277,8 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (PageHighMem (page));Signature · 1 line
2.17.2
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v1 7/8] PM / Hibernate: use pfn_to_online_page()
In reply to David HildenbrandQuoted 12 lines · David Hildenbrand
Let's use
pfn_to_online_page()instead ofpfn_to_page()when checking for saveable pages to not save/restore offline memory sections.Cc "Rafael J. Wysocki" <[email protected]>Cc Pavel Machek <[email protected]>Cc Len Brown <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Suggested-by Michal Hocko <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
I have only a very vague understanding of this specific code but I do not really see any real reason for checking offlined ranges. Also offline pfn ranges might have uninitialized struct pages so making any decisions of the struct page is basically undefined behavior.
Quoted 32 lines
--- kernel/power/snapshot.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)kernel/power/snapshot.c +4
-4@@ -1215,8 +1215,8 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (!PageHighMem (page));@@ -1277,8 +1277,8 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (PageHighMem (page));Signature · 1 line
2.17.2
Signature · 7 lines
Michal Hocko SUSE Labs _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 7/8] PM / Hibernate: use pfn_to_online_page()
In reply to David HildenbrandQuoted 4 lines · David Hildenbrand
Let's use
pfn_to_online_page()instead ofpfn_to_page()when checking for saveable pages to not save/restore offline memory sections.Cc "Rafael J. Wysocki" <[email protected]>
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Re: [PATCH v1 7/8] PM / Hibernate: use pfn_to_online_page()
In reply to David HildenbrandQuoted 4 lines · David Hildenbrand
Let's use
pfn_to_online_page()instead ofpfn_to_page()when checking for saveable pages to not save/restore offline memory sections.Cc "Rafael J. Wysocki" <[email protected]>
Signature · 2 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 7/8] PM / Hibernate: use pfn_to_online_page()
In reply to David HildenbrandQuoted 12 lines · CET David Hildenbrand
Let's use
pfn_to_online_page()instead ofpfn_to_page()when checking for saveable pages to not save/restore offline memory sections.Cc "Rafael J. Wysocki" <[email protected]>Cc Pavel Machek <[email protected]>Cc Len Brown <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Suggested-by Michal Hocko <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 31 lines
--- kernel/power/snapshot.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)kernel/power/snapshot.c +4
-4@@ -1215,8 +1215,8 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (!PageHighMem (page));@@ -1277,8 +1277,8 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (PageHighMem (page));
Re: [PATCH v1 7/8] PM / Hibernate: use pfn_to_online_page()
In reply to David HildenbrandQuoted 12 lines · CET David Hildenbrand
Let's use
pfn_to_online_page()instead ofpfn_to_page()when checking for saveable pages to not save/restore offline memory sections.Cc "Rafael J. Wysocki" <[email protected]>Cc Pavel Machek <[email protected]>Cc Len Brown <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Suggested-by Michal Hocko <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 21 lines
--- kernel/power/snapshot.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)kernel/power/snapshot.c +2
-2@@ -1215,8 +1215,8 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (!PageHighMem (page));
kernel/power/snapshot.c static struct page *saveable_page(struct zone *zone, unsigned long pfn) · line 1284
Quoted 10 lines
kernel/power/snapshot.c +2
-2@@ -1277,8 +1277,8 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (PageHighMem (page));
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 7/8 PM / Hibernate: use pfn_to_online_page()
In reply to David HildenbrandLet's use pfn_to_online_page() instead of pfn_to_page() when checking for saveable pages to not save/restore offline memory sections.
--- kernel/power/snapshot.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/power/snapshot.c +4 -4
@@ -1215,8 +1215,8 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (!PageHighMem (page));
@@ -1277,8 +1277,8 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (!pfn_valid (pfn))return NULL ; - page =pfn_to_page (pfn); -if (page_zone (page) != zone) + page =pfn_to_online_page (pfn); +if (!page ||page_zone (page) != zone)return NULL ;BUG_ON (PageHighMem (page));
Signature · 7 lines
2.17.2 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 8/8 PM / Hibernate: exclude all PageOffline() pages
In reply to David HildenbrandThe content of pages that are marked PG_offline is not of interest (e.g. inflated by a balloon driver), let's skip these pages.
--- kernel/power/snapshot.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
kernel/power/snapshot.c +4 -1
@@ -1222,7 +1222,7 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)BUG_ON (!PageHighMem (page));if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page) || -PageReserved (page)) +PageReserved (page) ||PageOffline (page))return NULL ;if (page_is_guard (page))
@@ -1286,6 +1286,9 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page))return NULL ; +if (PageOffline (page)) +return NULL ; +if (PageReserved (page) && (!kernel_page_present (page) ||pfn_is_nosave (pfn)))return NULL ;
Signature · 1 line
2.17.2
Re: [PATCH v1 8/8] PM / Hibernate: exclude all PageOffline() pages
Quoted 12 lines · CET David Hildenbrand
The content of pages that are marked
PG_offlineis not of interest (e.g. inflated by a balloon driver), let's skip these pages.Cc "Rafael J. Wysocki" <[email protected]>Cc Pavel Machek <[email protected]>Cc Len Brown <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Acked-by Pavel Machek <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 28 lines
--- kernel/power/snapshot.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)kernel/power/snapshot.c +4
-1@@ -1222,7 +1222,7 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)BUG_ON (!PageHighMem (page));if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page) || -PageReserved (page)) +PageReserved (page) ||PageOffline (page))return NULL ;if (page_is_guard (page))@@ -1286,6 +1286,9 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page))return NULL ; +if (PageOffline (page)) +return NULL ; +if (PageReserved (page) && (!kernel_page_present (page) ||pfn_is_nosave (pfn)))return NULL ;
Re: [PATCH v1 8/8] PM / Hibernate: exclude all PageOffline() pages
In reply to David HildenbrandQuoted 12 lines · CET David Hildenbrand
The content of pages that are marked
PG_offlineis not of interest (e.g. inflated by a balloon driver), let's skip these pages.Cc "Rafael J. Wysocki" <[email protected]>Cc Pavel Machek <[email protected]>Cc Len Brown <[email protected]>Cc Andrew Morton <[email protected]>Cc Matthew Wilcox <[email protected]>Cc Michal Hocko <[email protected]>Cc "Michael S. Tsirkin" <[email protected]>Acked-by Pavel Machek <[email protected]>Signed-off-by David Hildenbrand <[email protected]>
Quoted 19 lines
--- kernel/power/snapshot.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)kernel/power/snapshot.c +1
-1@@ -1222,7 +1222,7 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)BUG_ON (!PageHighMem (page));if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page) || -PageReserved (page)) +PageReserved (page) ||PageOffline (page))return NULL ;if (page_is_guard (page))
kernel/power/snapshot.c static struct page *saveable_page(struct zone *zone, unsigned long pfn) · line 1294
Quoted 9 lines
kernel/power/snapshot.c +3
-0@@ -1286,6 +1286,9 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page))return NULL ; +if (PageOffline (page)) +return NULL ; +if (PageReserved (page) && (!kernel_page_present (page) ||pfn_is_nosave (pfn)))return NULL ;
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 8/8] PM / Hibernate: exclude all PageOffline() pages
In reply to David HildenbrandIf you are adding PageOffline(page) to the condition list of the already existing if in saveable_highmem_page(), why explicitly add it as a separate statement in saveable_page()?
It would seem more consistent to make the second check:
- if (swsusp_page_is_forbidden(page) || swsusp_page_is_free(page)) + if (swsusp_page_is_forbidden(page) || swsusp_page_is_free(page) ||
+ PageOffline(page))
instead.
It's admittedly a nit but it just seems cleaner to either do that or, if your intention was to separate the Page checks from the swsusp checks, to break the calls to PageReserved() and PageOffline() into their own check in saveable_highmem_page().
Thanks!
-- Bill
Quoted 26 lines · David Hildenbrand
kernel/power/snapshot.c +4
-1@@ -1222,7 +1222,7 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)BUG_ON (!PageHighMem (page));if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page) || -PageReserved (page)) +PageReserved (page) ||PageOffline (page))return NULL ;if (page_is_guard (page))@@ -1286,6 +1286,9 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page))return NULL ; +if (PageOffline (page)) +return NULL ; +if (PageReserved (page) && (!kernel_page_present (page) ||pfn_is_nosave (pfn)))return NULL ;Signature · 1 line
2.17.2
Re: [PATCH v1 8/8] PM / Hibernate: exclude all PageOffline() pages
Quoted 9 lines · William Kucharski
If you are adding PageOffline(page) to the condition list of the already existing if in
saveable_highmem_page(), why explicitly add it as a separate statement insaveable_page()?It would seem more consistent to make the second check:
(quoted diff) +2
-1- if (swsusp_page_is_forbidden(page) || swsusp_page_is_free(page)) + if (swsusp_page_is_forbidden(page) || swsusp_page_is_free(page) || + PageOffline(page))
Quoted 5 lines
instead.
It's admittedly a nit but it just seems cleaner to either do that or, if your intention was to separate the Page checks from the swsusp checks, to break the calls to
PageReserved()andPageOffline()into their own check insaveable_highmem_page().
I'll split PageReserved() and PageOffline() off from the swsusp checks, thanks for your comment!
Quoted 3 lines
Thanks!
-- Bill
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v1 8/8] PM / Hibernate: exclude all PageOffline() pages
In reply to William KucharskiQuoted 9 lines · William Kucharski
If you are adding PageOffline(page) to the condition list of the already existing if in
saveable_highmem_page(), why explicitly add it as a separate statement insaveable_page()?It would seem more consistent to make the second check:
(quoted diff) +2
-1- if (swsusp_page_is_forbidden(page) || swsusp_page_is_free(page)) + if (swsusp_page_is_forbidden(page) || swsusp_page_is_free(page) || + PageOffline(page))
Quoted 5 lines
instead.
It's admittedly a nit but it just seems cleaner to either do that or, if your intention was to separate the Page checks from the swsusp checks, to break the calls to
PageReserved()andPageOffline()into their own check insaveable_highmem_page().
I'll split PageReserved() and PageOffline() off from the swsusp checks, thanks for your comment!
Quoted 3 lines
Thanks!
-- Bill
Signature · 8 lines
Thanks, David / dhildenb _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1 8/8] PM / Hibernate: exclude all PageOffline() pages
In reply to David HildenbrandIf you are adding PageOffline(page) to the condition list of the already existing if in saveable_highmem_page(), why explicitly add it as a separate statement in saveable_page()?
It would seem more consistent to make the second check:
- if (swsusp_page_is_forbidden(page) || swsusp_page_is_free(page)) + if (swsusp_page_is_forbidden(page) || swsusp_page_is_free(page) ||
+ PageOffline(page))
instead.
It's admittedly a nit but it just seems cleaner to either do that or, if your intention was to separate the Page checks from the swsusp checks, to break the calls to PageReserved() and PageOffline() into their own check in saveable_highmem_page().
Thanks!
-- Bill
Quoted 26 lines · David Hildenbrand
kernel/power/snapshot.c +4
-1@@ -1222,7 +1222,7 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)BUG_ON (!PageHighMem (page));if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page) || -PageReserved (page)) +PageReserved (page) ||PageOffline (page))return NULL ;if (page_is_guard (page))@@ -1286,6 +1286,9 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page))return NULL ; +if (PageOffline (page)) +return NULL ; +if (PageReserved (page) && (!kernel_page_present (page) ||pfn_is_nosave (pfn)))return NULL ;Signature · 1 line
2.17.2
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 8/8 PM / Hibernate: exclude all PageOffline() pages
In reply to David HildenbrandThe content of pages that are marked PG_offline is not of interest (e.g. inflated by a balloon driver), let's skip these pages.
--- kernel/power/snapshot.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
kernel/power/snapshot.c +4 -1
@@ -1222,7 +1222,7 @@ static struct page *saveable_highmem_page(struct zone *zone, unsigned long pfn)BUG_ON (!PageHighMem (page));if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page) || -PageReserved (page)) +PageReserved (page) ||PageOffline (page))return NULL ;if (page_is_guard (page))
@@ -1286,6 +1286,9 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)if (swsusp_page_is_forbidden (page) ||swsusp_page_is_free (page))return NULL ; +if (PageOffline (page)) +return NULL ; +if (PageReserved (page) && (!kernel_page_present (page) ||pfn_is_nosave (pfn)))return NULL ;
Signature · 7 lines
2.17.2 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 1/1 makedumpfile: exclude pages that are logically offline
In reply to David HildenbrandLinux marks pages that are logically offline via a page flag (map count). Such pages e.g. include pages infated as part of a balloon driver or pages that were not actually onlined when onlining the whole section.
While the hypervisor usually allows to read such inflated memory, we basically read and dump data that is completely irrelevant. Also, this might result in quite some overhead in the hypervisor. In addition, we saw some problems under Hyper-V, whereby we can crash the kernel by dumping, when reading memory of a partially onlined memory segment (for memory added by the Hyper-V balloon driver).
Therefore, don't read and dump pages that are marked as being logically offline.
--- makedumpfile.c | 34 ++++++++++++++++++++++++++++++---- makedumpfile.h | 1 + 2 files changed, 31 insertions(+), 4 deletions(-)
makedumpfile.c +30 -4
@@ -88,6 +88,7 @@ mdf_pfn_t pfn_cache_private;mdf_pfn_t pfn_user;mdf_pfn_t pfn_free;mdf_pfn_t pfn_hwpoison; +mdf_pfn_t pfn_offline;mdf_pfn_t num_dumped;
@@ -249,6 +250,21 @@ isHugetlb(unsigned long dtor) && (SYMBOL(free_huge_page) == dtor)); } +static int +isOffline (unsigned long flags,unsigned int _mapcount) +{ +if (NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE ) ==NOT_FOUND_NUMBER ) +return FALSE; + +if (flags & (1UL <<NUMBER (PG_slab))) +return FALSE; + +if (_mapcount == (int )NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE )) +return TRUE; + +return FALSE; +} +static int is_cache_page (unsigned long flags) {
@@ -2287,6 +2303,8 @@ write_vmcoreinfo_data(void)WRITE_NUMBER ("PG_hwpoison" , PG_hwpoison);WRITE_NUMBER ("PAGE_BUDDY_MAPCOUNT_VALUE" ,PAGE_BUDDY_MAPCOUNT_VALUE ); +WRITE_NUMBER ("PAGE_OFFLINE_MAPCOUNT_VALUE" , +PAGE_OFFLINE_MAPCOUNT_VALUE );WRITE_NUMBER ("phys_base" , phys_base);WRITE_NUMBER ("HUGETLB_PAGE_DTOR" ,HUGETLB_PAGE_DTOR );
@@ -2687,6 +2705,7 @@ read_vmcoreinfo(void)READ_SRCFILE ("pud_t" , pud_t);READ_NUMBER ("PAGE_BUDDY_MAPCOUNT_VALUE" ,PAGE_BUDDY_MAPCOUNT_VALUE ); +READ_NUMBER ("PAGE_OFFLINE_MAPCOUNT_VALUE" ,PAGE_OFFLINE_MAPCOUNT_VALUE );READ_NUMBER ("phys_base" , phys_base);#ifdef __aarch64__READ_NUMBER ("VA_BITS" ,VA_BITS );
@@ -6041,6 +6060,12 @@ __exclude_unnecessary_pages(unsigned long mem_map,else if (isHWPOISON (flags)) { pfn_counter = &pfn_hwpoison; } +/* +* Exclude pages that are logically offline. +*/ +else if (isOffline (flags, _mapcount)) { + pfn_counter = &pfn_offline; + }/* * Unexcludable page */
@@ -7522,7 +7547,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)*/ if (info->flag_cyclic) { pfn_zero = pfn_cache = pfn_cache_private =0 ; - pfn_user = pfn_free = pfn_hwpoison =0 ; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline =0 ; pfn_memhole = info->max_mapnr; }
@@ -8804,7 +8829,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d* Reset counter for debug message. */ pfn_zero = pfn_cache = pfn_cache_private =0 ; - pfn_user = pfn_free = pfn_hwpoison =0 ; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline =0 ; pfn_memhole = info->max_mapnr;/*
@@ -9749,7 +9774,7 @@ print_report(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; shrinking = (pfn_original - pfn_excluded) *100 ; shrinking = shrinking / pfn_original;
@@ -9763,6 +9788,7 @@ print_report(void)REPORT_MSG (" User process data pages : 0x%016llx\n" , pfn_user);REPORT_MSG (" Free pages : 0x%016llx\n" , pfn_free);REPORT_MSG (" Hwpoison pages : 0x%016llx\n" , pfn_hwpoison); +REPORT_MSG (" Offline pages : 0x%016llx\n" , pfn_offline);REPORT_MSG (" Remaining pages : 0x%016llx\n" , pfn_original - pfn_excluded); REPORT_MSG(" (The number of pages is reduced to %lld%%.)\n",
@@ -9790,7 +9816,7 @@ print_mem_usage(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; shrinking = (pfn_original - pfn_excluded) *100 ; shrinking = shrinking / pfn_original; total_size = info->page_size * pfn_original;
makedumpfile.h +1 -0
@@ -1927,6 +1927,7 @@ struct number_table {long PG_hwpoison;long PAGE_BUDDY_MAPCOUNT_VALUE ; +long PAGE_OFFLINE_MAPCOUNT_VALUE ;long SECTION_SIZE_BITS ;long MAX_PHYSMEM_BITS ;long HUGETLB_PAGE_DTOR ;
Signature · 1 line
2.17.2
RE: [PATCH v1] makedumpfile: exclude pages that are logically offline
Hi David,
Quoted 33 lines
Linux marks pages that are logically offline via a page flag (map count). Such pages e.g. include pages infated as part of a balloon driver or pages that were not actually onlined when onlining the whole section.
While the hypervisor usually allows to read such inflated memory, we basically read and dump data that is completely irrelevant. Also, this might result in quite some overhead in the hypervisor. In addition, we saw some problems under Hyper-V, whereby we can crash the kernel by dumping, when reading memory of a partially onlined memory segment (for memory added by the Hyper-V balloon driver).
Therefore, don't read and dump pages that are marked as being logically offline.
Signed-off-by David Hildenbrand <[email protected]>--- makedumpfile.c | 34 ++++++++++++++++++++++++++++++---- makedumpfile.h | 1 + 2 files changed, 31 insertions(+), 4 deletions(-)makedumpfile.c +1
-0@@ -88,6 +88,7 @@ mdf_pfn_t pfn_cache_private;mdf_pfn_t pfn_user;mdf_pfn_t pfn_free;mdf_pfn_t pfn_hwpoison; +mdf_pfn_t pfn_offline;mdf_pfn_t num_dumped;
makedumpfile.c isHugetlb(unsigned long dtor) · line 257
Quoted 8 lines
makedumpfile.c +5
-0@@ -249,6 +250,21 @@ isHugetlb(unsigned long dtor) && (SYMBOL(free_huge_page) == dtor)); } +static int +isOffline (unsigned long flags,unsigned int _mapcount) +{ +if (NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE ) ==NOT_FOUND_NUMBER ) +return FALSE;
This is NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE), isn't it? If so, I will correct it when merging.
Otherwise, looks good to me.
Thanks!
Kazu
Quoted 102 lines
makedumpfile.c +24
-4+ +if (flags & (1UL <<NUMBER (PG_slab))) +return FALSE; + +if (_mapcount == (int )NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE )) +return TRUE; + +return FALSE; +} +static int is_cache_page (unsigned long flags) {@@ -2287,6 +2303,8 @@ write_vmcoreinfo_data(void)WRITE_NUMBER ("PG_hwpoison" , PG_hwpoison);WRITE_NUMBER ("PAGE_BUDDY_MAPCOUNT_VALUE" ,PAGE_BUDDY_MAPCOUNT_VALUE ); +WRITE_NUMBER ("PAGE_OFFLINE_MAPCOUNT_VALUE" , +PAGE_OFFLINE_MAPCOUNT_VALUE );WRITE_NUMBER ("phys_base" , phys_base);WRITE_NUMBER ("HUGETLB_PAGE_DTOR" ,HUGETLB_PAGE_DTOR );@@ -2687,6 +2705,7 @@ read_vmcoreinfo(void)READ_SRCFILE ("pud_t" , pud_t);READ_NUMBER ("PAGE_BUDDY_MAPCOUNT_VALUE" ,PAGE_BUDDY_MAPCOUNT_VALUE ); +READ_NUMBER ("PAGE_OFFLINE_MAPCOUNT_VALUE" ,PAGE_OFFLINE_MAPCOUNT_VALUE );READ_NUMBER ("phys_base" , phys_base);#ifdef __aarch64__READ_NUMBER ("VA_BITS" ,VA_BITS );@@ -6041,6 +6060,12 @@ __exclude_unnecessary_pages(unsigned long mem_map,else if (isHWPOISON (flags)) { pfn_counter = &pfn_hwpoison; } +/* +* Exclude pages that are logically offline. +*/ +else if (isOffline (flags, _mapcount)) { + pfn_counter = &pfn_offline; + }/* * Unexcludable page */ @@ -7522,7 +7547,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)*/ if (info->flag_cyclic) { pfn_zero = pfn_cache = pfn_cache_private =0 ; - pfn_user = pfn_free = pfn_hwpoison =0 ; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline =0 ; pfn_memhole = info->max_mapnr; }@@ -8804,7 +8829,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d* Reset counter for debug message. */ pfn_zero = pfn_cache = pfn_cache_private =0 ; - pfn_user = pfn_free = pfn_hwpoison =0 ; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline =0 ; pfn_memhole = info->max_mapnr;/* @@ -9749,7 +9774,7 @@ print_report(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; shrinking = (pfn_original - pfn_excluded) *100 ; shrinking = shrinking / pfn_original;@@ -9763,6 +9788,7 @@ print_report(void)REPORT_MSG (" User process data pages : 0x%016llx\n" , pfn_user);REPORT_MSG (" Free pages : 0x%016llx\n" , pfn_free);REPORT_MSG (" Hwpoison pages : 0x%016llx\n" , pfn_hwpoison); +REPORT_MSG (" Offline pages : 0x%016llx\n" , pfn_offline);REPORT_MSG (" Remaining pages : 0x%016llx\n" , pfn_original - pfn_excluded); REPORT_MSG(" (The number of pages is reduced to %lld%%.)\n",@@ -9790,7 +9816,7 @@ print_mem_usage(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; shrinking = (pfn_original - pfn_excluded) *100 ; shrinking = shrinking / pfn_original; total_size = info->page_size * pfn_original;makedumpfile.h +1
-1@@ -1927,6 +1927,7 @@ struct number_table {long PG_hwpoison;long PAGE_BUDDY_MAPCOUNT_VALUE ; +long PAGE_OFFLINE_MAPCOUNT_VALUE ;long SECTION_SIZE_BITS ;long MAX_PHYSMEM_BITS ;long HUGETLB_PAGE_DTOR ; --2.17.2
Re: [PATCH v1] makedumpfile: exclude pages that are logically offline
Quoted 47 lines · Kazuhito Hagio
Hi David,
Quoted 33 lines
Linux marks pages that are logically offline via a page flag (map count). Such pages e.g. include pages infated as part of a balloon driver or pages that were not actually onlined when onlining the whole section.
While the hypervisor usually allows to read such inflated memory, we basically read and dump data that is completely irrelevant. Also, this might result in quite some overhead in the hypervisor. In addition, we saw some problems under Hyper-V, whereby we can crash the kernel by dumping, when reading memory of a partially onlined memory segment (for memory added by the Hyper-V balloon driver).
Therefore, don't read and dump pages that are marked as being logically offline.
Signed-off-by David Hildenbrand <[email protected]>--- makedumpfile.c | 34 ++++++++++++++++++++++++++++++---- makedumpfile.h | 1 + 2 files changed, 31 insertions(+), 4 deletions(-)makedumpfile.c +1
-0@@ -88,6 +88,7 @@ mdf_pfn_t pfn_cache_private;mdf_pfn_t pfn_user;mdf_pfn_t pfn_free;mdf_pfn_t pfn_hwpoison; +mdf_pfn_t pfn_offline;mdf_pfn_t num_dumped;makedumpfile.c isHugetlb(unsigned long dtor) · line 257
Quoted 8 lines
makedumpfile.c +5
-0@@ -249,6 +250,21 @@ isHugetlb(unsigned long dtor) && (SYMBOL(free_huge_page) == dtor)); } +static int +isOffline (unsigned long flags,unsigned int _mapcount) +{ +if (NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE ) ==NOT_FOUND_NUMBER ) +return FALSE;This is
NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE), isn't it? If so, I will correct it when merging.
Quoted 4 lines
Otherwise, looks good to me.
Thanks!
Kazu
Indeed,
I will most probably resend either way along with a new mm series!
Thanks a lot!
Signature · 3 lines
Thanks, David / dhildenb
Re: [PATCH v1] makedumpfile: exclude pages that are logically offline
In reply to Kazuhito HagioQuoted 47 lines · Kazuhito Hagio
Hi David,
Quoted 33 lines
Linux marks pages that are logically offline via a page flag (map count). Such pages e.g. include pages infated as part of a balloon driver or pages that were not actually onlined when onlining the whole section.
While the hypervisor usually allows to read such inflated memory, we basically read and dump data that is completely irrelevant. Also, this might result in quite some overhead in the hypervisor. In addition, we saw some problems under Hyper-V, whereby we can crash the kernel by dumping, when reading memory of a partially onlined memory segment (for memory added by the Hyper-V balloon driver).
Therefore, don't read and dump pages that are marked as being logically offline.
Signed-off-by David Hildenbrand <[email protected]>--- makedumpfile.c | 34 ++++++++++++++++++++++++++++++---- makedumpfile.h | 1 + 2 files changed, 31 insertions(+), 4 deletions(-)makedumpfile.c +1
-0@@ -88,6 +88,7 @@ mdf_pfn_t pfn_cache_private;mdf_pfn_t pfn_user;mdf_pfn_t pfn_free;mdf_pfn_t pfn_hwpoison; +mdf_pfn_t pfn_offline;mdf_pfn_t num_dumped;makedumpfile.c isHugetlb(unsigned long dtor) · line 257
Quoted 8 lines
makedumpfile.c +5
-0@@ -249,6 +250,21 @@ isHugetlb(unsigned long dtor) && (SYMBOL(free_huge_page) == dtor)); } +static int +isOffline (unsigned long flags,unsigned int _mapcount) +{ +if (NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE ) ==NOT_FOUND_NUMBER ) +return FALSE;This is
NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE), isn't it? If so, I will correct it when merging.
Quoted 4 lines
Otherwise, looks good to me.
Thanks!
Kazu
Indeed,
I will most probably resend either way along with a new mm series!
Thanks a lot!
Signature · 8 lines
Thanks, David / dhildenb _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH v1] makedumpfile: exclude pages that are logically offline
In reply to David HildenbrandHi David,
Quoted 33 lines
Linux marks pages that are logically offline via a page flag (map count). Such pages e.g. include pages infated as part of a balloon driver or pages that were not actually onlined when onlining the whole section.
While the hypervisor usually allows to read such inflated memory, we basically read and dump data that is completely irrelevant. Also, this might result in quite some overhead in the hypervisor. In addition, we saw some problems under Hyper-V, whereby we can crash the kernel by dumping, when reading memory of a partially onlined memory segment (for memory added by the Hyper-V balloon driver).
Therefore, don't read and dump pages that are marked as being logically offline.
Signed-off-by David Hildenbrand <[email protected]>--- makedumpfile.c | 34 ++++++++++++++++++++++++++++++---- makedumpfile.h | 1 + 2 files changed, 31 insertions(+), 4 deletions(-)makedumpfile.c +1
-0@@ -88,6 +88,7 @@ mdf_pfn_t pfn_cache_private;mdf_pfn_t pfn_user;mdf_pfn_t pfn_free;mdf_pfn_t pfn_hwpoison; +mdf_pfn_t pfn_offline;mdf_pfn_t num_dumped;
makedumpfile.c isHugetlb(unsigned long dtor) · line 257
Quoted 8 lines
makedumpfile.c +5
-0@@ -249,6 +250,21 @@ isHugetlb(unsigned long dtor) && (SYMBOL(free_huge_page) == dtor)); } +static int +isOffline (unsigned long flags,unsigned int _mapcount) +{ +if (NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE ) ==NOT_FOUND_NUMBER ) +return FALSE;
This is NUMBER(PAGE_OFFLINE_MAPCOUNT_VALUE), isn't it? If so, I will correct it when merging.
Otherwise, looks good to me.
Thanks!
Kazu
Quoted 100 lines
makedumpfile.c +24
-4+ +if (flags & (1UL <<NUMBER (PG_slab))) +return FALSE; + +if (_mapcount == (int )NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE )) +return TRUE; + +return FALSE; +} +static int is_cache_page (unsigned long flags) {@@ -2287,6 +2303,8 @@ write_vmcoreinfo_data(void)WRITE_NUMBER ("PG_hwpoison" , PG_hwpoison);WRITE_NUMBER ("PAGE_BUDDY_MAPCOUNT_VALUE" ,PAGE_BUDDY_MAPCOUNT_VALUE ); +WRITE_NUMBER ("PAGE_OFFLINE_MAPCOUNT_VALUE" , +PAGE_OFFLINE_MAPCOUNT_VALUE );WRITE_NUMBER ("phys_base" , phys_base);WRITE_NUMBER ("HUGETLB_PAGE_DTOR" ,HUGETLB_PAGE_DTOR );@@ -2687,6 +2705,7 @@ read_vmcoreinfo(void)READ_SRCFILE ("pud_t" , pud_t);READ_NUMBER ("PAGE_BUDDY_MAPCOUNT_VALUE" ,PAGE_BUDDY_MAPCOUNT_VALUE ); +READ_NUMBER ("PAGE_OFFLINE_MAPCOUNT_VALUE" ,PAGE_OFFLINE_MAPCOUNT_VALUE );READ_NUMBER ("phys_base" , phys_base);#ifdef __aarch64__READ_NUMBER ("VA_BITS" ,VA_BITS );@@ -6041,6 +6060,12 @@ __exclude_unnecessary_pages(unsigned long mem_map,else if (isHWPOISON (flags)) { pfn_counter = &pfn_hwpoison; } +/* +* Exclude pages that are logically offline. +*/ +else if (isOffline (flags, _mapcount)) { + pfn_counter = &pfn_offline; + }/* * Unexcludable page */ @@ -7522,7 +7547,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)*/ if (info->flag_cyclic) { pfn_zero = pfn_cache = pfn_cache_private =0 ; - pfn_user = pfn_free = pfn_hwpoison =0 ; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline =0 ; pfn_memhole = info->max_mapnr; }@@ -8804,7 +8829,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d* Reset counter for debug message. */ pfn_zero = pfn_cache = pfn_cache_private =0 ; - pfn_user = pfn_free = pfn_hwpoison =0 ; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline =0 ; pfn_memhole = info->max_mapnr;/* @@ -9749,7 +9774,7 @@ print_report(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; shrinking = (pfn_original - pfn_excluded) *100 ; shrinking = shrinking / pfn_original;@@ -9763,6 +9788,7 @@ print_report(void)REPORT_MSG (" User process data pages : 0x%016llx\n" , pfn_user);REPORT_MSG (" Free pages : 0x%016llx\n" , pfn_free);REPORT_MSG (" Hwpoison pages : 0x%016llx\n" , pfn_hwpoison); +REPORT_MSG (" Offline pages : 0x%016llx\n" , pfn_offline);REPORT_MSG (" Remaining pages : 0x%016llx\n" , pfn_original - pfn_excluded); REPORT_MSG(" (The number of pages is reduced to %lld%%.)\n",@@ -9790,7 +9816,7 @@ print_mem_usage(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; shrinking = (pfn_original - pfn_excluded) *100 ; shrinking = shrinking / pfn_original; total_size = info->page_size * pfn_original;makedumpfile.h +1
-1@@ -1927,6 +1927,7 @@ struct number_table {long PG_hwpoison;long PAGE_BUDDY_MAPCOUNT_VALUE ; +long PAGE_OFFLINE_MAPCOUNT_VALUE ;long SECTION_SIZE_BITS ;long MAX_PHYSMEM_BITS ;long HUGETLB_PAGE_DTOR ; --
Quoted 2 lines
2.17.2
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 1/1 makedumpfile: exclude pages that are logically offline
In reply to David HildenbrandLinux marks pages that are logically offline via a page flag (map count). Such pages e.g. include pages infated as part of a balloon driver or pages that were not actually onlined when onlining the whole section.
While the hypervisor usually allows to read such inflated memory, we basically read and dump data that is completely irrelevant. Also, this might result in quite some overhead in the hypervisor. In addition, we saw some problems under Hyper-V, whereby we can crash the kernel by dumping, when reading memory of a partially onlined memory segment (for memory added by the Hyper-V balloon driver).
Therefore, don't read and dump pages that are marked as being logically offline.
--- makedumpfile.c | 34 ++++++++++++++++++++++++++++++---- makedumpfile.h | 1 + 2 files changed, 31 insertions(+), 4 deletions(-)
makedumpfile.c +30 -4
@@ -88,6 +88,7 @@ mdf_pfn_t pfn_cache_private;mdf_pfn_t pfn_user;mdf_pfn_t pfn_free;mdf_pfn_t pfn_hwpoison; +mdf_pfn_t pfn_offline;mdf_pfn_t num_dumped;
@@ -249,6 +250,21 @@ isHugetlb(unsigned long dtor) && (SYMBOL(free_huge_page) == dtor)); } +static int +isOffline (unsigned long flags,unsigned int _mapcount) +{ +if (NUMBER (PAGE_BUDDY_MAPCOUNT_VALUE ) ==NOT_FOUND_NUMBER ) +return FALSE; + +if (flags & (1UL <<NUMBER (PG_slab))) +return FALSE; + +if (_mapcount == (int )NUMBER (PAGE_OFFLINE_MAPCOUNT_VALUE )) +return TRUE; + +return FALSE; +} +static int is_cache_page (unsigned long flags) {
@@ -2287,6 +2303,8 @@ write_vmcoreinfo_data(void)WRITE_NUMBER ("PG_hwpoison" , PG_hwpoison);WRITE_NUMBER ("PAGE_BUDDY_MAPCOUNT_VALUE" ,PAGE_BUDDY_MAPCOUNT_VALUE ); +WRITE_NUMBER ("PAGE_OFFLINE_MAPCOUNT_VALUE" , +PAGE_OFFLINE_MAPCOUNT_VALUE );WRITE_NUMBER ("phys_base" , phys_base);WRITE_NUMBER ("HUGETLB_PAGE_DTOR" ,HUGETLB_PAGE_DTOR );
@@ -2687,6 +2705,7 @@ read_vmcoreinfo(void)READ_SRCFILE ("pud_t" , pud_t);READ_NUMBER ("PAGE_BUDDY_MAPCOUNT_VALUE" ,PAGE_BUDDY_MAPCOUNT_VALUE ); +READ_NUMBER ("PAGE_OFFLINE_MAPCOUNT_VALUE" ,PAGE_OFFLINE_MAPCOUNT_VALUE );READ_NUMBER ("phys_base" , phys_base);#ifdef __aarch64__READ_NUMBER ("VA_BITS" ,VA_BITS );
@@ -6041,6 +6060,12 @@ __exclude_unnecessary_pages(unsigned long mem_map,else if (isHWPOISON (flags)) { pfn_counter = &pfn_hwpoison; } +/* +* Exclude pages that are logically offline. +*/ +else if (isOffline (flags, _mapcount)) { + pfn_counter = &pfn_offline; + }/* * Unexcludable page */
@@ -7522,7 +7547,7 @@ write_elf_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page)*/ if (info->flag_cyclic) { pfn_zero = pfn_cache = pfn_cache_private =0 ; - pfn_user = pfn_free = pfn_hwpoison =0 ; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline =0 ; pfn_memhole = info->max_mapnr; }
@@ -8804,7 +8829,7 @@ write_kdump_pages_and_bitmap_cyclic(struct cache_data *cd_header, struct cache_d* Reset counter for debug message. */ pfn_zero = pfn_cache = pfn_cache_private =0 ; - pfn_user = pfn_free = pfn_hwpoison =0 ; + pfn_user = pfn_free = pfn_hwpoison = pfn_offline =0 ; pfn_memhole = info->max_mapnr;/*
@@ -9749,7 +9774,7 @@ print_report(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; shrinking = (pfn_original - pfn_excluded) *100 ; shrinking = shrinking / pfn_original;
@@ -9763,6 +9788,7 @@ print_report(void)REPORT_MSG (" User process data pages : 0x%016llx\n" , pfn_user);REPORT_MSG (" Free pages : 0x%016llx\n" , pfn_free);REPORT_MSG (" Hwpoison pages : 0x%016llx\n" , pfn_hwpoison); +REPORT_MSG (" Offline pages : 0x%016llx\n" , pfn_offline);REPORT_MSG (" Remaining pages : 0x%016llx\n" , pfn_original - pfn_excluded); REPORT_MSG(" (The number of pages is reduced to %lld%%.)\n",
@@ -9790,7 +9816,7 @@ print_mem_usage(void) pfn_original = info->max_mapnr - pfn_memhole; pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private - + pfn_user + pfn_free + pfn_hwpoison; + + pfn_user + pfn_free + pfn_hwpoison + pfn_offline; shrinking = (pfn_original - pfn_excluded) *100 ; shrinking = shrinking / pfn_original; total_size = info->page_size * pfn_original;
makedumpfile.h +1 -0
@@ -1927,6 +1927,7 @@ struct number_table {long PG_hwpoison;long PAGE_BUDDY_MAPCOUNT_VALUE ; +long PAGE_OFFLINE_MAPCOUNT_VALUE ;long SECTION_SIZE_BITS ;long MAX_PHYSMEM_BITS ;long HUGETLB_PAGE_DTOR ;
Signature · 7 lines
2.17.2 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel