mm: balloon infrastructure cleanups
Subsystems (14)
David Hildenbrand (Red Hat)
maintainerv3 cover mm: balloon infrastructure cleanups
I started with wanting to remove the dependency of the balloon infrastructure on the page lock, but ended up performing various other cleanups, some of which I had on my todo list for years.
This series heavily cleans up and simplifies our balloon infrastructure, including our balloon page migration functionality.
With this series, we no longer make use of the page lock for PageOffline pages as part of the balloon infrastructure (preparing for memdescs where PageOffline pages won't have any such lock), and simplifies migration handling such that refcounting can more easily be adjusted later (long-term focus is for PageOffline pages to not have a refcount either).
Plenty of related cleanups.
Heavily compile-tested and heavily runtime-tested with virtio-balloon. PPC CMM and the VMware balloon are untested and I'd appreciate a helping hand from people that have suitable environments.
Not CCing maintainers for the vmscan.c and migrate.c change as they are rather trivial and I don't want to patchbomb them.
--
v2 -> v3:
* "mm/balloon_compaction: centralize basic page migration handling" -> Remove newline
-> Talk about "page" instead of "old page"
-> avoid the switch statement
* "mm/balloon_compaction: move internal helpers to balloon_compaction.c" -> No need to carry over the "inline"
* "fixup mm: rename CONFIG_BALLOON_COMPACTION to CONFIG_BALLOON_MIGRATION -> Remove some more "balloon page compaction" thingies
* "MAINTAINERS: move memory balloon infrastructure to "MEMORY MANAGEMENT - BALLOON"
-> CC the virt list and use the proper mail address
* "mm/balloon_compaction: improve comments for WARN_ON_ONCE(!b_dev_info) -> Added
v1 -> v2:
* Rebased to latest mm/mm-unstable without any conflicts
* Minor patch description/subject and comment fixups
* Retested
David Hildenbrand (Red Hat) (24):
vmw_balloon: adjust BALLOON_DEFLATE when deflating while migrating
vmw_balloon: remove vmballoon_compaction_init() powerpc/pseries/cmm: remove cmm_balloon_compaction_init() mm/balloon_compaction: improve comments for WARN_ON_ONCE(!b_dev_info) mm/balloon_compaction: centralize basic page migration handling mm/balloon_compaction: centralize adjust_managed_page_count() handling
vmw_balloon: stop using the balloon_dev_info lock mm/balloon_compaction: use a device-independent balloon (list) lock mm/balloon_compaction: remove dependency on page lock mm/balloon_compaction: make balloon_mops static mm/balloon_compaction: drop fs.h include from balloon_compaction.h drivers/virtio/virtio_balloon: stop using balloon_page_push/pop() mm/balloon_compaction: remove balloon_page_push/pop() mm/balloon_compaction: fold balloon_mapping_gfp_mask() into
balloon_page_alloc()
mm/balloon_compaction: move internal helpers to balloon_compaction.c mm/balloon_compaction: assert that the balloon_pages_lock is held mm/balloon_compaction: mark remaining functions for having proper
kerneldoc
mm/balloon_compaction: remove "extern" from functions mm/vmscan: drop inclusion of balloon_compaction.h
mm: rename balloon_compaction.(c|h) to balloon.(c|h) mm/kconfig: make BALLOON_COMPACTION depend on MIGRATION
mm: rename CONFIG_BALLOON_COMPACTION to CONFIG_BALLOON_MIGRATION
mm: rename CONFIG_MEMORY_BALLOON -> CONFIG_BALLOON
MAINTAINERS: move memory balloon infrastructure to "MEMORY MANAGEMENT
- BALLOON"
.../admin-guide/mm/memory-hotplug.rst | 8 +-
Documentation/core-api/mm-api.rst | 2 +-
MAINTAINERS | 13 +-
arch/powerpc/platforms/pseries/Kconfig | 2 +-
arch/powerpc/platforms/pseries/cmm.c | 53 +----
drivers/misc/Kconfig | 2 +-
drivers/misc/vmw_balloon.c | 105 +++------
drivers/virtio/Kconfig | 2 +-
drivers/virtio/virtio_balloon.c | 64 ++----
include/linux/balloon.h | 77 +++++++
include/linux/balloon_compaction.h | 160 --------------
include/linux/vm_event_item.h | 8 +-
mm/Kconfig | 23 +-
mm/Makefile | 2 +-
mm/{balloon_compaction.c => balloon.c} | 200 ++++++++++++------
mm/memory_hotplug.c | 4 +-
mm/migrate.c | 2 +-
mm/vmscan.c | 1 -
mm/vmstat.c | 8 +-
19 files changed, 319 insertions(+), 417 deletions(-)
create mode 100644 include/linux/balloon.h
delete mode 100644 include/linux/balloon_compaction.h
rename mm/{balloon_compaction.c => balloon.c} (58%)base-commit: f8ed52ac0cfbddff992bb9600941bfe51e1e385a
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 1/24 vmw_balloon: adjust BALLOON_DEFLATE when deflating while migrating
When we're effectively deflating the balloon while migrating a page because inflating the new page failed, we're not adjusting BALLOON_DEFLATE.
Let's do that. This is a preparation for factoring out this handling to the core code, making it work in a similar way first.
As this (deflating while migrating because of inflation error) is a corner case that I don't really expect to happen in practice and the stats are not that crucial, this likely doesn't classify as a fix.
--- drivers/misc/vmw_balloon.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/misc/vmw_balloon.c +2 -0
@@ -1818,6 +1818,8 @@ static int vmballoon_migratepage(struct balloon_dev_info *b_dev_info,if (status ==VMW_BALLOON_SUCCESS ) {balloon_page_insert (&b->b_dev_info, newpage);__count_vm_event (BALLOON_MIGRATE ); + }else { +__count_vm_event (BALLOON_DEFLATE ); }/*
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 2/24 vmw_balloon: remove vmballoon_compaction_init()
In reply to David Hildenbrand (Red Hat)Now that there is not a lot of logic left, let's just inline setting up the migration function and drop all these excessive comments that are not really required (or true) anymore.
To avoid #ifdef in the caller we can instead use IS_ENABLED() and make the compiler happy by only providing the function declaration.
--- drivers/misc/vmw_balloon.c | 30 +++++------------------------- 1 file changed, 5 insertions(+), 25 deletions(-)
drivers/misc/vmw_balloon.c +5 -25
@@ -1833,27 +1833,10 @@ static int vmballoon_migratepage(struct balloon_dev_info *b_dev_info,up_read (&b->conf_sem); return ret; } - -/** -* vmballoon_compaction_init() - initialized compaction for the balloon. -* -* @b: pointer to the balloon. -* -* If during the initialization a failure occurred, this function does not -* perform cleanup. The caller must call vmballoon_compaction_deinit() in this -* case. -* -* Return: zero on success or error code on failure. -*/ -static __init void vmballoon_compaction_init (struct vmballoon *b) -{ - b->b_dev_info.migratepage = vmballoon_migratepage; -} -#else /* CONFIG_BALLOON_COMPACTION */ -static inline void vmballoon_compaction_init (struct vmballoon *b) -{ -} +int vmballoon_migratepage (struct balloon_dev_info *b_dev_info, +struct page *newpage,struct page *page, +enum migrate_mode mode);#endif /* CONFIG_BALLOON_COMPACTION */ static int __init vmballoon_init (void )
@@ -1873,12 +1856,9 @@ static int __init vmballoon_init(void)if (error)return error; -/* -* Initialization of compaction must be done after the call to -* balloon_devinfo_init() . -*/ balloon_devinfo_init (&balloon.b_dev_info); -vmballoon_compaction_init (&balloon); +if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) + balloon.b_dev_info.migratepage = vmballoon_migratepage;INIT_LIST_HEAD (&balloon.huge_pages);spin_lock_init (&balloon.comm_lock);
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 3/24 powerpc/pseries/cmm: remove cmm_balloon_compaction_init()
In reply to David Hildenbrand (Red Hat)Now that there is not a lot of logic left, let's just inline setting up the migration function.
To avoid #ifdef in the caller we can instead use IS_ENABLED() and make the compiler happy by only providing the function declaration.
Now that the function is gone, drop the "out_balloon_compaction" label. Note that before commit 68f2736a8583 ("mm: Convert all PageMovable users to movable_operations") we actually had to undo something, now not anymore.
--- arch/powerpc/platforms/pseries/cmm.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-)
arch/powerpc/platforms/pseries/cmm.c +5 -11
@@ -548,15 +548,9 @@ static int cmm_migratepage(struct balloon_dev_info *b_dev_info, return 0; } - -static void cmm_balloon_compaction_init (void ) -{ - b_dev_info.migratepage = cmm_migratepage; -}#else /* CONFIG_BALLOON_COMPACTION */ -static void cmm_balloon_compaction_init (void ) -{ -} +int cmm_migratepage (struct balloon_dev_info *b_dev_info,struct page *newpage, +struct page *page,enum migrate_mode mode);#endif /* CONFIG_BALLOON_COMPACTION */ /**
@@ -573,11 +567,12 @@ static int cmm_init(void)return -EOPNOTSUPP ;balloon_devinfo_init (&b_dev_info); -cmm_balloon_compaction_init (); +if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) + b_dev_info.migratepage = cmm_migratepage; rc =register_oom_notifier (&cmm_oom_nb);if (rc <0 ) - goto out_balloon_compaction; +return rc;if ((rc =register_reboot_notifier (&cmm_reboot_nb))) goto out_oom_notifier;
@@ -606,7 +601,6 @@ static int cmm_init(void)unregister_reboot_notifier (&cmm_reboot_nb); out_oom_notifier:unregister_oom_notifier (&cmm_oom_nb); -out_balloon_compaction:return rc; }
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 4/24 mm/balloon_compaction: improve comments for WARN_ON_ONCE(!b_dev_info)
In reply to David Hildenbrand (Red Hat)Let's clarify a bit by extending the comments.
--- mm/balloon_compaction.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-)
mm/balloon_compaction.c +10 -2
@@ -222,7 +222,11 @@ static void balloon_page_putback(struct page *page)struct balloon_dev_info *b_dev_info =balloon_page_device (page);unsigned long flags; -/* Isolated balloon pages cannot get deflated. */ +/* +* When we isolated the page, the page was still inflated in a balloon +* device. As isolated balloon pages cannot get deflated, we still have +* a balloon device here. +*/ if (WARN_ON_ONCE (!b_dev_info))return ;
@@ -241,7 +245,11 @@ static int balloon_page_migrate(struct page *newpage, struct page *page,VM_BUG_ON_PAGE (!PageLocked (page), page);VM_BUG_ON_PAGE (!PageLocked (newpage), newpage); -/* Isolated balloon pages cannot get deflated. */ +/* +* When we isolated the page, the page was still inflated in a balloon +* device. As isolated balloon pages cannot get deflated, we still have +* a balloon device here. +*/ if (WARN_ON_ONCE (!balloon))return -EAGAIN ;
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 5/24 mm/balloon_compaction: centralize basic page migration handling
In reply to David Hildenbrand (Red Hat)Let's update the balloon page references, the balloon page list, the BALLOON_MIGRATE counter and the isolated-pages counter in balloon_page_migrate(), after letting the balloon->migratepage() callback deal with the actual inflation+deflation.
Note that we now perform the balloon list modifications outside of any implementation-specific locks: which is fine, there is nothing special about these page actions that the lock would be protecting.
The old page is already no longer in the list (isolated) and the new page is not yet in the list.
Let's use -ENOENT to communicate the special "inflation of new page failed after already deflating the old page" to balloon_page_migrate() so it can handle it accordingly.
While at it, rename balloon->b_dev_info to make it match the other functions. Also, drop the comment above balloon_page_migrate(), which seems unnecessary.
--- arch/powerpc/platforms/pseries/cmm.c | 16 --------- drivers/misc/vmw_balloon.c | 49 +++++----------------------- drivers/virtio/virtio_balloon.c | 12 ------- mm/balloon_compaction.c | 31 +++++++++++++++--- 4 files changed, 35 insertions(+), 73 deletions(-)
arch/powerpc/platforms/pseries/cmm.c +0 -16
@@ -501,8 +501,6 @@ static int cmm_migratepage(struct balloon_dev_info *b_dev_info, struct page *newpage, struct page *page, enum migrate_mode mode) { - unsigned long flags; -/* * loan/"inflate" the newpage first. *
@@ -517,9 +515,6 @@ static int cmm_migratepage(struct balloon_dev_info *b_dev_info,return -EBUSY ; } -/* balloon page list reference */ -get_page (newpage); -/* * When we migrate a page to a different zone, we have to fixup the * count of both involved zones as we adjusted the managed page count
@@ -530,22 +525,11 @@ static int cmm_migratepage(struct balloon_dev_info *b_dev_info, adjust_managed_page_count(newpage, -1); } -spin_lock_irqsave (&b_dev_info->pages_lock, flags); -balloon_page_insert (b_dev_info, newpage); -__count_vm_event (BALLOON_MIGRATE ); - b_dev_info->isolated_pages--; -spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); -/* * activate/"deflate" the old page. We ignore any errors just like the * other callers. */ plpar_page_set_active (page); - -balloon_page_finalize (page); -/* balloon page list reference */ -put_page (page); -return 0 ; }#else /* CONFIG_BALLOON_COMPACTION */
drivers/misc/vmw_balloon.c +8 -41
76 lines not on this page · raw message
drivers/virtio/virtio_balloon.c +0 -12
@@ -827,7 +827,6 @@ static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info, {struct virtio_balloon *vb =container_of (vb_dev_info, struct virtio_balloon, vb_dev_info); -unsigned long flags;/* * In order to avoid lock contention while migrating pages concurrently
@@ -840,8 +839,6 @@ static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,if (!mutex_trylock (&vb->balloon_lock))return -EAGAIN ; -get_page (newpage);/* balloon reference */ -/* * When we migrate a page to a different zone and adjusted the * managed page count when inflating, we have to fixup the count of
@@ -854,11 +851,6 @@ static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info, }/* balloon's page migration 1st step -- inflate "newpage" */ -spin_lock_irqsave (&vb_dev_info->pages_lock, flags); -balloon_page_insert (vb_dev_info, newpage); - vb_dev_info->isolated_pages--; -__count_vm_event (BALLOON_MIGRATE ); -spin_unlock_irqrestore (&vb_dev_info->pages_lock, flags); vb->num_pfns =VIRTIO_BALLOON_PAGES_PER_PAGE ;set_page_pfns (vb, vb->pfns, newpage);tell_host (vb, vb->inflate_vq);
@@ -869,10 +861,6 @@ static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,tell_host (vb, vb->deflate_vq);mutex_unlock (&vb->balloon_lock); - -balloon_page_finalize (page); -put_page (page);/* balloon reference */ - return 0; }#endif /* CONFIG_BALLOON_COMPACTION */
mm/balloon_compaction.c +27 -4
@@ -236,11 +236,12 @@ static void balloon_page_putback(struct page *page)spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); } -/* move_to_new_page() counterpart for a ballooned page */ static int balloon_page_migrate (struct page *newpage,struct page *page,enum migrate_mode mode) { -struct balloon_dev_info *balloon =balloon_page_device (page); +struct balloon_dev_info *b_dev_info =balloon_page_device (page); +unsigned long flags; +int rc;VM_BUG_ON_PAGE (!PageLocked (page), page);VM_BUG_ON_PAGE (!PageLocked (newpage), newpage);
@@ -250,10 +251,32 @@ static int balloon_page_migrate(struct page *newpage, struct page *page,* device. As isolated balloon pages cannot get deflated, we still have * a balloon device here. */ -if (WARN_ON_ONCE (!balloon)) +if (WARN_ON_ONCE (!b_dev_info))return -EAGAIN ; -return balloon->migratepage (balloon, newpage, page, mode); + rc = b_dev_info->migratepage (b_dev_info, newpage, page, mode); +if (rc <0 && rc != -ENOENT ) +return rc; + +spin_lock_irqsave (&b_dev_info->pages_lock, flags); +if (!rc) { +/* Insert the new page into the balloon list. */ +get_page (newpage); +balloon_page_insert (b_dev_info, newpage); +__count_vm_event (BALLOON_MIGRATE ); + }else { +/* Old page was deflated but new page not inflated. */ +__count_vm_event (BALLOON_DEFLATE ); + } + + b_dev_info->isolated_pages--; +spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); + +/* Free the now-deflated page we isolated in balloon_page_isolate(). */ +balloon_page_finalize (page); +put_page (page); + +return 0 ; }const struct movable_operations balloon_mops = {
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 6/24 mm/balloon_compaction: centralize adjust_managed_page_count() handling
In reply to David Hildenbrand (Red Hat)Let's centralize it, by allowing for the driver to enable this handling through a new flag (bool for now) in the balloon device info.
Note that we now adjust the counter when adding/removing a page into the balloon list: when removing a page to deflate it, it will now happen before the driver communicated with hypervisor, not afterwards.
This shouldn't make a difference in practice.
--- arch/powerpc/platforms/pseries/cmm.c | 13 +------------ drivers/virtio/virtio_balloon.c | 19 ++----------------- include/linux/balloon_compaction.h | 2 ++ mm/balloon_compaction.c | 17 +++++++++++++++++ 4 files changed, 22 insertions(+), 29 deletions(-)
arch/powerpc/platforms/pseries/cmm.c +1 -12
@@ -165,7 +165,6 @@ static long cmm_alloc_pages(long nr)balloon_page_enqueue (&b_dev_info, page);atomic_long_inc (&loaned_pages); -adjust_managed_page_count (page,-1 ); nr--; }
@@ -190,7 +189,6 @@ static long cmm_free_pages(long nr)if (!page)break ;plpar_page_set_active (page); -adjust_managed_page_count (page,1 );__free_page (page);atomic_long_dec (&loaned_pages); nr--;
@@ -515,16 +513,6 @@ static int cmm_migratepage(struct balloon_dev_info *b_dev_info,return -EBUSY ; } -/* -* When we migrate a page to a different zone, we have to fixup the -* count of both involved zones as we adjusted the managed page count -* when inflating. -*/ -if (page_zone (page) !=page_zone (newpage)) { -adjust_managed_page_count (page,1 ); -adjust_managed_page_count (newpage,-1 ); - } -/* * activate/"deflate" the old page. We ignore any errors just like the * other callers.
@@ -551,6 +539,7 @@ static int cmm_init(void)return -EOPNOTSUPP ;balloon_devinfo_init (&b_dev_info); + b_dev_info.adjust_managed_page_count = true;if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) b_dev_info.migratepage = cmm_migratepage;
drivers/virtio/virtio_balloon.c +2 -17
@@ -274,9 +274,6 @@ static unsigned int fill_balloon(struct virtio_balloon *vb, size_t num)set_page_pfns (vb, vb->pfns + vb->num_pfns, page); vb->num_pages +=VIRTIO_BALLOON_PAGES_PER_PAGE ; -if (!virtio_has_feature (vb->vdev, -VIRTIO_BALLOON_F_DEFLATE_ON_OOM )) -adjust_managed_page_count (page,-1 ); vb->num_pfns +=VIRTIO_BALLOON_PAGES_PER_PAGE ; }
@@ -295,9 +292,6 @@ static void release_pages_balloon(struct virtio_balloon *vb,struct page *page, *next;list_for_each_entry_safe (page, next, pages, lru) { -if (!virtio_has_feature (vb->vdev, -VIRTIO_BALLOON_F_DEFLATE_ON_OOM )) -adjust_managed_page_count (page,1 );list_del (&page->lru);put_page (page);/* balloon reference */ }
@@ -839,17 +833,6 @@ static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,if (!mutex_trylock (&vb->balloon_lock))return -EAGAIN ; -/* -* When we migrate a page to a different zone and adjusted the -* managed page count when inflating, we have to fixup the count of -* both involved zones. -*/ -if (!virtio_has_feature (vb->vdev,VIRTIO_BALLOON_F_DEFLATE_ON_OOM ) && -page_zone (page) !=page_zone (newpage)) { -adjust_managed_page_count (page,1 ); -adjust_managed_page_count (newpage,-1 ); - } -/* balloon's page migration 1st step -- inflate "newpage" */ vb->num_pfns =VIRTIO_BALLOON_PAGES_PER_PAGE ;set_page_pfns (vb, vb->pfns, newpage);
@@ -958,6 +941,8 @@ static int virtballoon_probe(struct virtio_device *vdev)if (err) goto out_free_vb; +if (!virtio_has_feature (vb->vdev,VIRTIO_BALLOON_F_DEFLATE_ON_OOM )) + vb->vb_dev_info.adjust_managed_page_count = true;#ifdef CONFIG_BALLOON_COMPACTION vb->vb_dev_info.migratepage = virtballoon_migratepage;#endif
include/linux/balloon_compaction.h +2 -0
@@ -56,6 +56,7 @@ struct balloon_dev_info {struct list_head pages;/* Pages enqueued & handled to Host */ int (*migratepage)(struct balloon_dev_info *,struct page *newpage,struct page *page,enum migrate_mode mode); +bool adjust_managed_page_count; };extern struct page *balloon_page_alloc (void );
@@ -73,6 +74,7 @@ static inline void balloon_devinfo_init(struct balloon_dev_info *balloon)spin_lock_init (&balloon->pages_lock);INIT_LIST_HEAD (&balloon->pages); balloon->migratepage =NULL ; + balloon->adjust_managed_page_count = false; }#ifdef CONFIG_BALLOON_COMPACTION
mm/balloon_compaction.c +17 -0
@@ -23,6 +23,8 @@ static void balloon_page_enqueue_one(struct balloon_dev_info *b_dev_info,BUG_ON (!trylock_page (page));balloon_page_insert (b_dev_info, page);unlock_page (page); +if (b_dev_info->adjust_managed_page_count) +adjust_managed_page_count (page,-1 );__count_vm_event (BALLOON_INFLATE );inc_node_page_state (page,NR_BALLOON_PAGES ); }
@@ -95,6 +97,8 @@ size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info,continue ;list_del (&page->lru); +if (b_dev_info->adjust_managed_page_count) +adjust_managed_page_count (page,1 );balloon_page_finalize (page);__count_vm_event (BALLOON_DEFLATE );list_add (&page->lru, pages);
@@ -264,9 +268,22 @@ static int balloon_page_migrate(struct page *newpage, struct page *page,get_page (newpage);balloon_page_insert (b_dev_info, newpage);__count_vm_event (BALLOON_MIGRATE ); + +if (b_dev_info->adjust_managed_page_count && +page_zone (page) !=page_zone (newpage)) { +/* +* When we migrate a page to a different zone we +* have to fixup the count of both involved zones. +*/ +adjust_managed_page_count (page,1 ); +adjust_managed_page_count (newpage,-1 ); + } }else {/* Old page was deflated but new page not inflated. */ __count_vm_event (BALLOON_DEFLATE ); + +if (b_dev_info->adjust_managed_page_count) +adjust_managed_page_count (page,1 ); } b_dev_info->isolated_pages--;
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 7/24 vmw_balloon: stop using the balloon_dev_info lock
In reply to David Hildenbrand (Red Hat)Let's not piggy-back on the existing lock and use a separate lock for the huge page list. Now that we use a separate lock, there is no need to disable interrupts, so use the non-irqsave variants. We only required the irqsave variants because of the balloon device lock.
This is a preparation for changing the locking used to protect balloon_dev_info.
While at it, talk about "page migration" instead of "page compaction". We'll change that in core code soon as well.
--- drivers/misc/vmw_balloon.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
drivers/misc/vmw_balloon.c +12 -8
@@ -354,10 +354,15 @@ struct vmballoon {/** * @huge_pages - list of the inflated 2MB pages. * -* Protected by @b_dev_info.pages_lock . +* Protected by @huge_pages_lock. */ struct list_head huge_pages; +/** +* @huge_pages_lock: lock for the list of inflated 2MB pages. +*/ +spinlock_t huge_pages_lock; +/** * @vmci_doorbell. *
@@ -987,7 +992,6 @@ static void vmballoon_enqueue_page_list(struct vmballoon *b,unsigned int *n_pages, enum vmballoon_page_size_type page_size) { - unsigned long flags; struct page *page;if (page_size ==VMW_BALLOON_4K_PAGE ) {
@@ -995,9 +999,9 @@ static void vmballoon_enqueue_page_list(struct vmballoon *b, }else {/* * Keep the huge pages in a local list which is not available -* for the balloon compaction mechanism. +* for the balloon page migration. */ -spin_lock_irqsave (&b->b_dev_info.pages_lock, flags); +spin_lock (&b->huge_pages_lock);list_for_each_entry (page, pages, lru) {vmballoon_mark_page_offline (page,VMW_BALLOON_2M_PAGE );
@@ -1006,7 +1010,7 @@ static void vmballoon_enqueue_page_list(struct vmballoon *b,list_splice_init (pages, &b->huge_pages);__count_vm_events (BALLOON_INFLATE , *n_pages *vmballoon_page_in_frames (VMW_BALLOON_2M_PAGE )); -spin_unlock_irqrestore (&b->b_dev_info.pages_lock, flags); +spin_unlock (&b->huge_pages_lock); } *n_pages =0 ;
@@ -1033,7 +1037,6 @@ static void vmballoon_dequeue_page_list(struct vmballoon *b, {struct page *page, *tmp;unsigned int i =0 ; -unsigned long flags;/* In the case of 4k pages, use the compaction infrastructure */ if (page_size ==VMW_BALLOON_4K_PAGE ) {
@@ -1043,7 +1046,7 @@ static void vmballoon_dequeue_page_list(struct vmballoon *b, }/* 2MB pages */ -spin_lock_irqsave (&b->b_dev_info.pages_lock, flags); +spin_lock (&b->huge_pages_lock);list_for_each_entry_safe (page, tmp, &b->huge_pages, lru) {vmballoon_mark_page_online (page,VMW_BALLOON_2M_PAGE );
@@ -1054,7 +1057,7 @@ static void vmballoon_dequeue_page_list(struct vmballoon *b,__count_vm_events (BALLOON_DEFLATE , i *vmballoon_page_in_frames (VMW_BALLOON_2M_PAGE )); -spin_unlock_irqrestore (&b->b_dev_info.pages_lock, flags); +spin_unlock (&b->huge_pages_lock); *n_pages = i; }
@@ -1828,6 +1831,7 @@ static int __init vmballoon_init(void) balloon.b_dev_info.migratepage = vmballoon_migratepage;INIT_LIST_HEAD (&balloon.huge_pages); +spin_lock_init (&balloon.huge_pages_lock);spin_lock_init (&balloon.comm_lock);init_rwsem (&balloon.conf_sem); balloon.vmci_doorbell =VMCI_INVALID_HANDLE ;
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 8/24 mm/balloon_compaction: use a device-independent balloon (list) lock
In reply to David Hildenbrand (Red Hat)In order to remove the dependency on the page lock for balloon pages, we need a lock that is independent of the page.
It's crucial that we can handle the scenario where balloon deflation (clearing page->private) can race with page isolation (using page->private to obtain the balloon_dev_info where the lock currently resides).
The current lock in balloon_dev_info is therefore not suitable.
Fortunately, we never really have more than a single balloon device per VM, so we can just keep it simple and use a static lock to protect all balloon devices.
Based on this change we will remove the dependency on the page lock next.
--- include/linux/balloon_compaction.h | 6 ++---- mm/balloon_compaction.c | 34 ++++++++++++++++++------------ 2 files changed, 22 insertions(+), 18 deletions(-)
include/linux/balloon_compaction.h +2 -4
@@ -21,10 +21,10 @@* i. Setting the PG_movable_ops flag and page->private with the following * lock order * +-page_lock(page); -* +--spin_lock_irq(&b_dev_info->pages_lock); +* +--spin_lock_irq(&balloon_pages_lock); * * ii. isolation or dequeueing procedure must remove the page from balloon -* device page list under b_dev_info->pages_lock. +* device page list under balloon_pages_lock * * The functions provided by this interface are placed to help on coping with * the aforementioned balloon page corner case, as well as to ensure the simple
@@ -52,7 +52,6 @@*/ struct balloon_dev_info {unsigned long isolated_pages;/* # of isolated pages for migration */ -spinlock_t pages_lock;/* Protection to pages list */ struct list_head pages;/* Pages enqueued & handled to Host */ int (*migratepage)(struct balloon_dev_info *,struct page *newpage,struct page *page,enum migrate_mode mode);
@@ -71,7 +70,6 @@ extern size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info,static inline void balloon_devinfo_init (struct balloon_dev_info *balloon) { balloon->isolated_pages =0 ; -spin_lock_init (&balloon->pages_lock);INIT_LIST_HEAD (&balloon->pages); balloon->migratepage =NULL ; balloon->adjust_managed_page_count = false;
mm/balloon_compaction.c +20 -14
@@ -11,6 +11,12 @@#include <linux/export.h> #include <linux/balloon_compaction.h> +/* +* Lock protecting the balloon_dev_info of all devices. We don't really +* expect more than one device. +*/ +static DEFINE_SPINLOCK (balloon_pages_lock); +static void balloon_page_enqueue_one (struct balloon_dev_info *b_dev_info,struct page *page) {
@@ -47,13 +53,13 @@ size_t balloon_page_list_enqueue(struct balloon_dev_info *b_dev_info,unsigned long flags;size_t n_pages =0 ; -spin_lock_irqsave (&b_dev_info->pages_lock, flags); +spin_lock_irqsave (&balloon_pages_lock, flags);list_for_each_entry_safe (page, tmp, pages, lru) {list_del (&page->lru);balloon_page_enqueue_one (b_dev_info, page); n_pages++; } -spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); +spin_unlock_irqrestore (&balloon_pages_lock, flags);return n_pages; }EXPORT_SYMBOL_GPL (balloon_page_list_enqueue);
@@ -83,7 +89,7 @@ size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info,unsigned long flags;size_t n_pages =0 ; -spin_lock_irqsave (&b_dev_info->pages_lock, flags); +spin_lock_irqsave (&balloon_pages_lock, flags);list_for_each_entry_safe (page, tmp, &b_dev_info->pages, lru) {if (n_pages == n_req_pages)break ;
@@ -106,7 +112,7 @@ size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info,dec_node_page_state (page,NR_BALLOON_PAGES ); n_pages++; } -spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); +spin_unlock_irqrestore (&balloon_pages_lock, flags);return n_pages; }
@@ -149,9 +155,9 @@ void balloon_page_enqueue(struct balloon_dev_info *b_dev_info, {unsigned long flags; -spin_lock_irqsave (&b_dev_info->pages_lock, flags); +spin_lock_irqsave (&balloon_pages_lock, flags);balloon_page_enqueue_one (b_dev_info, page); -spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); +spin_unlock_irqrestore (&balloon_pages_lock, flags); }EXPORT_SYMBOL_GPL (balloon_page_enqueue);
@@ -191,11 +197,11 @@ struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)* BUG() here, otherwise the balloon driver may get stuck in * an infinite loop while attempting to release all its pages. */ -spin_lock_irqsave (&b_dev_info->pages_lock, flags); +spin_lock_irqsave (&balloon_pages_lock, flags);if (unlikely (list_empty (&b_dev_info->pages) && !b_dev_info->isolated_pages))BUG (); -spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); +spin_unlock_irqrestore (&balloon_pages_lock, flags);return NULL ; } return list_first_entry(&pages, struct page, lru);
@@ -213,10 +219,10 @@ static bool balloon_page_isolate(struct page *page, isolate_mode_t mode)if (!b_dev_info)return false; -spin_lock_irqsave (&b_dev_info->pages_lock, flags); +spin_lock_irqsave (&balloon_pages_lock, flags);list_del (&page->lru); b_dev_info->isolated_pages++; -spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); +spin_unlock_irqrestore (&balloon_pages_lock, flags);return true; }
@@ -234,10 +240,10 @@ static void balloon_page_putback(struct page *page)if (WARN_ON_ONCE (!b_dev_info))return ; -spin_lock_irqsave (&b_dev_info->pages_lock, flags); +spin_lock_irqsave (&balloon_pages_lock, flags);list_add (&page->lru, &b_dev_info->pages); b_dev_info->isolated_pages--; -spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); +spin_unlock_irqrestore (&balloon_pages_lock, flags); } static int balloon_page_migrate(struct page *newpage, struct page *page,
@@ -262,7 +268,7 @@ static int balloon_page_migrate(struct page *newpage, struct page *page,if (rc <0 && rc != -ENOENT )return rc; -spin_lock_irqsave (&b_dev_info->pages_lock, flags); +spin_lock_irqsave (&balloon_pages_lock, flags);if (!rc) {/* Insert the new page into the balloon list. */ get_page (newpage);
@@ -287,7 +293,7 @@ static int balloon_page_migrate(struct page *newpage, struct page *page, } b_dev_info->isolated_pages--; -spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); +spin_unlock_irqrestore (&balloon_pages_lock, flags);/* Free the now-deflated page we isolated in balloon_page_isolate(). */ balloon_page_finalize (page);
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 9/24 mm/balloon_compaction: remove dependency on page lock
In reply to David Hildenbrand (Red Hat)Let's stop using the page lock in balloon code and instead use only the balloon_device_lock.
As soon as we set the PG_movable_ops flag, we might now get isolation callbacks for that page as we are no longer holding the page lock. In there, we'll simply synchronize using the balloon_device_lock.
So in balloon_page_isolate() lookup the balloon_dev_info through page->private under balloon_device_lock.
It's crucial that we update page->private under the balloon_device_lock, so the isolation callback can properly deal with concurrent deflation.
Consequently, make sure that balloon_page_finalize() is called under balloon_device_lock as we remove a page from the list and clear page->private. balloon_page_insert() is already called with the balloon_device_lock held.
Note that the core will still lock the pages, for example in isolate_movable_ops_page(). The lock is there still relevant for handling the PageMovableOpsIsolated flag, but that can be later changed to use an atomic test-and-set instead, or moved into the movable_ops backends.
--- include/linux/balloon_compaction.h | 25 ++++++++++---------- mm/balloon_compaction.c | 38 ++++++++++-------------------- 2 files changed, 25 insertions(+), 38 deletions(-)
include/linux/balloon_compaction.h +13 -12
@@ -12,25 +12,27 @@* is derived from the page type (PageOffline()) combined with the * PG_movable_ops flag (PageMovableOps()). * +* Once the page type and the PG_movable_ops are set, migration code +* can initiate page isolation by invoking the +* movable_operations()->isolate_page() callback +* +* As long as page->private is set, the page is either on the balloon list +* or isolated for migration. If page->private is not set, the page is +* either still getting inflated, or was deflated to be freed by the balloon +* driver soon. Isolation is impossible in both cases. +* * As the page isolation scanning step a compaction thread does is a lockless * procedure (from a page standpoint), it might bring some racy situations while * performing balloon page compaction. In order to sort out these racy scenarios * and safely perform balloon's page compaction and migration we must, always, * ensure following these simple rules: * -* i. Setting the PG_movable_ops flag and page->private with the following -* lock order -* +-page_lock(page); -* +--spin_lock_irq(&balloon_pages_lock); +* i. Inflation/deflation must set/clear page->private under the +* balloon_pages_lock * * ii. isolation or dequeueing procedure must remove the page from balloon * device page list under balloon_pages_lock * -* The functions provided by this interface are placed to help on coping with -* the aforementioned balloon page corner case, as well as to ensure the simple -* set of exposed rules are satisfied while we are dealing with balloon pages -* compaction / migration. -* * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini <[email protected]> */ #ifndef _LINUX_BALLOON_COMPACTION_H
@@ -93,8 +95,7 @@ static inline struct balloon_dev_info *balloon_page_device(struct page *page)* @balloon : pointer to balloon device * @page : page to be assigned as a 'balloon page' * -* Caller must ensure the page is locked and the spin_lock protecting balloon -* pages list is held before inserting a page into the balloon device. +* Caller must ensure the balloon_pages_lock is held. */ static inline void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page)
@@ -119,7 +120,7 @@ static inline gfp_t balloon_mapping_gfp_mask(void)* balloon list for release to the page allocator * @page: page to be released to the page allocator * -* Caller must ensure that the page is locked. +* Caller must ensure the balloon_pages_lock is held. */ static inline void balloon_page_finalize (struct page *page) {
mm/balloon_compaction.c +12 -26
@@ -20,15 +20,7 @@ static DEFINE_SPINLOCK(balloon_pages_lock);static void balloon_page_enqueue_one (struct balloon_dev_info *b_dev_info,struct page *page) { -/* -* Block others from accessing the 'page' when we get around to -* establishing additional references. We should be the only one -* holding a reference to the 'page' at this point. If we are not, then -* memory corruption is possible and we should stop execution. -*/ -BUG_ON (!trylock_page (page));balloon_page_insert (b_dev_info, page); -unlock_page (page);if (b_dev_info->adjust_managed_page_count)adjust_managed_page_count (page,-1 );__count_vm_event (BALLOON_INFLATE );
@@ -93,22 +85,12 @@ size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info,list_for_each_entry_safe (page, tmp, &b_dev_info->pages, lru) {if (n_pages == n_req_pages)break ; - -/* -* Block others from accessing the 'page' while we get around to -* establishing additional references and preparing the 'page' -* to be released by the balloon driver. -*/ -if (!trylock_page (page)) -continue ; -list_del (&page->lru);if (b_dev_info->adjust_managed_page_count)adjust_managed_page_count (page,1 );balloon_page_finalize (page);__count_vm_event (BALLOON_DEFLATE );list_add (&page->lru, pages); -unlock_page (page);dec_node_page_state (page,NR_BALLOON_PAGES ); n_pages++; }
@@ -213,13 +195,19 @@ EXPORT_SYMBOL_GPL(balloon_page_dequeue);static bool balloon_page_isolate (struct page *page,isolate_mode_t mode) { -struct balloon_dev_info *b_dev_info =balloon_page_device (page); +struct balloon_dev_info *b_dev_info;unsigned long flags; -if (!b_dev_info) -return false; -spin_lock_irqsave (&balloon_pages_lock, flags); + b_dev_info =balloon_page_device (page); +if (!b_dev_info) { +/* +* The page already got deflated and removed from the +* balloon list. +*/ +spin_unlock_irqrestore (&balloon_pages_lock, flags); +return false; + }list_del (&page->lru); b_dev_info->isolated_pages++;spin_unlock_irqrestore (&balloon_pages_lock, flags);
@@ -253,9 +241,6 @@ static int balloon_page_migrate(struct page *newpage, struct page *page,unsigned long flags;int rc; -VM_BUG_ON_PAGE (!PageLocked (page), page); -VM_BUG_ON_PAGE (!PageLocked (newpage), newpage); -/* * When we isolated the page, the page was still inflated in a balloon * device. As isolated balloon pages cannot get deflated, we still have
@@ -293,10 +278,11 @@ static int balloon_page_migrate(struct page *newpage, struct page *page, } b_dev_info->isolated_pages--; -spin_unlock_irqrestore (&balloon_pages_lock, flags);/* Free the now-deflated page we isolated in balloon_page_isolate(). */ balloon_page_finalize (page); +spin_unlock_irqrestore (&balloon_pages_lock, flags); +put_page (page);return 0 ;
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 10/24 mm/balloon_compaction: make balloon_mops static
In reply to David Hildenbrand (Red Hat)There is no need to expose this anymore, so let's just make it static.
--- include/linux/balloon_compaction.h | 1 - mm/balloon_compaction.c | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-)
include/linux/balloon_compaction.h +0 -1
@@ -78,7 +78,6 @@ static inline void balloon_devinfo_init(struct balloon_dev_info *balloon) }#ifdef CONFIG_BALLOON_COMPACTION -extern const struct movable_operations balloon_mops;/* * balloon_page_device - get the b_dev_info descriptor for the balloon device * that enqueues the given page.
mm/balloon_compaction.c +1 -1
@@ -288,7 +288,7 @@ static int balloon_page_migrate(struct page *newpage, struct page *page,return 0 ; } -const struct movable_operations balloon_mops = { +static const struct movable_operations balloon_mops = { .migrate_page = balloon_page_migrate, .isolate_page = balloon_page_isolate, .putback_page = balloon_page_putback,
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 11/24 mm/balloon_compaction: drop fs.h include from balloon_compaction.h
In reply to David Hildenbrand (Red Hat)Ever since commit 68f2736a8583 ("mm: Convert all PageMovable users to movable_operations") we no longer store an inode in balloon_dev_info, so we can stop including "fs.h".
--- include/linux/balloon_compaction.h | 1 - 1 file changed, 1 deletion(-)
include/linux/balloon_compaction.h +0 -1
@@ -42,7 +42,6 @@#include <linux/migrate.h> #include <linux/gfp.h> #include <linux/err.h> -#include <linux/fs.h> #include <linux/list.h> /*
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 12/24 drivers/virtio/virtio_balloon: stop using balloon_page_push/pop()
In reply to David Hildenbrand (Red Hat)Let's stop using these functions so we can remove them. They look like belonging to the balloon API for managing the device balloon list when really they are just simple helpers only used by virtio-balloon.
Let's just inline them and switch to a properlist_for_each_entry_safe().
--- drivers/virtio/virtio_balloon.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-)
drivers/virtio/virtio_balloon.c +15 -10
@@ -242,8 +242,8 @@ static void set_page_pfns(struct virtio_balloon *vb,static unsigned int fill_balloon (struct virtio_balloon *vb,size_t num) {unsigned int num_allocated_pages; +struct page *page, *next;unsigned int num_pfns; -struct page *page;LIST_HEAD (pages);/* We can only do one array worth at a time. */
@@ -262,14 +262,15 @@ static unsigned int fill_balloon(struct virtio_balloon *vb, size_t num)break ; } -balloon_page_push (&pages, page); +list_add (&page->lru, &pages); }mutex_lock (&vb->balloon_lock); vb->num_pfns =0 ; -while ((page =balloon_page_pop (&pages))) { +list_for_each_entry_safe (page, next, &pages, lru) { +list_del (&page->lru);balloon_page_enqueue (&vb->vb_dev_info, page);set_page_pfns (vb, vb->pfns + vb->num_pfns, page);
@@ -474,15 +475,19 @@ static inline s64 towards_target(struct virtio_balloon *vb)static unsigned long return_free_pages_to_mm (struct virtio_balloon *vb,unsigned long num_to_return) { -struct page *page; -unsigned long num_returned; +unsigned long num_returned =0 ; +struct page *page, *next; + +if (unlikely (!num_to_return)) +return 0 ;spin_lock_irq (&vb->free_page_list_lock); -for (num_returned =0 ; num_returned < num_to_return; num_returned++) { - page =balloon_page_pop (&vb->free_page_list); -if (!page) -break ; + +list_for_each_entry_safe (page, next, &vb->free_page_list, lru) { +list_del (&page->lru);__free_pages (page,VIRTIO_BALLOON_HINT_BLOCK_ORDER ); +if (++num_returned == num_to_return) +break ; } vb->num_free_page_blocks -= num_returned;spin_unlock_irq (&vb->free_page_list_lock);
@@ -717,7 +722,7 @@ static int get_free_page_and_send(struct virtio_balloon *vb) }virtqueue_kick (vq);spin_lock_irq (&vb->free_page_list_lock); -balloon_page_push (&vb->free_page_list, page); +list_add (&page->lru, &vb->free_page_list); vb->num_free_page_blocks++;spin_unlock_irq (&vb->free_page_list_lock); }else {
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 13/24 mm/balloon_compaction: remove balloon_page_push/pop()
In reply to David Hildenbrand (Red Hat)Let's remove these helpers as they are unused now.
--- include/linux/balloon_compaction.h | 30 ------------------------------ mm/balloon_compaction.c | 5 ++--- 2 files changed, 2 insertions(+), 33 deletions(-)
include/linux/balloon_compaction.h +0 -30
@@ -126,34 +126,4 @@ static inline void balloon_page_finalize(struct page *page)set_page_private (page,0 ); /* PageOffline is sticky until the page is freed to the buddy. */ } - -/* -* balloon_page_push - insert a page into a page list. -* @head : pointer to list -* @page : page to be added -* -* Caller must ensure the page is private and protect the list. -*/ -static inline void balloon_page_push (struct list_head *pages,struct page *page) -{ -list_add (&page->lru, pages); -} - -/* -* balloon_page_pop - remove a page from a page list. -* @head : pointer to list -* @page : page to be added -* -* Caller must ensure the page is private and protect the list. -*/ -static inline struct page *balloon_page_pop (struct list_head *pages) -{ - struct page *page = list_first_entry_or_null(pages, struct page, lru); - -if (!page) -return NULL ; - -list_del (&page->lru); -return page; -}#endif /* _LINUX_BALLOON_COMPACTION_H */
mm/balloon_compaction.c +2 -3
@@ -128,9 +128,8 @@ EXPORT_SYMBOL_GPL(balloon_page_alloc);* Drivers must call this function to properly enqueue a new allocated balloon * page before definitively removing the page from the guest system. * -* Drivers must not call balloon_page_enqueue on pages that have been pushed to -* a list with balloon_page_push before removing them with balloon_page_pop. To -* enqueue a list of pages, use balloon_page_list_enqueue instead. +* Drivers must not enqueue pages while page->lru is still in +* use, and must not use page->lru until a page was unqueued again. */ void balloon_page_enqueue (struct balloon_dev_info *b_dev_info,struct page *page)
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 14/24 mm/balloon_compaction: fold balloon_mapping_gfp_mask() into balloon_page_alloc()
In reply to David Hildenbrand (Red Hat)Let's just remove balloon_mapping_gfp_mask().
--- include/linux/balloon_compaction.h | 7 ------- mm/balloon_compaction.c | 12 ++++++++---- 2 files changed, 8 insertions(+), 11 deletions(-)
include/linux/balloon_compaction.h +0 -7
@@ -106,13 +106,6 @@ static inline void balloon_page_insert(struct balloon_dev_info *balloon,list_add (&page->lru, &balloon->pages); } -static inline gfp_t balloon_mapping_gfp_mask (void ) -{ -if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) -return GFP_HIGHUSER_MOVABLE ; -return GFP_HIGHUSER ; -} -/* * balloon_page_finalize - prepare a balloon page that was removed from the * balloon list for release to the page allocator
mm/balloon_compaction.c +8 -4
@@ -112,10 +112,14 @@ EXPORT_SYMBOL_GPL(balloon_page_list_dequeue);*/ struct page *balloon_page_alloc (void ) { -struct page *page =alloc_page (balloon_mapping_gfp_mask () | - __GFP_NOMEMALLOC | __GFP_NORETRY | - __GFP_NOWARN); -return page; +gfp_t gfp_flags = __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN; + +if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) + gfp_flags |=GFP_HIGHUSER_MOVABLE ; +else + gfp_flags |=GFP_HIGHUSER ; + +return alloc_page (gfp_flags); }EXPORT_SYMBOL_GPL (balloon_page_alloc);
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 15/24 mm/balloon_compaction: move internal helpers to balloon_compaction.c
In reply to David Hildenbrand (Red Hat)Let's move the helpers that are not required by drivers anymore.
While at it, drop the doc of balloon_page_device() as it is trivial.
--- include/linux/balloon_compaction.h | 44 ------------------------------ mm/balloon_compaction.c | 38 ++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 44 deletions(-)
include/linux/balloon_compaction.h +0 -44
@@ -75,48 +75,4 @@ static inline void balloon_devinfo_init(struct balloon_dev_info *balloon) balloon->migratepage =NULL ; balloon->adjust_managed_page_count = false; } - -#ifdef CONFIG_BALLOON_COMPACTION -/* -* balloon_page_device - get the b_dev_info descriptor for the balloon device -* that enqueues the given page. -*/ -static inline struct balloon_dev_info *balloon_page_device (struct page *page) -{ -return (struct balloon_dev_info *)page_private (page); -} -#endif /* CONFIG_BALLOON_COMPACTION */ - -/* -* balloon_page_insert - insert a page into the balloon's page list and make -* the page->private assignment accordingly. -* @balloon : pointer to balloon device -* @page : page to be assigned as a 'balloon page' -* -* Caller must ensure the balloon_pages_lock is held. -*/ -static inline void balloon_page_insert (struct balloon_dev_info *balloon, -struct page *page) -{ -__SetPageOffline (page); -if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) { -SetPageMovableOps (page); -set_page_private (page, (unsigned long )balloon); - } -list_add (&page->lru, &balloon->pages); -} - -/* -* balloon_page_finalize - prepare a balloon page that was removed from the -* balloon list for release to the page allocator -* @page: page to be released to the page allocator -* -* Caller must ensure the balloon_pages_lock is held. -*/ -static inline void balloon_page_finalize (struct page *page) -{ -if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) -set_page_private (page,0 ); -/* PageOffline is sticky until the page is freed to the buddy. */ -}#endif /* _LINUX_BALLOON_COMPACTION_H */
mm/balloon_compaction.c +38 -0
@@ -17,6 +17,44 @@*/ static DEFINE_SPINLOCK (balloon_pages_lock); +static struct balloon_dev_info *balloon_page_device (struct page *page) +{ +return (struct balloon_dev_info *)page_private (page); +} + +/* +* balloon_page_insert - insert a page into the balloon's page list and make +* the page->private assignment accordingly. +* @balloon : pointer to balloon device +* @page : page to be assigned as a 'balloon page' +* +* Caller must ensure the balloon_pages_lock is held. +*/ +static void balloon_page_insert (struct balloon_dev_info *balloon, +struct page *page) +{ +__SetPageOffline (page); +if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) { +SetPageMovableOps (page); +set_page_private (page, (unsigned long )balloon); + } +list_add (&page->lru, &balloon->pages); +} + +/* +* balloon_page_finalize - prepare a balloon page that was removed from the +* balloon list for release to the page allocator +* @page: page to be released to the page allocator +* +* Caller must ensure the balloon_pages_lock is held. +*/ +static void balloon_page_finalize (struct page *page) +{ +if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) +set_page_private (page,0 ); +/* PageOffline is sticky until the page is freed to the buddy. */ +} +static void balloon_page_enqueue_one (struct balloon_dev_info *b_dev_info,struct page *page) {
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 16/24 mm/balloon_compaction: assert that the balloon_pages_lock is held
In reply to David Hildenbrand (Red Hat)Let's add some sanity checks for holding the balloon_pages_lock when we're effectively inflating/deflating a page.
--- mm/balloon_compaction.c | 2 ++ 1 file changed, 2 insertions(+)
mm/balloon_compaction.c +2 -0
@@ -33,6 +33,7 @@ static struct balloon_dev_info *balloon_page_device(struct page *page)static void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) { +lockdep_assert_held (&balloon_pages_lock);__SetPageOffline (page);if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) {SetPageMovableOps (page);
@@ -50,6 +51,7 @@ static void balloon_page_insert(struct balloon_dev_info *balloon,*/ static void balloon_page_finalize (struct page *page) { +lockdep_assert_held (&balloon_pages_lock);if (IS_ENABLED (CONFIG_BALLOON_COMPACTION ))set_page_private (page,0 );/* PageOffline is sticky until the page is freed to the buddy. */
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 17/24 mm/balloon_compaction: mark remaining functions for having proper kerneldoc
In reply to David Hildenbrand (Red Hat)Looks like all we are missing for proper kerneldoc is another "*".
--- mm/balloon_compaction.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
mm/balloon_compaction.c +5 -5
@@ -22,7 +22,7 @@ static struct balloon_dev_info *balloon_page_device(struct page *page)return (struct balloon_dev_info *)page_private (page); } -/* +/** * balloon_page_insert - insert a page into the balloon's page list and make * the page->private assignment accordingly. * @balloon : pointer to balloon device
@@ -42,7 +42,7 @@ static void balloon_page_insert(struct balloon_dev_info *balloon,list_add (&page->lru, &balloon->pages); } -/* +/** * balloon_page_finalize - prepare a balloon page that was removed from the * balloon list for release to the page allocator * @page: page to be released to the page allocator
@@ -140,7 +140,7 @@ size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info, }EXPORT_SYMBOL_GPL (balloon_page_list_dequeue); -/* +/** * balloon_page_alloc - allocates a new page for insertion into the balloon * page list. *
@@ -163,7 +163,7 @@ struct page *balloon_page_alloc(void) }EXPORT_SYMBOL_GPL (balloon_page_alloc); -/* +/** * balloon_page_enqueue - inserts a new page into the balloon page list. * * @b_dev_info: balloon device descriptor where we will insert a new page
@@ -186,7 +186,7 @@ void balloon_page_enqueue(struct balloon_dev_info *b_dev_info, }EXPORT_SYMBOL_GPL (balloon_page_enqueue); -/* +/** * balloon_page_dequeue - removes a page from balloon's page list and returns * its address to allow the driver to release the page. * @b_dev_info: balloon device descriptor where we will grab a page from.
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 18/24 mm/balloon_compaction: remove "extern" from functions
In reply to David Hildenbrand (Red Hat)Adding "extern" to functions is frowned-upon. Let's just get rid of it for all functions here.
--- include/linux/balloon_compaction.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
include/linux/balloon_compaction.h +8 -8
@@ -59,14 +59,14 @@ struct balloon_dev_info {bool adjust_managed_page_count; }; -extern struct page *balloon_page_alloc (void ); -extern void balloon_page_enqueue (struct balloon_dev_info *b_dev_info, -struct page *page); -extern struct page *balloon_page_dequeue (struct balloon_dev_info *b_dev_info); -extern size_t balloon_page_list_enqueue (struct balloon_dev_info *b_dev_info, -struct list_head *pages); -extern size_t balloon_page_list_dequeue (struct balloon_dev_info *b_dev_info, -struct list_head *pages,size_t n_req_pages); +struct page *balloon_page_alloc (void ); +void balloon_page_enqueue (struct balloon_dev_info *b_dev_info, +struct page *page); +struct page *balloon_page_dequeue (struct balloon_dev_info *b_dev_info); +size_t balloon_page_list_enqueue (struct balloon_dev_info *b_dev_info, +struct list_head *pages); +size_t balloon_page_list_dequeue (struct balloon_dev_info *b_dev_info, +struct list_head *pages,size_t n_req_pages);static inline void balloon_devinfo_init (struct balloon_dev_info *balloon) {
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 19/24 mm/vmscan: drop inclusion of balloon_compaction.h
In reply to David Hildenbrand (Red Hat)Before commit b1123ea6d3b3 ("mm: balloon: use general non-lru movable page feature"), the include was required because of isolated_balloon_page().
It's no longer required, so let's remove it.
--- mm/vmscan.c | 1 - 1 file changed, 1 deletion(-)
mm/vmscan.c +0 -1
@@ -63,7 +63,6 @@#include <asm/div64.h> #include <linux/swapops.h> -#include <linux/balloon_compaction.h> #include <linux/sched/sysctl.h> #include "internal.h"
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 20/24 mm: rename balloon_compaction.(c|h) to balloon.(c|h)
In reply to David Hildenbrand (Red Hat)Even without CONFIG_BALLOON_COMPACTION this infrastructure implements basic list and page management for a memory balloon.
---
Documentation/core-api/mm-api.rst | 2 +-
MAINTAINERS | 4 ++--
arch/powerpc/platforms/pseries/cmm.c | 2 +-
drivers/misc/vmw_balloon.c | 2 +-
drivers/virtio/virtio_balloon.c | 2 +-
include/linux/{balloon_compaction.h => balloon.h} | 11 +++++------
mm/Makefile | 2 +-
mm/{balloon_compaction.c => balloon.c} | 7 +++----
8 files changed, 15 insertions(+), 17 deletions(-)
rename include/linux/{balloon_compaction.h => balloon.h} (92%)
rename mm/{balloon_compaction.c => balloon.c} (98%)Documentation/core-api/mm-api.rst +1 -1
@@ -130,5 +130,5 @@ More Memory Management Functions .. kernel-doc:: mm/vmscan.c .. kernel-doc:: mm/memory_hotplug.c .. kernel-doc:: mm/mmu_notifier.c -.. kernel-doc:: mm/balloon_compaction.c +.. kernel-doc:: mm/balloon.c .. kernel-doc:: mm/huge_memory.c
MAINTAINERS +2 -2
@@ -27536,9 +27536,9 @@ M: David Hildenbrand <[email protected]> L: [email protected] S: Maintained F: drivers/virtio/virtio_balloon.c -F: include/linux/balloon_compaction.h +F: include/linux/balloon.h F: include/uapi/linux/virtio_balloon.h -F: mm/balloon_compaction.c +F: mm/balloon.c VIRTIO BLOCK AND SCSI DRIVERS M: "Michael S. Tsirkin" <[email protected]>
arch/powerpc/platforms/pseries/cmm.c +1 -1
@@ -19,7 +19,7 @@#include <linux/stringify.h> #include <linux/swap.h> #include <linux/device.h> -#include <linux/balloon_compaction.h> +#include <linux/balloon.h> #include <asm/firmware.h> #include <asm/hvcall.h> #include <asm/mmu.h>
drivers/misc/vmw_balloon.c +1 -1
@@ -29,7 +29,7 @@#include <linux/rwsem.h> #include <linux/slab.h> #include <linux/spinlock.h> -#include <linux/balloon_compaction.h> +#include <linux/balloon.h> #include <linux/vmw_vmci_defs.h> #include <linux/vmw_vmci_api.h> #include <asm/hypervisor.h>
drivers/virtio/virtio_balloon.c +1 -1
@@ -13,7 +13,7 @@#include <linux/delay.h> #include <linux/slab.h> #include <linux/module.h> -#include <linux/balloon_compaction.h> +#include <linux/balloon.h> #include <linux/oom.h> #include <linux/wait.h> #include <linux/mm.h>
include/linux/balloon.h renamed from include/linux/balloon_compaction.h +5 -6
@@ -1,8 +1,7 @@/* SPDX-License-Identifier: GPL-2.0 */ /* -* include/linux/balloon_compaction.h -* -* Common interface definitions for making balloon pages movable by compaction. +* Common interface for implementing a memory balloon, including support +* for migration of pages inflated in a memory balloon. * * Balloon page migration makes use of the general "movable_ops page migration" * feature.
@@ -35,8 +34,8 @@* * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini <[email protected]> */ -#ifndef _LINUX_BALLOON_COMPACTION_H -#define _LINUX_BALLOON_COMPACTION_H +#ifndef _LINUX_BALLOON_H +#define _LINUX_BALLOON_H#include <linux/pagemap.h> #include <linux/page-flags.h> #include <linux/migrate.h>
@@ -75,4 +74,4 @@ static inline void balloon_devinfo_init(struct balloon_dev_info *balloon) balloon->migratepage =NULL ; balloon->adjust_managed_page_count = false; } -#endif /* _LINUX_BALLOON_COMPACTION_H */ +#endif /* _LINUX_BALLOON_H */
mm/Makefile +1 -1
@@ -122,7 +122,7 @@ obj-$(CONFIG_CMA) += cma.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_NUMA_MEMBLKS) += numa_memblks.o obj-$(CONFIG_NUMA_EMU) += numa_emulation.o -obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o +obj-$(CONFIG_MEMORY_BALLOON) += balloon.o obj-$(CONFIG_PAGE_EXTENSION) += page_ext.o obj-$(CONFIG_PAGE_TABLE_CHECK) += page_table_check.o obj-$(CONFIG_CMA_DEBUGFS) += cma_debug.o
mm/balloon.c renamed from mm/balloon_compaction.c +3 -4
@@ -1,15 +1,14 @@// SPDX-License-Identifier: GPL-2.0-only /* -* mm/balloon_compaction.c -* -* Common interface for making balloon pages movable by compaction. +* Common interface for implementing a memory balloon, including support +* for migration of pages inflated in a memory balloon. * * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini <[email protected]> */ #include <linux/mm.h> #include <linux/slab.h> #include <linux/export.h> -#include <linux/balloon_compaction.h> +#include <linux/balloon.h> /* * Lock protecting the balloon_dev_info of all devices. We don't really
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 21/24 mm/kconfig: make BALLOON_COMPACTION depend on MIGRATION
In reply to David Hildenbrand (Red Hat)Migration support for balloon memory depends on MIGRATION not COMPACTION. Compaction is simply another user of page migration.
The last dependency on compaction.c was effectively removed with commit 3d388584d599 ("mm: convert "movable" flag in page->mapping to a page flag"). Ever since, everything for handling movable_ops page migration resides in core migration code.
So let's change the dependency and adjust the description + help text.
We'll rename BALLOON_COMPACTION separately next.
--- mm/Kconfig | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-)
mm/Kconfig +7 -10
@@ -599,17 +599,14 @@ config MEMORY_BALLOON # # support for memory balloon compaction config BALLOON_COMPACTION - bool "Allow for balloon memory compaction/migration" + bool "Allow for balloon memory migration" default y - depends on COMPACTION && MEMORY_BALLOON - help - Memory fragmentation introduced by ballooning might reduce - significantly the number of 2MB contiguous memory blocks that can be - used within a guest, thus imposing performance penalties associated - with the reduced number of transparent huge pages that could be used - by the guest workload. Allowing the compaction & migration for memory - pages enlisted as being part of memory balloon devices avoids the - scenario aforementioned and helps improving memory defragmentation. + depends on MIGRATION && MEMORY_BALLOON + help + Allow for migration of pages inflated in a memory balloon such that + they can be allocated from memory areas only available for movable + allocations (e.g., ZONE_MOVABLE, CMA) and such that they can be + migrated for memory defragmentation purposes by memory compaction. # # support for memory compaction
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 22/24 mm: rename CONFIG_BALLOON_COMPACTION to CONFIG_BALLOON_MIGRATION
In reply to David Hildenbrand (Red Hat)While compaction depends on migration, the other direction is not the case. So let's make it clearer that this is all about migration of balloon pages.
Adjust all comments/docs in the core to talk about "migration" instead of "compaction".
While at it add some "/* CONFIG_BALLOON_MIGRATION */".
--- Documentation/admin-guide/mm/memory-hotplug.rst | 8 ++++---- arch/powerpc/platforms/pseries/cmm.c | 8 ++++---- drivers/misc/vmw_balloon.c | 8 ++++---- drivers/virtio/virtio_balloon.c | 6 +++--- include/linux/balloon.h | 12 ++++++------ include/linux/vm_event_item.h | 4 ++-- mm/Kconfig | 4 ++-- mm/balloon.c | 10 +++++----- mm/memory_hotplug.c | 4 ++-- mm/migrate.c | 2 +- mm/vmstat.c | 4 ++-- 11 files changed, 35 insertions(+), 35 deletions(-)
Documentation/admin-guide/mm/memory-hotplug.rst +4 -4
@@ -603,11 +603,11 @@ ZONE_MOVABLE, especially when fine-tuning zone ratios: memory for metadata and page tables in the direct map; having a lot of offline memory blocks is not a typical case, though. -- Memory ballooning without balloon compaction is incompatible with - ZONE_MOVABLE. Only some implementations, such as virtio-balloon and - pseries CMM, fully support balloon compaction. +- Memory ballooning without support for balloon memory migration is incompatible + with ZONE_MOVABLE. Only some implementations, such as virtio-balloon and + pseries CMM, fully support balloon memory migration. - Further, the CONFIG_BALLOON_COMPACTION kernel configuration option might be + Further, the CONFIG_BALLOON_MIGRATION kernel configuration option might be disabled. In that case, balloon inflation will only perform unmovable allocations and silently create a zone imbalance, usually triggered by inflation requests from the hypervisor.
arch/powerpc/platforms/pseries/cmm.c +4 -4
@@ -494,7 +494,7 @@ static struct notifier_block cmm_mem_nb = { .priority = CMM_MEM_HOTPLUG_PRI }; -#ifdef CONFIG_BALLOON_COMPACTION +#ifdef CONFIG_BALLOON_MIGRATION static int cmm_migratepage (struct balloon_dev_info *b_dev_info,struct page *newpage,struct page *page,enum migrate_mode mode)
@@ -520,10 +520,10 @@ static int cmm_migratepage(struct balloon_dev_info *b_dev_info,plpar_page_set_active (page); return 0; } -#else /* CONFIG_BALLOON_COMPACTION */ +#else /* CONFIG_BALLOON_MIGRATION */ int cmm_migratepage (struct balloon_dev_info *b_dev_info,struct page *newpage,struct page *page,enum migrate_mode mode); -#endif /* CONFIG_BALLOON_COMPACTION */ +#endif /* CONFIG_BALLOON_MIGRATION */ /** * cmm_init - Module initialization
@@ -540,7 +540,7 @@ static int cmm_init(void)balloon_devinfo_init (&b_dev_info); b_dev_info.adjust_managed_page_count = true; -if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) +if (IS_ENABLED (CONFIG_BALLOON_MIGRATION )) b_dev_info.migratepage = cmm_migratepage; rc =register_oom_notifier (&cmm_oom_nb);
drivers/misc/vmw_balloon.c +4 -4
@@ -1719,7 +1719,7 @@ static inline void vmballoon_debugfs_exit(struct vmballoon *b)#endif /* CONFIG_DEBUG_FS */ -#ifdef CONFIG_BALLOON_COMPACTION +#ifdef CONFIG_BALLOON_MIGRATION /** * vmballoon_migratepage() - migrates a balloon page. * @b_dev_info: balloon device information descriptor.
@@ -1803,11 +1803,11 @@ static int vmballoon_migratepage(struct balloon_dev_info *b_dev_info,up_read (&b->conf_sem); return ret; } -#else /* CONFIG_BALLOON_COMPACTION */ +#else /* CONFIG_BALLOON_MIGRATION */ int vmballoon_migratepage (struct balloon_dev_info *b_dev_info,struct page *newpage,struct page *page,enum migrate_mode mode); -#endif /* CONFIG_BALLOON_COMPACTION */ +#endif /* CONFIG_BALLOON_MIGRATION */ static int __init vmballoon_init (void ) {
@@ -1827,7 +1827,7 @@ static int __init vmballoon_init(void)return error;balloon_devinfo_init (&balloon.b_dev_info); -if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) +if (IS_ENABLED (CONFIG_BALLOON_MIGRATION )) balloon.b_dev_info.migratepage = vmballoon_migratepage;INIT_LIST_HEAD (&balloon.huge_pages);
drivers/virtio/virtio_balloon.c +3 -3
@@ -802,7 +802,7 @@ static void report_free_page_func(struct work_struct *work) } } -#ifdef CONFIG_BALLOON_COMPACTION +#ifdef CONFIG_BALLOON_MIGRATION /* * virtballoon_migratepage - perform the balloon page migration on behalf of * a compaction thread. (called under page lock)
@@ -851,7 +851,7 @@ static int virtballoon_migratepage(struct balloon_dev_info *vb_dev_info,mutex_unlock (&vb->balloon_lock); return 0; } -#endif /* CONFIG_BALLOON_COMPACTION */ +#endif /* CONFIG_BALLOON_MIGRATION */ static unsigned long shrink_free_pages (struct virtio_balloon *vb,unsigned long pages_to_free)
@@ -948,7 +948,7 @@ static int virtballoon_probe(struct virtio_device *vdev)if (!virtio_has_feature (vb->vdev,VIRTIO_BALLOON_F_DEFLATE_ON_OOM )) vb->vb_dev_info.adjust_managed_page_count = true; -#ifdef CONFIG_BALLOON_COMPACTION +#ifdef CONFIG_BALLOON_MIGRATION vb->vb_dev_info.migratepage = virtballoon_migratepage;#endif if (virtio_has_feature (vdev,VIRTIO_BALLOON_F_FREE_PAGE_HINT )) {
include/linux/balloon.h +6 -6
@@ -22,9 +22,9 @@* * As the page isolation scanning step a compaction thread does is a lockless * procedure (from a page standpoint), it might bring some racy situations while -* performing balloon page compaction. In order to sort out these racy scenarios -* and safely perform balloon's page compaction and migration we must, always, -* ensure following these simple rules: +* performing balloon page migration. In order to sort out these racy scenarios +* and safely perform balloon's page migration we must, always, ensure following +* these simple rules: * * i. Inflation/deflation must set/clear page->private under the * balloon_pages_lock
@@ -45,10 +45,10 @@/* * Balloon device information descriptor. -* This struct is used to allow the common balloon compaction interface +* This struct is used to allow the common balloon page migration interface * procedures to find the proper balloon device holding memory pages they'll -* have to cope for page compaction / migration, as well as it serves the -* balloon driver as a page book-keeper for its registered balloon devices. +* have to cope for page migration, as well as it serves the balloon driver as +* a page book-keeper for its registered balloon devices. */ struct balloon_dev_info {unsigned long isolated_pages;/* # of isolated pages for migration */
include/linux/vm_event_item.h +2 -2
@@ -125,9 +125,9 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,#ifdef CONFIG_MEMORY_BALLOON BALLOON_INFLATE, BALLOON_DEFLATE, -#ifdef CONFIG_BALLOON_COMPACTION +#ifdef CONFIG_BALLOON_MIGRATION BALLOON_MIGRATE, -#endif +#endif /* CONFIG_BALLOON_MIGRATION */ #endif #ifdef CONFIG_DEBUG_TLBFLUSH NR_TLB_REMOTE_FLUSH, /* cpu tried to flush others' tlbs */
mm/Kconfig +2 -2
@@ -597,8 +597,8 @@ config MEMORY_BALLOON bool # -# support for memory balloon compaction -config BALLOON_COMPACTION +# support for memory balloon page migration +config BALLOON_MIGRATION bool "Allow for balloon memory migration" default y depends on MIGRATION && MEMORY_BALLOON
mm/balloon.c +5 -5
@@ -34,7 +34,7 @@ static void balloon_page_insert(struct balloon_dev_info *balloon, {lockdep_assert_held (&balloon_pages_lock);__SetPageOffline (page); -if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) { +if (IS_ENABLED (CONFIG_BALLOON_MIGRATION )) {SetPageMovableOps (page);set_page_private (page, (unsigned long )balloon); }
@@ -51,7 +51,7 @@ static void balloon_page_insert(struct balloon_dev_info *balloon,static void balloon_page_finalize (struct page *page) {lockdep_assert_held (&balloon_pages_lock); -if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) +if (IS_ENABLED (CONFIG_BALLOON_MIGRATION ))set_page_private (page,0 );/* PageOffline is sticky until the page is freed to the buddy. */ }
@@ -153,7 +153,7 @@ struct page *balloon_page_alloc(void) {gfp_t gfp_flags = __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN; -if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) +if (IS_ENABLED (CONFIG_BALLOON_MIGRATION )) gfp_flags |=GFP_HIGHUSER_MOVABLE ;else gfp_flags |=GFP_HIGHUSER ;
@@ -232,7 +232,7 @@ struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info) }EXPORT_SYMBOL_GPL (balloon_page_dequeue); -#ifdef CONFIG_BALLOON_COMPACTION +#ifdef CONFIG_BALLOON_MIGRATION static bool balloon_page_isolate (struct page *page,isolate_mode_t mode)
@@ -342,4 +342,4 @@ static int __init balloon_init(void) }core_initcall (balloon_init); -#endif /* CONFIG_BALLOON_COMPACTION */ +#endif /* CONFIG_BALLOON_MIGRATION */
mm/memory_hotplug.c +2 -2
@@ -946,8 +946,8 @@ static struct zone *default_kernel_zone_for_pfn(int nid, unsigned long start_pfn* We rely on "present pages" instead of "managed pages", as the latter is * highly unreliable and dynamic in virtualized environments, and does not * consider boot time allocations. For example, memory ballooning adjusts the -* managed pages when inflating/deflating the balloon, and balloon compaction -* can even migrate inflated pages between zones. +* managed pages when inflating/deflating the balloon, and balloon page +* migration can even migrate inflated pages between zones. * * Using "present pages" is better but some things to keep in mind are: *
mm/migrate.c +1 -1
@@ -88,7 +88,7 @@ static const struct movable_operations *page_movable_ops(struct page *page)* back to the buddy. */ if (PageOffline (page)) -/* Only balloon compaction sets PageOffline pages movable. */ +/* Only balloon page migration sets PageOffline pages movable. */ return offline_movable_ops;if (PageZsmalloc (page))return zsmalloc_movable_ops;
mm/vmstat.c +2 -2
@@ -1422,9 +1422,9 @@ const char * const vmstat_text[] = { #ifdef CONFIG_MEMORY_BALLOON [I(BALLOON_INFLATE)] = "balloon_inflate", [I (BALLOON_DEFLATE )] ="balloon_deflate" , -#ifdef CONFIG_BALLOON_COMPACTION +#ifdef CONFIG_BALLOON_MIGRATION [I (BALLOON_MIGRATE )] ="balloon_migrate" , -#endif +#endif /* CONFIG_BALLOON_MIGRATION */ #endif /* CONFIG_MEMORY_BALLOON */ #ifdef CONFIG_DEBUG_TLBFLUSH [I(NR_TLB_REMOTE_FLUSH)] = "nr_tlb_remote_flush",
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 23/24 mm: rename CONFIG_MEMORY_BALLOON -> CONFIG_BALLOON
In reply to David Hildenbrand (Red Hat)Let's make it consistent with the naming of the files but also with the naming of CONFIG_BALLOON_MIGRATION.
While at it, add a "/* CONFIG_BALLOON */".
--- arch/powerpc/platforms/pseries/Kconfig | 2 +- drivers/misc/Kconfig | 2 +- drivers/virtio/Kconfig | 2 +- include/linux/vm_event_item.h | 4 ++-- mm/Kconfig | 4 ++-- mm/Makefile | 2 +- mm/vmstat.c | 4 ++-- 7 files changed, 10 insertions(+), 10 deletions(-)
arch/powerpc/platforms/pseries/Kconfig +1 -1
@@ -120,7 +120,7 @@ config PPC_SMLPAR config CMM tristate "Collaborative memory management" depends on PPC_SMLPAR - select MEMORY_BALLOON + select BALLOON default y help Select this option, if you want to enable the kernel interface
drivers/misc/Kconfig +1 -1
@@ -410,7 +410,7 @@ config DS1682 config VMWARE_BALLOON tristate "VMware Balloon Driver" depends on VMWARE_VMCI && X86 && HYPERVISOR_GUEST - select MEMORY_BALLOON + select BALLOON help This is VMware physical memory management driver which acts like a "balloon" that can be inflated to reclaim physical pages
drivers/virtio/Kconfig +1 -1
@@ -112,7 +112,7 @@ config VIRTIO_PMEM config VIRTIO_BALLOON tristate "Virtio balloon driver" depends on VIRTIO - select MEMORY_BALLOON + select BALLOON select PAGE_REPORTING help This driver supports increasing and decreasing the amount
include/linux/vm_event_item.h +2 -2
@@ -122,13 +122,13 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, THP_SWPOUT, THP_SWPOUT_FALLBACK,#endif -#ifdef CONFIG_MEMORY_BALLOON +#ifdef CONFIG_BALLOON BALLOON_INFLATE, BALLOON_DEFLATE,#ifdef CONFIG_BALLOON_MIGRATION BALLOON_MIGRATE,#endif /* CONFIG_BALLOON_MIGRATION */ -#endif +#endif /* CONFIG_BALLOON */ #ifdef CONFIG_DEBUG_TLBFLUSH NR_TLB_REMOTE_FLUSH ,/* cpu tried to flush others' tlbs */ NR_TLB_REMOTE_FLUSH_RECEIVED,/* cpu received ipi for flush */
mm/Kconfig +2 -2
@@ -593,7 +593,7 @@ config SPLIT_PMD_PTLOCKS # # support for memory balloon -config MEMORY_BALLOON +config BALLOON bool #
@@ -601,7 +601,7 @@ config MEMORY_BALLOON config BALLOON_MIGRATION bool "Allow for balloon memory migration" default y - depends on MIGRATION && MEMORY_BALLOON + depends on MIGRATION && BALLOON help Allow for migration of pages inflated in a memory balloon such that they can be allocated from memory areas only available for movable
mm/Makefile +1 -1
@@ -122,7 +122,7 @@ obj-$(CONFIG_CMA) += cma.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_NUMA_MEMBLKS) += numa_memblks.o obj-$(CONFIG_NUMA_EMU) += numa_emulation.o -obj-$(CONFIG_MEMORY_BALLOON) += balloon.o +obj-$(CONFIG_BALLOON) += balloon.o obj-$(CONFIG_PAGE_EXTENSION) += page_ext.o obj-$(CONFIG_PAGE_TABLE_CHECK) += page_table_check.o obj-$(CONFIG_CMA_DEBUGFS) += cma_debug.o
mm/vmstat.c +2 -2
@@ -1419,13 +1419,13 @@ const char * const vmstat_text[] = { [I(THP_SWPOUT)] = "thp_swpout", [I (THP_SWPOUT_FALLBACK )] ="thp_swpout_fallback" , #endif -#ifdef CONFIG_MEMORY_BALLOON +#ifdef CONFIG_BALLOON [I(BALLOON_INFLATE)] = "balloon_inflate", [I (BALLOON_DEFLATE )] ="balloon_deflate" , #ifdef CONFIG_BALLOON_MIGRATION [I (BALLOON_MIGRATE )] ="balloon_migrate" , #endif /* CONFIG_BALLOON_MIGRATION */ -#endif /* CONFIG_MEMORY_BALLOON */ +#endif /* CONFIG_BALLOON */ #ifdef CONFIG_DEBUG_TLBFLUSH [I(NR_TLB_REMOTE_FLUSH)] = "nr_tlb_remote_flush", [I(NR_TLB_REMOTE_FLUSH_RECEIVED)] = "nr_tlb_remote_flush_received",
Signature · 1 line
2.52.0
David Hildenbrand (Red Hat)
maintainerPATCH v3 24/24 MAINTAINERS: move memory balloon infrastructure to "MEMORY MANAGEMENT - BALLOON"
In reply to David Hildenbrand (Red Hat)Nowadays, there is nothing virtio-balloon specific anymore about these files, the basic infrastructure is used by multiple memory balloon drivers.
For now we'll route it through Andrew's tree, maybe in some future it makes sense to route this through a separate tree.
--- MAINTAINERS | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
MAINTAINERS +11 -2
@@ -16454,6 +16454,17 @@ T: quilt git://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new F: mm/ F: tools/mm/ +MEMORY MANAGEMENT - BALLOON +M: Andrew Morton <[email protected]> +M: David Hildenbrand <[email protected]> +L: [email protected] +L: [email protected] +S: Maintained +W: http://www.linux-mm.org +T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm +F: include/linux/balloon.h +F: mm/balloon.c + MEMORY MANAGEMENT - CORE M: Andrew Morton <[email protected]> M: David Hildenbrand <[email protected]>
@@ -27536,9 +27547,7 @@ M: David Hildenbrand <[email protected]> L: [email protected] S: Maintained F: drivers/virtio/virtio_balloon.c -F: include/linux/balloon.h F: include/uapi/linux/virtio_balloon.h -F: mm/balloon.c VIRTIO BLOCK AND SCSI DRIVERS M: "Michael S. Tsirkin" <[email protected]>
Signature · 1 line
2.52.0
Andrew Morton
maintainerQuoted 6 lines · On Tue, 20 Jan 2026 00:01:08 +0100 "David Hildenbrand (Red Hat)"…
I started with wanting to remove the dependency of the balloon infrastructure on the page lock, but ended up performing various other cleanups, some of which I had on my todo list for years.
This series heavily cleans up and simplifies our balloon infrastructure, including our balloon page migration functionality.
Updated, thanks.
fwiw, below is how v3 altered mm.git:
include/linux/balloon.h renamed from include/linux/balloon.h~b +5 -5
@@ -22,9 +22,9 @@* * As the page isolation scanning step a compaction thread does is a lockless * procedure (from a page standpoint), it might bring some racy situations while -* performing balloon page compaction. In order to sort out these racy scenarios -* and safely perform balloon's page compaction and migration we must, always, -* ensure following these simple rules: +* performing balloon page migration. In order to sort out these racy scenarios +* and safely perform balloon's page migration we must, always, ensure following +* these simple rules: * * i. Inflation/deflation must set/clear page->private under the * balloon_pages_lock
@@ -47,8 +47,8 @@* Balloon device information descriptor. * This struct is used to allow the common balloon page migration interface * procedures to find the proper balloon device holding memory pages they'll -* have to cope for page compaction / migration, as well as it serves the -* balloon driver as a page book-keeper for its registered balloon devices. +* have to cope for page migration, as well as it serves the balloon driver as +* a page book-keeper for its registered balloon devices. */ struct balloon_dev_info {unsigned long isolated_pages;/* # of isolated pages for migration */
mm/balloon.c renamed from mm/balloon.c~b +18 -16
@@ -16,7 +16,7 @@*/ static DEFINE_SPINLOCK (balloon_pages_lock); -static inline struct balloon_dev_info *balloon_page_device (struct page *page) +static struct balloon_dev_info *balloon_page_device (struct page *page) {return (struct balloon_dev_info *)page_private (page); }
@@ -29,7 +29,7 @@ static inline struct balloon_dev_info *b* * Caller must ensure the balloon_pages_lock is held. */ -static inline void balloon_page_insert (struct balloon_dev_info *balloon, +static void balloon_page_insert (struct balloon_dev_info *balloon,struct page *page) {lockdep_assert_held (&balloon_pages_lock);
@@ -48,7 +48,7 @@ static inline void balloon_page_insert(s* * Caller must ensure the balloon_pages_lock is held. */ -static inline void balloon_page_finalize (struct page *page) +static void balloon_page_finalize (struct page *page) {lockdep_assert_held (&balloon_pages_lock);if (IS_ENABLED (CONFIG_BALLOON_MIGRATION ))
@@ -262,7 +262,11 @@ static void balloon_page_putback(structstruct balloon_dev_info *b_dev_info =balloon_page_device (page);unsigned long flags; -/* Isolated balloon pages cannot get deflated. */ +/* +* When we isolated the page, the page was still inflated in a balloon +* device. As isolated balloon pages cannot get deflated, we still have +* a balloon device here. +*/ if (WARN_ON_ONCE (!b_dev_info))return ;
@@ -279,18 +283,22 @@ static int balloon_page_migrate(struct punsigned long flags;int rc; -/* Isolated balloon pages cannot get deflated. */ +/* +* When we isolated the page, the page was still inflated in a balloon +* device. As isolated balloon pages cannot get deflated, we still have +* a balloon device here. +*/ if (WARN_ON_ONCE (!b_dev_info))return -EAGAIN ; rc = b_dev_info->migratepage (b_dev_info, newpage, page, mode); -switch (rc) { -case 0 : -spin_lock_irqsave (&balloon_pages_lock, flags); +if (rc <0 && rc != -ENOENT ) +return rc; +spin_lock_irqsave (&balloon_pages_lock, flags); +if (!rc) {/* Insert the new page into the balloon list. */ get_page (newpage); -balloon_page_insert (b_dev_info, newpage);__count_vm_event (BALLOON_MIGRATE );
@@ -303,18 +311,12 @@ static int balloon_page_migrate(struct padjust_managed_page_count (page,1 );adjust_managed_page_count (newpage,-1 ); } -break ; -case -ENOENT : -spin_lock_irqsave (&balloon_pages_lock, flags); - + }else {/* Old page was deflated but new page not inflated. */ __count_vm_event (BALLOON_DEFLATE );if (b_dev_info->adjust_managed_page_count)adjust_managed_page_count (page,1 ); -break ; -default : -return rc; } b_dev_info->isolated_pages--;
_
David Hildenbrand (Red Hat)
maintainerRe: [PATCH v3 15/24] mm/balloon_compaction: move internal helpers to balloon_compaction.c
In reply to David Hildenbrand (Red Hat)Quoted 104 lines · David Hildenbrand (Red Hat)
Let's move the helpers that are not required by drivers anymore.
While at it, drop the doc of
balloon_page_device()as it is trivial.Reviewed-by Lorenzo Stoakes <[email protected]>Signed-off-by David Hildenbrand (Red Hat) <[email protected]>--- include/linux/balloon_compaction.h | 44 ------------------------------ mm/balloon_compaction.c | 38 ++++++++++++++++++++++++++ 2 files changed, 38 insertions(+), 44 deletions(-)include/linux/balloon_compaction.h +0
-44@@ -75,48 +75,4 @@ static inline void balloon_devinfo_init(struct balloon_dev_info *balloon) balloon->migratepage =NULL ; balloon->adjust_managed_page_count = false; } - -#ifdef CONFIG_BALLOON_COMPACTION -/* -* balloon_page_device - get the b_dev_info descriptor for the balloon device -* that enqueues the given page. -*/ -static inline struct balloon_dev_info *balloon_page_device (struct page *page) -{ -return (struct balloon_dev_info *)page_private (page); -} -#endif /* CONFIG_BALLOON_COMPACTION */ - -/* -* balloon_page_insert - insert a page into the balloon's page list and make -* the page->private assignment accordingly. -* @balloon : pointer to balloon device -* @page : page to be assigned as a 'balloon page' -* -* Caller must ensure the balloon_pages_lock is held. -*/ -static inline void balloon_page_insert (struct balloon_dev_info *balloon, -struct page *page) -{ -__SetPageOffline (page); -if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) { -SetPageMovableOps (page); -set_page_private (page, (unsigned long )balloon); - } -list_add (&page->lru, &balloon->pages); -} - -/* -* balloon_page_finalize - prepare a balloon page that was removed from the -* balloon list for release to the page allocator -* @page: page to be released to the page allocator -* -* Caller must ensure the balloon_pages_lock is held. -*/ -static inline void balloon_page_finalize (struct page *page) -{ -if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) -set_page_private (page,0 ); -/* PageOffline is sticky until the page is freed to the buddy. */ -}#endif /* _LINUX_BALLOON_COMPACTION_H */ mm/balloon_compaction.c +31
-0@@ -17,6 +17,44 @@*/ static DEFINE_SPINLOCK (balloon_pages_lock); +static struct balloon_dev_info *balloon_page_device (struct page *page) +{ +return (struct balloon_dev_info *)page_private (page); +} + +/* +* balloon_page_insert - insert a page into the balloon's page list and make +* the page->private assignment accordingly. +* @balloon : pointer to balloon device +* @page : page to be assigned as a 'balloon page' +* +* Caller must ensure the balloon_pages_lock is held. +*/ +static void balloon_page_insert (struct balloon_dev_info *balloon, +struct page *page) +{ +__SetPageOffline (page); +if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) { +SetPageMovableOps (page); +set_page_private (page, (unsigned long )balloon); + } +list_add (&page->lru, &balloon->pages); +} + +/* +* balloon_page_finalize - prepare a balloon page that was removed from the +* balloon list for release to the page allocator +* @page: page to be released to the page allocator +* +* Caller must ensure the balloon_pages_lock is held. +*/
mm/balloon_compaction.c line 60
Quoted 10 lines
mm/balloon_compaction.c +7
-0+static void balloon_page_finalize (struct page *page) +{ +if (IS_ENABLED (CONFIG_BALLOON_COMPACTION )) +set_page_private (page,0 ); +/* PageOffline is sticky until the page is freed to the buddy. */ +} +static void balloon_page_enqueue_one (struct balloon_dev_info *b_dev_info,struct page *page) {
The following on top to silence [1]. Results in two contextual conflicts in later patches which is easy to resolve. As an alternative, we could throw in a __maybe_unused but this is cleaner.
[1] https://lkml.kernel.org/r/[email protected]
From a02a0de0670a6be9a34d06eea3ba41297ce547dc Mon Sep 17 00:00:00 2001
From: "David Hildenbrand (Red Hat)" <[email protected]>
Date: Tue, 20 Jan 2026 07:37:39 +0100
Subject: [PATCH] fixup: mm/balloon_compaction: move internal helpers to
balloon_compaction.c
Move balloon_page_device() under CONFIG_BALLOON_COMPACTION.
--- mm/balloon_compaction.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-)
mm/balloon_compaction.c +4 -5
@@ -17,11 +17,6 @@*/ static DEFINE_SPINLOCK (balloon_pages_lock); -static struct balloon_dev_info *balloon_page_device (struct page *page) -{ -return (struct balloon_dev_info *)page_private (page); -} -/* * balloon_page_insert - insert a page into the balloon's page list and make * the page->private assignment accordingly.
@@ -232,6 +227,10 @@ struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)EXPORT_SYMBOL_GPL (balloon_page_dequeue);#ifdef CONFIG_BALLOON_COMPACTION +static struct balloon_dev_info *balloon_page_device (struct page *page) +{ +return (struct balloon_dev_info *)page_private (page); +}static bool balloon_page_isolate (struct page *page,isolate_mode_t mode)
Signature · 7 lines
2.52.0 -- Cheers David