Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8DF5E20CCDC for ; Sat, 31 Jan 2026 22:24:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769898274; cv=none; b=Olgn4f2tdLgWk2FnW+6r/T028HqOnwfHAoKXXqIOC0UyF+puPO+uOxd3AEd2CZTGwjr7181XZrslPWt3XVtcPaABfc4uiYLgMOHurSWFxoYGH7iGd3AL1iumKsYa5mSuIAU+TNVEDVhVuzixhk2+NCQhNzJKm35DlAieDTtlEJ0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769898274; c=relaxed/simple; bh=4ikCsQesbkphlo7iMsDlX3TpZ3rhCqdR67Fnaypi+/0=; h=Date:To:From:Subject:Message-Id; b=bGGZ1pI0+RqSqndYg6LPBl6i7DRrY/WG+VbiEOV91wQLLzZowsIpJpALdORDGpH2tiuTEFcQiBRjOJE1P4L2N+4+/Ao0w+VvYuir5DqL7C0wJYP3hp05u7wV+1wn8CUp7PeMUVsnVRZgcPbAVyKmgmJ17eC5ZJWNQH2sCNC9SLc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=X7WVuW/y; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="X7WVuW/y" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4AD05C4CEF1; Sat, 31 Jan 2026 22:24:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1769898274; bh=4ikCsQesbkphlo7iMsDlX3TpZ3rhCqdR67Fnaypi+/0=; h=Date:To:From:Subject:From; b=X7WVuW/ysD16p5m3gZIgtcHc4V60+5EjZEFBCMpShoo66CQR3Ez6jW72AhDr4nPZV CQ/3/4CKVPAeQzyzKRjZmNi0rpw5uoFcz1nv1oEe6F61p9Y1KH4CZ38KPY9WdQjUKP C/tCEcE7I+gzIo51jZpujSs0ku21roRlNTgWQajg= Date: Sat, 31 Jan 2026 14:24:33 -0800 To: mm-commits@vger.kernel.org,ziy@nvidia.com,xuanzhuo@linux.alibaba.com,vbabka@suse.cz,surenb@google.com,sj@kernel.org,rppt@kernel.org,osalvador@suse.de,npiggin@gmail.com,mst@redhat.com,mpe@ellerman.id.au,mhocko@suse.com,maddy@linux.ibm.com,lorenzo.stoakes@oracle.com,liam.howlett@oracle.com,jerrin.shaji-george@broadcom.com,jasowang@redhat.com,gregkh@linuxfoundation.org,eperezma@redhat.com,corbet@lwn.net,christophe.leroy@csgroup.eu,arnd@arndb.de,david@kernel.org,akpm@linux-foundation.org From: Andrew Morton Subject: [merged mm-stable] mm-rename-balloon_compactionch-to-balloonch.patch removed from -mm tree Message-Id: <20260131222434.4AD05C4CEF1@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The quilt patch titled Subject: mm: rename balloon_compaction.(c|h) to balloon.(c|h) has been removed from the -mm tree. Its filename was mm-rename-balloon_compactionch-to-balloonch.patch This patch was dropped because it was merged into the mm-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm ------------------------------------------------------ From: "David Hildenbrand (Red Hat)" Subject: mm: rename balloon_compaction.(c|h) to balloon.(c|h) Date: Tue, 20 Jan 2026 00:01:28 +0100 Even without CONFIG_BALLOON_COMPACTION this infrastructure implements basic list and page management for a memory balloon. Link: https://lkml.kernel.org/r/20260119230133.3551867-21-david@kernel.org Signed-off-by: David Hildenbrand (Red Hat) Reviewed-by: Lorenzo Stoakes Acked-by: Michael S. Tsirkin Cc: Arnd Bergmann Cc: Christophe Leroy Cc: Eugenio Pérez Cc: Greg Kroah-Hartman Cc: Jason Wang Cc: Jerrin Shaji George Cc: Jonathan Corbet Cc: Liam Howlett Cc: Madhavan Srinivasan Cc: Michael Ellerman Cc: Michal Hocko Cc: Mike Rapoport Cc: Nicholas Piggin Cc: Oscar Salvador Cc: SeongJae Park Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Xuan Zhuo Cc: Zi Yan Signed-off-by: Andrew Morton --- Documentation/core-api/mm-api.rst | 2 MAINTAINERS | 4 arch/powerpc/platforms/pseries/cmm.c | 2 drivers/misc/vmw_balloon.c | 2 drivers/virtio/virtio_balloon.c | 2 include/linux/balloon.h | 77 +++++ include/linux/balloon_compaction.h | 78 ----- mm/Makefile | 2 mm/balloon.c | 344 ++++++++++++++++++++++++ mm/balloon_compaction.c | 345 ------------------------- 10 files changed, 428 insertions(+), 430 deletions(-) --- a/arch/powerpc/platforms/pseries/cmm.c~mm-rename-balloon_compactionch-to-balloonch +++ a/arch/powerpc/platforms/pseries/cmm.c @@ -19,7 +19,7 @@ #include #include #include -#include +#include #include #include #include --- a/Documentation/core-api/mm-api.rst~mm-rename-balloon_compactionch-to-balloonch +++ a/Documentation/core-api/mm-api.rst @@ -130,5 +130,5 @@ More Memory Management Functions .. kernel-doc:: mm/vmscan.c .. kernel-doc:: mm/memory_hotplug.c .. kernel-doc:: mm/mmu_notifier.c -.. kernel-doc:: mm/balloon_compaction.c +.. kernel-doc:: mm/balloon.c .. kernel-doc:: mm/huge_memory.c --- a/drivers/misc/vmw_balloon.c~mm-rename-balloon_compactionch-to-balloonch +++ a/drivers/misc/vmw_balloon.c @@ -29,7 +29,7 @@ #include #include #include -#include +#include #include #include #include --- a/drivers/virtio/virtio_balloon.c~mm-rename-balloon_compactionch-to-balloonch +++ a/drivers/virtio/virtio_balloon.c @@ -13,7 +13,7 @@ #include #include #include -#include +#include #include #include #include diff --git a/include/linux/balloon_compaction.h a/include/linux/balloon_compaction.h deleted file mode 100644 --- a/include/linux/balloon_compaction.h +++ /dev/null @@ -1,78 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * include/linux/balloon_compaction.h - * - * Common interface definitions for making balloon pages movable by compaction. - * - * Balloon page migration makes use of the general "movable_ops page migration" - * feature. - * - * page->private is used to reference the responsible balloon device. - * That these pages have movable_ops, and which movable_ops apply, - * is derived from the page type (PageOffline()) combined with the - * PG_movable_ops flag (PageMovableOps()). - * - * Once the page type and the PG_movable_ops are set, migration code - * can initiate page isolation by invoking the - * movable_operations()->isolate_page() callback - * - * As long as page->private is set, the page is either on the balloon list - * or isolated for migration. If page->private is not set, the page is - * either still getting inflated, or was deflated to be freed by the balloon - * driver soon. Isolation is impossible in both cases. - * - * As the page isolation scanning step a compaction thread does is a lockless - * procedure (from a page standpoint), it might bring some racy situations while - * performing balloon page compaction. In order to sort out these racy scenarios - * and safely perform balloon's page compaction and migration we must, always, - * ensure following these simple rules: - * - * i. Inflation/deflation must set/clear page->private under the - * balloon_pages_lock - * - * ii. isolation or dequeueing procedure must remove the page from balloon - * device page list under balloon_pages_lock - * - * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini - */ -#ifndef _LINUX_BALLOON_COMPACTION_H -#define _LINUX_BALLOON_COMPACTION_H -#include -#include -#include -#include -#include -#include - -/* - * Balloon device information descriptor. - * This struct is used to allow the common balloon compaction interface - * procedures to find the proper balloon device holding memory pages they'll - * have to cope for page compaction / migration, as well as it serves the - * balloon driver as a page book-keeper for its registered balloon devices. - */ -struct balloon_dev_info { - unsigned long isolated_pages; /* # of isolated pages for migration */ - struct list_head pages; /* Pages enqueued & handled to Host */ - int (*migratepage)(struct balloon_dev_info *, struct page *newpage, - struct page *page, enum migrate_mode mode); - bool adjust_managed_page_count; -}; - -struct page *balloon_page_alloc(void); -void balloon_page_enqueue(struct balloon_dev_info *b_dev_info, - struct page *page); -struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info); -size_t balloon_page_list_enqueue(struct balloon_dev_info *b_dev_info, - struct list_head *pages); -size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info, - struct list_head *pages, size_t n_req_pages); - -static inline void balloon_devinfo_init(struct balloon_dev_info *balloon) -{ - balloon->isolated_pages = 0; - INIT_LIST_HEAD(&balloon->pages); - balloon->migratepage = NULL; - balloon->adjust_managed_page_count = false; -} -#endif /* _LINUX_BALLOON_COMPACTION_H */ diff --git a/include/linux/balloon.h a/include/linux/balloon.h new file mode 100644 --- /dev/null +++ a/include/linux/balloon.h @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Common interface for implementing a memory balloon, including support + * for migration of pages inflated in a memory balloon. + * + * Balloon page migration makes use of the general "movable_ops page migration" + * feature. + * + * page->private is used to reference the responsible balloon device. + * That these pages have movable_ops, and which movable_ops apply, + * is derived from the page type (PageOffline()) combined with the + * PG_movable_ops flag (PageMovableOps()). + * + * Once the page type and the PG_movable_ops are set, migration code + * can initiate page isolation by invoking the + * movable_operations()->isolate_page() callback + * + * As long as page->private is set, the page is either on the balloon list + * or isolated for migration. If page->private is not set, the page is + * either still getting inflated, or was deflated to be freed by the balloon + * driver soon. Isolation is impossible in both cases. + * + * As the page isolation scanning step a compaction thread does is a lockless + * procedure (from a page standpoint), it might bring some racy situations while + * performing balloon page compaction. In order to sort out these racy scenarios + * and safely perform balloon's page compaction and migration we must, always, + * ensure following these simple rules: + * + * i. Inflation/deflation must set/clear page->private under the + * balloon_pages_lock + * + * ii. isolation or dequeueing procedure must remove the page from balloon + * device page list under balloon_pages_lock + * + * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini + */ +#ifndef _LINUX_BALLOON_H +#define _LINUX_BALLOON_H +#include +#include +#include +#include +#include +#include + +/* + * Balloon device information descriptor. + * This struct is used to allow the common balloon compaction interface + * procedures to find the proper balloon device holding memory pages they'll + * have to cope for page compaction / migration, as well as it serves the + * balloon driver as a page book-keeper for its registered balloon devices. + */ +struct balloon_dev_info { + unsigned long isolated_pages; /* # of isolated pages for migration */ + struct list_head pages; /* Pages enqueued & handled to Host */ + int (*migratepage)(struct balloon_dev_info *, struct page *newpage, + struct page *page, enum migrate_mode mode); + bool adjust_managed_page_count; +}; + +struct page *balloon_page_alloc(void); +void balloon_page_enqueue(struct balloon_dev_info *b_dev_info, + struct page *page); +struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info); +size_t balloon_page_list_enqueue(struct balloon_dev_info *b_dev_info, + struct list_head *pages); +size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info, + struct list_head *pages, size_t n_req_pages); + +static inline void balloon_devinfo_init(struct balloon_dev_info *balloon) +{ + balloon->isolated_pages = 0; + INIT_LIST_HEAD(&balloon->pages); + balloon->migratepage = NULL; + balloon->adjust_managed_page_count = false; +} +#endif /* _LINUX_BALLOON_H */ --- a/MAINTAINERS~mm-rename-balloon_compactionch-to-balloonch +++ a/MAINTAINERS @@ -27546,9 +27546,9 @@ M: David Hildenbrand L: virtualization@lists.linux.dev S: Maintained F: drivers/virtio/virtio_balloon.c -F: include/linux/balloon_compaction.h +F: include/linux/balloon.h F: include/uapi/linux/virtio_balloon.h -F: mm/balloon_compaction.c +F: mm/balloon.c VIRTIO BLOCK AND SCSI DRIVERS M: "Michael S. Tsirkin" diff --git a/mm/balloon.c a/mm/balloon.c new file mode 100644 --- /dev/null +++ a/mm/balloon.c @@ -0,0 +1,344 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Common interface for implementing a memory balloon, including support + * for migration of pages inflated in a memory balloon. + * + * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini + */ +#include +#include +#include +#include + +/* + * Lock protecting the balloon_dev_info of all devices. We don't really + * expect more than one device. + */ +static DEFINE_SPINLOCK(balloon_pages_lock); + +/** + * balloon_page_insert - insert a page into the balloon's page list and make + * the page->private assignment accordingly. + * @balloon : pointer to balloon device + * @page : page to be assigned as a 'balloon page' + * + * Caller must ensure the balloon_pages_lock is held. + */ +static void balloon_page_insert(struct balloon_dev_info *balloon, + struct page *page) +{ + lockdep_assert_held(&balloon_pages_lock); + __SetPageOffline(page); + if (IS_ENABLED(CONFIG_BALLOON_COMPACTION)) { + SetPageMovableOps(page); + set_page_private(page, (unsigned long)balloon); + } + list_add(&page->lru, &balloon->pages); +} + +/** + * balloon_page_finalize - prepare a balloon page that was removed from the + * balloon list for release to the page allocator + * @page: page to be released to the page allocator + * + * Caller must ensure the balloon_pages_lock is held. + */ +static void balloon_page_finalize(struct page *page) +{ + lockdep_assert_held(&balloon_pages_lock); + if (IS_ENABLED(CONFIG_BALLOON_COMPACTION)) + set_page_private(page, 0); + /* PageOffline is sticky until the page is freed to the buddy. */ +} + +static void balloon_page_enqueue_one(struct balloon_dev_info *b_dev_info, + struct page *page) +{ + balloon_page_insert(b_dev_info, page); + if (b_dev_info->adjust_managed_page_count) + adjust_managed_page_count(page, -1); + __count_vm_event(BALLOON_INFLATE); + inc_node_page_state(page, NR_BALLOON_PAGES); +} + +/** + * balloon_page_list_enqueue() - inserts a list of pages into the balloon page + * list. + * @b_dev_info: balloon device descriptor where we will insert a new page to + * @pages: pages to enqueue - allocated using balloon_page_alloc. + * + * Driver must call this function to properly enqueue balloon pages before + * definitively removing them from the guest system. + * + * Return: number of pages that were enqueued. + */ +size_t balloon_page_list_enqueue(struct balloon_dev_info *b_dev_info, + struct list_head *pages) +{ + struct page *page, *tmp; + unsigned long flags; + size_t n_pages = 0; + + spin_lock_irqsave(&balloon_pages_lock, flags); + list_for_each_entry_safe(page, tmp, pages, lru) { + list_del(&page->lru); + balloon_page_enqueue_one(b_dev_info, page); + n_pages++; + } + spin_unlock_irqrestore(&balloon_pages_lock, flags); + return n_pages; +} +EXPORT_SYMBOL_GPL(balloon_page_list_enqueue); + +/** + * balloon_page_list_dequeue() - removes pages from balloon's page list and + * returns a list of the pages. + * @b_dev_info: balloon device descriptor where we will grab a page from. + * @pages: pointer to the list of pages that would be returned to the caller. + * @n_req_pages: number of requested pages. + * + * Driver must call this function to properly de-allocate a previous enlisted + * balloon pages before definitively releasing it back to the guest system. + * This function tries to remove @n_req_pages from the ballooned pages and + * return them to the caller in the @pages list. + * + * Note that this function may fail to dequeue some pages even if the balloon + * isn't empty - since the page list can be temporarily empty due to compaction + * of isolated pages. + * + * Return: number of pages that were added to the @pages list. + */ +size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info, + struct list_head *pages, size_t n_req_pages) +{ + struct page *page, *tmp; + unsigned long flags; + size_t n_pages = 0; + + spin_lock_irqsave(&balloon_pages_lock, flags); + list_for_each_entry_safe(page, tmp, &b_dev_info->pages, lru) { + if (n_pages == n_req_pages) + break; + list_del(&page->lru); + if (b_dev_info->adjust_managed_page_count) + adjust_managed_page_count(page, 1); + balloon_page_finalize(page); + __count_vm_event(BALLOON_DEFLATE); + list_add(&page->lru, pages); + dec_node_page_state(page, NR_BALLOON_PAGES); + n_pages++; + } + spin_unlock_irqrestore(&balloon_pages_lock, flags); + + return n_pages; +} +EXPORT_SYMBOL_GPL(balloon_page_list_dequeue); + +/** + * balloon_page_alloc - allocates a new page for insertion into the balloon + * page list. + * + * Driver must call this function to properly allocate a new balloon page. + * Driver must call balloon_page_enqueue before definitively removing the page + * from the guest system. + * + * Return: struct page for the allocated page or NULL on allocation failure. + */ +struct page *balloon_page_alloc(void) +{ + gfp_t gfp_flags = __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN; + + if (IS_ENABLED(CONFIG_BALLOON_COMPACTION)) + gfp_flags |= GFP_HIGHUSER_MOVABLE; + else + gfp_flags |= GFP_HIGHUSER; + + return alloc_page(gfp_flags); +} +EXPORT_SYMBOL_GPL(balloon_page_alloc); + +/** + * balloon_page_enqueue - inserts a new page into the balloon page list. + * + * @b_dev_info: balloon device descriptor where we will insert a new page + * @page: new page to enqueue - allocated using balloon_page_alloc. + * + * Drivers must call this function to properly enqueue a new allocated balloon + * page before definitively removing the page from the guest system. + * + * Drivers must not enqueue pages while page->lru is still in + * use, and must not use page->lru until a page was unqueued again. + */ +void balloon_page_enqueue(struct balloon_dev_info *b_dev_info, + struct page *page) +{ + unsigned long flags; + + spin_lock_irqsave(&balloon_pages_lock, flags); + balloon_page_enqueue_one(b_dev_info, page); + spin_unlock_irqrestore(&balloon_pages_lock, flags); +} +EXPORT_SYMBOL_GPL(balloon_page_enqueue); + +/** + * balloon_page_dequeue - removes a page from balloon's page list and returns + * its address to allow the driver to release the page. + * @b_dev_info: balloon device descriptor where we will grab a page from. + * + * Driver must call this function to properly dequeue a previously enqueued page + * before definitively releasing it back to the guest system. + * + * Caller must perform its own accounting to ensure that this + * function is called only if some pages are actually enqueued. + * + * Note that this function may fail to dequeue some pages even if there are + * some enqueued pages - since the page list can be temporarily empty due to + * the compaction of isolated pages. + * + * TODO: remove the caller accounting requirements, and allow caller to wait + * until all pages can be dequeued. + * + * Return: struct page for the dequeued page, or NULL if no page was dequeued. + */ +struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info) +{ + unsigned long flags; + LIST_HEAD(pages); + int n_pages; + + n_pages = balloon_page_list_dequeue(b_dev_info, &pages, 1); + + if (n_pages != 1) { + /* + * If we are unable to dequeue a balloon page because the page + * list is empty and there are no isolated pages, then something + * went out of track and some balloon pages are lost. + * BUG() here, otherwise the balloon driver may get stuck in + * an infinite loop while attempting to release all its pages. + */ + spin_lock_irqsave(&balloon_pages_lock, flags); + if (unlikely(list_empty(&b_dev_info->pages) && + !b_dev_info->isolated_pages)) + BUG(); + spin_unlock_irqrestore(&balloon_pages_lock, flags); + return NULL; + } + return list_first_entry(&pages, struct page, lru); +} +EXPORT_SYMBOL_GPL(balloon_page_dequeue); + +#ifdef CONFIG_BALLOON_COMPACTION +static struct balloon_dev_info *balloon_page_device(struct page *page) +{ + return (struct balloon_dev_info *)page_private(page); +} + +static bool balloon_page_isolate(struct page *page, isolate_mode_t mode) + +{ + struct balloon_dev_info *b_dev_info; + unsigned long flags; + + spin_lock_irqsave(&balloon_pages_lock, flags); + b_dev_info = balloon_page_device(page); + if (!b_dev_info) { + /* + * The page already got deflated and removed from the + * balloon list. + */ + spin_unlock_irqrestore(&balloon_pages_lock, flags); + return false; + } + list_del(&page->lru); + b_dev_info->isolated_pages++; + spin_unlock_irqrestore(&balloon_pages_lock, flags); + + return true; +} + +static void balloon_page_putback(struct page *page) +{ + struct balloon_dev_info *b_dev_info = balloon_page_device(page); + unsigned long flags; + + /* + * When we isolated the page, the page was still inflated in a balloon + * device. As isolated balloon pages cannot get deflated, we still have + * a balloon device here. + */ + if (WARN_ON_ONCE(!b_dev_info)) + return; + + spin_lock_irqsave(&balloon_pages_lock, flags); + list_add(&page->lru, &b_dev_info->pages); + b_dev_info->isolated_pages--; + spin_unlock_irqrestore(&balloon_pages_lock, flags); +} + +static int balloon_page_migrate(struct page *newpage, struct page *page, + enum migrate_mode mode) +{ + struct balloon_dev_info *b_dev_info = balloon_page_device(page); + unsigned long flags; + int rc; + + /* + * When we isolated the page, the page was still inflated in a balloon + * device. As isolated balloon pages cannot get deflated, we still have + * a balloon device here. + */ + if (WARN_ON_ONCE(!b_dev_info)) + return -EAGAIN; + + rc = b_dev_info->migratepage(b_dev_info, newpage, page, mode); + if (rc < 0 && rc != -ENOENT) + return rc; + + spin_lock_irqsave(&balloon_pages_lock, flags); + if (!rc) { + /* Insert the new page into the balloon list. */ + get_page(newpage); + balloon_page_insert(b_dev_info, newpage); + __count_vm_event(BALLOON_MIGRATE); + + if (b_dev_info->adjust_managed_page_count && + page_zone(page) != page_zone(newpage)) { + /* + * When we migrate a page to a different zone we + * have to fixup the count of both involved zones. + */ + adjust_managed_page_count(page, 1); + adjust_managed_page_count(newpage, -1); + } + } else { + /* Old page was deflated but new page not inflated. */ + __count_vm_event(BALLOON_DEFLATE); + + if (b_dev_info->adjust_managed_page_count) + adjust_managed_page_count(page, 1); + } + + b_dev_info->isolated_pages--; + + /* Free the now-deflated page we isolated in balloon_page_isolate(). */ + balloon_page_finalize(page); + spin_unlock_irqrestore(&balloon_pages_lock, flags); + + put_page(page); + + return 0; +} + +static const struct movable_operations balloon_mops = { + .migrate_page = balloon_page_migrate, + .isolate_page = balloon_page_isolate, + .putback_page = balloon_page_putback, +}; + +static int __init balloon_init(void) +{ + return set_movable_ops(&balloon_mops, PGTY_offline); +} +core_initcall(balloon_init); + +#endif /* CONFIG_BALLOON_COMPACTION */ diff --git a/mm/balloon_compaction.c a/mm/balloon_compaction.c deleted file mode 100644 --- a/mm/balloon_compaction.c +++ /dev/null @@ -1,345 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * mm/balloon_compaction.c - * - * Common interface for making balloon pages movable by compaction. - * - * Copyright (C) 2012, Red Hat, Inc. Rafael Aquini - */ -#include -#include -#include -#include - -/* - * Lock protecting the balloon_dev_info of all devices. We don't really - * expect more than one device. - */ -static DEFINE_SPINLOCK(balloon_pages_lock); - -/** - * balloon_page_insert - insert a page into the balloon's page list and make - * the page->private assignment accordingly. - * @balloon : pointer to balloon device - * @page : page to be assigned as a 'balloon page' - * - * Caller must ensure the balloon_pages_lock is held. - */ -static void balloon_page_insert(struct balloon_dev_info *balloon, - struct page *page) -{ - lockdep_assert_held(&balloon_pages_lock); - __SetPageOffline(page); - if (IS_ENABLED(CONFIG_BALLOON_COMPACTION)) { - SetPageMovableOps(page); - set_page_private(page, (unsigned long)balloon); - } - list_add(&page->lru, &balloon->pages); -} - -/** - * balloon_page_finalize - prepare a balloon page that was removed from the - * balloon list for release to the page allocator - * @page: page to be released to the page allocator - * - * Caller must ensure the balloon_pages_lock is held. - */ -static void balloon_page_finalize(struct page *page) -{ - lockdep_assert_held(&balloon_pages_lock); - if (IS_ENABLED(CONFIG_BALLOON_COMPACTION)) - set_page_private(page, 0); - /* PageOffline is sticky until the page is freed to the buddy. */ -} - -static void balloon_page_enqueue_one(struct balloon_dev_info *b_dev_info, - struct page *page) -{ - balloon_page_insert(b_dev_info, page); - if (b_dev_info->adjust_managed_page_count) - adjust_managed_page_count(page, -1); - __count_vm_event(BALLOON_INFLATE); - inc_node_page_state(page, NR_BALLOON_PAGES); -} - -/** - * balloon_page_list_enqueue() - inserts a list of pages into the balloon page - * list. - * @b_dev_info: balloon device descriptor where we will insert a new page to - * @pages: pages to enqueue - allocated using balloon_page_alloc. - * - * Driver must call this function to properly enqueue balloon pages before - * definitively removing them from the guest system. - * - * Return: number of pages that were enqueued. - */ -size_t balloon_page_list_enqueue(struct balloon_dev_info *b_dev_info, - struct list_head *pages) -{ - struct page *page, *tmp; - unsigned long flags; - size_t n_pages = 0; - - spin_lock_irqsave(&balloon_pages_lock, flags); - list_for_each_entry_safe(page, tmp, pages, lru) { - list_del(&page->lru); - balloon_page_enqueue_one(b_dev_info, page); - n_pages++; - } - spin_unlock_irqrestore(&balloon_pages_lock, flags); - return n_pages; -} -EXPORT_SYMBOL_GPL(balloon_page_list_enqueue); - -/** - * balloon_page_list_dequeue() - removes pages from balloon's page list and - * returns a list of the pages. - * @b_dev_info: balloon device descriptor where we will grab a page from. - * @pages: pointer to the list of pages that would be returned to the caller. - * @n_req_pages: number of requested pages. - * - * Driver must call this function to properly de-allocate a previous enlisted - * balloon pages before definitively releasing it back to the guest system. - * This function tries to remove @n_req_pages from the ballooned pages and - * return them to the caller in the @pages list. - * - * Note that this function may fail to dequeue some pages even if the balloon - * isn't empty - since the page list can be temporarily empty due to compaction - * of isolated pages. - * - * Return: number of pages that were added to the @pages list. - */ -size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info, - struct list_head *pages, size_t n_req_pages) -{ - struct page *page, *tmp; - unsigned long flags; - size_t n_pages = 0; - - spin_lock_irqsave(&balloon_pages_lock, flags); - list_for_each_entry_safe(page, tmp, &b_dev_info->pages, lru) { - if (n_pages == n_req_pages) - break; - list_del(&page->lru); - if (b_dev_info->adjust_managed_page_count) - adjust_managed_page_count(page, 1); - balloon_page_finalize(page); - __count_vm_event(BALLOON_DEFLATE); - list_add(&page->lru, pages); - dec_node_page_state(page, NR_BALLOON_PAGES); - n_pages++; - } - spin_unlock_irqrestore(&balloon_pages_lock, flags); - - return n_pages; -} -EXPORT_SYMBOL_GPL(balloon_page_list_dequeue); - -/** - * balloon_page_alloc - allocates a new page for insertion into the balloon - * page list. - * - * Driver must call this function to properly allocate a new balloon page. - * Driver must call balloon_page_enqueue before definitively removing the page - * from the guest system. - * - * Return: struct page for the allocated page or NULL on allocation failure. - */ -struct page *balloon_page_alloc(void) -{ - gfp_t gfp_flags = __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN; - - if (IS_ENABLED(CONFIG_BALLOON_COMPACTION)) - gfp_flags |= GFP_HIGHUSER_MOVABLE; - else - gfp_flags |= GFP_HIGHUSER; - - return alloc_page(gfp_flags); -} -EXPORT_SYMBOL_GPL(balloon_page_alloc); - -/** - * balloon_page_enqueue - inserts a new page into the balloon page list. - * - * @b_dev_info: balloon device descriptor where we will insert a new page - * @page: new page to enqueue - allocated using balloon_page_alloc. - * - * Drivers must call this function to properly enqueue a new allocated balloon - * page before definitively removing the page from the guest system. - * - * Drivers must not enqueue pages while page->lru is still in - * use, and must not use page->lru until a page was unqueued again. - */ -void balloon_page_enqueue(struct balloon_dev_info *b_dev_info, - struct page *page) -{ - unsigned long flags; - - spin_lock_irqsave(&balloon_pages_lock, flags); - balloon_page_enqueue_one(b_dev_info, page); - spin_unlock_irqrestore(&balloon_pages_lock, flags); -} -EXPORT_SYMBOL_GPL(balloon_page_enqueue); - -/** - * balloon_page_dequeue - removes a page from balloon's page list and returns - * its address to allow the driver to release the page. - * @b_dev_info: balloon device descriptor where we will grab a page from. - * - * Driver must call this function to properly dequeue a previously enqueued page - * before definitively releasing it back to the guest system. - * - * Caller must perform its own accounting to ensure that this - * function is called only if some pages are actually enqueued. - * - * Note that this function may fail to dequeue some pages even if there are - * some enqueued pages - since the page list can be temporarily empty due to - * the compaction of isolated pages. - * - * TODO: remove the caller accounting requirements, and allow caller to wait - * until all pages can be dequeued. - * - * Return: struct page for the dequeued page, or NULL if no page was dequeued. - */ -struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info) -{ - unsigned long flags; - LIST_HEAD(pages); - int n_pages; - - n_pages = balloon_page_list_dequeue(b_dev_info, &pages, 1); - - if (n_pages != 1) { - /* - * If we are unable to dequeue a balloon page because the page - * list is empty and there are no isolated pages, then something - * went out of track and some balloon pages are lost. - * BUG() here, otherwise the balloon driver may get stuck in - * an infinite loop while attempting to release all its pages. - */ - spin_lock_irqsave(&balloon_pages_lock, flags); - if (unlikely(list_empty(&b_dev_info->pages) && - !b_dev_info->isolated_pages)) - BUG(); - spin_unlock_irqrestore(&balloon_pages_lock, flags); - return NULL; - } - return list_first_entry(&pages, struct page, lru); -} -EXPORT_SYMBOL_GPL(balloon_page_dequeue); - -#ifdef CONFIG_BALLOON_COMPACTION -static struct balloon_dev_info *balloon_page_device(struct page *page) -{ - return (struct balloon_dev_info *)page_private(page); -} - -static bool balloon_page_isolate(struct page *page, isolate_mode_t mode) - -{ - struct balloon_dev_info *b_dev_info; - unsigned long flags; - - spin_lock_irqsave(&balloon_pages_lock, flags); - b_dev_info = balloon_page_device(page); - if (!b_dev_info) { - /* - * The page already got deflated and removed from the - * balloon list. - */ - spin_unlock_irqrestore(&balloon_pages_lock, flags); - return false; - } - list_del(&page->lru); - b_dev_info->isolated_pages++; - spin_unlock_irqrestore(&balloon_pages_lock, flags); - - return true; -} - -static void balloon_page_putback(struct page *page) -{ - struct balloon_dev_info *b_dev_info = balloon_page_device(page); - unsigned long flags; - - /* - * When we isolated the page, the page was still inflated in a balloon - * device. As isolated balloon pages cannot get deflated, we still have - * a balloon device here. - */ - if (WARN_ON_ONCE(!b_dev_info)) - return; - - spin_lock_irqsave(&balloon_pages_lock, flags); - list_add(&page->lru, &b_dev_info->pages); - b_dev_info->isolated_pages--; - spin_unlock_irqrestore(&balloon_pages_lock, flags); -} - -static int balloon_page_migrate(struct page *newpage, struct page *page, - enum migrate_mode mode) -{ - struct balloon_dev_info *b_dev_info = balloon_page_device(page); - unsigned long flags; - int rc; - - /* - * When we isolated the page, the page was still inflated in a balloon - * device. As isolated balloon pages cannot get deflated, we still have - * a balloon device here. - */ - if (WARN_ON_ONCE(!b_dev_info)) - return -EAGAIN; - - rc = b_dev_info->migratepage(b_dev_info, newpage, page, mode); - if (rc < 0 && rc != -ENOENT) - return rc; - - spin_lock_irqsave(&balloon_pages_lock, flags); - if (!rc) { - /* Insert the new page into the balloon list. */ - get_page(newpage); - balloon_page_insert(b_dev_info, newpage); - __count_vm_event(BALLOON_MIGRATE); - - if (b_dev_info->adjust_managed_page_count && - page_zone(page) != page_zone(newpage)) { - /* - * When we migrate a page to a different zone we - * have to fixup the count of both involved zones. - */ - adjust_managed_page_count(page, 1); - adjust_managed_page_count(newpage, -1); - } - } else { - /* Old page was deflated but new page not inflated. */ - __count_vm_event(BALLOON_DEFLATE); - - if (b_dev_info->adjust_managed_page_count) - adjust_managed_page_count(page, 1); - } - - b_dev_info->isolated_pages--; - - /* Free the now-deflated page we isolated in balloon_page_isolate(). */ - balloon_page_finalize(page); - spin_unlock_irqrestore(&balloon_pages_lock, flags); - - put_page(page); - - return 0; -} - -static const struct movable_operations balloon_mops = { - .migrate_page = balloon_page_migrate, - .isolate_page = balloon_page_isolate, - .putback_page = balloon_page_putback, -}; - -static int __init balloon_init(void) -{ - return set_movable_ops(&balloon_mops, PGTY_offline); -} -core_initcall(balloon_init); - -#endif /* CONFIG_BALLOON_COMPACTION */ --- a/mm/Makefile~mm-rename-balloon_compactionch-to-balloonch +++ a/mm/Makefile @@ -122,7 +122,7 @@ obj-$(CONFIG_CMA) += cma.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_NUMA_MEMBLKS) += numa_memblks.o obj-$(CONFIG_NUMA_EMU) += numa_emulation.o -obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o +obj-$(CONFIG_MEMORY_BALLOON) += balloon.o obj-$(CONFIG_PAGE_EXTENSION) += page_ext.o obj-$(CONFIG_PAGE_TABLE_CHECK) += page_table_check.o obj-$(CONFIG_CMA_DEBUGFS) += cma_debug.o _ Patches currently in -mm which might be from david@kernel.org are mm-move-pte-table-reclaim-code-to-memoryc.patch mm-memory-handle-non-split-locks-correctly-in-zap_empty_pte_table.patch