mm/balloon_compaction: fixes and cleanups
Subsystems (12)
v2 cover mm/balloon_compaction: fixes and cleanups
I've checked compilation of linux-next/x86 for allnoconfig, defconfig and defconfig + kvmconfig + virtio-balloon with and without balloon-compaction. For stable kernels first three patches should be enough.
changes since v1:
mm/balloon_compaction: ignore anonymous pages
* no changes
mm/balloon_compaction: keep ballooned pages away from normal migration path
* fix compilation without CONFIG_BALLOON_COMPACTION
mm/balloon_compaction: isolate balloon pages without lru_lock
* no changes
mm: introduce common page state for ballooned memory
* move __Set/ClearPageBalloon into linux/mm.h
* remove inc/dec_zone_page_state from __Set/ClearPageBalloon
mm/balloon_compaction: use common page ballooning
* call inc/dec_zone_page_state from balloon_page_insert/delete
mm/balloon_compaction: general cleanup
* fix compilation without CONFIG_MIGRATION
* fix compilation without CONFIG_BALLOON_COMPACTION
---
Konstantin Khlebnikov (6):
mm/balloon_compaction: ignore anonymous pages mm/balloon_compaction: keep ballooned pages away from normal migration path mm/balloon_compaction: isolate balloon pages without lru_lock
mm: introduce common page state for ballooned memory mm/balloon_compaction: use common page ballooning mm/balloon_compaction: general cleanup
Documentation/filesystems/proc.txt | 2 drivers/base/node.c | 16 +- drivers/virtio/Kconfig | 1 drivers/virtio/virtio_balloon.c | 77 +++-------- fs/proc/meminfo.c | 6 + fs/proc/page.c | 3 include/linux/balloon_compaction.h | 223 ++++++-------------------------- include/linux/migrate.h | 11 -- include/linux/mm.h | 20 +++ include/linux/mmzone.h | 3 include/linux/pagemap.h | 18 --- include/uapi/linux/kernel-page-flags.h | 1 mm/Kconfig | 7 + mm/Makefile | 3 mm/balloon_compaction.c | 219 ++++++++++--------------------- mm/compaction.c | 9 + mm/migrate.c | 29 +--- mm/vmscan.c | 2 mm/vmstat.c | 8 + tools/vm/page-types.c | 1 20 files changed, 210 insertions(+), 449 deletions(-)
--
Signature
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v2 1/6 mm/balloon_compaction: ignore anonymous pages
From: Konstantin Khlebnikov <[email protected]>
Sasha Levin reported KASAN splash inside isolate_migratepages_range(). Problem is in function __is_movable_balloon_page() which tests AS_BALLOON_MAP in page->mapping->flags. This function has no protection against anonymous pages. As result it tried to check address space flags in inside anon-vma.
--- include/linux/balloon_compaction.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/balloon_compaction.h +1 -1
@@ -128,7 +128,7 @@ static inline bool page_flags_cleared(struct page *page)static inline bool __is_movable_balloon_page (struct page *page) {struct address_space *mapping = page->mapping; -return mapping_balloon (mapping); +return !PageAnon (page) &&mapping_balloon (mapping); }/*
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v2 2/6 mm/balloon_compaction: keep ballooned pages away from normal migration path
In reply to Konstantin KhlebnikovFrom: Konstantin Khlebnikov <[email protected]>
Proper testing shows yet another problem in balloon migration: it works only once for each page. balloon_page_movable() check page flags and page_count. In __unmap_and_move page is locked, reference counter is elevated, so balloon_page_movable() _always_ fails here. As result in __unmap_and_move() migration goes to the normal migration path.
Balloon ->migratepage() is so special, it returns MIGRATEPAGE_BALLOON_SUCCESS instead of MIGRATEPAGE_SUCCESS. After that in move_to_new_page() successfully migrated page got NULL into its mapping pointer and loses connectivity with balloon and ability for further migration.
It's safe to use __is_movable_balloon_page here: page is isolated and pinned.
--- include/linux/balloon_compaction.h | 5 +++++ mm/migrate.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-)
include/linux/balloon_compaction.h +5 -0
@@ -258,6 +258,11 @@ static inline void balloon_page_delete(struct page *page)list_del (&page->lru); } +static inline bool __is_movable_balloon_page (struct page *page) +{ +return false; +} +static inline bool balloon_page_movable (struct page *page) {return false;
mm/migrate.c +1 -1
@@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage, } } -if (unlikely (balloon_page_movable (page))) { +if (unlikely (__is_movable_balloon_page (page))) {/* * A ballooned page does not need any special attention from * physical to virtual reverse mapping procedures.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v2 3/6 mm/balloon_compaction: isolate balloon pages without lru_lock
In reply to Konstantin KhlebnikovFrom: Konstantin Khlebnikov <[email protected]>
LRU-lock isn't required for balloon page isolation. This check makes migration of some ballooned pages mostly impossible because isolate_migratepages_range() drops LRU lock periodically.
--- mm/compaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/compaction.c +1 -1
@@ -643,7 +643,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,*/ if (!PageLRU (page)) {if (unlikely (balloon_page_movable (page))) { -if (locked &&balloon_page_isolate (page)) { +if (balloon_page_isolate (page)) {/* Successfully isolated */ goto isolate_success; }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v2 4/6 mm: introduce common page state for ballooned memory
In reply to Konstantin KhlebnikovFrom: Konstantin Khlebnikov <[email protected]>
This patch adds page state PageBallon() and functions __Set/ClearPageBalloon. Like PageBuddy() PageBalloon() looks like page-flag but actually this is special state of page->_mapcount counter. There is no conflict because ballooned pages cannot be mapped and cannot be in buddy allocator.
Ballooned pages are counted in vmstat counter NR_BALLOON_PAGES, it's shown them in /proc/meminfo and /proc/meminfo. Also this patch it exports PageBallon into userspace via /proc/kpageflags as KPF_BALLOON.
All new code is under CONFIG_MEMORY_BALLOON, it should be selected by ballooning driver which wants use this feature.
--- Documentation/filesystems/proc.txt | 2 ++ drivers/base/node.c | 16 ++++++++++------ fs/proc/meminfo.c | 6 ++++++ fs/proc/page.c | 3 +++ include/linux/mm.h | 20 ++++++++++++++++++++ include/linux/mmzone.h | 3 +++ include/uapi/linux/kernel-page-flags.h | 1 + mm/Kconfig | 5 +++++ mm/vmstat.c | 8 +++++++- tools/vm/page-types.c | 1 + 10 files changed, 58 insertions(+), 7 deletions(-)
Documentation/filesystems/proc.txt +2 -0
@@ -796,6 +796,7 @@ VmallocTotal: 112216 kB VmallocUsed: 428 kB VmallocChunk: 111088 kB AnonHugePages: 49152 kB +BalloonPages: 0 kB MemTotal: Total usable ram (i.e. physical ram minus a few reserved bits and the kernel binary code)
@@ -838,6 +839,7 @@ MemAvailable: An estimate of how much memory is available for starting new Writeback: Memory which is actively being written back to the disk AnonPages: Non-file backed pages mapped into userspace page tables AnonHugePages: Non-file backed huge pages mapped into userspace page tables +BalloonPages: Memory which was ballooned, not included into MemTotal Mapped: files which have been mmaped, such as libraries Slab: in-kernel data structures cache SReclaimable: Part of Slab, that might be reclaimed, such as caches
drivers/base/node.c +10 -6
@@ -120,6 +120,9 @@ static ssize_t node_read_meminfo(struct device *dev,#ifdef CONFIG_TRANSPARENT_HUGEPAGE "Node %d AnonHugePages: %8lu kB\n"#endif +#ifdef CONFIG_MEMORY_BALLOON +"Node %d BalloonPages: %8lu kB\n" +#endif , nid,K (node_page_state (nid,NR_FILE_DIRTY )), nid, K(node_page_state(nid, NR_WRITEBACK)),
@@ -136,14 +139,15 @@ static ssize_t node_read_meminfo(struct device *dev, nid,K (node_page_state (nid,NR_SLAB_RECLAIMABLE ) +node_page_state (nid,NR_SLAB_UNRECLAIMABLE )), nid,K (node_page_state (nid,NR_SLAB_RECLAIMABLE )), -#ifdef CONFIG_TRANSPARENT_HUGEPAGE nid,K (node_page_state (nid,NR_SLAB_UNRECLAIMABLE )) - , nid, -K (node_page_state (nid,NR_ANON_TRANSPARENT_HUGEPAGES ) * - HPAGE_PMD_NR)); -#else - nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE))); +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + ,nid,K (node_page_state (nid, +NR_ANON_TRANSPARENT_HUGEPAGES ) *HPAGE_PMD_NR ) +#endif +#ifdef CONFIG_MEMORY_BALLOON + ,nid,K (node_page_state (nid,NR_BALLOON_PAGES )) #endif + ); n +=hugetlb_report_node_meminfo (nid, buf + n);return n; }
fs/proc/meminfo.c +6 -0
@@ -138,6 +138,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)#ifdef CONFIG_TRANSPARENT_HUGEPAGE "AnonHugePages: %8lu kB\n"#endif +#ifdef CONFIG_MEMORY_BALLOON +"BalloonPages: %8lu kB\n" +#endif ,K (i.totalram), K(i.freeram),
@@ -193,6 +196,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v) ,K(global_page_state(NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR)#endif +#ifdef CONFIG_MEMORY_BALLOON + ,K(global_page_state(NR_BALLOON_PAGES)) +#endif );hugetlb_report_meminfo (m);
fs/proc/page.c +3 -0
@@ -133,6 +133,9 @@ u64 stable_page_flags(struct page *page)if (PageBuddy (page)) u |=1 <<KPF_BUDDY ; +if (PageBalloon (page)) + u |=1 <<KPF_BALLOON ; + u |=kpf_copy_bit (k,KPF_LOCKED , PG_locked); u |=kpf_copy_bit (k,KPF_SLAB , PG_slab);
include/linux/mm.h +20 -0
@@ -554,6 +554,26 @@ static inline void __ClearPageBuddy(struct page *page)atomic_set (&page->_mapcount,-1 ); } +#define PAGE_BALLOON_MAPCOUNT_VALUE (-256) + +static inline int PageBalloon (struct page *page) +{ +return IS_ENABLED (CONFIG_MEMORY_BALLOON ) && +atomic_read (&page->_mapcount) ==PAGE_BALLOON_MAPCOUNT_VALUE ; +} + +static inline void __SetPageBalloon (struct page *page) +{ +VM_BUG_ON_PAGE (atomic_read (&page->_mapcount) !=-1 , page); +atomic_set (&page->_mapcount,PAGE_BALLOON_MAPCOUNT_VALUE ); +} + +static inline void __ClearPageBalloon (struct page *page) +{ +VM_BUG_ON_PAGE (!PageBalloon (page), page); +atomic_set (&page->_mapcount,-1 ); +} +void put_page (struct page *page);void put_pages_list (struct list_head *pages);
include/linux/mmzone.h +3 -0
@@ -157,6 +157,9 @@ enum zone_stat_item {WORKINGSET_NODERECLAIM ,NR_ANON_TRANSPARENT_HUGEPAGES ,NR_FREE_CMA_PAGES , +#ifdef CONFIG_MEMORY_BALLOON +NR_BALLOON_PAGES , +#endif NR_VM_ZONE_STAT_ITEMS };/*
include/uapi/linux/kernel-page-flags.h +1 -0
@@ -31,6 +31,7 @@#define KPF_KSM 21#define KPF_THP 22 +#define KPF_BALLOON 23#endif /* _UAPILINUX_KERNEL_PAGE_FLAGS_H */
mm/Kconfig +5 -0
@@ -228,6 +228,11 @@ config ARCH_ENABLE_SPLIT_PMD_PTLOCK boolean # +# support for memory ballooning +config MEMORY_BALLOON + boolean + +# # support for memory balloon compaction config BALLOON_COMPACTION bool "Allow for balloon memory compaction/migration"
mm/vmstat.c +7 -1
@@ -751,7 +751,7 @@ static void walk_zones_in_node(struct seq_file *m, pg_data_t *pgdat, TEXT_FOR_HIGHMEM(xx) xx "_movable",const char *const vmstat_text[] = { -/* Zoned VM counters */ +/* enum zone_stat_item countes */ "nr_free_pages" ,"nr_alloc_batch" ,"nr_inactive_anon" ,
@@ -794,10 +794,16 @@ const char * const vmstat_text[] = {"workingset_nodereclaim" ,"nr_anon_transparent_hugepages" ,"nr_free_cma" , +#ifdef CONFIG_MEMORY_BALLOON +"nr_balloon_pages" , +#endif + +/* enum writeback_stat_item counters */ "nr_dirty_threshold" ,"nr_dirty_background_threshold" , #ifdef CONFIG_VM_EVENT_COUNTERS +/* enum vm_event_item counters */ "pgpgin" ,"pgpgout" , "pswpin",
tools/vm/page-types.c +1 -0
@@ -132,6 +132,7 @@ static const char * const page_flag_names[] = { [KPF_NOPAGE] = "n:nopage", [KPF_KSM] = "x:ksm", [KPF_THP] = "t:thp", + [KPF_BALLOON] = "o:balloon", [KPF_RESERVED] = "r:reserved", [KPF_MLOCKED] = "m:mlocked",
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v2 5/6 mm/balloon_compaction: use common page ballooning
In reply to Konstantin KhlebnikovFrom: Konstantin Khlebnikov <[email protected]>
This patch replaces checking AS_BALLOON_MAP in page->mapping->flags with PageBalloon which is stored directly in the struct page. All code of balloon_compaction now under CONFIG_MEMORY_BALLOON.
--- drivers/virtio/Kconfig | 1 include/linux/balloon_compaction.h | 135 ++++++------------------------------ mm/Kconfig | 2 - mm/Makefile | 3 + mm/balloon_compaction.c | 7 +- mm/compaction.c | 9 +- mm/migrate.c | 6 +- mm/vmscan.c | 2 - 8 files changed, 39 insertions(+), 126 deletions(-)
drivers/virtio/Kconfig +1 -0
@@ -25,6 +25,7 @@ config VIRTIO_PCI config VIRTIO_BALLOON tristate "Virtio balloon driver" depends on VIRTIO + select MEMORY_BALLOON ---help--- This driver supports increasing and decreasing the amount of memory within a KVM guest.
include/linux/balloon_compaction.h +23 -112
169 lines not on this page · raw message
mm/Kconfig +1 -1
@@ -237,7 +237,7 @@ config MEMORY_BALLOON config BALLOON_COMPACTION bool "Allow for balloon memory compaction/migration" def_bool y - depends on COMPACTION && VIRTIO_BALLOON + depends on COMPACTION && MEMORY_BALLOON help Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can be
mm/Makefile +2 -1
@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \ readahead.o swap.o truncate.o vmscan.o shmem.o \ util.o mmzone.o vmstat.o backing-dev.o \ mm_init.o mmu_context.o percpu.o slab_common.o \ - compaction.o balloon_compaction.o vmacache.o \ + compaction.o vmacache.o \ interval_tree.o list_lru.o workingset.o \ iov_iter.o $(mmu-y)
@@ -64,3 +64,4 @@ obj-$(CONFIG_ZBUD) += zbud.o obj-$(CONFIG_ZSMALLOC) += zsmalloc.o obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o obj-$(CONFIG_CMA) += cma.o +obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
mm/balloon_compaction.c +3 -4
@@ -239,8 +239,7 @@ bool balloon_page_isolate(struct page *page)* Prevent concurrent compaction threads from isolating * an already isolated balloon page by refcount check. */ -if (__is_movable_balloon_page (page) && -page_count (page) ==2 ) { +if (PageBalloon (page) &&page_count (page) ==2 ) {__isolate_balloon_page (page);unlock_page (page);return true;
@@ -261,7 +260,7 @@ void balloon_page_putback(struct page *page)*/ lock_page (page); -if (__is_movable_balloon_page (page)) { +if (PageBalloon (page)) {__putback_balloon_page (page);/* drop the extra ref count taken for page isolation */ put_page (page);
@@ -286,7 +285,7 @@ int balloon_page_migrate(struct page *newpage,*/ BUG_ON (!trylock_page (newpage)); -if (WARN_ON (!__is_movable_balloon_page (page))) { +if (WARN_ON (!PageBalloon (page))) {dump_page (page,"not movable balloon page" );unlock_page (newpage);return rc;
mm/compaction.c +4 -5
@@ -642,11 +642,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,* Skip any other type of page */ if (!PageLRU (page)) { -if (unlikely (balloon_page_movable (page))) { -if (balloon_page_isolate (page)) { -/* Successfully isolated */ - goto isolate_success; - } +if (unlikely (PageBalloon (page)) && +balloon_page_isolate (page)) { +/* Successfully isolated */ + goto isolate_success; }continue ; }
mm/migrate.c +4 -2
@@ -92,7 +92,7 @@ void putback_movable_pages(struct list_head *l)list_del (&page->lru);dec_zone_page_state (page,NR_ISOLATED_ANON +page_is_file_cache (page)); -if (unlikely (isolated_balloon_page (page))) +if (unlikely (PageBalloon (page)))balloon_page_putback (page);else putback_lru_page (page);
@@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage, } } -if (unlikely (__is_movable_balloon_page (page))) { +if (unlikely (PageBalloon (page))) {/* * A ballooned page does not need any special attention from * physical to virtual reverse mapping procedures.
@@ -952,6 +952,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page, rc =__unmap_and_move (page, newpage, force, mode); +#ifdef CONFIG_MEMORY_BALLOON if (unlikely (rc ==MIGRATEPAGE_BALLOON_SUCCESS )) {/* * A ballooned page has been migrated already.
@@ -963,6 +964,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,balloon_page_free (page); return MIGRATEPAGE_SUCCESS; } +#endif out:if (rc != -EAGAIN ) {/*
mm/vmscan.c +1 -1
@@ -1160,7 +1160,7 @@ unsigned long reclaim_clean_pages_from_list(struct zone *zone,list_for_each_entry_safe (page, next, page_list, lru) {if (page_is_file_cache (page) && !PageDirty (page) && - !isolated_balloon_page (page)) { + !PageBalloon (page)) {ClearPageActive (page);list_move (&page->lru, &clean_pages); }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
PATCH v2 6/6 mm/balloon_compaction: general cleanup
In reply to Konstantin KhlebnikovFrom: Konstantin Khlebnikov <[email protected]>
* move special branch for balloon migraion into migrate_pages
* remove special mapping for balloon and its flag AS_BALLOON_MAP
* embed struct balloon_dev_info into struct virtio_balloon
* cleanup balloon_page_dequeue, kill balloon_page_free
--- drivers/virtio/virtio_balloon.c | 77 ++++--------- include/linux/balloon_compaction.h | 127 ++++++--------------- include/linux/migrate.h | 11 -- include/linux/pagemap.h | 18 --- mm/balloon_compaction.c | 214 ++++++++++++------------------------ mm/migrate.c | 29 +---- 6 files changed, 134 insertions(+), 342 deletions(-)
drivers/virtio/virtio_balloon.c +21 -56
@@ -59,7 +59,7 @@ struct virtio_balloon* Each page on this list adds VIRTIO_BALLOON_PAGES_PER_PAGE * to num_pages above. */ -struct balloon_dev_info *vb_dev_info; +struct balloon_dev_info vb_dev_info;/* Synchronize access/update to this struct virtio_balloon elements */ struct mutex balloon_lock;
@@ -127,7 +127,7 @@ static void set_page_pfns(u32 pfns[], struct page *page)static void fill_balloon (struct virtio_balloon *vb,size_t num) { -struct balloon_dev_info *vb_dev_info = vb->vb_dev_info; +struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;/* We can only do one array worth at a time. */ num =min (num,ARRAY_SIZE (vb->pfns));
@@ -163,15 +163,15 @@ static void release_pages_by_pfn(const u32 pfns[], unsigned int num)/* Find pfns pointing at start of each page, get pages and free them. */ for (i =0 ; i < num; i +=VIRTIO_BALLOON_PAGES_PER_PAGE ) {struct page *page =balloon_pfn_to_page (pfns[i]); -balloon_page_free (page);adjust_managed_page_count (page,1 ); +put_page (page); } }static void leak_balloon (struct virtio_balloon *vb,size_t num) {struct page *page; -struct balloon_dev_info *vb_dev_info = vb->vb_dev_info; +struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;/* We can only do one array worth at a time. */ num =min (num,ARRAY_SIZE (vb->pfns));
@@ -353,12 +353,11 @@ static int init_vqs(struct virtio_balloon *vb)return 0 ; } -static const struct address_space_operations virtio_balloon_aops;#ifdef CONFIG_BALLOON_COMPACTION /* * virtballoon_migratepage - perform the balloon page migration on behalf of * a compation thread. (called under page lock) -* @mapping: the page->mapping which will be assigned to the new migrated page. +* @vb_dev_info: the balloon device * @newpage: page that will replace the isolated page after migration finishes. * @page : the isolated (old) page that is about to be migrated to newpage. * @mode : compaction mode -- not used for balloon page migration.
@@ -373,17 +372,13 @@ static const struct address_space_operations virtio_balloon_aops;* This function preforms the balloon page migration task. * Called through balloon_mapping->a_ops->migratepage */ -static int virtballoon_migratepage (struct address_space *mapping, +static int virtballoon_migratepage (struct balloon_dev_info *vb_dev_info,struct page *newpage,struct page *page,enum migrate_mode mode) { -struct balloon_dev_info *vb_dev_info =balloon_page_device (page); -struct virtio_balloon *vb; +struct virtio_balloon *vb =container_of (vb_dev_info, + struct virtio_balloon, vb_dev_info);unsigned long flags; -BUG_ON (!vb_dev_info); - - vb = vb_dev_info->balloon_device; -/* * In order to avoid lock contention while migrating pages concurrently * to leak_balloon() or fill_balloon() we just give up the balloon_lock
@@ -395,42 +390,34 @@ static int virtballoon_migratepage(struct address_space *mapping,if (!mutex_trylock (&vb->balloon_lock))return -EAGAIN ; +get_page (newpage);/* balloon reference */ +/* balloon's page migration 1st step -- inflate "newpage" */ spin_lock_irqsave (&vb_dev_info->pages_lock, flags); -balloon_page_insert (newpage, mapping, &vb_dev_info->pages); +balloon_page_insert (vb_dev_info, newpage); vb_dev_info->isolated_pages--;spin_unlock_irqrestore (&vb_dev_info->pages_lock, flags); vb->num_pfns =VIRTIO_BALLOON_PAGES_PER_PAGE ;set_page_pfns (vb->pfns, newpage);tell_host (vb, vb->inflate_vq); -/* -* balloon's page migration 2nd step -- deflate "page" -* -* It's safe to delete page->lru here because this page is at -* an isolated migration list, and this step is expected to happen here -*/ -balloon_page_delete (page); +/* balloon's page migration 2nd step -- deflate "page" */ +balloon_page_delete (page, true); vb->num_pfns =VIRTIO_BALLOON_PAGES_PER_PAGE ;set_page_pfns (vb->pfns, page);tell_host (vb, vb->deflate_vq);mutex_unlock (&vb->balloon_lock); - return MIGRATEPAGE_BALLOON_SUCCESS; -} +put_page (page);/* balloon reference */ -/* define the balloon_mapping->a_ops callback to allow balloon page migration */ -static const struct address_space_operations virtio_balloon_aops = { - .migratepage = virtballoon_migratepage, -}; + return MIGRATEPAGE_SUCCESS; +}#endif /* CONFIG_BALLOON_COMPACTION */ static int virtballoon_probe (struct virtio_device *vdev) {struct virtio_balloon *vb; -struct address_space *vb_mapping; -struct balloon_dev_info *vb_devinfo;int err; vdev->priv = vb =kmalloc (sizeof (*vb),GFP_KERNEL );
@@ -446,30 +433,14 @@ static int virtballoon_probe(struct virtio_device *vdev) vb->vdev = vdev; vb->need_stats_update =0 ; - vb_devinfo =balloon_devinfo_alloc (vb); -if (IS_ERR (vb_devinfo)) { - err =PTR_ERR (vb_devinfo); - goto out_free_vb; - } - - vb_mapping =balloon_mapping_alloc (vb_devinfo, - (balloon_compaction_check ()) ? - &virtio_balloon_aops :NULL ); -if (IS_ERR (vb_mapping)) { -/* -* IS_ERR(vb_mapping) && PTR_ERR(vb_mapping) == -EOPNOTSUPP -* This means !CONFIG_BALLOON_COMPACTION, otherwise we get off. -*/ - err =PTR_ERR (vb_mapping); -if (err != -EOPNOTSUPP ) - goto out_free_vb_devinfo; - } - - vb->vb_dev_info = vb_devinfo; +balloon_devinfo_init (&vb->vb_dev_info); +#ifdef CONFIG_BALLOON_COMPACTION + vb->vb_dev_info.migratepage = virtballoon_migratepage; +#endif err =init_vqs (vb);if (err) - goto out_free_vb_mapping; + goto out_free_vb; vb->thread =kthread_run (balloon, vb,"vballoon" );if (IS_ERR (vb->thread)) {
@@ -481,10 +452,6 @@ static int virtballoon_probe(struct virtio_device *vdev) out_del_vqs: vdev->config->del_vqs (vdev); -out_free_vb_mapping: -balloon_mapping_free (vb_mapping); -out_free_vb_devinfo: -balloon_devinfo_free (vb_devinfo); out_free_vb:kfree (vb); out:
@@ -510,8 +477,6 @@ static void virtballoon_remove(struct virtio_device *vdev)kthread_stop (vb->thread);remove_common (vb); -balloon_mapping_free (vb->vb_dev_info->mapping); -balloon_devinfo_free (vb->vb_dev_info);kfree (vb); }
include/linux/balloon_compaction.h +38 -89
188 lines not on this page · raw message
include/linux/migrate.h +1 -10
@@ -13,18 +13,9 @@ typedef void free_page_t(struct page *page, unsigned long private);* Return values from addresss_space_operations.migratepage(): * - negative errno on page migration failure; * - zero on page migration success; -* -* The balloon page migration introduces this special case where a 'distinct' -* return code is used to flag a successful page migration to unmap_and_move(). -* This approach is necessary because page migration can race against balloon -* deflation procedure, and for such case we could introduce a nasty page leak -* if a successfully migrated balloon page gets released concurrently with -* migration's unmap_and_move() wrap-up steps. */ #define MIGRATEPAGE_SUCCESS 0 -#define MIGRATEPAGE_BALLOON_SUCCESS 1/* special ret code for balloon page -* sucessful migration case. -*/ +enum migrate_reason {MR_COMPACTION ,MR_MEMORY_FAILURE ,
include/linux/pagemap.h +1 -17
@@ -24,8 +24,7 @@ enum mapping_flags {AS_ENOSPC = __GFP_BITS_SHIFT +1 ,/* ENOSPC on async write */ AS_MM_ALL_LOCKS = __GFP_BITS_SHIFT +2 ,/* under mm_take_all_locks() */ AS_UNEVICTABLE = __GFP_BITS_SHIFT +3 ,/* e.g., ramdisk, SHM_LOCK */ -AS_BALLOON_MAP = __GFP_BITS_SHIFT +4 ,/* balloon page special map */ - AS_EXITING = __GFP_BITS_SHIFT + 5, /* final truncate in progress */ + AS_EXITING = __GFP_BITS_SHIFT + 4, /* final truncate in progress */ };static inline void mapping_set_error (struct address_space *mapping,int error)
@@ -55,21 +54,6 @@ static inline int mapping_unevictable(struct address_space *mapping)return !!mapping; } -static inline void mapping_set_balloon (struct address_space *mapping) -{ -set_bit (AS_BALLOON_MAP , &mapping->flags); -} - -static inline void mapping_clear_balloon (struct address_space *mapping) -{ -clear_bit (AS_BALLOON_MAP , &mapping->flags); -} - -static inline int mapping_balloon (struct address_space *mapping) -{ -return mapping &&test_bit (AS_BALLOON_MAP , &mapping->flags); -} -static inline void mapping_set_exiting (struct address_space *mapping) {set_bit (AS_EXITING , &mapping->flags);
mm/balloon_compaction.c +69 -145
274 lines not on this page · raw message
mm/migrate.c +4 -25
@@ -873,18 +873,6 @@ static int __unmap_and_move(struct page *page, struct page *newpage, } } -if (unlikely (PageBalloon (page))) { -/* -* A ballooned page does not need any special attention from -* physical to virtual reverse mapping procedures. -* Skip any attempt to unmap PTEs or to remap swap cache, -* in order to avoid burning cycles at rmap level, and perform -* the page migration right away (proteced by page lock). -*/ - rc =balloon_page_migrate (newpage, page, mode); - goto out_unlock; - } -/* * Corner case handling: * 1. When a new swap-cache page is read into, it is added to the LRU
@@ -952,19 +940,6 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page, rc =__unmap_and_move (page, newpage, force, mode); -#ifdef CONFIG_MEMORY_BALLOON -if (unlikely (rc ==MIGRATEPAGE_BALLOON_SUCCESS )) { -/* -* A ballooned page has been migrated already. -* Now, it's the time to wrap-up counters, -* handle the page back to Buddy and return. -*/ -dec_zone_page_state (page,NR_ISOLATED_ANON + -page_is_file_cache (page)); -balloon_page_free (page); -return MIGRATEPAGE_SUCCESS ; - } -#endif out:if (rc != -EAGAIN ) {/*
@@ -1139,6 +1114,10 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, rc =unmap_and_move_huge_page (get_new_page, put_new_page, private, page, pass >2 , mode); + else if (PageBalloon(page)) + rc =balloon_page_migrate (get_new_page, + put_new_page, private, + page, pass >2 , mode); else rc =unmap_and_move (get_new_page, put_new_page, private, page, pass >2 , mode);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 1/6] mm/balloon_compaction: ignore anonymous pages
In reply to Konstantin KhlebnikovQuoted 21 lines · Konstantin Khlebnikov
From: Konstantin Khlebnikov <[email protected]>
Sasha Levin reported KASAN splash inside
isolate_migratepages_range(). Problem is in function__is_movable_balloon_page()which testsAS_BALLOON_MAPinpage->mapping->flags. This function has no protection against anonymous pages. As result it tried to check address space flags in inside anon-vma.Signed-off-by Konstantin Khlebnikov <[email protected]>Reported-by Sasha Levin <[email protected]>Cc stable <[email protected]> # v3.8--- include/linux/balloon_compaction.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)include/linux/balloon_compaction.h +0
-0
include/linux/balloon_compaction.h static inline bool page_flags_cleared(struct page *page) · line 134
Quoted 8 lines
include/linux/balloon_compaction.h +1
-1@@ -128,7 +128,7 @@ static inline bool page_flags_cleared(struct page *page)static inline bool __is_movable_balloon_page (struct page *page) {struct address_space *mapping = page->mapping; -return mapping_balloon (mapping); +return !PageAnon (page) &&mapping_balloon (mapping); }/*
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 2/6] mm/balloon_compaction: keep ballooned pages away from normal migration path
In reply to Konstantin KhlebnikovQuoted 44 lines · Konstantin Khlebnikov
From: Konstantin Khlebnikov <[email protected]>
Proper testing shows yet another problem in balloon migration: it works only once for each page.
balloon_page_movable()check page flags andpage_count. In__unmap_and_movepage is locked, reference counter is elevated, soballoon_page_movable()_always_ fails here. As result in__unmap_and_move()migration goes to the normal migration path.Balloon ->migratepage() is so special, it returns
MIGRATEPAGE_BALLOON_SUCCESSinstead ofMIGRATEPAGE_SUCCESS. After that inmove_to_new_page()successfully migrated page got NULL into its mapping pointer and loses connectivity with balloon and ability for further migration.It's safe to use
__is_movable_balloon_pagehere: page is isolated and pinned.Signed-off-by Konstantin Khlebnikov <[email protected]>Cc stable <[email protected]> # v3.8--- include/linux/balloon_compaction.h | 5 +++++ mm/migrate.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-)include/linux/balloon_compaction.h +5
-0@@ -258,6 +258,11 @@ static inline void balloon_page_delete(struct page *page)list_del (&page->lru); } +static inline bool __is_movable_balloon_page (struct page *page) +{ +return false; +} +static inline bool balloon_page_movable (struct page *page) {return false;mm/migrate.c +0
-0
mm/migrate.c static int __unmap_and_move(struct page *page, struct page *newpage, · line 879
Quoted 8 lines
mm/migrate.c +1
-1@@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage, } } -if (unlikely (balloon_page_movable (page))) { +if (unlikely (__is_movable_balloon_page (page))) {/* * A ballooned page does not need any special attention from * physical to virtual reverse mapping procedures.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 3/6] mm/balloon_compaction: isolate balloon pages without lru_lock
In reply to Konstantin KhlebnikovQuoted 18 lines · Konstantin Khlebnikov
From: Konstantin Khlebnikov <[email protected]>
LRU-lock isn't required for balloon page isolation. This check makes migration of some ballooned pages mostly impossible because
isolate_migratepages_range()drops LRU lock periodically.Signed-off-by Konstantin Khlebnikov <[email protected]>Cc stable <[email protected]> # v3.8--- mm/compaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/compaction.c +0
-0
mm/compaction.c isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, · line 649
Quoted 8 lines
mm/compaction.c +1
-1@@ -643,7 +643,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,*/ if (!PageLRU (page)) {if (unlikely (balloon_page_movable (page))) { -if (locked &&balloon_page_isolate (page)) { +if (balloon_page_isolate (page)) {/* Successfully isolated */ goto isolate_success; }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
In reply to Konstantin KhlebnikovQuoted 231 lines · Konstantin Khlebnikov
From: Konstantin Khlebnikov <[email protected]>
This patch adds page state
PageBallon()and functions __Set/ClearPageBalloon. LikePageBuddy()PageBalloon()looks like page-flag but actually this is special state ofpage->_mapcountcounter. There is no conflict because ballooned pages cannot be mapped and cannot be in buddy allocator.Ballooned pages are counted in vmstat counter
NR_BALLOON_PAGES, it's shown them in/proc/meminfoand/proc/meminfo. Also this patch it exports PageBallon into userspace via/proc/kpageflagsasKPF_BALLOON.All new code is under
CONFIG_MEMORY_BALLOON, it should be selected by ballooning driver which wants use this feature.Signed-off-by Konstantin Khlebnikov <[email protected]>--- Documentation/filesystems/proc.txt | 2 ++ drivers/base/node.c | 16 ++++++++++------ fs/proc/meminfo.c | 6 ++++++ fs/proc/page.c | 3 +++ include/linux/mm.h | 20 ++++++++++++++++++++ include/linux/mmzone.h | 3 +++ include/uapi/linux/kernel-page-flags.h | 1 + mm/Kconfig | 5 +++++ mm/vmstat.c | 8 +++++++- tools/vm/page-types.c | 1 + 10 files changed, 58 insertions(+), 7 deletions(-)Documentation/filesystems/proc.txt +2
-0@@ -796,6 +796,7 @@ VmallocTotal: 112216 kB VmallocUsed: 428 kB VmallocChunk: 111088 kB AnonHugePages: 49152 kB +BalloonPages: 0 kB MemTotal: Total usable ram (i.e. physical ram minus a few reserved bits and the kernel binary code)@@ -838,6 +839,7 @@ MemAvailable: An estimate of how much memory is available for starting new Writeback: Memory which is actively being written back to the disk AnonPages: Non-file backed pages mapped into userspace page tables AnonHugePages: Non-file backed huge pages mapped into userspace page tables +BalloonPages: Memory which was ballooned, not included into MemTotal Mapped: files which have been mmaped, such as libraries Slab: in-kernel data structures cache SReclaimable: Part of Slab, that might be reclaimed, such as cachesdrivers/base/node.c +10
-6@@ -120,6 +120,9 @@ static ssize_t node_read_meminfo(struct device *dev,#ifdef CONFIG_TRANSPARENT_HUGEPAGE "Node %d AnonHugePages: %8lu kB\n"#endif +#ifdef CONFIG_MEMORY_BALLOON +"Node %d BalloonPages: %8lu kB\n" +#endif , nid,K (node_page_state (nid,NR_FILE_DIRTY )), nid, K(node_page_state(nid, NR_WRITEBACK)),@@ -136,14 +139,15 @@ static ssize_t node_read_meminfo(struct device *dev, nid,K (node_page_state (nid,NR_SLAB_RECLAIMABLE ) +node_page_state (nid,NR_SLAB_UNRECLAIMABLE )), nid,K (node_page_state (nid,NR_SLAB_RECLAIMABLE )), -#ifdef CONFIG_TRANSPARENT_HUGEPAGE nid,K (node_page_state (nid,NR_SLAB_UNRECLAIMABLE )) - , nid, -K (node_page_state (nid,NR_ANON_TRANSPARENT_HUGEPAGES ) * - HPAGE_PMD_NR)); -#else - nid, K(node_page_state(nid, NR_SLAB_UNRECLAIMABLE))); +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + ,nid,K (node_page_state (nid, +NR_ANON_TRANSPARENT_HUGEPAGES ) *HPAGE_PMD_NR ) +#endif +#ifdef CONFIG_MEMORY_BALLOON + ,nid,K (node_page_state (nid,NR_BALLOON_PAGES )) #endif + ); n +=hugetlb_report_node_meminfo (nid, buf + n);return n; }fs/proc/meminfo.c +6
-0@@ -138,6 +138,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v)#ifdef CONFIG_TRANSPARENT_HUGEPAGE "AnonHugePages: %8lu kB\n"#endif +#ifdef CONFIG_MEMORY_BALLOON +"BalloonPages: %8lu kB\n" +#endif ,K (i.totalram), K(i.freeram),@@ -193,6 +196,9 @@ static int meminfo_proc_show(struct seq_file *m, void *v) ,K(global_page_state(NR_ANON_TRANSPARENT_HUGEPAGES) * HPAGE_PMD_NR)#endif +#ifdef CONFIG_MEMORY_BALLOON + ,K(global_page_state(NR_BALLOON_PAGES)) +#endif );hugetlb_report_meminfo (m);fs/proc/page.c +3
-0@@ -133,6 +133,9 @@ u64 stable_page_flags(struct page *page)if (PageBuddy (page)) u |=1 <<KPF_BUDDY ; +if (PageBalloon (page)) + u |=1 <<KPF_BALLOON ; + u |=kpf_copy_bit (k,KPF_LOCKED , PG_locked); u |=kpf_copy_bit (k,KPF_SLAB , PG_slab);include/linux/mm.h +20
-0@@ -554,6 +554,26 @@ static inline void __ClearPageBuddy(struct page *page)atomic_set (&page->_mapcount,-1 ); } +#define PAGE_BALLOON_MAPCOUNT_VALUE (-256) + +static inline int PageBalloon (struct page *page) +{ +return IS_ENABLED (CONFIG_MEMORY_BALLOON ) && +atomic_read (&page->_mapcount) ==PAGE_BALLOON_MAPCOUNT_VALUE ; +} + +static inline void __SetPageBalloon (struct page *page) +{ +VM_BUG_ON_PAGE (atomic_read (&page->_mapcount) !=-1 , page); +atomic_set (&page->_mapcount,PAGE_BALLOON_MAPCOUNT_VALUE ); +} + +static inline void __ClearPageBalloon (struct page *page) +{ +VM_BUG_ON_PAGE (!PageBalloon (page), page); +atomic_set (&page->_mapcount,-1 ); +} +void put_page (struct page *page);void put_pages_list (struct list_head *pages);include/linux/mmzone.h +3
-0@@ -157,6 +157,9 @@ enum zone_stat_item {WORKINGSET_NODERECLAIM ,NR_ANON_TRANSPARENT_HUGEPAGES ,NR_FREE_CMA_PAGES , +#ifdef CONFIG_MEMORY_BALLOON +NR_BALLOON_PAGES , +#endif NR_VM_ZONE_STAT_ITEMS };/* include/uapi/linux/kernel-page-flags.h +1
-0@@ -31,6 +31,7 @@#define KPF_KSM 21#define KPF_THP 22 +#define KPF_BALLOON 23#endif /* _UAPILINUX_KERNEL_PAGE_FLAGS_H */ mm/Kconfig +5
-0@@ -228,6 +228,11 @@ config ARCH_ENABLE_SPLIT_PMD_PTLOCK boolean # +# support for memory ballooning +config MEMORY_BALLOON + boolean + +# # support for memory balloon compaction config BALLOON_COMPACTION bool "Allow for balloon memory compaction/migration"mm/vmstat.c +7
-1@@ -751,7 +751,7 @@ static void walk_zones_in_node(struct seq_file *m, pg_data_t *pgdat, TEXT_FOR_HIGHMEM(xx) xx "_movable",const char *const vmstat_text[] = { -/* Zoned VM counters */ +/* enum zone_stat_item countes */ "nr_free_pages" ,"nr_alloc_batch" ,"nr_inactive_anon" ,@@ -794,10 +794,16 @@ const char * const vmstat_text[] = {"workingset_nodereclaim" ,"nr_anon_transparent_hugepages" ,"nr_free_cma" , +#ifdef CONFIG_MEMORY_BALLOON +"nr_balloon_pages" , +#endif + +/* enum writeback_stat_item counters */ "nr_dirty_threshold" ,"nr_dirty_background_threshold" , #ifdef CONFIG_VM_EVENT_COUNTERS +/* enum vm_event_item counters */ "pgpgin" ,"pgpgout" , "pswpin",tools/vm/page-types.c +0
-0
tools/vm/page-types.c static const char * const page_flag_names[] = { · line 138
Quoted 7 lines
tools/vm/page-types.c +1
-0@@ -132,6 +132,7 @@ static const char * const page_flag_names[] = { [KPF_NOPAGE] = "n:nopage", [KPF_KSM] = "x:ksm", [KPF_THP] = "t:thp", + [KPF_BALLOON] = "o:balloon", [KPF_RESERVED] = "r:reserved", [KPF_MLOCKED] = "m:mlocked",
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 5/6] mm/balloon_compaction: use common page ballooning
In reply to Konstantin KhlebnikovQuoted 335 lines · Konstantin Khlebnikov
From: Konstantin Khlebnikov <[email protected]>
This patch replaces checking
AS_BALLOON_MAPinpage->mapping->flagswith PageBalloon which is stored directly in the struct page. All code ofballoon_compactionnow underCONFIG_MEMORY_BALLOON.Signed-off-by Konstantin Khlebnikov <[email protected]>--- drivers/virtio/Kconfig | 1 include/linux/balloon_compaction.h | 135 ++++++------------------------------ mm/Kconfig | 2 - mm/Makefile | 3 + mm/balloon_compaction.c | 7 +- mm/compaction.c | 9 +- mm/migrate.c | 6 +- mm/vmscan.c | 2 - 8 files changed, 39 insertions(+), 126 deletions(-)drivers/virtio/Kconfig +1
-0@@ -25,6 +25,7 @@ config VIRTIO_PCI config VIRTIO_BALLOON tristate "Virtio balloon driver" depends on VIRTIO + select MEMORY_BALLOON ---help--- This driver supports increasing and decreasing the amount of memory within a KVM guest.include/linux/balloon_compaction.h +23
-112@@ -46,6 +46,8 @@#include <linux/gfp.h> #include <linux/err.h> +#ifdef CONFIG_MEMORY_BALLOON +/* * Balloon device information descriptor. * This struct is used to allow the common balloon compaction interface @@ -93,91 +95,6 @@ static inline void balloon_page_free(struct page *page)__free_page (page); } -#ifdef CONFIG_BALLOON_COMPACTION -extern bool balloon_page_isolate (struct page *page); -extern void balloon_page_putback (struct page *page); -extern int balloon_page_migrate (struct page *newpage, -struct page *page,enum migrate_mode mode); -extern struct address_space -*balloon_mapping_alloc (struct balloon_dev_info *b_dev_info, -const struct address_space_operations *a_ops); - -static inline void balloon_mapping_free (struct address_space *balloon_mapping) -{ -kfree (balloon_mapping); -} - -/* -* page_flags_cleared - helper to perform balloon @page ->flags tests. -* -* As balloon pages are obtained from buddy and we do not play with page->flags -* at driver level (exception made when we get the page lock for compaction), -* we can safely identify a ballooned page by checking if the -* PAGE_FLAGS_CHECK_AT_PREP page->flags are all cleared. This approach also -* helps us skip ballooned pages that are locked for compaction or release, thus -* mitigating their racy check at balloon_page_movable() -*/ -static inline bool page_flags_cleared (struct page *page) -{ -return !(page->flags &PAGE_FLAGS_CHECK_AT_PREP ); -} - -/* -* __is_movable_balloon_page - helper to perform @page mapping->flags tests -*/ -static inline bool __is_movable_balloon_page (struct page *page) -{ -struct address_space *mapping = page->mapping; -return !PageAnon (page) &&mapping_balloon (mapping); -} - -/* -* balloon_page_movable - test page->mapping->flags to identify balloon pages -* that can be moved by compaction/migration. -* -* This function is used at core compaction's page isolation scheme, therefore -* most pages exposed to it are not enlisted as balloon pages and so, to avoid -* undesired side effects like racing against __free_pages(), we cannot afford -* holding the page locked while testing page->mapping->flags here. -* -* As we might return false positives in the case of a balloon page being just -* released under us, the page->mapping->flags need to be re-tested later, -* under the proper page lock, at the functions that will be coping with the -* balloon page case. -*/ -static inline bool balloon_page_movable (struct page *page) -{ -/* -* Before dereferencing and testing mapping->flags, let's make sure -* this is not a page that uses ->mapping in a different way -*/ -if (page_flags_cleared (page) && !page_mapped (page) && -page_count (page) ==1 ) -return __is_movable_balloon_page (page); - -return false; -} - -/* -* isolated_balloon_page - identify an isolated balloon page on private -* compaction/migration page lists. -* -* After a compaction thread isolates a balloon page for migration, it raises -* the page refcount to prevent concurrent compaction threads from re-isolating -* the same page. For that reason putback_movable_pages(), or other routines -* that need to identify isolated balloon pages on private pagelists, cannot -* rely on balloon_page_movable() to accomplish the task. -*/ -static inline bool isolated_balloon_page (struct page *page) -{ -/* Already isolated balloon pages, by default, have a raised refcount */ -if (page_flags_cleared (page) && !page_mapped (page) && -page_count (page) >=2 ) -return __is_movable_balloon_page (page); - -return false; -} -/* * balloon_page_insert - insert a page into the balloon's page list and make * the page->mapping assignment accordingly. @@ -192,6 +109,8 @@ static inline void balloon_page_insert(struct page *page,struct address_space *mapping, struct list_head *head) { +__SetPageBalloon (page ); +inc_zone_page_state (page,NR_BALLOON_PAGES ); page->mapping = mapping;list_add (&page->lru, head); }@@ -206,10 +125,29 @@ static inline void balloon_page_insert(struct page *page,*/ static inline void balloon_page_delete (struct page *page) { +__ClearPageBalloon (page); +dec_zone_page_state (page,NR_BALLOON_PAGES ); page->mapping =NULL ;list_del (&page->lru); } +#endif /* CONFIG_MEMORY_BALLOON */ + +#ifdef CONFIG_BALLOON_COMPACTION + +extern bool balloon_page_isolate (struct page *page); +extern void balloon_page_putback (struct page *page); +extern int balloon_page_migrate (struct page *newpage, +struct page *page,enum migrate_mode mode); +extern struct address_space +*balloon_mapping_alloc (struct balloon_dev_info *b_dev_info, +const struct address_space_operations *a_ops); + +static inline void balloon_mapping_free (struct address_space *balloon_mapping) +{ +kfree (balloon_mapping); +} +/* * balloon_page_device - get the b_dev_info descriptor for the balloon device * that enqueues the given page. @@ -246,33 +184,6 @@ static inline void balloon_mapping_free(struct address_space *balloon_mapping)return ; } -static inline void balloon_page_insert (struct page *page, -struct address_space *mapping, -struct list_head *head) -{ -list_add (&page->lru, head); -} - -static inline void balloon_page_delete (struct page *page) -{ -list_del (&page->lru); -} - -static inline bool __is_movable_balloon_page (struct page *page) -{ -return false; -} - -static inline bool balloon_page_movable (struct page *page) -{ -return false; -} - -static inline bool isolated_balloon_page (struct page *page) -{ -return false; -} -static inline bool balloon_page_isolate (struct page *page) {return false;mm/Kconfig +1
-1@@ -237,7 +237,7 @@ config MEMORY_BALLOON config BALLOON_COMPACTION bool "Allow for balloon memory compaction/migration" def_bool y - depends on COMPACTION && VIRTIO_BALLOON + depends on COMPACTION && MEMORY_BALLOON help Memory fragmentation introduced by ballooning might reduce significantly the number of 2MB contiguous memory blocks that can bemm/Makefile +2
-1@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill.o fadvise.o \ readahead.o swap.o truncate.o vmscan.o shmem.o \ util.o mmzone.o vmstat.o backing-dev.o \ mm_init.o mmu_context.o percpu.o slab_common.o \ - compaction.o balloon_compaction.o vmacache.o \ + compaction.o vmacache.o \ interval_tree.o list_lru.o workingset.o \ iov_iter.o $(mmu-y)@@ -64,3 +64,4 @@ obj-$(CONFIG_ZBUD) += zbud.o obj-$(CONFIG_ZSMALLOC) += zsmalloc.o obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o obj-$(CONFIG_CMA) += cma.o +obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.omm/balloon_compaction.c +3
-4@@ -239,8 +239,7 @@ bool balloon_page_isolate(struct page *page)* Prevent concurrent compaction threads from isolating * an already isolated balloon page by refcount check. */ -if (__is_movable_balloon_page (page) && -page_count (page) ==2 ) { +if (PageBalloon (page) &&page_count (page) ==2 ) {__isolate_balloon_page (page);unlock_page (page);return true;@@ -261,7 +260,7 @@ void balloon_page_putback(struct page *page)*/ lock_page (page); -if (__is_movable_balloon_page (page)) { +if (PageBalloon (page)) {__putback_balloon_page (page);/* drop the extra ref count taken for page isolation */ put_page (page);@@ -286,7 +285,7 @@ int balloon_page_migrate(struct page *newpage,*/ BUG_ON (!trylock_page (newpage)); -if (WARN_ON (!__is_movable_balloon_page (page))) { +if (WARN_ON (!PageBalloon (page))) {dump_page (page,"not movable balloon page" );unlock_page (newpage);return rc;mm/compaction.c +4
-5@@ -642,11 +642,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,* Skip any other type of page */ if (!PageLRU (page)) { -if (unlikely (balloon_page_movable (page))) { -if (balloon_page_isolate (page)) { -/* Successfully isolated */ - goto isolate_success; - } +if (unlikely (PageBalloon (page)) && +balloon_page_isolate (page)) { +/* Successfully isolated */ + goto isolate_success; }continue ; }mm/migrate.c +4
-2@@ -92,7 +92,7 @@ void putback_movable_pages(struct list_head *l)list_del (&page->lru);dec_zone_page_state (page,NR_ISOLATED_ANON +page_is_file_cache (page)); -if (unlikely (isolated_balloon_page (page))) +if (unlikely (PageBalloon (page)))balloon_page_putback (page);else putback_lru_page (page);@@ -873,7 +873,7 @@ static int __unmap_and_move(struct page *page, struct page *newpage, } } -if (unlikely (__is_movable_balloon_page (page))) { +if (unlikely (PageBalloon (page))) {/* * A ballooned page does not need any special attention from * physical to virtual reverse mapping procedures. @@ -952,6 +952,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page, rc =__unmap_and_move (page, newpage, force, mode); +#ifdef CONFIG_MEMORY_BALLOON if (unlikely (rc ==MIGRATEPAGE_BALLOON_SUCCESS )) {/* * A ballooned page has been migrated already. @@ -963,6 +964,7 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page,balloon_page_free (page); return MIGRATEPAGE_SUCCESS; } +#endif out:if (rc != -EAGAIN ) {/* mm/vmscan.c +0
-0
mm/vmscan.c unsigned long reclaim_clean_pages_from_list(struct zone *zone, · line 1166
Quoted 8 lines
mm/vmscan.c +1
-1@@ -1160,7 +1160,7 @@ unsigned long reclaim_clean_pages_from_list(struct zone *zone,list_for_each_entry_safe (page, next, page_list, lru) {if (page_is_file_cache (page) && !PageDirty (page) && - !isolated_balloon_page (page)) { + !PageBalloon (page)) {ClearPageActive (page);list_move (&page->lru, &clean_pages); }
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
In reply to Konstantin KhlebnikovQuoted 786 lines · Konstantin Khlebnikov
From: Konstantin Khlebnikov <[email protected]>
* move special branch for balloon migraion into
migrate_pages
* remove special mapping for balloon and its flagAS_BALLOON_MAP
* embed structballoon_dev_infointo structvirtio_balloon
* cleanupballoon_page_dequeue, killballoon_page_freeSigned-off-by Konstantin Khlebnikov <[email protected]>--- drivers/virtio/virtio_balloon.c | 77 ++++--------- include/linux/balloon_compaction.h | 127 ++++++--------------- include/linux/migrate.h | 11 -- include/linux/pagemap.h | 18 --- mm/balloon_compaction.c | 214 ++++++++++++------------------------ mm/migrate.c | 29 +---- 6 files changed, 134 insertions(+), 342 deletions(-)drivers/virtio/virtio_balloon.c +21
-56@@ -59,7 +59,7 @@ struct virtio_balloon* Each page on this list adds VIRTIO_BALLOON_PAGES_PER_PAGE * to num_pages above. */ -struct balloon_dev_info *vb_dev_info; +struct balloon_dev_info vb_dev_info;/* Synchronize access/update to this struct virtio_balloon elements */ struct mutex balloon_lock;@@ -127,7 +127,7 @@ static void set_page_pfns(u32 pfns[], struct page *page)static void fill_balloon (struct virtio_balloon *vb,size_t num) { -struct balloon_dev_info *vb_dev_info = vb->vb_dev_info; +struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;/* We can only do one array worth at a time. */ num =min (num,ARRAY_SIZE (vb->pfns));@@ -163,15 +163,15 @@ static void release_pages_by_pfn(const u32 pfns[], unsigned int num)/* Find pfns pointing at start of each page, get pages and free them. */ for (i =0 ; i < num; i +=VIRTIO_BALLOON_PAGES_PER_PAGE ) {struct page *page =balloon_pfn_to_page (pfns[i]); -balloon_page_free (page);adjust_managed_page_count (page,1 ); +put_page (page); } }static void leak_balloon (struct virtio_balloon *vb,size_t num) {struct page *page; -struct balloon_dev_info *vb_dev_info = vb->vb_dev_info; +struct balloon_dev_info *vb_dev_info = &vb->vb_dev_info;/* We can only do one array worth at a time. */ num =min (num,ARRAY_SIZE (vb->pfns));@@ -353,12 +353,11 @@ static int init_vqs(struct virtio_balloon *vb)return 0 ; } -static const struct address_space_operations virtio_balloon_aops;#ifdef CONFIG_BALLOON_COMPACTION /* * virtballoon_migratepage - perform the balloon page migration on behalf of * a compation thread. (called under page lock) -* @mapping: the page->mapping which will be assigned to the new migrated page. +* @vb_dev_info: the balloon device * @newpage: page that will replace the isolated page after migration finishes. * @page : the isolated (old) page that is about to be migrated to newpage. * @mode : compaction mode -- not used for balloon page migration. @@ -373,17 +372,13 @@ static const struct address_space_operations virtio_balloon_aops;* This function preforms the balloon page migration task. * Called through balloon_mapping->a_ops->migratepage */ -static int virtballoon_migratepage (struct address_space *mapping, +static int virtballoon_migratepage (struct balloon_dev_info *vb_dev_info,struct page *newpage,struct page *page,enum migrate_mode mode) { -struct balloon_dev_info *vb_dev_info =balloon_page_device (page); -struct virtio_balloon *vb; +struct virtio_balloon *vb =container_of (vb_dev_info, + struct virtio_balloon, vb_dev_info);unsigned long flags; -BUG_ON (!vb_dev_info); - - vb = vb_dev_info->balloon_device; -/* * In order to avoid lock contention while migrating pages concurrently * to leak_balloon() or fill_balloon() we just give up the balloon_lock @@ -395,42 +390,34 @@ static int virtballoon_migratepage(struct address_space *mapping,if (!mutex_trylock (&vb->balloon_lock))return -EAGAIN ; +get_page (newpage);/* balloon reference */ +/* balloon's page migration 1st step -- inflate "newpage" */ spin_lock_irqsave (&vb_dev_info->pages_lock, flags); -balloon_page_insert (newpage, mapping, &vb_dev_info->pages); +balloon_page_insert (vb_dev_info, newpage); vb_dev_info->isolated_pages--;spin_unlock_irqrestore (&vb_dev_info->pages_lock, flags); vb->num_pfns =VIRTIO_BALLOON_PAGES_PER_PAGE ;set_page_pfns (vb->pfns, newpage);tell_host (vb, vb->inflate_vq); -/* -* balloon's page migration 2nd step -- deflate "page" -* -* It's safe to delete page->lru here because this page is at -* an isolated migration list, and this step is expected to happen here -*/ -balloon_page_delete (page); +/* balloon's page migration 2nd step -- deflate "page" */ +balloon_page_delete (page, true); vb->num_pfns =VIRTIO_BALLOON_PAGES_PER_PAGE ;set_page_pfns (vb->pfns, page);tell_host (vb, vb->deflate_vq);mutex_unlock (&vb->balloon_lock); - return MIGRATEPAGE_BALLOON_SUCCESS; -} +put_page (page);/* balloon reference */ -/* define the balloon_mapping->a_ops callback to allow balloon page migration */ -static const struct address_space_operations virtio_balloon_aops = { - .migratepage = virtballoon_migratepage, -}; + return MIGRATEPAGE_SUCCESS; +}#endif /* CONFIG_BALLOON_COMPACTION */ static int virtballoon_probe (struct virtio_device *vdev) {struct virtio_balloon *vb; -struct address_space *vb_mapping; -struct balloon_dev_info *vb_devinfo;int err; vdev->priv = vb =kmalloc (sizeof (*vb),GFP_KERNEL );@@ -446,30 +433,14 @@ static int virtballoon_probe(struct virtio_device *vdev) vb->vdev = vdev; vb->need_stats_update =0 ; - vb_devinfo =balloon_devinfo_alloc (vb); -if (IS_ERR (vb_devinfo)) { - err =PTR_ERR (vb_devinfo); - goto out_free_vb; - } - - vb_mapping =balloon_mapping_alloc (vb_devinfo, - (balloon_compaction_check ()) ? - &virtio_balloon_aops :NULL ); -if (IS_ERR (vb_mapping)) { -/* -* IS_ERR(vb_mapping) && PTR_ERR(vb_mapping) == -EOPNOTSUPP -* This means !CONFIG_BALLOON_COMPACTION, otherwise we get off. -*/ - err =PTR_ERR (vb_mapping); -if (err != -EOPNOTSUPP ) - goto out_free_vb_devinfo; - } - - vb->vb_dev_info = vb_devinfo; +balloon_devinfo_init (&vb->vb_dev_info); +#ifdef CONFIG_BALLOON_COMPACTION + vb->vb_dev_info.migratepage = virtballoon_migratepage; +#endif err =init_vqs (vb);if (err) - goto out_free_vb_mapping; + goto out_free_vb; vb->thread =kthread_run (balloon, vb,"vballoon" );if (IS_ERR (vb->thread)) {@@ -481,10 +452,6 @@ static int virtballoon_probe(struct virtio_device *vdev) out_del_vqs: vdev->config->del_vqs (vdev); -out_free_vb_mapping: -balloon_mapping_free (vb_mapping); -out_free_vb_devinfo: -balloon_devinfo_free (vb_devinfo); out_free_vb:kfree (vb); out:@@ -510,8 +477,6 @@ static void virtballoon_remove(struct virtio_device *vdev)kthread_stop (vb->thread);remove_common (vb); -balloon_mapping_free (vb->vb_dev_info->mapping); -balloon_devinfo_free (vb->vb_dev_info);kfree (vb); }include/linux/balloon_compaction.h +38
-89@@ -56,96 +56,58 @@* balloon driver as a page book-keeper for its registered balloon devices. */ struct balloon_dev_info { -void *balloon_device;/* balloon device descriptor */ -struct address_space *mapping;/* balloon special page->mapping */ unsigned long isolated_pages;/* # of isolated pages for migration */ spinlock_t pages_lock;/* Protection to pages list */ struct list_head pages;/* Pages enqueued & handled to Host */ +int (*migratepage)(struct balloon_dev_info *,struct page *newpage, +struct page *page,enum migrate_mode mode); }; -extern struct page *balloon_page_enqueue (struct balloon_dev_info *b_dev_info); -extern struct page *balloon_page_dequeue (struct balloon_dev_info *b_dev_info); -extern struct balloon_dev_info *balloon_devinfo_alloc ( -void *balloon_dev_descriptor); - -static inline void balloon_devinfo_free (struct balloon_dev_info *b_dev_info) +static inline void balloon_devinfo_init (struct balloon_dev_info *b_dev_info) { -kfree (b_dev_info); + b_dev_info->isolated_pages =0 ; +spin_lock_init (&b_dev_info->pages_lock); +INIT_LIST_HEAD (&b_dev_info->pages); + b_dev_info->migratepage =NULL ; } -/* -* balloon_page_free - release a balloon page back to the page free lists -* @page: ballooned page to be set free -* -* This function must be used to properly set free an isolated/dequeued balloon -* page at the end of a sucessful page migration, or at the balloon driver's -* page release procedure. -*/ -static inline void balloon_page_free (struct page *page) -{ -/* -* Balloon pages always get an extra refcount before being isolated -* and before being dequeued to help on sorting out fortuite colisions -* between a thread attempting to isolate and another thread attempting -* to release the very same balloon page. -* -* Before we handle the page back to Buddy, lets drop its extra refcnt. -*/ -put_page (page); -__free_page (page); -} +extern struct page *balloon_page_enqueue (struct balloon_dev_info *b_dev_info); +extern struct page *balloon_page_dequeue (struct balloon_dev_info *b_dev_info);/* -* balloon_page_insert - insert a page into the balloon's page list and make -* the page->mapping assignment accordingly. -* @page : page to be assigned as a 'balloon page' -* @mapping : allocated special 'balloon_mapping' -* @head : balloon's device page list head +* balloon_page_insert - insert a page into the balloon's page list, +* mark and account it accordingly. +* @b_dev_info : pinter to ballon device +* @page : page to be assigned as a 'balloon page' * * Caller must ensure the page is locked and the spin_lock protecting balloon * pages list is held before inserting a page into the balloon device. */ -static inline void balloon_page_insert (struct page *page, -struct address_space *mapping, -struct list_head *head) +static inline void +balloon_page_insert (struct balloon_dev_info *b_dev_info,struct page *page) {__SetPageBalloon (page);inc_zone_page_state (page,NR_BALLOON_PAGES ); - page->mapping = mapping; -list_add (&page->lru, head); +set_page_private (page, (unsigned long )b_dev_info); +list_add (&page->lru, &b_dev_info->pages); }/* * balloon_page_delete - delete a page from balloon's page list and clear -* the page->mapping assignement accordingly. +* the ballon page mark accordingly. * @page : page to be released from balloon's page list +* @isolated: already isolated, do not delete from list * * Caller must ensure the page is locked and the spin_lock protecting balloon * pages list is held before deleting a page from the balloon device. */ -static inline void balloon_page_delete (struct page *page) +static inline void balloon_page_delete (struct page *page,bool isolated) {__ClearPageBalloon (page);dec_zone_page_state (page,NR_BALLOON_PAGES ); - page->mapping =NULL ; -list_del (&page->lru); -} - -#endif /* CONFIG_MEMORY_BALLOON */ - -#ifdef CONFIG_BALLOON_COMPACTION - -extern bool balloon_page_isolate (struct page *page); -extern void balloon_page_putback (struct page *page); -extern int balloon_page_migrate (struct page *newpage, -struct page *page,enum migrate_mode mode); -extern struct address_space -*balloon_mapping_alloc (struct balloon_dev_info *b_dev_info, -const struct address_space_operations *a_ops); - -static inline void balloon_mapping_free (struct address_space *balloon_mapping) -{ -kfree (balloon_mapping); +set_page_private (page,0 ); +if (!isolated) +list_del (&page->lru); }/* @@ -154,35 +116,25 @@ static inline void balloon_mapping_free(struct address_space *balloon_mapping)*/ static inline struct balloon_dev_info *balloon_page_device (struct page *page) { -struct address_space *mapping = page->mapping; -if (likely (mapping)) -return mapping->private_data; - -return NULL ; +return (struct balloon_dev_info *)page_private (page); } -static inline gfp_t balloon_mapping_gfp_mask (void ) -{ -return GFP_HIGHUSER_MOVABLE ; -} +#endif /* CONFIG_MEMORY_BALLOON */ -static inline bool balloon_compaction_check (void ) -{ -return true; -} +#ifdef CONFIG_BALLOON_COMPACTION +extern bool balloon_page_isolate (struct page *page); +extern void balloon_page_putback (struct page *page); -#else /* !CONFIG_BALLOON_COMPACTION */ +int balloon_page_migrate (new_page_t get_new_page,free_page_t put_new_page, +unsigned long private,struct page *page, +int force,enum migrate_mode mode); -static inline void *balloon_mapping_alloc (void *balloon_device, -const struct address_space_operations *a_ops) +static inline gfp_t balloon_mapping_gfp_mask (void ) { -return ERR_PTR (-EOPNOTSUPP ); +return GFP_HIGHUSER_MOVABLE ; } -static inline void balloon_mapping_free (struct address_space *balloon_mapping) -{ -return ; -} +#else /* !CONFIG_BALLOON_COMPACTION */ static inline bool balloon_page_isolate (struct page *page) {@@ -194,10 +146,11 @@ static inline void balloon_page_putback(struct page *page)return ; } -static inline int balloon_page_migrate (struct page *newpage, -struct page *page,enum migrate_mode mode) +static inline int balloon_page_migrate (new_page_t get_new_page, +free_page_t put_new_page,unsigned long private, +struct page *page,int force,enum migrate_mode mode) { -return 0 ; +return -EAGAIN ; }static inline gfp_t balloon_mapping_gfp_mask (void )@@ -205,9 +158,5 @@ static inline gfp_t balloon_mapping_gfp_mask(void)return GFP_HIGHUSER ; } -static inline bool balloon_compaction_check (void ) -{ -return false; -}#endif /* CONFIG_BALLOON_COMPACTION */ #endif /* _LINUX_BALLOON_COMPACTION_H */ include/linux/migrate.h +1
-10@@ -13,18 +13,9 @@ typedef void free_page_t(struct page *page, unsigned long private);* Return values from addresss_space_operations.migratepage(): * - negative errno on page migration failure; * - zero on page migration success; -* -* The balloon page migration introduces this special case where a 'distinct' -* return code is used to flag a successful page migration to unmap_and_move(). -* This approach is necessary because page migration can race against balloon -* deflation procedure, and for such case we could introduce a nasty page leak -* if a successfully migrated balloon page gets released concurrently with -* migration's unmap_and_move() wrap-up steps. */ #define MIGRATEPAGE_SUCCESS 0 -#define MIGRATEPAGE_BALLOON_SUCCESS 1/* special ret code for balloon page -* sucessful migration case. -*/ +enum migrate_reason {MR_COMPACTION ,MR_MEMORY_FAILURE ,include/linux/pagemap.h +1
-17@@ -24,8 +24,7 @@ enum mapping_flags {AS_ENOSPC = __GFP_BITS_SHIFT +1 ,/* ENOSPC on async write */ AS_MM_ALL_LOCKS = __GFP_BITS_SHIFT +2 ,/* under mm_take_all_locks() */ AS_UNEVICTABLE = __GFP_BITS_SHIFT +3 ,/* e.g., ramdisk, SHM_LOCK */ -AS_BALLOON_MAP = __GFP_BITS_SHIFT +4 ,/* balloon page special map */ - AS_EXITING = __GFP_BITS_SHIFT + 5, /* final truncate in progress */ + AS_EXITING = __GFP_BITS_SHIFT + 4, /* final truncate in progress */ };static inline void mapping_set_error (struct address_space *mapping,int error)@@ -55,21 +54,6 @@ static inline int mapping_unevictable(struct address_space *mapping)return !!mapping; } -static inline void mapping_set_balloon (struct address_space *mapping) -{ -set_bit (AS_BALLOON_MAP , &mapping->flags); -} - -static inline void mapping_clear_balloon (struct address_space *mapping) -{ -clear_bit (AS_BALLOON_MAP , &mapping->flags); -} - -static inline int mapping_balloon (struct address_space *mapping) -{ -return mapping &&test_bit (AS_BALLOON_MAP , &mapping->flags); -} -static inline void mapping_set_exiting (struct address_space *mapping) {set_bit (AS_EXITING , &mapping->flags);mm/balloon_compaction.c +69
-145@@ -11,32 +11,6 @@#include <linux/balloon_compaction.h> /* -* balloon_devinfo_alloc - allocates a balloon device information descriptor. -* @balloon_dev_descriptor: pointer to reference the balloon device which -* this struct balloon_dev_info will be servicing. -* -* Driver must call it to properly allocate and initialize an instance of -* struct balloon_dev_info which will be used to reference a balloon device -* as well as to keep track of the balloon device page list. -*/ -struct balloon_dev_info *balloon_devinfo_alloc (void *balloon_dev_descriptor) -{ -struct balloon_dev_info *b_dev_info; - b_dev_info =kmalloc (sizeof (*b_dev_info),GFP_KERNEL ); -if (!b_dev_info) -return ERR_PTR (-ENOMEM ); - - b_dev_info->balloon_device = balloon_dev_descriptor; - b_dev_info->mapping =NULL ; - b_dev_info->isolated_pages =0 ; -spin_lock_init (&b_dev_info->pages_lock); -INIT_LIST_HEAD (&b_dev_info->pages); - -return b_dev_info; -} -EXPORT_SYMBOL_GPL (balloon_devinfo_alloc); - -/* * balloon_page_enqueue - allocates a new page and inserts it into the balloon * page list. * @b_dev_info: balloon device decriptor where we will insert a new page to @@ -61,7 +35,7 @@ struct page *balloon_page_enqueue(struct balloon_dev_info *b_dev_info)*/ BUG_ON (!trylock_page (page));spin_lock_irqsave (&b_dev_info->pages_lock, flags); -balloon_page_insert (page, b_dev_info->mapping, &b_dev_info->pages); +balloon_page_insert (b_dev_info, page);spin_unlock_irqrestore (&b_dev_info->pages_lock, flags);unlock_page (page);return page;@@ -81,12 +55,10 @@ EXPORT_SYMBOL_GPL(balloon_page_enqueue);*/ struct page *balloon_page_dequeue (struct balloon_dev_info *b_dev_info) { -struct page *page, *tmp; +struct page *page;unsigned long flags; -bool dequeued_page; - dequeued_page = false; -list_for_each_entry_safe (page, tmp, &b_dev_info->pages, lru) { +list_for_each_entry (page, &b_dev_info->pages, lru) {/* * Block others from accessing the 'page' while we get around * establishing additional references and preparing the 'page' @@ -94,98 +66,32 @@ struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)*/ if (trylock_page (page)) {spin_lock_irqsave (&b_dev_info->pages_lock, flags); -/* -* Raise the page refcount here to prevent any wrong -* attempt to isolate this page, in case of coliding -* with balloon_page_isolate() just after we release -* the page lock. -* -* balloon_page_free() will take care of dropping -* this extra refcount later. -*/ -get_page (page); -balloon_page_delete (page); +balloon_page_delete (page, false);spin_unlock_irqrestore (&b_dev_info->pages_lock, flags);unlock_page (page); - dequeued_page = true; -break ; +return page; } } -if (!dequeued_page) { -/* -* If we are unable to dequeue a balloon page because the page -* list is empty and there is no isolated pages, then something -* went out of track and some balloon pages are lost. -* BUG() here, otherwise the balloon driver may get stuck into -* an infinite loop while attempting to release all its pages. -*/ -spin_lock_irqsave (&b_dev_info->pages_lock, flags); -if (unlikely (list_empty (&b_dev_info->pages) && - !b_dev_info->isolated_pages)) -BUG (); -spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); - page =NULL ; - } -return page; +/* +* If we are unable to dequeue a balloon page because the page +* list is empty and there is no isolated pages, then something +* went out of track and some balloon pages are lost. +* BUG() here, otherwise the balloon driver may get stuck into +* an infinite loop while attempting to release all its pages. +*/ +spin_lock_irqsave (&b_dev_info->pages_lock, flags); +BUG_ON (list_empty (&b_dev_info->pages) && !b_dev_info->isolated_pages); +spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); +return NULL ; }EXPORT_SYMBOL_GPL (balloon_page_dequeue);#ifdef CONFIG_BALLOON_COMPACTION -/* -* balloon_mapping_alloc - allocates a special ->mapping for ballooned pages. -* @b_dev_info: holds the balloon device information descriptor. -* @a_ops: balloon_mapping address_space_operations descriptor. -* -* Driver must call it to properly allocate and initialize an instance of -* struct address_space which will be used as the special page->mapping for -* balloon device enlisted page instances. -*/ -struct address_space *balloon_mapping_alloc (struct balloon_dev_info *b_dev_info, -const struct address_space_operations *a_ops) -{ -struct address_space *mapping; - - mapping =kmalloc (sizeof (*mapping),GFP_KERNEL ); -if (!mapping) -return ERR_PTR (-ENOMEM ); - -/* -* Give a clean 'zeroed' status to all elements of this special -* balloon page->mapping struct address_space instance. -*/ -address_space_init_once (mapping); - -/* -* Set mapping->flags appropriately, to allow balloon pages -* ->mapping identification. -*/ -mapping_set_balloon (mapping); -mapping_set_gfp_mask (mapping,balloon_mapping_gfp_mask ()); - -/* balloon's page->mapping->a_ops callback descriptor */ - mapping->a_ops = a_ops; - -/* -* Establish a pointer reference back to the balloon device descriptor -* this particular page->mapping will be servicing. -* This is used by compaction / migration procedures to identify and -* access the balloon device pageset while isolating / migrating pages. -* -* As some balloon drivers can register multiple balloon devices -* for a single guest, this also helps compaction / migration to -* properly deal with multiple balloon pagesets, when required. -*/ - mapping->private_data = b_dev_info; - b_dev_info->mapping = mapping; - -return mapping; -} -EXPORT_SYMBOL_GPL (balloon_mapping_alloc);static inline void __isolate_balloon_page (struct page *page) { -struct balloon_dev_info *b_dev_info = page->mapping->private_data; +struct balloon_dev_info *b_dev_info =balloon_page_device (page);unsigned long flags;spin_lock_irqsave (&b_dev_info->pages_lock, flags);list_del (&page->lru);@@ -195,7 +101,7 @@ static inline void __isolate_balloon_page(struct page *page)static inline void __putback_balloon_page (struct page *page) { -struct balloon_dev_info *b_dev_info = page->mapping->private_data; +struct balloon_dev_info *b_dev_info =balloon_page_device (page);unsigned long flags;spin_lock_irqsave (&b_dev_info->pages_lock, flags);list_add (&page->lru, &b_dev_info->pages);@@ -203,12 +109,6 @@ static inline void __putback_balloon_page(struct page *page)spin_unlock_irqrestore (&b_dev_info->pages_lock, flags); } -static inline int __migrate_balloon_page (struct address_space *mapping, -struct page *newpage,struct page *page,enum migrate_mode mode) -{ -return page->mapping->a_ops->migratepage (mapping, newpage, page, mode); -} -/* __isolate_lru_page() counterpart for a ballooned page */ bool balloon_page_isolate (struct page *page) {@@ -251,6 +151,57 @@ bool balloon_page_isolate(struct page *page)return false; } +int balloon_page_migrate (new_page_t get_new_page,free_page_t put_new_page, +unsigned long private,struct page *page, +int force,enum migrate_mode mode) +{ +struct balloon_dev_info *balloon =balloon_page_device (page); +struct page *newpage; +int *result =NULL ; +int rc = -EAGAIN ; + +if (!balloon || !balloon->migratepage) +return -EAGAIN ; + + newpage =get_new_page (page, private, &result); +if (!newpage) +return -ENOMEM ; + +if (!trylock_page (newpage)) +BUG (); + +if (!trylock_page (page)) { +if (!force || mode !=MIGRATE_SYNC ) + goto out; +lock_page (page); + } + + rc = balloon->migratepage (balloon, newpage, page, mode); + +unlock_page (page); +out: +unlock_page (newpage); + +if (rc != -EAGAIN ) { +dec_zone_page_state (page,NR_ISOLATED_FILE ); +list_del (&page->lru); +put_page (page); + } + +if (rc !=MIGRATEPAGE_SUCCESS && put_new_page) +put_new_page (newpage, private); +else +put_page (newpage); + +if (result) { +if (rc) + *result = rc; +else + *result =page_to_nid (newpage); + } +return rc; +} +/* putback_lru_page() counterpart for a ballooned page */ void balloon_page_putback (struct page *page) {@@ -271,31 +222,4 @@ void balloon_page_putback(struct page *page) unlock_page(page); } -/* move_to_new_page() counterpart for a ballooned page */ -int balloon_page_migrate (struct page *newpage, -struct page *page,enum migrate_mode mode) -{ -struct address_space *mapping; -int rc = -EAGAIN ; - -/* -* Block others from accessing the 'newpage' when we get around to -* establishing additional references. We should be the only one -* holding a reference to the 'newpage' at this point. -*/ -BUG_ON (!trylock_page (newpage)); - -if (WARN_ON (!PageBalloon (page))) { -dump_page (page,"not movable balloon page" ); -unlock_page (newpage); -return rc; - } - - mapping = page->mapping; -if (mapping) - rc =__migrate_balloon_page (mapping, newpage, page, mode); - -unlock_page (newpage); -return rc; -}#endif /* CONFIG_BALLOON_COMPACTION */ mm/migrate.c +0
-25@@ -873,18 +873,6 @@ static int __unmap_and_move(struct page *page, struct page *newpage, } } -if (unlikely (PageBalloon (page))) { -/* -* A ballooned page does not need any special attention from -* physical to virtual reverse mapping procedures. -* Skip any attempt to unmap PTEs or to remap swap cache, -* in order to avoid burning cycles at rmap level, and perform -* the page migration right away (proteced by page lock). -*/ - rc =balloon_page_migrate (newpage, page, mode); - goto out_unlock; - } -/* * Corner case handling: * 1. When a new swap-cache page is read into, it is added to the LRU @@ -952,19 +940,6 @@ static int unmap_and_move(new_page_t get_new_page, free_page_t put_new_page, rc =__unmap_and_move (page, newpage, force, mode); -#ifdef CONFIG_MEMORY_BALLOON -if (unlikely (rc ==MIGRATEPAGE_BALLOON_SUCCESS )) { -/* -* A ballooned page has been migrated already. -* Now, it's the time to wrap-up counters, -* handle the page back to Buddy and return. -*/ -dec_zone_page_state (page,NR_ISOLATED_ANON + -page_is_file_cache (page)); -balloon_page_free (page); -return MIGRATEPAGE_SUCCESS ; - } -#endif out:if (rc != -EAGAIN ) {/*
mm/migrate.c int migrate_pages(struct list_head *from, new_page_t get_new_page, · line 1123
Quoted 10 lines
mm/migrate.c +4
-0@@ -1139,6 +1114,10 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, rc =unmap_and_move_huge_page (get_new_page, put_new_page, private, page, pass >2 , mode); + else if (PageBalloon(page)) + rc =balloon_page_migrate (get_new_page, + put_new_page, private, + page, pass >2 , mode); else rc =unmap_and_move (get_new_page, put_new_page, private, page, pass >2 , mode);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerRe: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
In reply to Konstantin KhlebnikovQuoted 11 lines · On Sat, 30 Aug 2014 20:41:20 +0400 Konstantin Khlebnikov <[email protected]>…
This patch adds page state
PageBallon()and functions __Set/ClearPageBalloon. LikePageBuddy()PageBalloon()looks like page-flag but actually this is special state ofpage->_mapcountcounter. There is no conflict because ballooned pages cannot be mapped and cannot be in buddy allocator.Ballooned pages are counted in vmstat counter
NR_BALLOON_PAGES, it's shown them in/proc/meminfoand/proc/meminfo. Also this patch it exports PageBallon into userspace via/proc/kpageflagsasKPF_BALLOON.All new code is under
CONFIG_MEMORY_BALLOON, it should be selected by ballooning driver which wants use this feature.
The delta from the (fixed) v1 is below.
What's up with those Kconfig/Makefile changes? We're now including a pile of balloon code into vmlinux when CONFIG_MEMORY_BALLOON=n? These changes were not changelogged?
Did we really need to put the BalloonPages count into per-zone vmstat, global vmstat and /proc/meminfo? Seems a bit overkillish - why so important?
Consuming another page flag is a big deal. We keep on nearly running out and one day we'll run out for real. page-flags-layout.h is incomprehensible. How many flags do we have left (worst-case) with this change? Is there no other way? Needs extraordinary justification, please.
drivers/virtio/Kconfig | 1 - include/linux/mm.h | 14 ++++++++++++-- mm/Makefile | 3 +-- mm/balloon_compaction.c | 16 ---------------- 4 files changed, 13 insertions(+), 21 deletions(-)
drivers/virtio/Kconfig renamed from drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 +0 -1
@@ -25,7 +25,6 @@ config VIRTIO_PCI config VIRTIO_BALLOON tristate "Virtio balloon driver" depends on VIRTIO - select MEMORY_BALLOON ---help--- This driver supports increasing and decreasing the amount of memory within a KVM guest.
include/linux/mm.h renamed from include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 +12 -2
@@ -561,8 +561,18 @@ static inline int PageBalloon(struct pagreturn IS_ENABLED (CONFIG_MEMORY_BALLOON ) &&atomic_read (&page->_mapcount) ==PAGE_BALLOON_MAPCOUNT_VALUE ; } -void __SetPageBalloon (struct page *page); -void __ClearPageBalloon (struct page *page); + +static inline void __SetPageBalloon (struct page *page) +{ +VM_BUG_ON_PAGE (atomic_read (&page->_mapcount) !=-1 , page); +atomic_set (&page->_mapcount,PAGE_BALLOON_MAPCOUNT_VALUE ); +} + +static inline void __ClearPageBalloon (struct page *page) +{ +VM_BUG_ON_PAGE (!PageBalloon (page), page); +atomic_set (&page->_mapcount,-1 ); +}void put_page (struct page *page);void put_pages_list (struct list_head *pages);
mm/Makefile renamed from mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 +1 -2
@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill. readahead.o swap.o truncate.o vmscan.o shmem.o \ util.o mmzone.o vmstat.o backing-dev.o \ mm_init.o mmu_context.o percpu.o slab_common.o \ - compaction.o vmacache.o \ + compaction.o balloon_compaction.o vmacache.o \ interval_tree.o list_lru.o workingset.o \ iov_iter.o $(mmu-y)
@@ -64,4 +64,3 @@ obj-$(CONFIG_ZBUD) += zbud.o obj-$(CONFIG_ZSMALLOC) += zsmalloc.o obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o obj-$(CONFIG_CMA) += cma.o -obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.o
mm/balloon_compaction.c renamed from mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 +0 -16
@@ -10,22 +10,6 @@#include <linux/export.h> #include <linux/balloon_compaction.h> -void __SetPageBalloon (struct page *page) -{ -VM_BUG_ON_PAGE (atomic_read (&page->_mapcount) !=-1 , page); -atomic_set (&page->_mapcount,PAGE_BALLOON_MAPCOUNT_VALUE ); -inc_zone_page_state (page,NR_BALLOON_PAGES ); -} -EXPORT_SYMBOL (__SetPageBalloon); - -void __ClearPageBalloon (struct page *page) -{ -VM_BUG_ON_PAGE (!PageBalloon (page), page); -atomic_set (&page->_mapcount,-1 ); -dec_zone_page_state (page,NR_BALLOON_PAGES ); -} -EXPORT_SYMBOL (__ClearPageBalloon); -/* * balloon_devinfo_alloc - allocates a balloon device information descriptor. * @balloon_dev_descriptor: pointer to reference the balloon device which
_
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerRe: [PATCH v2 5/6] mm/balloon_compaction: use common page ballooning
In reply to Konstantin KhlebnikovQuoted 3 lines · On Sat, 30 Aug 2014 20:41:23 +0400 Konstantin Khlebnikov <[email protected]>…
This patch replaces checking
AS_BALLOON_MAPinpage->mapping->flagswith PageBalloon which is stored directly in the struct page. All code ofballoon_compactionnow underCONFIG_MEMORY_BALLOON.
argh, now the bogus Kconfig/Makefile changes get removed again.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerRe: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
In reply to Konstantin KhlebnikovQuoted 4 lines · On Sat, 30 Aug 2014 20:41:27 +0400 Konstantin Khlebnikov <[email protected]>…
* move special branch for balloon migraion into
migrate_pages
* remove special mapping for balloon and its flagAS_BALLOON_MAP
* embed structballoon_dev_infointo structvirtio_balloon
* cleanupballoon_page_dequeue, killballoon_page_free
Not sure what's going on here - your include/linux/balloon_compaction.h seems significantly different from mine.
I think I'll just drop this patch - it's quite inconvenient to have a large "general cleanup" coming after a stack of significant functional changes. It makes review, debug, fix, merge and reversion harder. Let's worry about it later.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerRe: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
Quoted 9 lines · On Fri, 12 Sep 2014 17:04:04 -0700 Andrew Morton <[email protected]>…
Quoted 4 lines · On Sat, 30 Aug 2014 20:41:27 +0400 Konstantin Khlebnikov <[email protected]>…
* move special branch for balloon migraion into
migrate_pages
* remove special mapping for balloon and its flagAS_BALLOON_MAP
* embed structballoon_dev_infointo structvirtio_balloon
* cleanupballoon_page_dequeue, killballoon_page_freeNot sure what's going on here - your
include/linux/balloon_compaction.hseems significantly different from mine.
OK, I worked it out.
Quoted 4 lines
I think I'll just drop this patch - it's quite inconvenient to have a large "general cleanup" coming after a stack of significant functional changes. It makes review, debug, fix, merge and reversion harder. Let's worry about it later.
But I'm still thinking we should defer this one?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerQuoted 19 lines · On Sat, 30 Aug 2014 20:41:06 +0400 Konstantin Khlebnikov <[email protected]>…
I've checked compilation of linux-next/x86 for allnoconfig, defconfig and defconfig + kvmconfig + virtio-balloon with and without balloon-compaction. For stable kernels first three patches should be enough.
changes since v1:
mm/balloon_compaction: ignore anonymous pages
* no changes
mm/balloon_compaction: keep ballooned pages away from normal migration path
* fix compilation withoutCONFIG_BALLOON_COMPACTION
mm/balloon_compaction: isolate balloon pages withoutlru_lock
* no changesmm: introduce common page state for ballooned memory
* move __Set/ClearPageBalloon intolinux/mm.h
* remove inc/dec_zone_page_state from __Set/ClearPageBalloon
Quoted 7 lines
mm/balloon_compaction: use common page ballooning
* call inc/dec_zone_page_state from balloon_page_insert/delete
mm/balloon_compaction: general cleanup
* fix compilation withoutCONFIG_MIGRATION
* fix compilation withoutCONFIG_BALLOON_COMPACTION
The patch "selftests/vm/transhuge-stress: stress test for memory compaction" has silently and mysteriously vanished?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 24 lines · Andrew Morton
Quoted 22 lines · On Sat, 30 Aug 2014 20:41:06 +0400 Konstantin Khlebnikov <[email protected]>…
I've checked compilation of linux-next/x86 for allnoconfig, defconfig and defconfig + kvmconfig + virtio-balloon with and without balloon-compaction. For stable kernels first three patches should be enough.
changes since v1:
mm/balloon_compaction: ignore anonymous pages
* no changes
mm/balloon_compaction: keep ballooned pages away from normal migration path
* fix compilation withoutCONFIG_BALLOON_COMPACTION
mm/balloon_compaction: isolate balloon pages withoutlru_lock
* no changesmm: introduce common page state for ballooned memory
* move __Set/ClearPageBalloon intolinux/mm.h
* remove inc/dec_zone_page_state from __Set/ClearPageBalloon
mm/balloon_compaction: use common page ballooning
* call inc/dec_zone_page_state from balloon_page_insert/delete
Quoted 8 lines
Quoted 4 lines
mm/balloon_compaction: general cleanup
* fix compilation withoutCONFIG_MIGRATION
* fix compilation withoutCONFIG_BALLOON_COMPACTIONThe patch "selftests/vm/transhuge-stress: stress test for memory compaction" has silently and mysteriously vanished?
It's unchanged and has no direct connection to this patchset. So I've dropped it. If you like it you can keep the v1 version.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
In reply to Andrew MortonQuoted 11 lines · Andrew Morton
Quoted 9 lines · On Sat, 30 Aug 2014 20:41:20 +0400 Konstantin Khlebnikov <[email protected]>…
This patch adds page state
PageBallon()and functions __Set/ClearPageBalloon. LikePageBuddy()PageBalloon()looks like page-flag but actually this is special state ofpage->_mapcountcounter. There is no conflict because ballooned pages cannot be mapped and cannot be in buddy allocator.Ballooned pages are counted in vmstat counter
NR_BALLOON_PAGES, it's shown them in/proc/meminfoand/proc/meminfo. Also this patch it exports PageBallon into userspace via/proc/kpageflagsasKPF_BALLOON.
Quoted 8 lines
Quoted 2 lines
All new code is under
CONFIG_MEMORY_BALLOON, it should be selected by ballooning driver which wants use this feature.The delta from the (fixed) v1 is below.
What's up with those Kconfig/Makefile changes? We're now including a pile of balloon code into vmlinux when
CONFIG_MEMORY_BALLOON=n? These changes were not changelogged?
It's been moved into next patch, and not mentioned in changes. Sorry.
Quoted 4 lines
Did we really need to put the BalloonPages count into per-zone vmstat, global vmstat and
/proc/meminfo? Seems a bit overkillish - why so important?
Balloon grabs random pages, their distribution among numa nodes might be important.
But I know nobody who uses numa-aware vm together with ballooning.
Probably it's better to drop per-zone vmstat and line from meminfo, global vmstat counter should be enough.
Quoted 6 lines
Consuming another page flag is a big deal. We keep on nearly running out and one day we'll run out for real.
page-flags-layout.his incomprehensible. How many flags do we have left (worst-case) with this change? Is there no other way? Needs extraordinary justification, please.
PageBalloon is not a page flags, it's like PageBuddy -- special state of _mapcount (-256 in this case).
The same was in v1 and is written in the comment above.
Quoted 85 lines
drivers/virtio/Kconfig | 1 - include/linux/mm.h | 14 ++++++++++++-- mm/Makefile | 3 +-- mm/balloon_compaction.c | 16 ---------------- 4 files changed, 13 insertions(+), 21 deletions(-)diff -puN
drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2drivers/virtio/Kconfigdrivers/virtio/Kconfig renamed from drivers/virtio/Kconfig~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 +0
-1@@ -25,7 +25,6 @@ config VIRTIO_PCI config VIRTIO_BALLOON tristate "Virtio balloon driver" depends on VIRTIO - select MEMORY_BALLOON ---help--- This driver supports increasing and decreasing the amount of memory within a KVM guest.diff -puN
include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2include/linux/mm.hinclude/linux/mm.h renamed from include/linux/mm.h~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 +12
-2@@ -561,8 +561,18 @@ static inline int PageBalloon(struct pagreturn IS_ENABLED (CONFIG_MEMORY_BALLOON ) &&atomic_read (&page->_mapcount) ==PAGE_BALLOON_MAPCOUNT_VALUE ; } -void __SetPageBalloon (struct page *page); -void __ClearPageBalloon (struct page *page); + +static inline void __SetPageBalloon (struct page *page) +{ +VM_BUG_ON_PAGE (atomic_read (&page->_mapcount) !=-1 , page); +atomic_set (&page->_mapcount,PAGE_BALLOON_MAPCOUNT_VALUE ); +} + +static inline void __ClearPageBalloon (struct page *page) +{ +VM_BUG_ON_PAGE (!PageBalloon (page), page); +atomic_set (&page->_mapcount,-1 ); +}void put_page (struct page *page);void put_pages_list (struct list_head *pages);diff -puN
mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2mm/Makefilemm/Makefile renamed from mm/Makefile~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 +1
-2@@ -16,7 +16,7 @@ obj-y := filemap.o mempool.o oom_kill. readahead.o swap.o truncate.o vmscan.o shmem.o \ util.o mmzone.o vmstat.o backing-dev.o \ mm_init.o mmu_context.o percpu.o slab_common.o \ - compaction.o vmacache.o \ + compaction.o balloon_compaction.o vmacache.o \ interval_tree.o list_lru.o workingset.o \ iov_iter.o $(mmu-y)@@ -64,4 +64,3 @@ obj-$(CONFIG_ZBUD) += zbud.o obj-$(CONFIG_ZSMALLOC) += zsmalloc.o obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o obj-$(CONFIG_CMA) += cma.o -obj-$(CONFIG_MEMORY_BALLOON) += balloon_compaction.odiff -puN
mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2mm/balloon_compaction.cmm/balloon_compaction.c renamed from mm/balloon_compaction.c~mm-introduce-common-page-state-for-ballooned-memory-fix-v2 +0
-16@@ -10,22 +10,6 @@#include <linux/export.h> #include <linux/balloon_compaction.h> -void __SetPageBalloon (struct page *page) -{ -VM_BUG_ON_PAGE (atomic_read (&page->_mapcount) !=-1 , page); -atomic_set (&page->_mapcount,PAGE_BALLOON_MAPCOUNT_VALUE ); -inc_zone_page_state (page,NR_BALLOON_PAGES ); -} -EXPORT_SYMBOL (__SetPageBalloon); - -void __ClearPageBalloon (struct page *page) -{ -VM_BUG_ON_PAGE (!PageBalloon (page), page); -atomic_set (&page->_mapcount,-1 ); -dec_zone_page_state (page,NR_BALLOON_PAGES ); -} -EXPORT_SYMBOL (__ClearPageBalloon); -/* * balloon_devinfo_alloc - allocates a balloon device information descriptor. * @balloon_dev_descriptor: pointer to reference the balloon device which
Quoted 2 lines
_
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerRe: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
Quoted 11 lines · On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 4 lines
Did we really need to put the BalloonPages count into per-zone vmstat, global vmstat and
/proc/meminfo? Seems a bit overkillish - why so important?Balloon grabs random pages, their distribution among numa nodes might be important.
But I know nobody who uses numa-aware vm together with ballooning.Probably it's better to drop per-zone vmstat and line from meminfo, global vmstat counter should be enough.
Yes, the less we add the better - we can always add stuff later if there is a demonstrated need.
Quoted 10 lines
Quoted 6 lines
Consuming another page flag is a big deal. We keep on nearly running out and one day we'll run out for real.
page-flags-layout.his incomprehensible. How many flags do we have left (worst-case) with this change? Is there no other way? Needs extraordinary justification, please.PageBalloon is not a page flags, it's like PageBuddy -- special state of
_mapcount(-256 in this case).
The same was in v1 and is written in the comment above.
oop sorry, I got confused about KPF_BALLOON.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 6/6] mm/balloon_compaction: general cleanup
In reply to Andrew MortonQuoted 12 lines · Andrew Morton
Quoted 9 lines · On Fri, 12 Sep 2014 17:04:04 -0700 Andrew Morton <[email protected]>…
Quoted 4 lines · On Sat, 30 Aug 2014 20:41:27 +0400 Konstantin Khlebnikov <[email protected]>…
* move special branch for balloon migraion into
migrate_pages
* remove special mapping for balloon and its flagAS_BALLOON_MAP
* embed structballoon_dev_infointo structvirtio_balloon
* cleanupballoon_page_dequeue, killballoon_page_freeNot sure what's going on here - your
include/linux/balloon_compaction.hseems significantly different from mine.
Quoted 8 lines
OK, I worked it out.
Quoted 4 lines
I think I'll just drop this patch - it's quite inconvenient to have a large "general cleanup" coming after a stack of significant functional changes. It makes review, debug, fix, merge and reversion harder. Let's worry about it later.
But I'm still thinking we should defer this one?
It seems in this case massive cleanup before fixies leads to much bigger mess. I've separated fixes specially for merging into stable kernels.
The rest patches mostly rewrites this code using new approatch. I don't know how to make it more reviewable, it's easier to write a new version from the scratch.
Probably rework should be untied form fixes and sent as separate patchset.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
In reply to Andrew MortonQuoted 11 lines · Andrew Morton
Quoted 9 lines · On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 4 lines
Did we really need to put the BalloonPages count into per-zone vmstat, global vmstat and
/proc/meminfo? Seems a bit overkillish - why so important?Balloon grabs random pages, their distribution among numa nodes might be important.
But I know nobody who uses numa-aware vm together with ballooning.
Quoted 5 lines
Quoted 2 lines
Probably it's better to drop per-zone vmstat and line from meminfo, global vmstat counter should be enough.
Yes, the less we add the better - we can always add stuff later if there is a demonstrated need.
Ok. (I guess incremental patches are more convenient for you) Here is two fixes which remove redundant per-zone counters and adds three vmstat counters: "balloon_inflate", "balloon_deflate" and "balloon_migrate".
This statistic seems more useful than current state snapshot. Size of balloon is just a difference between "inflate" and "deflate".
Quoted 13 lines
Quoted 11 lines
Quoted 7 lines
Consuming another page flag is a big deal. We keep on nearly running out and one day we'll run out for real.
page-flags-layout.his incomprehensible. How many flags do we have left (worst-case) with this change? Is there no other way? Needs extraordinary justification, please.PageBalloon is not a page flags, it's like PageBuddy -- special state of
_mapcount(-256 in this case).
The same was in v1 and is written in the comment above.oop sorry, I got confused about
KPF_BALLOON.
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
In reply to Konstantin KhlebnikovQuoted 6 lines · Konstantin Khlebnikov
Quoted 3 lines
Did we really need to put the BalloonPages count into per-zone vmstat,
Quoted 2 lines
global vmstat and
/proc/meminfo? Seems a bit overkillish - why so important?Balloon grabs random pages, their distribution among numa nodes might be important.
But I know nobody who uses numa-aware vm together with ballooning.
*cough*. me?
Obviously there's no need to keep that per-zone vmstat just for me, but right now NUMA on KVM works just fine and does a good job of catching NUMA issues without having to run on actual metal.
Thanks,
Sasha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerRe: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
In reply to Konstantin KhlebnikovQuoted 19 lines · On Sat, 13 Sep 2014 12:22:23 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 11 lines · Andrew Morton
Quoted 9 lines · On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 4 lines
Did we really need to put the BalloonPages count into per-zone vmstat, global vmstat and
/proc/meminfo? Seems a bit overkillish - why so important?Balloon grabs random pages, their distribution among numa nodes might be important.
But I know nobody who uses numa-aware vm together with ballooning.Quoted 5 lines
Quoted 2 lines
Probably it's better to drop per-zone vmstat and line from meminfo, global vmstat counter should be enough.
Yes, the less we add the better - we can always add stuff later if there is a demonstrated need.
Quoted 7 lines
Ok. (I guess incremental patches are more convenient for you) Here is two fixes which remove redundant per-zone counters and adds three vmstat counters: "
balloon_inflate", "balloon_deflate" and "balloon_migrate".This statistic seems more useful than current state snapshot. Size of balloon is just a difference between "inflate" and "deflate".
This is a complete mess.
Your second patch (which is actually the first one) called "fix for mm-introduce-common-page-state-for-ballooned-memory-fix-v2" is indeed a fix for mm-introduce-common-page-state-for-ballooned-memory-fix-v2.patch and has a changelog.
I eventually worked out that your first patch (whcih is actually the second one) called "fix for mm-balloon_compaction-use-common-page-ballooning-v2" assumes that mm-balloon_compaction-general-cleanup.patch has been applied. Does it actually fix mm-balloon_compaction-use-common-page-ballooning-v2.patch? I can't tell, because the "general cleanup" is in the way.
So I'm going to send "fix for mm-balloon_compaction-use-common-page-ballooning-v2" to Linus separately, but it has no changelog at all.
Please always write changelogs. Please never send more than one patch in a single email. Please be *consistent* in filenames, patch titles, email subjects, etc.
Please send me a changelog for the below patch.
From: Konstantin Khlebnikov <[email protected]>
Subject: mm-balloon_compaction-use-common-page-ballooning-v2-fix-1
---
drivers/virtio/virtio_balloon.c | 1 + include/linux/balloon_compaction.h | 2 -- mm/balloon_compaction.c | 2 ++ 3 files changed, 3 insertions(+), 2 deletions(-)
drivers/virtio/virtio_balloon.c renamed from drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 +1 -0
@@ -395,6 +395,7 @@ static int virtballoon_migratepage(struc/* balloon's page migration 1st step -- inflate "newpage" */ spin_lock_irqsave (&vb_dev_info->pages_lock, flags);balloon_page_insert (vb_dev_info, newpage); +__count_vm_event (BALLOON_MIGRATE ); vb_dev_info->isolated_pages--;spin_unlock_irqrestore (&vb_dev_info->pages_lock, flags); vb->num_pfns =VIRTIO_BALLOON_PAGES_PER_PAGE ;
include/linux/balloon_compaction.h renamed from include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 +0 -2
@@ -87,7 +87,6 @@ static inline void balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page) {__SetPageBalloon (page); -inc_zone_page_state (page,NR_BALLOON_PAGES );set_page_private (page, (unsigned long )b_dev_info);list_add (&page->lru, &b_dev_info->pages); }
@@ -104,7 +103,6 @@ balloon_page_insert(struct balloon_dev_istatic inline void balloon_page_delete (struct page *page,bool isolated) {__ClearPageBalloon (page); -dec_zone_page_state (page,NR_BALLOON_PAGES );set_page_private (page,0 );if (!isolated)list_del (&page->lru);
mm/balloon_compaction.c renamed from mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 +2 -0
@@ -36,6 +36,7 @@ struct page *balloon_page_enqueue(structBUG_ON (!trylock_page (page));spin_lock_irqsave (&b_dev_info->pages_lock, flags);balloon_page_insert (b_dev_info, page); +__count_vm_event (BALLOON_INFLATE );spin_unlock_irqrestore (&b_dev_info->pages_lock, flags);unlock_page (page);return page;
@@ -67,6 +68,7 @@ struct page *balloon_page_dequeue(structif (trylock_page (page)) {spin_lock_irqsave (&b_dev_info->pages_lock, flags);balloon_page_delete (page, false); +__count_vm_event (BALLOON_DEFLATE );spin_unlock_irqrestore (&b_dev_info->pages_lock, flags);unlock_page (page);return page;
_
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
Quoted 26 lines · Andrew Morton
Quoted 24 lines · On Sat, 13 Sep 2014 12:22:23 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 11 lines · Andrew Morton
Quoted 9 lines · On Sat, 13 Sep 2014 09:26:49 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 4 lines
Did we really need to put the BalloonPages count into per-zone vmstat, global vmstat and
/proc/meminfo? Seems a bit overkillish - why so important?Balloon grabs random pages, their distribution among numa nodes might be important.
But I know nobody who uses numa-aware vm together with ballooning.Quoted 5 lines
Quoted 2 lines
Probably it's better to drop per-zone vmstat and line from meminfo, global vmstat counter should be enough.
Yes, the less we add the better - we can always add stuff later if there is a demonstrated need.
Ok. (I guess incremental patches are more convenient for you) Here is two fixes which remove redundant per-zone counters and adds three vmstat counters: "
balloon_inflate", "balloon_deflate" and "balloon_migrate".
Quoted 4 lines
Quoted 2 lines
This statistic seems more useful than current state snapshot. Size of balloon is just a difference between "inflate" and "deflate".
This is a complete mess.
nod
Quoted 7 lines
Your second patch (which is actually the first one) called "fix for mm-introduce-common-page-state-for-ballooned-memory-fix-v2" is indeed a fix for mm-introduce-common-page-state-for-ballooned-memory-fix-v2.patch and has a changelog.
Quoted 6 lines
I eventually worked out that your first patch (whcih is actually the second one) called "fix for mm-balloon_compaction-use-common-page-ballooning-v2" assumes that mm-balloon_compaction-general-cleanup.patch has been applied. Does it actually fix mm-balloon_compaction-use-common-page-ballooning-v2.patch? I can't tell, because the "general cleanup" is in the way.
Oops I did it again.
Quoted 4 lines
So I'm going to send "fix for mm-balloon_compaction-use-common-page-ballooning-v2" to Linus separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually fixes and stresstest. This is gone too far, now balloon won't compile in the middle of patchset. Just tell me and I'll redo the rest.
Quoted 4 lines
Please always write changelogs. Please never send more than one patch in a single email. Please be *consistent* in filenames, patch titles, email subjects, etc.
That patch is supposed to be merged into patch as a fix.
Quoted 2 lines
Please send me a changelog for the below patch.
Ok, sure.
Quoted 1 line
From: Konstantin Khlebnikov <[email protected]>
Subject: mm/balloon_compaction: use vmstat counters
This is fix for "mm/balloon_compaction: use common page ballooning". it reverts per-zone balloon counters and replaces them with vmstat counters:
"balloon_inflate", "balloon_deflate" and "balloon_migrate".
Per-zone balloon counters have been reverted after discussion but reverting them from balloon_compaction conflicts with massive cleanup in this code. Thus this change ends up as a separate patch. Sorry for the mess.
Quoted 56 lines
---
drivers/virtio/virtio_balloon.c | 1 + include/linux/balloon_compaction.h | 2 -- mm/balloon_compaction.c | 2 ++ 3 files changed, 3 insertions(+), 2 deletions(-)diff -puN
drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1drivers/virtio/virtio_balloon.cdrivers/virtio/virtio_balloon.c renamed from drivers/virtio/virtio_balloon.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 +1
-0@@ -395,6 +395,7 @@ static int virtballoon_migratepage(struc/* balloon's page migration 1st step -- inflate "newpage" */ spin_lock_irqsave (&vb_dev_info->pages_lock, flags);balloon_page_insert (vb_dev_info, newpage); +__count_vm_event (BALLOON_MIGRATE ); vb_dev_info->isolated_pages--;spin_unlock_irqrestore (&vb_dev_info->pages_lock, flags); vb->num_pfns =VIRTIO_BALLOON_PAGES_PER_PAGE ;diff -puN
include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1include/linux/balloon_compaction.hinclude/linux/balloon_compaction.h renamed from include/linux/balloon_compaction.h~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 +0
-2@@ -87,7 +87,6 @@ static inline void balloon_page_insert(struct balloon_dev_info *b_dev_info, struct page *page) {__SetPageBalloon (page); -inc_zone_page_state (page,NR_BALLOON_PAGES );set_page_private (page, (unsigned long )b_dev_info);list_add (&page->lru, &b_dev_info->pages); }@@ -104,7 +103,6 @@ balloon_page_insert(struct balloon_dev_istatic inline void balloon_page_delete (struct page *page,bool isolated) {__ClearPageBalloon (page); -dec_zone_page_state (page,NR_BALLOON_PAGES );set_page_private (page,0 );if (!isolated)list_del (&page->lru);diff -puN
mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1mm/balloon_compaction.cmm/balloon_compaction.c renamed from mm/balloon_compaction.c~mm-balloon_compaction-use-common-page-ballooning-v2-fix-1 +2
-0@@ -36,6 +36,7 @@ struct page *balloon_page_enqueue(structBUG_ON (!trylock_page (page));spin_lock_irqsave (&b_dev_info->pages_lock, flags);balloon_page_insert (b_dev_info, page); +__count_vm_event (BALLOON_INFLATE );spin_unlock_irqrestore (&b_dev_info->pages_lock, flags);unlock_page (page);return page;@@ -67,6 +68,7 @@ struct page *balloon_page_dequeue(structif (trylock_page (page)) {spin_lock_irqsave (&b_dev_info->pages_lock, flags);balloon_page_delete (page, false); +__count_vm_event (BALLOON_DEFLATE );spin_unlock_irqrestore (&b_dev_info->pages_lock, flags);unlock_page (page);return page;
Quoted 2 lines
_
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Andrew Morton
maintainerRe: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
Quoted 8 lines · On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 4 lines
So I'm going to send "fix for mm-balloon_compaction-use-common-page-ballooning-v2" to Linus separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually fixes and stresstest. This is gone too far, now balloon won't compile in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch mm-introduce-common-page-state-for-ballooned-memory.patch mm-balloon_compaction-use-common-page-ballooning.patch mm-balloon_compaction-general-cleanup.patch mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Please go through it and send out a new version?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
Quoted 22 lines · Andrew Morton
Quoted 8 lines · On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 4 lines
So I'm going to send "fix for mm-balloon_compaction-use-common-page-ballooning-v2" to Linus separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually fixes and stresstest. This is gone too far, now balloon won't compile in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch mm-introduce-common-page-state-for-ballooned-memory.patch mm-balloon_compaction-use-common-page-ballooning.patch mm-balloon_compaction-general-cleanup.patch mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Quoted 3 lines
Please go through it and send out a new version?
I've found yet another bug in this code. It seems here is a nest. balloon_page_dequeue can race with balloon_page_isolate:balloon_page_isolate can remove page from list between llist_for_each_entry_safe and trylock_page in balloon_page_dequeue. balloon_page_dequeue runs under mutex_lock(&vb->balloon_lock); both of them lock page using trylock_page so race is tight but it is not impossible.
Probably it's really easier to rewrite it than to fix bugs one by one =/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
Quoted 28 lines · Konstantin Khlebnikov
Quoted 22 lines · Andrew Morton
Quoted 8 lines · On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 4 lines
So I'm going to send "fix for mm-balloon_compaction-use-common-page-ballooning-v2" to Linus separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually fixes and stresstest. This is gone too far, now balloon won't compile in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch mm-introduce-common-page-state-for-ballooned-memory.patch mm-balloon_compaction-use-common-page-ballooning.patch mm-balloon_compaction-general-cleanup.patch mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Quoted 3 lines
Please go through it and send out a new version?
Quoted 7 lines
I've found yet another bug in this code. It seems here is a nest.
balloon_page_dequeuecan race withballoon_page_isolate:balloon_page_isolatecan remove page from list betweenllist_for_each_entry_safeandtrylock_pageinballoon_page_dequeue.balloon_page_dequeueruns undermutex_lock(&vb->balloon_lock); both of them lock page usingtrylock_pageso race is tight but it is not impossible.
Plausible to happen if stress testing compaction simultaneously with freezing/unloading the balloon driver. As you noted, it's quite tight despite not impossible. Nice catch.
Quoted 1 line
Probably it's really easier to rewrite it than to fix bugs one by one =/
I'm not against a rewrite, but I don't think that rewriting the code to get rid of such bugs changes the fact we still have to address them in the actual placed code as we go on finding them. That's why I thought your inital changeset fine, with patches for stable going first and code overhaul for next following them up.
For this race you spotted, I think a simple change like the following might be enough (not-tested)
mm/balloon_compaction.c +0 -0
@@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(struct
balloon_dev_info *b_dev_info)
* to be released by the balloon driver.
*/
if (trylock_page(page)) {
+ /*
+ * Skip dequeue attempt for this page to a later round
+ * if balloon_page_isolate() has sucessfully isolated
+ * it just before we got the page lock here.
+ */+ if (page_count(page) != 1) {
+ unlock_page(page);
+ continue
+ }
+
spin_lock_irqsave(&b_dev_info->pages_lock, flags); /*
* Raise the page refcount here to prevent any
* wrong
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
Quoted 41 lines · Rafael Aquini
Quoted 28 lines · Konstantin Khlebnikov
Quoted 22 lines · Andrew Morton
Quoted 8 lines · On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 4 lines
So I'm going to send "fix for mm-balloon_compaction-use-common-page-ballooning-v2" to Linus separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually fixes and stresstest. This is gone too far, now balloon won't compile in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch mm-introduce-common-page-state-for-ballooned-memory.patch mm-balloon_compaction-use-common-page-ballooning.patch mm-balloon_compaction-general-cleanup.patch mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Quoted 3 lines
Please go through it and send out a new version?
Quoted 7 lines
I've found yet another bug in this code. It seems here is a nest.
balloon_page_dequeuecan race withballoon_page_isolate:balloon_page_isolatecan remove page from list betweenllist_for_each_entry_safeandtrylock_pageinballoon_page_dequeue.balloon_page_dequeueruns undermutex_lock(&vb->balloon_lock); both of them lock page usingtrylock_pageso race is tight but it is not impossible.Plausible to happen if stress testing compaction simultaneously with freezing/unloading the balloon driver. As you noted, it's quite tight despite not impossible. Nice catch.
Quoted 8 lines
Quoted 1 line
Probably it's really easier to rewrite it than to fix bugs one by one =/
I'm not against a rewrite, but I don't think that rewriting the code to get rid of such bugs changes the fact we still have to address them in the actual placed code as we go on finding them. That's why I thought your inital changeset fine, with patches for stable going first and code overhaul for next following them up.
For this race you spotted, I think a simple change like the following might be enough (not-tested)
This locking scheme is too fragile and uncommon.
What about this:
* special page->_mapcount marks ballooned pages
* page->private points to balloon (directly, without intermediate mapping)
* flag PagePrivate means page currently in balloon page list (i.e. not isolated, like PageLRU for normal pages)
* lock_page protects all of them
balloon_page_dequeue() will delete page from balloon list only if it's not isolated, also it always clears page->private and balloon mark. put-back rechecks mark after locking the page and releases it as normal page if mark is gone.
Quoted 22 lines
mm/balloon_compaction.c +10
-0@@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(structballoon_dev_info *b_dev_info) * to be released by the balloon driver. */ if (trylock_page (page)) { +/* +* Skip dequeue attempt for this page to a later round +* if balloon_page_isolate() has sucessfully isolated +* it just before we got the page lock here. +*/ +if (page_count (page) !=1 ) { +unlock_page (page); + continue + } +spin_lock_irqsave (&b_dev_info->pages_lock, flags);/*
Quoted 3 lines
* Raise the page refcount here to prevent any
* wrong
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
Quoted 48 lines · Konstantin Khlebnikov
Quoted 47 lines · Rafael Aquini
Quoted 28 lines · Konstantin Khlebnikov
Quoted 22 lines · Andrew Morton
Quoted 8 lines · On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 4 lines
So I'm going to send "fix for mm-balloon_compaction-use-common-page-ballooning-v2" to Linus separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually fixes and stresstest. This is gone too far, now balloon won't compile in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch mm-introduce-common-page-state-for-ballooned-memory.patch mm-balloon_compaction-use-common-page-ballooning.patch mm-balloon_compaction-general-cleanup.patch mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Quoted 3 lines
Please go through it and send out a new version?
Quoted 7 lines
I've found yet another bug in this code. It seems here is a nest.
balloon_page_dequeuecan race withballoon_page_isolate:balloon_page_isolatecan remove page from list betweenllist_for_each_entry_safeandtrylock_pageinballoon_page_dequeue.balloon_page_dequeueruns undermutex_lock(&vb->balloon_lock); both of them lock page usingtrylock_pageso race is tight but it is not impossible.Plausible to happen if stress testing compaction simultaneously with freezing/unloading the balloon driver. As you noted, it's quite tight despite not impossible. Nice catch.
Quoted 1 line
Probably it's really easier to rewrite it than to fix bugs one by one =/
I'm not against a rewrite, but I don't think that rewriting the code to get rid of such bugs changes the fact we still have to address them in the actual placed code as we go on finding them. That's why I thought your inital changeset fine, with patches for stable going first and code overhaul for next following them up.
Quoted 5 lines
Quoted 2 lines
For this race you spotted, I think a simple change like the following might be enough (not-tested)
This locking scheme is too fragile and uncommon.
page_lock and refcounting was what I had at my disposal to sort these races out since I haven't thoutgh on a special page->_mapcount when designing this feature. It's the way other page races are sorted out. Not arguing it's not fragile, but it's the way code is layed out since some time, so we must check the feasibility of a total overhaul for stable branches.
Quoted 8 lines
What about this:
* special
page->_mapcountmarks ballooned pages
*page->privatepoints to balloon (directly, without intermediate mapping)
* flag PagePrivate means page currently in balloon page list (i.e. not isolated, like PageLRU for normal pages)
*lock_pageprotects all of them
Quoted 5 lines
balloon_page_dequeue()will delete page from balloon list only if it's not isolated, also it always clearspage->privateand balloon mark. put-back rechecks mark after locking the page and releases it as normal page if mark is gone.
I have already agreed with you here, since the changes above are mostly from your original overhaul proposal. It's a much better approach for that balloon code, no doubts. Thanks for doing it. Only thing we need to take care here is about its requirement on changing the semantics for those interfaces might turn the changes unfeasible for old stable branches. If we can ignore this mentioned fact entirely, I don't see why not going with your idea all branches across, otherwise I think we should overhaul the code for -next, and send pontual fixes for stable.
Cheers,
-- Rafael
Quoted 17 lines
Quoted 17 lines
mm/balloon_compaction.c +7
-0@@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(structballoon_dev_info *b_dev_info) * to be released by the balloon driver. */ if (trylock_page (page)) { +/* +* Skip dequeue attempt for this page to a later round +* if balloon_page_isolate() has sucessfully isolated +* it just before we got the page lock here. +*/ +if (page_count (page) !=1 ) { +unlock_page (page);
Quoted 8 lines
Quoted 8 lines
mm/balloon_compaction.c +3
-0+ continue + } +spin_lock_irqsave (&b_dev_info->pages_lock, flags);/* * Raise the page refcount here to prevent any
* wrong
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Re: [PATCH v2 4/6] mm: introduce common page state for ballooned memory
Quoted 77 lines · Rafael Aquini
Quoted 48 lines · Konstantin Khlebnikov
Quoted 47 lines · Rafael Aquini
Quoted 28 lines · Konstantin Khlebnikov
Quoted 22 lines · Andrew Morton
Quoted 8 lines · On Sat, 20 Sep 2014 09:25:01 +0400 Konstantin Khlebnikov <[email protected]>…
Quoted 4 lines
So I'm going to send "fix for mm-balloon_compaction-use-common-page-ballooning-v2" to Linus separately, but it has no changelog at all.
Probably it would be better if you drop everything except actually fixes and stresstest. This is gone too far, now balloon won't compile in the middle of patchset. Just tell me and I'll redo the rest.
I think it's best if I drop everything:
mm-balloon_compaction-ignore-anonymous-pages.patch mm-balloon_compaction-keep-ballooned-pages-away-from-normal-migration-path.patch mm-balloon_compaction-isolate-balloon-pages-without-lru_lock.patch selftests-vm-transhuge-stress-stress-test-for-memory-compaction.patch mm-introduce-common-page-state-for-ballooned-memory.patch mm-balloon_compaction-use-common-page-ballooning.patch mm-balloon_compaction-general-cleanup.patch mm-balloon_compaction-use-common-page-ballooning-v2-fix-1.patch
Quoted 3 lines
Please go through it and send out a new version?
Quoted 7 lines
I've found yet another bug in this code. It seems here is a nest.
balloon_page_dequeuecan race withballoon_page_isolate:balloon_page_isolatecan remove page from list betweenllist_for_each_entry_safeandtrylock_pageinballoon_page_dequeue.balloon_page_dequeueruns undermutex_lock(&vb->balloon_lock); both of them lock page usingtrylock_pageso race is tight but it is not impossible.Plausible to happen if stress testing compaction simultaneously with freezing/unloading the balloon driver. As you noted, it's quite tight despite not impossible. Nice catch.
Quoted 1 line
Probably it's really easier to rewrite it than to fix bugs one by one =/
I'm not against a rewrite, but I don't think that rewriting the code to get rid of such bugs changes the fact we still have to address them in the actual placed code as we go on finding them. That's why I thought your inital changeset fine, with patches for stable going first and code overhaul for next following them up.
Quoted 5 lines
Quoted 2 lines
For this race you spotted, I think a simple change like the following might be enough (not-tested)
This locking scheme is too fragile and uncommon.
page_lockand refcounting was what I had at my disposal to sort these races out since I haven't thoutgh on a specialpage->_mapcountwhen designing this feature. It's the way other page races are sorted out. Not arguing it's not fragile, but it's the way code is layed out since some time, so we must check the feasibility of a total overhaul for stable branches.Quoted 8 lines
What about this:
* special
page->_mapcountmarks ballooned pages
*page->privatepoints to balloon (directly, without intermediate mapping)
* flag PagePrivate means page currently in balloon page list (i.e. not isolated, like PageLRU for normal pages)
*lock_pageprotects all of themQuoted 5 lines
balloon_page_dequeue()will delete page from balloon list only if it's not isolated, also it always clearspage->privateand balloon mark. put-back rechecks mark after locking the page and releases it as normal page if mark is gone.
Quoted 9 lines
I have already agreed with you here, since the changes above are mostly from your original overhaul proposal. It's a much better approach for that balloon code, no doubts. Thanks for doing it. Only thing we need to take care here is about its requirement on changing the semantics for those interfaces might turn the changes unfeasible for old stable branches. If we can ignore this mentioned fact entirely, I don't see why not going with your idea all branches across, otherwise I think we should overhaul the code for -next, and send pontual fixes for stable.
Ok, I'll try to implement this approach in suitable for stable branches way. Keeping several different versions is overkill for this code.
Quoted 21 lines
Cheers,
-- RafaelQuoted 17 lines
Quoted 17 lines
mm/balloon_compaction.c +7
-0@@ -93,6 +93,16 @@ struct page *balloon_page_dequeue(structballoon_dev_info *b_dev_info) * to be released by the balloon driver. */ if (trylock_page (page)) { +/* +* Skip dequeue attempt for this page to a later round +* if balloon_page_isolate() has sucessfully isolated +* it just before we got the page lock here. +*/ +if (page_count (page) !=1 ) { +unlock_page (page);
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
mm/balloon_compaction.c +3
-0+ continue + } +spin_lock_irqsave (&b_dev_info->pages_lock, flags);/* * Raise the page refcount here to prevent any
* wrong
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>