Skip to contentSkip to conversation

Volatile Ranges v9

PATCH In review John Stultz 6 participants 39 messages Oct 3, 2013 – Oct 8, 2013

Subsystems (16)
lore
Share & tools
lore
#0 lore raw

cover Volatile Ranges v9

So its been awhile since the last release of the volatile ranges patches, and while Minchan and I have been busy with other things, we have been slowly chipping away at issues and differences trying to get a patchset that we both agree on.

There's still a few smaller issues, but we figured any further polishing of the patch series in private would be unproductive and it would be much better to send the patches out for review and comment and get some wider opinions.

Whats new in v9:
* Updated to v3.11
* Added vrange purging logic to purge anonymous pages on swapless systems
* Added logic to allocate the vroot structure dynamically to avoid added overhead to mm and address_space structures
* Lots of minor tweaks, changes and cleanups

Still TODO:
* Sort out better solution for clearing volatility on new mmaps
- Minchan has a different approach here
* Sort out apparent shrinker livelock that occasionally crops up under severe pressure

Feedback or thoughts here would be particularly helpful!

As is apparent from the author list, Minchan has really been the one doing the heavy lifting here, and I've only been finding and fixing a few bugs, refactoring the code for readability, and trying to clarify commit messages. So many many thanks to Minchan here for all his great work, and putting up with my sometimes misguided "editing".

Also, thanks to Dhaval for his maintaining and vastly improving the volatile ranges test suite, which can be found here:
https://github.com/volatile-ranges-test/vranges-test

These patches can also be pulled from git here:
git://git.linaro.org/people/jstultz/android-dev.git dev/vrange-v9

We'd really welcome any feedback and comments on the patch series.

thanks
-john

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>

John Stultz (2):
vrange: Clear volatility on new mmaps
vrange: Add support for volatile ranges on file mappings

Minchan Kim (12):
vrange: Add basic data structure and functions
vrange: Add vrange support to mm_structs
vrange: Add new vrange(2) system call
vrange: Add basic functions to purge volatile pages
vrange: Purge volatile pages when memory is tight
vrange: Send SIGBUS when user try to access purged page
vrange: Add vrange LRU list for purging
vrange: Add core shrinking logic for swapless system
vrange: Purging vrange-anon pages from shrinker
vrange: Support background purging for vrange-file
vrange: Allocate vroot dynamically
vrange: Add vmstat counter about purged page

 arch/x86/syscalls/syscall_64.tbl       |    1 +
 fs/inode.c                             |    4 +
 include/linux/fs.h                     |    4 +
 include/linux/mm_types.h               |    4 +
 include/linux/rmap.h                   |   11 +-
 include/linux/swap.h                   |    6 +-
 include/linux/syscalls.h               |    2 +
 include/linux/vm_event_item.h          |    2 +
 include/linux/vrange.h                 |   84 +++
 include/linux/vrange_types.h           |   28 +
 include/uapi/asm-generic/mman-common.h |    3 +
 kernel/fork.c                          |   12 +
 kernel/sys_ni.c                        |    1 +
 lib/Makefile                           |    2 +-
 mm/Makefile                            |    2 +-
 mm/internal.h                          |    2 -
 mm/ksm.c                               |    2 +-
 mm/memory.c                            |   27 +
 mm/mincore.c                           |    5 +-
 mm/mmap.c                              |    5 +
 mm/rmap.c                              |   28 +-
 mm/vmscan.c                            |   17 +-
 mm/vmstat.c                            |    2 +
 mm/vrange.c                            | 1196 ++++++++++++++++++++++++++++++++
 24 files changed, 1429 insertions(+), 21 deletions(-)
 create mode 100644 include/linux/vrange.h
 create mode 100644 include/linux/vrange_types.h
 create mode 100644 mm/vrange.c
Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#1 lore raw

PATCH 1/14 vrange: Add basic data structure and functions

From: Minchan Kim <[email protected]>

This patch adds vrange data structure and core management functions.

The vrange uses the generic interval tree as main data structure because it handles address range, which fits well for this purpose.

The vrange_add/vrange_remove are core functions for the vrange() system call that will be introduced in a following patch.

The vrange_add inserts new address range into interval tree. If new address range crosses over existing volatile range, existing volatile range will be expanded to cover new range.

Thus, if existing volatile range has purged state, new extended range will inherit that purged state. If new address range is inside existing range, we ignore it.

vrange_remove removes the address range, returning the purged state of the address ranges.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>

[jstultz: Heavy rework and cleanups to make this infrastructure more easily reused for both file and anonymous pages]

---
 include/linux/vrange.h       |  48 ++++++++++++
 include/linux/vrange_types.h |  25 ++++++
 lib/Makefile                 |   2 +-
 mm/Makefile                  |   2 +-
 mm/vrange.c                  | 183 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 258 insertions(+), 2 deletions(-)
 create mode 100644 include/linux/vrange.h
 create mode 100644 include/linux/vrange_types.h
 create mode 100644 mm/vrange.c
include/linux/vrange.h new file +48 -0
new file mode 100644
index 0000000..0d378a5
@@ -0,0 +1,48 @@
+#ifndef _LINUX_VRANGE_H
+#define _LINUX_VRANGE_H
+
+#include <linux/vrange_types.h>
+#include <linux/mm.h>
+
+#define vrange_from_node(node_ptr) \
+	container_of(node_ptr, struct vrange, node)
+
+#define vrange_entry(ptr) \
+	container_of(ptr, struct vrange, node.rb)
+
+#ifdef CONFIG_MMU
+
+static inline void vrange_root_init(struct vrange_root *vroot, int type,
+								void *object)
+{
+	vroot->type = type;
+	vroot->v_rb = RB_ROOT;
+	mutex_init(&vroot->v_lock);
+	vroot->object = object;
+}
+
+static inline void vrange_lock(struct vrange_root *vroot)
+{
+	mutex_lock(&vroot->v_lock);
+}
+
+static inline void vrange_unlock(struct vrange_root *vroot)
+{
+	mutex_unlock(&vroot->v_lock);
+}
+
+static inline int vrange_type(struct vrange *vrange)
+{
+	return vrange->owner->type;
+}
+
+extern void vrange_root_cleanup(struct vrange_root *vroot);
+
+#else
+
+static inline void vrange_root_init(struct vrange_root *vroot,
+					int type, void *object) {};
+static inline void vrange_root_cleanup(struct vrange_root *vroot) {};
+
+#endif
+#endif /* _LINIUX_VRANGE_H */
include/linux/vrange_types.h new file +25 -0
new file mode 100644
index 0000000..0d48b42
@@ -0,0 +1,25 @@
+#ifndef _LINUX_VRANGE_TYPES_H
+#define _LINUX_VRANGE_TYPES_H
+
+#include <linux/mutex.h>
+#include <linux/interval_tree.h>
+
+enum vrange_type {
+	VRANGE_MM,
+	VRANGE_FILE,
+};
+
+struct vrange_root {
+	struct rb_root v_rb;		/* vrange rb tree */
+	struct mutex v_lock;		/* Protect v_rb */
+	enum vrange_type type;		/* range root type */
+	void *object;			/* pointer to mm_struct or mapping */
+};
+
+struct vrange {
+	struct interval_tree_node node;
+	struct vrange_root *owner;
+	int purged;
+};
+#endif
+
lib/Makefile +1 -1
index 7baccfd..c8739ee 100644
@@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
 	 sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
 	 proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
 	 is_single_threaded.o plist.o decompress.o kobject_uevent.o \
-	 earlycpio.o percpu-refcount.o
+	 earlycpio.o percpu-refcount.o interval_tree.o
 
 obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
 lib-$(CONFIG_MMU) += ioremap.o
mm/Makefile +1 -1
index f008033..54928af 100644
@@ -5,7 +5,7 @@
 mmu-y			:= nommu.o
 mmu-$(CONFIG_MMU)	:= fremap.o highmem.o madvise.o memory.o mincore.o \
 			   mlock.o mmap.o mprotect.o mremap.o msync.o rmap.o \
-			   vmalloc.o pagewalk.o pgtable-generic.o
+			   vmalloc.o pagewalk.o pgtable-generic.o vrange.o
 
 ifdef CONFIG_CROSS_MEMORY_ATTACH
 mmu-$(CONFIG_MMU)	+= process_vm_access.o
mm/vrange.c new file +183 -0

183 lines not on this page · raw message

Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#2 lore raw

PATCH 2/14 vrange: Add vrange support to mm_structs

In reply to John Stultz

From: Minchan Kim <[email protected]>

This patch addes vroot on mm_struct so process can set volatile ranges on anonymous memory.

This is somewhat wasteful, as it increases the mm struct even if the process doesn't use vrange syscall. So a later patch will provide dynamically allocated vroots.

One of note on this patch is vrange_fork. Since we do allocations while holding a lock on the vrange, its possible it could deadlock with direct reclaim's purging logic. For this reason, vrange_fork uses GFP_NOIO for its allocations.

If vrange_fork fails, it isn't a critical problem. Since the result is the child process's pages won't be volatile/purgable, which could cause additional memory pressure, but won't cause problematic application behavior (since volatile pages are only purged at the kernels' discretion). This is thought to be more desirable then having fork fail.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
[jstultz: Bit of refactoring. Comment cleanups]
---
 include/linux/mm_types.h |  4 ++++
 include/linux/vrange.h   |  7 ++++++-
 kernel/fork.c            | 11 +++++++++++
 mm/vrange.c              | 40 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 61 insertions(+), 1 deletion(-)
include/linux/mm_types.h +4 -0
index faf4b7c..5d8cdc3 100644
@@ -13,6 +13,7 @@
 #include <linux/page-debug-flags.h>
 #include <linux/uprobes.h>
 #include <linux/page-flags-layout.h>
+#include <linux/vrange_types.h>
 #include <asm/page.h>
 #include <asm/mmu.h>
 
@@ -349,6 +350,9 @@ struct mm_struct {
 						 */
 
 
+#ifdef CONFIG_MMU
+	struct vrange_root vroot;
+#endif
 	unsigned long hiwater_rss;	/* High-watermark of RSS usage */
 	unsigned long hiwater_vm;	/* High-water virtual memory usage */
 
include/linux/vrange.h +6 -1
index 0d378a5..2b96ee1 100644
@@ -37,12 +37,17 @@ static inline int vrange_type(struct vrange *vrange)
 }
 
 extern void vrange_root_cleanup(struct vrange_root *vroot);
-
+extern int vrange_fork(struct mm_struct *new,
+					struct mm_struct *old);
 #else
 
 static inline void vrange_root_init(struct vrange_root *vroot,
 					int type, void *object) {};
 static inline void vrange_root_cleanup(struct vrange_root *vroot) {};
+static inline int vrange_fork(struct mm_struct *new, struct mm_struct *old)
+{
+	return 0;
+}
 
 #endif
 #endif /* _LINIUX_VRANGE_H */
kernel/fork.c +11 -0
index bf46287..ceb38bf 100644
@@ -71,6 +71,7 @@
 #include <linux/signalfd.h>
 #include <linux/uprobes.h>
 #include <linux/aio.h>
+#include <linux/vrange.h>
 
 #include <asm/pgtable.h>
 #include <asm/pgalloc.h>
@@ -377,6 +378,14 @@ static int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
 	retval = khugepaged_fork(mm, oldmm);
 	if (retval)
 		goto out;
+	/*
+	 * Note: vrange_fork can fail in the case of ENOMEM, but
+	 * this only results in the child not having any active
+	 * volatile ranges. This is not harmful. Thus in this case
+	 * the child will not see any pages purged unless it remarks
+	 * them as volatile.
+	 */
+	vrange_fork(mm, oldmm);
 
 	prev = NULL;
 	for (mpnt = oldmm->mmap; mpnt; mpnt = mpnt->vm_next) {
@@ -538,6 +547,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p)
 	mm->nr_ptes = 0;
 	memset(&mm->rss_stat, 0, sizeof(mm->rss_stat));
 	spin_lock_init(&mm->page_table_lock);
+	vrange_root_init(&mm->vroot, VRANGE_MM, mm);
 	mm_init_aio(mm);
 	mm_init_owner(mm, p);
 
@@ -609,6 +619,7 @@ void mmput(struct mm_struct *mm)
 
 	if (atomic_dec_and_test(&mm->mm_users)) {
 		uprobe_clear_state(mm);
+		vrange_root_cleanup(&mm->vroot);
 		exit_aio(mm);
 		ksm_exit(mm);
 		khugepaged_exit(mm); /* must run before exit_mmap */
mm/vrange.c +40 -0
index 866566c..4ddcc3e9 100644
@@ -181,3 +181,43 @@ void vrange_root_cleanup(struct vrange_root *vroot)
 	vrange_unlock(vroot);
 }
 
+/*
+ * It's okay to fail vrange_fork because worst case is child process
+ * can't have copied own vrange data structure so that pages in the
+ * vrange couldn't be purged. It would be better rather than failing
+ * fork.
+ */
+int vrange_fork(struct mm_struct *new_mm, struct mm_struct *old_mm)
+{
+	struct vrange_root *new, *old;
+	struct vrange *range, *new_range;
+	struct rb_node *next;
+
+	new = &new_mm->vroot;
+	old = &old_mm->vroot;
+
+	vrange_lock(old);
+	next = rb_first(&old->v_rb);
+	while (next) {
+		range = vrange_entry(next);
+		next = rb_next(next);
+		/*
+		 * We can't use GFP_KERNEL because direct reclaim's
+		 * purging logic on vrange could be deadlock by
+		 * vrange_lock.
+		 */
+		new_range = __vrange_alloc(GFP_NOIO);
+		if (!new_range)
+			goto fail;
+		__vrange_set(new_range, range->node.start,
+					range->node.last, range->purged);
+		__vrange_add(new_range, new);
+
+	}
+	vrange_unlock(old);
+	return 0;
+fail:
+	vrange_unlock(old);
+	vrange_root_cleanup(new);
+	return -ENOMEM;
+}
Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#3 lore raw

PATCH 3/14 vrange: Clear volatility on new mmaps

In reply to John Stultz

At lsf-mm, the issue was brought up that there is a precedence with interfaces like mlock, such that new mappings in a pre-existing range do no inherit the mlock state.

This is mostly because mlock only modifies the existing vmas, and so any new mmaps create new vmas, which won't be mlocked.

Since volatility is not stored in the vma (for good cause, specifically as we'd have to have manage file volatility differently from anonymous and we're likely to manage volatility on small chunks of memory, which would cause lots of vma splitting and churn), this patch clears volitility on new mappings, to ensure that we don't inherit volatility if memory in an existing volatile range is unmapped and then re-mapped with something else.

Thus, this patch forces any volatility to be cleared on mmap.

XXX: We expect this patch to be not well loved by mm folks, and are open to alternative methods here. Its more of a place holder to address the issue from lsf-mm and hopefully will spur some further discussion.

Minchan does have an alternative solution, but I'm not a big fan of it yet, so this simpler approach is a placeholder for now.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
---
 include/linux/vrange.h | 2 ++
 mm/mmap.c              | 5 +++++
 mm/vrange.c            | 8 ++++++++
 3 files changed, 15 insertions(+)
include/linux/vrange.h +2 -0
index 2b96ee1..ef153c8 100644
@@ -36,6 +36,8 @@ static inline int vrange_type(struct vrange *vrange)
 	return vrange->owner->type;
 }
 
+extern int vrange_clear(struct vrange_root *vroot,
+				unsigned long start, unsigned long end);
 extern void vrange_root_cleanup(struct vrange_root *vroot);
 extern int vrange_fork(struct mm_struct *new,
 					struct mm_struct *old);
mm/mmap.c +5 -0
index f9c97d1..ed7056f 100644
@@ -36,6 +36,7 @@
 #include <linux/sched/sysctl.h>
 #include <linux/notifier.h>
 #include <linux/memory.h>
+#include <linux/vrange.h>
 
 #include <asm/uaccess.h>
 #include <asm/cacheflush.h>
@@ -1502,6 +1503,10 @@ unsigned long mmap_region(struct file *file, unsigned long addr,
 	/* Clear old maps */
 	error = -ENOMEM;
 munmap_back:
+
+	/* zap any volatile ranges */
+	vrange_clear(&mm->vroot, addr, addr + len);
+
 	if (find_vma_links(mm, addr, addr + len, &prev, &rb_link, &rb_parent)) {
 		if (do_munmap(mm, addr, len))
 			return -ENOMEM;
mm/vrange.c +8 -0
index 4ddcc3e9..f2d1588 100644
@@ -166,6 +166,14 @@ static int vrange_remove(struct vrange_root *vroot,
 	return 0;
 }
 
+int vrange_clear(struct vrange_root *vroot,
+					unsigned long start, unsigned long end)
+{
+	int purged;
+
+	return vrange_remove(vroot, start, end - 1, &purged);
+}
+
 void vrange_root_cleanup(struct vrange_root *vroot)
 {
 	struct vrange *range;
Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#4 lore raw

PATCH 4/14 vrange: Add support for volatile ranges on file mappings

In reply to John Stultz

Like with the mm struct, this patch add basic support for volatile ranges on file address_space structures. This allows for volatile ranges to be set on mmapped files that can be shared between processes.

The semantics on the volatile range sharing is that the volatility is shared, just as the data is shared. Thus if one process marks the range as volatile, the data is volatile in all processes that have those pages mapped.

It is advised that processes coordinate when using volatile ranges on shared mappings (much as they must coordinate when writing to shared data).

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
---
 fs/inode.c         | 4 ++++
 include/linux/fs.h | 4 ++++
 2 files changed, 8 insertions(+)
fs/inode.c +4 -0
index d6dfb09..5364f91 100644
@@ -17,6 +17,7 @@
 #include <linux/prefetch.h>
 #include <linux/buffer_head.h> /* for inode_has_buffers */
 #include <linux/ratelimit.h>
+#include <linux/vrange.h>
 #include "internal.h"
 
 /*
@@ -352,6 +353,7 @@ void address_space_init_once(struct address_space *mapping)
 	spin_lock_init(&mapping->private_lock);
 	mapping->i_mmap = RB_ROOT;
 	INIT_LIST_HEAD(&mapping->i_mmap_nonlinear);
+	vrange_root_init(&mapping->vroot, VRANGE_FILE, mapping);
 }
 EXPORT_SYMBOL(address_space_init_once);
 
@@ -1419,6 +1421,8 @@ static void iput_final(struct inode *inode)
 		inode_lru_list_del(inode);
 	spin_unlock(&inode->i_lock);
 
+	vrange_root_cleanup(&inode->i_mapping->vroot);
+
 	evict(inode);
 }
 
include/linux/fs.h +4 -0
index 9818747..6ec2953 100644
@@ -28,6 +28,7 @@
 #include <linux/lockdep.h>
 #include <linux/percpu-rwsem.h>
 #include <linux/blk_types.h>
+#include <linux/vrange_types.h>
 
 #include <asm/byteorder.h>
 #include <uapi/linux/fs.h>
@@ -413,6 +414,9 @@ struct address_space {
 	struct rb_root		i_mmap;		/* tree of private and shared mappings */
 	struct list_head	i_mmap_nonlinear;/*list VM_NONLINEAR mappings */
 	struct mutex		i_mmap_mutex;	/* protect tree, count, list */
+#ifdef CONFIG_MMU
+	struct vrange_root	vroot;
+#endif
 	/* Protected by tree_lock together with the radix tree */
 	unsigned long		nrpages;	/* number of total pages */
 	pgoff_t			writeback_index;/* writeback starts here */
Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#5 lore raw

PATCH 5/14 vrange: Add new vrange(2) system call

In reply to John Stultz

From: Minchan Kim <[email protected]>

This patch adds new system call sys_vrange.

NAME
vrange - Mark or unmark range of memory as volatile

SYNOPSIS

	int vrange(unsigned_long start, size_t length, int mode,
			 int *purged);

DESCRIPTION
Applications can use vrange(2) to advise the kernel how it should handle paging I/O in this VM area. The idea is to help the kernel discard pages of vrange instead of reclaiming when memory pressure happens. It means kernel doesn't discard any pages of vrange if there is no memory pressure.

mode:
VRANGE_VOLATILE
hint to kernel so VM can discard in vrange pages when memory pressure happens.
VRANGE_NONVOLATILE
hint to kernel so VM doesn't discard vrange pages any more.

If user try to access purged memory without VRANGE_NOVOLATILE call, he can encounter SIGBUS if the page was discarded by kernel.

purged: Pointer to an integer which will return 1 if mode == VRANGE_NONVOLATILE and any page in the affected range was purged. If purged returns zero during a mode == VRANGE_NONVOLATILE call, it means all of the pages in the range are intact.

RETURN VALUE
On success vrange returns the number of bytes marked or unmarked. Similar to write(), it may return fewer bytes then specified if it ran into a problem.

If an error is returned, no changes were made.

ERRORS
EINVAL This error can occur for the following reasons:
* The value length is negative or not page size units.
* addr is not page-aligned
* mode not a valid value.

ENOMEM Not enough memory

EFAULT purged pointer is invalid

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
---
 arch/x86/syscalls/syscall_64.tbl       |   1 +
 include/linux/syscalls.h               |   2 +
 include/uapi/asm-generic/mman-common.h |   3 +
 kernel/sys_ni.c                        |   1 +
 mm/vrange.c                            | 164 +++++++++++++++++++++++++++++++++
 5 files changed, 171 insertions(+)
arch/x86/syscalls/syscall_64.tbl +1 -0
index 38ae65d..dc332bd 100644
@@ -320,6 +320,7 @@
 311	64	process_vm_writev	sys_process_vm_writev
 312	common	kcmp			sys_kcmp
 313	common	finit_module		sys_finit_module
+314	common	vrange			sys_vrange
 
 #
 # x32-specific system call numbers start at 512 to avoid cache impact
include/linux/syscalls.h +2 -0
index 84662ec..0997165 100644
@@ -846,4 +846,6 @@ asmlinkage long sys_process_vm_writev(pid_t pid,
 asmlinkage long sys_kcmp(pid_t pid1, pid_t pid2, int type,
 			 unsigned long idx1, unsigned long idx2);
 asmlinkage long sys_finit_module(int fd, const char __user *uargs, int flags);
+asmlinkage long sys_vrange(unsigned long start, size_t len, int mode,
+		int __user *purged);
 #endif
include/uapi/asm-generic/mman-common.h +3 -0
index 4164529..9be120b 100644
@@ -66,4 +66,7 @@
 #define MAP_HUGE_SHIFT	26
 #define MAP_HUGE_MASK	0x3f
 
+#define VRANGE_VOLATILE		0	/* unpin pages so VM can discard them */
+#define VRANGE_NONVOLATILE	1	/* pin pages so VM can't discard them */
+
 #endif /* __ASM_GENERIC_MMAN_COMMON_H */
kernel/sys_ni.c +1 -0
index 7078052..f40070e 100644
@@ -175,6 +175,7 @@ cond_syscall(sys_mremap);
 cond_syscall(sys_remap_file_pages);
 cond_syscall(compat_sys_move_pages);
 cond_syscall(compat_sys_migrate_pages);
+cond_syscall(sys_vrange);
 
 /* block-layer dependent */
 cond_syscall(sys_bdflush);
mm/vrange.c +164 -0

173 lines not on this page · raw message

Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#6 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 7 lines · John Stultz

From: Minchan Kim <[email protected]>

This patch adds new system call sys_vrange.

NAME
vrange - Mark or unmark range of memory as volatile

vrange() is about as nondescriptive as one can get -- there is exactly one letter that has any connection with that this does.

Quoted 19 lines

SYNOPSIS

	int vrange(unsigned_long start, size_t length, int mode,
			 int *purged);

DESCRIPTION
Applications can use vrange(2) to advise the kernel how it should handle paging I/O in this VM area. The idea is to help the kernel discard pages of vrange instead of reclaiming when memory pressure happens. It means kernel doesn't discard any pages of vrange if there is no memory pressure.

mode:
VRANGE_VOLATILE
hint to kernel so VM can discard in vrange pages when memory pressure happens.
VRANGE_NONVOLATILE
hint to kernel so VM doesn't discard vrange pages any more.

Quoted 8 lines

If user try to access purged memory without VRANGE_NOVOLATILE call, he can encounter SIGBUS if the page was discarded by kernel.

purged: Pointer to an integer which will return 1 if mode == VRANGE_NONVOLATILE and any page in the affected range was purged. If purged returns zero during a mode == VRANGE_NONVOLATILE call, it means all of the pages in the range are intact.

I'm a bit confused about the "purged"

Quoted 8 lines

From an earlier version of the patch:

- What's different with madvise(DONTNEED)?

System call semantic

DONTNEED makes sure user always can see zero-fill pages after he calls madvise while vrange can see data or encounter SIGBUS.

This difference doesn't seem to be a huge one. The other one seems to be the blocking status of MADV_DONTNEED, which perhaps may be better handled by adding an option (MADV_LAZY) perhaps?

That way we would have lazy vs. immediate, and zero versus SIGBUS.

I see from the change history of the patch that this was an madvise() at some point, but was changed into a separate system call at some point, does anyone remember why that was? A quick look through my LKML archives doesn't really make it clear.

-hpa

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#7 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 10 lines · H. Peter Anvin
Quoted 7 lines · John Stultz

From: Minchan Kim <[email protected]>

This patch adds new system call sys_vrange.

NAME
vrange - Mark or unmark range of memory as volatile

vrange() is about as nondescriptive as one can get -- there is exactly one letter that has any connection with that this does.

Hrm. Any suggestions? Would volatile_range() be better?

Quoted 36 lines
Quoted 20 lines

SYNOPSIS

	int vrange(unsigned_long start, size_t length, int mode,
			 int *purged);

DESCRIPTION
Applications can use vrange(2) to advise the kernel how it should handle paging I/O in this VM area. The idea is to help the kernel discard pages of vrange instead of reclaiming when memory pressure happens. It means kernel doesn't discard any pages of vrange if there is no memory pressure.

mode:
VRANGE_VOLATILE
hint to kernel so VM can discard in vrange pages when memory pressure happens.
VRANGE_NONVOLATILE
hint to kernel so VM doesn't discard vrange pages any more.

Quoted 8 lines

If user try to access purged memory without VRANGE_NOVOLATILE call, he can encounter SIGBUS if the page was discarded by kernel.

purged: Pointer to an integer which will return 1 if mode == VRANGE_NONVOLATILE and any page in the affected range was purged. If purged returns zero during a mode == VRANGE_NONVOLATILE call, it means all of the pages in the range are intact.

I'm a bit confused about the "purged"

From an earlier version of the patch:

Quoted 4 lines

- What's different with madvise(DONTNEED)?

System call semantic

Quoted 7 lines
Quoted 2 lines

DONTNEED makes sure user always can see zero-fill pages after he calls madvise while vrange can see data or encounter SIGBUS.

This difference doesn't seem to be a huge one. The other one seems to be the blocking status of MADV_DONTNEED, which perhaps may be better handled by adding an option (MADV_LAZY) perhaps?

That way we would have lazy vs. immediate, and zero versus SIGBUS.

And some sort of lazy-cancling call as well.

Quoted 5 lines

I see from the change history of the patch that this was an madvise() at some point, but was changed into a separate system call at some point, does anyone remember why that was? A quick look through my LKML archives doesn't really make it clear.

The reason we can't use madvise, is that to properly handle error cases and report the pruge state, we need an extra argument.

In much earlier versions, we just returned an error when setting NONVOLATILE if the data was purged. However, since we have to possibly do allocations when marking a range as non-volatile, we needed a way to properly handle that allocation failing. We can't just return ENOMEM, as we may have already marked purged memory as non-volatile.

Thus, that's why with vrange, we return the number of bytes modified, along with the purge state. That way, if an error does occur we can return the purge state of the bytes successfully modified, and only return an error if nothing was changed, much like when a write fails.

thanks
-john

#8 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 15 lines · John Stultz
Quoted 5 lines

I see from the change history of the patch that this was an madvise() at some point, but was changed into a separate system call at some point, does anyone remember why that was? A quick look through my LKML archives doesn't really make it clear.

The reason we can't use madvise, is that to properly handle error cases and report the pruge state, we need an extra argument.

In much earlier versions, we just returned an error when setting NONVOLATILE if the data was purged. However, since we have to possibly do allocations when marking a range as non-volatile, we needed a way to properly handle that allocation failing. We can't just return ENOMEM, as we may have already marked purged memory as non-volatile.

Quoted 5 lines

Thus, that's why with vrange, we return the number of bytes modified, along with the purge state. That way, if an error does occur we can return the purge state of the bytes successfully modified, and only return an error if nothing was changed, much like when a write fails.

I am not clear at all what the "purge state" is in this case.

-hpa

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#9 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 14 lines · H. Peter Anvin
Quoted 13 lines · John Stultz
Quoted 4 lines

I see from the change history of the patch that this was an madvise() at some point, but was changed into a separate system call at some point, does anyone remember why that was? A quick look through my LKML archives doesn't really make it clear.

The reason we can't use madvise, is that to properly handle error cases and report the pruge state, we need an extra argument.

In much earlier versions, we just returned an error when setting NONVOLATILE if the data was purged. However, since we have to possibly do allocations when marking a range as non-volatile, we needed a way to properly handle that allocation failing. We can't just return ENOMEM, as we may have already marked purged memory as non-volatile.

Quoted 6 lines
Quoted 5 lines

Thus, that's why with vrange, we return the number of bytes modified, along with the purge state. That way, if an error does occur we can return the purge state of the bytes successfully modified, and only return an error if nothing was changed, much like when a write fails.

I am not clear at all what the "purge state" is in this case.

You mark a chunk of memory as volatile, then at some point later, mark its as non-volatile. The purge state tells you if the memory is still there, or if we threw it out due to memory pressure. This lets the application regnerate the purged data before continuing on.

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#10 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 6 lines · John Stultz

You mark a chunk of memory as volatile, then at some point later, mark its as non-volatile. The purge state tells you if the memory is still there, or if we threw it out due to memory pressure. This lets the application regnerate the purged data before continuing on.

And wouldn't this apply to MADV_DONTNEED just as well? Perhaps what we should do is an enhanced madvise() call?

-hpa

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#11 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 8 lines · H. Peter Anvin
Quoted 5 lines · John Stultz

You mark a chunk of memory as volatile, then at some point later, mark its as non-volatile. The purge state tells you if the memory is still there, or if we threw it out due to memory pressure. This lets the application regnerate the purged data before continuing on.

And wouldn't this apply to MADV_DONTNEED just as well? Perhaps what we should do is an enhanced madvise() call?

Well, I think MADV_DONTNEED doesn't *have* do to anything at all. Its advisory after all. So it may immediately wipe out any data, but it may not.

Those advisory semantics work fine w/ VRANGE_VOLATILE. However, VRANGE_NONVOLATILE is not quite advisory, its telling the system that it requires the memory at the specified range to not be volatile, and we need to correctly inform userland how much was changed and if any of the memory we did change to non-volatile was purged since being set volatile.

In that way it is sort of different from madvise. Some sort of an madvise2 could be done, but then the extra purge state argument would be oddly defined for any other mode.

Is your main concern here just wanting to have a zero-fill mode with volatile ranges? Or do you really want to squeeze this in to the madvise call interface?

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#12 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 12 lines · John Stultz
Quoted 3 lines
Quoted 1 line

And wouldn't this apply to MADV_DONTNEED just as well? Perhaps what we should do is an enhanced madvise() call?

Well, I think MADV_DONTNEED doesn't *have* do to anything at all. Its advisory after all. So it may immediately wipe out any data, but it may not.

Those advisory semantics work fine w/ VRANGE_VOLATILE. However, VRANGE_NONVOLATILE is not quite advisory, its telling the system that it requires the memory at the specified range to not be volatile, and we need to correctly inform userland how much was changed and if any of the memory we did change to non-volatile was purged since being set volatile.

Quoted 7 lines

In that way it is sort of different from madvise. Some sort of an madvise2 could be done, but then the extra purge state argument would be oddly defined for any other mode.

Is your main concern here just wanting to have a zero-fill mode with volatile ranges? Or do you really want to squeeze this in to the madvise call interface?

The point is that MADV_DONTNEED is very similar in that sense, especially if allowed to be lazy. It makes a lot of sense to permit both scrubbing modes orthogonally.

The point you're making has to do with withdrawal of permission to flush on demand, which is a result of having the lazy mode (ongoing permission) and having to be able to withdraw such permission.

	-0hpa

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#13 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Hello Peter,

Quoted 21 lines · H. Peter Anvin
Quoted 12 lines · John Stultz
Quoted 3 lines
Quoted 1 line

And wouldn't this apply to MADV_DONTNEED just as well? Perhaps what we should do is an enhanced madvise() call?

Well, I think MADV_DONTNEED doesn't *have* do to anything at all. Its advisory after all. So it may immediately wipe out any data, but it may not.

Those advisory semantics work fine w/ VRANGE_VOLATILE. However, VRANGE_NONVOLATILE is not quite advisory, its telling the system that it requires the memory at the specified range to not be volatile, and we need to correctly inform userland how much was changed and if any of the memory we did change to non-volatile was purged since being set volatile.

Quoted 7 lines

In that way it is sort of different from madvise. Some sort of an madvise2 could be done, but then the extra purge state argument would be oddly defined for any other mode.

Is your main concern here just wanting to have a zero-fill mode with volatile ranges? Or do you really want to squeeze this in to the madvise call interface?

Quoted 7 lines

The point is that MADV_DONTNEED is very similar in that sense, especially if allowed to be lazy. It makes a lot of sense to permit both scrubbing modes orthogonally.

The point you're making has to do with withdrawal of permission to flush on demand, which is a result of having the lazy mode (ongoing permission) and having to be able to withdraw such permission.

I'm sorry I could not understand what you wanted to say. Could you elaborate a bit?

Thanks.

Quoted 9 lines
	-0hpa

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#14 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 26 lines · Minchan Kim

Hello Peter,

Quoted 23 lines · H. Peter Anvin
Quoted 11 lines · John Stultz
Quoted 2 lines

And wouldn't this apply to MADV_DONTNEED just as well? Perhaps what we should do is an enhanced madvise() call?

Well, I think MADV_DONTNEED doesn't *have* do to anything at all. Its advisory after all. So it may immediately wipe out any data, but it may not.

Those advisory semantics work fine w/ VRANGE_VOLATILE. However, VRANGE_NONVOLATILE is not quite advisory, its telling the system that it requires the memory at the specified range to not be volatile, and we need to correctly inform userland how much was changed and if any of the memory we did change to non-volatile was purged since being set volatile.

Quoted 7 lines

In that way it is sort of different from madvise. Some sort of an madvise2 could be done, but then the extra purge state argument would be oddly defined for any other mode.

Is your main concern here just wanting to have a zero-fill mode with volatile ranges? Or do you really want to squeeze this in to the madvise call interface?

The point is that MADV_DONTNEED is very similar in that sense, especially if allowed to be lazy. It makes a lot of sense to permit both scrubbing modes orthogonally.

Quoted 5 lines
Quoted 3 lines

The point you're making has to do with withdrawal of permission to flush on demand, which is a result of having the lazy mode (ongoing permission) and having to be able to withdraw such permission.

I'm sorry I could not understand what you wanted to say. Could you elaborate a bit?

My understanding of his point is that VRANGE_VOLATILE is like a lazy MADV_DONTNEED (with sigbus, rather then zero fill on fault), suggests that we should find a way to have VRANGE_VOLATILE be something like MADV_DONTNEED|MADV_LAZY|MADV_SIGBUS_FAULT, instead of adding a new syscall. This would provide more options, since one could instead just do MADV_DONTNEED|MADV_LAZY if they wanted zero-fill faults.

And indeed, for the VRANGE_VOLATILE case, we could do something like that, but the unresolved problem I see is that that we still need to handle the VRANGE_NONVOLATILE case, and the madvise() interface doesn't seem to accomodate the needed semantics well.

thanks
-john

#15 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 30 lines · John Stultz
Quoted 29 lines · Minchan Kim

Hello Peter,

Quoted 26 lines · H. Peter Anvin
Quoted 11 lines · John Stultz
Quoted 2 lines

And wouldn't this apply to MADV_DONTNEED just as well? Perhaps what we should do is an enhanced madvise() call?

Well, I think MADV_DONTNEED doesn't *have* do to anything at all. Its advisory after all. So it may immediately wipe out any data, but it may not.

Those advisory semantics work fine w/ VRANGE_VOLATILE. However, VRANGE_NONVOLATILE is not quite advisory, its telling the system that it requires the memory at the specified range to not be volatile, and we need to correctly inform userland how much was changed and if any of the memory we did change to non-volatile was purged since being set volatile.

Quoted 7 lines

In that way it is sort of different from madvise. Some sort of an madvise2 could be done, but then the extra purge state argument would be oddly defined for any other mode.

Is your main concern here just wanting to have a zero-fill mode with volatile ranges? Or do you really want to squeeze this in to the madvise call interface?

The point is that MADV_DONTNEED is very similar in that sense, especially if allowed to be lazy. It makes a lot of sense to permit both scrubbing modes orthogonally.

The point you're making has to do with withdrawal of permission to flush on demand, which is a result of having the lazy mode (ongoing permission) and having to be able to withdraw such permission.

Quoted 8 lines
Quoted 2 lines

I'm sorry I could not understand what you wanted to say. Could you elaborate a bit?

My understanding of his point is that VRANGE_VOLATILE is like a lazy MADV_DONTNEED (with sigbus, rather then zero fill on fault), suggests that we should find a way to have VRANGE_VOLATILE be something like MADV_DONTNEED|MADV_LAZY|MADV_SIGBUS_FAULT, instead of adding a new syscall. This would provide more options, since one could instead just do MADV_DONTNEED|MADV_LAZY if they wanted zero-fill faults.

Hmm, actually, I have thought VRANGE_SIGBUS option because Address/Thread sanitizer people wanted it as you know and someone might want it, too.

I agree it's orthogonal but not sure MADV_LAZY and MADV_SIGBUS_FAULT can be used for other combination of advise except MADV_DONTNEED so it might confuse userland without benefit.

Quoted 5 lines

And indeed, for the VRANGE_VOLATILE case, we could do something like that, but the unresolved problem I see is that that we still need to handle the VRANGE_NONVOLATILE case, and the madvise() interface doesn't seem to accomodate the needed semantics well.

VRANGE_VOLATILE case could be a problem. In my mind, I had an idea to return purged state when we call vrange(VRANGE_VOLATILE) because kernel could purge them as soon as vrange(VRANGE_VOLATILE) called if memory is really tight so userland can notice "purging" earlier and kernel can discard them more efficiently.

Quoted 10 lines

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#16 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 53 lines · Minchan Kim
Quoted 30 lines · John Stultz
Quoted 29 lines · Minchan Kim

Hello Peter,

Quoted 26 lines · H. Peter Anvin
Quoted 11 lines · John Stultz
Quoted 2 lines

And wouldn't this apply to MADV_DONTNEED just as well? Perhaps what we should do is an enhanced madvise() call?

Well, I think MADV_DONTNEED doesn't *have* do to anything at all. Its advisory after all. So it may immediately wipe out any data, but it may not.

Those advisory semantics work fine w/ VRANGE_VOLATILE. However, VRANGE_NONVOLATILE is not quite advisory, its telling the system that it requires the memory at the specified range to not be volatile, and we need to correctly inform userland how much was changed and if any of the memory we did change to non-volatile was purged since being set volatile.

Quoted 7 lines

In that way it is sort of different from madvise. Some sort of an madvise2 could be done, but then the extra purge state argument would be oddly defined for any other mode.

Is your main concern here just wanting to have a zero-fill mode with volatile ranges? Or do you really want to squeeze this in to the madvise call interface?

The point is that MADV_DONTNEED is very similar in that sense, especially if allowed to be lazy. It makes a lot of sense to permit both scrubbing modes orthogonally.

The point you're making has to do with withdrawal of permission to flush on demand, which is a result of having the lazy mode (ongoing permission) and having to be able to withdraw such permission.

Quoted 8 lines
Quoted 2 lines

I'm sorry I could not understand what you wanted to say. Could you elaborate a bit?

My understanding of his point is that VRANGE_VOLATILE is like a lazy MADV_DONTNEED (with sigbus, rather then zero fill on fault), suggests that we should find a way to have VRANGE_VOLATILE be something like MADV_DONTNEED|MADV_LAZY|MADV_SIGBUS_FAULT, instead of adding a new syscall. This would provide more options, since one could instead just do MADV_DONTNEED|MADV_LAZY if they wanted zero-fill faults.

Hmm, actually, I have thought VRANGE_SIGBUS option because Address/Thread sanitizer people wanted it as you know and someone might want it, too.

I agree it's orthogonal but not sure MADV_LAZY and MADV_SIGBUS_FAULT can be used for other combination of advise except MADV_DONTNEED so it might confuse userland without benefit.

Quoted 5 lines

And indeed, for the VRANGE_VOLATILE case, we could do something like that, but the unresolved problem I see is that that we still need to handle the VRANGE_NONVOLATILE case, and the madvise() interface doesn't seem to accomodate the needed semantics well.

Quoted 6 lines

VRANGE_VOLATILE case could be a problem. In my mind, I had an idea to return purged state when we call vrange(VRANGE_VOLATILE) because kernel could purge them as soon as vrange(VRANGE_VOLATILE) called if memory is really tight so userland can notice "purging" earlier and kernel can discard them more efficiently.

And we should return the number of bytes marked but madvise returns error.

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#17 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

In reply to Minchan Kim
Quoted 5 lines · Minchan Kim
Quoted 5 lines

The point is that MADV_DONTNEED is very similar in that sense, especially if allowed to be lazy. It makes a lot of sense to permit both scrubbing modes orthogonally.

Quoted 7 lines
Quoted 3 lines

The point you're making has to do with withdrawal of permission to flush on demand, which is a result of having the lazy mode (ongoing permission) and having to be able to withdraw such permission.

I'm sorry I could not understand what you wanted to say. Could you elaborate a bit?

Basically, you need this because of MADV_LAZY or the equivalent, so it would be applicable to a similar variant of madvise().

As such I would suggest that an madvise4() call would be appropriate.

-hpa

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#18 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 14 lines · H. Peter Anvin
Quoted 5 lines · Minchan Kim
Quoted 5 lines

The point is that MADV_DONTNEED is very similar in that sense, especially if allowed to be lazy. It makes a lot of sense to permit both scrubbing modes orthogonally.

Quoted 7 lines
Quoted 3 lines

The point you're making has to do with withdrawal of permission to flush on demand, which is a result of having the lazy mode (ongoing permission) and having to be able to withdraw such permission.

I'm sorry I could not understand what you wanted to say. Could you elaborate a bit?

Quoted 6 lines

Basically, you need this because of MADV_LAZY or the equivalent, so it would be applicable to a similar variant of madvise().

As such I would suggest that an madvise4() call would be appropriate.

-hpa

Maybe, int madvise5(addr, length, MADV_DONTNEED|MADV_LAZY|MADV_SIGBUS,

        &purged, &ret);

Another reason to make it hard is that madvise(2) is tight coupled with with vmas split/merge. It needs mmap_sem's write-side lock and it hurt anon-vrange test performance much heavily and userland might want to make volatile range with small unit like "page size" so it's undesireable to make it with vma. Then, we should filter out to avoid vma split/merge in implementation if only MADV_LAZY case? Doable but it could make code complicated and lost consistency with other variant of madvise.

I think it would be better to implement MADV_FREE if you really want MADV_LAZY(http://www.unix.com/man-page/FreeBSD/2/madvise/) which is differnt with volatile range and vrange is more advanced function, IMHO because MADV_FREE's cost would be proporational to range size due to page table/page descriptor operations.

Signature · 2 lines
Kind regards,
Minchan Kim
#19 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 10 lines

Maybe, int madvise5(addr, length, MADV_DONTNEED|MADV_LAZY|MADV_SIGBUS,

         &purged, &ret);

Another reason to make it hard is that madvise(2) is tight coupled with with vmas split/merge. It needs mmap_sem's write-side lock and it hurt anon-vrange test performance much heavily and userland might want to make volatile range with small unit like "page size" so it's undesireable to make it with vma. Then, we should filter out to avoid vma split/merge in implementation if only MADV_LAZY case? Doable but it could make code complicated and lost consistency with other variant of madvise.

I haven't seen your performance test result. Could please point out URLs?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#20 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Hi KOSAKI,

Quoted 11 lines · KOSAKI Motohiro
Quoted 10 lines

Maybe, int madvise5(addr, length, MADV_DONTNEED|MADV_LAZY|MADV_SIGBUS,

        &purged, &ret);

Another reason to make it hard is that madvise(2) is tight coupled with with vmas split/merge. It needs mmap_sem's write-side lock and it hurt anon-vrange test performance much heavily and userland might want to make volatile range with small unit like "page size" so it's undesireable to make it with vma. Then, we should filter out to avoid vma split/merge in implementation if only MADV_LAZY case? Doable but it could make code complicated and lost consistency with other variant of madvise.

Quoted 1 line

I haven't seen your performance test result. Could please point out URLs?

https://lkml.org/lkml/2013/3/12/105

Signature · 2 lines
Kind regards,
Minchan Kim
#21 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 14 lines · Minchan Kim

Hi KOSAKI,

Quoted 11 lines · KOSAKI Motohiro
Quoted 10 lines

Maybe, int madvise5(addr, length, MADV_DONTNEED|MADV_LAZY|MADV_SIGBUS,

         &purged, &ret);

Another reason to make it hard is that madvise(2) is tight coupled with with vmas split/merge. It needs mmap_sem's write-side lock and it hurt anon-vrange test performance much heavily and userland might want to make volatile range with small unit like "page size" so it's undesireable to make it with vma. Then, we should filter out to avoid vma split/merge in implementation if only MADV_LAZY case? Doable but it could make code complicated and lost consistency with other variant of madvise.

Quoted 3 lines
Quoted 1 line

I haven't seen your performance test result. Could please point out URLs?

https://lkml.org/lkml/2013/3/12/105

It's not comparison with and without vma merge. I'm interest how much benefit vmas operation avoiding have.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#22 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 15 lines · KOSAKI Motohiro
Quoted 14 lines · Minchan Kim

Hi KOSAKI,

Quoted 11 lines · KOSAKI Motohiro
Quoted 10 lines

Maybe, int madvise5(addr, length, MADV_DONTNEED|MADV_LAZY|MADV_SIGBUS,

        &purged, &ret);

Another reason to make it hard is that madvise(2) is tight coupled with with vmas split/merge. It needs mmap_sem's write-side lock and it hurt anon-vrange test performance much heavily and userland might want to make volatile range with small unit like "page size" so it's undesireable to make it with vma. Then, we should filter out to avoid vma split/merge in implementation if only MADV_LAZY case? Doable but it could make code complicated and lost consistency with other variant of madvise.

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

I haven't seen your performance test result. Could please point out URLs?

https://lkml.org/lkml/2013/3/12/105

It's not comparison with and without vma merge. I'm interest how much benefit vmas operation avoiding have.

I had an number but lost it so I should set up it in my KVM machine :( And I needed old kernel 3.7.0 for testing vma-based approach.

DRAM:2G, CPU : 12

kernel 3.7.0

jemalloc: 20527 records/s
jemalloc vma based approach : 5360 records/s

vrange call made worse because every thread stuck with mmap_sem.

kernel 3.11.0

jemalloc: 21176 records/s
jemalloc vroot tree approach: 103637 records/s

It could enhance 5 times.

Signature · 2 lines
Kind regards,
Minchan Kim
#23 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 33 lines · Minchan Kim
Quoted 15 lines · KOSAKI Motohiro
Quoted 14 lines · Minchan Kim

Hi KOSAKI,

Quoted 11 lines · KOSAKI Motohiro
Quoted 10 lines

Maybe, int madvise5(addr, length, MADV_DONTNEED|MADV_LAZY|MADV_SIGBUS,

        &purged, &ret);

Another reason to make it hard is that madvise(2) is tight coupled with with vmas split/merge. It needs mmap_sem's write-side lock and it hurt anon-vrange test performance much heavily and userland might want to make volatile range with small unit like "page size" so it's undesireable to make it with vma. Then, we should filter out to avoid vma split/merge in implementation if only MADV_LAZY case? Doable but it could make code complicated and lost consistency with other variant of madvise.

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

I haven't seen your performance test result. Could please point out URLs?

https://lkml.org/lkml/2013/3/12/105

It's not comparison with and without vma merge. I'm interest how much benefit vmas operation avoiding have.

I had an number but lost it so I should set up it in my KVM machine :( And I needed old kernel 3.7.0 for testing vma-based approach.

DRAM:2G, CPU : 12

kernel 3.7.0

jemalloc: 20527 records/s
jemalloc vma based approach : 5360 records/s

Quoted 8 lines

vrange call made worse because every thread stuck with mmap_sem.

kernel 3.11.0

jemalloc: 21176 records/s
jemalloc vroot tree approach: 103637 records/s

It could enhance 5 times.

And please keep in mind that vrange's user might want to small vrange like PAGE_SIZE. If we go with vma-based approach, we would consume memory with lots of vm_area_struct.

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#24 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

In reply to John Stultz

Hello, John and Peter

Quoted 78 lines · John Stultz
Quoted 10 lines · H. Peter Anvin
Quoted 7 lines · John Stultz

From: Minchan Kim <[email protected]>

This patch adds new system call sys_vrange.

NAME
vrange - Mark or unmark range of memory as volatile

vrange() is about as nondescriptive as one can get -- there is exactly one letter that has any connection with that this does.

Hrm. Any suggestions? Would volatile_range() be better?

Quoted 36 lines
Quoted 20 lines

SYNOPSIS

	int vrange(unsigned_long start, size_t length, int mode,
			 int *purged);

DESCRIPTION
Applications can use vrange(2) to advise the kernel how it should handle paging I/O in this VM area. The idea is to help the kernel discard pages of vrange instead of reclaiming when memory pressure happens. It means kernel doesn't discard any pages of vrange if there is no memory pressure.

mode:
VRANGE_VOLATILE
hint to kernel so VM can discard in vrange pages when memory pressure happens.
VRANGE_NONVOLATILE
hint to kernel so VM doesn't discard vrange pages any more.

Quoted 8 lines

If user try to access purged memory without VRANGE_NOVOLATILE call, he can encounter SIGBUS if the page was discarded by kernel.

purged: Pointer to an integer which will return 1 if mode == VRANGE_NONVOLATILE and any page in the affected range was purged. If purged returns zero during a mode == VRANGE_NONVOLATILE call, it means all of the pages in the range are intact.

I'm a bit confused about the "purged"

From an earlier version of the patch:

Quoted 4 lines

- What's different with madvise(DONTNEED)?

System call semantic

Quoted 7 lines
Quoted 2 lines

DONTNEED makes sure user always can see zero-fill pages after he calls madvise while vrange can see data or encounter SIGBUS.

This difference doesn't seem to be a huge one. The other one seems to be the blocking status of MADV_DONTNEED, which perhaps may be better handled by adding an option (MADV_LAZY) perhaps?

That way we would have lazy vs. immediate, and zero versus SIGBUS.

And some sort of lazy-cancling call as well.

Quoted 5 lines

I see from the change history of the patch that this was an madvise() at some point, but was changed into a separate system call at some point, does anyone remember why that was? A quick look through my LKML archives doesn't really make it clear.

The reason we can't use madvise, is that to properly handle error cases and report the pruge state, we need an extra argument.

In much earlier versions, we just returned an error when setting NONVOLATILE if the data was purged. However, since we have to possibly do allocations when marking a range as non-volatile, we needed a way to properly handle that allocation failing. We can't just return ENOMEM, as we may have already marked purged memory as non-volatile.

Quoted 4 lines

Thus, that's why with vrange, we return the number of bytes modified, along with the purge state. That way, if an error does occur we can return the purge state of the bytes successfully modified, and only return an error if nothing was changed, much like when a write fails.

As well, we might need addtional argument VRANGE_FULL/VRANGE_PARTIAL for vrange system call. I discussed it long time ago but omitted it for early easy review phase. It is requested by Mozilla fork and of course I think it makes sense to me.

https://lkml.org/lkml/2013/3/22/20

In short, if you mark a range with VRANGE_FULL, kernel can discard all of pages within the range if memory is tight while kernel can discard part of pages in the vrange if you mark the range with VRANGE_PARTIAL.

Quoted 11 lines

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Signature · 2 lines
Kind regards,
Minchan Kim
#25 lore raw

Re: [PATCH 05/14] vrange: Add new vrange(2) system call

Quoted 26 lines · Minchan Kim

Hello, John and Peter

Quoted 14 lines · John Stultz
Quoted 4 lines · H. Peter Anvin

I see from the change history of the patch that this was an madvise() at some point, but was changed into a separate system call at some point, does anyone remember why that was? A quick look through my LKML archives doesn't really make it clear.

The reason we can't use madvise, is that to properly handle error cases and report the pruge state, we need an extra argument.

In much earlier versions, we just returned an error when setting NONVOLATILE if the data was purged. However, since we have to possibly do allocations when marking a range as non-volatile, we needed a way to properly handle that allocation failing. We can't just return ENOMEM, as we may have already marked purged memory as non-volatile.

Quoted 4 lines

Thus, that's why with vrange, we return the number of bytes modified, along with the purge state. That way, if an error does occur we can return the purge state of the bytes successfully modified, and only return an error if nothing was changed, much like when a write fails.

As well, we might need addtional argument VRANGE_FULL/VRANGE_PARTIAL for vrange system call. I discussed it long time ago but omitted it for early easy review phase. It is requested by Mozilla fork and of course I think it makes sense to me.

Quoted 5 lines

https://lkml.org/lkml/2013/3/22/20

In short, if you mark a range with VRANGE_FULL, kernel can discard all of pages within the range if memory is tight while kernel can discard part of pages in the vrange if you mark the range with VRANGE_PARTIAL.

Yea, I'm still not particularly fond of userland being able to specify the purging semantics, but as we discussed earlier, this can be debated in finer detail as an extension to the merged interface. :)

thanks
-john

#26 lore raw

PATCH 6/14 vrange: Add basic functions to purge volatile pages

In reply to John Stultz

From: Minchan Kim <[email protected]>

This patch adds discard_vpage and related functions to purge anonymous and file volatile pages.

It is in preparation for purging volatile pages when memory is tight. The logic to trigger purge volatile pages will be introduced in the next patch.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
[jstultz: Reworked to add purging of file pages, commit log tweaks]
---
 include/linux/vrange.h |   9 +++
 mm/internal.h          |   2 -
 mm/vrange.c            | 185 +++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 194 insertions(+), 2 deletions(-)
include/linux/vrange.h +9 -0
index ef153c8..778902d 100644
@@ -41,6 +41,9 @@ extern int vrange_clear(struct vrange_root *vroot,
 extern void vrange_root_cleanup(struct vrange_root *vroot);
 extern int vrange_fork(struct mm_struct *new,
 					struct mm_struct *old);
+int discard_vpage(struct page *page);
+bool vrange_addr_volatile(struct vm_area_struct *vma, unsigned long addr);
+
 #else
 
 static inline void vrange_root_init(struct vrange_root *vroot,
@@ -51,5 +54,11 @@ static inline int vrange_fork(struct mm_struct *new, struct mm_struct *old)
 	return 0;
 }
 
+static inline bool vrange_addr_volatile(struct vm_area_struct *vma,
+					unsigned long addr)
+{
+	return false;
+}
+static inline int discard_vpage(struct page *page) { return 0 };
 #endif
 #endif /* _LINIUX_VRANGE_H */
mm/internal.h +0 -2
index 4390ac6..c2c6a93 100644
@@ -223,10 +223,8 @@ static inline void mlock_migrate_page(struct page *newpage, struct page *page)
 
 extern pmd_t maybe_pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma);
 
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
 extern unsigned long vma_address(struct page *page,
 				 struct vm_area_struct *vma);
-#endif
 #else /* !CONFIG_MMU */
 static inline int mlocked_vma_newpage(struct vm_area_struct *v, struct page *p)
 {
mm/vrange.c +185 -0

200 lines not on this page · raw message

Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#27 lore raw

Re: [PATCH 06/14] vrange: Add basic functions to purge volatile pages

Quoted 7 lines · John Stultz
mm/vrange.c +7 -0
+static void try_to_discard_one(struct vrange_root *vroot, struct page *page,
+				struct vm_area_struct *vma, unsigned long addr)
+{
+	struct mm_struct *mm = vma->vm_mm;
+	pte_t *pte;
+	pte_t pteval;
+	spinlock_t *ptl;

mm/vrange.c SYSCALL_DEFINE4(vrange, unsigned long, start, · line 464

Quoted 10 lines
mm/vrange.c +10 -0
+
+	VM_BUG_ON(!PageLocked(page));
+
+	pte = page_check_address(page, mm, addr, &ptl, 0);
+	if (!pte)
+		return;
+
+	BUG_ON(vma->vm_flags & (VM_SPECIAL|VM_LOCKED|VM_MIXEDMAP|VM_HUGETLB));
+
+	flush_cache_page(vma, addr, page_to_pfn(page));

It seems that this patch is different in your GIT repo
(git://git.linaro.org/people/jstultz/android-dev.git dev/vrange-v9). In GIT it is missing the fix: s/address/addr.

Best regards,
Krzysztof

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#28 lore raw

PATCH 7/14 vrange: Purge volatile pages when memory is tight

In reply to John Stultz

From: Minchan Kim <[email protected]>

This patch adds purging logic of volatile pages into direct reclaim path so that if vrange pages is selected as victim by VM, they could be discarded rather than swapping out.

Direct purging doesn't consider volatile page's age because it would be better to free the page rather than swapping out another working set pages. This makes sense because userspace specifies "please remove free these pages when memory is tight" via the vrange syscall.

This however is an in-kernel behavior and the purging logic could later change. Applications should not assume anything about the volatile page purging order, much as they shouldn't assume anything about the page swapout order.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
---
 include/linux/rmap.h | 11 +++++++----
 mm/ksm.c             |  2 +-
 mm/rmap.c            | 28 ++++++++++++++++++++--------
 mm/vmscan.c          | 17 +++++++++++++++--
 4 files changed, 43 insertions(+), 15 deletions(-)
include/linux/rmap.h +7 -4
index 6dacb93..f38185d 100644
@@ -181,10 +181,11 @@ static inline void page_dup_rmap(struct page *page)
 /*
  * Called from mm/vmscan.c to handle paging out
  */
-int page_referenced(struct page *, int is_locked,
-			struct mem_cgroup *memcg, unsigned long *vm_flags);
+int page_referenced(struct page *, int is_locked, struct mem_cgroup *memcg,
+				unsigned long *vm_flags, int *is_vrange);
 int page_referenced_one(struct page *, struct vm_area_struct *,
-	unsigned long address, unsigned int *mapcount, unsigned long *vm_flags);
+			unsigned long address, unsigned int *mapcount,
+			unsigned long *vm_flags, int *is_vrange);
 
 #define TTU_ACTION(x) ((x) & TTU_ACTION_MASK)
 
@@ -249,9 +250,11 @@ int rmap_walk(struct page *page, int (*rmap_one)(struct page *,
 
 static inline int page_referenced(struct page *page, int is_locked,
 				  struct mem_cgroup *memcg,
-				  unsigned long *vm_flags)
+				  unsigned long *vm_flags,
+				  int *is_vrange)
 {
 	*vm_flags = 0;
+	*is_vrange = 0;
 	return 0;
 }
 
mm/ksm.c +1 -1
index b6afe0c..debc20c 100644
@@ -1932,7 +1932,7 @@ again:
 				continue;
 
 			referenced += page_referenced_one(page, vma,
-				rmap_item->address, &mapcount, vm_flags);
+				rmap_item->address, &mapcount, vm_flags, NULL);
 			if (!search_new_forks || !mapcount)
 				break;
 		}
mm/rmap.c +20 -8
index b2e29ac..f929f22 100644
@@ -57,6 +57,7 @@
 #include <linux/migrate.h>
 #include <linux/hugetlb.h>
 #include <linux/backing-dev.h>
+#include <linux/vrange.h>
 
 #include <asm/tlbflush.h>
 
@@ -662,7 +663,7 @@ int page_mapped_in_vma(struct page *page, struct vm_area_struct *vma)
  */
 int page_referenced_one(struct page *page, struct vm_area_struct *vma,
 			unsigned long address, unsigned int *mapcount,
-			unsigned long *vm_flags)
+			unsigned long *vm_flags, int *is_vrange)
 {
 	struct mm_struct *mm = vma->vm_mm;
 	int referenced = 0;
@@ -724,6 +725,11 @@ int page_referenced_one(struct page *page, struct vm_area_struct *vma,
 				referenced++;
 		}
 		pte_unmap_unlock(pte, ptl);
+		if (is_vrange && vrange_addr_volatile(vma, address)) {
+			*is_vrange = 1;
+			*mapcount = 0; /* break ealry from loop */
+			goto out;
+		}
 	}
 
 	(*mapcount)--;
@@ -736,7 +742,7 @@ out:
 
 static int page_referenced_anon(struct page *page,
 				struct mem_cgroup *memcg,
-				unsigned long *vm_flags)
+				unsigned long *vm_flags, int *is_vrange)
 {
 	unsigned int mapcount;
 	struct anon_vma *anon_vma;
@@ -761,7 +767,8 @@ static int page_referenced_anon(struct page *page,
 		if (memcg && !mm_match_cgroup(vma->vm_mm, memcg))
 			continue;
 		referenced += page_referenced_one(page, vma, address,
-						  &mapcount, vm_flags);
+						  &mapcount, vm_flags,
+						  is_vrange);
 		if (!mapcount)
 			break;
 	}
@@ -785,7 +792,7 @@ static int page_referenced_anon(struct page *page,
  */
 static int page_referenced_file(struct page *page,
 				struct mem_cgroup *memcg,
-				unsigned long *vm_flags)
+				unsigned long *vm_flags, int *is_vrange)
 {
 	unsigned int mapcount;
 	struct address_space *mapping = page->mapping;
@@ -826,7 +833,8 @@ static int page_referenced_file(struct page *page,
 		if (memcg && !mm_match_cgroup(vma->vm_mm, memcg))
 			continue;
 		referenced += page_referenced_one(page, vma, address,
-						  &mapcount, vm_flags);
+						  &mapcount, vm_flags,
+						  is_vrange);
 		if (!mapcount)
 			break;
 	}
@@ -841,6 +849,7 @@ static int page_referenced_file(struct page *page,
  * @is_locked: caller holds lock on the page
  * @memcg: target memory cgroup
  * @vm_flags: collect encountered vma->vm_flags who actually referenced the page
+ * @is_vrange: Is @page in vrange?
  *
  * Quick test_and_clear_referenced for all mappings to a page,
  * returns the number of ptes which referenced the page.
@@ -848,7 +857,8 @@ static int page_referenced_file(struct page *page,
 int page_referenced(struct page *page,
 		    int is_locked,
 		    struct mem_cgroup *memcg,
-		    unsigned long *vm_flags)
+		    unsigned long *vm_flags,
+		    int *is_vrange)
 {
 	int referenced = 0;
 	int we_locked = 0;
@@ -867,10 +877,12 @@ int page_referenced(struct page *page,
 								vm_flags);
 		else if (PageAnon(page))
 			referenced += page_referenced_anon(page, memcg,
-								vm_flags);
+								vm_flags,
+								is_vrange);
 		else if (page->mapping)
 			referenced += page_referenced_file(page, memcg,
-								vm_flags);
+								vm_flags,
+								is_vrange);
 		if (we_locked)
 			unlock_page(page);
 
mm/vmscan.c +15 -2
index 2cff0d4..ab377b6 100644
@@ -43,6 +43,7 @@
 #include <linux/sysctl.h>
 #include <linux/oom.h>
 #include <linux/prefetch.h>
+#include <linux/vrange.h>
 
 #include <asm/tlbflush.h>
 #include <asm/div64.h>
@@ -610,17 +611,19 @@ enum page_references {
 	PAGEREF_RECLAIM,
 	PAGEREF_RECLAIM_CLEAN,
 	PAGEREF_KEEP,
+	PAGEREF_DISCARD,
 	PAGEREF_ACTIVATE,
 };
 
 static enum page_references page_check_references(struct page *page,
 						  struct scan_control *sc)
 {
+	int is_vrange = 0;
 	int referenced_ptes, referenced_page;
 	unsigned long vm_flags;
 
 	referenced_ptes = page_referenced(page, 1, sc->target_mem_cgroup,
-					  &vm_flags);
+					  &vm_flags, &is_vrange);
 	referenced_page = TestClearPageReferenced(page);
 
 	/*
@@ -630,6 +633,13 @@ static enum page_references page_check_references(struct page *page,
 	if (vm_flags & VM_LOCKED)
 		return PAGEREF_RECLAIM;
 
+	/*
+	 * If volatile page is reached on LRU's tail, we discard the
+	 * page without considering recycle the page.
+	 */
+	if (is_vrange)
+		return PAGEREF_DISCARD;
+
 	if (referenced_ptes) {
 		if (PageSwapBacked(page))
 			return PAGEREF_ACTIVATE;
@@ -859,6 +869,9 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 			goto activate_locked;
 		case PAGEREF_KEEP:
 			goto keep_locked;
+		case PAGEREF_DISCARD:
+			if (may_enter_fs && !discard_vpage(page))
+				goto free_it;
 		case PAGEREF_RECLAIM:
 		case PAGEREF_RECLAIM_CLEAN:
 			; /* try to reclaim the page below */
@@ -1614,7 +1627,7 @@ static void shrink_active_list(unsigned long nr_to_scan,
 		}
 
 		if (page_referenced(page, 0, sc->target_mem_cgroup,
-				    &vm_flags)) {
+				    &vm_flags, NULL)) {
 			nr_rotated += hpage_nr_pages(page);
 			/*
 			 * Identify referenced, file-backed active pages and
Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#29 lore raw

Re: [PATCH 07/14] vrange: Purge volatile pages when memory is tight

include/linux/rmap.h int rmap_walk(struct page *page, int (*rmap_one)(struct page *, · line 259

Quoted 10 lines · John Stultz

static inline int page_referenced(struct page *page, int is_locked,
struct mem_cgroup *memcg,

include/linux/rmap.h +3 -1
-                                 unsigned long *vm_flags)
+                                 unsigned long *vm_flags,
+                                 int *is_vrange)
 {
        *vm_flags = 0;
+       *is_vrange = 0;
        return 0;
 }

I don't know if it is appropriate to add a parameter in such a core function for an optional functionality. Maybe the is_vrange flag should be squashed into the vm_flags ? I am not sure .

--

Regards,
Zhan Jianyu

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#30 lore raw

Re: [PATCH 07/14] vrange: Purge volatile pages when memory is tight

Quoted 11 lines · Zhan Jianyu

include/linux/rmap.h int rmap_walk(struct page *page, int (*rmap_one)(struct page *, · line 259

Quoted 10 lines · John Stultz

static inline int page_referenced(struct page *page, int is_locked,
struct mem_cgroup *memcg,

include/linux/rmap.h +3 -1
-                                 unsigned long *vm_flags)
+                                 unsigned long *vm_flags,
+                                 int *is_vrange)
 {
        *vm_flags = 0;
+       *is_vrange = 0;
        return 0;
 }
Quoted 3 lines

I don't know if it is appropriate to add a parameter in such a core function for an optional functionality. Maybe the is_vrange flag should be squashed into the vm_flags ? I am not sure .

Yea, this wasn't either Minchan or I were particularly fond of, but with the vm_flags exausted, there wasn't a clear way to do so without doing the rmap traversal again.

Other suggestions? Extending the vm_flags to 64bits is something many better mm devs have tried to merge unsuccessfully, so I'm hesitant to try pushing it myself.

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#31 lore raw

PATCH 8/14 vrange: Send SIGBUS when user try to access purged page

In reply to John Stultz

From: Minchan Kim <[email protected]>

By vrange(2) semantic, a user should see SIGBUG if they try to access purged page without marking the memory as non-voaltile (ie, vrange(...VRANGE_NOVOLATILE)).

This allows for optimistic traversal of volatile pages, without having to mark them non-volatile first and the SIGBUS allows applications to trap and fixup the purged range before accessing them again.

This patch implements it by adding SWP_VRANGE so it consumes one from MAX_SWAPFILES. It means worst case of MAX_SWAPFILES in 32 bit is 32 - 2 - 1 - 1 = 28. I think it's still enough for everybody. If someone complains about that and thinks we shouldn't consume it, I will change it with (swp_type 0, pgoffset 0) which is header of swap which couldn't be allocated as swp_pte for swapout so we can use it.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
---
 include/linux/swap.h   |  6 +++++-
 include/linux/vrange.h | 20 ++++++++++++++++++++
 mm/memory.c            | 27 +++++++++++++++++++++++++++
 mm/mincore.c           |  5 ++++-
 mm/vrange.c            | 20 +++++++++++++++++++-
 5 files changed, 75 insertions(+), 3 deletions(-)
include/linux/swap.h +5 -1
index d95cde5..7fd1006 100644
@@ -49,6 +49,9 @@ static inline int current_is_kswapd(void)
  * actions on faults.
  */
 
+#define SWP_VRANGE_NUM 1
+#define SWP_VRANGE	(MAX_SWAPFILES + SWP_HWPOISON_NUM + SWP_MIGRATION_NUM)
+
 /*
  * NUMA node memory migration support
  */
@@ -71,7 +74,8 @@ static inline int current_is_kswapd(void)
 #endif
 
 #define MAX_SWAPFILES \
-	((1 << MAX_SWAPFILES_SHIFT) - SWP_MIGRATION_NUM - SWP_HWPOISON_NUM)
+	((1 << MAX_SWAPFILES_SHIFT) - SWP_MIGRATION_NUM - SWP_HWPOISON_NUM \
+			- SWP_VRANGE_NUM)
 
 /*
  * Magic header for a swap area. The first part of the union is
include/linux/vrange.h +20 -0
index 778902d..50b9131 100644
@@ -3,6 +3,8 @@
 
 #include <linux/vrange_types.h>
 #include <linux/mm.h>
+#include <linux/swap.h>
+#include <linux/swapops.h>
 
 #define vrange_from_node(node_ptr) \
 	container_of(node_ptr, struct vrange, node)
@@ -12,6 +14,16 @@
 
 #ifdef CONFIG_MMU
 
+static inline swp_entry_t make_vrange_entry(void)
+{
+	return swp_entry(SWP_VRANGE, 0);
+}
+
+static inline int is_vrange_entry(swp_entry_t entry)
+{
+	return swp_type(entry) == SWP_VRANGE;
+}
+
 static inline void vrange_root_init(struct vrange_root *vroot, int type,
 								void *object)
 {
@@ -44,6 +56,9 @@ extern int vrange_fork(struct mm_struct *new,
 int discard_vpage(struct page *page);
 bool vrange_addr_volatile(struct vm_area_struct *vma, unsigned long addr);
 
+extern bool vrange_addr_purged(struct vm_area_struct *vma,
+					unsigned long address);
+
 #else
 
 static inline void vrange_root_init(struct vrange_root *vroot,
@@ -60,5 +75,10 @@ static inline bool vrange_addr_volatile(struct vm_area_struct *vma,
 	return false;
 }
 static inline int discard_vpage(struct page *page) { return 0 };
+static inline bool vrange_addr_purged(struct vm_area_struct *vma,
+					unsigned long address)
+{
+	return false;
+};
 #endif
 #endif /* _LINIUX_VRANGE_H */
mm/memory.c +27 -0
index af84bc0..e33dbce 100644
@@ -59,6 +59,7 @@
 #include <linux/gfp.h>
 #include <linux/migrate.h>
 #include <linux/string.h>
+#include <linux/vrange.h>
 
 #include <asm/io.h>
 #include <asm/pgalloc.h>
@@ -831,6 +832,8 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm,
 	if (unlikely(!pte_present(pte))) {
 		if (!pte_file(pte)) {
 			swp_entry_t entry = pte_to_swp_entry(pte);
+			if (is_vrange_entry(entry))
+				goto out_set_pte;
 
 			if (swap_duplicate(entry) < 0)
 				return entry.val;
@@ -1174,6 +1177,8 @@ again:
 				print_bad_pte(vma, addr, ptent, NULL);
 		} else {
 			swp_entry_t entry = pte_to_swp_entry(ptent);
+			if (is_vrange_entry(entry))
+				goto out;
 
 			if (!non_swap_entry(entry))
 				rss[MM_SWAPENTS]--;
@@ -1190,6 +1195,7 @@ again:
 			if (unlikely(!free_swap_and_cache(entry)))
 				print_bad_pte(vma, addr, ptent, NULL);
 		}
+out:
 		pte_clear_not_present_full(mm, addr, pte, tlb->fullmm);
 	} while (pte++, addr += PAGE_SIZE, addr != end);
 
@@ -3715,15 +3721,36 @@ int handle_pte_fault(struct mm_struct *mm,
 
 	entry = *pte;
 	if (!pte_present(entry)) {
+		swp_entry_t vrange_entry;
+
 		if (pte_none(entry)) {
 			if (vma->vm_ops) {
 				if (likely(vma->vm_ops->fault))
 					return do_linear_fault(mm, vma, address,
 						pte, pmd, flags, entry);
 			}
+anon:
 			return do_anonymous_page(mm, vma, address,
 						 pte, pmd, flags);
 		}
+
+		vrange_entry = pte_to_swp_entry(entry);
+		if (unlikely(is_vrange_entry(vrange_entry))) {
+			if (!vrange_addr_purged(vma, address)) {
+				/* zap pte */
+				ptl = pte_lockptr(mm, pmd);
+				spin_lock(ptl);
+				if (unlikely(!pte_same(*pte, entry)))
+					goto unlock;
+				flush_cache_page(vma, address, pte_pfn(*pte));
+				ptep_clear_flush(vma, address, pte);
+				pte_unmap_unlock(pte, ptl);
+				goto anon;
+			}
+
+			return VM_FAULT_SIGBUS;
+		}
+
 		if (pte_file(entry))
 			return do_nonlinear_fault(mm, vma, address,
 					pte, pmd, flags, entry);
mm/mincore.c +4 -1
index da2be56..2a95eef 100644
@@ -15,6 +15,7 @@
 #include <linux/swap.h>
 #include <linux/swapops.h>
 #include <linux/hugetlb.h>
+#include <linux/vrange.h>
 
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>
@@ -129,7 +130,9 @@ static void mincore_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
 		} else { /* pte is a swap entry */
 			swp_entry_t entry = pte_to_swp_entry(pte);
 
-			if (is_migration_entry(entry)) {
+			if (is_vrange_entry(entry))
+				*vec = 0;
+			else if (is_migration_entry(entry)) {
 				/* migration entries are always uptodate */
 				*vec = 1;
 			} else {
mm/vrange.c +19 -1
index c72e72d..c19a966 100644
@@ -10,7 +10,6 @@
 #include <linux/rmap.h>
 #include <linux/hugetlb.h>
 #include "internal.h"
-#include <linux/swap.h>
 #include <linux/mmu_notifier.h>
 
 static struct kmem_cache *vrange_cachep;
@@ -430,6 +429,24 @@ bool vrange_addr_volatile(struct vm_area_struct *vma, unsigned long addr)
 	return ret;
 }
 
+bool vrange_addr_purged(struct vm_area_struct *vma, unsigned long addr)
+{
+	struct vrange_root *vroot;
+	struct vrange *range;
+	unsigned long vstart_idx;
+	bool ret = false;
+
+	vroot = __vma_to_vroot(vma);
+	vstart_idx = __vma_addr_to_index(vma, addr);
+
+	vrange_lock(vroot);
+	range = __vrange_find(vroot, vstart_idx, vstart_idx + PAGE_SIZE - 1);
+	if (range && range->purged)
+		ret = true;
+	vrange_unlock(vroot);
+	return ret;
+}
+
 /* Caller should hold vrange_lock */
 static void do_purge(struct vrange_root *vroot,
 		unsigned long start_idx, unsigned long end_idx)
@@ -473,6 +490,7 @@ static void try_to_discard_one(struct vrange_root *vroot, struct page *page,
 	page_remove_rmap(page);
 	page_cache_release(page);
 
+	set_pte_at(mm, addr, pte, swp_entry_to_pte(make_vrange_entry()));
 	pte_unmap_unlock(pte, ptl);
 	mmu_notifier_invalidate_page(mm, addr);
 
Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#32 lore raw

PATCH 9/14 vrange: Add vrange LRU list for purging

In reply to John Stultz

From: Minchan Kim <[email protected]>

This patch adds vrange LRU list for managing vranges to purge by something (In this implementation, I will use slab shrinker introduced by upcoming patches).

This is necessary to purge vranges on swapless system because currently the VM only ages anonymous pages if the system has a swap device.

In this case, because we would otherwise be duplicating the page LRUs tracking of hot/cold pages, we utilize a vrange LRU, to manage the shrinking order. Thus the shrinker will discard the entire vrange at once, and vranges are purged in the order they are marked volatile.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
---
 include/linux/vrange_types.h |  2 ++
 mm/vrange.c                  | 61 ++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 58 insertions(+), 5 deletions(-)
include/linux/vrange_types.h +2 -0
index 0d48b42..d7d451c 100644
@@ -20,6 +20,8 @@ struct vrange {
 	struct interval_tree_node node;
 	struct vrange_root *owner;
 	int purged;
+	struct list_head lru;
+	atomic_t refcount;
 };
 #endif
 
mm/vrange.c +56 -5

124 lines not on this page · raw message

Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#33 lore raw

PATCH 10/14 vrange: Add core shrinking logic for swapless system

In reply to John Stultz

From: Minchan Kim <[email protected]>

This patch adds the core volatile range shrinking logic needed to allow volatile range purging to function on swapless systems.

This patch does not wire in the specific range purging logic, but that will be added in the following patches.

The reason I use shrinker is that Dave and Glauber are trying to make slab shrinker being aware of node/memcg so if the patchset reach on mainline, we also can support node/memcg in vrange, easily.

Another reason I selected slab shrinker is that normally slab shrinker is called after normal reclaim of file-backed page(ex, page cache) so reclaiming preference would be this, I expect.(TODO: invstigate and might need more tunes in reclaim path)

        page cache -> vrange by slab shrinking -> anon page

It does make sense because page cache can have stream data so there is no point to shrink vrange pages if there are lots of streaming pages in page cache.

In this version, I didn't check it works well but it's design concept so we can make it work via modify page reclaim path.
I will have more experiment.

One of disadvantage with using slab shrink is that slab shrinker isn't called in using memcg so memcg-noswap system cannot take advantage of it. Hmm, Maybe I will jump into relcaim code to hook some point to control vrange page shrinking more freely.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
[jstultz: Renamed some functions and minor cleanups]
---
 mm/vrange.c | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 86 insertions(+), 3 deletions(-)
mm/vrange.c +86 -3
index 33e3ac1..e7c5a25 100644
@@ -25,11 +25,19 @@ static inline unsigned int vrange_size(struct vrange *range)
 	return range->node.last + 1 - range->node.start;
 }
 
+static int shrink_vrange(struct shrinker *s, struct shrink_control *sc);
+
+static struct shrinker vrange_shrinker = {
+	.shrink = shrink_vrange,
+	.seeks = DEFAULT_SEEKS
+};
+
 static int __init vrange_init(void)
 {
 	INIT_LIST_HEAD(&vrange_list.list);
 	mutex_init(&vrange_list.lock);
 	vrange_cachep = KMEM_CACHE(vrange, SLAB_PANIC);
+	register_shrinker(&vrange_shrinker);
 	return 0;
 }
 module_init(vrange_init);
@@ -58,9 +66,14 @@ static void __vrange_free(struct vrange *range)
 static inline void __vrange_lru_add(struct vrange *range)
 {
 	mutex_lock(&vrange_list.lock);
-	WARN_ON(!list_empty(&range->lru));
-	list_add(&range->lru, &vrange_list.list);
-	vrange_list.size += vrange_size(range);
+	/*
+	 * We need this check because it could be raced with
+	 * shrink_vrange and vrange_resize
+	 */
+	if (list_empty(&range->lru)) {
+		list_add(&range->lru, &vrange_list.list);
+		vrange_list.size += vrange_size(range);
+	}
 	mutex_unlock(&vrange_list.lock);
 }
 
@@ -84,6 +97,14 @@ static void __vrange_add(struct vrange *range, struct vrange_root *vroot)
 	__vrange_lru_add(range);
 }
 
+static inline int __vrange_get(struct vrange *vrange)
+{
+	if (!atomic_inc_not_zero(&vrange->refcount))
+		return 0;
+
+	return 1;
+}
+
 static inline void __vrange_put(struct vrange *range)
 {
 	if (atomic_dec_and_test(&range->refcount)) {
@@ -647,3 +668,65 @@ int discard_vpage(struct page *page)
 
 	return 1;
 }
+
+static struct vrange *vrange_isolate(void)
+{
+	struct vrange *vrange = NULL;
+	mutex_lock(&vrange_list.lock);
+	while (!list_empty(&vrange_list.list)) {
+		vrange = list_entry(vrange_list.list.prev,
+				struct vrange, lru);
+		list_del_init(&vrange->lru);
+		vrange_list.size -= vrange_size(vrange);
+
+		/* vrange is going to destroy */
+		if (__vrange_get(vrange))
+			break;
+
+		vrange = NULL;
+	}
+
+	mutex_unlock(&vrange_list.lock);
+	return vrange;
+}
+
+static unsigned int discard_vrange(struct vrange *vrange)
+{
+	return 0;
+}
+
+static int shrink_vrange(struct shrinker *s, struct shrink_control *sc)
+{
+	struct vrange *range = NULL;
+	long nr_to_scan = sc->nr_to_scan;
+	long size = vrange_list.size;
+
+	if (!nr_to_scan)
+		return size;
+
+	if (sc->nr_to_scan && !(sc->gfp_mask & __GFP_IO))
+		return -1;
+
+	while (size > 0 && nr_to_scan > 0) {
+		range = vrange_isolate();
+		if (!range)
+			break;
+
+		/* range is removing so don't bother */
+		if (!range->owner) {
+			__vrange_put(range);
+			size -= vrange_size(range);
+			nr_to_scan -= vrange_size(range);
+			continue;
+		}
+
+		if (discard_vrange(range) < 0)
+			__vrange_lru_add(range);
+		__vrange_put(range);
+
+		size -= vrange_size(range);
+		nr_to_scan -= vrange_size(range);
+	}
+
+	return size;
+}
Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#34 lore raw

PATCH 11/14 vrange: Purging vrange-anon pages from shrinker

In reply to John Stultz

From: Minchan Kim <[email protected]>

This patch provides the logic to discard anonymous vranges from the shrinker, by generating the page list for the volatile ranges setting the ptes volatile, and discarding the pages.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
[jstultz: Code tweaks and commit log rewording]
---
 mm/vrange.c | 179 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 178 insertions(+), 1 deletion(-)
mm/vrange.c +178 -1
index e7c5a25..c6bc32f 100644
@@ -11,6 +11,8 @@
 #include <linux/hugetlb.h>
 #include "internal.h"
 #include <linux/mmu_notifier.h>
+#include <linux/mm_inline.h>
+#include <linux/migrate.h>
 
 static struct kmem_cache *vrange_cachep;
 
@@ -20,6 +22,11 @@ static struct vrange_list {
 	struct mutex lock;
 } vrange_list;
 
+struct vrange_walker {
+	struct vm_area_struct *vma;
+	struct list_head *pagelist;
+};
+
 static inline unsigned int vrange_size(struct vrange *range)
 {
 	return range->node.last + 1 - range->node.start;
@@ -690,11 +697,181 @@ static struct vrange *vrange_isolate(void)
 	return vrange;
 }
 
-static unsigned int discard_vrange(struct vrange *vrange)
+static unsigned int discard_vrange_pagelist(struct list_head *page_list)
+{
+	struct page *page;
+	unsigned int nr_discard = 0;
+	LIST_HEAD(ret_pages);
+	LIST_HEAD(free_pages);
+
+	while (!list_empty(page_list)) {
+		int err;
+		page = list_entry(page_list->prev, struct page, lru);
+		list_del(&page->lru);
+		if (!trylock_page(page)) {
+			list_add(&page->lru, &ret_pages);
+			continue;
+		}
+
+		/*
+		 * discard_vpage returns unlocked page if it
+		 * is successful
+		 */
+		err = discard_vpage(page);
+		if (err) {
+			unlock_page(page);
+			list_add(&page->lru, &ret_pages);
+			continue;
+		}
+
+		ClearPageActive(page);
+		list_add(&page->lru, &free_pages);
+		dec_zone_page_state(page, NR_ISOLATED_ANON);
+		nr_discard++;
+	}
+
+	free_hot_cold_page_list(&free_pages, 1);
+	list_splice(&ret_pages, page_list);
+	return nr_discard;
+}
+
+static void vrange_pte_entry(pte_t pteval, unsigned long address,
+		unsigned ptent_size, struct mm_walk *walk)
+{
+	struct page *page;
+	struct vrange_walker *vw = walk->private;
+	struct vm_area_struct *vma = vw->vma;
+	struct list_head *pagelist = vw->pagelist;
+
+	if (pte_none(pteval))
+		return;
+
+	if (!pte_present(pteval))
+		return;
+
+	page = vm_normal_page(vma, address, pteval);
+	if (unlikely(!page))
+		return;
+
+	if (!PageLRU(page) || PageLocked(page))
+		return;
+
+	/* TODO : Support THP */
+	if (unlikely(PageCompound(page)))
+		return;
+
+	if (isolate_lru_page(page))
+		return;
+
+	list_add(&page->lru, pagelist);
+
+	VM_BUG_ON(page_is_file_cache(page));
+	inc_zone_page_state(page, NR_ISOLATED_ANON);
+}
+
+static int vrange_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
+		struct mm_walk *walk)
 {
+	pte_t *pte;
+	spinlock_t *ptl;
+
+	pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl);
+	for (; addr != end; pte++, addr += PAGE_SIZE)
+		vrange_pte_entry(*pte, addr, PAGE_SIZE, walk);
+	pte_unmap_unlock(pte - 1, ptl);
+	cond_resched();
+
 	return 0;
 }
 
+static unsigned int discard_vma_pages(struct mm_struct *mm,
+		struct vm_area_struct *vma, unsigned long start,
+		unsigned long end)
+{
+	unsigned int ret = 0;
+	LIST_HEAD(pagelist);
+	struct vrange_walker vw;
+	struct mm_walk vrange_walk = {
+		.pmd_entry = vrange_pte_range,
+		.mm = vma->vm_mm,
+		.private = &vw,
+	};
+
+	vw.pagelist = &pagelist;
+	vw.vma = vma;
+
+	walk_page_range(start, end, &vrange_walk);
+
+	if (!list_empty(&pagelist))
+		ret = discard_vrange_pagelist(&pagelist);
+
+	putback_lru_pages(&pagelist);
+	return ret;
+}
+
+/*
+ * vrange->owner isn't stable because caller doesn't hold vrange_lock
+ * so avoid touching vrange->owner.
+ */
+static int __discard_vrange_anon(struct mm_struct *mm, struct vrange *vrange,
+					unsigned int *ret_discard)
+{
+	struct vm_area_struct *vma;
+	unsigned int nr_discard = 0;
+	unsigned long start = vrange->node.start;
+	unsigned long end = vrange->node.last + 1;
+	int ret = 0;
+
+	/* It prevent to destroy vma when the process exist */
+	if (!atomic_inc_not_zero(&mm->mm_users))
+		return ret;
+
+	if (!down_read_trylock(&mm->mmap_sem)) {
+		mmput(mm);
+		ret = -EBUSY;
+		goto out; /* this vrange could be retried */
+	}
+
+	vma = find_vma(mm, start);
+	if (!vma || (vma->vm_start >= end))
+		goto out_unlock;
+
+	for (; vma; vma = vma->vm_next) {
+		if (vma->vm_start >= end)
+			break;
+		BUG_ON(vma->vm_flags & (VM_SPECIAL|VM_LOCKED|VM_MIXEDMAP|
+					VM_HUGETLB));
+		cond_resched();
+		nr_discard += discard_vma_pages(mm, vma,
+				max_t(unsigned long, start, vma->vm_start),
+				min_t(unsigned long, end, vma->vm_end));
+	}
+out_unlock:
+	up_read(&mm->mmap_sem);
+	mmput(mm);
+	*ret_discard = nr_discard;
+out:
+	return ret;
+}
+
+static int discard_vrange(struct vrange *vrange)
+{
+	int ret = 0;
+	struct mm_struct *mm;
+	struct vrange_root *vroot;
+	unsigned int nr_discard = 0;
+	vroot = vrange->owner;
+
+	/* TODO : handle VRANGE_FILE */
+	if (vroot->type != VRANGE_MM)
+		goto out;
+
+	mm = vroot->object;
+	ret = __discard_vrange_anon(mm, vrange, &nr_discard);
+out:
+	return nr_discard;
+}
+
 static int shrink_vrange(struct shrinker *s, struct shrink_control *sc)
 {
 	struct vrange *range = NULL;
Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#35 lore raw

PATCH 12/14 vrange: Support background purging for vrange-file

In reply to John Stultz

From: Minchan Kim <[email protected]>

Add support to purge vrange file pages via the shrinker interface.

This is useful, since some filesystems like shmem/tmpfs use anonymous pages, which won't be aged off the page LRU if swap is disabled.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
[jstultz: Commit message tweaks]
---
 mm/vrange.c | 56 +++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 49 insertions(+), 7 deletions(-)
mm/vrange.c +49 -7
index c6bc32f..3f21dc9 100644
@@ -13,6 +13,7 @@
 #include <linux/mmu_notifier.h>
 #include <linux/mm_inline.h>
 #include <linux/migrate.h>
+#include <linux/pagevec.h>
 
 static struct kmem_cache *vrange_cachep;
 
@@ -854,21 +855,62 @@ out:
 	return ret;
 }
 
+static int __discard_vrange_file(struct address_space *mapping,
+			struct vrange *vrange, unsigned int *ret_discard)
+{
+	struct pagevec pvec;
+	pgoff_t index;
+	int i;
+	unsigned int nr_discard = 0;
+	unsigned long start_idx = vrange->node.start;
+	unsigned long end_idx = vrange->node.last;
+	const pgoff_t start = start_idx >> PAGE_CACHE_SHIFT;
+	pgoff_t end = end_idx >> PAGE_CACHE_SHIFT;
+	LIST_HEAD(pagelist);
+
+	pagevec_init(&pvec, 0);
+	index = start;
+	while (index <= end && pagevec_lookup(&pvec, mapping, index,
+			min(end - index, (pgoff_t)PAGEVEC_SIZE - 1) + 1)) {
+		for (i = 0; i < pagevec_count(&pvec); i++) {
+			struct page *page = pvec.pages[i];
+			index = page->index;
+			if (index > end)
+				break;
+			if (isolate_lru_page(page))
+				continue;
+			list_add(&page->lru, &pagelist);
+			inc_zone_page_state(page, NR_ISOLATED_ANON);
+		}
+		pagevec_release(&pvec);
+		cond_resched();
+		index++;
+	}
+
+	if (!list_empty(&pagelist))
+		nr_discard = discard_vrange_pagelist(&pagelist);
+
+	*ret_discard = nr_discard;
+	putback_lru_pages(&pagelist);
+
+	return 0;
+}
+
 static int discard_vrange(struct vrange *vrange)
 {
 	int ret = 0;
-	struct mm_struct *mm;
 	struct vrange_root *vroot;
 	unsigned int nr_discard = 0;
 	vroot = vrange->owner;
 
-	/* TODO : handle VRANGE_FILE */
-	if (vroot->type != VRANGE_MM)
-		goto out;
+	if (vroot->type == VRANGE_MM) {
+		struct mm_struct *mm = vroot->object;
+		ret = __discard_vrange_anon(mm, vrange, &nr_discard);
+	} else if (vroot->type == VRANGE_FILE) {
+		struct address_space *mapping = vroot->object;
+		ret = __discard_vrange_file(mapping, vrange, &nr_discard);
+	}
 
-	mm = vroot->object;
-	ret = __discard_vrange_anon(mm, vrange, &nr_discard);
-out:
 	return nr_discard;
 }
 
Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#36 lore raw

PATCH 13/14 vrange: Allocate vroot dynamically

In reply to John Stultz

From: Minchan Kim <[email protected]>

This patch allocates vroot dynamically when vrange syscall is called so if anybody doesn't call vrange syscall, we don't waste memory space occupied by vroot.

The vroot is allocated by SLAB_DESTROY_BY_RCU, thus because we can't guarantee vroot's validity when we are about to access vroot of a different process, the rules are as follows:

1. rcu_read_lock
2. checkt vroot == NULL
3. increment vroot's refcount
4. rcu_read_unlock
5. vrange_lock(vroot)
6. get vrange from tree
7. vrange->owenr == vroot check again because vroot can be allocated for another one in same RCU period.

If we're accessing the vroot from our own context, we can skip the rcu & extra checking, since we know the vroot won't disappear from under us while we're running.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
[jstultz: Commit rewording, renamed functions, added helper functions]
---
 fs/inode.c                   |   4 +-
 include/linux/fs.h           |   2 +-
 include/linux/mm_types.h     |   2 +-
 include/linux/vrange_types.h |   1 +
 kernel/fork.c                |   5 +-
 mm/mmap.c                    |   2 +-
 mm/vrange.c                  | 257 +++++++++++++++++++++++++++++++++++++++++--
 7 files changed, 255 insertions(+), 18 deletions(-)
fs/inode.c +2 -2
index 5364f91..f5b8990 100644
@@ -353,7 +353,6 @@ void address_space_init_once(struct address_space *mapping)
 	spin_lock_init(&mapping->private_lock);
 	mapping->i_mmap = RB_ROOT;
 	INIT_LIST_HEAD(&mapping->i_mmap_nonlinear);
-	vrange_root_init(&mapping->vroot, VRANGE_FILE, mapping);
 }
 EXPORT_SYMBOL(address_space_init_once);
 
@@ -1421,7 +1420,8 @@ static void iput_final(struct inode *inode)
 		inode_lru_list_del(inode);
 	spin_unlock(&inode->i_lock);
 
-	vrange_root_cleanup(&inode->i_mapping->vroot);
+	vrange_root_cleanup(inode->i_mapping->vroot);
+	inode->i_mapping->vroot = NULL;
 
 	evict(inode);
 }
include/linux/fs.h +1 -1
index 6ec2953..32ef488 100644
@@ -415,7 +415,7 @@ struct address_space {
 	struct list_head	i_mmap_nonlinear;/*list VM_NONLINEAR mappings */
 	struct mutex		i_mmap_mutex;	/* protect tree, count, list */
 #ifdef CONFIG_MMU
-	struct vrange_root	vroot;
+	struct vrange_root	*vroot;
 #endif
 	/* Protected by tree_lock together with the radix tree */
 	unsigned long		nrpages;	/* number of total pages */
include/linux/mm_types.h +1 -1
index 5d8cdc3..ad7e2fc 100644
@@ -351,7 +351,7 @@ struct mm_struct {
 
 
 #ifdef CONFIG_MMU
-	struct vrange_root vroot;
+	struct vrange_root *vroot;
 #endif
 	unsigned long hiwater_rss;	/* High-watermark of RSS usage */
 	unsigned long hiwater_vm;	/* High-water virtual memory usage */
include/linux/vrange_types.h +1 -0
index d7d451c..c4ef8b6 100644
@@ -14,6 +14,7 @@ struct vrange_root {
 	struct mutex v_lock;		/* Protect v_rb */
 	enum vrange_type type;		/* range root type */
 	void *object;			/* pointer to mm_struct or mapping */
+	atomic_t refcount;
 };
 
 struct vrange {
kernel/fork.c +3 -2
index ceb38bf..16d58ca 100644
@@ -545,9 +545,9 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p)
 		(current->mm->flags & MMF_INIT_MASK) : default_dump_filter;
 	mm->core_state = NULL;
 	mm->nr_ptes = 0;
+	mm->vroot = NULL;
 	memset(&mm->rss_stat, 0, sizeof(mm->rss_stat));
 	spin_lock_init(&mm->page_table_lock);
-	vrange_root_init(&mm->vroot, VRANGE_MM, mm);
 	mm_init_aio(mm);
 	mm_init_owner(mm, p);
 
@@ -619,7 +619,8 @@ void mmput(struct mm_struct *mm)
 
 	if (atomic_dec_and_test(&mm->mm_users)) {
 		uprobe_clear_state(mm);
-		vrange_root_cleanup(&mm->vroot);
+		vrange_root_cleanup(mm->vroot);
+		mm->vroot = NULL;
 		exit_aio(mm);
 		ksm_exit(mm);
 		khugepaged_exit(mm); /* must run before exit_mmap */
mm/mmap.c +1 -1
index ed7056f..cb2f9e0 100644
@@ -1505,7 +1505,7 @@ unsigned long mmap_region(struct file *file, unsigned long addr,
 munmap_back:
 
 	/* zap any volatile ranges */
-	vrange_clear(&mm->vroot, addr, addr + len);
+	vrange_clear(mm->vroot, addr, addr + len);
 
 	if (find_vma_links(mm, addr, addr + len, &prev, &rb_link, &rb_parent)) {
 		if (do_munmap(mm, addr, len))
mm/vrange.c +246 -11

378 lines not on this page · raw message

Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#37 lore raw

PATCH 14/14 vrange: Add vmstat counter about purged page

In reply to John Stultz

From: Minchan Kim <[email protected]>

This patch adds the number of purged page in vmstat so admin can see how many of volatile pages are discarded by VM until now.

Cc Andrew Morton <[email protected]>
Cc Android Kernel Team <[email protected]>
Cc Robert Love <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Dmitry Adamushko <[email protected]>
Cc Dave Chinner <[email protected]>
Cc Neil Brown <[email protected]>
Cc Andrea Righi <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Aneesh Kumar K.V <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc Dhaval Giani <[email protected]>
Cc Jan Kara <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Rob Clark <[email protected]>
Cc Minchan Kim <[email protected]>
---
 include/linux/vm_event_item.h |  2 ++
 mm/vmstat.c                   |  2 ++
 mm/vrange.c                   | 10 ++++++++++
 3 files changed, 14 insertions(+)
include/linux/vm_event_item.h +2 -0
index bd6cf61..c4aea92 100644
@@ -25,6 +25,8 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
 		FOR_ALL_ZONES(PGALLOC),
 		PGFREE, PGACTIVATE, PGDEACTIVATE,
 		PGFAULT, PGMAJFAULT,
+		PGDISCARD_DIRECT,
+		PGDISCARD_KSWAPD,
 		FOR_ALL_ZONES(PGREFILL),
 		FOR_ALL_ZONES(PGSTEAL_KSWAPD),
 		FOR_ALL_ZONES(PGSTEAL_DIRECT),
mm/vmstat.c +2 -0
index 20c2ef4..4f35f46 100644
@@ -756,6 +756,8 @@ const char * const vmstat_text[] = {
 
 	"pgfault",
 	"pgmajfault",
+	"pgdiscard_direct",
+	"pgdiscard_kswapd",
 
 	TEXTS_FOR_ZONES("pgrefill")
 	TEXTS_FOR_ZONES("pgsteal_kswapd")
mm/vrange.c +10 -0
index c30e3dd..8931fab 100644
@@ -894,6 +894,10 @@ int discard_vpage(struct page *page)
 
 		if (page_freeze_refs(page, 1)) {
 			unlock_page(page);
+			if (current_is_kswapd())
+				count_vm_event(PGDISCARD_KSWAPD);
+			else
+				count_vm_event(PGDISCARD_DIRECT);
 			return 0;
 		}
 	}
@@ -1144,6 +1148,12 @@ static int discard_vrange(struct vrange *vrange)
 		ret = __discard_vrange_file(mapping, vrange, &nr_discard);
 	}
 
+	if (!ret) {
+		if (current_is_kswapd())
+			count_vm_events(PGDISCARD_KSWAPD, nr_discard);
+		else
+			count_vm_events(PGDISCARD_DIRECT, nr_discard);
+	}
 out:
 	__vroot_put(vroot);
 	return nr_discard;
Signature · 7 lines
1.8.1.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#38 lore raw
In reply to John Stultz
Quoted 18 lines · John Stultz

So its been awhile since the last release of the volatile ranges patches, and while Minchan and I have been busy with other things, we have been slowly chipping away at issues and differences trying to get a patchset that we both agree on.

There's still a few smaller issues, but we figured any further polishing of the patch series in private would be unproductive and it would be much better to send the patches out for review and comment and get some wider opinions.

Whats new in v9:
* Updated to v3.11
* Added vrange purging logic to purge anonymous pages on swapless systems
* Added logic to allocate the vroot structure dynamically to avoid added overhead to mm and address_space structures
* Lots of minor tweaks, changes and cleanups

Quoted 7 lines

Still TODO:
* Sort out better solution for clearing volatility on new mmaps
- Minchan has a different approach here
* Sort out apparent shrinker livelock that occasionally crops up under severe pressure

Feedback or thoughts here would be particularly helpful!

Andrew noted that I've forgotten to provide sufficient overview of what volatile ranges does, and given its been while, folks may want a quick introduction/reminder.

Volatile ranges provides a method for userland to inform the kernel that a range of memory is safe to discard (ie: can be regenerated) but userspace may want to try access it in the future. It can be thought of as similar to MADV_DONTNEED, but that the actual freeing of the memory is delayed and only done under memory pressure, and the user can try to cancel the action and be able to quickly access any unpurged pages. The idea originated from Android's ashmem, but I've since learned that other OSes provide similar functionality.

This funcitonality allows for a number of interesting uses:
* Userland caches that have kernel triggered eviction under memory pressure. This allows for the kernel to "rightsize" userspace caches for current system-wide workload. Things like image bitmap caches, or rendered HTML in a hidden browser tab, where the data is not visible and can be regenerated if needed, are good examples.

* Opportunistic freeing of memory that may be quickly reused. Minchan has done a malloc implementation where free() marks the pages as volatile, allowing the kernel to reclaim under pressure. This avoids the unmapping and remapping of anonymous pages on free/malloc. So if userland wants to malloc memory quickly after the free, it just needs to mark the pages as non-volatile, and only purged pages will have to be faulted back in.

The syscall interface is defined in patch 5/14 in this series, but briefly there are two ways to utilze the functionality:

Explicit marking method:
1) Userland marks a range of memory that can be regenerated if necessary as volatile
2) Before accessing the memory again, userland marks the memroy as nonvolatile, and the kernel will provide notifcation if any pages in the range has been purged.

Optimistic method:
1) Userland marks a large range of data as volatile
2) Userland continues to access the data as it needs.
3) If userland accesses a page that has been purged, the kernel will send a SIGBUS
4) Userspace can trap the SIGBUS, mark the afected pages as non-volatile, and refill the data as needed before continuing on

Other details:
The interface takes a range of memory, which can cover anonymous pages as well as mmapped file pages. In the case that the pages are from a shared mmapped file, the volatility set on those file pages is global. Thus much as writes to those pages are shared to other processes, pages marked volatile will be volatile to any other processes that have the file mapped as well. It is advised that processes coordinate when using volatile ranges on shared mappings (much as they must coordinate when writing to shared data). Any uncleared volatility on mmapped files will last until the the file is closed by all users (ie: volatility isn't persistent on disk).

Volatility on anonymous pages are inherited across forks, but cleared on exec.

You can read more about the history of volatile ranges here:
http://permalink.gmane.org/gmane.linux.kernel.mm/98848
http://permalink.gmane.org/gmane.linux.kernel.mm/98676
https://lwn.net/Articles/522135/
https://lwn.net/Kernel/Index/#Volatile_ranges

thanks
-john

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>