Skip to contentSkip to conversation

Support vranges on files

RFC PATCH In review John Stultz 2 participants 15 messages Apr 3, 2013 – Apr 10, 2013

Subsystems (8)
Memory management x86 VFS MM core +4 more subsystems
lore
Share & tools
lore
#0 lore raw

cover Support vranges on files

This patchset is against Minchan's vrange work here:

	https://lkml.org/lkml/2013/3/12/105

Extending it to support volatile ranges on files. In effect providing the same functionality of my earlier file based volatile range patches on-top of Minchan's anonymous volatile range work.

Volatile ranges on files are different then on anonymous memory, because the volatility state can be shared between multiple applications. This makes storing the volatile ranges exclusively in the mm_struct (or in vmas as in Minchan's earlier work) inappropriate.

The patchset starts with some minor cleanup.

Then we introduce the idea of a vrange_root, which provides a interval-tree root and a lock to protect the tree. This structure can then be stored in the mm_struct or in an addres_space. Then the same infrastructure can be used to manage volatile ranges on both anonymous and file backed memory.

Next we introduce a parallel fvrange() syscall for creating volatile ranges directly against files.

And finally, we change the range pruging logic to be able to handle both anonymous and file volatile ranges.

Now there are some quirks still to be resolved with the approach used here. The biggest one being the vrange() call can't be used to create volatile ranges against mmapped files. Instead only the fvrange() can be used to create file backed volatile ranges.

This could be overcome by iterating across all the process VMAs to determine if they're anonymous or file based, and if file-based, create a VMA sized volatile range on the mapping pointed to by the VMA.

But this would have downsides, as Minchan has been clear that he wants to optmize the vrange() calls so that it is very cheap to create and destroy volatile ranges. Having simple per-process ranges be created means we don't have to iterate across the vmas in the range to determine if they're anonymous or file backed. Instead the current vrange() code just creates per process ranges (which may or may not cover mmapped file data), but will only purge anonymous pages in that range. This keeps the vrange() call cheap.

Additionally, just creating or destroying a single range is very simple to do, and requires a fixed amount of memory known up front. Thus we can allocate needed data prior to making any modifications.

But If we were to create a range that crosses anonymous and file backed pages, it must create or destroy multiple per-process or per-file ranges. This could require an unknown number of allocations, opening the possibility of getting an ENOMEM half-way through the operation, leaving the volatile range partially created or destroyed.

So to keep this simple for this first pass, for now we have two syscalls for two types of volatile ranges.

Let me know if you have any thoughts or comments. I'm sure there's plenty of room for improvement here.

In the meantime I'll be playing with some different approaches to try to handle single volatile ranges that cross file and anonymous vmas.

The entire queue, both Minchan's changes and mine can be found here:
git://git.linaro.org/people/jstultz/android-dev.git dev/vrange-minchan

thanks
-john

Cc Michael Kerrisk <[email protected]>
Cc Arun Sharma <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Neil Brown <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc KAMEZAWA Hiroyuki <[email protected]>
Cc Jason Evans <[email protected]>
Cc Paul Turner <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Minchan Kim <[email protected]>

John Stultz (4):
vrange: Make various vrange.c local functions static
vrange: Introduce vrange_root to make vrange structures more flexible
vrange: Support fvrange() syscall for file based volatile ranges
vrange: Enable purging of file backed volatile ranges

 arch/x86/syscalls/syscall_64.tbl |    1 +
 fs/file_table.c                  |    5 +
 fs/inode.c                       |    2 +
 fs/proc/task_mmu.c               |   10 +-
 include/linux/fs.h               |    2 +
 include/linux/mm_types.h         |    4 +-
 include/linux/vrange.h           |   60 ++++---
 include/linux/vrange_types.h     |   22 +++
 kernel/fork.c                    |    2 +-
 mm/vrange.c                      |  334 ++++++++++++++++++++++++++------------
 10 files changed, 308 insertions(+), 134 deletions(-)
 create mode 100644 include/linux/vrange_types.h
Signature · 7 lines
1.7.10.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#1 lore raw

RFC PATCH 1/4 vrange: Make various vrange.c local functions static

Make a number of local functions in vrange.c static.

Cc Michael Kerrisk <[email protected]>
Cc Arun Sharma <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Neil Brown <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc KAMEZAWA Hiroyuki <[email protected]>
Cc Jason Evans <[email protected]>
Cc Paul Turner <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Minchan Kim <[email protected]>
---
 mm/vrange.c |   18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)
mm/vrange.c +9 -9
index c0c5d50..d07884d 100644
@@ -45,7 +45,7 @@ static inline void __set_vrange(struct vrange *range,
 	range->node.last = end_idx;
 }
 
-void lru_add_vrange(struct vrange *vrange)
+static void lru_add_vrange(struct vrange *vrange)
 {
 	spin_lock(&lru_lock);
 	WARN_ON(!list_empty(&vrange->lru));
@@ -53,7 +53,7 @@ void lru_add_vrange(struct vrange *vrange)
 	spin_unlock(&lru_lock);
 }
 
-void lru_remove_vrange(struct vrange *vrange)
+static void lru_remove_vrange(struct vrange *vrange)
 {
 	spin_lock(&lru_lock);
 	if (!list_empty(&vrange->lru))
@@ -130,7 +130,7 @@ static inline void range_resize(struct rb_root *root,
 	__add_range(range, root, mm);
 }
 
-int add_vrange(struct mm_struct *mm,
+static int add_vrange(struct mm_struct *mm,
 			unsigned long start, unsigned long end)
 {
 	struct rb_root *root;
@@ -172,7 +172,7 @@ out:
 	return 0;
 }
 
-int remove_vrange(struct mm_struct *mm,
+static int remove_vrange(struct mm_struct *mm,
 		unsigned long start, unsigned long end)
 {
 	struct rb_root *root;
@@ -292,7 +292,7 @@ out:
 	return ret;
 }
 
-bool __vrange_address(struct mm_struct *mm,
+static bool __vrange_address(struct mm_struct *mm,
 			unsigned long start, unsigned long end)
 {
 	struct rb_root *root = &mm->v_rb;
@@ -387,7 +387,7 @@ static void __vrange_purge(struct mm_struct *mm,
 	}
 }
 
-int try_to_discard_one(struct page *page, struct vm_area_struct *vma,
+static int try_to_discard_one(struct page *page, struct vm_area_struct *vma,
 		unsigned long address)
 {
 	struct mm_struct *mm = vma->vm_mm;
@@ -602,7 +602,7 @@ static int vrange_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end,
 
 }
 
-unsigned int discard_vma_pages(struct zone *zone, struct mm_struct *mm,
+static unsigned int discard_vma_pages(struct zone *zone, struct mm_struct *mm,
 		struct vm_area_struct *vma, unsigned long start,
 		unsigned long end, unsigned int nr_to_discard)
 {
@@ -669,7 +669,7 @@ out:
  * Get next victim vrange from LRU and hold a vrange refcount
  * and vrange->mm's refcount.
  */
-struct vrange *get_victim_vrange(void)
+static struct vrange *get_victim_vrange(void)
 {
 	struct mm_struct *mm;
 	struct vrange *vrange = NULL;
@@ -711,7 +711,7 @@ struct vrange *get_victim_vrange(void)
 	return vrange;
 }
 
-void put_victim_range(struct vrange *vrange)
+static void put_victim_range(struct vrange *vrange)
 {
 	put_vrange(vrange);
 	mmdrop(vrange->mm);
Signature · 7 lines
1.7.10.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#2 lore raw

RFC PATCH 2/4 vrange: Introduce vrange_root to make vrange structures more flexible

In reply to John Stultz

Instead of having the vrange trees hanging directly off of the mm_struct, use a vrange_root structure, which will allow us to have vrange_roots that hang off the mm_struct for anonomous memory, as well as address_space structures for file backed memory.

Cc Michael Kerrisk <[email protected]>
Cc Arun Sharma <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Neil Brown <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc KAMEZAWA Hiroyuki <[email protected]>
Cc Jason Evans <[email protected]>
Cc Paul Turner <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Minchan Kim <[email protected]>
---
 fs/proc/task_mmu.c           |   10 +--
 include/linux/mm_types.h     |    4 +-
 include/linux/vrange.h       |   35 +++++-----
 include/linux/vrange_types.h |   21 ++++++
 kernel/fork.c                |    2 +-
 mm/vrange.c                  |  156 ++++++++++++++++++++++--------------------
 6 files changed, 126 insertions(+), 102 deletions(-)
 create mode 100644 include/linux/vrange_types.h
fs/proc/task_mmu.c +5 -5
index df009f0..11f63d4 100644
@@ -391,13 +391,13 @@ static void *v_start(struct seq_file *m, loff_t *pos)
 	if (!mm || IS_ERR(mm))
 		return mm;
 
-	vrange_lock(mm);
-	root = &mm->v_rb;
+	vrange_lock(&mm->vroot);
+	root = &mm->vroot.v_rb;
 
-	if (RB_EMPTY_ROOT(&mm->v_rb))
+	if (RB_EMPTY_ROOT(&mm->vroot.v_rb))
 		goto out;
 
-	next = rb_first(&mm->v_rb);
+	next = rb_first(&mm->vroot.v_rb);
 	range = vrange_entry(next);
 	while(n > 0 && range) {
 		n--;
@@ -432,7 +432,7 @@ static void v_stop(struct seq_file *m, void *v)
 	struct proc_vrange_private *priv = m->private;
 	if (priv->task) {
 		struct mm_struct *mm = priv->task->mm;
-		vrange_unlock(mm);
+		vrange_unlock(&mm->vroot);
 		mmput(mm);
 		put_task_struct(priv->task);
 	}
include/linux/mm_types.h +2 -2
index 080bf74..2e02a6d 100644
@@ -14,6 +14,7 @@
 #include <linux/uprobes.h>
 #include <linux/page-flags-layout.h>
 #include <linux/mutex.h>
+#include <linux/vrange_types.h>
 #include <asm/page.h>
 #include <asm/mmu.h>
 
@@ -353,8 +354,7 @@ struct mm_struct {
 
 
 #ifdef CONFIG_MMU
-	struct rb_root v_rb;		/* vrange rb tree */
-	struct mutex v_lock;		/* Protect v_rb */
+	struct vrange_root vroot;
 #endif
 	unsigned long hiwater_rss;	/* High-watermark of RSS usage */
 	unsigned long hiwater_vm;	/* High-water virtual memory usage */
include/linux/vrange.h +16 -19
index 4bcec40..b9b219c 100644
@@ -1,42 +1,39 @@
 #ifndef _LINUX_VRANGE_H
 #define _LINUX_VRANGE_H
 
-#include <linux/mutex.h>
-#include <linux/interval_tree.h>
+#include <linux/vrange_types.h>
 #include <linux/mm.h>
 
-struct vrange {
-	struct interval_tree_node node;
-	bool purged;
-	struct mm_struct *mm;
-	struct list_head lru; /* protected by lru_lock */
-	atomic_t refcount;
-};
-
 #define vrange_entry(ptr) \
 	container_of(ptr, struct vrange, node.rb)
 
 #ifdef CONFIG_MMU
-struct mm_struct;
 
 static inline void mm_init_vrange(struct mm_struct *mm)
 {
-	mm->v_rb = RB_ROOT;
-	mutex_init(&mm->v_lock);
+	mm->vroot.v_rb = RB_ROOT;
+	mutex_init(&mm->vroot.v_lock);
+}
+
+static inline void vrange_lock(struct vrange_root *vroot)
+{
+	mutex_lock(&vroot->v_lock);
 }
 
-static inline void vrange_lock(struct mm_struct *mm)
+static inline void vrange_unlock(struct vrange_root *vroot)
 {
-	mutex_lock(&mm->v_lock);
+	mutex_unlock(&vroot->v_lock);
 }
 
-static inline void vrange_unlock(struct mm_struct *mm)
+static inline struct mm_struct *vrange_get_owner_mm(struct vrange *vrange)
 {
-	mutex_unlock(&mm->v_lock);
+
+	return container_of(vrange->owner, struct mm_struct, vroot);
 }
 
-extern void exit_vrange(struct mm_struct *mm);
+
 void vrange_init(void);
+extern void mm_exit_vrange(struct mm_struct *mm);
 int discard_vpage(struct page *page);
 bool vrange_address(struct mm_struct *mm, unsigned long start,
 			unsigned long end);
@@ -50,7 +47,7 @@ void lru_move_vrange_to_head(struct mm_struct *mm, unsigned long address);
 
 static inline void vrange_init(void) {};
 static inline void mm_init_vrange(struct mm_struct *mm) {};
-static inline void exit_vrange(struct mm_struct *mm);
+static inline void mm_exit_vrange(struct mm_struct *mm);
 
 static inline bool vrange_address(struct mm_struct *mm, unsigned long start,
 		unsigned long end) { return false; };
include/linux/vrange_types.h new file +21 -0
new file mode 100644
index 0000000..bede336
@@ -0,0 +1,21 @@
+#ifndef _LINUX_VRANGE_TYPES_H
+#define _LINUX_VRANGE_TYPES_H
+
+#include <linux/mutex.h>
+#include <linux/interval_tree.h>
+
+struct vrange_root {
+	struct rb_root v_rb;		/* vrange rb tree */
+	struct mutex v_lock;		/* Protect v_rb */
+};
+
+
+struct vrange {
+	struct interval_tree_node node;
+	struct vrange_root *owner;
+	bool purged;
+	struct list_head lru; /* protected by lru_lock */
+	atomic_t refcount;
+};
+#endif
+
kernel/fork.c +1 -1
index e3aa120..f2da4a0 100644
@@ -614,7 +614,7 @@ void mmput(struct mm_struct *mm)
 
 	if (atomic_dec_and_test(&mm->mm_users)) {
 		uprobe_clear_state(mm);
-		exit_vrange(mm);
+		mm_exit_vrange(mm);
 		exit_aio(mm);
 		ksm_exit(mm);
 		khugepaged_exit(mm); /* must run before exit_mmap */
mm/vrange.c +81 -75

410 lines not on this page · raw message

Signature · 7 lines
1.7.10.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#3 lore raw

RFC PATCH 3/4 vrange: Support fvrange() syscall for file based volatile ranges

In reply to John Stultz

Add vrange support on addres_space structures, and add fvrange() syscall for creating ranges on address_space structures.

Cc Michael Kerrisk <[email protected]>
Cc Arun Sharma <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Neil Brown <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc KAMEZAWA Hiroyuki <[email protected]>
Cc Jason Evans <[email protected]>
Cc Paul Turner <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Minchan Kim <[email protected]>
---
 arch/x86/syscalls/syscall_64.tbl |    1 +
 fs/file_table.c                  |    5 +++
 fs/inode.c                       |    2 ++
 include/linux/fs.h               |    2 ++
 include/linux/vrange.h           |   19 +++++++++-
 include/linux/vrange_types.h     |    1 +
 mm/vrange.c                      |   72 +++++++++++++++++++++++++++++++++++++-
 7 files changed, 100 insertions(+), 2 deletions(-)
arch/x86/syscalls/syscall_64.tbl +1 -0
index dc332bd..910d9f3 100644
@@ -321,6 +321,7 @@
 312	common	kcmp			sys_kcmp
 313	common	finit_module		sys_finit_module
 314	common	vrange			sys_vrange
+315	common	fvrange			sys_fvrange
 
 #
 # x32-specific system call numbers start at 512 to avoid cache impact
fs/file_table.c +5 -0
index cd4d87a..61c8aaa 100644
@@ -26,6 +26,7 @@
 #include <linux/hardirq.h>
 #include <linux/task_work.h>
 #include <linux/ima.h>
+#include <linux/vrange.h>
 
 #include <linux/atomic.h>
 
@@ -244,6 +245,10 @@ static void __fput(struct file *file)
 			file->f_op->fasync(-1, file, 0);
 	}
 	ima_file_free(file);
+
+	/* drop all vranges on last close */
+	mapping_exit_vrange(inode->i_mapping);
+
 	if (file->f_op && file->f_op->release)
 		file->f_op->release(inode, file);
 	security_file_free(file);
fs/inode.c +2 -0
index f5f7c06..4707c95 100644
@@ -17,6 +17,7 @@
 #include <linux/prefetch.h>
 #include <linux/buffer_head.h> /* for inode_has_buffers */
 #include <linux/ratelimit.h>
+#include <linux/vrange.h>
 #include "internal.h"
 
 /*
@@ -350,6 +351,7 @@ void address_space_init_once(struct address_space *mapping)
 	spin_lock_init(&mapping->private_lock);
 	mapping->i_mmap = RB_ROOT;
 	INIT_LIST_HEAD(&mapping->i_mmap_nonlinear);
+	mapping_init_vrange(mapping);
 }
 EXPORT_SYMBOL(address_space_init_once);
 
include/linux/fs.h +2 -0
index 2c28271..6f86c7c 100644
@@ -27,6 +27,7 @@
 #include <linux/lockdep.h>
 #include <linux/percpu-rwsem.h>
 #include <linux/blk_types.h>
+#include <linux/vrange_types.h>
 
 #include <asm/byteorder.h>
 #include <uapi/linux/fs.h>
@@ -411,6 +412,7 @@ struct address_space {
 	struct rb_root		i_mmap;		/* tree of private and shared mappings */
 	struct list_head	i_mmap_nonlinear;/*list VM_NONLINEAR mappings */
 	struct mutex		i_mmap_mutex;	/* protect tree, count, list */
+	struct vrange_root	vroot;
 	/* Protected by tree_lock together with the radix tree */
 	unsigned long		nrpages;	/* number of total pages */
 	pgoff_t			writeback_index;/* writeback starts here */
include/linux/vrange.h +18 -1
index b9b219c..91960eb 100644
@@ -3,6 +3,7 @@
 
 #include <linux/vrange_types.h>
 #include <linux/mm.h>
+#include <linux/fs.h>
 
 #define vrange_entry(ptr) \
 	container_of(ptr, struct vrange, node.rb)
@@ -11,10 +12,19 @@
 
 static inline void mm_init_vrange(struct mm_struct *mm)
 {
+	mm->vroot.type = VRANGE_ANON;
 	mm->vroot.v_rb = RB_ROOT;
 	mutex_init(&mm->vroot.v_lock);
 }
 
+static inline void mapping_init_vrange(struct address_space *mapping)
+{
+	mapping->vroot.type = VRANGE_FILE;
+	mapping->vroot.v_rb = RB_ROOT;
+	mutex_init(&mapping->vroot.v_lock);
+}
+
+
 static inline void vrange_lock(struct vrange_root *vroot)
 {
 	mutex_lock(&vroot->v_lock);
@@ -25,15 +35,22 @@ static inline void vrange_unlock(struct vrange_root *vroot)
 	mutex_unlock(&vroot->v_lock);
 }
 
-static inline struct mm_struct *vrange_get_owner_mm(struct vrange *vrange)
+static inline int vrange_type(struct vrange *vrange)
 {
+	return vrange->owner->type;
+}
 
+static inline struct mm_struct *vrange_get_owner_mm(struct vrange *vrange)
+{
+	if (vrange_type(vrange) != VRANGE_ANON)
+		return NULL;
 	return container_of(vrange->owner, struct mm_struct, vroot);
 }
 
 
 void vrange_init(void);
 extern void mm_exit_vrange(struct mm_struct *mm);
+extern void mapping_exit_vrange(struct address_space *mapping);
 int discard_vpage(struct page *page);
 bool vrange_address(struct mm_struct *mm, unsigned long start,
 			unsigned long end);
include/linux/vrange_types.h +1 -0
index bede336..c7154e4 100644
@@ -7,6 +7,7 @@
 struct vrange_root {
 	struct rb_root v_rb;		/* vrange rb tree */
 	struct mutex v_lock;		/* Protect v_rb */
+	enum {VRANGE_ANON, VRANGE_FILE} type; /* range root type */
 };
 
 
mm/vrange.c +71 -1

108 lines not on this page · raw message

Signature · 7 lines
1.7.10.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#4 lore raw

RFC PATCH 4/4 vrange: Enable purging of file backed volatile ranges

In reply to John Stultz

Rework the victim range selection to also support
file backed volatile ranges.

Cc Michael Kerrisk <[email protected]>
Cc Arun Sharma <[email protected]>
Cc Mel Gorman <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Dave Hansen <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Neil Brown <[email protected]>
Cc Mike Hommey <[email protected]>
Cc Taras Glek <[email protected]>
Cc KOSAKI Motohiro <[email protected]>
Cc KAMEZAWA Hiroyuki <[email protected]>
Cc Jason Evans <[email protected]>
Cc Paul Turner <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Michel Lespinasse <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Minchan Kim <[email protected]>
---
 include/linux/vrange.h |    8 ++++
 mm/vrange.c            |  118 +++++++++++++++++++++++++++++++++---------------
 2 files changed, 89 insertions(+), 37 deletions(-)
include/linux/vrange.h +8 -0
index 91960eb..bada2bd 100644
@@ -47,6 +47,14 @@ static inline struct mm_struct *vrange_get_owner_mm(struct vrange *vrange)
 	return container_of(vrange->owner, struct mm_struct, vroot);
 }
 
+static inline
+struct address_space *vrange_get_owner_mapping(struct vrange *vrange)
+{
+	if (vrange_type(vrange) != VRANGE_FILE)
+		return NULL;
+	return container_of(vrange->owner, struct address_space, vroot);
+}
+
 
 void vrange_init(void);
 extern void mm_exit_vrange(struct mm_struct *mm);
mm/vrange.c +81 -37

163 lines not on this page · raw message

Signature · 7 lines
1.7.10.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#5 lore raw
In reply to John Stultz

Hey John,

First of all, I should confess I just glanced your code and poped several questions. If I miss something, please slap me.

Quoted 14 lines · John Stultz

This patchset is against Minchan's vrange work here:

	https://lkml.org/lkml/2013/3/12/105

Extending it to support volatile ranges on files. In effect providing the same functionality of my earlier file based volatile range patches on-top of Minchan's anonymous volatile range work.

Volatile ranges on files are different then on anonymous memory, because the volatility state can be shared between multiple applications. This makes storing the volatile ranges exclusively in the mm_struct (or in vmas as in Minchan's earlier work) inappropriate.

Quoted 7 lines

The patchset starts with some minor cleanup.

Then we introduce the idea of a vrange_root, which provides a interval-tree root and a lock to protect the tree. This structure can then be stored in the mm_struct or in an addres_space. Then the same infrastructure can be used to manage volatile ranges on both anonymous and file backed memory.

Thanks for the above two patches. It is a nice cleanup.

Quoted 3 lines

Next we introduce a parallel fvrange() syscall for creating volatile ranges directly against files.

Okay. It seems you want to replace ashmem interface with fvrange. I dobut we have to eat a slot for system call. Can't we add "int fd" in vrange systemcall without inventing new wheel?

Quoted 3 lines

And finally, we change the range pruging logic to be able to handle both anonymous and file volatile ranges.

Okay. Then, what's the semantic file-vrange?

There is a file F. Process A mapped some part of file into his address space. Then, Process B calls fvrange same part. As I looked over your code, it purges the range although process B is using now. Right? Is it your intention? Maybe isn't.

Let's define fvrange's semantic same with anon-vrange. If there is a process using range with non-volatile, at least, we shouldn't purge at all.

So your [4/4] should investigate all processes mapped the page atomically. You could do it with i_mmap_mutex and vrange_lock and percolate the logic into try_to_discard_vpage.

Quoted 4 lines

Now there are some quirks still to be resolved with the approach used here. The biggest one being the vrange() call can't be used to create volatile ranges against mmapped files. Instead only the

Why?

Quoted 1 line

fvrange() can be used to create file backed volatile ranges.

I could't understand your point. It would be better to explain my thought firstly then, you could point out something I am missing now. Look below.

Quoted 5 lines

This could be overcome by iterating across all the process VMAs to determine if they're anonymous or file based, and if file-based, create a VMA sized volatile range on the mapping pointed to by the VMA.

It needs just when we start to discard pages. Simply, it is related to reclaim path, NOT system call path so it's not a problem.

Quoted 9 lines

But this would have downsides, as Minchan has been clear that he wants to optmize the vrange() calls so that it is very cheap to create and destroy volatile ranges. Having simple per-process ranges be created means we don't have to iterate across the vmas in the range to determine if they're anonymous or file backed. Instead the current vrange() code just creates per process ranges (which may or may not cover mmapped file data), but will only purge anonymous pages in that range. This keeps the vrange() call cheap.

Right.

Quoted 8 lines

Additionally, just creating or destroying a single range is very simple to do, and requires a fixed amount of memory known up front. Thus we can allocate needed data prior to making any modifications.

But If we were to create a range that crosses anonymous and file backed pages, it must create or destroy multiple per-process or per-file ranges. This could require an unknown number of allocations,

This is a part I can fail to parse your opinion.

Quoted 5 lines

opening the possibility of getting an ENOMEM half-way through the operation, leaving the volatile range partially created or destroyed.

So to keep this simple for this first pass, for now we have two syscalls for two types of volatile ranges.

My idea is following as

vrange(fd, start, len, mode, behavior)

A) fd = 0

1) system call context - vrange system call registers new vrange in mm_struct.
2) Add new vrange into LRU
3) reclaim context - walk with rmap to confirm all processes make the range with volatile -> discard

B) fd = 1

1) system call context - vrange system call registers new vrange in address_space
2) Add new vrange into LRU
3) reclaim context - walk with rmap to confirm all processes make the range with volatile -> discard

What's the problem in this logic?

Quoted 14 lines

Let me know if you have any thoughts or comments. I'm sure there's plenty of room for improvement here.

In the meantime I'll be playing with some different approaches to try to handle single volatile ranges that cross file and anonymous vmas.

The entire queue, both Minchan's changes and mine can be found here:
git://git.linaro.org/people/jstultz/android-dev.git dev/vrange-minchan

thanks
-john

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#6 lore raw
Quoted 6 lines · Minchan Kim
Quoted 2 lines · John Stultz

Next we introduce a parallel fvrange() syscall for creating volatile ranges directly against files.

Okay. It seems you want to replace ashmem interface with fvrange. I dobut we have to eat a slot for system call. Can't we add "int fd" in vrange systemcall without inventing new wheel?

Sure, that would be doable. I just added the new syscall to make the differences in functionality clear. Once the subtleties are understood, we can condense things down if we think its best.

Quoted 8 lines
Quoted 2 lines

And finally, we change the range pruging logic to be able to handle both anonymous and file volatile ranges.

Okay. Then, what's the semantic file-vrange?

There is a file F. Process A mapped some part of file into his address space. Then, Process B calls fvrange same part. As I looked over your code, it purges the range although process B is using now. Right? Is it your intention? Maybe isn't.

Not sure if you're example has a type-o and you meant "process A is using it"? If so, yes. The point is the volatility is shared and consistent across all users of the file, in the same way the data in the file is shared. If process B punched a hole in the file, process A would see the effect immediately. With volatile ranges, the hole punching is just delayed and possibly done later by the kernel, in effect on behalf of process B, so the behavior is the same.

Consider the case where we could have two processes mmap a tmpfs file in order to create a circular buffer shared between them. You could then have a producer/consumer relationship with two processes where any data not between the head & tail offsets were marked volatile. The producer would mark tail+size non-volatile, write the data, and update the tail offset. The consumer would read data from the head offset, mark the just-read range as volatile, and update the offset.

In this example, the producer would be the only process to mark data non-volatile, while the consumer would be the only one marking ranges volatile. Thus the state of volatility would need to be an attribute of the file, not the process, in the same way the shared data is.

Is that clear?

Quoted 3 lines

Let's define fvrange's semantic same with anon-vrange. If there is a process using range with non-volatile, at least, we shouldn't purge at all.

So this I'm not in agreement with.

Anonymous pages are for the most part not shared, except via COW. And for the COW case, yes, I agree, we shouldn't purge those pages.

Similarly (and I have yet to handle this in the code), for private mapped files, those pages shouldn't be purged either (or purging them shouldn't affect the private mapped pages - not sure which direction to go here).

But for shared mapped files, we need to keep the volatility state shared as well.

Quoted 4 lines
Quoted 3 lines

Now there are some quirks still to be resolved with the approach used here. The biggest one being the vrange() call can't be used to create volatile ranges against mmapped files. Instead only the

Why?

As explained above, the volatility is shared like the data. The current vrange() code creates per-mm volatile ranges, which aren't shared.

Quoted 6 lines
Quoted 2 lines

fvrange() can be used to create file backed volatile ranges.

I could't understand your point. It would be better to explain my thought firstly then, you could point out something I am missing now. Look below.

Quoted 6 lines
Quoted 4 lines

This could be overcome by iterating across all the process VMAs to determine if they're anonymous or file based, and if file-based, create a VMA sized volatile range on the mapping pointed to by the VMA.

It needs just when we start to discard pages. Simply, it is related to reclaim path, NOT system call path so it's not a problem.

The reason we can't defer this to only the reclaim path is if volatile ranges on shared mappings are stored in the mm_struct, if process A sets up a volatile range on a shared mapping, but stores the volatility in its own mm, then process B wants to clear the volatility on the range, process B would have to iterate over all processes that have those file vmas mapped and change them.

Additionally if process A sets up a volatile range on a shared mapped file, then quits, the volatility state dies with that process.

Either way, its not just a simple matter of handling data on your own mm_struct. That's fine for the process' own anonymous memory, but doesn't work for shared file mappings.

Quoted 11 lines
Quoted 9 lines

But this would have downsides, as Minchan has been clear that he wants to optmize the vrange() calls so that it is very cheap to create and destroy volatile ranges. Having simple per-process ranges be created means we don't have to iterate across the vmas in the range to determine if they're anonymous or file backed. Instead the current vrange() code just creates per process ranges (which may or may not cover mmapped file data), but will only purge anonymous pages in that range. This keeps the vrange() call cheap.

Right.

Quoted 8 lines
Quoted 7 lines

Additionally, just creating or destroying a single range is very simple to do, and requires a fixed amount of memory known up front. Thus we can allocate needed data prior to making any modifications.

But If we were to create a range that crosses anonymous and file backed pages, it must create or destroy multiple per-process or per-file ranges. This could require an unknown number of allocations,

This is a part I can fail to parse your opinion.

So if we were in the vrange() code to iterate over all the VMAs in the range, creating VMA sizes ranges on either the mm_struct or the backing address_space where appropriate, its possible that we could hit an ENOMEM half way through the operation. This would leaving the range in an inconsistent state: partially marked, and potentially causing us to lose the purged state on the subranges.

Quoted 4 lines
Quoted 4 lines

opening the possibility of getting an ENOMEM half-way through the operation, leaving the volatile range partially created or destroyed.

Quoted 8 lines
Quoted 2 lines

So to keep this simple for this first pass, for now we have two syscalls for two types of volatile ranges.

My idea is following as

vrange(fd, start, len, mode, behavior)

A) fd = 0

Well we'd probably need to use -1 or something that would be an invalid fd here.

And really, I think having separate interfaces might be good, just as there are separate madvise() and fadvise() calls (and when all this is done, we may need to re-visit the new syscall vs new madvise/fadvise flags decision).

Quoted 8 lines

1) system call context - vrange system call registers new vrange in mm_struct.
2) Add new vrange into LRU
3) reclaim context - walk with rmap to confirm all processes make

    the range with volatile -> discard

B) fd = 1

The fd would just need to be valid right, not 1.

Quoted 7 lines

1) system call context - vrange system call registers new vrange in address_space
2) Add new vrange into LRU
3) reclaim context - walk with rmap to confirm all processes make

    the range with volatile -> discard

What's the problem in this logic?

The problem is only if in the first case, the volatile range being created crosses over both anonymous and shared file mmap pages. In that case we have to create appropriate sub-ranges on the mm_struct, and sub-ranges on the address_space of the mmaped file.

This isn't impossible to do, but again, the handling of errors mid-way through creating subranges is problematic (there may be yet a way around it, I just haven't thought of it yet).

Thus with my patches, I simplified the problem a bit by partitioning it into two separate problems and two separate interfaces: Volatile ranges that are created by the vrange() call won't affect mmaped pages, only anonymous pages. We may create a range that covers them, but the volatility isn't shared with other processes and the purging logic still skips file pages. If you want to to create a volatile range on file pages, you have to use fvrange().

Of course, my patchset has its own inconsistencies too, since if a range is marked non-volatile that covers a mmapped file that has been marked volatile, that volatility would persist. So I probably should return an error if the vrange call covers any mmapped files.

Also, to be clear, I'm not saying that we *have* to partition these operations into two separate behaviors, but I think having two separate behaviors at first helps makes clear the subtleties of the differences between them.

Let me know if any of this helps your understanding. :)

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#7 lore raw

Hi John,

Quoted 4 lines · John Stultz
Quoted 3 lines · Minchan Kim
Quoted 2 lines · John Stultz

Next we introduce a parallel fvrange() syscall for creating volatile ranges directly against files.

Quoted 8 lines
Quoted 3 lines

Okay. It seems you want to replace ashmem interface with fvrange. I dobut we have to eat a slot for system call. Can't we add "int fd" in vrange systemcall without inventing new wheel?

Sure, that would be doable. I just added the new syscall to make the differences in functionality clear.
Once the subtleties are understood, we can condense things down if we think its best.

Fair enough.

Quoted 28 lines
Quoted 10 lines
Quoted 4 lines

And finally, we change the range pruging logic to be able to handle both anonymous and file volatile ranges.

Okay. Then, what's the semantic file-vrange?

There is a file F. Process A mapped some part of file into his address space. Then, Process B calls fvrange same part. As I looked over your code, it purges the range although process B is using now. Right? Is it your intention? Maybe isn't.

Not sure if you're example has a type-o and you meant "process A is using it"? If so, yes. The point is the volatility is shared and consistent across all users of the file, in the same way the data in the file is shared. If process B punched a hole in the file, process A would see the effect immediately. With volatile ranges, the hole punching is just delayed and possibly done later by the kernel, in effect on behalf of process B, so the behavior is the same.

Consider the case where we could have two processes mmap a tmpfs file in order to create a circular buffer shared between them. You could then have a producer/consumer relationship with two processes where any data not between the head & tail offsets were marked volatile. The producer would mark tail+size non-volatile, write the data, and update the tail offset. The consumer would read data from the head offset, mark the just-read range as volatile, and update the offset.

Quoted 7 lines

In this example, the producer would be the only process to mark data non-volatile, while the consumer would be the only one marking ranges volatile. Thus the state of volatility would need to be an attribute of the file, not the process, in the same way the shared data is.

Is that clear?

Yes, I got your point that you meant shared mapping. Let's enumerate more examples.

1. Process A mapped FILE A with MAP_SHARED
Process B mapped FILE A with MAP_SHARED
Process C calls fvrange
Discard all pages of process A and B -> Make sense to me.

2. Process A mapped FILE A with MAP_PRIVATE and is using it with read-only Process B mapped FILE A with MAP_PRIVATE and is using it with write-only Process C calls fvrange

What does it happens? I expect process A lost all pages while process B keeps COWed pages.

3. Process A mapped FILE A with MAP_PRIVATE and is using it with read/write Process C calls fvrange

Some pages non-COWed in process A are lost while some pages COWed are kept. Mixing.

Above all are your intention?
It would be very clear if you should have wrote down semantic you intent about private mapped file and shared mapped file. ;-)

Quoted 8 lines
Quoted 6 lines

Let's define fvrange's semantic same with anon-vrange. If there is a process using range with non-volatile, at least, we shouldn't purge at all.

So this I'm not in agreement with.

I got your point.

Quoted 8 lines

Anonymous pages are for the most part not shared, except via COW. And for the COW case, yes, I agree, we shouldn't purge those pages.

Similarly (and I have yet to handle this in the code), for private mapped files, those pages shouldn't be purged either (or purging them shouldn't affect the private mapped pages - not sure which direction to go here).

Yeb. It's questionable.
It seems fallocate for punch hole removes non-COWed pages although they are mapped privately if I didn't miss something to read code. If I was right, it looks very strange to me. COWed pages remain in memory while NOT-YET-COWed pages are discarded. :( Ho, Hmm.

Quoted 5 lines

But for shared mapped files, we need to keep the volatility state shared as well.

Quoted 8 lines
Quoted 4 lines
Quoted 3 lines

Now there are some quirks still to be resolved with the approach used here. The biggest one being the vrange() call can't be used to create volatile ranges against mmapped files. Instead only the

Why?

As explained above, the volatility is shared like the data. The current vrange() code creates per-mm volatile ranges, which aren't shared.

Strictly speaking, we can do it by only per-mm volatile range, I think. But the concern if we choose the approach is that what you mention in below is we have to iterate all process's mm_sturct to check in system call context. Of course, I don't like it and too bad design.

Quoted 15 lines
Quoted 8 lines
Quoted 4 lines

fvrange() can be used to create file backed volatile ranges.

I could't understand your point. It would be better to explain my thought firstly then, you could point out something I am missing now. Look below.

Quoted 6 lines
Quoted 4 lines

This could be overcome by iterating across all the process VMAs to determine if they're anonymous or file based, and if file-based, create a VMA sized volatile range on the mapping pointed to by the VMA.

It needs just when we start to discard pages. Simply, it is related to reclaim path, NOT system call path so it's not a problem.

Quoted 6 lines

The reason we can't defer this to only the reclaim path is if volatile ranges on shared mappings are stored in the mm_struct, if process A sets up a volatile range on a shared mapping, but stores the volatility in its own mm, then process B wants to clear the volatility on the range, process B would have to iterate over all processes that have those file vmas mapped and change them.

Right. I think iterating all of relevant vmas isn't big cost in normal situation but it could be rather bigger when the memory pressure is severe, especially for file-backed pages because it's not even read/write lock.
I'd like to minimize the system call overhead if possible.

Quoted 4 lines

Additionally if process A sets up a volatile range on a shared mapped file, then quits, the volatility state dies with that process.

Yes, so don't you want to use vrange system call for mmaped-file range at the moment?

Quoted 4 lines

Either way, its not just a simple matter of handling data on your own mm_struct. That's fine for the process' own anonymous memory, but doesn't work for shared file mappings.

Agreed.

Quoted 44 lines
Quoted 13 lines
Quoted 11 lines

But this would have downsides, as Minchan has been clear that he wants to optmize the vrange() calls so that it is very cheap to create and destroy volatile ranges. Having simple per-process ranges be created means we don't have to iterate across the vmas in the range to determine if they're anonymous or file backed. Instead the current vrange() code just creates per process ranges (which may or may not cover mmapped file data), but will only purge anonymous pages in that range. This keeps the vrange() call cheap.

Right.

Quoted 8 lines
Quoted 7 lines

Additionally, just creating or destroying a single range is very simple to do, and requires a fixed amount of memory known up front. Thus we can allocate needed data prior to making any modifications.

But If we were to create a range that crosses anonymous and file backed pages, it must create or destroy multiple per-process or per-file ranges. This could require an unknown number of allocations,

This is a part I can fail to parse your opinion.

So if we were in the vrange() code to iterate over all the VMAs in the range, creating VMA sizes ranges on either the mm_struct or the backing address_space where appropriate, its possible that we could hit an ENOMEM half way through the operation. This would leaving the range in an inconsistent state: partially marked, and potentially causing us to lose the purged state on the subranges.

Quoted 4 lines
Quoted 4 lines

opening the possibility of getting an ENOMEM half-way through the operation, leaving the volatile range partially created or destroyed.

Quoted 8 lines
Quoted 2 lines

So to keep this simple for this first pass, for now we have two syscalls for two types of volatile ranges.

My idea is following as

vrange(fd, start, len, mode, behavior)

A) fd = 0

Quoted 7 lines

Well we'd probably need to use -1 or something that would be an invalid fd here.

And really, I think having separate interfaces might be good, just as there are separate madvise() and fadvise() calls (and when all this is done, we may need to re-visit the new syscall vs new madvise/fadvise flags decision).

It does make sense in this phase where we are still RFC.

Quoted 19 lines
Quoted 9 lines

1) system call context - vrange system call registers new vrange in mm_struct.
2) Add new vrange into LRU
3) reclaim context - walk with rmap to confirm all processes make the range with volatile -> discard

B) fd = 1

The fd would just need to be valid right, not 1.

Quoted 7 lines

1) system call context - vrange system call registers new vrange in address_space
2) Add new vrange into LRU
3) reclaim context - walk with rmap to confirm all processes make the range with volatile -> discard

What's the problem in this logic?

Quoted 8 lines

The problem is only if in the first case, the volatile range being created crosses over both anonymous and shared file mmap pages. In that case we have to create appropriate sub-ranges on the mm_struct, and sub-ranges on the address_space of the mmaped file.

This isn't impossible to do, but again, the handling of errors mid-way through creating subranges is problematic (there may be yet a way around it, I just haven't thought of it yet).

Fair enough.

Quoted 9 lines

Thus with my patches, I simplified the problem a bit by partitioning it into two separate problems and two separate interfaces: Volatile ranges that are created by the vrange() call won't affect mmaped pages, only anonymous pages. We may create a range that covers them, but the volatility isn't shared with other processes and the purging logic still skips file pages. If you want to to create a volatile range on file pages, you have to use fvrange().

Okay, I got your intention by this paragraph.
You don't want to handle file pages with vrange() and want to use fvrange for file pages. I don't oppose it but please write down why we did like you explained to me on above.
It would make reviewers happier.

Quoted 5 lines

Of course, my patchset has its own inconsistencies too, since if a range is marked non-volatile that covers a mmapped file that has been marked volatile, that volatility would persist. So I probably should return an error if the vrange call covers any mmapped files.

Hmm, if you intend to separate anon and file with vrange and fvrange's separate data structure, it's no problem?

Quoted 6 lines

Also, to be clear, I'm not saying that we *have* to partition these operations into two separate behaviors, but I think having two separate behaviors at first helps makes clear the subtleties of the differences between them.

I got your point and I am thinking about that more.

Quoted 3 lines

Let me know if any of this helps your understanding. :)

Thank you very much. John!

Looking forward to seeing you in SF.

Quoted 9 lines

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#8 lore raw

Hello John,

As you know, userland people wanted to handle vrange with mmaped pointer rather than fd-based and see the SIGBUS so I thought more about semantic of vrange and want to make it very clear and easy. So I suggest below semantic(Of course, it's not rock solid).

mvrange(start_addr, lengh, mode, behavior)

It's same with that I suggested lately but different name, just adding prefix "m". It's per-process model(ie, mm_struct vrange) so if process is exited, "volatility" isn't valid any more. It isn't a problem in anonymous but could be in file-vrange so let's introduce fvrange for covering the problem.

fvrange(int fd, start_offset, length, mode, behavior)

First of all, let's see mvrange with anonymous and file page POV.

1) anon-mvrange

The page in volaitle range will be purged only if all of processes marked the range as volatile.

If A process calls mvrange and is forked, vrange could be copied from parent to child so not-yet-COWed pages could be purged unless either one of both processes marks NO_VOLATILE explicitly.

Of course, COWed page could be purged easily because there is no link any more.

2) file-mvrange

A page in volatile range will be purged only if all of processes mapped the page marked it as volatile AND there is no process mapped the page as "private". IOW, all of the process mapped the page should map it with "shared" for purging.

So, all of processes should mark each address range in own process context if they want to collaborate with shared mapped file and gaurantee there is no process mapped the range with "private".

Of course, volatility state will be terminated as the process is gone.

3) fvrange

It's same with 2) but volatility state could be persistent in address_space until someone calls fvrange(NO_VOLATILE).
So it could remove the weakness of 2).

What do you think about above semantic?

If you don't have any problem, we could implement it. I think 1) and 2) could be handled with my base code for anon-vrange handling with tweaking file-vrange and need your new patches in address_space for handling 3).

Quoted 313 lines · Minchan Kim

Hi John,

Quoted 4 lines · John Stultz
Quoted 3 lines · Minchan Kim
Quoted 2 lines · John Stultz

Next we introduce a parallel fvrange() syscall for creating volatile ranges directly against files.

Quoted 8 lines
Quoted 3 lines

Okay. It seems you want to replace ashmem interface with fvrange. I dobut we have to eat a slot for system call. Can't we add "int fd" in vrange systemcall without inventing new wheel?

Sure, that would be doable. I just added the new syscall to make the differences in functionality clear.
Once the subtleties are understood, we can condense things down if we think its best.

Fair enough.

Quoted 28 lines
Quoted 10 lines
Quoted 4 lines

And finally, we change the range pruging logic to be able to handle both anonymous and file volatile ranges.

Okay. Then, what's the semantic file-vrange?

There is a file F. Process A mapped some part of file into his address space. Then, Process B calls fvrange same part. As I looked over your code, it purges the range although process B is using now. Right? Is it your intention? Maybe isn't.

Not sure if you're example has a type-o and you meant "process A is using it"? If so, yes. The point is the volatility is shared and consistent across all users of the file, in the same way the data in the file is shared. If process B punched a hole in the file, process A would see the effect immediately. With volatile ranges, the hole punching is just delayed and possibly done later by the kernel, in effect on behalf of process B, so the behavior is the same.

Consider the case where we could have two processes mmap a tmpfs file in order to create a circular buffer shared between them. You could then have a producer/consumer relationship with two processes where any data not between the head & tail offsets were marked volatile. The producer would mark tail+size non-volatile, write the data, and update the tail offset. The consumer would read data from the head offset, mark the just-read range as volatile, and update the offset.

Quoted 7 lines

In this example, the producer would be the only process to mark data non-volatile, while the consumer would be the only one marking ranges volatile. Thus the state of volatility would need to be an attribute of the file, not the process, in the same way the shared data is.

Is that clear?

Yes, I got your point that you meant shared mapping. Let's enumerate more examples.

1. Process A mapped FILE A with MAP_SHARED
Process B mapped FILE A with MAP_SHARED
Process C calls fvrange
Discard all pages of process A and B -> Make sense to me.

2. Process A mapped FILE A with MAP_PRIVATE and is using it with read-only Process B mapped FILE A with MAP_PRIVATE and is using it with write-only Process C calls fvrange

What does it happens? I expect process A lost all pages while process B keeps COWed pages.

3. Process A mapped FILE A with MAP_PRIVATE and is using it with read/write Process C calls fvrange

Some pages non-COWed in process A are lost while some pages COWed are kept. Mixing.

Above all are your intention?
It would be very clear if you should have wrote down semantic you intent about private mapped file and shared mapped file. ;-)

Quoted 8 lines
Quoted 6 lines

Let's define fvrange's semantic same with anon-vrange. If there is a process using range with non-volatile, at least, we shouldn't purge at all.

So this I'm not in agreement with.

I got your point.

Quoted 8 lines

Anonymous pages are for the most part not shared, except via COW. And for the COW case, yes, I agree, we shouldn't purge those pages.

Similarly (and I have yet to handle this in the code), for private mapped files, those pages shouldn't be purged either (or purging them shouldn't affect the private mapped pages - not sure which direction to go here).

Yeb. It's questionable.
It seems fallocate for punch hole removes non-COWed pages although they are mapped privately if I didn't miss something to read code. If I was right, it looks very strange to me. COWed pages remain in memory while NOT-YET-COWed pages are discarded. :( Ho, Hmm.

Quoted 5 lines

But for shared mapped files, we need to keep the volatility state shared as well.

Quoted 8 lines
Quoted 4 lines
Quoted 3 lines

Now there are some quirks still to be resolved with the approach used here. The biggest one being the vrange() call can't be used to create volatile ranges against mmapped files. Instead only the

Why?

As explained above, the volatility is shared like the data. The current vrange() code creates per-mm volatile ranges, which aren't shared.

Strictly speaking, we can do it by only per-mm volatile range, I think. But the concern if we choose the approach is that what you mention in below is we have to iterate all process's mm_sturct to check in system call context. Of course, I don't like it and too bad design.

Quoted 15 lines
Quoted 8 lines
Quoted 4 lines

fvrange() can be used to create file backed volatile ranges.

I could't understand your point. It would be better to explain my thought firstly then, you could point out something I am missing now. Look below.

Quoted 6 lines
Quoted 4 lines

This could be overcome by iterating across all the process VMAs to determine if they're anonymous or file based, and if file-based, create a VMA sized volatile range on the mapping pointed to by the VMA.

It needs just when we start to discard pages. Simply, it is related to reclaim path, NOT system call path so it's not a problem.

Quoted 6 lines

The reason we can't defer this to only the reclaim path is if volatile ranges on shared mappings are stored in the mm_struct, if process A sets up a volatile range on a shared mapping, but stores the volatility in its own mm, then process B wants to clear the volatility on the range, process B would have to iterate over all processes that have those file vmas mapped and change them.

Right. I think iterating all of relevant vmas isn't big cost in normal situation but it could be rather bigger when the memory pressure is severe, especially for file-backed pages because it's not even read/write lock.
I'd like to minimize the system call overhead if possible.

Quoted 4 lines

Additionally if process A sets up a volatile range on a shared mapped file, then quits, the volatility state dies with that process.

Yes, so don't you want to use vrange system call for mmaped-file range at the moment?

Quoted 4 lines

Either way, its not just a simple matter of handling data on your own mm_struct. That's fine for the process' own anonymous memory, but doesn't work for shared file mappings.

Agreed.

Quoted 44 lines
Quoted 13 lines
Quoted 11 lines

But this would have downsides, as Minchan has been clear that he wants to optmize the vrange() calls so that it is very cheap to create and destroy volatile ranges. Having simple per-process ranges be created means we don't have to iterate across the vmas in the range to determine if they're anonymous or file backed. Instead the current vrange() code just creates per process ranges (which may or may not cover mmapped file data), but will only purge anonymous pages in that range. This keeps the vrange() call cheap.

Right.

Quoted 8 lines
Quoted 7 lines

Additionally, just creating or destroying a single range is very simple to do, and requires a fixed amount of memory known up front. Thus we can allocate needed data prior to making any modifications.

But If we were to create a range that crosses anonymous and file backed pages, it must create or destroy multiple per-process or per-file ranges. This could require an unknown number of allocations,

This is a part I can fail to parse your opinion.

So if we were in the vrange() code to iterate over all the VMAs in the range, creating VMA sizes ranges on either the mm_struct or the backing address_space where appropriate, its possible that we could hit an ENOMEM half way through the operation. This would leaving the range in an inconsistent state: partially marked, and potentially causing us to lose the purged state on the subranges.

Quoted 4 lines
Quoted 4 lines

opening the possibility of getting an ENOMEM half-way through the operation, leaving the volatile range partially created or destroyed.

Quoted 8 lines
Quoted 2 lines

So to keep this simple for this first pass, for now we have two syscalls for two types of volatile ranges.

My idea is following as

vrange(fd, start, len, mode, behavior)

A) fd = 0

Quoted 7 lines

Well we'd probably need to use -1 or something that would be an invalid fd here.

And really, I think having separate interfaces might be good, just as there are separate madvise() and fadvise() calls (and when all this is done, we may need to re-visit the new syscall vs new madvise/fadvise flags decision).

It does make sense in this phase where we are still RFC.

Quoted 19 lines
Quoted 9 lines

1) system call context - vrange system call registers new vrange in mm_struct.
2) Add new vrange into LRU
3) reclaim context - walk with rmap to confirm all processes make the range with volatile -> discard

B) fd = 1

The fd would just need to be valid right, not 1.

Quoted 7 lines

1) system call context - vrange system call registers new vrange in address_space
2) Add new vrange into LRU
3) reclaim context - walk with rmap to confirm all processes make the range with volatile -> discard

What's the problem in this logic?

Quoted 8 lines

The problem is only if in the first case, the volatile range being created crosses over both anonymous and shared file mmap pages. In that case we have to create appropriate sub-ranges on the mm_struct, and sub-ranges on the address_space of the mmaped file.

This isn't impossible to do, but again, the handling of errors mid-way through creating subranges is problematic (there may be yet a way around it, I just haven't thought of it yet).

Fair enough.

Quoted 9 lines

Thus with my patches, I simplified the problem a bit by partitioning it into two separate problems and two separate interfaces: Volatile ranges that are created by the vrange() call won't affect mmaped pages, only anonymous pages. We may create a range that covers them, but the volatility isn't shared with other processes and the purging logic still skips file pages. If you want to to create a volatile range on file pages, you have to use fvrange().

Okay, I got your intention by this paragraph.
You don't want to handle file pages with vrange() and want to use fvrange for file pages. I don't oppose it but please write down why we did like you explained to me on above.
It would make reviewers happier.

Quoted 5 lines

Of course, my patchset has its own inconsistencies too, since if a range is marked non-volatile that covers a mmapped file that has been marked volatile, that volatility would persist. So I probably should return an error if the vrange call covers any mmapped files.

Hmm, if you intend to separate anon and file with vrange and fvrange's separate data structure, it's no problem?

Quoted 6 lines

Also, to be clear, I'm not saying that we *have* to partition these operations into two separate behaviors, but I think having two separate behaviors at first helps makes clear the subtleties of the differences between them.

I got your point and I am thinking about that more.

Quoted 3 lines

Let me know if any of this helps your understanding. :)

Thank you very much. John!

Looking forward to seeing you in SF.

Quoted 9 lines

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#9 lore raw
Quoted 24 lines · Minchan Kim

Hello John,

As you know, userland people wanted to handle vrange with mmaped pointer rather than fd-based and see the SIGBUS so I thought more about semantic of vrange and want to make it very clear and easy. So I suggest below semantic(Of course, it's not rock solid).

mvrange(start_addr, lengh, mode, behavior)

It's same with that I suggested lately but different name, just adding prefix "m". It's per-process model(ie, mm_struct vrange) so if process is exited, "volatility" isn't valid any more. It isn't a problem in anonymous but could be in file-vrange so let's introduce fvrange for covering the problem.

fvrange(int fd, start_offset, length, mode, behavior)

First of all, let's see mvrange with anonymous and file page POV.

1) anon-mvrange

The page in volaitle range will be purged only if all of processes marked the range as volatile.

Quoted 6 lines

If A process calls mvrange and is forked, vrange could be copied from parent to child so not-yet-COWed pages could be purged unless either one of both processes marks NO_VOLATILE explicitly.

Of course, COWed page could be purged easily because there is no link any more.

Ack. This seems reasonable.

Quoted 7 lines

2) file-mvrange

A page in volatile range will be purged only if all of processes mapped the page marked it as volatile AND there is no process mapped the page as "private". IOW, all of the process mapped the page should map it with "shared" for purging.

Quoted 5 lines

So, all of processes should mark each address range in own process context if they want to collaborate with shared mapped file and gaurantee there is no process mapped the range with "private".

Of course, volatility state will be terminated as the process is gone.

This case doesn't seem ideal to me, but is sort of how the current code works to avoid the complexity of dealing with memory volatile ranges that cross page types (file/anonymous). Although the current code just doesn't purge file pages marked with mvrange().

I'd much prefer file-mvrange calls to behave identically to fvrange calls.

The important point here is that the kernel doesn't *have* to purge anything ever. Its the kernel's discretion as to which volatile pages to purge when. So its easier for now to simply not purge file pages marked volatile via mvolatile.

There however is the inconsistency that file pages marked volatile via fvrange, then are marked non-volatile via mvrange() might still be purged. That is broken in my mind, and still needs to be addressed. The easiest out is probably just to return an error if any of the mvrange calls cover file pages. But I'd really like a better fix.

Quoted 7 lines

3) fvrange

It's same with 2) but volatility state could be persistent in address_space until someone calls fvrange(NO_VOLATILE).
So it could remove the weakness of 2).

What do you think about above semantic?

I'd still like mvrange() calls on shared mapped files to be stored on the address_space.

Quoted 3 lines

If you don't have any problem, we could implement it. I think 1) and 2) could be handled with my base code for anon-vrange handling with tweaking file-vrange and need your new patches in address_space for handling 3).

I think we can get it sorted out. It might just take a few iterations.

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#10 lore raw
Quoted 46 lines · John Stultz
Quoted 24 lines · Minchan Kim

Hello John,

As you know, userland people wanted to handle vrange with mmaped pointer rather than fd-based and see the SIGBUS so I thought more about semantic of vrange and want to make it very clear and easy. So I suggest below semantic(Of course, it's not rock solid).

mvrange(start_addr, lengh, mode, behavior)

It's same with that I suggested lately but different name, just adding prefix "m". It's per-process model(ie, mm_struct vrange) so if process is exited, "volatility" isn't valid any more. It isn't a problem in anonymous but could be in file-vrange so let's introduce fvrange for covering the problem.

fvrange(int fd, start_offset, length, mode, behavior)

First of all, let's see mvrange with anonymous and file page POV.

1) anon-mvrange

The page in volaitle range will be purged only if all of processes marked the range as volatile.

Quoted 6 lines

If A process calls mvrange and is forked, vrange could be copied from parent to child so not-yet-COWed pages could be purged unless either one of both processes marks NO_VOLATILE explicitly.

Of course, COWed page could be purged easily because there is no link any more.

Ack. This seems reasonable.

Quoted 11 lines

2) file-mvrange

A page in volatile range will be purged only if all of processes mapped the page marked it as volatile AND there is no process mapped the page as "private". IOW, all of the process mapped the page should map it with "shared" for purging.

So, all of processes should mark each address range in own process context if they want to collaborate with shared mapped file and gaurantee there is no process mapped the range with "private".

Quoted 6 lines
Quoted 1 line

Of course, volatility state will be terminated as the process is gone.

This case doesn't seem ideal to me, but is sort of how the current code works to avoid the complexity of dealing with memory volatile ranges that cross page types (file/anonymous). Although the current code just doesn't purge file pages marked with mvrange().

Personally, I don't think it's to avoid the complexity of implemenation. I thought explict declaration volatility on range before using would be more clear for userspace programmer.
Otherwise, he can encounter SIGBUS and got confused easily.

Frankly speaking, I don't like to remain volatility permanently although relavant processes go away and it could make processs using the file much error-prone and hard to debug it.

Anyway, do you agree my suggestion that "we should not purge any page if a process are using now with non-shared(ie, private)"?

Quoted 6 lines

I'd much prefer file-mvrange calls to behave identically to fvrange calls.

The important point here is that the kernel doesn't *have* to purge anything ever. Its the kernel's discretion as to which volatile pages to purge when. So its easier for now to simply not purge file

Right.

Quoted 1 line

pages marked volatile via mvolatile.

NP but we should write down vague description. User try to use it in file-backed pages and got disappointed, then is reluctant to use it any more. :)

I'm not saying that let's write down description implementation specific but want to say them at least new system call can affect anonymous or file or both, at least from the beginning. Just hope.

Quoted 6 lines

There however is the inconsistency that file pages marked volatile via fvrange, then are marked non-volatile via mvrange() might still be purged. That is broken in my mind, and still needs to be addressed. The easiest out is probably just to return an error if any of the mvrange calls cover file pages. But I'd really like a

It needs vma enumeration and mmap_sem read-lock.
It could hurt anon-vrange performance severely.

Quoted 1 line

better fix.

Another idea is that we can move per-mm vrange element to address_space when the process goes away if the element covers file-backd vma. But I'm still very not sure whether we should keep it persistent.

Quoted 14 lines
Quoted 8 lines

3) fvrange

It's same with 2) but volatility state could be persistent in address_space until someone calls fvrange(NO_VOLATILE).
So it could remove the weakness of 2).
What do you think about above semantic?

I'd still like mvrange() calls on shared mapped files to be stored on the address_space.

Quoted 5 lines
Quoted 3 lines

If you don't have any problem, we could implement it. I think 1) and 2) could be handled with my base code for anon-vrange handling with tweaking file-vrange and need your new patches in address_space for handling 3).

I think we can get it sorted out. It might just take a few iterations.

Sure!

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#11 lore raw
Quoted 51 lines · Minchan Kim
Quoted 45 lines · John Stultz
Quoted 24 lines · Minchan Kim

Hello John,

As you know, userland people wanted to handle vrange with mmaped pointer rather than fd-based and see the SIGBUS so I thought more about semantic of vrange and want to make it very clear and easy. So I suggest below semantic(Of course, it's not rock solid).

mvrange(start_addr, lengh, mode, behavior)

It's same with that I suggested lately but different name, just adding prefix "m". It's per-process model(ie, mm_struct vrange) so if process is exited, "volatility" isn't valid any more. It isn't a problem in anonymous but could be in file-vrange so let's introduce fvrange for covering the problem.

fvrange(int fd, start_offset, length, mode, behavior)

First of all, let's see mvrange with anonymous and file page POV.

1) anon-mvrange

The page in volaitle range will be purged only if all of processes marked the range as volatile.

Quoted 6 lines

If A process calls mvrange and is forked, vrange could be copied from parent to child so not-yet-COWed pages could be purged unless either one of both processes marks NO_VOLATILE explicitly.

Of course, COWed page could be purged easily because there is no link any more.

Ack. This seems reasonable.

Quoted 11 lines

2) file-mvrange

A page in volatile range will be purged only if all of processes mapped the page marked it as volatile AND there is no process mapped the page as "private". IOW, all of the process mapped the page should map it with "shared" for purging.

So, all of processes should mark each address range in own process context if they want to collaborate with shared mapped file and gaurantee there is no process mapped the range with "private".

Quoted 5 lines
Quoted 1 line

Of course, volatility state will be terminated as the process is gone.

This case doesn't seem ideal to me, but is sort of how the current code works to avoid the complexity of dealing with memory volatile ranges that cross page types (file/anonymous). Although the current code just doesn't purge file pages marked with mvrange().

Quoted 8 lines

Personally, I don't think it's to avoid the complexity of implemenation. I thought explict declaration volatility on range before using would be more clear for userspace programmer.
Otherwise, he can encounter SIGBUS and got confused easily.

Frankly speaking, I don't like to remain volatility permanently although relavant processes go away and it could make processs using the file much error-prone and hard to debug it.

So this is maybe is a contentious point we'll have to work out.

Maybe could you describe some use cases you envision where someone would want to mark pages volatile on a file that could be accidentally shared? Or how you think the per-mm sense of volatility would be beneficial in those use-cases?

The use cases I envision where volatility would be used are when any sharing would be coordinated between processes.
Again, that producer/consumer example from before where the empty portion of a very large circular buffer could be made volatile, scaling the actual memory usage to the actual need.

And really the same concern would likely apply in the common case when multiple applications mmap (shared) a file, but use fvrange() to mark the data as volatile. This is exactly the use case the Android ashmem interface works for. In that case, once the data is marked volatile, it should remain volatile until someone who has the file open marks it as non-volatile. The only time we clear the volatility is when the file is closed by all users.

I think the concern about surprising an application that isn't expecting volatility is odd, since if an application jumped in and punched a hole in the data, that could surprise other applications as well. If you're going to use a file that can be shared, applications have to deal with potential changes to that file by others.

To me, the value in using volatile ranges on the file data is exactly because the file data can be shared. So it makes sense to me to have the volatility state be like the data in the file. I guess the only exception in my case is that if all the references to a file are closed, we can clear the volatility (since we don't have a sane way for the volatility to persist past that point).

One question that might help resolve this: Would having some sort of volatility checking interface be helpful in easing your concern about applications being surprised by volatility?

Quoted 2 lines

Anyway, do you agree my suggestion that "we should not purge any page if a process are using now with non-shared(ie, private)"?

Yes, or if we do purge any pages, they should not affect the private mapped pages (in other words, the COW link should be broken - as the backing page has in-effect been written to by purging).

Quoted 7 lines
Quoted 5 lines

I'd much prefer file-mvrange calls to behave identically to fvrange calls.

The important point here is that the kernel doesn't *have* to purge anything ever. Its the kernel's discretion as to which volatile pages to purge when. So its easier for now to simply not purge file

Right.

Quoted 8 lines
Quoted 1 line

pages marked volatile via mvolatile.

NP but we should write down vague description. User try to use it in file-backed pages and got disappointed, then is reluctant to use it any more. :)

I'm not saying that let's write down description implementation specific but want to say them at least new system call can affect anonymous or file or both, at least from the beginning. Just hope.

I'd like to make it generic enough that we have some flexibility to modify the puring rules if we find its more optimal. But I agree, the desired semantics of what could occur should be clear.

Quoted 7 lines
Quoted 5 lines

There however is the inconsistency that file pages marked volatile via fvrange, then are marked non-volatile via mvrange() might still be purged. That is broken in my mind, and still needs to be addressed. The easiest out is probably just to return an error if any of the mvrange calls cover file pages. But I'd really like a

It needs vma enumeration and mmap_sem read-lock.
It could hurt anon-vrange performance severely.

True. And performance needs to be good if this hinting interface is to be used easily. Although I worry about performance trumping sane semantics. So let me try to implement the desired behavior and we can measure the difference.

Quoted 4 lines
Quoted 1 line

better fix.

Another idea is that we can move per-mm vrange element to address_space when the process goes away if the element covers file-backd vma. But I'm still very not sure whether we should keep it persistent.

I really think the persistence of file-backed volatile ranges (as long as someone has the file open or a mapping to it) is important. Again, I think of the volatility really being a state of the page, but since a page-based approach is too costly, we're optimizing it into mm_struct state or address_space state.

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#12 lore raw
Quoted 61 lines · John Stultz
Quoted 51 lines · Minchan Kim
Quoted 45 lines · John Stultz
Quoted 24 lines · Minchan Kim

Hello John,

As you know, userland people wanted to handle vrange with mmaped pointer rather than fd-based and see the SIGBUS so I thought more about semantic of vrange and want to make it very clear and easy. So I suggest below semantic(Of course, it's not rock solid).

mvrange(start_addr, lengh, mode, behavior)

It's same with that I suggested lately but different name, just adding prefix "m". It's per-process model(ie, mm_struct vrange) so if process is exited, "volatility" isn't valid any more. It isn't a problem in anonymous but could be in file-vrange so let's introduce fvrange for covering the problem.

fvrange(int fd, start_offset, length, mode, behavior)

First of all, let's see mvrange with anonymous and file page POV.

1) anon-mvrange

The page in volaitle range will be purged only if all of processes marked the range as volatile.

Quoted 6 lines

If A process calls mvrange and is forked, vrange could be copied from parent to child so not-yet-COWed pages could be purged unless either one of both processes marks NO_VOLATILE explicitly.

Of course, COWed page could be purged easily because there is no link any more.

Ack. This seems reasonable.

Quoted 11 lines

2) file-mvrange

A page in volatile range will be purged only if all of processes mapped the page marked it as volatile AND there is no process mapped the page as "private". IOW, all of the process mapped the page should map it with "shared" for purging.

So, all of processes should mark each address range in own process context if they want to collaborate with shared mapped file and gaurantee there is no process mapped the range with "private".

Quoted 5 lines
Quoted 1 line

Of course, volatility state will be terminated as the process is gone.

This case doesn't seem ideal to me, but is sort of how the current code works to avoid the complexity of dealing with memory volatile ranges that cross page types (file/anonymous). Although the current code just doesn't purge file pages marked with mvrange().

Quoted 8 lines

Personally, I don't think it's to avoid the complexity of implemenation. I thought explict declaration volatility on range before using would be more clear for userspace programmer.
Otherwise, he can encounter SIGBUS and got confused easily.

Frankly speaking, I don't like to remain volatility permanently although relavant processes go away and it could make processs using the file much error-prone and hard to debug it.

Quoted 6 lines

So this is maybe is a contentious point we'll have to work out.

Maybe could you describe some use cases you envision where someone would want to mark pages volatile on a file that could be accidentally shared? Or how you think the per-mm sense of volatility would be beneficial in those use-cases?

My concern point is that following as

1. Process A calls mvrange for file F.
2. Process A is killed by someone or own BUG
3. Process B maps F with shared in his address space
4. Memory pressure happens
5. Process B is killed by SIGBUS but Process B really can't know why he was killed because he can't know anyone who open F except himself.

Quoted 11 lines

The use cases I envision where volatility would be used are when any sharing would be coordinated between processes.
Again, that producer/consumer example from before where the empty portion of a very large circular buffer could be made volatile, scaling the actual memory usage to the actual need.

And really the same concern would likely apply in the common case when multiple applications mmap (shared) a file, but use fvrange() to mark the data as volatile. This is exactly the use case the Android ashmem interface works for. In that case, once the data is

I don't know Android ashmem interface well but if it works as I mentioned early, I think it's not good interface.

Quoted 3 lines

marked volatile, it should remain volatile until someone who has the file open marks it as non-volatile. The only time we clear the volatility is when the file is closed by all users.

Yes. We need it that clear volatile ranges when the file is closed by ball users. That's what we need and blow my concern out.

Quoted 7 lines

I think the concern about surprising an application that isn't expecting volatility is odd, since if an application jumped in and punched a hole in the data, that could surprise other applications as well. If you're going to use a file that can be shared, applications have to deal with potential changes to that file by others.

True. My concern is delayed punching without any client of fd and there is no interface to detect some range of file is volatile state or not. It means anyone mapped a file with shared could encunter SIGBUS although he try to best effort to check it with lsof before using.

Quoted 7 lines

To me, the value in using volatile ranges on the file data is exactly because the file data can be shared. So it makes sense to me to have the volatility state be like the data in the file. I guess the only exception in my case is that if all the references to a file are closed, we can clear the volatility (since we don't have a sane way for the volatility to persist past that point).

Agree if you provide to clear out volatility when file are closed by all stakeholder.

Quoted 4 lines

One question that might help resolve this: Would having some sort of volatility checking interface be helpful in easing your concern about applications being surprised by volatility?

If we can provide above things, I think we don't need such interface until someone want it with reasonable logic.

Quoted 36 lines
Quoted 4 lines

Anyway, do you agree my suggestion that "we should not purge any page if a process are using now with non-shared(ie, private)"?

Yes, or if we do purge any pages, they should not affect the private mapped pages (in other words, the COW link should be broken - as the backing page has in-effect been written to by purging).

Quoted 7 lines
Quoted 5 lines

I'd much prefer file-mvrange calls to behave identically to fvrange calls.

The important point here is that the kernel doesn't *have* to purge anything ever. Its the kernel's discretion as to which volatile pages to purge when. So its easier for now to simply not purge file

Right.

Quoted 8 lines
Quoted 1 line

pages marked volatile via mvolatile.

NP but we should write down vague description. User try to use it in file-backed pages and got disappointed, then is reluctant to use it any more. :)

I'm not saying that let's write down description implementation specific but want to say them at least new system call can affect anonymous or file or both, at least from the beginning. Just hope.

I'd like to make it generic enough that we have some flexibility to modify the puring rules if we find its more optimal. But I agree, the desired semantics of what could occur should be clear.

Quoted 5 lines
Quoted 5 lines

There however is the inconsistency that file pages marked volatile via fvrange, then are marked non-volatile via mvrange() might still be purged. That is broken in my mind, and still needs to be addressed. The easiest out is probably just to return an error if any of the mvrange calls cover file pages. But I'd really like a

Quoted 7 lines
Quoted 2 lines

It needs vma enumeration and mmap_sem read-lock.
It could hurt anon-vrange performance severely.

True. And performance needs to be good if this hinting interface is to be used easily. Although I worry about performance trumping sane semantics. So let me try to implement the desired behavior and we can measure the difference.

NP. But keep in mind that mmap_sem was really terrible for performance when I took a expereiment(ie, concurrent page fault by many threads while a thread calls mmap).
I guess primary reason is CONFIG_MUTEX_SPIN_ON_OWNER.
So at least, we should avoid it by introducing new mode like VOLATILE_ANON|VOLATILE_FILE|VOLATILE_BOTH if we want to support mvrange-file and mvragne interface was thing userland people really want although ashmem have used fd-based model.

Thanks.

Quoted 21 lines
Quoted 6 lines
Quoted 3 lines

better fix.

Another idea is that we can move per-mm vrange element to address_space when the process goes away if the element covers file-backd vma. But I'm still very not sure whether we should keep it persistent.

I really think the persistence of file-backed volatile ranges (as long as someone has the file open or a mapping to it) is important. Again, I think of the volatility really being a state of the page, but since a page-based approach is too costly, we're optimizing it into mm_struct state or address_space state.

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#13 lore raw
Quoted 6 lines · Minchan Kim
Quoted 3 lines · John Stultz

marked volatile, it should remain volatile until someone who has the file open marks it as non-volatile. The only time we clear the volatility is when the file is closed by all users.

Yes. We need it that clear volatile ranges when the file is closed by ball users. That's what we need and blow my concern out.

Ok, sorry this wasn't more clear. In all the implementations I've pushed, the volatility only persists as long as someone holds the file open. Once its closed by all users, the volatility is cleared.

Hopefully that calms your worries here. :)

Quoted 10 lines
Quoted 6 lines

I think the concern about surprising an application that isn't expecting volatility is odd, since if an application jumped in and punched a hole in the data, that could surprise other applications as well. If you're going to use a file that can be shared, applications have to deal with potential changes to that file by others.

True. My concern is delayed punching without any client of fd and there is no interface to detect some range of file is volatile state or not. It means anyone mapped a file with shared could encunter SIGBUS although he try to best effort to check it with lsof before using.

I'll grant the SIGBUG semantics create the potential for stranger behavior then usual, but I think the use cases are still attractive enough to try to make it work.

Quoted 8 lines
Quoted 6 lines

To me, the value in using volatile ranges on the file data is exactly because the file data can be shared. So it makes sense to me to have the volatility state be like the data in the file. I guess the only exception in my case is that if all the references to a file are closed, we can clear the volatility (since we don't have a sane way for the volatility to persist past that point).

Agree if you provide to clear out volatility when file are closed by all stakeholder.

Agreed.

Quoted 5 lines
Quoted 3 lines

One question that might help resolve this: Would having some sort of volatility checking interface be helpful in easing your concern about applications being surprised by volatility?

If we can provide above things, I think we don't need such interface until someone want it with reasonable logic.

Sure, I just wanted to know if you saw a need right away. For now we can leave it be.

Quoted 4 lines
Quoted 4 lines

True. And performance needs to be good if this hinting interface is to be used easily. Although I worry about performance trumping sane semantics. So let me try to implement the desired behavior and we can measure the difference.

Quoted 8 lines

NP. But keep in mind that mmap_sem was really terrible for performance when I took a expereiment(ie, concurrent page fault by many threads while a thread calls mmap).
I guess primary reason is CONFIG_MUTEX_SPIN_ON_OWNER.
So at least, we should avoid it by introducing new mode like VOLATILE_ANON|VOLATILE_FILE|VOLATILE_BOTH if we want to support mvrange-file and mvragne interface was thing userland people really want although ashmem have used fd-based model.

The VOLATILE_ANON|VOLATILE_FILE|VOLATILE_BOTH may be an interesting compromise.

Though, if one marks a VOLATILE_ANON range on an address that's an mmaped file, how do we detect this and provide a sane error value without checking the vmas?

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#14 lore raw
Quoted 11 lines · John Stultz
Quoted 6 lines · Minchan Kim
Quoted 3 lines · John Stultz

marked volatile, it should remain volatile until someone who has the file open marks it as non-volatile. The only time we clear the volatility is when the file is closed by all users.

Yes. We need it that clear volatile ranges when the file is closed by ball users. That's what we need and blow my concern out.

Ok, sorry this wasn't more clear. In all the implementations I've pushed, the volatility only persists as long as someone holds the file open. Once its closed by all users, the volatility is cleared.

I now confirmed it with your implementation.
Sorry for the confusing without looking into your code in detail. :(

Quoted 2 lines

Hopefully that calms your worries here. :)

Yeb.

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

I think the concern about surprising an application that isn't expecting volatility is odd, since if an application jumped in and punched a hole in the data, that could surprise other applications as well. If you're going to use a file that can be shared, applications have to deal with potential changes to that file by others.

Quoted 8 lines
Quoted 4 lines

True. My concern is delayed punching without any client of fd and there is no interface to detect some range of file is volatile state or not. It means anyone mapped a file with shared could encunter SIGBUS although he try to best effort to check it with lsof before using.

I'll grant the SIGBUG semantics create the potential for stranger behavior then usual, but I think the use cases are still attractive enough to try to make it work.

Indeed.

Quoted 36 lines
Quoted 10 lines
Quoted 8 lines

To me, the value in using volatile ranges on the file data is exactly because the file data can be shared. So it makes sense to me to have the volatility state be like the data in the file. I guess the only exception in my case is that if all the references to a file are closed, we can clear the volatility (since we don't have a sane way for the volatility to persist past that point).

Agree if you provide to clear out volatility when file are closed by all stakeholder.

Agreed.

Quoted 5 lines
Quoted 3 lines

One question that might help resolve this: Would having some sort of volatility checking interface be helpful in easing your concern about applications being surprised by volatility?

If we can provide above things, I think we don't need such interface until someone want it with reasonable logic.

Sure, I just wanted to know if you saw a need right away. For now we can leave it be.

Quoted 4 lines
Quoted 4 lines

True. And performance needs to be good if this hinting interface is to be used easily. Although I worry about performance trumping sane semantics. So let me try to implement the desired behavior and we can measure the difference.

Quoted 8 lines

NP. But keep in mind that mmap_sem was really terrible for performance when I took a expereiment(ie, concurrent page fault by many threads while a thread calls mmap).
I guess primary reason is CONFIG_MUTEX_SPIN_ON_OWNER.
So at least, we should avoid it by introducing new mode like VOLATILE_ANON|VOLATILE_FILE|VOLATILE_BOTH if we want to support mvrange-file and mvragne interface was thing userland people really want although ashmem have used fd-based model.

Quoted 7 lines

The VOLATILE_ANON|VOLATILE_FILE|VOLATILE_BOTH may be an interesting compromise.

Though, if one marks a VOLATILE_ANON range on an address that's an mmaped file, how do we detect this and provide a sane error value without checking the vmas?

Should we check vma?
If there are conflict with existing vrange type, just return an -EINVAL?

Quoted 9 lines

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Signature · 8 lines
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>