Volatile Ranges (v12) & LSF-MM discussion fodder

This patch introduces the vrange() syscall, which allows for specifying ranges of memory as volatile, and able to be discarded by the system.

This initial patch simply adds the syscall, and the vma handling, splitting and merging the vmas as needed, and marking them with VM_VOLATILE.

No purging or discarding of volatile ranges is done at this point.

Example man page:

NAME
vrange - Mark or unmark range of memory as volatile

SYNOPSIS
ssize_t vrange(unsigned_long start, size_t length,
unsigned_long mode, unsigned_long flags, int *purged);

DESCRIPTION
Applications can use vrange(2) to advise kernel that pages of anonymous mapping in the given VM area can be reclaimed without swapping (or can no longer be reclaimed without swapping). The idea is that application can help kernel with page reclaim under memory pressure by specifying data it can easily regenerate and thus kernel can discard the data if needed.

mode:
VRANGE_VOLATILE
Informs the kernel that the VM can discard in pages in the specified range when under memory pressure.
VRANGE_NONVOLATILE
Informs the kernel that the VM can no longer discard pages in this range.

flags: Currently no flags are supported.

purged: Pointer to an integer which will return 1 if mode == VRANGE_NONVOLATILE and any page in the affected range was purged. If purged returns zero during a mode == VRANGE_NONVOLATILE call, it means all of the pages in the range are intact.

If a process accesses volatile memory which has been purged, and was not set as non volatile via a VRANGE_NONVOLATILE call, it will recieve a SIGBUS.

RETURN VALUE
On success vrange returns the number of bytes marked or unmarked. Similar to write(), it may return fewer bytes then specified if it ran into a problem.

When using VRANGE_NON_VOLATILE, if the return value is smaller then the specified length, then the value specified by the purged

arch/x86/syscalls/syscall_64.tbl +1 -0

index a12bddc..7ae3940 100644
@@ -322,6 +322,7 @@
 313	common	finit_module		sys_finit_module
 314	common	sched_setattr		sys_sched_setattr
 315	common	sched_getattr		sys_sched_getattr
+316	common	vrange			sys_vrange
 
 #
 # x32-specific system call numbers start at 512 to avoid cache impact

include/linux/mm.h +1 -0

index c1b7414..a1f11da 100644
@@ -117,6 +117,7 @@ extern unsigned int kobjsize(const void *objp);
 #define VM_IO           0x00004000	/* Memory mapped I/O or similar */
 
 					/* Used by sys_madvise() */
+#define VM_VOLATILE	0x00001000	/* VMA is volatile */
 #define VM_SEQ_READ	0x00008000	/* App will access data sequentially */
 #define VM_RAND_READ	0x00010000	/* App will not benefit from clustered reads */
 

include/linux/vrange.h new file +8 -0

new file mode 100644
index 0000000..6e5331e
@@ -0,0 +1,8 @@
+#ifndef _LINUX_VRANGE_H
+#define _LINUX_VRANGE_H
+
+#define VRANGE_NONVOLATILE 0
+#define VRANGE_VOLATILE 1
+#define VRANGE_VALID_FLAGS (0) /* Don't yet support any flags */
+
+#endif /* _LINUX_VRANGE_H */

mm/Makefile +1 -1

index 310c90a..20229e2 100644
@@ -16,7 +16,7 @@ obj-y			:= filemap.o mempool.o oom_kill.o fadvise.o \
 			   readahead.o swap.o truncate.o vmscan.o shmem.o \
 			   util.o mmzone.o vmstat.o backing-dev.o \
 			   mm_init.o mmu_context.o percpu.o slab_common.o \
-			   compaction.o balloon_compaction.o \
+			   compaction.o balloon_compaction.o vrange.o \
 			   interval_tree.o list_lru.o $(mmu-y)
 
 obj-y += init-mm.o

mm/vrange.c new file +3 -0

new file mode 100644
index 0000000..2f8e2ce
@@ -0,0 +1,173 @@
+#include <linux/syscalls.h>
+#include <linux/vrange.h>
+#include <linux/mm_inline.h>

mm/vrange.c new file +144 -0

+ *
+ * Core logic of sys_volatile. Iterates over the VMAs in the specified
+ * range, and marks or clears them as VM_VOLATILE, splitting or merging them
+ * as needed.
+ *
+ * Returns the number of bytes successfully modified.
+ *
+ * Returns error only if no bytes were modified.
+ */
+static ssize_t do_vrange(struct mm_struct *mm, unsigned long start,
+				unsigned long end, unsigned long mode,
+				unsigned long flags, int *purged)
+{
+	struct vm_area_struct *vma, *prev;
+	unsigned long orig_start = start;
+	ssize_t count = 0, ret = 0;
+
+	down_read(&mm->mmap_sem);
+
+	vma = find_vma_prev(mm, start, &prev);
+	if (vma && start > vma->vm_start)
+		prev = vma;
+
+	for (;;) {
+		unsigned long new_flags;
+		pgoff_t pgoff;
+		unsigned long tmp;
+
+		if (!vma)
+			goto out;
+
+		if (vma->vm_flags & (VM_SPECIAL|VM_LOCKED|VM_MIXEDMAP|
+					VM_HUGETLB))
+			goto out;
+
+		/* We don't support volatility on files for now */
+		if (vma->vm_file) {
+			ret = -EINVAL;
+			goto out;
+		}
+
+		/* return ENOMEM if we're trying to mark unmapped pages */
+		if (start < vma->vm_start) {
+			ret = -ENOMEM;
+			goto out;
+		}
+
+		new_flags = vma->vm_flags;
+
+		tmp = vma->vm_end;
+		if (end < tmp)
+			tmp = end;
+
+		switch (mode) {
+		case VRANGE_VOLATILE:
+			new_flags |= VM_VOLATILE;
+			break;
+		case VRANGE_NONVOLATILE:
+			new_flags &= ~VM_VOLATILE;
+		}
+
+		pgoff = vma->vm_pgoff + ((start - vma->vm_start) >> PAGE_SHIFT);
+		prev = vma_merge(mm, prev, start, tmp, new_flags,
+					vma->anon_vma, vma->vm_file, pgoff,
+					vma_policy(vma));
+		if (prev)
+			goto success;
+
+		if (start != vma->vm_start) {
+			ret = split_vma(mm, vma, start, 1);
+			if (ret)
+				goto out;
+		}
+
+		if (tmp != vma->vm_end) {
+			ret = split_vma(mm, vma, tmp, 0);
+			if (ret)
+				goto out;
+		}
+
+		prev = vma;
+success:
+		vma->vm_flags = new_flags;
+
+		/* update count to distance covered so far*/
+		count = tmp - orig_start;
+
+		start = tmp;
+		if (start < prev->vm_end)
+			start = prev->vm_end;
+		if (start >= end)
+			goto out;
+		vma = prev->vm_next;
+	}
+out:
+	up_read(&mm->mmap_sem);
+
+	/* report bytes successfully marked, even if we're exiting on error */
+	if (count)
+		return count;
+
+	return ret;
+}
+
+
+/**
+ * sys_vrange - Marks specified range as volatile or non-volatile.
+ *
+ * Validates the syscall inputs and calls do_vrange(), then copies the
+ * purged flag back out to userspace.
+ *
+ * Returns the number of bytes successfully modified.
+ * Returns error only if no bytes were modified.
+ */
+SYSCALL_DEFINE5(vrange, unsigned long, start, size_t, len, unsigned long, mode,
+			unsigned long, flags, int __user *, purged)
+{
+	unsigned long end;
+	struct mm_struct *mm = current->mm;
+	ssize_t ret = -EINVAL;
+	int p = 0;
+
+	if (flags & ~VRANGE_VALID_FLAGS)
+		goto out;
+
+	if (start & ~PAGE_MASK)
+		goto out;
+
+	len &= PAGE_MASK;
+	if (!len)
+		goto out;
+
+	end = start + len;
+	if (end < start)
+		goto out;
+
+	if (start >= TASK_SIZE)
+		goto out;
+
+	if (purged) {
+		/* Test pointer is valid before making any changes */
+		if (put_user(p, purged))
+			return -EFAULT;
+	}

Quoted 7 lines · John Stultz

RETURN VALUE
On success vrange returns the number of bytes marked or unmarked. Similar to write(), it may return fewer bytes then specified if it ran into a problem.

When using VRANGE_NON_VOLATILE, if the return value is smaller then the specified length, then the value specified by the purged

        ^^^ than

Also I'm not sure why *purged is set only if the return value is smaller than the specified legth. Won't the interface be more logical if we set *purged to appropriate value in all cases?

mm/vrange.c line 167

Quoted 9 lines

mm/vrange.c new file +9 -0

+     ret = do_vrange(mm, start, end, mode, flags, &p);
+
+     if (purged) {
+             if (put_user(p, purged)) {
+                     /*
+                      * This would be bad, since we've modified volatilty
+                      * and the change in purged state would be lost.
+                      */
+                     WARN_ONCE(1, "vrange: purge state possibly lost\n");

I think this can happen when the application has several threads and
vrange() in one thread races with munmap() in another thread. So WARN_ONCE() doesn't look appropriate (kernel shouldn't spew warnings about application programming bugs)... I'd just return -EFAULT. I know information will be lost but userspace is doing something utterly stupid.

This patch introduces the vrange() syscall, which allows for specifying ranges of memory as volatile, and able to be discarded by the system.

This initial patch simply adds the syscall, and the vma handling, splitting and merging the vmas as needed, and marking them with VM_VOLATILE.

No purging or discarding of volatile ranges is done at this point.

Example man page:

NAME
vrange - Mark or unmark range of memory as volatile

SYNOPSIS
ssize_t vrange(unsigned_long start, size_t length,
unsigned_long mode, unsigned_long flags, int *purged);

DESCRIPTION
Applications can use vrange(2) to advise kernel that pages of anonymous mapping in the given VM area can be reclaimed without swapping (or can no longer be reclaimed without swapping). The idea is that application can help kernel with page reclaim under memory pressure by specifying data it can easily regenerate and thus kernel can discard the data if needed.

mode:
VRANGE_VOLATILE
Informs the kernel that the VM can discard in pages in the specified range when under memory pressure.
VRANGE_NONVOLATILE
Informs the kernel that the VM can no longer discard pages in this range.

flags: Currently no flags are supported.

purged: Pointer to an integer which will return 1 if mode == VRANGE_NONVOLATILE and any page in the affected range was purged. If purged returns zero during a mode == VRANGE_NONVOLATILE call, it means all of the pages in the range are intact.

If a process accesses volatile memory which has been purged, and was not set as non volatile via a VRANGE_NONVOLATILE call, it will recieve a SIGBUS.

RETURN VALUE
On success vrange returns the number of bytes marked or unmarked. Similar to write(), it may return fewer bytes then specified if it ran into a problem.

When using VRANGE_NON_VOLATILE, if the return value is smaller then the specified length, then the value specified by the purged pointer will be set to 1 if any of the pages specified in the return value as successfully marked non-volatile had been purged.

If an error is returned, no changes were made.

ERRORS
EINVAL This error can occur for the following reasons:
* The value length is negative or not page size units.
* addr is not page-aligned
* mode not a valid value.
* flags is not a valid value.

ENOMEM Not enough memory

ENOMEM Addresses in the specified range are not currently mapped,
or are outside the address space of the process.

EFAULT Purged pointer is invalid

This a simplified implementation which reuses some of the logic from Minchan's earlier efforts. So credit to Minchan for his work.

Cc Andrew Morton <[email protected]>

Cc Android Kernel Team <[email protected]>

Cc Johannes Weiner <[email protected]>

Cc Robert Love <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Dave Hansen <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Dmitry Adamushko <[email protected]>

Cc Neil Brown <[email protected]>

Cc Andrea Arcangeli <[email protected]>

Cc Mike Hommey <[email protected]>

Cc Taras Glek <[email protected]>

Cc Jan Kara <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Michel Lespinasse <[email protected]>

Cc Minchan Kim <[email protected]>

Cc [email protected] <[email protected]>

Signed-off-by John Stultz <[email protected]>

---
 arch/x86/syscalls/syscall_64.tbl |   1 +
 include/linux/mm.h               |   1 +
 include/linux/vrange.h           |   8 ++
 mm/Makefile                      |   2 +-
 mm/vrange.c                      | 173 +++++++++++++++++++++++++++++++++++++++
 5 files changed, 184 insertions(+), 1 deletion(-)
 create mode 100644 include/linux/vrange.h
 create mode 100644 mm/vrange.c

arch/x86/syscalls/syscall_64.tbl +1 -0

index a12bddc..7ae3940 100644
@@ -322,6 +322,7 @@
 313    common  finit_module            sys_finit_module
 314    common  sched_setattr           sys_sched_setattr
 315    common  sched_getattr           sys_sched_getattr
+316    common  vrange                  sys_vrange
 
 #
 # x32-specific system call numbers start at 512 to avoid cache impact

include/linux/mm.h +1 -0

index c1b7414..a1f11da 100644
@@ -117,6 +117,7 @@ extern unsigned int kobjsize(const void *objp);
 #define VM_IO           0x00004000     /* Memory mapped I/O or similar */
 
                                        /* Used by sys_madvise() */
+#define VM_VOLATILE    0x00001000      /* VMA is volatile */
 #define VM_SEQ_READ    0x00008000      /* App will access data sequentially */
 #define VM_RAND_READ   0x00010000      /* App will not benefit from clustered reads */
 

include/linux/vrange.h new file +0 -0

new file mode 100644
index 0000000..6e5331e

include/linux/vrange.h new file +3 -0

+#define VRANGE_VALID_FLAGS (0) /* Don't yet support any flags */
+
+#endif /* _LINUX_VRANGE_H */

mm/Makefile +1 -1

index 310c90a..20229e2 100644
@@ -16,7 +16,7 @@ obj-y                 := filemap.o mempool.o oom_kill.o fadvise.o \
                           readahead.o swap.o truncate.o vmscan.o shmem.o \
                           util.o mmzone.o vmstat.o backing-dev.o \
                           mm_init.o mmu_context.o percpu.o slab_common.o \
-                          compaction.o balloon_compaction.o \
+                          compaction.o balloon_compaction.o vrange.o \
                           interval_tree.o list_lru.o $(mmu-y)
 
 obj-y += init-mm.o

mm/vrange.c new file +21 -0

new file mode 100644
index 0000000..2f8e2ce
@@ -0,0 +1,173 @@
+#include <linux/syscalls.h>
+#include <linux/vrange.h>
+#include <linux/mm_inline.h>
+#include <linux/pagemap.h>
+#include <linux/rmap.h>
+#include <linux/hugetlb.h>
+#include <linux/mmu_notifier.h>
+#include <linux/mm_inline.h>
+#include "internal.h"
+
+
+/**
+ * do_vrange - Marks or clears VMAs in the range (start-end) as VM_VOLATILE
+ *
+ * Core logic of sys_volatile. Iterates over the VMAs in the specified
+ * range, and marks or clears them as VM_VOLATILE, splitting or merging them
+ * as needed.
+ *
+ * Returns the number of bytes successfully modified.
+ *
+ * Returns error only if no bytes were modified.

mm/vrange.c new file +103 -0

+
+       vma = find_vma_prev(mm, start, &prev);
+       if (vma && start > vma->vm_start)
+               prev = vma;
+
+       for (;;) {
+               unsigned long new_flags;
+               pgoff_t pgoff;
+               unsigned long tmp;
+
+               if (!vma)
+                       goto out;
+
+               if (vma->vm_flags & (VM_SPECIAL|VM_LOCKED|VM_MIXEDMAP|
+                                       VM_HUGETLB))
+                       goto out;
+
+               /* We don't support volatility on files for now */
+               if (vma->vm_file) {
+                       ret = -EINVAL;
+                       goto out;
+               }
+
+               /* return ENOMEM if we're trying to mark unmapped pages */
+               if (start < vma->vm_start) {
+                       ret = -ENOMEM;
+                       goto out;
+               }
+
+               new_flags = vma->vm_flags;
+
+               tmp = vma->vm_end;
+               if (end < tmp)
+                       tmp = end;
+
+               switch (mode) {
+               case VRANGE_VOLATILE:
+                       new_flags |= VM_VOLATILE;
+                       break;
+               case VRANGE_NONVOLATILE:
+                       new_flags &= ~VM_VOLATILE;
+               }
+
+               pgoff = vma->vm_pgoff + ((start - vma->vm_start) >> PAGE_SHIFT);
+               prev = vma_merge(mm, prev, start, tmp, new_flags,
+                                       vma->anon_vma, vma->vm_file, pgoff,
+                                       vma_policy(vma));
+               if (prev)
+                       goto success;
+
+               if (start != vma->vm_start) {
+                       ret = split_vma(mm, vma, start, 1);
+                       if (ret)
+                               goto out;
+               }
+
+               if (tmp != vma->vm_end) {
+                       ret = split_vma(mm, vma, tmp, 0);
+                       if (ret)
+                               goto out;
+               }
+
+               prev = vma;
+success:
+               vma->vm_flags = new_flags;
+
+               /* update count to distance covered so far*/
+               count = tmp - orig_start;
+
+               start = tmp;
+               if (start < prev->vm_end)
+                       start = prev->vm_end;
+               if (start >= end)
+                       goto out;
+               vma = prev->vm_next;
+       }
+out:
+       up_read(&mm->mmap_sem);
+
+       /* report bytes successfully marked, even if we're exiting on error */
+       if (count)
+               return count;
+
+       return ret;
+}
+
+
+/**
+ * sys_vrange - Marks specified range as volatile or non-volatile.
+ *
+ * Validates the syscall inputs and calls do_vrange(), then copies the
+ * purged flag back out to userspace.
+ *
+ * Returns the number of bytes successfully modified.
+ * Returns error only if no bytes were modified.
+ */
+SYSCALL_DEFINE5(vrange, unsigned long, start, size_t, len, unsigned long, mode,
+                       unsigned long, flags, int __user *, purged)
+{
+       unsigned long end;
+       struct mm_struct *mm = current->mm;
+       ssize_t ret = -EINVAL;
+       int p = 0;

mm/vrange.c new file +13 -0

+
+       end = start + len;
+       if (end < start)
+               goto out;
+
+       if (start >= TASK_SIZE)
+               goto out;
+
+       if (purged) {
+               /* Test pointer is valid before making any changes */
+               if (put_user(p, purged))
+                       return -EFAULT;
+       }

Quoted 4 lines · John Stultz

RETURN VALUE
On success vrange returns the number of bytes marked or unmarked. Similar to write(), it may return fewer bytes then specified if it ran into a problem.

This explanation doesn't match your implementation. You return the last VMA - orig_start.
That said, when some hole is there at middle of the range marked (or unmarked) bytes
aren't match the return value.

Quoted 7 lines

When using VRANGE_NON_VOLATILE, if the return value is smaller then the specified length, then the value specified by the purged pointer will be set to 1 if any of the pages specified in the return value as successfully marked non-volatile had been purged.

If an error is returned, no changes were made.

At least, this explanation doesn't match the implementation. When you find file mappings, you don't rollback prior changes.

include/linux/vrange.h line 5

Quoted 12 lines

include/linux/vrange.h new file +5 -0

new file mode 100644
index 0000000..6e5331e
@@ -0,0 +1,8 @@
+#ifndef _LINUX_VRANGE_H
+#define _LINUX_VRANGE_H
+
+#define VRANGE_NONVOLATILE 0
+#define VRANGE_VOLATILE 1

Maybe, moving uapi is better?

include/linux/vrange.h line 7

Quoted 2 lines

include/linux/vrange.h new file +2 -0

+
+       down_read(&mm->mmap_sem);

This should be down_write. VMA split and merge require write lock.

include/linux/vrange.h

Quoted 4 lines

include/linux/vrange.h new file +4 -0

+
+       len &= PAGE_MASK;
+       if (!len)
+               goto out;

This code doesn't match the explanation of "not page size units."

include/linux/vrange.h

Quoted 7 lines

include/linux/vrange.h new file +7 -0

+       if (purged) {
+               if (put_user(p, purged)) {
+                       /*
+                        * This would be bad, since we've modified volatilty
+                        * and the change in purged state would be lost.
+                        */
+                       WARN_ONCE(1, "vrange: purge state possibly lost\n");

Don't do that.
If userland app unmap the page between do_vrange and here, it's just their fault, not kernel.
Therefore kernel warning make no sense. Please just move 1st put_user to here.

Users of volatile ranges will need to know if memory was discarded. This patch adds the purged state tracking required to inform userland when it marks memory as non-volatile that some memory in that range was purged and needs to be regenerated.

This simplified implementation which uses some of the logic from Minchan's earlier efforts, so credit to Minchan for his work.

Cc Andrew Morton <[email protected]>

Cc Android Kernel Team <[email protected]>

Cc Johannes Weiner <[email protected]>

Cc Robert Love <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Dave Hansen <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Dmitry Adamushko <[email protected]>

Cc Neil Brown <[email protected]>

Cc Andrea Arcangeli <[email protected]>

Cc Mike Hommey <[email protected]>

Cc Taras Glek <[email protected]>

Cc Jan Kara <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Michel Lespinasse <[email protected]>

Cc Minchan Kim <[email protected]>

Cc [email protected] <[email protected]>

Signed-off-by John Stultz <[email protected]>

---
 include/linux/swap.h    | 15 ++++++++--
 include/linux/swapops.h | 10 +++++++
 include/linux/vrange.h  |  3 ++
 mm/vrange.c             | 75 +++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 101 insertions(+), 2 deletions(-)

include/linux/swap.h +13 -2

index 46ba0c6..18c12f9 100644
@@ -70,8 +70,19 @@ static inline int current_is_kswapd(void)
 #define SWP_HWPOISON_NUM 0
 #endif
 
-#define MAX_SWAPFILES \
-	((1 << MAX_SWAPFILES_SHIFT) - SWP_MIGRATION_NUM - SWP_HWPOISON_NUM)
+
+/*
+ * Purged volatile range pages
+ */
+#define SWP_VRANGE_PURGED_NUM 1
+#define SWP_VRANGE_PURGED (MAX_SWAPFILES + SWP_HWPOISON_NUM + SWP_MIGRATION_NUM)
+
+
+#define MAX_SWAPFILES ((1 << MAX_SWAPFILES_SHIFT)	\
+				- SWP_MIGRATION_NUM	\
+				- SWP_HWPOISON_NUM	\
+				- SWP_VRANGE_PURGED_NUM	\
+			)
 
 /*
  * Magic header for a swap area. The first part of the union is

include/linux/swapops.h +10 -0

index c0f7526..84f43d9 100644
@@ -161,6 +161,16 @@ static inline int is_write_migration_entry(swp_entry_t entry)
 
 #endif
 
+static inline swp_entry_t make_vpurged_entry(void)
+{
+	return swp_entry(SWP_VRANGE_PURGED, 0);
+}
+
+static inline int is_vpurged_entry(swp_entry_t entry)
+{
+	return swp_type(entry) == SWP_VRANGE_PURGED;
+}
+
 #ifdef CONFIG_MEMORY_FAILURE
 /*
  * Support for hardware poisoned pages

include/linux/vrange.h +3 -0

index 6e5331e..986fa85 100644
@@ -1,6 +1,9 @@
 #ifndef _LINUX_VRANGE_H
 #define _LINUX_VRANGE_H
 
+#include <linux/swap.h>
+#include <linux/swapops.h>
+
 #define VRANGE_NONVOLATILE 0
 #define VRANGE_VOLATILE 1
 #define VRANGE_VALID_FLAGS (0) /* Don't yet support any flags */

mm/vrange.c +3 -0

index 2f8e2ce..1ff3cbd 100644
@@ -8,6 +8,76 @@
 #include <linux/mm_inline.h>
 #include "internal.h"
 
+struct vrange_walker {
+	struct vm_area_struct *vma;
+	int page_was_purged;

mm/vrange.c +62 -0

+ */
+static int vrange_check_purged_pte(pmd_t *pmd, unsigned long addr,
+					unsigned long end, struct mm_walk *walk)
+{
+	struct vrange_walker *vw = walk->private;
+	pte_t *pte;
+	spinlock_t *ptl;
+
+	if (pmd_trans_huge(*pmd))
+		return 0;
+	if (pmd_trans_unstable(pmd))
+		return 0;
+
+	pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl);
+	for (; addr != end; pte++, addr += PAGE_SIZE) {
+		if (!pte_present(*pte)) {
+			swp_entry_t vrange_entry = pte_to_swp_entry(*pte);
+
+			if (unlikely(is_vpurged_entry(vrange_entry))) {
+				vw->page_was_purged = 1;
+				break;
+			}
+		}
+	}
+	pte_unmap_unlock(pte - 1, ptl);
+	cond_resched();
+
+	return 0;
+}
+
+
+/**
+ * vrange_check_purged - Sets up a mm_walk to check for purged pages
+ *
+ * Sets up and calls wa_page_range() to check for purge pages.
+ *
+ * Returns 1 if pages in the range were purged, 0 otherwise.
+ */
+static int vrange_check_purged(struct mm_struct *mm,
+					 struct vm_area_struct *vma,
+					 unsigned long start,
+					 unsigned long end)
+{
+	struct vrange_walker vw;
+	struct mm_walk vrange_walk = {
+		.pmd_entry = vrange_check_purged_pte,
+		.mm = vma->vm_mm,
+		.private = &vw,
+	};
+	vw.page_was_purged = 0;
+	vw.vma = vma;
+
+	walk_page_range(start, end, &vrange_walk);
+
+	return vw.page_was_purged;
+
+}
 
 /**
  * do_vrange - Marks or clears VMAs in the range (start-end) as VM_VOLATILE

@@ -106,6 +176,11 @@ success:
 		vma = prev->vm_next;
 	}
 out:
+	if (count && (mode == VRANGE_NONVOLATILE))
+		*purged = vrange_check_purged(mm, vma,
+						orig_start,
+						orig_start+count);
+
 	up_read(&mm->mmap_sem);
 
 	/* report bytes successfully marked, even if we're exiting on error */

Signature · 1 line

1.8.3.2

Quoted 2 lines · John Stultz

+ *
+ * Sets the vrange_walker.pages_purged to 1 if any were purged.

                              ^^^ page_was_purged

Users of volatile ranges will need to know if memory was discarded. This patch adds the purged state tracking required to inform userland when it marks memory as non-volatile that some memory in that range was purged and needs to be regenerated.

This simplified implementation which uses some of the logic from Minchan's earlier efforts, so credit to Minchan for his work.

Cc Andrew Morton <[email protected]>

Cc Android Kernel Team <[email protected]>

Cc Johannes Weiner <[email protected]>

Cc Robert Love <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Dave Hansen <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Dmitry Adamushko <[email protected]>

Cc Neil Brown <[email protected]>

Cc Andrea Arcangeli <[email protected]>

Cc Mike Hommey <[email protected]>

Cc Taras Glek <[email protected]>

Cc Jan Kara <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Michel Lespinasse <[email protected]>

Cc Minchan Kim <[email protected]>

Cc [email protected] <[email protected]>

Signed-off-by John Stultz <[email protected]>

---
 include/linux/swap.h    | 15 ++++++++--
 include/linux/swapops.h | 10 +++++++
 include/linux/vrange.h  |  3 ++
 mm/vrange.c             | 75 +++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 101 insertions(+), 2 deletions(-)

include/linux/swap.h +3 -2

index 46ba0c6..18c12f9 100644
@@ -70,8 +70,19 @@ static inline int current_is_kswapd(void)
 #define SWP_HWPOISON_NUM 0
 #endif
 
-#define MAX_SWAPFILES \
-       ((1 << MAX_SWAPFILES_SHIFT) - SWP_MIGRATION_NUM - SWP_HWPOISON_NUM)
+
+/*
+ * Purged volatile range pages

Quoted 42 lines · John Stultz

Users of volatile ranges will need to know if memory was discarded. This patch adds the purged state tracking required to inform userland when it marks memory as non-volatile that some memory in that range was purged and needs to be regenerated.

This simplified implementation which uses some of the logic from Minchan's earlier efforts, so credit to Minchan for his work.

Cc Andrew Morton <[email protected]>

Cc Android Kernel Team <[email protected]>

Cc Johannes Weiner <[email protected]>

Cc Robert Love <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Dave Hansen <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Dmitry Adamushko <[email protected]>

Cc Neil Brown <[email protected]>

Cc Andrea Arcangeli <[email protected]>

Cc Mike Hommey <[email protected]>

Cc Taras Glek <[email protected]>

Cc Jan Kara <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Michel Lespinasse <[email protected]>

Cc Minchan Kim <[email protected]>

Cc [email protected] <[email protected]>

Signed-off-by John Stultz <[email protected]>

---
 include/linux/swap.h    | 15 ++++++++--
 include/linux/swapops.h | 10 +++++++
 include/linux/vrange.h  |  3 ++
 mm/vrange.c             | 75 +++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 101 insertions(+), 2 deletions(-)

include/linux/swap.h +0 -0

index 46ba0c6..18c12f9 100644
@@ -70,8 +70,19 @@ static inline int current_is_kswapd(void)
 #define SWP_HWPOISON_NUM 0
 #endif
 

Quoted 5 lines

include/linux/swap.h +3 -2

-#define MAX_SWAPFILES \
-       ((1 << MAX_SWAPFILES_SHIFT) - SWP_MIGRATION_NUM - SWP_HWPOISON_NUM)
+
+/*
+ * Purged volatile range pages

include/linux/swap.h static inline int current_is_kswapd(void) · line 85

Quoted 10 lines

include/linux/swap.h +10 -0

+ */
+#define SWP_VRANGE_PURGED_NUM 1
+#define SWP_VRANGE_PURGED (MAX_SWAPFILES + SWP_HWPOISON_NUM + SWP_MIGRATION_NUM)
+
+
+#define MAX_SWAPFILES ((1 << MAX_SWAPFILES_SHIFT)      \
+                               - SWP_MIGRATION_NUM     \
+                               - SWP_HWPOISON_NUM      \
+                               - SWP_VRANGE_PURGED_NUM \
+                       )

This change hwpoison and migration tag number. maybe ok, maybe not.

I'd suggest to use younger number than hwpoison. (That's why hwpoison uses younger number than migration)

Quoted 1 line

This change hwpoison and migration tag number. maybe ok, maybe not.

Though depending on config can't these tag numbers change anyway?

Quoted 2 lines

I'd suggest to use younger number than hwpoison. (That's why hwpoison uses younger number than migration)

So I can, but the way these are defined makes the results seem pretty terrible:

#define SWP_MIGRATION_WRITE (MAX_SWAPFILES + SWP_HWPOISON_NUM \
+ SWP_MVOLATILE_PURGED_NUM + 1)

Particularly when:
#define MAX_SWAPFILES ((1 << MAX_SWAPFILES_SHIFT) \
- SWP_MIGRATION_NUM \
- SWP_HWPOISON_NUM \
- SWP_MVOLATILE_PURGED_NUM \
)

Its a lot of unnecessary mental gymnastics. Yuck.

Would a general cleanup like the following be ok to try to make this more extensible?

thanks

(quoted diff) +0 -1

-john

include/linux/swap.h +7 -0

index 3507115..21387df 100644
@@ -49,29 +49,38 @@ static inline int current_is_kswapd(void)
  * actions on faults.
  */
 
+enum {
+       /*
+        * NOTE: We use the high bits here (subtracting from
+        * 1<<MAX_SWPFILES_SHIFT), so to preserve the values insert
+        * new entries here at the top of the enum, not at the bottom
+        */
+#ifdef CONFIG_MEMORY_FAILURE

Quoted 2 lines

Quoted 1 line

This change hwpoison and migration tag number. maybe ok, maybe not.

Though depending on config can't these tag numbers change anyway?

I don't think distro disable any of these.

Quoted 41 lines

Quoted 2 lines

I'd suggest to use younger number than hwpoison. (That's why hwpoison uses younger number than migration)

So I can, but the way these are defined makes the results seem pretty terrible:

#define SWP_MIGRATION_WRITE (MAX_SWAPFILES + SWP_HWPOISON_NUM \
+ SWP_MVOLATILE_PURGED_NUM + 1)

Particularly when:
#define MAX_SWAPFILES ((1 << MAX_SWAPFILES_SHIFT) \
- SWP_MIGRATION_NUM \
- SWP_HWPOISON_NUM \
- SWP_MVOLATILE_PURGED_NUM \
)

Its a lot of unnecessary mental gymnastics. Yuck.

Would a general cleanup like the following be ok to try to make this more extensible?

thanks

(quoted diff) +0 -1

-john

include/linux/swap.h +10 -0

index 3507115..21387df 100644
@@ -49,29 +49,38 @@ static inline int current_is_kswapd(void)
  * actions on faults.
  */
 
+enum {
+       /*
+        * NOTE: We use the high bits here (subtracting from
+        * 1<<MAX_SWPFILES_SHIFT), so to preserve the values insert
+        * new entries here at the top of the enum, not at the bottom
+        */
+#ifdef CONFIG_MEMORY_FAILURE
+       SWP_HWPOISON_NR,
+#endif
+#ifdef CONFIG_MIGRATION

include/linux/swap.h static inline int current_is_kswapd(void) · line 68

Quoted 7 lines

include/linux/swap.h +7 -0

+       SWP_MIGRATION_READ_NR,
+       SWP_MIGRATION_WRITE_NR,
+#endif
+       SWP_MAX_NR,
+};
+#define MAX_SWAPFILES ((1 << MAX_SWAPFILES_SHIFT) - SWP_MAX_NR)
+

I don't see any benefit of this code. At least, SWP_MAX_NR is suck.

The name doesn't match the actual meanings.

mm/vrange.c +18 -0

+/**
+ * vrange_check_purged_pte - Checks ptes for purged pages
+ *
+ * Iterates over the ptes in the pmd checking if they have
+ * purged swap entries.
+ *
+ * Sets the vrange_walker.pages_purged to 1 if any were purged.
+ */
+static int vrange_check_purged_pte(pmd_t *pmd, unsigned long addr,
+                                       unsigned long end, struct mm_walk *walk)
+{
+       struct vrange_walker *vw = walk->private;
+       pte_t *pte;
+       spinlock_t *ptl;
+
+       if (pmd_trans_huge(*pmd))
+               return 0;
+       if (pmd_trans_unstable(pmd))

Quoted 24 lines

mm/vrange.c +24 -0

+/**
+ * vrange_check_purged_pte - Checks ptes for purged pages
+ *
+ * Iterates over the ptes in the pmd checking if they have
+ * purged swap entries.
+ *
+ * Sets the vrange_walker.pages_purged to 1 if any were purged.
+ */
+static int vrange_check_purged_pte(pmd_t *pmd, unsigned long addr,
+                                       unsigned long end, struct mm_walk *walk)
+{
+       struct vrange_walker *vw = walk->private;
+       pte_t *pte;
+       spinlock_t *ptl;
+
+       if (pmd_trans_huge(*pmd))
+               return 0;
+       if (pmd_trans_unstable(pmd))
+               return 0;
+
+       pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl);
+       for (; addr != end; pte++, addr += PAGE_SIZE) {
+               if (!pte_present(*pte)) {
+                       swp_entry_t vrange_entry = pte_to_swp_entry(*pte);

mm/vrange.c line 44

Quoted 4 lines

mm/vrange.c +4 -0

+
+                       if (unlikely(is_vpurged_entry(vrange_entry))) {
+                               vw->page_was_purged = 1;
+                               break;

This function only detect there is vpurge entry or not. But VRANGE_NONVOLATILE should remove all vpurge entries. Otherwise, non-volatiled range still makes SIGBUS.

Quoted 24 lines · KOSAKI Motohiro

Quoted 24 lines

mm/vrange.c +24 -0

+/**
+ * vrange_check_purged_pte - Checks ptes for purged pages
+ *
+ * Iterates over the ptes in the pmd checking if they have
+ * purged swap entries.
+ *
+ * Sets the vrange_walker.pages_purged to 1 if any were purged.
+ */
+static int vrange_check_purged_pte(pmd_t *pmd, unsigned long addr,
+                                       unsigned long end, struct mm_walk *walk)
+{
+       struct vrange_walker *vw = walk->private;
+       pte_t *pte;
+       spinlock_t *ptl;
+
+       if (pmd_trans_huge(*pmd))
+               return 0;
+       if (pmd_trans_unstable(pmd))
+               return 0;
+
+       pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl);
+       for (; addr != end; pte++, addr += PAGE_SIZE) {
+               if (!pte_present(*pte)) {
+                       swp_entry_t vrange_entry = pte_to_swp_entry(*pte);

Quoted 8 lines

mm/vrange.c line 44

Quoted 4 lines

mm/vrange.c +4 -0

+
+                       if (unlikely(is_vpurged_entry(vrange_entry))) {
+                               vw->page_was_purged = 1;
+                               break;

This function only detect there is vpurge entry or not. But VRANGE_NONVOLATILE should remove all vpurge entries. Otherwise, non-volatiled range still makes SIGBUS.

So in the following patch (3/5), we only SIGBUS if the swap entry is_vpurged_entry() && the vma is still marked volatile, so this shouldn't be an issue.

Quoted 33 lines · John Stultz

Quoted 31 lines · KOSAKI Motohiro

Quoted 18 lines

mm/vrange.c +18 -0

+/**
+ * vrange_check_purged_pte - Checks ptes for purged pages
+ *
+ * Iterates over the ptes in the pmd checking if they have
+ * purged swap entries.
+ *
+ * Sets the vrange_walker.pages_purged to 1 if any were purged.
+ */
+static int vrange_check_purged_pte(pmd_t *pmd, unsigned long addr,
+                                       unsigned long end, struct mm_walk *walk)
+{
+       struct vrange_walker *vw = walk->private;
+       pte_t *pte;
+       spinlock_t *ptl;
+
+       if (pmd_trans_huge(*pmd))
+               return 0;
+       if (pmd_trans_unstable(pmd))

mm/vrange.c line 44

Quoted 10 lines

mm/vrange.c +10 -0

+               return 0;
+
+       pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl);
+       for (; addr != end; pte++, addr += PAGE_SIZE) {
+               if (!pte_present(*pte)) {
+                       swp_entry_t vrange_entry = pte_to_swp_entry(*pte);
+
+                       if (unlikely(is_vpurged_entry(vrange_entry))) {
+                               vw->page_was_purged = 1;
+                               break;

This function only detect there is vpurge entry or not. But VRANGE_NONVOLATILE should remove all vpurge entries. Otherwise, non-volatiled range still makes SIGBUS.

Quoted 3 lines

So in the following patch (3/5), we only SIGBUS if the swap entry is_vpurged_entry() && the vma is still marked volatile, so this shouldn't be an issue.

When VOLATILE -> NON-VOLATILE -> VOLATILE transition happen, the page immediately marked "was purged"?

I don't understand why vma check help.

This patch adds the hooks in the vmscan logic to discard volatile pages and mark their pte as purged. With this, volatile pages will be purged under pressure, and their ptes swap entry's marked. If the purged pages are accessed before being marked non-volatile, we catch this and send a SIGBUS.

This is a simplified implementation that uses logic from Minchan's earlier efforts, so credit to Minchan for his work.

Cc Andrew Morton <[email protected]>

Cc Android Kernel Team <[email protected]>

Cc Johannes Weiner <[email protected]>

Cc Robert Love <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Dave Hansen <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Dmitry Adamushko <[email protected]>

Cc Neil Brown <[email protected]>

Cc Andrea Arcangeli <[email protected]>

Cc Mike Hommey <[email protected]>

Cc Taras Glek <[email protected]>

Cc Jan Kara <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Michel Lespinasse <[email protected]>

Cc Minchan Kim <[email protected]>

Cc [email protected] <[email protected]>

Signed-off-by John Stultz <[email protected]>

---
 include/linux/vrange.h |   2 +
 mm/internal.h          |   2 -
 mm/memory.c            |  21 +++++++++
 mm/rmap.c              |   5 +++
 mm/vmscan.c            |  12 ++++++
 mm/vrange.c            | 114 +++++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 154 insertions(+), 2 deletions(-)

include/linux/vrange.h +2 -0

index 986fa85..d93ad21 100644
@@ -8,4 +8,6 @@
 #define VRANGE_VOLATILE 1
 #define VRANGE_VALID_FLAGS (0) /* Don't yet support any flags */
 
+extern int discard_vpage(struct page *page);
+
 #endif /* _LINUX_VRANGE_H */

mm/internal.h +0 -2

index 29e1e76..ea66bf9 100644
@@ -225,10 +225,8 @@ static inline void mlock_migrate_page(struct page *newpage, struct page *page)
 
 extern pmd_t maybe_pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma);
 
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
 extern unsigned long vma_address(struct page *page,
                                 struct vm_area_struct *vma);
-#endif
 #else /* !CONFIG_MMU */
 static inline int mlocked_vma_newpage(struct vm_area_struct *v, struct page *p)
 {

mm/memory.c +8 -0

index 22dfa61..db5f4da 100644
@@ -60,6 +60,7 @@
 #include <linux/migrate.h>
 #include <linux/string.h>
 #include <linux/dma-debug.h>
+#include <linux/vrange.h>
 
 #include <asm/io.h>
 #include <asm/pgalloc.h>

@@ -3643,6 +3644,8 @@ static int handle_pte_fault(struct mm_struct *mm,
 
        entry = *pte;
        if (!pte_present(entry)) {
+               swp_entry_t vrange_entry;
+retry:
                if (pte_none(entry)) {
                        if (vma->vm_ops) {
                                if (likely(vma->vm_ops->fault))

@@ -3652,6 +3655,24 @@ static int handle_pte_fault(struct mm_struct *mm,
                        return do_anonymous_page(mm, vma, address,
                                                 pte, pmd, flags);
                }
+
+               vrange_entry = pte_to_swp_entry(entry);
+               if (unlikely(is_vpurged_entry(vrange_entry))) {
+                       if (vma->vm_flags & VM_VOLATILE)
+                               return VM_FAULT_SIGBUS;

mm/memory.c +3 -0

+               }
+
+
                if (pte_file(entry))
                        return do_nonlinear_fault(mm, vma, address,
                                        pte, pmd, flags, entry);

mm/rmap.c +5 -0

index d9d4231..2b6f079 100644
@@ -728,6 +728,11 @@ int page_referenced_one(struct page *page, struct vm_area_struct *vma,
                                referenced++;
                }
                pte_unmap_unlock(pte, ptl);
+               if (vma->vm_flags & VM_VOLATILE) {
+                       pra->mapcount = 0;
+                       pra->vm_flags |= VM_VOLATILE;
+                       return SWAP_FAIL;
+               }
        }
 
        if (referenced) {

mm/vmscan.c +1 -0

index a9c74b4..34f159a 100644
@@ -43,6 +43,7 @@
 #include <linux/sysctl.h>
 #include <linux/oom.h>
 #include <linux/prefetch.h>
+#include <linux/vrange.h>
 
 #include <asm/tlbflush.h>
 #include <asm/div64.h>

mm/vmscan.c +7 -0

        PAGEREF_ACTIVATE,
 };
 

@@ -703,6 +705,13 @@ static enum page_references page_check_references(struct page *page,
        if (vm_flags & VM_LOCKED)
                return PAGEREF_RECLAIM;
 
+       /*
+        * If volatile page is reached on LRU's tail, we discard the
+        * page without considering recycle the page.
+        */
+       if (vm_flags & VM_VOLATILE)
+               return PAGEREF_DISCARD;
+
        if (referenced_ptes) {
                if (PageSwapBacked(page))
                        return PAGEREF_ACTIVATE;

mm/vmscan.c +1 -0

+                               goto free_it;
                case PAGEREF_KEEP:
                        goto keep_locked;
                case PAGEREF_RECLAIM:

mm/vrange.c +69 -0

index 1ff3cbd..28ceb6f 100644
@@ -246,3 +246,117 @@ SYSCALL_DEFINE5(vrange, unsigned long, start, size_t, len, unsigned long, mode,
 out:
        return ret;
 }
+
+
+/**
+ * try_to_discard_one - Purge a volatile page from a vma
+ *
+ * Finds the pte for a page in a vma, marks the pte as purged
+ * and release the page.
+ */
+static void try_to_discard_one(struct page *page, struct vm_area_struct *vma)
+{
+       struct mm_struct *mm = vma->vm_mm;
+       pte_t *pte;
+       pte_t pteval;
+       spinlock_t *ptl;
+       unsigned long addr;
+
+       VM_BUG_ON(!PageLocked(page));
+
+       addr = vma_address(page, vma);
+       pte = page_check_address(page, mm, addr, &ptl, 0);
+       if (!pte)
+               return;
+
+       BUG_ON(vma->vm_flags & (VM_SPECIAL|VM_LOCKED|VM_MIXEDMAP|VM_HUGETLB));
+
+       flush_cache_page(vma, addr, page_to_pfn(page));
+       pteval = ptep_clear_flush(vma, addr, pte);
+
+       update_hiwater_rss(mm);
+       if (PageAnon(page))
+               dec_mm_counter(mm, MM_ANONPAGES);
+       else
+               dec_mm_counter(mm, MM_FILEPAGES);
+
+       page_remove_rmap(page);
+       page_cache_release(page);
+
+       set_pte_at(mm, addr, pte,
+                               swp_entry_to_pte(make_vpurged_entry()));
+
+       pte_unmap_unlock(pte, ptl);
+       mmu_notifier_invalidate_page(mm, addr);
+
+}
+
+/**
+ * try_to_discard_vpage - check vma chain and discard from vmas marked volatile
+ *
+ * Goes over all the vmas that hold a page, and where the vmas are volatile,
+ * purge the page from the vma.
+ *
+ * Returns 0 on success, -1 on error.
+ */
+static int try_to_discard_vpage(struct page *page)
+{
+       struct anon_vma *anon_vma;
+       struct anon_vma_chain *avc;
+       pgoff_t pgoff;
+
+       anon_vma = page_lock_anon_vma_read(page);
+       if (!anon_vma)
+               return -1;
+
+       pgoff = page->index << (PAGE_CACHE_SHIFT - PAGE_SHIFT);
+       /*
+        * During interating the loop, some processes could see a page as
+        * purged while others could see a page as not-purged because we have
+        * no global lock between parent and child for protecting vrange system
+        * call during this loop. But it's not a problem because the page is

mm/vrange.c +16 -0

+               try_to_discard_one(page, vma);
+       }
+       page_unlock_anon_vma_read(anon_vma);
+       return 0;
+}
+
+
+/**
+ * discard_vpage - If possible, discard the specified volatile page
+ *
+ * Attempts to discard a volatile page, and if needed frees the swap page
+ *
+ * Returns 0 on success, -1 on error.
+ */
+int discard_vpage(struct page *page)
+{

mm/vrange.c +7 -1

+                       unlock_page(page);
+                       return 0;
+               }
+       }
+
+       return -1;
+}
--

1.8.3.2

On Fri, Mar 21, 2014 at 2:17 PM, John Stultz <[email protected]> wrote:

mm/vmscan.c +1 -0

@@ -683,6 +684,7 @@ enum page_references {
        PAGEREF_RECLAIM,
        PAGEREF_RECLAIM_CLEAN,
        PAGEREF_KEEP,
+       PAGEREF_DISCARD,

"discard" is alread used in various place for another meanings. another name is better.

Quoted 17 lines

PAGEREF_ACTIVATE,
};

mm/vmscan.c +7 -0

@@ -703,6 +705,13 @@ static enum page_references page_check_references(struct page *page,
        if (vm_flags & VM_LOCKED)
                return PAGEREF_RECLAIM;
 
+       /*
+        * If volatile page is reached on LRU's tail, we discard the
+        * page without considering recycle the page.
+        */
+       if (vm_flags & VM_VOLATILE)
+               return PAGEREF_DISCARD;
+
        if (referenced_ptes) {
                if (PageSwapBacked(page))

mm/vmscan.c static unsigned long shrink_page_list(struct list_head *page_list, · line 943

Quoted 7 lines

mm/vmscan.c +2 -0

                        return PAGEREF_ACTIVATE;

@@ -930,6 +939,9 @@ static unsigned long shrink_page_list(struct list_head *page_list,
                switch (references) {
                case PAGEREF_ACTIVATE:
                        goto activate_locked;
+               case PAGEREF_DISCARD:
+                       if (may_enter_fs && !discard_vpage(page))

Wny may-enter-fs is needed? discard_vpage never enter FS.

Quoted 5 lines

mm/vrange.c +5 -0

+       /*
+        * During interating the loop, some processes could see a page as
+        * purged while others could see a page as not-purged because we have
+        * no global lock between parent and child for protecting vrange system
+        * call during this loop. But it's not a problem because the page is

mm/vrange.c SYSCALL_DEFINE5(vrange, unsigned long, start, size_t, len, unsigned long, mode, · line 327

Quoted 10 lines

mm/vrange.c +10 -0

+        * not *SHARED* page but *COW* page so parent and child can see other
+        * data anytime. The worst case by this race is a page was purged
+        * but couldn't be discarded so it makes unnecessary page fault but
+        * it wouldn't be severe.
+        */
+       anon_vma_interval_tree_foreach(avc, &anon_vma->rb_root, pgoff, pgoff) {
+               struct vm_area_struct *vma = avc->vma;
+
+               if (!(vma->vm_flags & VM_VOLATILE))
+                       continue;

When you find !VM_VOLATILE vma, we have no reason to continue pte zapping. Isn't it?

Quoted 20 lines

mm/vrange.c +19 -0

+               try_to_discard_one(page, vma);
+       }
+       page_unlock_anon_vma_read(anon_vma);
+       return 0;
+}
+
+
+/**
+ * discard_vpage - If possible, discard the specified volatile page
+ *
+ * Attempts to discard a volatile page, and if needed frees the swap page
+ *
+ * Returns 0 on success, -1 on error.
+ */
+int discard_vpage(struct page *page)
+{
+       VM_BUG_ON(!PageLocked(page));
+       VM_BUG_ON(PageLRU(page));
+

mm/vrange.c SYSCALL_DEFINE5(vrange, unsigned long, start, size_t, len, unsigned long, mode, · line 353

Quoted 7 lines

mm/vrange.c +7 -0

+       /* XXX - for now we only support anonymous volatile pages */
+       if (!PageAnon(page))
+               return -1;
+
+       if (!try_to_discard_vpage(page)) {
+               if (PageSwapCache(page))
+                       try_to_free_swap(page);

This looks strange. try_to_free_swap can't handle vpurge pseudo entry.

Review

Quoted 2 lines

(quoted diff) +2 -0

+
+               if (page_freeze_refs(page, 1)) {

Where is page_unfreeze_refs() for the pair of this?

One issue that some potential users were concerned about, was that they wanted to ensure that all the pages from one volatile range were purged before we purge pages from a different volatile range. This would prevent the case where they have 4 large objects, and the system purges one page from each object, casuing all of the objects to have to be re-created.

The counter-point to this case, is when an application is using the SIGBUS semantics to continue to access pages after they have been marked volatile. In that case, the desire was that the most recently touched pages be purged last, and only the "cold" pages be purged from the specified range.

Instead of adding option flags for the various usage model (at least initially), one way of getting a solutoin for both uses would be to have the act of marking pages as volatile in effect mark the pages as accessed. Since all of the pages in the range would be marked together, they would be of the same "age" and would (approximately) be purged together. Further, if any pages in the range were accessed after being marked volatile, they would be moved to the end of the lru and be purged later.

Currently we don't shrink/scan the anonymous lrus when swap is off. This is problematic for volatile range purging on swapless systems/

This patch naievely changes the vmscan code to continue scanning and shrinking the lrus even when there is no swap.

It obviously has performance issues.

Thoughts on how best to implement this would be appreciated.

Cc Andrew Morton <[email protected]>

Cc Android Kernel Team <[email protected]>

Cc Johannes Weiner <[email protected]>

Cc Robert Love <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Dave Hansen <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Dmitry Adamushko <[email protected]>

Cc Neil Brown <[email protected]>

Cc Andrea Arcangeli <[email protected]>

Cc Mike Hommey <[email protected]>

Cc Taras Glek <[email protected]>

Cc Jan Kara <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Michel Lespinasse <[email protected]>

Cc Minchan Kim <[email protected]>

Cc [email protected] <[email protected]>

Signed-off-by John Stultz <[email protected]>

---
  mm/vmscan.c | 26 ++++----------------------
  1 file changed, 4 insertions(+), 22 deletions(-)

mm/vmscan.c +0 -0

index 34f159a..07b0a8c 100644

Quoted 49 lines · John Stultz

Currently we don't shrink/scan the anonymous lrus when swap is off. This is problematic for volatile range purging on swapless systems/

This patch naievely changes the vmscan code to continue scanning and shrinking the lrus even when there is no swap.

It obviously has performance issues.

Thoughts on how best to implement this would be appreciated.

Cc Andrew Morton <[email protected]>

Cc Android Kernel Team <[email protected]>

Cc Johannes Weiner <[email protected]>

Cc Robert Love <[email protected]>

Cc Mel Gorman <[email protected]>

Cc Hugh Dickins <[email protected]>

Cc Dave Hansen <[email protected]>

Cc Rik van Riel <[email protected]>

Cc Dmitry Adamushko <[email protected]>

Cc Neil Brown <[email protected]>

Cc Andrea Arcangeli <[email protected]>

Cc Mike Hommey <[email protected]>

Cc Taras Glek <[email protected]>

Cc Jan Kara <[email protected]>

Cc KOSAKI Motohiro <[email protected]>

Cc Michel Lespinasse <[email protected]>

Cc Minchan Kim <[email protected]>

Cc [email protected] <[email protected]>

Signed-off-by John Stultz <[email protected]>

---
  mm/vmscan.c | 26 ++++----------------------
  1 file changed, 4 insertions(+), 22 deletions(-)

mm/vmscan.c +2 -3

index 34f159a..07b0a8c 100644
@@ -155,9 +155,8 @@ static unsigned long zone_reclaimable_pages(struct
 zone *zone)
        nr = zone_page_state(zone, NR_ACTIVE_FILE) +
             zone_page_state(zone, NR_INACTIVE_FILE);
 
-       if (get_nr_swap_pages() > 0)
-               nr += zone_page_state(zone, NR_ACTIVE_ANON) +
-                     zone_page_state(zone, NR_INACTIVE_ANON);
+       nr += zone_page_state(zone, NR_ACTIVE_ANON) +
+             zone_page_state(zone, NR_INACTIVE_ANON);
 

mm/vmscan.c static unsigned long zone_reclaimable_pages(struct zone *zone) · line 161

Quoted 1 line

mm/vmscan.c +0 -0

        return nr;

Not all of the anonymous pages will be reclaimable.

Is there some counter that keeps track of how many volatile range pages there are in each zone?

Quoted 11 lines

mm/vmscan.c +2 -2

@@ -2181,8 +2166,8 @@ static inline bool should_continue_reclaim(struct
 zone *zone,
         */
        pages_for_compaction = (2UL << sc->order);
        inactive_lru_pages = zone_page_state(zone, NR_INACTIVE_FILE);
-       if (get_nr_swap_pages() > 0)
-               inactive_lru_pages += zone_page_state(zone,
 NR_INACTIVE_ANON);
+       inactive_lru_pages += zone_page_state(zone, NR_INACTIVE_ANON);
+
        if (sc->nr_reclaimed < pages_for_compaction &&

Quoted 1 line

inactive_lru_pages > pages_for_compaction)

Not sure this is a good idea, since the pages may not actually be reclaimable, and the inactive list will continue to be refilled indefinitely...

If there was a counter of the number of volatile range pages in a zone, this would be easier.

Of course, the overhead of keeping such a counter might be too high for what volatile ranges are designed for...

Optimistic method:
1) Userland marks a large range of data as volatile
2) Userland continues to access the data as it needs.
3) If userland accesses a page that has been purged, the kernel will send a SIGBUS
4) Userspace can trap the SIGBUS, mark the affected pages as non-volatile, and refill the data as needed before continuing on

[ I tried to bring this up during LSFMM but it got drowned out.
Trying again :) ]

Quoted 7 lines · John Stultz

Optimistic method:
1) Userland marks a large range of data as volatile
2) Userland continues to access the data as it needs.
3) If userland accesses a page that has been purged, the kernel will send a SIGBUS
4) Userspace can trap the SIGBUS, mark the affected pages as non-volatile, and refill the data as needed before continuing on

As far as I understand, if a pointer to volatile memory makes it into a syscall and the fault is trapped in kernel space, there won't be a SIGBUS, the syscall will just return -EFAULT.

Handling this would mean annotating every syscall invocation to check for -EFAULT, refill the data, and then restart the syscall. This is complicated even before taking external libraries into account, which may not propagate syscall returns properly or may not be reentrant at the necessary granularity.

Another option is to never pass volatile memory pointers into the kernel, but that too means that knowledge of volatility has to travel alongside the pointers, which will either result in more complexity throughout the application or severely limited scope of volatile memory usage.

Either way, optimistic volatile pointers are nowhere near as transparent to the application as the above description suggests, which makes this usecase not very interesting, IMO. If we can support it at little cost, why not, but I don't think we should complicate the common usecases to support this one.

Either way, optimistic volatile pointers are nowhere near as transparent to the application as the above description suggests, which makes this usecase not very interesting, IMO.

Quoted 3 lines · Johannes Weiner

Either way, optimistic volatile pointers are nowhere near as transparent to the application as the above description suggests, which makes this usecase not very interesting, IMO.

... however, I think you're still derating the value way too much. The case of user space doing elastic memory management is more and more common, and for a lot of those applications it is perfectly reasonable to either not do system calls or to have to devolatilize first.

Quoted 8 lines · H. Peter Anvin

Quoted 3 lines · Johannes Weiner

Either way, optimistic volatile pointers are nowhere near as transparent to the application as the above description suggests, which makes this usecase not very interesting, IMO.

... however, I think you're still derating the value way too much. The case of user space doing elastic memory management is more and more common, and for a lot of those applications it is perfectly reasonable to either not do system calls or to have to devolatilize first.

The SIGBUS is only in cases where the memory is set as volatile and _then_ accessed, right?

John, this was something that the Mozilla guys asked for, right? Any idea why this isn't ever a problem for them?

Quoted 11 lines · Dave Hansen

Quoted 8 lines · H. Peter Anvin

Quoted 3 lines · Johannes Weiner

Either way, optimistic volatile pointers are nowhere near as transparent to the application as the above description suggests, which makes this usecase not very interesting, IMO.

... however, I think you're still derating the value way too much. The case of user space doing elastic memory management is more and more common, and for a lot of those applications it is perfectly reasonable to either not do system calls or to have to devolatilize first.

The SIGBUS is only in cases where the memory is set as volatile and _then_ accessed, right?

Not just set volatile and then accessed, but when a volatile page has been purged and then accessed without being made non-volatile.

Quoted 2 lines

John, this was something that the Mozilla guys asked for, right? Any idea why this isn't ever a problem for them?

So one of their use cases for it is for library text. Basically they want to decompress a compressed library file into memory. Then they plan to mark the uncompressed pages volatile, and then be able to call into it. Ideally for them, the kernel would only purge cold pages, leaving the hot pages in memory. When they traverse a purged page, they handle the SIGBUS and patch the page up.

Now.. this is not what I'd consider a normal use case, but was hoping to illustrate some of the more interesting uses and demonstrate the interfaces flexibility.

Also it provided a clear example of benefits to doing LRU based cold-page purging rather then full object purging. Though I think the same could be demonstrated in a simpler case of a large cache of objects that the applications wants to mark volatile in one pass, unmarking sub-objects as it needs.

Quoted 5 lines · John Stultz

Quoted 4 lines · Dave Hansen

Quoted 1 line · H. Peter Anvin

On 04/01/2014 02:21 PM, Johannes Weiner wrote:

John, this was something that the Mozilla guys asked for, right? Any idea why this isn't ever a problem for them?

Quoted 6 lines

So one of their use cases for it is for library text. Basically they want to decompress a compressed library file into memory. Then they plan to mark the uncompressed pages volatile, and then be able to call into it. Ideally for them, the kernel would only purge cold pages, leaving the hot pages in memory. When they traverse a purged page, they handle the SIGBUS and patch the page up.

How big are these libraries compared to overall system size?

Quoted 3 lines

Now.. this is not what I'd consider a normal use case, but was hoping to illustrate some of the more interesting uses and demonstrate the interfaces flexibility.

I'm just dying to hear a "normal" use case then. :)

Quoted 6 lines · Johannes Weiner

Quoted 5 lines · John Stultz

Quoted 4 lines · Dave Hansen

Quoted 1 line · H. Peter Anvin

On 04/01/2014 02:21 PM, Johannes Weiner wrote:

John, this was something that the Mozilla guys asked for, right? Any idea why this isn't ever a problem for them?

Quoted 8 lines

Quoted 6 lines

So one of their use cases for it is for library text. Basically they want to decompress a compressed library file into memory. Then they plan to mark the uncompressed pages volatile, and then be able to call into it. Ideally for them, the kernel would only purge cold pages, leaving the hot pages in memory. When they traverse a purged page, they handle the SIGBUS and patch the page up.

How big are these libraries compared to overall system size?

Mike or Taras would have to refresh my memory on this detail. My recollection is it mostly has to do with keeping the on-disk size of the library small, so it can load off of slow media very quickly.

Quoted 5 lines

Quoted 3 lines

Now.. this is not what I'd consider a normal use case, but was hoping to illustrate some of the more interesting uses and demonstrate the interfaces flexibility.

I'm just dying to hear a "normal" use case then. :)

So the more "normal" use cause would be marking objects volatile and then non-volatile w/o accessing them in-between. In this case the zero-fill vs SIGBUS semantics don't really matter, its really just a trade off in how we handle applications deviating (intentionally or not) from this use case.

So to maybe flesh out the context here for folks who are following along (but weren't in the hallway at LSF :), Johannes made a fairly interesting proposal (Johannes: Please correct me here where I'm maybe slightly off here) to use only the dirty bits of the ptes to mark a page as volatile. Then the kernel could reclaim these clean pages as it needed, and when we marked the range as non-volatile, the pages would be re-dirtied and if any of the pages were missing, we could return a flag with the purged state. This had some different semantics then what I've been working with for awhile (for example, any writes to pages would implicitly clear volatility), so I wasn't completely comfortable with it, but figured I'd think about it to see if it could be done. Particularly since it would in some ways simplify tmpfs/shm shared volatility that I'd eventually like to do.

After thinking it over in the hallway, I talked some of the details w/ Johnnes and there was one issue that while w/ anonymous memory, we can still add a VM_VOLATILE flag on the vma, so we can get SIGBUS semantics, but since on shared volatile ranges, we don't have anything to hang a volatile flag on w/o adding some new vma like structure to the address_space structure (much as we did in the past w/ earlier volatile range implementations). This would negate much of the point of using the dirty bits to simplify the shared volatility implementation.

Thus Johannes is reasonably questioning the need for SIGBUS semantics, since if it wasn't needed, the simpler page-cleaning based volatility could potentially be used.

Now, while for the case I'm personally most interested in (ashmem), zero-fill would technically be ok, since that's what Android does. Even so, I don't think its the best approach for the interface, since applications may end up quite surprised by the results when they accidentally don't follow the "don't touch volatile pages" rule.

That point beside, I think the other problem with the page-cleaning volatility approach is that there are other awkward side effects. For
example: Say an application marks a range as volatile. One page in the range is then purged. The application, due to a bug or otherwise, reads the volatile range. This causes the page to be zero-filled in, and the application silently uses the corrupted data (which isn't great). More problematic though, is that by faulting the page in, they've in effect lost the purge state for that page. When the application then goes to mark the range as non-volatile, all pages are present, so we'd return that no pages were purged. From an application perspective this is pretty ugly.

Johannes: Any thoughts on this potential issue with your proposal? Am I missing something else?

Quoted 12 lines · John Stultz

That point beside, I think the other problem with the page-cleaning volatility approach is that there are other awkward side effects. For
example: Say an application marks a range as volatile. One page in the range is then purged. The application, due to a bug or otherwise, reads the volatile range. This causes the page to be zero-filled in, and the application silently uses the corrupted data (which isn't great). More problematic though, is that by faulting the page in, they've in effect lost the purge state for that page. When the application then goes to mark the range as non-volatile, all pages are present, so we'd return that no pages were purged. From an application perspective this is pretty ugly.

Quoted 2 lines

Johannes: Any thoughts on this potential issue with your proposal? Am I missing something else?

No, this is accurate. However, I don't really see how this is different than any other use-after-free bug. If you access malloc memory after free(), you might receive a SIGSEGV, you might see random data, you might corrupt somebody else's data. This certainly isn't nice, but it's not exactly new behavior, is it?

Quoted 13 lines · Johannes Weiner

Quoted 12 lines · John Stultz

That point beside, I think the other problem with the page-cleaning volatility approach is that there are other awkward side effects. For
example: Say an application marks a range as volatile. One page in the range is then purged. The application, due to a bug or otherwise, reads the volatile range. This causes the page to be zero-filled in, and the application silently uses the corrupted data (which isn't great). More problematic though, is that by faulting the page in, they've in effect lost the purge state for that page. When the application then goes to mark the range as non-volatile, all pages are present, so we'd return that no pages were purged. From an application perspective this is pretty ugly.

Quoted 8 lines

Quoted 2 lines

Johannes: Any thoughts on this potential issue with your proposal? Am I missing something else?

No, this is accurate. However, I don't really see how this is different than any other use-after-free bug. If you access malloc memory after free(), you might receive a SIGSEGV, you might see random data, you might corrupt somebody else's data. This certainly isn't nice, but it's not exactly new behavior, is it?

The part that troubles me is that I see the purged state as kernel data being corrupted by userland in this case. The kernel will tell userspace that no pages were purged, even though they were. Only because userspace made an errant read of a page, and got garbage data back.

Quoted 21 lines · John Stultz

Quoted 13 lines · Johannes Weiner

Quoted 12 lines · John Stultz

That point beside, I think the other problem with the page-cleaning volatility approach is that there are other awkward side effects. For
example: Say an application marks a range as volatile. One page in the range is then purged. The application, due to a bug or otherwise, reads the volatile range. This causes the page to be zero-filled in, and the application silently uses the corrupted data (which isn't great). More problematic though, is that by faulting the page in, they've in effect lost the purge state for that page. When the application then goes to mark the range as non-volatile, all pages are present, so we'd return that no pages were purged. From an application perspective this is pretty ugly.

Quoted 7 lines

Quoted 2 lines

Johannes: Any thoughts on this potential issue with your proposal? Am I missing something else?

No, this is accurate. However, I don't really see how this is different than any other use-after-free bug. If you access malloc memory after free(), you might receive a SIGSEGV, you might see random data, you might corrupt somebody else's data. This certainly isn't nice, but it's not exactly new behavior, is it?

Quoted 5 lines

The part that troubles me is that I see the purged state as kernel data being corrupted by userland in this case. The kernel will tell userspace that no pages were purged, even though they were. Only because userspace made an errant read of a page, and got garbage data back.

That sounds overly dramatic to me. First of all, this data still reflects accurately the actions of userspace in this situation. And secondly, the kernel does not rely on this data to be meaningful from a userspace perspective to function correctly.

It's really nothing but a use-after-free bug that has consequences for no-one but the faulty application. The thing that IS new is that even a read is enough to corrupt your data in this case.

MADV_REVIVE could return 0 if all pages in the specified range were present, -Esomething if otherwise. That would be semantically sound even if userspace messes up.

Quoted 27 lines · Johannes Weiner

Quoted 21 lines · John Stultz

Quoted 13 lines · Johannes Weiner

Quoted 12 lines · John Stultz

That point beside, I think the other problem with the page-cleaning volatility approach is that there are other awkward side effects. For
example: Say an application marks a range as volatile. One page in the range is then purged. The application, due to a bug or otherwise, reads the volatile range. This causes the page to be zero-filled in, and the application silently uses the corrupted data (which isn't great). More problematic though, is that by faulting the page in, they've in effect lost the purge state for that page. When the application then goes to mark the range as non-volatile, all pages are present, so we'd return that no pages were purged. From an application perspective this is pretty ugly.

Quoted 7 lines

Quoted 2 lines

Johannes: Any thoughts on this potential issue with your proposal? Am I missing something else?

No, this is accurate. However, I don't really see how this is different than any other use-after-free bug. If you access malloc memory after free(), you might receive a SIGSEGV, you might see random data, you might corrupt somebody else's data. This certainly isn't nice, but it's not exactly new behavior, is it?

Quoted 5 lines

The part that troubles me is that I see the purged state as kernel data being corrupted by userland in this case. The kernel will tell userspace that no pages were purged, even though they were. Only because userspace made an errant read of a page, and got garbage data back.

Quoted 4 lines

That sounds overly dramatic to me. First of all, this data still reflects accurately the actions of userspace in this situation. And secondly, the kernel does not rely on this data to be meaningful from a userspace perspective to function correctly.

<insert dramatic-chipmunk video w/ text overlay "errant read corrupted volatile page purge state!!!!1">

Maybe you're right, but I feel this is the sort of thing application developers would be surprised and annoyed by.

Quoted 7 lines

It's really nothing but a use-after-free bug that has consequences for no-one but the faulty application. The thing that IS new is that even a read is enough to corrupt your data in this case.

MADV_REVIVE could return 0 if all pages in the specified range were present, -Esomething if otherwise. That would be semantically sound even if userspace messes up.

So its semantically more of just a combined mincore+dirty operation.. and nothing more?

What are other folks thinking about this? Although I don't particularly like it, I probably could go along with Johannes' approach, forgoing SIGBUS for zero-fill and adapting the semantics that are in my mind a bit stranger. This would allow for ashmem-like style behavior w/ the additional write-clears-volatile-state and read-clears-purged-state constraints (which I don't think would be problematic for Android, but am not totally sure).

But I do worry that these semantics are easier for kernel-mm-developers to grasp, but are much much harder for application developers to understand.

Quoted 7 lines · Johannes Weiner

It's really nothing but a use-after-free bug that has consequences for no-one but the faulty application. The thing that IS new is that even a read is enough to corrupt your data in this case.

MADV_REVIVE could return 0 if all pages in the specified range were present, -Esomething if otherwise. That would be semantically sound even if userspace messes up.

So its semantically more of just a combined mincore+dirty operation.. and nothing more?

What are other folks thinking about this? Although I don't particularly like it, I probably could go along with Johannes' approach, forgoing SIGBUS for zero-fill and adapting the semantics that are in my mind a bit stranger. This would allow for ashmem-like style behavior w/ the additional write-clears-volatile-state and read-clears-purged-state constraints (which I don't think would be problematic for Android, but am not totally sure).

But I do worry that these semantics are easier for kernel-mm-developers to grasp, but are much much harder for application developers to understand.

Quoted 6 lines · Johannes Weiner

Quoted 5 lines · John Stultz

Quoted 4 lines · Dave Hansen

Quoted 1 line · H. Peter Anvin

On 04/01/2014 02:21 PM, Johannes Weiner wrote:

John, this was something that the Mozilla guys asked for, right? Any idea why this isn't ever a problem for them?

Quoted 8 lines

Quoted 6 lines

So one of their use cases for it is for library text. Basically they want to decompress a compressed library file into memory. Then they plan to mark the uncompressed pages volatile, and then be able to call into it. Ideally for them, the kernel would only purge cold pages, leaving the hot pages in memory. When they traverse a purged page, they handle the SIGBUS and patch the page up.

How big are these libraries compared to overall system size?

Mike or Taras would have to refresh my memory on this detail. My recollection is it mostly has to do with keeping the on-disk size of the library small, so it can load off of slow media very quickly.

Quoted 5 lines

Quoted 3 lines

Now.. this is not what I'd consider a normal use case, but was hoping to illustrate some of the more interesting uses and demonstrate the interfaces flexibility.

I'm just dying to hear a "normal" use case then. :)

So the more "normal" use cause would be marking objects volatile and then non-volatile w/o accessing them in-between. In this case the zero-fill vs SIGBUS semantics don't really matter, its really just a trade off in how we handle applications deviating (intentionally or not) from this use case.

So to maybe flesh out the context here for folks who are following along (but weren't in the hallway at LSF :), Johannes made a fairly interesting proposal (Johannes: Please correct me here where I'm maybe slightly off here) to use only the dirty bits of the ptes to mark a page as volatile. Then the kernel could reclaim these clean pages as it needed, and when we marked the range as non-volatile, the pages would be re-dirtied and if any of the pages were missing, we could

return a flag with the purged state. This had some different semantics then what I've been working with for awhile (for example, any writes to pages would implicitly clear volatility), so I wasn't completely comfortable with it, but figured I'd think about it to see if it could be done. Particularly since it would in some ways simplify tmpfs/shm shared volatility that I'd eventually like to do.

After thinking it over in the hallway, I talked some of the details w/ Johnnes and there was one issue that while w/ anonymous memory, we can still add a VM_VOLATILE flag on the vma, so we can get SIGBUS semantics, but since on shared volatile ranges, we don't have anything to hang a volatile flag on w/o adding some new vma like structure to the address_space structure (much as we did in the past w/ earlier volatile range implementations). This would negate much of the point of using the dirty bits to simplify the shared volatility implementation.

Thus Johannes is reasonably questioning the need for SIGBUS semantics, since if it wasn't needed, the simpler page-cleaning based volatility could potentially be used.

Now, while for the case I'm personally most interested in (ashmem), zero-fill would technically be ok, since that's what Android does. Even so, I don't think its the best approach for the interface, since applications may end up quite surprised by the results when they accidentally don't follow the "don't touch volatile pages" rule.

That point beside, I think the other problem with the page-cleaning volatility approach is that there are other awkward side effects. For
example: Say an application marks a range as volatile. One page in the range is then purged. The application, due to a bug or otherwise, reads the volatile range. This causes the page to be zero-filled in, and the application silently uses the corrupted data (which isn't great). More problematic though, is that by faulting the page in, they've in effect lost the purge state for that page. When the application then goes to mark the range as non-volatile, all pages are present, so we'd return that no pages were purged. From an application perspective this is pretty ugly.

Johannes: Any thoughts on this potential issue with your proposal? Am I missing something else?

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Quoted 1 line · Johannes Weiner

I'm just dying to hear a "normal" use case then. :)

So the more "normal" use cause would be marking objects volatile and then non-volatile w/o accessing them in-between. In this case the zero-fill vs SIGBUS semantics don't really matter, its really just a trade off in how we handle applications deviating (intentionally or not) from this use case.

So to maybe flesh out the context here for folks who are following along (but weren't in the hallway at LSF :), Johannes made a fairly interesting proposal (Johannes: Please correct me here where I'm maybe slightly off here) to use only the dirty bits of the ptes to mark a page as volatile. Then the kernel could reclaim these clean pages as it needed, and when we marked the range as non-volatile, the pages would be re-dirtied and if any of the pages were missing, we could return a flag with the purged state. This had some different semantics then what I've been working with for awhile (for example, any writes to pages would implicitly clear volatility), so I wasn't completely comfortable with it, but figured I'd think about it to see if it could be done. Particularly since it would in some ways simplify tmpfs/shm shared volatility that I'd eventually like to do.

Now, while for the case I'm personally most interested in (ashmem), zero-fill would technically be ok, since that's what Android does. Even so, I don't think its the best approach for the interface, since applications may end up quite surprised by the results when they accidentally don't follow the "don't touch volatile pages" rule.

That point beside, I think the other problem with the page-cleaning volatility approach is that there are other awkward side effects. For
example: Say an application marks a range as volatile. One page in the range is then purged. The application, due to a bug or otherwise, reads the volatile range. This causes the page to be zero-filled in, and the application silently uses the corrupted data (which isn't great). More problematic though, is that by faulting the page in, they've in effect lost the purge state for that page. When the application then goes to mark the range as non-volatile, all pages are present, so we'd return that no pages were purged. From an application perspective this is pretty ugly.

Quoted 21 lines · John Stultz

Quoted 1 line · Johannes Weiner

I'm just dying to hear a "normal" use case then. :)

So the more "normal" use cause would be marking objects volatile and then non-volatile w/o accessing them in-between. In this case the zero-fill vs SIGBUS semantics don't really matter, its really just a trade off in how we handle applications deviating (intentionally or not) from this use case.

So to maybe flesh out the context here for folks who are following along (but weren't in the hallway at LSF :), Johannes made a fairly interesting proposal (Johannes: Please correct me here where I'm maybe slightly off here) to use only the dirty bits of the ptes to mark a page as volatile. Then the kernel could reclaim these clean pages as it needed, and when we marked the range as non-volatile, the pages would be re-dirtied and if any of the pages were missing, we could return a flag with the purged state. This had some different semantics then what I've been working with for awhile (for example, any writes to pages would implicitly clear volatility), so I wasn't completely comfortable with it, but figured I'd think about it to see if it could be done. Particularly since it would in some ways simplify tmpfs/shm shared volatility that I'd eventually like to do.

...

Quoted 6 lines

Now, while for the case I'm personally most interested in (ashmem), zero-fill would technically be ok, since that's what Android does. Even so, I don't think its the best approach for the interface, since applications may end up quite surprised by the results when they accidentally don't follow the "don't touch volatile pages" rule.

Quoted 11 lines

That point beside, I think the other problem with the page-cleaning volatility approach is that there are other awkward side effects. For
example: Say an application marks a range as volatile. One page in the range is then purged. The application, due to a bug or otherwise, reads the volatile range. This causes the page to be zero-filled in, and the application silently uses the corrupted data (which isn't great). More problematic though, is that by faulting the page in, they've in effect lost the purge state for that page. When the application then goes to mark the range as non-volatile, all pages are present, so we'd return that no pages were purged. From an application perspective this is pretty ugly.

The write-implicitly-clears-volatile semantics would actually be an advantage for some use cases. If you have a volatile cache of many sub-page-size objects, the application can just include at the start of each page "int present, in_use;". "present" is set to non-zero before marking volatile, and when the application wants unmark as volatile it writes to "in_use" and tests the value of "present". No need for a syscall at all, although it does take a minor fault.

The syscall would be better for the case of large objects, though.

Or is that fatally flawed?

Quoted 18 lines · John Stultz

Quoted 11 lines · Dave Hansen

Quoted 8 lines · H. Peter Anvin

Quoted 3 lines · Johannes Weiner

Either way, optimistic volatile pointers are nowhere near as transparent to the application as the above description suggests, which makes this usecase not very interesting, IMO.

... however, I think you're still derating the value way too much. The case of user space doing elastic memory management is more and more common, and for a lot of those applications it is perfectly reasonable to either not do system calls or to have to devolatilize first.

The SIGBUS is only in cases where the memory is set as volatile and _then_ accessed, right?

Not just set volatile and then accessed, but when a volatile page has been purged and then accessed without being made non-volatile.

Quoted 2 lines

John, this was something that the Mozilla guys asked for, right? Any idea why this isn't ever a problem for them?

Quoted 6 lines

So one of their use cases for it is for library text. Basically they want to decompress a compressed library file into memory. Then they plan to mark the uncompressed pages volatile, and then be able to call into it. Ideally for them, the kernel would only purge cold pages, leaving the hot pages in memory. When they traverse a purged page, they handle the SIGBUS and patch the page up.

How big are these libraries compared to overall system size?

Quoted 4 lines

Now.. this is not what I'd consider a normal use case, but was hoping to illustrate some of the more interesting uses and demonstrate the interfaces flexibility.

I'm just dying to hear a "normal" use case then. :)

Quoted 5 lines

Also it provided a clear example of benefits to doing LRU based cold-page purging rather then full object purging. Though I think the same could be demonstrated in a simpler case of a large cache of objects that the applications wants to mark volatile in one pass, unmarking sub-objects as it needs.

Agreed.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

[ I tried to bring this up during LSFMM but it got drowned out.
Trying again :) ]

Quoted 7 lines · John Stultz

Optimistic method:
1) Userland marks a large range of data as volatile
2) Userland continues to access the data as it needs.
3) If userland accesses a page that has been purged, the kernel will send a SIGBUS
4) Userspace can trap the SIGBUS, mark the affected pages as non-volatile, and refill the data as needed before continuing on

As far as I understand, if a pointer to volatile memory makes it into a syscall and the fault is trapped in kernel space, there won't be a SIGBUS, the syscall will just return -EFAULT.

Handling this would mean annotating every syscall invocation to check for -EFAULT, refill the data, and then restart the syscall. This is complicated even before taking external libraries into account, which may not propagate syscall returns properly or may not be reentrant at the necessary granularity.

Another option is to never pass volatile memory pointers into the kernel, but that too means that knowledge of volatility has to travel alongside the pointers, which will either result in more complexity throughout the application or severely limited scope of volatile memory usage.

Either way, optimistic volatile pointers are nowhere near as transparent to the application as the above description suggests, which makes this usecase not very interesting, IMO. If we can support it at little cost, why not, but I don't think we should complicate the common usecases to support this one.

So, maybe its best to ignore the fact that folks want to do semi-crazy user-space faulting via SIGBUS. At least to start with. Lets look at the semantic for the "normal" mark volatile, never touch the pages until you mark non-volatile - basically where accessing volatile pages is similar to a use-after-free bug.

So, for the most part, I'd say the proposed SIGBUS semantics don't complicate things for this basic use-case, at least when compared with things like zero-fill. If an applications accidentally accessed a purged volatile page, I think SIGBUS is the right thing to do. They most likely immediately crash, but its better then them moving along with silent corruption because they're mucking with zero-filled pages.

So between zero-fill and SIGBUS, I think SIGBUS makes the most sense. If you have a third option you're thinking of, I'd of course be interested in hearing it.

Now... once you've chosen SIGBUS semantics, there will be folks who will try to exploit the fact that we get SIGBUS on purged page access (at least on the user-space side) and will try to access pages that are volatile until they are purged and try to then handle the SIGBUS to fix things up. Those folks exploiting that will have to be particularly careful not to pass volatile data to the kernel, and if they do they'll have to be smart enough to handle the EFAULT, etc. That's really all their problem, because they're being clever. :)

Quoted 27 lines · Johannes Weiner

[ I tried to bring this up during LSFMM but it got drowned out.
Trying again :) ]

Quoted 7 lines · John Stultz

Optimistic method:
1) Userland marks a large range of data as volatile
2) Userland continues to access the data as it needs.
3) If userland accesses a page that has been purged, the kernel will send a SIGBUS
4) Userspace can trap the SIGBUS, mark the affected pages as non-volatile, and refill the data as needed before continuing on

As far as I understand, if a pointer to volatile memory makes it into a syscall and the fault is trapped in kernel space, there won't be a SIGBUS, the syscall will just return -EFAULT.

Handling this would mean annotating every syscall invocation to check for -EFAULT, refill the data, and then restart the syscall. This is complicated even before taking external libraries into account, which may not propagate syscall returns properly or may not be reentrant at the necessary granularity.

Another option is to never pass volatile memory pointers into the kernel, but that too means that knowledge of volatility has to travel alongside the pointers, which will either result in more complexity throughout the application or severely limited scope of volatile memory usage.

Quoted 5 lines

Either way, optimistic volatile pointers are nowhere near as transparent to the application as the above description suggests, which makes this usecase not very interesting, IMO. If we can support it at little cost, why not, but I don't think we should complicate the common usecases to support this one.

So yea, thanks again for all the feedback at LSF-MM! I'm trying to get things integrated for a v13 here shortly (although with visitors in town this week it may not happen until next week).

So, maybe its best to ignore the fact that folks want to do semi-crazy user-space faulting via SIGBUS. At least to start with. Lets look at the semantic for the "normal" mark volatile, never touch the pages until you mark non-volatile - basically where accessing volatile pages is similar to a use-after-free bug.

So, for the most part, I'd say the proposed SIGBUS semantics don't complicate things for this basic use-case, at least when compared with things like zero-fill. If an applications accidentally accessed a purged volatile page, I think SIGBUS is the right thing to do. They most likely immediately crash, but its better then them moving along with silent corruption because they're mucking with zero-filled pages.

So between zero-fill and SIGBUS, I think SIGBUS makes the most sense. If you have a third option you're thinking of, I'd of course be interested in hearing it.

So between zero-fill and SIGBUS, I'd prefer the one which results in the simpler user interface / fewer system calls.

Quoted 4 lines · Johannes Weiner

So between zero-fill and SIGBUS, I'd prefer the one which results in the simpler user interface / fewer system calls.

The use cases are different; I believe this should be a user space option.

Quoted 6 lines · H. Peter Anvin

Quoted 4 lines · Johannes Weiner

So between zero-fill and SIGBUS, I'd prefer the one which results in the simpler user interface / fewer system calls.

Quoted 2 lines

The use cases are different; I believe this should be a user space option.

Case in point, for example: imagine a JIT. You *really* don't want to zero-fill memory behind the back of your JIT, as all zero memory may not be a trapping instruction (it isn't on x86, for example, and if you are unlucky you may be modifying *part* of an instruction.)

Hence my follow-up question in the other mail about how large we expect such code caches to become in practice in relationship to overall system memory. Are code caches interesting reclaim candidates to begin with? Are they big enough to make the machine thrash/swap otherwise?

Quoted 5 lines · Johannes Weiner

Hence my follow-up question in the other mail about how large we expect such code caches to become in practice in relationship to overall system memory. Are code caches interesting reclaim candidates to begin with? Are they big enough to make the machine thrash/swap otherwise?

A big chunk of the use cases here are for swapless systems anyway, so this is the *only* way for them to reclaim anonymous memory. Their choices are either to be constantly throwing away and rebuilding these objects, or to leave them in memory effectively pinned.

In practice I did see ashmem (the Android thing that we're trying to replace) get used a lot by the Android web browser when I was playing with it. John said that it got used for storing decompressed copies of images.

Quoted 12 lines · Dave Hansen

Quoted 5 lines · Johannes Weiner

Hence my follow-up question in the other mail about how large we expect such code caches to become in practice in relationship to overall system memory. Are code caches interesting reclaim candidates to begin with? Are they big enough to make the machine thrash/swap otherwise?

A big chunk of the use cases here are for swapless systems anyway, so this is the *only* way for them to reclaim anonymous memory. Their choices are either to be constantly throwing away and rebuilding these objects, or to leave them in memory effectively pinned.

Quoted 4 lines

In practice I did see ashmem (the Android thing that we're trying to replace) get used a lot by the Android web browser when I was playing with it. John said that it got used for storing decompressed copies of images.

Although images are a simpler case where its easier to not touch volatile pages. I think Johannes is mostly concerned about cases where volatile pages are being accessed while they are volatile, which the Mozilla folks are so far the only viable case (in my mind... folks may have others) where they intentionally want to access pages while they're volatile and thus require SIGBUS semantics.

I suspect handling the SIGBUS and patching up the purged page you trapped on is likely much to complicated for most use cases. But I do think SIGBUS is preferable to zero-fill on purged page access, just because its likely to be easier to debug applications.

Quoted 4 lines · John Stultz

I suspect handling the SIGBUS and patching up the purged page you trapped on is likely much to complicated for most use cases. But I do think SIGBUS is preferable to zero-fill on purged page access, just because its likely to be easier to debug applications.

Fully agreed, but it seems a bit overkill to add a separate syscall, a range-tree on top of shmem address_spaces, and an essentially new programming model based on SIGBUS userspace fault handling (incl. all the complexities and confusion this inevitably will bring when people DO end up passing these pointers into kernel space) just to be a bit nicer about use-after-free bugs in applications.