[PATCH v2][RFC] add MAP_UNLOCKED mmap flag

Gleb Natapov

author

2009-10-06 17:02 UTC #0 lore raw

RFC PATCH v2 1/1 add MAP_UNLOCKED mmap flag

If application does mlockall(MCL_FUTURE) it is no longer possible to mmap file bigger than main memory or allocate big area of anonymous memory. Sometimes it is desirable to lock everything related to program execution into memory, but still be able to mmap big file or allocate huge amount of memory and allow OS to swap them on demand. MAP_UNLOCKED allows to do that.

Signed-off-by Gleb Natapov <[email protected]>

---

v1->v2
- adding new flag to all archs
- fixing typo

arch/alpha/include/asm/mman.h +1 -0

index 99c56d4..cfc51ac 100644
@@ -30,6 +30,7 @@
 #define MAP_NONBLOCK	0x40000		/* do not block on IO */
 #define MAP_STACK	0x80000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x100000	/* create a huge page mapping */
+#define MAP_UNLOCKED	0x200000	/* force page unlocking */
 
 #define MS_ASYNC	1		/* sync memory asynchronously */
 #define MS_SYNC		2		/* synchronous memory sync */

arch/mips/include/asm/mman.h +1 -0

index a2250f3..c161e27 100644
@@ -48,6 +48,7 @@
 #define MAP_NONBLOCK	0x20000		/* do not block on IO */
 #define MAP_STACK	0x40000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x80000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x100000	/* force page unlocking */
 
 /*
  * Flags for msync

arch/parisc/include/asm/mman.h +1 -0

index 9749c8a..4e8b9bf 100644
@@ -24,6 +24,7 @@
 #define MAP_NONBLOCK	0x20000		/* do not block on IO */
 #define MAP_STACK	0x40000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x80000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x100000	/* force page unlocking */
 
 #define MS_SYNC		1		/* synchronous memory sync */
 #define MS_ASYNC	2		/* sync memory asynchronously */

arch/powerpc/include/asm/mman.h +1 -0

index d4a7f64..7d33f01 100644
@@ -27,6 +27,7 @@
 #define MAP_NONBLOCK	0x10000		/* do not block on IO */
 #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x80000		/* force page unlocking */
 
 #ifdef __KERNEL__
 #ifdef CONFIG_PPC64

arch/sparc/include/asm/mman.h +1 -0

index c3029ad..f80d203 100644
@@ -22,6 +22,7 @@
 #define MAP_NONBLOCK	0x10000		/* do not block on IO */
 #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x80000		/* force page unlocking */
 
 #ifdef __KERNEL__
 #ifndef __ASSEMBLY__

arch/xtensa/include/asm/mman.h +1 -0

index fca4db4..c62bcd8 100644
@@ -55,6 +55,7 @@
 #define MAP_NONBLOCK	0x20000		/* do not block on IO */
 #define MAP_STACK	0x40000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x80000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x100000	/* force page unlocking */
 
 /*
  * Flags for msync

include/asm-generic/mman.h +1 -0

index 32c8bd6..59e0f29 100644
@@ -12,6 +12,7 @@
 #define MAP_NONBLOCK	0x10000		/* do not block on IO */
 #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x80000         /* force page unlocking */
 
 #define MCL_CURRENT	1		/* lock all current mappings */
 #define MCL_FUTURE	2		/* lock all future mappings */

mm/mmap.c +3 -0

index 73f5e4b..7c2abdb 100644
@@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 		if (!can_do_mlock())
 			return -EPERM;
 
+        if (flags & MAP_UNLOKED)
+                vm_flags &= ~VM_LOCKED;
+
 	/* mlock MCL_FUTURE? */
 	if (vm_flags & VM_LOCKED) {
 		unsigned long locked, lock_limit;

--
Gleb.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Christoph Lameter

Reviewed-by on v2 1/1

2009-10-06 18:06 UTC #1 lore raw

Looks good.

Reviewed-by Christoph Lameter <[email protected]>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Johannes Weiner

2009-10-06 18:34 UTC #2 lore raw

In reply to Gleb Natapov

Quoted 17 lines · Gleb Natapov

include/asm-generic/mman.h +1 -0

index 32c8bd6..59e0f29 100644
@@ -12,6 +12,7 @@
 #define MAP_NONBLOCK	0x10000		/* do not block on IO */
 #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x80000         /* force page unlocking */
 
 #define MCL_CURRENT	1		/* lock all current mappings */
 #define MCL_FUTURE	2		/* lock all future mappings */

mm/mmap.c +0 -0

index 73f5e4b..7c2abdb 100644

mm/mmap.c unsigned long do_mmap_pgoff(struct file *file, unsigned long addr, · line 989

Quoted 5 lines

mm/mmap.c +2 -0

@@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 		if (!can_do_mlock())
 			return -EPERM;
 
+        if (flags & MAP_UNLOKED)
+                vm_flags &= ~VM_LOCKED;

That needs changing into MAP_UNLOCKED as well.

Should we do something special about (MAP_UNLOCKED | MAP_LOCKED)?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Gleb Natapov

author

2009-10-06 18:53 UTC #3 lore raw

Quoted 21 lines · Johannes Weiner

Quoted 20 lines · Gleb Natapov

include/asm-generic/mman.h +1 -0

index 32c8bd6..59e0f29 100644
@@ -12,6 +12,7 @@
 #define MAP_NONBLOCK	0x10000		/* do not block on IO */
 #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x80000         /* force page unlocking */
 
 #define MCL_CURRENT	1		/* lock all current mappings */
 #define MCL_FUTURE	2		/* lock all future mappings */

mm/mmap.c +0 -0

index 73f5e4b..7c2abdb 100644
@@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 		if (!can_do_mlock())
 			return -EPERM;

Quoted 5 lines

mm/mmap.c unsigned long do_mmap_pgoff(struct file *file, unsigned long addr, · line 989
Quoted 2 lines
mm/mmap.c +2 -0
+        if (flags & MAP_UNLOKED)
+                vm_flags &= ~VM_LOCKED;
That needs changing into MAP_UNLOCKED as well.

That's for editing patch manually :(

Quoted 1 line

Should we do something special about (MAP_UNLOCKED | MAP_LOCKED)?

It is simpler to just ignore it. What do you think?

--
Gleb.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Johannes Weiner

2009-10-08 09:37 UTC #4 lore raw

Quoted 22 lines · Gleb Natapov

Quoted 21 lines · Johannes Weiner

Quoted 20 lines · Gleb Natapov

include/asm-generic/mman.h +1 -0

index 32c8bd6..59e0f29 100644
@@ -12,6 +12,7 @@
 #define MAP_NONBLOCK	0x10000		/* do not block on IO */
 #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x80000         /* force page unlocking */
 
 #define MCL_CURRENT	1		/* lock all current mappings */
 #define MCL_FUTURE	2		/* lock all future mappings */

mm/mmap.c +0 -0

index 73f5e4b..7c2abdb 100644
@@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 		if (!can_do_mlock())
 			return -EPERM;

Quoted 6 lines

Quoted 4 lines
mm/mmap.c unsigned long do_mmap_pgoff(struct file *file, unsigned long addr, · line 989
Quoted 2 lines
mm/mmap.c +2 -0
+        if (flags & MAP_UNLOKED)
+                vm_flags &= ~VM_LOCKED;
Should we do something special about (MAP_UNLOCKED | MAP_LOCKED)?
It is simpler to just ignore it. What do you think?

I think we should filter it out. It's unlocking even when MAP_LOCKED is set and doing mlock-specific checks also no mlock will happen.

Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Gleb Natapov

author

2009-10-08 09:40 UTC #5 lore raw

Quoted 26 lines · Johannes Weiner

Quoted 25 lines · Gleb Natapov

Quoted 23 lines · Johannes Weiner

Quoted 22 lines · Gleb Natapov

include/asm-generic/mman.h +1 -0

index 32c8bd6..59e0f29 100644
@@ -12,6 +12,7 @@
 #define MAP_NONBLOCK	0x10000		/* do not block on IO */
 #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x80000         /* force page unlocking */
 
 #define MCL_CURRENT	1		/* lock all current mappings */
 #define MCL_FUTURE	2		/* lock all future mappings */

mm/mmap.c +2 -0

index 73f5e4b..7c2abdb 100644
@@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 		if (!can_do_mlock())
 			return -EPERM;
 
+        if (flags & MAP_UNLOKED)
+                vm_flags &= ~VM_LOCKED;

Quoted 7 lines

Quoted 3 lines
Quoted 1 line
Should we do something special about (MAP_UNLOCKED | MAP_LOCKED)?
It is simpler to just ignore it. What do you think?
I think we should filter it out. It's unlocking even when MAP_LOCKED is set and doing mlock-specific checks also no mlock will happen.

What do you mean by "filter it out"? Return error. mlock-specific checks don't make much sense indeed since locking will not happen.

--
Gleb.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Johannes Weiner

2009-10-08 11:03 UTC #6 lore raw

Quoted 27 lines · Gleb Natapov

Quoted 26 lines · Johannes Weiner

Quoted 25 lines · Gleb Natapov

Quoted 23 lines · Johannes Weiner

Quoted 22 lines · Gleb Natapov

include/asm-generic/mman.h +1 -0

index 32c8bd6..59e0f29 100644
@@ -12,6 +12,7 @@
 #define MAP_NONBLOCK	0x10000		/* do not block on IO */
 #define MAP_STACK	0x20000		/* give out an address that is best suited for process/thread stacks */
 #define MAP_HUGETLB	0x40000		/* create a huge page mapping */
+#define MAP_UNLOCKED	0x80000         /* force page unlocking */
 
 #define MCL_CURRENT	1		/* lock all current mappings */
 #define MCL_FUTURE	2		/* lock all future mappings */

mm/mmap.c +2 -0

index 73f5e4b..7c2abdb 100644
@@ -985,6 +985,9 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 		if (!can_do_mlock())
 			return -EPERM;
 
+        if (flags & MAP_UNLOKED)
+                vm_flags &= ~VM_LOCKED;

Quoted 8 lines

Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Should we do something special about (MAP_UNLOCKED | MAP_LOCKED)?
It is simpler to just ignore it. What do you think?
I think we should filter it out. It's unlocking even when MAP_LOCKED is set and doing mlock-specific checks also no mlock will happen.
What do you mean by "filter it out"? Return error.

Yes, sorry for being vague. I mean returning -EINVAL for both flags being set.

Hannes

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>