mlock: operate on any regions with protection != PROT_NONE

PATCH In review Michel Lespinasse 4 participants 7 messages Feb 1, 2011 – Feb 3, 2011

Subsystems (2)

Memory management Memory mapping

lore

Share & tools

lore

Michel Lespinasse

author

2011-02-01 01:03 UTC #0 lore raw

PATCH 1/1 mlock: operate on any regions with protection != PROT_NONE

As Tao Ma noticed, change 5ecfda0 breaks blktrace. This is because blktrace mmaps a file with PROT_WRITE permissions but without PROT_READ, so my attempt to not unnecessarity break COW during mlock ended up causing mlock to fail with a permission problem.

I am proposing to let mlock ignore vma protection in all cases except PROT_NONE. In particular, mlock should not fail for PROT_WRITE regions (as in the blktrace case, which broke at 5ecfda0) or for PROT_EXEC regions (which seem to me like they were always broken).

Please review. I am proposing this as a candidate for 2.6.38 inclusion, because of the behavior change with blktrace.

mm/mlock.c +7 -0

index 13e81ee..c3924c7f 100644
@@ -178,6 +178,13 @@ static long __mlock_vma_pages_range(struct vm_area_struct *vma,
 	if ((vma->vm_flags & (VM_WRITE | VM_SHARED)) == VM_WRITE)
 		gup_flags |= FOLL_WRITE;
 
+	/*
+	 * We want mlock to succeed for regions that have any permissions
+	 * other than PROT_NONE.
+	 */
+	if (vma->vm_flags & (VM_READ | VM_WRITE | VM_EXEC))
+		gup_flags |= FOLL_FORCE;
+
 	if (vma->vm_flags & VM_LOCKED)
 		gup_flags |= FOLL_MLOCK;

Signature · 9 lines

Michel "Walken" Lespinasse
A program is never fully debugged until the last user dies.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Michel Lespinasse

author

2011-02-01 01:09 UTC #1 lore raw

(forgot the signoff...)

Quoted 5 lines · Michel Lespinasse

As Tao Ma noticed, change 5ecfda0 breaks blktrace. This is because blktrace mmaps a file with PROT_WRITE permissions but without PROT_READ, so my attempt to not unnecessarity break COW during mlock ended up causing mlock to fail with a permission problem.

Quoted 7 lines

I am proposing to let mlock ignore vma protection in all cases except PROT_NONE. In particular, mlock should not fail for PROT_WRITE regions (as in the blktrace case, which broke at 5ecfda0) or for PROT_EXEC regions (which seem to me like they were always broken).
Please review. I am proposing this as a candidate for 2.6.38 inclusion, because of the behavior change with blktrace.

Signed-off-by Michel Lespinasse <[email protected]>

Quoted 17 lines

mm/mlock.c +7 -0

index 13e81ee..c3924c7f 100644
@@ -178,6 +178,13 @@ static long __mlock_vma_pages_range(struct vm_area_struct *vma,
        if ((vma->vm_flags & (VM_WRITE | VM_SHARED)) == VM_WRITE)
                gup_flags |= FOLL_WRITE;
 
+       /*
+        * We want mlock to succeed for regions that have any permissions
+        * other than PROT_NONE.
+        */
+       if (vma->vm_flags & (VM_READ | VM_WRITE | VM_EXEC))
+               gup_flags |= FOLL_FORCE;
+
        if (vma->vm_flags & VM_LOCKED)
                gup_flags |= FOLL_MLOCK;

Signature · 9 lines

Michel "Walken" Lespinasse
A program is never fully debugged until the last user dies.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Linus Torvalds

2011-02-01 06:00 UTC #2 lore raw

In reply to Michel Lespinasse

Quoted 3 lines · Michel Lespinasse

I am proposing to let mlock ignore vma protection in all cases except PROT_NONE.

What's so special about PROT_NONE? If you want to mlock something without actually being able to then fault that in, why not?

IOW, why wouldn't it be right to just make FOLL_FORCE be unconditional in mlock?

Linus

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Michel Lespinasse

author

2011-02-01 06:36 UTC #3 lore raw

Quoted 9 lines · Linus Torvalds

Quoted 3 lines · Michel Lespinasse
I am proposing to let mlock ignore vma protection in all cases except PROT_NONE.
What's so special about PROT_NONE? If you want to mlock something without actually being able to then fault that in, why not?
IOW, why wouldn't it be right to just make FOLL_FORCE be unconditional in mlock?

I agree this would be the most logical thing to do, but I'm afraid people would complain about it as it'd be yet another behavior change.

I don't have the entire context here, but PROT_NONE regions are actually common in modern userspace. It seems that for most shared libraries, ld.so creates 4 vmas, one of them being just under 2MB and with PROT_NONE protection. From my shell, if I do cat /proc/$$/maps, I see:
[...]
7f41ebdb5000-7f41ebdc5000 r-xp 00000000 fc:03 669498
/usr/lib/zsh/4.3.10/zsh/computil.so
7f41ebdc5000-7f41ebfc4000 ---p 00010000 fc:03 669498
/usr/lib/zsh/4.3.10/zsh/computil.so
7f41ebfc4000-7f41ebfc5000 r--p 0000f000 fc:03 669498
/usr/lib/zsh/4.3.10/zsh/computil.so
7f41ebfc5000-7f41ebfc6000 rw-p 00010000 fc:03 669498
/usr/lib/zsh/4.3.10/zsh/computil.so
7f41ebfc6000-7f41ebfce000 r-xp 00000000 fc:03 669508
/usr/lib/zsh/4.3.10/zsh/parameter.so
7f41ebfce000-7f41ec1ce000 ---p 00008000 fc:03 669508
/usr/lib/zsh/4.3.10/zsh/parameter.so
7f41ec1ce000-7f41ec1cf000 r--p 00008000 fc:03 669508
/usr/lib/zsh/4.3.10/zsh/parameter.so
7f41ec1cf000-7f41ec1d0000 rw-p 00009000 fc:03 669508
/usr/lib/zsh/4.3.10/zsh/parameter.so
[...]

I don't know why userspace does that, but these regions currently never get any page into RSS, even when processes call mlockall(). I am told that we need to preserve this property.

Signature · 9 lines

Michel "Walken" Lespinasse
A program is never fully debugged until the last user dies.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Rik van Riel

2011-02-01 17:57 UTC #4 lore raw

In reply to Linus Torvalds

Quoted 9 lines · Linus Torvalds

Quoted 3 lines · Michel Lespinasse
I am proposing to let mlock ignore vma protection in all cases except PROT_NONE.
What's so special about PROT_NONE? If you want to mlock something without actually being able to then fault that in, why not?
IOW, why wouldn't it be right to just make FOLL_FORCE be unconditional in mlock?

I could think of a combination of reasons.

Specifically, some libc/linker magic will set up PROT_NONE areas for programs automatically.

Some programs use mlockall to lock themselves into memory, with no idea that PROT_NONE areas were set up behind its back.

Faulting in the PROT_NONE memory will result is wasted memory, without the application even realizing it.

Signature · 8 lines

All rights reversed

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Rik van Riel

Acked-by on v1 1/1

2011-02-01 17:58 UTC #5 lore raw

In reply to Michel Lespinasse

Quoted 5 lines · Michel Lespinasse

As Tao Ma noticed, change 5ecfda0 breaks blktrace. This is because blktrace mmaps a file with PROT_WRITE permissions but without PROT_READ, so my attempt to not unnecessarity break COW during mlock ended up causing mlock to fail with a permission problem.

Quoted 7 lines

I am proposing to let mlock ignore vma protection in all cases except PROT_NONE. In particular, mlock should not fail for PROT_WRITE regions (as in the blktrace case, which broke at 5ecfda0) or for PROT_EXEC regions (which seem to me like they were always broken).
Please review. I am proposing this as a candidate for 2.6.38 inclusion, because of the behavior change with blktrace.

Acked-by Rik van Riel <[email protected]>

Signature · 8 lines

All rights reversed

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

KOSAKI Motohiro

Reviewed-by

2011-02-03 00:57 UTC #6 lore raw

Quoted 11 lines · Michel Lespinasse

Quoted 10 lines
As Tao Ma noticed, change 5ecfda0 breaks blktrace. This is because blktrace mmaps a file with PROT_WRITE permissions but without PROT_READ, so my attempt to not unnecessarity break COW during mlock ended up causing mlock to fail with a permission problem.
I am proposing to let mlock ignore vma protection in all cases except PROT_NONE. In particular, mlock should not fail for PROT_WRITE regions (as in the blktrace case, which broke at 5ecfda0) or for PROT_EXEC regions (which seem to me like they were always broken).

Quoted 4 lines

Quoted 2 lines
Please review. I am proposing this as a candidate for 2.6.38 inclusion, because of the behavior change with blktrace.
Acked-by Rik van Riel <[email protected]>

Reviewed-by KOSAKI Motohiro <[email protected]>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>