a question about mlockall() and mprotect()

When we call mlockall(), we will add VM_LOCKED to the vma, if the vma prot is ---p,

then mm_populate -> get_user_pages will not alloc memory.

I find it said "ignore errors" in mm_populate()
static inline void mm_populate(unsigned long addr, unsigned long len)

{
	/* Ignore errors */
	(void) __mm_populate(addr, len, 1);
}

And later we call mprotect() to change the prot, then it is still not alloc memory for the mlocked vma.

My question is that, shall we alloc memory if the prot changed, and who(kernel, glibc, user) should alloc the memory?

Quoted 2 lines · Xishi Qiu

When we call mlockall(), we will add VM_LOCKED to the vma, if the vma prot is ---p,

not sure what you mean here. apply_mlockall_flags will set the flag on all vmas except for special mappings (mlock_fixup). This phase will cause that memory reclaim will not free already mapped pages in those vmas (see page_check_references and the lazy mlock pages move to unevictable LRUs).

Quoted 1 line

then mm_populate -> get_user_pages will not alloc memory.

mm_populate all the vmas with pages. Well there are certainly some constrains - e.g. memory cgroup hard limit might be hit and so the faulting might fail.

Quoted 6 lines

I find it said "ignore errors" in mm_populate()
static inline void mm_populate(unsigned long addr, unsigned long len)

{
	/* Ignore errors */
	(void) __mm_populate(addr, len, 1);
}

But we do not report the failure because any failure past apply_mlockall_flags would be tricky to handle. We have already dropped the mmap_sem lock so some other address space operations could have interfered.

Quoted 5 lines

And later we call mprotect() to change the prot, then it is still not alloc memory for the mlocked vma.

My question is that, shall we alloc memory if the prot changed, and who(kernel, glibc, user) should alloc the memory?

I do not understand your question but if you are asking how to get pages to map your vmas then touching that area will fault the memory in.

Quoted 28 lines · Michal Hocko

Quoted 2 lines · Xishi Qiu

When we call mlockall(), we will add VM_LOCKED to the vma, if the vma prot is ---p,

not sure what you mean here. apply_mlockall_flags will set the flag on all vmas except for special mappings (mlock_fixup). This phase will cause that memory reclaim will not free already mapped pages in those vmas (see page_check_references and the lazy mlock pages move to unevictable LRUs).

Quoted 1 line

then mm_populate -> get_user_pages will not alloc memory.

mm_populate all the vmas with pages. Well there are certainly some constrains - e.g. memory cgroup hard limit might be hit and so the faulting might fail.

Quoted 6 lines

I find it said "ignore errors" in mm_populate()
static inline void mm_populate(unsigned long addr, unsigned long len)

{
	/* Ignore errors */
	(void) __mm_populate(addr, len, 1);
}

But we do not report the failure because any failure past apply_mlockall_flags would be tricky to handle. We have already dropped the mmap_sem lock so some other address space operations could have interfered.

Quoted 8 lines

Quoted 5 lines

And later we call mprotect() to change the prot, then it is still not alloc memory for the mlocked vma.

My question is that, shall we alloc memory if the prot changed, and who(kernel, glibc, user) should alloc the memory?

I do not understand your question but if you are asking how to get pages to map your vmas then touching that area will fault the memory in.

Hi Michal,

syscall mlockall() will first apply the VM_LOCKED to the vma, then call mm_populate() to map the vmas.

mm_populate
populate_vma_page_range
__get_user_pages
check_vma_flags
And the above path maybe return -EFAULT in some case, right?

If we call mprotect() to change the prot of vma, just let check_vma_flags() return 0, then we will get the mlocked pages in following page-fault, right?

My question is that, shall we map the vmas immediately when
the prot changed? If we should map it immediately, who(kernel, glibc, user) do this step?

Quoted 50 lines · Xishi Qiu

Quoted 28 lines · Michal Hocko

Quoted 2 lines · Xishi Qiu

When we call mlockall(), we will add VM_LOCKED to the vma, if the vma prot is ---p,

not sure what you mean here. apply_mlockall_flags will set the flag on all vmas except for special mappings (mlock_fixup). This phase will cause that memory reclaim will not free already mapped pages in those vmas (see page_check_references and the lazy mlock pages move to unevictable LRUs).

Quoted 1 line

then mm_populate -> get_user_pages will not alloc memory.

mm_populate all the vmas with pages. Well there are certainly some constrains - e.g. memory cgroup hard limit might be hit and so the faulting might fail.

Quoted 6 lines

I find it said "ignore errors" in mm_populate()
static inline void mm_populate(unsigned long addr, unsigned long len)

{
	/* Ignore errors */
	(void) __mm_populate(addr, len, 1);
}

But we do not report the failure because any failure past apply_mlockall_flags would be tricky to handle. We have already dropped the mmap_sem lock so some other address space operations could have interfered.

Quoted 8 lines

Quoted 5 lines

And later we call mprotect() to change the prot, then it is still not alloc memory for the mlocked vma.

My question is that, shall we alloc memory if the prot changed, and who(kernel, glibc, user) should alloc the memory?

I do not understand your question but if you are asking how to get pages to map your vmas then touching that area will fault the memory in.

Hi Michal,

syscall mlockall() will first apply the VM_LOCKED to the vma, then call mm_populate() to map the vmas.

mm_populate
populate_vma_page_range
__get_user_pages
check_vma_flags
And the above path maybe return -EFAULT in some case, right?

Quoted 3 lines

If we call mprotect() to change the prot of vma, just let check_vma_flags() return 0, then we will get the mlocked pages in following page-fault, right?

Any future page fault to the existing vma will result in the mlocked page. That is what VM_LOCKED guarantess.

Quoted 3 lines

My question is that, shall we map the vmas immediately when
the prot changed? If we should map it immediately, who(kernel, glibc, user) do this step?

This is still very fuzzy. What are you actually trying to achieve?

On 2017/9/26 17:02, Michal Hocko wrote:

Quoted 1 line

This is still very fuzzy. What are you actually trying to achieve?

I don't expect page fault any more after mlock.

Quoted 58 lines · Michal Hocko

Quoted 50 lines · Xishi Qiu

Quoted 28 lines · Michal Hocko

Quoted 2 lines · Xishi Qiu

When we call mlockall(), we will add VM_LOCKED to the vma, if the vma prot is ---p,

not sure what you mean here. apply_mlockall_flags will set the flag on all vmas except for special mappings (mlock_fixup). This phase will cause that memory reclaim will not free already mapped pages in those vmas (see page_check_references and the lazy mlock pages move to unevictable LRUs).

Quoted 1 line

then mm_populate -> get_user_pages will not alloc memory.

mm_populate all the vmas with pages. Well there are certainly some constrains - e.g. memory cgroup hard limit might be hit and so the faulting might fail.

Quoted 6 lines

I find it said "ignore errors" in mm_populate()
static inline void mm_populate(unsigned long addr, unsigned long len)

{
	/* Ignore errors */
	(void) __mm_populate(addr, len, 1);
}

But we do not report the failure because any failure past apply_mlockall_flags would be tricky to handle. We have already dropped the mmap_sem lock so some other address space operations could have interfered.

Quoted 8 lines

Quoted 5 lines

And later we call mprotect() to change the prot, then it is still not alloc memory for the mlocked vma.

My question is that, shall we alloc memory if the prot changed, and who(kernel, glibc, user) should alloc the memory?

I do not understand your question but if you are asking how to get pages to map your vmas then touching that area will fault the memory in.

Hi Michal,

syscall mlockall() will first apply the VM_LOCKED to the vma, then call mm_populate() to map the vmas.

mm_populate
populate_vma_page_range
__get_user_pages
check_vma_flags
And the above path maybe return -EFAULT in some case, right?

Quoted 3 lines

If we call mprotect() to change the prot of vma, just let check_vma_flags() return 0, then we will get the mlocked pages in following page-fault, right?

Any future page fault to the existing vma will result in the mlocked page. That is what VM_LOCKED guarantess.

Quoted 5 lines

Quoted 3 lines

My question is that, shall we map the vmas immediately when
the prot changed? If we should map it immediately, who(kernel, glibc, user) do this step?

This is still very fuzzy. What are you actually trying to achieve?

I don't expect page fault any more after mlock.

.

Quoted 4 lines · Xishi Qiu

Quoted 1 line

This is still very fuzzy. What are you actually trying to achieve?

I don't expect page fault any more after mlock.

Our apps is some thing like RT, and page-fault maybe cause a lot of time, e.g. lock, mem reclaim ..., so I use mlock and don't want page fault any more.

Quoted 6 lines · Xishi Qiu

Quoted 4 lines · Xishi Qiu

Quoted 1 line

This is still very fuzzy. What are you actually trying to achieve?

I don't expect page fault any more after mlock.

Quoted 3 lines

Our apps is some thing like RT, and page-fault maybe cause a lot of time, e.g. lock, mem reclaim ..., so I use mlock and don't want page fault any more.

Why does your app then have restricted mprotect when calling mlockall() and only later adjusts the mprotect?

Quoted 7 lines · Vlastimil Babka

Quoted 6 lines · Xishi Qiu

Quoted 4 lines · Xishi Qiu

Quoted 1 line

This is still very fuzzy. What are you actually trying to achieve?

I don't expect page fault any more after mlock.

Quoted 6 lines

Quoted 3 lines

Our apps is some thing like RT, and page-fault maybe cause a lot of time, e.g. lock, mem reclaim ..., so I use mlock and don't want page fault any more.

Why does your app then have restricted mprotect when calling mlockall() and only later adjusts the mprotect?

Ahh, OK I see what is goging on. So you have PROT_NONE vma at the time mlockall and then later mprotect it something else and want to fault all that memory at the mprotect time?

So basically to do
---

mm/mprotect.c +0 -0

index 6d3e2f082290..b665b5d1c544 100644

Quoted 27 lines · Michal Hocko

Quoted 7 lines · Vlastimil Babka

Quoted 6 lines · Xishi Qiu

Quoted 4 lines · Xishi Qiu

Quoted 1 line

This is still very fuzzy. What are you actually trying to achieve?

I don't expect page fault any more after mlock.

Quoted 6 lines

Quoted 3 lines

Our apps is some thing like RT, and page-fault maybe cause a lot of time, e.g. lock, mem reclaim ..., so I use mlock and don't want page fault any more.

Why does your app then have restricted mprotect when calling mlockall() and only later adjusts the mprotect?

Ahh, OK I see what is goging on. So you have PROT_NONE vma at the time mlockall and then later mprotect it something else and want to fault all that memory at the mprotect time?

So basically to do
---

mm/mprotect.c +0 -0

index 6d3e2f082290..b665b5d1c544 100644

mm/mprotect.c mprotect_fixup(struct vm_area_struct *vma, struct vm_area_struct **pprev, · line 375

Quoted 8 lines

mm/mprotect.c +1 -1

@@ -369,7 +369,7 @@ mprotect_fixup(struct vm_area_struct *vma, struct vm_area_struct **pprev,
 	 * Private VM_LOCKED VMA becoming writable: trigger COW to avoid major
 	 * fault on access.
 	 */
-	if ((oldflags & (VM_WRITE | VM_SHARED | VM_LOCKED)) == VM_LOCKED &&
+	if ((oldflags & (VM_WRITE | VM_LOCKED)) == VM_LOCKED &&
 			(newflags & VM_WRITE)) {
 		populate_vma_page_range(vma, start, end, NULL);
 	}

Hi Michal,

My kernel is v3.10, and I missed this code, thank you reminding me.

Quoted 40 lines · Xishi Qiu

Quoted 27 lines · Michal Hocko

Quoted 7 lines · Vlastimil Babka

Quoted 6 lines · Xishi Qiu

Quoted 4 lines · Xishi Qiu

Quoted 1 line

This is still very fuzzy. What are you actually trying to achieve?

I don't expect page fault any more after mlock.

Quoted 6 lines

Quoted 3 lines

Our apps is some thing like RT, and page-fault maybe cause a lot of time, e.g. lock, mem reclaim ..., so I use mlock and don't want page fault any more.

Why does your app then have restricted mprotect when calling mlockall() and only later adjusts the mprotect?

Ahh, OK I see what is goging on. So you have PROT_NONE vma at the time mlockall and then later mprotect it something else and want to fault all that memory at the mprotect time?

So basically to do
---

mm/mprotect.c +0 -0

index 6d3e2f082290..b665b5d1c544 100644

mm/mprotect.c mprotect_fixup(struct vm_area_struct *vma, struct vm_area_struct **pprev, · line 375

Quoted 8 lines

mm/mprotect.c +1 -1

@@ -369,7 +369,7 @@ mprotect_fixup(struct vm_area_struct *vma, struct vm_area_struct **pprev,
 	 * Private VM_LOCKED VMA becoming writable: trigger COW to avoid major
 	 * fault on access.
 	 */
-	if ((oldflags & (VM_WRITE | VM_SHARED | VM_LOCKED)) == VM_LOCKED &&
+	if ((oldflags & (VM_WRITE | VM_LOCKED)) == VM_LOCKED &&
 			(newflags & VM_WRITE)) {
 		populate_vma_page_range(vma, start, end, NULL);
 	}

Hi Michal,

Quoted 1 line

My kernel is v3.10, and I missed this code, thank you reminding me.

I guess I didn't get your answer. Does the above diff resolves your problem?

Quoted 43 lines · Michal Hocko

Quoted 41 lines · Xishi Qiu

Quoted 27 lines · Michal Hocko

Quoted 7 lines · Vlastimil Babka

Quoted 6 lines · Xishi Qiu

Quoted 4 lines · Xishi Qiu

Quoted 1 line

This is still very fuzzy. What are you actually trying to achieve?

I don't expect page fault any more after mlock.

Quoted 6 lines

Quoted 3 lines

Our apps is some thing like RT, and page-fault maybe cause a lot of time, e.g. lock, mem reclaim ..., so I use mlock and don't want page fault any more.

Why does your app then have restricted mprotect when calling mlockall() and only later adjusts the mprotect?

Ahh, OK I see what is goging on. So you have PROT_NONE vma at the time mlockall and then later mprotect it something else and want to fault all that memory at the mprotect time?

So basically to do
---

mm/mprotect.c +0 -0

index 6d3e2f082290..b665b5d1c544 100644

mm/mprotect.c mprotect_fixup(struct vm_area_struct *vma, struct vm_area_struct **pprev, · line 375

Quoted 8 lines

mm/mprotect.c +1 -1

@@ -369,7 +369,7 @@ mprotect_fixup(struct vm_area_struct *vma, struct vm_area_struct **pprev,
 	 * Private VM_LOCKED VMA becoming writable: trigger COW to avoid major
 	 * fault on access.
 	 */
-	if ((oldflags & (VM_WRITE | VM_SHARED | VM_LOCKED)) == VM_LOCKED &&
+	if ((oldflags & (VM_WRITE | VM_LOCKED)) == VM_LOCKED &&
 			(newflags & VM_WRITE)) {
 		populate_vma_page_range(vma, start, end, NULL);
 	}

Hi Michal,

My kernel is v3.10, and I missed this code, thank you reminding me.

Quoted 2 lines

I guess I didn't get your answer. Does the above diff resolves your problem?

Hi Michal,

This upstream patch 36f881883c57941bb32d25cea6524f9612ab5a2c has already resolve my problem, thank you for your attention.