mm: do not increment pgfault stats when page fault handler retries
Subsystems (2)
Suren Baghdasaryan
reviewerPATCH 1/1 mm: do not increment pgfault stats when page fault handler retries
If the page fault handler requests a retry, we will count the fault multiple times. This is a relatively harmless problem as the retry paths are not often requested, and the only user-visible problem is that the fault counter will be slightly higher than it should be. Nevertheless, userspace only took one fault, and should not see the fact that the kernel had to retry the fault multiple times.
---
Patch applies cleanly over linux-next and mm-unstable
mm/memory.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
mm/memory.c +10 -6
@@ -5212,17 +5212,16 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,__set_current_state (TASK_RUNNING ); -count_vm_event (PGFAULT ); -count_memcg_event_mm (vma->vm_mm,PGFAULT ); - ret =sanitize_fault_flags (vma, &flags);if (ret) -return ret; + goto out;if (!arch_vma_access_permitted (vma, flags &FAULT_FLAG_WRITE , flags &FAULT_FLAG_INSTRUCTION , - flags &FAULT_FLAG_REMOTE )) -return VM_FAULT_SIGSEGV ; + flags &FAULT_FLAG_REMOTE )) { + ret =VM_FAULT_SIGSEGV ; + goto out; + }/* * Enable the memcg OOM handling for faults triggered in user
@@ -5253,6 +5252,11 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, }mm_account_fault (regs, address, flags, ret); +out: +if (!(ret &VM_FAULT_RETRY )) { +count_vm_event (PGFAULT ); +count_memcg_event_mm (vma->vm_mm,PGFAULT ); + }return ret; }
Signature · 1 line
2.40.0.634.g4ca3ef3211-goog
Quoted 8 lines · Suren Baghdasaryan
If the page fault handler requests a retry, we will count the fault multiple times. This is a relatively harmless problem as the retry paths are not often requested, and the only user-visible problem is that the fault counter will be slightly higher than it should be. Nevertheless, userspace only took one fault, and should not see the fact that the kernel had to retry the fault multiple times.
Fixes 6b4c9f446981 ("filemap: drop the mmap_sem for all blocking operations")
I know I suggested this fixes line, but I think it's actually been here much longer, perhaps since
Michel, what do you think?
Quoted 50 lines
Signed-off-by Suren Baghdasaryan <[email protected]>Reviewed-by Matthew Wilcox (Oracle) <[email protected]>---
Patch applies cleanly over linux-next and mm-unstablemm/memory.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)mm/memory.c +10
-6@@ -5212,17 +5212,16 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,__set_current_state (TASK_RUNNING ); -count_vm_event (PGFAULT ); -count_memcg_event_mm (vma->vm_mm,PGFAULT ); - ret =sanitize_fault_flags (vma, &flags);if (ret) -return ret; + goto out;if (!arch_vma_access_permitted (vma, flags &FAULT_FLAG_WRITE , flags &FAULT_FLAG_INSTRUCTION , - flags &FAULT_FLAG_REMOTE )) -return VM_FAULT_SIGSEGV ; + flags &FAULT_FLAG_REMOTE )) { + ret =VM_FAULT_SIGSEGV ; + goto out; + }/* * Enable the memcg OOM handling for faults triggered in user @@ -5253,6 +5252,11 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, }mm_account_fault (regs, address, flags, ret); +out: +if (!(ret &VM_FAULT_RETRY )) { +count_vm_event (PGFAULT ); +count_memcg_event_mm (vma->vm_mm,PGFAULT ); + }return ret; }Signature · 1 line
2.40.0.634.g4ca3ef3211-goog
Hi, Suren,
Quoted 44 lines · Suren Baghdasaryan
If the page fault handler requests a retry, we will count the fault multiple times. This is a relatively harmless problem as the retry paths are not often requested, and the only user-visible problem is that the fault counter will be slightly higher than it should be. Nevertheless, userspace only took one fault, and should not see the fact that the kernel had to retry the fault multiple times.
Fixes 6b4c9f446981 ("filemap: drop the mmap_sem for all blocking operations")Signed-off-by Suren Baghdasaryan <[email protected]>Reviewed-by Matthew Wilcox (Oracle) <[email protected]>---
Patch applies cleanly over linux-next and mm-unstablemm/memory.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)mm/memory.c +5
-6@@ -5212,17 +5212,16 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,__set_current_state (TASK_RUNNING ); -count_vm_event (PGFAULT ); -count_memcg_event_mm (vma->vm_mm,PGFAULT ); - ret =sanitize_fault_flags (vma, &flags);if (ret) -return ret; + goto out;if (!arch_vma_access_permitted (vma, flags &FAULT_FLAG_WRITE , flags &FAULT_FLAG_INSTRUCTION , - flags &FAULT_FLAG_REMOTE )) -return VM_FAULT_SIGSEGV ; + flags &FAULT_FLAG_REMOTE )) { + ret =VM_FAULT_SIGSEGV ; + goto out; + }/* * Enable the memcg OOM handling for faults triggered in user
mm/memory.c vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, · line 5254
Quoted 3 lines
mm/memory.c +0
-0@@ -5253,6 +5252,11 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, }mm_account_fault (regs, address, flags, ret);
Here is the mm_account_fault() function taking care of some other accountings. Perhaps good to put things into it?
It also already ignores invalid faults:
if (ret & (VM_FAULT_ERROR | VM_FAULT_RETRY)) return;
I see that you may also want to account for sigbus, however I really don't know why. Explanations would be great when it would matter. So far it makes sense to me if we skip both RETRY or ERROR cases.
mm/memory.c vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, · line 5258
Quoted 4 lines
mm/memory.c +4
-0+out: +if (!(ret &VM_FAULT_RETRY )) { +count_vm_event (PGFAULT ); +count_memcg_event_mm (vma->vm_mm,PGFAULT );
There is one thing worth noticing is here vma may or may not be valid depending on the retval of the fault.
RETRY is exactly one of the cases that accessing vma may be unsafe due to releasing of mmap read lock. The other one is the recently added VM_FAULT_COMPLETE. So if we want to move this chunk (or any vma reference) to be later we need to consider a valid vma / mm being there first, or we're prone to accessing a vma that has already been released, I think.
Quoted 8 lines
mm/memory.c +1
-0+ }return ret; }Signature · 1 line
2.40.0.634.g4ca3ef3211-goog
Thanks,
Signature · 1 line
Peter Xu
Suren Baghdasaryan
reviewerQuoted 2 lines · Peter Xu
Hi, Suren,
Hi Peter,
Quoted 48 lines · Suren Baghdasaryan
Quoted 46 lines
If the page fault handler requests a retry, we will count the fault multiple times. This is a relatively harmless problem as the retry paths are not often requested, and the only user-visible problem is that the fault counter will be slightly higher than it should be. Nevertheless, userspace only took one fault, and should not see the fact that the kernel had to retry the fault multiple times.
Fixes 6b4c9f446981 ("filemap: drop the mmap_sem for all blocking operations")Signed-off-by Suren Baghdasaryan <[email protected]>Reviewed-by Matthew Wilcox (Oracle) <[email protected]>---
Patch applies cleanly over linux-next and mm-unstablemm/memory.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)mm/memory.c +5
-6@@ -5212,17 +5212,16 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,__set_current_state (TASK_RUNNING ); -count_vm_event (PGFAULT ); -count_memcg_event_mm (vma->vm_mm,PGFAULT ); - ret =sanitize_fault_flags (vma, &flags);if (ret) -return ret; + goto out;if (!arch_vma_access_permitted (vma, flags &FAULT_FLAG_WRITE , flags &FAULT_FLAG_INSTRUCTION , - flags &FAULT_FLAG_REMOTE )) -return VM_FAULT_SIGSEGV ; + flags &FAULT_FLAG_REMOTE )) { + ret =VM_FAULT_SIGSEGV ; + goto out; + }/* * Enable the memcg OOM handling for faults triggered in user @@ -5253,6 +5252,11 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, }
Quoted 4 lines
mm/memory.c vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, · line 5254
Quoted 1 line
mm/memory.c +0
-0mm_account_fault (regs, address, flags, ret);Here is the
mm_account_fault()function taking care of some other accountings. Perhaps good to put things into it?
That seems appropriate. Let me take a closer look.
Quoted 5 lines
It also already ignores invalid faults:
if (ret & (
VM_FAULT_ERROR|VM_FAULT_RETRY))
return;
Can there be a case of (!VM_FAULT_ERROR && VM_FAULT_RETRY) - basically we need to retry but no errors happened? If so then this condition would double-count pagefaults in such cases. If such return code is impossible then it's the same as checking for VM_FAULT_RETRY.
Quoted 4 lines
I see that you may also want to account for sigbus, however I really don't know why. Explanations would be great when it would matter. So far it makes sense to me if we skip both RETRY or ERROR cases.
Accounting in case of a sigbus is not affected by this patch I think. We account for sigbus or any other error cases because there was a pagefault and we need to account for it. Whether we failed to handle it or not should not affect the count. We skip the retry case because we know the same fault will be retried. If we don't skip then we will double-count this fault.
Quoted 6 lines
mm/memory.c vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, · line 5258
Quoted 5 lines
mm/memory.c +4
-0+out: +if (!(ret &VM_FAULT_RETRY )) { +count_vm_event (PGFAULT ); +count_memcg_event_mm (vma->vm_mm,PGFAULT );
Quoted 8 lines
There is one thing worth noticing is here vma may or may not be valid depending on the retval of the fault.
RETRY is exactly one of the cases that accessing vma may be unsafe due to releasing of mmap read lock. The other one is the recently added
VM_FAULT_COMPLETE. So if we want to move this chunk (or any vma reference) to be later we need to consider a valid vma / mm being there first, or we're prone to accessing a vma that has already been released, I think.
Good catch! I think you are right and I should have stored vma->vm_mm in the beginning and used it when calling count_memcg_event_mm(). I'll prepare a new patch which handles this correctly. Thanks,
Suren.
Quoted 18 lines
Quoted 9 lines
mm/memory.c +1
-1+ }return ret; } --2.40.0.634.g4ca3ef3211-goog
Thanks,
--
Peter Xu--
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
Suren Baghdasaryan
reviewerQuoted 78 lines · Suren Baghdasaryan
Quoted 2 lines · Peter Xu
Hi, Suren,
Hi Peter,
Quoted 48 lines · Suren Baghdasaryan
Quoted 46 lines
If the page fault handler requests a retry, we will count the fault multiple times. This is a relatively harmless problem as the retry paths are not often requested, and the only user-visible problem is that the fault counter will be slightly higher than it should be. Nevertheless, userspace only took one fault, and should not see the fact that the kernel had to retry the fault multiple times.
Fixes 6b4c9f446981 ("filemap: drop the mmap_sem for all blocking operations")Signed-off-by Suren Baghdasaryan <[email protected]>Reviewed-by Matthew Wilcox (Oracle) <[email protected]>---
Patch applies cleanly over linux-next and mm-unstablemm/memory.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)mm/memory.c +5
-6@@ -5212,17 +5212,16 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address,__set_current_state (TASK_RUNNING ); -count_vm_event (PGFAULT ); -count_memcg_event_mm (vma->vm_mm,PGFAULT ); - ret =sanitize_fault_flags (vma, &flags);if (ret) -return ret; + goto out;if (!arch_vma_access_permitted (vma, flags &FAULT_FLAG_WRITE , flags &FAULT_FLAG_INSTRUCTION , - flags &FAULT_FLAG_REMOTE )) -return VM_FAULT_SIGSEGV ; + flags &FAULT_FLAG_REMOTE )) { + ret =VM_FAULT_SIGSEGV ; + goto out; + }/* * Enable the memcg OOM handling for faults triggered in user @@ -5253,6 +5252,11 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, }Quoted 4 lines
mm/memory.c vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, · line 5254
Quoted 1 line
mm/memory.c +0
-0mm_account_fault (regs, address, flags, ret);Here is the
mm_account_fault()function taking care of some other accountings. Perhaps good to put things into it?That seems appropriate. Let me take a closer look.
Quoted 5 lines
It also already ignores invalid faults:
if (ret & (
VM_FAULT_ERROR|VM_FAULT_RETRY))
return;Can there be a case of (!VM_FAULT_ERROR &&
VM_FAULT_RETRY) - basically we need to retry but no errors happened? If so then this condition would double-count pagefaults in such cases. If such return code is impossible then it's the same as checking forVM_FAULT_RETRY.Quoted 4 lines
I see that you may also want to account for sigbus, however I really don't know why. Explanations would be great when it would matter. So far it makes sense to me if we skip both RETRY or ERROR cases.
Quoted 6 lines
Accounting in case of a sigbus is not affected by this patch I think. We account for sigbus or any other error cases because there was a pagefault and we need to account for it. Whether we failed to handle it or not should not affect the count. We skip the retry case because we know the same fault will be retried. If we don't skip then we will double-count this fault.
mm_account_fault() has a nice comment explaining why it skips errors and that now makes sense to me. Let me move the accounting there and see if others agree that's the right place.
Quoted 40 lines
Quoted 7 lines
mm/memory.c vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, · line 5258
Quoted 6 lines
mm/memory.c +4
-0+out: +if (!(ret &VM_FAULT_RETRY )) { +count_vm_event (PGFAULT ); +count_memcg_event_mm (vma->vm_mm,PGFAULT );Quoted 8 lines
There is one thing worth noticing is here vma may or may not be valid depending on the retval of the fault.
RETRY is exactly one of the cases that accessing vma may be unsafe due to releasing of mmap read lock. The other one is the recently added
VM_FAULT_COMPLETE. So if we want to move this chunk (or any vma reference) to be later we need to consider a valid vma / mm being there first, or we're prone to accessing a vma that has already been released, I think.Good catch! I think you are right and I should have stored
vma->vm_mmin the beginning and used it when callingcount_memcg_event_mm(). I'll prepare a new patch which handles this correctly. Thanks,
Suren.Quoted 18 lines
Quoted 9 lines
mm/memory.c +1
-1+ }return ret; } --2.40.0.634.g4ca3ef3211-goog
Thanks,
--
Peter Xu--
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
Hi, Suren,
Quoted 8 lines · Suren Baghdasaryan
Quoted 4 lines
It also already ignores invalid faults:
if (ret & (
VM_FAULT_ERROR|VM_FAULT_RETRY))
return;Can there be a case of (!VM_FAULT_ERROR &&
VM_FAULT_RETRY) - basically we need to retry but no errors happened? If so then this condition would double-count pagefaults in such cases.
If ret==VM_FAULT_RETRY it should return here already, so I assume mm_account_fault() itself is fine regarding fault retries?
Note that I think "ret & (VM_FAULT_ERROR | VM_FAULT_RETRY)" above means "either ERROR or RETRY we'll skip the accounting".
IMHO we should have 3 cases here:
- ERROR && !RETRY
error triggered of any kind
- RETRY && !ERROR
we need to try one more time
- !RETRY && !ERROR
we finished the fault
I don't think ERROR & RETRY can even be set at the same time so I assume there's no option 4) - a RETRY should imply no ERROR already, even though it's still incomplete so need another attempt.
Thanks,
Signature · 1 line
Peter Xu
Suren Baghdasaryan
reviewerQuoted 21 lines · Peter Xu
Hi, Suren,
Quoted 8 lines · Suren Baghdasaryan
Quoted 4 lines
It also already ignores invalid faults:
if (ret & (
VM_FAULT_ERROR|VM_FAULT_RETRY))
return;Can there be a case of (!VM_FAULT_ERROR &&
VM_FAULT_RETRY) - basically we need to retry but no errors happened? If so then this condition would double-count pagefaults in such cases.If ret==VM_FAULT_RETRY it should return here already, so I assume
mm_account_fault()itself is fine regarding fault retries?Note that I think "ret & (
VM_FAULT_ERROR|VM_FAULT_RETRY)" above means "either ERROR or RETRY we'll skip the accounting".IMHO we should have 3 cases here:
Quoted 8 lines
- ERROR && !RETRY
error triggered of any kind- RETRY && !ERROR
we need to try one more time- !RETRY && !ERROR
we finished the fault
After looking some more into mm_account_fault(), I think it would be fine to count the faults which produced errors. IIUC these counters represent the total number of faults, not the number of valid and successful faults. If so then I think simply using VM_FAULT_RETRY should be ok without considering all possible combinations. WDYT?
Quoted 10 lines
I don't think ERROR & RETRY can even be set at the same time so I assume there's no option 4) - a RETRY should imply no ERROR already, even though it's still incomplete so need another attempt.
Thanks,
--
Peter Xu
Suren Baghdasaryan
reviewerQuoted 29 lines · Suren Baghdasaryan
Quoted 27 lines · Peter Xu
Hi, Suren,
Quoted 8 lines · Suren Baghdasaryan
Quoted 4 lines
It also already ignores invalid faults:
if (ret & (
VM_FAULT_ERROR|VM_FAULT_RETRY))
return;Can there be a case of (!VM_FAULT_ERROR &&
VM_FAULT_RETRY) - basically we need to retry but no errors happened? If so then this condition would double-count pagefaults in such cases.If ret==VM_FAULT_RETRY it should return here already, so I assume
mm_account_fault()itself is fine regarding fault retries?Note that I think "ret & (
VM_FAULT_ERROR|VM_FAULT_RETRY)" above means "either ERROR or RETRY we'll skip the accounting".IMHO we should have 3 cases here:
- ERROR && !RETRY
error triggered of any kind- RETRY && !ERROR
we need to try one more time
Quoted 8 lines
Quoted 2 lines
- !RETRY && !ERROR
we finished the faultAfter looking some more into
mm_account_fault(), I think it would be fine to count the faults which produced errors. IIUC these counters represent the total number of faults, not the number of valid and successful faults. If so then I think simply usingVM_FAULT_RETRYshould be ok without considering all possible combinations. WDYT?
I posted v2 at https://lore.kernel.org/all/[email protected]/
Hopefully it's closer to what we want it to be.
Quoted 11 lines
Quoted 11 lines
I don't think ERROR & RETRY can even be set at the same time so I assume there's no option 4) - a RETRY should imply no ERROR already, even though it's still incomplete so need another attempt.
Thanks,
--
Peter Xu