PFs on pages pinned with get_user_pages()
Hi,
please could someone explain me under which circumstances a pagefault, either generated from kernel code or from userland code, can occur on pages which are pinned with get_user_pages()?
So far my understanding was that this can _never_ happen but I seems to be wrong. Under high memory pressure I get PFs on such pages raised from kernel code and the PFs are handled by do_swap_page(). When this happens, page_count is 3 but page_mapped() returns false.
Thanks in advance,
Frank
Signature · 1 line
Dr.-Ing. Frank Mehnert Sun Microsystems http://www.sun.com/
Peter Zijlstra
maintainerQuoted 10 lines · Frank Mehnert
Hi,
please could someone explain me under which circumstances a pagefault, either generated from kernel code or from userland code, can occur on pages which are pinned with
get_user_pages()?So far my understanding was that this can _never_ happen but I seems to be wrong. Under high memory pressure I get PFs on such pages raised from kernel code and the PFs are handled by
do_swap_page(). When this happens,page_countis 3 butpage_mapped()returns false.
Under memory pressure the page reclaim will first unmap the physical page from the virtual address range, and then try to free it.
Obviously the freeing bit fails if you hold a reference to it, but the unmap will work.
After that, userspace will have to (minor) fault the stuff back in.
Also, that same page-reclaim, or pdflush might decide to write out dirty data, which will also result in (minor) faults when userspace will re-dirty the pages.
Having a page reference will only avoid the physical page from getting removed from its current mapping (and thereby also pins the mapping).
Peter,
Quoted 5 lines · Peter Zijlstra
Quoted 4 lines · Frank Mehnert
please could someone explain me under which circumstances a pagefault, either generated from kernel code or from userland code, can occur on pages which are pinned with
get_user_pages()?
Quoted 7 lines
Quoted 4 lines
So far my understanding was that this can _never_ happen but I seems to be wrong. Under high memory pressure I get PFs on such pages raised from kernel code and the PFs are handled by
do_swap_page(). When this happens,page_countis 3 butpage_mapped()returns false.Under memory pressure the page reclaim will first unmap the physical page from the virtual address range, and then try to free it.
Which means the page table entry is removed but the physical page is not swapped out, right?
Quoted 2 lines
Obviously the freeing bit fails if you hold a reference to it, but the unmap will work.
Right.
Quoted 1 line
After that, userspace will have to (minor) fault the stuff back in.
So do_swap_page does only 'restore' the page table entry, no further reading from the swapfile is necessary?
Quoted 6 lines
Also, that same page-reclaim, or pdflush might decide to write out dirty data, which will also result in (minor) faults when userspace will re-dirty the pages.
Having a page reference will only avoid the physical page from getting removed from its current mapping (and thereby also pins the mapping).
Question: Is it possible to prevent these minor page faults at all?
Thank you very much for your answer!
Frank
Signature · 1 line
Dr.-Ing. Frank Mehnert Sun Microsystems http://www.sun.com/
Peter Zijlstra
maintainerQuoted 1 line · Frank Mehnert
Peter,
(please retain CC's)
Quoted 11 lines · Peter Zijlstra
Quoted 10 lines · Frank Mehnert
Quoted 8 lines
please could someone explain me under which circumstances a pagefault, either generated from kernel code or from userland code, can occur on pages which are pinned with
get_user_pages()?So far my understanding was that this can _never_ happen but I seems to be wrong. Under high memory pressure I get PFs on such pages raised from kernel code and the PFs are handled by
do_swap_page(). When this happens,page_countis 3 butpage_mapped()returns false.
Quoted 5 lines
Quoted 2 lines
Under memory pressure the page reclaim will first unmap the physical page from the virtual address range, and then try to free it.
Which means the page table entry is removed but the physical page is not swapped out, right?
Correct.
Quoted 9 lines
Quoted 2 lines
Obviously the freeing bit fails if you hold a reference to it, but the unmap will work.
Right.
Quoted 1 line
After that, userspace will have to (minor) fault the stuff back in.
So
do_swap_pagedoes only 'restore' the page table entry, no further reading from the swapfile is necessary?
Indeed.
Quoted 8 lines
Quoted 6 lines
Also, that same page-reclaim, or pdflush might decide to write out dirty data, which will also result in (minor) faults when userspace will re-dirty the pages.
Having a page reference will only avoid the physical page from getting removed from its current mapping (and thereby also pins the mapping).
Question: Is it possible to prevent these minor page faults at all?
Not without some serious tinkering to the VM -- and in the case of the dirty fault, not at all.
Why are you asking?
Quoted 30 lines · Peter Zijlstra
Quoted 1 line · Frank Mehnert
Peter,
(please retain CC's)
Quoted 11 lines · Peter Zijlstra
Quoted 10 lines · Frank Mehnert
Quoted 8 lines
please could someone explain me under which circumstances a pagefault, either generated from kernel code or from userland code, can occur on pages which are pinned with
get_user_pages()?So far my understanding was that this can _never_ happen but I seems to be wrong. Under high memory pressure I get PFs on such pages raised from kernel code and the PFs are handled by
do_swap_page(). When this happens,page_countis 3 butpage_mapped()returns false.Quoted 5 lines
Quoted 2 lines
Under memory pressure the page reclaim will first unmap the physical page from the virtual address range, and then try to free it.
Which means the page table entry is removed but the physical page is not swapped out, right?
Correct.
Quoted 6 lines
Quoted 2 lines
Obviously the freeing bit fails if you hold a reference to it, but the unmap will work.
Right.
Quoted 1 line
After that, userspace will have to (minor) fault the stuff back in.
[...]
Quoted 6 lines
Quoted 1 line
Question: Is it possible to prevent these minor page faults at all?
Not without some serious tinkering to the VM -- and in the case of the dirty fault, not at all.
Why are you asking?
I'm one of the VirtualBox developers. We are trying to fix the annoying kerneloops warning 'BUG: sleeping function called from invalid context' reported by the Fedora folks. This warning occurs when do_swap_page() calls lock_page() and in_atomic() returns true.
This warning appears when we touch into memory which is pinned with get_user_pages(). In VT-x/AMD-V mode we are executing some code in the context of the Linux kernel. To prevent scheduling of the current CPU core we disable the interripts. preempt_disable() would be probably the better choice but this would oops as well if CONFIG_PREEMPT is enabled.
Kind regards,
Frank
Signature · 1 line
Dr.-Ing. Frank Mehnert Sun Microsystems http://www.sun.com/
Peter Zijlstra
maintainerQuoted 10 lines · Frank Mehnert
I'm one of the VirtualBox developers. We are trying to fix the annoying kerneloops warning 'BUG: sleeping function called from invalid context' reported by the Fedora folks. This warning occurs when
do_swap_page()callslock_page()andin_atomic()returns true.This warning appears when we touch into memory which is pinned with
get_user_pages(). In VT-x/AMD-V mode we are executing some code in the context of the Linux kernel. To prevent scheduling of the current CPU core we disable the interripts.preempt_disable()would be probably the better choice but this would oops as well ifCONFIG_PREEMPTis enabled.
but to get there, you'd have to have called handle_mm_fault() which requires the mmap_sem, which should also give that might_sleep() warning.
That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.
Quoted 12 lines · Peter Zijlstra
Quoted 10 lines · Frank Mehnert
I'm one of the VirtualBox developers. We are trying to fix the annoying kerneloops warning 'BUG: sleeping function called from invalid context' reported by the Fedora folks. This warning occurs when
do_swap_page()callslock_page()andin_atomic()returns true.This warning appears when we touch into memory which is pinned with
get_user_pages(). In VT-x/AMD-V mode we are executing some code in the context of the Linux kernel. To prevent scheduling of the current CPU core we disable the interripts.preempt_disable()would be probably the better choice but this would oops as well ifCONFIG_PREEMPTis enabled.
Quoted 3 lines
but to get there, you'd have to have called
handle_mm_fault()which requires themmap_sem, which should also give thatmight_sleep()warning.
The stacktrace is
__might_sleep()
lock_page()
handle_mm_fault()
do_page_fault()
error_code
So yes, handle_mm_fault() is called. But I assume that down_read_trylock() succeeded before we were forced to call down_read().
Quoted 2 lines
That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.
The reason is that our code expects that to ensure syncing of the CPU state with the saved state. I fear it is quite difficult to change that...
Kind regards,
Frank
Signature · 1 line
Dr.-Ing. Frank Mehnert Sun Microsystems http://www.sun.com/
Peter Zijlstra
maintainerQuoted 25 lines · Frank Mehnert
Quoted 12 lines · Peter Zijlstra
Quoted 10 lines · Frank Mehnert
I'm one of the VirtualBox developers. We are trying to fix the annoying kerneloops warning 'BUG: sleeping function called from invalid context' reported by the Fedora folks. This warning occurs when
do_swap_page()callslock_page()andin_atomic()returns true.This warning appears when we touch into memory which is pinned with
get_user_pages(). In VT-x/AMD-V mode we are executing some code in the context of the Linux kernel. To prevent scheduling of the current CPU core we disable the interripts.preempt_disable()would be probably the better choice but this would oops as well ifCONFIG_PREEMPTis enabled.Quoted 3 lines
but to get there, you'd have to have called
handle_mm_fault()which requires themmap_sem, which should also give thatmight_sleep()warning.The stacktrace is
__might_sleep()
lock_page()
handle_mm_fault()
do_page_fault()
error_code
Quoted 8 lines
So yes,
handle_mm_fault()is called. But I assume thatdown_read_trylock()succeeded before we were forced to calldown_read().Quoted 2 lines
That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.
The reason is that our code expects that to ensure syncing of the CPU state with the saved state. I fear it is quite difficult to change that...
Ah, is that what KVM uses the preempt notifiers for? Could you too?
Quoted 8 lines · Peter Zijlstra
Quoted 6 lines
Quoted 2 lines
That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.
The reason is that our code expects that to ensure syncing of the CPU state with the saved state. I fear it is quite difficult to change that...
Ah, is that what KVM uses the preempt notifiers for? Could you too?
Right, that could be an option.
We will try to change our code which is a big effort as we try to keep the code as unique as possible between the different hosts we support (Linux, Solaris, Windows, Mac OS X).
Just to be sure: There is no other option than disabling interrupts or calling disable_preemption() to prevent scheduling?
Kind regards,
Frank
Signature · 1 line
Dr.-Ing. Frank Mehnert Sun Microsystems http://www.sun.com/
Peter Zijlstra
maintainerQuoted 10 lines · Frank Mehnert
Quoted 8 lines · Peter Zijlstra
Quoted 6 lines
Quoted 2 lines
That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.
The reason is that our code expects that to ensure syncing of the CPU state with the saved state. I fear it is quite difficult to change that...
Ah, is that what KVM uses the preempt notifiers for? Could you too?
Quoted 8 lines
Right, that could be an option.
We will try to change our code which is a big effort as we try to keep the code as unique as possible between the different hosts we support (Linux, Solaris, Windows, Mac OS X).
Just to be sure: There is no other option than disabling interrupts or calling
disable_preemption()to prevent scheduling?
Thing is, lock_page() and down_read() require to be able to schedule(), so there's no way around that.
So even if there was another way to disable scheduling, you'd still have the same problem.
Quoted 17 lines · Peter Zijlstra
Quoted 16 lines · Frank Mehnert
Quoted 8 lines · Peter Zijlstra
Quoted 6 lines
Quoted 2 lines
That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.
The reason is that our code expects that to ensure syncing of the CPU state with the saved state. I fear it is quite difficult to change that...
Ah, is that what KVM uses the preempt notifiers for? Could you too?
Right, that could be an option.
We will try to change our code which is a big effort as we try to keep the code as unique as possible between the different hosts we support (Linux, Solaris, Windows, Mac OS X).
Quoted 8 lines
Quoted 2 lines
Just to be sure: There is no other option than disabling interrupts or calling
disable_preemption()to prevent scheduling?Thing is,
lock_page()anddown_read()require to be able toschedule(), so there's no way around that.So even if there was another way to disable scheduling, you'd still have the same problem.
Yes, makes sense.
Back to my initial question: The problem arises for us because we depend on permanent mappings of memory which were
- allocated with alloc_pages() or alloc_page()
- mapped into ring 3 with remap_pfn_range() and
- pinned with get_user_pages()
There are potential pagefaults when touching into these ring-3-mappings from ring 0. So I assume we could prevent such pagefaults if we access that memory from ring-0-mappings, right? Unfortunately, the space for ring-0-mappings (< 1GB) is smaller than userland (~ 3GB), at least on 32-bit systems.
Kind regards,
Frank
Signature · 1 line
Dr.-Ing. Frank Mehnert Sun Microsystems http://www.sun.com/
Peter Zijlstra
maintainerQuoted 15 lines · Frank Mehnert
Quoted 5 lines
Thing is,
lock_page()anddown_read()require to be able toschedule(), so there's no way around that.So even if there was another way to disable scheduling, you'd still have the same problem.
Yes, makes sense.
Back to my initial question: The problem arises for us because we depend on permanent mappings of memory which were
- allocated with
alloc_pages()oralloc_page()
- mapped into ring 3 withremap_pfn_range()and
- pinned withget_user_pages()
Quoted 5 lines
There are potential pagefaults when touching into these ring-3-mappings from ring 0. So I assume we could prevent such pagefaults if we access that memory from ring-0-mappings, right? Unfortunately, the space for ring-0-mappings (< 1GB) is smaller than userland (~ 3GB), at least on 32-bit systems.
if you only need to access one or two pages, you could kmap_atomic() the actual pages from ring-0.
Peter Zijlstra
maintainerPATCH 1/1 x86: add might_sleep() to do_page_fault()
In reply to Frank MehnertVirtualBox calls do_page_fault() from an atomic context but runs into a might_sleep() way pas this point, cure that.
--- arch/x86/mm/fault.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-)
arch/x86/mm/fault.c +5 -0
@@ -908,6 +908,11 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code) }down_read (&mm->mmap_sem); } +/* +* The above down_read_trylock() might have succeeded in which case +* we'll have missed the might_sleep() from down_read(). +*/ +might_sleep (); vma =find_vma (mm, address);if (unlikely (!vma)) {
Re: [PATCH] x86: add might_sleep() to do_page_fault()
Quoted 13 lines · Peter Zijlstra
VirtualBox calls
do_page_fault()from an atomic context but runs into amight_sleep()way pas this point, cure that.Signed-off-by Peter Zijlstra <[email protected]>--- arch/x86/mm/fault.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-)arch/x86/mm/fault.c +0
-0
arch/x86/mm/fault.c void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code) · line 915
Quoted 8 lines
arch/x86/mm/fault.c +5
-0@@ -908,6 +908,11 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code) }down_read (&mm->mmap_sem); } +/* +* The above down_read_trylock() might have succeeded in which case +* we'll have missed the might_sleep() from down_read(). +*/ +might_sleep ();
should go into the 'else' branch i guess? In the down_read() case we already had the check.
Ingo
Peter Zijlstra
maintainerPATCH v2 1/1 x86: add might_sleep() to do_page_fault()
Quoted 3 lines
should go into the 'else' branch i guess? In the
down_read()case we already had the check.
True.
---
VirtualBox calls do_page_fault() from an atomic context but runs into a might_sleep() way pas this point, cure that.
--- arch/x86/mm/fault.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
arch/x86/mm/fault.c +6 -0
@@ -907,6 +907,12 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code)return ; }down_read (&mm->mmap_sem); + }else { +/* +* The above down_read_trylock() might have succeeded in which +* case we'll have missed the might_sleep() from down_read(). +*/ +might_sleep (); } vma =find_vma (mm, address);
Ingo Molnar
AppliedRe: [PATCH v2] x86: add might_sleep() to do_page_fault()
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines
should go into the 'else' branch i guess? In the
down_read()case we already had the check.True.
Quoted 8 lines
---
VirtualBox callsdo_page_fault()from an atomic context but runs into amight_sleep()way pas this point, cure that.Signed-off-by Peter Zijlstra <[email protected]>--- arch/x86/mm/fault.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
Applied to tip/x86/mm, thanks Peter!
Ingo