Skip to contentSkip to conversation

PFs on pages pinned with get_user_pages()

PATCH Superseded Frank Mehnert 3 participants 16 messages Jan 29, 2009 – Jan 30, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

please could someone explain me under which circumstances a pagefault, either generated from kernel code or from userland code, can occur on pages which are pinned with get_user_pages()?

So far my understanding was that this can _never_ happen but I seems to be wrong. Under high memory pressure I get PFs on such pages raised from kernel code and the PFs are handled by do_swap_page(). When this happens, page_count is 3 but page_mapped() returns false.

Thanks in advance,

Frank

Signature · 1 line
Dr.-Ing. Frank Mehnert    Sun Microsystems    http://www.sun.com/
maintainer
#1 lore raw
Quoted 10 lines · Frank Mehnert

Hi,

please could someone explain me under which circumstances a pagefault, either generated from kernel code or from userland code, can occur on pages which are pinned with get_user_pages()?

So far my understanding was that this can _never_ happen but I seems to be wrong. Under high memory pressure I get PFs on such pages raised from kernel code and the PFs are handled by do_swap_page(). When this happens, page_count is 3 but page_mapped() returns false.

Under memory pressure the page reclaim will first unmap the physical page from the virtual address range, and then try to free it.

Obviously the freeing bit fails if you hold a reference to it, but the unmap will work.

After that, userspace will have to (minor) fault the stuff back in.

Also, that same page-reclaim, or pdflush might decide to write out dirty data, which will also result in (minor) faults when userspace will re-dirty the pages.

Having a page reference will only avoid the physical page from getting removed from its current mapping (and thereby also pins the mapping).

#2 lore raw

Peter,

Quoted 5 lines · Peter Zijlstra
Quoted 4 lines · Frank Mehnert

please could someone explain me under which circumstances a pagefault, either generated from kernel code or from userland code, can occur on pages which are pinned with get_user_pages()?

Quoted 7 lines
Quoted 4 lines

So far my understanding was that this can _never_ happen but I seems to be wrong. Under high memory pressure I get PFs on such pages raised from kernel code and the PFs are handled by do_swap_page(). When this happens, page_count is 3 but page_mapped() returns false.

Under memory pressure the page reclaim will first unmap the physical page from the virtual address range, and then try to free it.

Which means the page table entry is removed but the physical page is not swapped out, right?

Quoted 2 lines

Obviously the freeing bit fails if you hold a reference to it, but the unmap will work.

Right.

Quoted 1 line

After that, userspace will have to (minor) fault the stuff back in.

So do_swap_page does only 'restore' the page table entry, no further reading from the swapfile is necessary?

Quoted 6 lines

Also, that same page-reclaim, or pdflush might decide to write out dirty data, which will also result in (minor) faults when userspace will re-dirty the pages.

Having a page reference will only avoid the physical page from getting removed from its current mapping (and thereby also pins the mapping).

Question: Is it possible to prevent these minor page faults at all?

Thank you very much for your answer!

Frank

Signature · 1 line
Dr.-Ing. Frank Mehnert    Sun Microsystems    http://www.sun.com/
maintainer
#3 lore raw
Quoted 1 line · Frank Mehnert

Peter,

(please retain CC's)

Quoted 11 lines · Peter Zijlstra
Quoted 10 lines · Frank Mehnert
Quoted 8 lines

please could someone explain me under which circumstances a pagefault, either generated from kernel code or from userland code, can occur on pages which are pinned with get_user_pages()?

So far my understanding was that this can _never_ happen but I seems to be wrong. Under high memory pressure I get PFs on such pages raised from kernel code and the PFs are handled by do_swap_page(). When this happens, page_count is 3 but page_mapped() returns false.

Quoted 5 lines
Quoted 2 lines

Under memory pressure the page reclaim will first unmap the physical page from the virtual address range, and then try to free it.

Which means the page table entry is removed but the physical page is not swapped out, right?

Correct.

Quoted 9 lines
Quoted 2 lines

Obviously the freeing bit fails if you hold a reference to it, but the unmap will work.

Right.

Quoted 1 line

After that, userspace will have to (minor) fault the stuff back in.

So do_swap_page does only 'restore' the page table entry, no further reading from the swapfile is necessary?

Indeed.

Quoted 8 lines
Quoted 6 lines

Also, that same page-reclaim, or pdflush might decide to write out dirty data, which will also result in (minor) faults when userspace will re-dirty the pages.

Having a page reference will only avoid the physical page from getting removed from its current mapping (and thereby also pins the mapping).

Question: Is it possible to prevent these minor page faults at all?

Not without some serious tinkering to the VM -- and in the case of the dirty fault, not at all.

Why are you asking?

#4 lore raw
Quoted 30 lines · Peter Zijlstra
Quoted 1 line · Frank Mehnert

Peter,

(please retain CC's)

Quoted 11 lines · Peter Zijlstra
Quoted 10 lines · Frank Mehnert
Quoted 8 lines

please could someone explain me under which circumstances a pagefault, either generated from kernel code or from userland code, can occur on pages which are pinned with get_user_pages()?

So far my understanding was that this can _never_ happen but I seems to be wrong. Under high memory pressure I get PFs on such pages raised from kernel code and the PFs are handled by do_swap_page(). When this happens, page_count is 3 but page_mapped() returns false.

Quoted 5 lines
Quoted 2 lines

Under memory pressure the page reclaim will first unmap the physical page from the virtual address range, and then try to free it.

Which means the page table entry is removed but the physical page is not swapped out, right?

Correct.

Quoted 6 lines
Quoted 2 lines

Obviously the freeing bit fails if you hold a reference to it, but the unmap will work.

Right.

Quoted 1 line

After that, userspace will have to (minor) fault the stuff back in.

[...]

Quoted 6 lines
Quoted 1 line

Question: Is it possible to prevent these minor page faults at all?

Not without some serious tinkering to the VM -- and in the case of the dirty fault, not at all.

Why are you asking?

I'm one of the VirtualBox developers. We are trying to fix the annoying kerneloops warning 'BUG: sleeping function called from invalid context' reported by the Fedora folks. This warning occurs when do_swap_page() calls lock_page() and in_atomic() returns true.

This warning appears when we touch into memory which is pinned with get_user_pages(). In VT-x/AMD-V mode we are executing some code in the context of the Linux kernel. To prevent scheduling of the current CPU core we disable the interripts. preempt_disable() would be probably the better choice but this would oops as well if CONFIG_PREEMPT is enabled.

Kind regards,

Frank

Signature · 1 line
Dr.-Ing. Frank Mehnert    Sun Microsystems    http://www.sun.com/
maintainer
#5 lore raw
Quoted 10 lines · Frank Mehnert

I'm one of the VirtualBox developers. We are trying to fix the annoying kerneloops warning 'BUG: sleeping function called from invalid context' reported by the Fedora folks. This warning occurs when do_swap_page() calls lock_page() and in_atomic() returns true.

This warning appears when we touch into memory which is pinned with get_user_pages(). In VT-x/AMD-V mode we are executing some code in the context of the Linux kernel. To prevent scheduling of the current CPU core we disable the interripts. preempt_disable() would be probably the better choice but this would oops as well if CONFIG_PREEMPT is enabled.

but to get there, you'd have to have called handle_mm_fault() which requires the mmap_sem, which should also give that might_sleep() warning.

That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.

#6 lore raw
Quoted 12 lines · Peter Zijlstra
Quoted 10 lines · Frank Mehnert

I'm one of the VirtualBox developers. We are trying to fix the annoying kerneloops warning 'BUG: sleeping function called from invalid context' reported by the Fedora folks. This warning occurs when do_swap_page() calls lock_page() and in_atomic() returns true.

This warning appears when we touch into memory which is pinned with get_user_pages(). In VT-x/AMD-V mode we are executing some code in the context of the Linux kernel. To prevent scheduling of the current CPU core we disable the interripts. preempt_disable() would be probably the better choice but this would oops as well if CONFIG_PREEMPT is enabled.

Quoted 3 lines

but to get there, you'd have to have called handle_mm_fault() which requires the mmap_sem, which should also give that might_sleep() warning.

The stacktrace is

__might_sleep()
lock_page()
handle_mm_fault()
do_page_fault()
error_code

So yes, handle_mm_fault() is called. But I assume that down_read_trylock() succeeded before we were forced to call down_read().

Quoted 2 lines

That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.

The reason is that our code expects that to ensure syncing of the CPU state with the saved state. I fear it is quite difficult to change that...

Kind regards,

Frank

Signature · 1 line
Dr.-Ing. Frank Mehnert    Sun Microsystems    http://www.sun.com/
maintainer
#7 lore raw
Quoted 25 lines · Frank Mehnert
Quoted 12 lines · Peter Zijlstra
Quoted 10 lines · Frank Mehnert

I'm one of the VirtualBox developers. We are trying to fix the annoying kerneloops warning 'BUG: sleeping function called from invalid context' reported by the Fedora folks. This warning occurs when do_swap_page() calls lock_page() and in_atomic() returns true.

This warning appears when we touch into memory which is pinned with get_user_pages(). In VT-x/AMD-V mode we are executing some code in the context of the Linux kernel. To prevent scheduling of the current CPU core we disable the interripts. preempt_disable() would be probably the better choice but this would oops as well if CONFIG_PREEMPT is enabled.

Quoted 3 lines

but to get there, you'd have to have called handle_mm_fault() which requires the mmap_sem, which should also give that might_sleep() warning.

The stacktrace is

__might_sleep()
lock_page()
handle_mm_fault()
do_page_fault()
error_code

Quoted 8 lines

So yes, handle_mm_fault() is called. But I assume that down_read_trylock() succeeded before we were forced to call down_read().

Quoted 2 lines

That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.

The reason is that our code expects that to ensure syncing of the CPU state with the saved state. I fear it is quite difficult to change that...

Ah, is that what KVM uses the preempt notifiers for? Could you too?

#8 lore raw
Quoted 8 lines · Peter Zijlstra
Quoted 6 lines
Quoted 2 lines

That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.

The reason is that our code expects that to ensure syncing of the CPU state with the saved state. I fear it is quite difficult to change that...

Ah, is that what KVM uses the preempt notifiers for? Could you too?

Right, that could be an option.

We will try to change our code which is a big effort as we try to keep the code as unique as possible between the different hosts we support (Linux, Solaris, Windows, Mac OS X).

Just to be sure: There is no other option than disabling interrupts or calling disable_preemption() to prevent scheduling?

Kind regards,

Frank

Signature · 1 line
Dr.-Ing. Frank Mehnert    Sun Microsystems    http://www.sun.com/
maintainer
#9 lore raw
Quoted 10 lines · Frank Mehnert
Quoted 8 lines · Peter Zijlstra
Quoted 6 lines
Quoted 2 lines

That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.

The reason is that our code expects that to ensure syncing of the CPU state with the saved state. I fear it is quite difficult to change that...

Ah, is that what KVM uses the preempt notifiers for? Could you too?

Quoted 8 lines

Right, that could be an option.

We will try to change our code which is a big effort as we try to keep the code as unique as possible between the different hosts we support (Linux, Solaris, Windows, Mac OS X).

Just to be sure: There is no other option than disabling interrupts or calling disable_preemption() to prevent scheduling?

Thing is, lock_page() and down_read() require to be able to schedule(), so there's no way around that.

So even if there was another way to disable scheduling, you'd still have the same problem.

#10 lore raw
Quoted 17 lines · Peter Zijlstra
Quoted 16 lines · Frank Mehnert
Quoted 8 lines · Peter Zijlstra
Quoted 6 lines
Quoted 2 lines

That aside, is there any reason you have to avoid scheduling? Otherwise I would just allow so and be done with it.

The reason is that our code expects that to ensure syncing of the CPU state with the saved state. I fear it is quite difficult to change that...

Ah, is that what KVM uses the preempt notifiers for? Could you too?

Right, that could be an option.

We will try to change our code which is a big effort as we try to keep the code as unique as possible between the different hosts we support (Linux, Solaris, Windows, Mac OS X).

Quoted 8 lines
Quoted 2 lines

Just to be sure: There is no other option than disabling interrupts or calling disable_preemption() to prevent scheduling?

Thing is, lock_page() and down_read() require to be able to schedule(), so there's no way around that.

So even if there was another way to disable scheduling, you'd still have the same problem.

Yes, makes sense.

Back to my initial question: The problem arises for us because we depend on permanent mappings of memory which were

- allocated with alloc_pages() or alloc_page()
- mapped into ring 3 with remap_pfn_range() and
- pinned with get_user_pages()

There are potential pagefaults when touching into these ring-3-mappings from ring 0. So I assume we could prevent such pagefaults if we access that memory from ring-0-mappings, right? Unfortunately, the space for ring-0-mappings (< 1GB) is smaller than userland (~ 3GB), at least on 32-bit systems.

Kind regards,

Frank

Signature · 1 line
Dr.-Ing. Frank Mehnert    Sun Microsystems    http://www.sun.com/
maintainer
#11 lore raw
Quoted 15 lines · Frank Mehnert
Quoted 5 lines

Thing is, lock_page() and down_read() require to be able to schedule(), so there's no way around that.

So even if there was another way to disable scheduling, you'd still have the same problem.

Yes, makes sense.

Back to my initial question: The problem arises for us because we depend on permanent mappings of memory which were

- allocated with alloc_pages() or alloc_page()
- mapped into ring 3 with remap_pfn_range() and
- pinned with get_user_pages()

Quoted 5 lines

There are potential pagefaults when touching into these ring-3-mappings from ring 0. So I assume we could prevent such pagefaults if we access that memory from ring-0-mappings, right? Unfortunately, the space for ring-0-mappings (< 1GB) is smaller than userland (~ 3GB), at least on 32-bit systems.

if you only need to access one or two pages, you could kmap_atomic() the actual pages from ring-0.

maintainer
#12 lore raw

PATCH 1/1 x86: add might_sleep() to do_page_fault()

In reply to Frank Mehnert

VirtualBox calls do_page_fault() from an atomic context but runs into a might_sleep() way pas this point, cure that.

---
 arch/x86/mm/fault.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)
arch/x86/mm/fault.c +5 -0
index 67e4df5..bb7f946 100644
@@ -908,6 +908,11 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code)
 		}
 		down_read(&mm->mmap_sem);
 	}
+	/*
+	 * The above down_read_trylock() might have succeeded in which case
+	 * we'll have missed the might_sleep() from down_read().
+	 */
+	might_sleep();
 
 	vma = find_vma(mm, address);
 	if (unlikely(!vma)) {
#13 lore raw

Re: [PATCH] x86: add might_sleep() to do_page_fault()

Quoted 13 lines · Peter Zijlstra

VirtualBox calls do_page_fault() from an atomic context but runs into a might_sleep() way pas this point, cure that.

---
 arch/x86/mm/fault.c |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)
arch/x86/mm/fault.c +0 -0
index 67e4df5..bb7f946 100644

arch/x86/mm/fault.c void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code) · line 915

Quoted 8 lines
arch/x86/mm/fault.c +5 -0
@@ -908,6 +908,11 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code)
 		}
 		down_read(&mm->mmap_sem);
 	}
+	/*
+	 * The above down_read_trylock() might have succeeded in which case
+	 * we'll have missed the might_sleep() from down_read().
+	 */
+	might_sleep();

should go into the 'else' branch i guess? In the down_read() case we already had the check.

Ingo

maintainer
#14 lore raw

PATCH v2 1/1 x86: add might_sleep() to do_page_fault()

Quoted 3 lines

should go into the 'else' branch i guess? In the down_read() case we already had the check.

True.

---
VirtualBox calls do_page_fault() from an atomic context but runs into a might_sleep() way pas this point, cure that.

---
 arch/x86/mm/fault.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)
arch/x86/mm/fault.c +6 -0
index 67e4df5..bfac289 100644
@@ -907,6 +907,12 @@ void __kprobes do_page_fault(struct pt_regs *regs, unsigned long error_code)
 			return;
 		}
 		down_read(&mm->mmap_sem);
+	} else {
+		/*
+		 * The above down_read_trylock() might have succeeded in which
+		 * case we'll have missed the might_sleep() from down_read().
+		 */
+		might_sleep();
 	}
 
 	vma = find_vma(mm, address);
Applied
#15 lore raw

Re: [PATCH v2] x86: add might_sleep() to do_page_fault()

Quoted 6 lines · Peter Zijlstra
Quoted 3 lines

should go into the 'else' branch i guess? In the down_read() case we already had the check.

True.

Quoted 8 lines

---
VirtualBox calls do_page_fault() from an atomic context but runs into a might_sleep() way pas this point, cure that.

---
 arch/x86/mm/fault.c |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

Applied to tip/x86/mm, thanks Peter!

Ingo