pte notifiers -- support for external page tables

RFC PATCH In review Avi Kivity 4 participants 9 messages Sep 5, 2007 – Sep 6, 2007

Subsystems (5)

Memory management KVM MM core Memory mapping +1 more subsystems

lore

Share & tools

lore

Avi Kivity

author

2007-09-05 19:32 UTC #0 lore raw

RFC PATCH 1/1 pte notifiers -- support for external page tables

[resend due to bad alias expansion resulting in some recipients
being bogus]

Some hardware and software systems maintain page tables outside the normal Linux page tables, which reference userspace memory. This includes Infiniband, other RDMA-capable devices, and kvm (with a pending patch).

Because these systems maintain external page tables (and external tlbs), Linux cannot demand page this memory and it must be locked. For kvm at least, this is a significant reduction in functionality.

This sample patch adds a new mechanism, pte notifiers, that allows drivers to register an interest in a changes to ptes. Whenever Linux changes a pte, it will call a notifier to allow the driver to adjust the external page table and flush its tlb.

Note that only one notifier is implemented, ->clear(), but others should be similar.

pte notifiers are different from paravirt_ops: they extend the normal page tables rather than replace them; and they provide high-level information such as the vma and the virtual address for the driver to use.

Signed-off-by Avi Kivity <avi-atKUWr5tajBWk0Htik3J/[email protected]>

include/linux/mm.h +4 -0

index 655094d..5d2bbee 100644
@@ -14,6 +14,7 @@
 #include <linux/debug_locks.h>
 #include <linux/backing-dev.h>
 #include <linux/mm_types.h>
+#include <linux/pte_notifier.h>
 
 struct mempolicy;
 struct anon_vma;

@@ -108,6 +109,9 @@ struct vm_area_struct {
 #ifndef CONFIG_MMU
 	atomic_t vm_usage;		/* refcount (VMAs shared if !MMU) */
 #endif
+#ifdef CONFIG_PTE_NOTIFIERS
+	struct list_head pte_notifier_list;
+#endif
 #ifdef CONFIG_NUMA
 	struct mempolicy *vm_policy;	/* NUMA policy for the VMA */
 #endif

include/linux/pte_notifier.h new file +52 -0

52 lines not on this page · raw message

mm/Kconfig +3 -0

index e24d348..7b10151 100644
@@ -176,3 +176,6 @@ config NR_QUICK
 config VIRT_TO_BUS
 	def_bool y
 	depends on !ARCH_NO_VIRT_TO_BUS
+
+config PTE_NOTIFIERS
+       bool

mm/Makefile +1 -0

index 245e33a..59f6a03 100644
@@ -29,4 +29,5 @@ obj-$(CONFIG_FS_XIP) += filemap_xip.o
 obj-$(CONFIG_MIGRATION) += migrate.o
 obj-$(CONFIG_SMP) += allocpercpu.o
 obj-$(CONFIG_QUICKLIST) += quicklist.o
+obj-$(CONFIG_PTE_NOTIFIERS) += pte_notifiers.o

mm/mmap.c +1 -0

index b653721..cc6c4fe 100644
@@ -1134,6 +1134,7 @@ munmap_back:
 	vma->vm_page_prot = protection_map[vm_flags &
 				(VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)];
 	vma->vm_pgoff = pgoff;
+	vma_init_pte_notifiers(vma);
 
 	if (file) {
 		error = -EINVAL;

mm/pte_notifier.c new file +32 -0

new file mode 100644
index 0000000..0b9076c
@@ -0,0 +1,32 @@
+
+#include <linux/pte_notifier.h>
+
+void vma_init_pte_notifiers(struct vm_area_struct *vma)
+{
+	INIT_LIST_HEAD(&vma->pte_notifier_list);
+}
+EXPORT_SYMBOL_GPL(vma_init_pte_notifiers);
+
+void vma_destroy_pte_notifiers(struct vm_area_struct *vma)
+{
+	struct pte_notifier *pn;
+	struct list_head *n;
+
+	list_for_each_entry_safe(pn, n, &vma->pte_notifier_list, link) {
+		pn->ops->close(__pn, vma);
+		__list_del(n);
+	}
+}
+
+void pte_notifier_register(struct pte_notifier *pn, struct vm_area_struct *vma)
+{
+	list_add(&pn->link, &vma->pte_notifier_list);
+}
+EXPORT_SYMBOL_GPL(pte_notifier_register);
+
+void pte_notifier_unregister(struct pte_notifier *pn)
+{
+	list_del(&pn->link);
+}
+EXPORT_SYMBOL_GPL(pte_notifier_unregister);
+

mm/rmap.c +1 -0

index 41ac397..3f61d38 100644
@@ -682,6 +682,7 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 	}
 
 	/* Nuke the page table entry. */
+	pte_notifier_call(vma, clear, address);
 	flush_cache_page(vma, address, page_to_pfn(page));
 	pteval = ptep_clear_flush(vma, address, pte);

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/

Rusty Russell

2007-09-05 19:56 UTC #1 lore raw

Quoted 6 lines · Avi Kivity

[resend due to bad alias expansion resulting in some recipients
being bogus]
Some hardware and software systems maintain page tables outside the normal Linux page tables, which reference userspace memory. This includes Infiniband, other RDMA-capable devices, and kvm (with a pending patch).

And lguest. I can't tell until I've actually implemented it, but I think it will seriously reduce the need for page pinning which is why only root can currently launch guests.

My concern is locking: this is called with the page lock held, and I guess we have to bump the guest out if it's currently running.

(Oh, and this means lguest needs to do a reverse mapping somehow, but I'll come up with something).

Cheers,
Rusty.

Avi Kivity

author

2007-09-05 20:17 UTC #2 lore raw

Quoted 5 lines · Rusty Russell

Quoted 3 lines · Avi Kivity
[resend due to bad alias expansion resulting in some recipients
being bogus]

Quoted 10 lines

Quoted 4 lines
Some hardware and software systems maintain page tables outside the normal Linux page tables, which reference userspace memory. This includes Infiniband, other RDMA-capable devices, and kvm (with a pending patch).
And lguest. I can't tell until I've actually implemented it, but I think it will seriously reduce the need for page pinning which is why only root can currently launch guests.

Ah yes, lguest.

Quoted 3 lines

My concern is locking: this is called with the page lock held, and I guess we have to bump the guest out if it's currently running.

This will complicate kvm's locking too. We usually take kvm->lock to do mmu ops, but that is now a mutex.

Signature · 8 lines

Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >>  http://get.splunk.com/

Shaohua Li

2007-09-06 04:28 UTC #3 lore raw

In reply to Avi Kivity

Quoted 16 lines · Avi Kivity

[resend due to bad alias expansion resulting in some recipients
being bogus]
Some hardware and software systems maintain page tables outside the normal Linux page tables, which reference userspace memory. This includes Infiniband, other RDMA-capable devices, and kvm (with a pending patch).
Because these systems maintain external page tables (and external tlbs), Linux cannot demand page this memory and it must be locked. For kvm at least, this is a significant reduction in functionality.
This sample patch adds a new mechanism, pte notifiers, that allows drivers to register an interest in a changes to ptes. Whenever Linux changes a pte, it will call a notifier to allow the driver to adjust the external page table and flush its tlb.

Quoted 7 lines

Note that only one notifier is implemented, ->clear(), but others should be similar.
pte notifiers are different from paravirt_ops: they extend the normal page tables rather than replace them; and they provide high-level information
such as the vma and the virtual address for the driver to use.

Looks great. So for kvm, all guest pages will be vma mapped? There are lock issues in kvm between kvm lock and page lock. Will shadow page table be still stored in page->private? If yes, the page->private must be cleaned before add_to_swap.

Thanks,
Shaohua

Avi Kivity

author

2007-09-06 08:38 UTC #4 lore raw

Quoted 21 lines · Shaohua Li

Quoted 19 lines · Avi Kivity
[resend due to bad alias expansion resulting in some recipients
being bogus]
Some hardware and software systems maintain page tables outside the normal Linux page tables, which reference userspace memory. This includes Infiniband, other RDMA-capable devices, and kvm (with a pending patch).
Because these systems maintain external page tables (and external tlbs), Linux cannot demand page this memory and it must be locked. For kvm at least, this is a significant reduction in functionality.
This sample patch adds a new mechanism, pte notifiers, that allows drivers to register an interest in a changes to ptes. Whenever Linux changes a pte, it will call a notifier to allow the driver to adjust the external page table and flush its tlb.
Note that only one notifier is implemented, ->clear(), but others should be similar.

Quoted 8 lines

Quoted 5 lines
pte notifiers are different from paravirt_ops: they extend the normal page tables rather than replace them; and they provide high-level information
such as the vma and the virtual address for the driver to use.
Looks great. So for kvm, all guest pages will be vma mapped? There are lock issues in kvm between kvm lock and page lock.

Yes, locking will be a headache.

Quoted 3 lines

Will shadow page table be still stored in page->private? If yes, the page->private must be cleaned before add_to_swap.

page->private can be in use by filesystems, so we will need to move rmap somewhere else.

Signature · 1 line

Any sufficiently difficult bug is indistinguishable from a feature.

Andi Kleen

2007-09-06 11:39 UTC #5 lore raw

In reply to Avi Kivity

Quoted 4 lines · Avi Kivity

pte notifiers are different from paravirt_ops: they extend the normal page tables rather than replace them; and they provide high-level information such as the vma and the virtual address for the driver to use.

Sounds like a locking horror to me. To do anything with page tables you need locks. Both for the kernel page tables and for your new tables.

What happens when people add all
things of complicated operations in these notifiers? That will likely happen and then everytime you change something in VM code they will break. This has the potential to increase the cost of maintaining VM code considerably, which would be a bad thing.

This is quite different from paravirt ops because low level pvops can typically run lockless by just doing some kind of hypercall directly. But that won't work for maintaining your custom page tables.

-Andi

Avi Kivity

author

2007-09-06 13:22 UTC #6 lore raw

[ugh, what happened to the cc-list?]

Quoted 10 lines · Andi Kleen

Quoted 4 lines · Avi Kivity
pte notifiers are different from paravirt_ops: they extend the normal page tables rather than replace them; and they provide high-level information such as the vma and the virtual address for the driver to use.
Sounds like a locking horror to me. To do anything with page tables you need locks. Both for the kernel page tables and for your new tables.

Quoted 9 lines

What happens when people add all
things of complicated operations in these notifiers? That will likely
happen and then everytime you change something in VM code they will break. This has the potential to increase the cost of maintaining VM code considerably, which would be a bad thing.
This is quite different from paravirt ops because low level pvops can typically run lockless by just doing some kind of hypercall directly. But that won't work for maintaining your custom page tables.

This is a real problem. I don't have a solution yet.

Obviously that needs to be addressed before something like this can go in; but as it's been done for the quadrics driver, presumably it is doable.

Signature · 1 line

Any sufficiently difficult bug is indistinguishable from a feature.

Avi Kivity

author

2007-09-06 15:17 UTC #7 lore raw

In reply to Andi Kleen

Quoted 10 lines · Andi Kleen

Quoted 4 lines · Avi Kivity
pte notifiers are different from paravirt_ops: they extend the normal page tables rather than replace them; and they provide high-level information such as the vma and the virtual address for the driver to use.
Sounds like a locking horror to me. To do anything with page tables you need locks. Both for the kernel page tables and for your new tables.

Quoted 9 lines

What happens when people add all
things of complicated operations in these notifiers? That will likely
happen and then everytime you change something in VM code they will break. This has the potential to increase the cost of maintaining VM code considerably, which would be a bad thing.
This is quite different from paravirt ops because low level pvops can typically run lockless by just doing some kind of hypercall directly. But that won't work for maintaining your custom page tables.

Okay, here's a possible fix: add ->lock() and ->unlock() callbacks, to be called when mmap_sem is taken either for read or write. Also add a ->release() for when the mm goes away to avoid the need to care about the entire data structure going away.

The notifier list would need to be kept sorted to avoid deadlocks.

Signature · 1 line

Any sufficiently difficult bug is indistinguishable from a feature.

Andi Kleen

2007-09-06 13:28 UTC #8 lore raw

In reply to Avi Kivity

Quoted 4 lines · Avi Kivity

pte notifiers are different from paravirt_ops: they extend the normal page tables rather than replace them; and they provide high-level information such as the vma and the virtual address for the driver to use.

Sounds like a locking horror to me. To do anything with page tables you need locks. Both for the kernel page tables and for your new tables.

-Andi

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>