: pte notifiers -- support for external page tables

RFC PATCH In review Avi Kivity 7 participants 15 messages Sep 5, 2007 – Sep 11, 2007

Subsystems (5)

Memory management KVM MM core Memory mapping +1 more subsystems

lore

Share & tools

lore

Avi Kivity

author

2007-09-05 16:38 UTC #0 lore raw

RFC PATCH 1/1 : pte notifiers -- support for external page tables

Some hardware and software systems maintain page tables outside the normal Linux page tables, which reference userspace memory. This includes Infiniband, other RDMA-capable devices, and kvm (with a pending patch).

Because these systems maintain external page tables (and external tlbs), Linux cannot demand page this memory and it must be locked. For kvm at least, this is a significant reduction in functionality.

This sample patch adds a new mechanism, pte notifiers, that allows drivers to register an interest in a changes to ptes. Whenever Linux changes a pte, it will call a notifier to allow the driver to adjust the external page table and flush its tlb.

Note that only one notifier is implemented, ->clear(), but others should be similar.

pte notifiers are different from paravirt_ops: they extend the normal page tables rather than replace them; and they provide high-level information such as the vma and the virtual address for the driver to use.

Signed-off-by Avi Kivity <[email protected]>

include/linux/mm.h +4 -0

index 655094d..5d2bbee 100644
@@ -14,6 +14,7 @@
 #include <linux/debug_locks.h>
 #include <linux/backing-dev.h>
 #include <linux/mm_types.h>
+#include <linux/pte_notifier.h>
 
 struct mempolicy;
 struct anon_vma;

@@ -108,6 +109,9 @@ struct vm_area_struct {
 #ifndef CONFIG_MMU
 	atomic_t vm_usage;		/* refcount (VMAs shared if !MMU) */
 #endif
+#ifdef CONFIG_PTE_NOTIFIERS
+	struct list_head pte_notifier_list;
+#endif
 #ifdef CONFIG_NUMA
 	struct mempolicy *vm_policy;	/* NUMA policy for the VMA */
 #endif

include/linux/pte_notifier.h new file +52 -0

52 lines not on this page · raw message

mm/Kconfig +3 -0

index e24d348..7b10151 100644
@@ -176,3 +176,6 @@ config NR_QUICK
 config VIRT_TO_BUS
 	def_bool y
 	depends on !ARCH_NO_VIRT_TO_BUS
+
+config PTE_NOTIFIERS
+       bool

mm/Makefile +1 -0

index 245e33a..59f6a03 100644
@@ -29,4 +29,5 @@ obj-$(CONFIG_FS_XIP) += filemap_xip.o
 obj-$(CONFIG_MIGRATION) += migrate.o
 obj-$(CONFIG_SMP) += allocpercpu.o
 obj-$(CONFIG_QUICKLIST) += quicklist.o
+obj-$(CONFIG_PTE_NOTIFIERS) += pte_notifiers.o

mm/mmap.c +1 -0

index b653721..cc6c4fe 100644
@@ -1134,6 +1134,7 @@ munmap_back:
 	vma->vm_page_prot = protection_map[vm_flags &
 				(VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)];
 	vma->vm_pgoff = pgoff;
+	vma_init_pte_notifiers(vma);
 
 	if (file) {
 		error = -EINVAL;

mm/pte_notifier.c new file +32 -0

new file mode 100644
index 0000000..0b9076c
@@ -0,0 +1,32 @@
+
+#include <linux/pte_notifier.h>
+
+void vma_init_pte_notifiers(struct vm_area_struct *vma)
+{
+	INIT_LIST_HEAD(&vma->pte_notifier_list);
+}
+EXPORT_SYMBOL_GPL(vma_init_pte_notifiers);
+
+void vma_destroy_pte_notifiers(struct vm_area_struct *vma)
+{
+	struct pte_notifier *pn;
+	struct list_head *n;
+
+	list_for_each_entry_safe(pn, n, &vma->pte_notifier_list, link) {
+		pn->ops->close(__pn, vma);
+		__list_del(n);
+	}
+}
+
+void pte_notifier_register(struct pte_notifier *pn, struct vm_area_struct *vma)
+{
+	list_add(&pn->link, &vma->pte_notifier_list);
+}
+EXPORT_SYMBOL_GPL(pte_notifier_register);
+
+void pte_notifier_unregister(struct pte_notifier *pn)
+{
+	list_del(&pn->link);
+}
+EXPORT_SYMBOL_GPL(pte_notifier_unregister);
+

mm/rmap.c +1 -0

index 41ac397..3f61d38 100644
@@ -682,6 +682,7 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 	}
 
 	/* Nuke the page table entry. */
+	pte_notifier_call(vma, clear, address);
 	flush_cache_page(vma, address, page_to_pfn(page));
 	pteval = ptep_clear_flush(vma, address, pte);

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Rik van Riel

2007-09-05 19:05 UTC #1 lore raw

Quoted 7 lines · Avi Kivity

This sample patch adds a new mechanism, pte notifiers, that allows drivers to register an interest in a changes to ptes. Whenever Linux changes a pte, it will call a notifier to allow the driver to adjust the external page table and flush its tlb.
Note that only one notifier is implemented, ->clear(), but others should be similar.

This approach makes a lot of sense.

mm/rmap.c static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma, · line 687

Quoted 11 lines

mm/rmap.c +1 -0

index 41ac397..3f61d38 100644
@@ -682,6 +682,7 @@ static int try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 	}
 
 	/* Nuke the page table entry. */
+	pte_notifier_call(vma, clear, address);
 	flush_cache_page(vma, address, page_to_pfn(page));
 	pteval = ptep_clear_flush(vma, address, pte);

If you want this to be useful to Infiniband, you should probably also hook up do_wp_page() in mm/memory.c, where a page table can be pointed to another page.

Probably the code in mm/mremap.c will need to be hooked up too.

Signature · 9 lines

Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is.  Each group
calls the other unpatriotic.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Avi Kivity

author

2007-09-05 19:14 UTC #2 lore raw

Quoted 17 lines · Rik van Riel

mm/rmap.c static int try_to_unmap_one(struct page *page, · line 688
Quoted 12 lines
mm/rmap.c +1 -0
index 41ac397..3f61d38 100644
@@ -682,6 +682,7 @@ static int try_to_unmap_one(struct page *page, 
 struct vm_area_struct *vma,
     }
 
     /* Nuke the page table entry. */
+    pte_notifier_call(vma, clear, address);
     flush_cache_page(vma, address, page_to_pfn(page));
     pteval = ptep_clear_flush(vma, address, pte);
If you want this to be useful to Infiniband, you should probably also hook up do_wp_page() in mm/memory.c, where a page table can be pointed to another page.

Quoted 2 lines

Probably the code in mm/mremap.c will need to be hooked up too.

I imagine that many of the paravirt_ops mmu hooks will need to be exposed as pte notifiers. This can't be done as part of the paravirt_ops code due to the need to pass high level data structures, though.

Signature · 7 lines

Any sufficiently difficult bug is indistinguishable from a feature.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Rik van Riel

2007-09-05 19:23 UTC #3 lore raw

Quoted 19 lines · Avi Kivity

Quoted 17 lines · Rik van Riel
mm/rmap.c static int try_to_unmap_one(struct page *page, · line 688
Quoted 12 lines
mm/rmap.c +1 -0
index 41ac397..3f61d38 100644
@@ -682,6 +682,7 @@ static int try_to_unmap_one(struct page *page, 
 struct vm_area_struct *vma,
     }
 
     /* Nuke the page table entry. */
+    pte_notifier_call(vma, clear, address);
     flush_cache_page(vma, address, page_to_pfn(page));
     pteval = ptep_clear_flush(vma, address, pte);
If you want this to be useful to Infiniband, you should probably also hook up do_wp_page() in mm/memory.c, where a page table can be pointed to another page.

Quoted 4 lines

Quoted 2 lines
Probably the code in mm/mremap.c will need to be hooked up too.
I imagine that many of the paravirt_ops mmu hooks will need to be exposed as pte notifiers. This can't be done as part of the paravirt_ops code due to the need to pass high level data structures, though.

Wait, I thought that paravirt_ops was all on the side of the guest kernel, where these host kernel operations are invisible?

Signature · 9 lines

Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is.  Each group
calls the other unpatriotic.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Avi Kivity

author

2007-09-05 19:32 UTC #4 lore raw

Quoted 6 lines · Rik van Riel

Quoted 2 lines
I imagine that many of the paravirt_ops mmu hooks will need to be exposed as pte notifiers. This can't be done as part of the paravirt_ops code due to the need to pass high level data structures, though.
Wait, I thought that paravirt_ops was all on the side of the guest kernel, where these host kernel operations are invisible?

It is, but the hooks are in much the same places. It could be argued that you'd embed pte notifiers in paravirt_ops for a host kernel, but that's not doable because pte notifiers use higher-level data strutures (like vmas).

Signature · 1 line

Any sufficiently difficult bug is indistinguishable from a feature.

Jeremy Fitzhardinge

2007-09-06 11:28 UTC #5 lore raw

Quoted 4 lines · Avi Kivity

It is, but the hooks are in much the same places. It could be argued that you'd embed pte notifiers in paravirt_ops for a host kernel, but that's not doable because pte notifiers use higher-level data strutures (like vmas).

Also, I wouldn't like to preclude the possibility of having a kernel that's both a guest and a host (ie, nested vmms).

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/

Jack Steiner

2007-09-05 20:40 UTC #6 lore raw

In reply to Avi Kivity

Quoted 4 lines · Avi Kivity

Some hardware and software systems maintain page tables outside the normal Linux page tables, which reference userspace memory. This includes Infiniband, other RDMA-capable devices, and kvm (with a pending patch).

I like it.

We have 2 special devices with external TLBs that can take advantage of this.

One suggestion - at least for what we need. Can the notifier be registered against the mm_struct instead of (or in addition to) the vma?

---jack

Avi Kivity

author

2007-09-05 20:40 UTC #7 lore raw

Quoted 8 lines · Jack Steiner

Quoted 5 lines · Avi Kivity
Some hardware and software systems maintain page tables outside the normal Linux page tables, which reference userspace memory. This includes Infiniband, other RDMA-capable devices, and kvm (with a pending patch).

Quoted 9 lines

I like it.
We have 2 special devices with external TLBs that can take advantage of this.
One suggestion - at least for what we need. Can the notifier be registered against the mm_struct instead of (or in addition to) the vma?

Yes. It's a lot simpler since this way we don't have to support vma creation/splitting/merging/destruction. There's a tiny performance hit for kvm, but it isn't worth the bother.

Will implement for v2 of this patch.

Signature · 7 lines

Any sufficiently difficult bug is indistinguishable from a feature.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Avi Kivity

author

2007-09-05 20:42 UTC #8 lore raw

In reply to Jack Steiner

[resend due to broken cc list in my original post]

Quoted 8 lines · Jack Steiner

Quoted 5 lines · Avi Kivity
Some hardware and software systems maintain page tables outside the normal Linux page tables, which reference userspace memory. This includes Infiniband, other RDMA-capable devices, and kvm (with a pending patch).

Quoted 9 lines

I like it.
We have 2 special devices with external TLBs that can take advantage of this.
One suggestion - at least for what we need. Can the notifier be registered against the mm_struct instead of (or in addition to) the vma?

Yes. It's a lot simpler since this way we don't have to support vma creation/splitting/merging/destruction. There's a tiny performance hit for kvm, but it isn't worth the bother.

Will implement for v2 of this patch.

Signature · 1 line

Any sufficiently difficult bug is indistinguishable from a feature.

Gleb Natapov

2007-09-06 06:24 UTC #9 lore raw

In reply to Avi Kivity

Quoted 4 lines · Avi Kivity

This sample patch adds a new mechanism, pte notifiers, that allows drivers to register an interest in a changes to ptes. Whenever Linux changes a pte, it will call a notifier to allow the driver to adjust the external page table and flush its tlb.

How is this different from http://lwn.net/Articles/133627/? AFAIR the patch was rejected because there was only one user for it and it was decided that it would be better to maintain it out of tree for a while.

--
Gleb.

Avi Kivity

author

2007-09-06 08:35 UTC #10 lore raw

Quoted 11 lines · Gleb Natapov

Quoted 5 lines · Avi Kivity
This sample patch adds a new mechanism, pte notifiers, that allows drivers to register an interest in a changes to ptes. Whenever Linux changes a pte, it will call a notifier to allow the driver to adjust the external page table and flush its tlb.
How is this different from http://lwn.net/Articles/133627/? AFAIR the patch was rejected because there was only one user for it and it was decided that it would be better to maintain it out of tree for a while.

Your patch is more complete.

There are now at least three users: you, kvm, and newer Infiniband HCAs. Care to resurrect the patch?

Signature · 7 lines

Any sufficiently difficult bug is indistinguishable from a feature.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Gleb Natapov

2007-09-06 08:41 UTC #11 lore raw

Quoted 14 lines · Avi Kivity

Quoted 8 lines · Gleb Natapov
Quoted 6 lines · Avi Kivity
This sample patch adds a new mechanism, pte notifiers, that allows drivers
to register an interest in a changes to ptes. Whenever Linux changes a pte, it will call a notifier to allow the driver to adjust the external page table and flush its tlb.
Quoted 4 lines
How is this different from http://lwn.net/Articles/133627/? AFAIR the patch was rejected because there was only one user for it and it was decided that it would be better to maintain it out of tree for a while.

Quoted 5 lines

Your patch is more complete.
There are now at least three users: you, kvm, and newer Infiniband HCAs. Care to resurrect the patch?

This is not my patch :) This is patch written by David Addison from Quadrics. I CCed him on my previous email. I just saw that you are trying to do something similar.

--
Gleb.

Andrew Hastings

2007-09-10 18:17 UTC #12 lore raw

In reply to Avi Kivity

Quoted 8 lines · Avi Kivity

Quoted 7 lines · Gleb Natapov
Quoted 5 lines · Avi Kivity
This sample patch adds a new mechanism, pte notifiers, that allows drivers to register an interest in a changes to ptes. Whenever Linux changes a pte, it will call a notifier to allow the driver to adjust the external page table and flush its tlb.

Quoted 8 lines

Quoted 4 lines
How is this different from http://lwn.net/Articles/133627/? AFAIR the patch was rejected because there was only one user for it and it was decided that it would be better to maintain it out of tree for a while.
Your patch is more complete.
There are now at least three users: you, kvm, and newer Infiniband HCAs. Care to resurrect the patch?

We (Cray) also use the ioproc patch. AFAIK the current maintainer is Dan Blueman at Quadrics.

-Andrew Hastings
Cray Inc.

Daniel J Blueman

2007-09-11 10:37 UTC #13 lore raw

Quoted 14 lines · Andrew Hastings

Quoted 13 lines · Avi Kivity
Quoted 11 lines · Gleb Natapov
Quoted 5 lines · Avi Kivity
This sample patch adds a new mechanism, pte notifiers, that allows drivers to register an interest in a changes to ptes. Whenever Linux changes a pte, it will call a notifier to allow the driver to adjust the external page table and flush its tlb.
How is this different from http://lwn.net/Articles/133627/? AFAIR the patch was rejected because there was only one user for it and it was decided that it would be better to maintain it out of tree for a while.

Quoted 5 lines

Quoted 3 lines
Your patch is more complete.
There are now at least three users: you, kvm, and newer Infiniband HCAs. Care to resurrect the patch?
We (Cray) also use the ioproc patch. AFAIK the current maintainer is Dan Blueman at Quadrics.

I should add that the IOPROC patches are maintained internally to loosely track mainline kernels; however, we do not generally release [1] these until they've passed quite a lot of validation (driven by customer demand mostly) on various configurations.

Quite a few large users/groups would benefit from this; the IOPROC patches have been stable for quite a while now, so are a good option.

If you have any feedback/suggestions that would help forward progress, I'm happy to hear and address them.

Thanks,
Daniel

--- [1]

http://www.quadrics.com/patches

Signature · 8 lines

Daniel J Blueman
Software Engineer, Quadrics Ltd

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

Gleb Natapov

2007-09-11 11:19 UTC #14 lore raw

Quoted 28 lines · Daniel J Blueman

Quoted 15 lines · Andrew Hastings
Quoted 14 lines · Avi Kivity
Quoted 12 lines · Gleb Natapov
Quoted 6 lines · Avi Kivity
This sample patch adds a new mechanism, pte notifiers, that allows drivers
to register an interest in a changes to ptes. Whenever Linux changes a pte, it will call a notifier to allow the driver to adjust the external page table and flush its tlb.
How is this different from http://lwn.net/Articles/133627/? AFAIR the patch was rejected because there was only one user for it and it was decided that it would be better to maintain it out of tree for a while.
Quoted 6 lines
Quoted 4 lines
Your patch is more complete.
There are now at least three users: you, kvm, and newer Infiniband HCAs. Care to resurrect the patch?
We (Cray) also use the ioproc patch. AFAIK the current maintainer is Dan Blueman at Quadrics.
I should add that the IOPROC patches are maintained internally to loosely track mainline kernels; however, we do not generally release [1] these until they've passed quite a lot of validation (driven by customer demand mostly) on various configurations.

Quoted 6 lines

Quite a few large users/groups would benefit from this; the IOPROC patches have been stable for quite a while now, so are a good option.
If you have any feedback/suggestions that would help forward progress, I'm happy to hear and address them.

Posting the patch against current kernel (-mm or mainline) here would be certainly helpful.

Thanks,

--
Gleb.