p2m: use GNTTABOP_unmap_and_duplicate if available
Subsystems (3)
PATCH v3 1/1 p2m: use GNTTABOP_unmap_and_duplicate if available
The new GNTTABOP_unmap_and_duplicate operation doesn't zero the mapping passed in new_addr, allowing us to perform batch unmaps in p2m code without requiring the use of a multicall.
---
Changes since v2:
* Rebase on top of 3.12-rc7
Changes since v1:
* Append check_duplicate_mfn at the end of remove_page_override.
Changes since RFC:
* Move shared code between _single and _batch to helper functions.
---
I don't currently have a NFS server (the one I had is currently not working due to SD card corruption) and I cannot set up one right now, so I've only tested this with a raw image stored in a local disk.
--- arch/x86/include/asm/xen/page.h | 4 +- arch/x86/xen/p2m.c | 132 ++++++++++++++++++++++++++++++----- drivers/xen/grant-table.c | 24 ++---- include/xen/interface/grant_table.h | 22 ++++++ 4 files changed, 146 insertions(+), 36 deletions(-)
arch/x86/include/asm/xen/page.h +2 -2
@@ -51,8 +51,8 @@ extern unsigned long set_phys_range_identity(unsigned long pfn_s,extern int m2p_add_override (unsigned long mfn,struct page *page,struct gnttab_map_grant_ref *kmap_op); -extern int m2p_remove_override (struct page *page, -struct gnttab_map_grant_ref *kmap_op); +extern int m2p_remove_override (struct page **pages, +struct gnttab_map_grant_ref *kmap_ops,int count);extern struct page *m2p_find_override (unsigned long mfn);extern unsigned long m2p_find_override_pfn (unsigned long mfn,unsigned long pfn);
arch/x86/xen/p2m.c +114 -18
164 lines not on this page · raw message
drivers/xen/grant-table.c +8 -16
@@ -66,6 +66,7 @@ static int gnttab_free_count;static grant_ref_t gnttab_free_head;static DEFINE_SPINLOCK (gnttab_list_lock);unsigned long xen_hvm_resume_frames; +bool gnttab_supports_duplicate;EXPORT_SYMBOL_GPL (xen_hvm_resume_frames); static union {
@@ -935,8 +936,7 @@ int gnttab_unmap_refs(struct gnttab_unmap_grant_ref *unmap_ops,struct gnttab_map_grant_ref *kmap_ops, struct page **pages, unsigned int count) { - int i, ret; -bool lazy = false; + int ret; ret =HYPERVISOR_grant_table_op (GNTTABOP_unmap_grant_ref, unmap_ops, count);if (ret)
@@ -945,20 +945,7 @@ int gnttab_unmap_refs(struct gnttab_unmap_grant_ref *unmap_ops,if (xen_feature (XENFEAT_auto_translated_physmap))return ret; -if (!in_interrupt () &¶virt_get_lazy_mode () ==PARAVIRT_LAZY_NONE ) { -arch_enter_lazy_mmu_mode (); - lazy = true; - } - -for (i =0 ; i < count; i++) { - ret =m2p_remove_override (pages[i], kmap_ops ? - &kmap_ops[i] :NULL ); -if (ret) -return ret; - } - -if (lazy) -arch_leave_lazy_mmu_mode (); + ret =m2p_remove_override (pages, kmap_ops, count);return ret; }
@@ -1246,6 +1233,11 @@ int gnttab_init(void) gnttab_free_count = nr_init_grefs -NR_RESERVED_ENTRIES ; gnttab_free_head =NR_RESERVED_ENTRIES ; +if (HYPERVISOR_grant_table_op (GNTTABOP_unmap_and_duplicate,NULL ,0 )) + gnttab_supports_duplicate = false; +else + gnttab_supports_duplicate = true; +printk ("Grant table initialized\n" );return 0 ;
include/xen/interface/grant_table.h +22 -0
@@ -428,6 +428,27 @@ struct gnttab_unmap_and_replace {DEFINE_GUEST_HANDLE_STRUCT (gnttab_unmap_and_replace);/* +* GNTTABOP_unmap_and_duplicate: Destroy one or more grant-reference mappings +* tracked by <handle> but atomically replace the page table entry with one +* pointing to the machine address under <new_addr>. +* NOTES: +* 1. The call may fail in an undefined manner if either mapping is not +* tracked by <handle>. +* 2. After executing a batch of unmaps, it is guaranteed that no stale +* mappings will remain in the device or host TLBs. +*/ +#define GNTTABOP_unmap_and_duplicate 12 +struct gnttab_unmap_and_duplicate { +/* IN parameters. */ +uint64_t host_addr; +uint64_t new_addr; +grant_handle_t handle; +/* OUT parameters. */ +int16_t status;/* => enum grant_status */ +}; +DEFINE_GUEST_HANDLE_STRUCT (gnttab_unmap_and_duplicate_t); + +/* * GNTTABOP_set_version: Request a particular version of the grant * table shared table structure. This operation can only be performed * once in any given domain. It must be performed before any grants
@@ -522,6 +543,7 @@ DEFINE_GUEST_HANDLE_STRUCT(gnttab_get_version);#define GNTST_bad_copy_arg (-10)/* copy arguments cross page boundary. */ #define GNTST_address_too_big (-11)/* transfer page address too large. */ #define GNTST_eagain (-12)/* Operation not done; try again. */ +#define GNTST_enosys (-13)/* Operation not implemented. */ #define GNTTABOP_error_msgs { \ "okay", \
Signature · 1 line
1.7.7.5 (Apple Git-26)
Quoted 1 line · Roger Pau Monne
The new
GNTTABOP_unmap_and_duplicateoperation
I don't see this op in mainline Xen anywhere...
Was it part of Stefano's original swiotlb for ARM stuff? If so we've dropped that approach for ARM and the new solution doesn't require hypervisor changes so the upstreaming of this hypercall ended, so you'd have to pick it up and complete that work if you want to use it.
Ian.
Quoted 1 line · Roger Pau Monne
The new
GNTTABOP_unmap_and_duplicateoperation
I don't see this op in mainline Xen anywhere...
Was it part of Stefano's original swiotlb for ARM stuff? If so we've dropped that approach for ARM and the new solution doesn't require hypervisor changes so the upstreaming of this hypercall ended, so you'd have to pick it up and complete that work if you want to use it.
Ian.
Quoted 9 lines · Ian Campbell
Quoted 1 line · Roger Pau Monne
The new
GNTTABOP_unmap_and_duplicateoperationI don't see this op in mainline Xen anywhere...
Was it part of Stefano's original swiotlb for ARM stuff? If so we've dropped that approach for ARM and the new solution doesn't require hypervisor changes so the upstreaming of this hypercall ended, so you'd have to pick it up and complete that work if you want to use it.
It's not part of the ARM stuff, the patch to add that OP is here:
http://lists.xen.org/archives/html/xen-devel/2013-07/msg02825.html
I guess I will need to resend that also.
Quoted 9 lines · Ian Campbell
Quoted 1 line · Roger Pau Monne
The new
GNTTABOP_unmap_and_duplicateoperationI don't see this op in mainline Xen anywhere...
Was it part of Stefano's original swiotlb for ARM stuff? If so we've dropped that approach for ARM and the new solution doesn't require hypervisor changes so the upstreaming of this hypercall ended, so you'd have to pick it up and complete that work if you want to use it.
It's not part of the ARM stuff, the patch to add that OP is here:
http://lists.xen.org/archives/html/xen-devel/2013-07/msg02825.html
I guess I will need to resend that also.
Quoted 4 lines · Roger Pau Monné
Quoted 3 lines · Ian Campbell
Quoted 1 line · Roger Pau Monne
The new
GNTTABOP_unmap_and_duplicateoperation
Quoted 8 lines
Quoted 6 lines
I don't see this op in mainline Xen anywhere...
Was it part of Stefano's original swiotlb for ARM stuff? If so we've dropped that approach for ARM and the new solution doesn't require hypervisor changes so the upstreaming of this hypercall ended, so you'd have to pick it up and complete that work if you want to use it.
It's not part of the ARM stuff,
oops, my mistake, sorry.
Quoted 5 lines
the patch to add that OP is here:
http://lists.xen.org/archives/html/xen-devel/2013-07/msg02825.html
I guess I will need to resend that also.
Yes, I think you want to get the hcall locked down before you add anything to the kernel to use it...
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel
Quoted 4 lines · Roger Pau Monné
Quoted 3 lines · Ian Campbell
Quoted 1 line · Roger Pau Monne
The new
GNTTABOP_unmap_and_duplicateoperation
Quoted 8 lines
Quoted 6 lines
I don't see this op in mainline Xen anywhere...
Was it part of Stefano's original swiotlb for ARM stuff? If so we've dropped that approach for ARM and the new solution doesn't require hypervisor changes so the upstreaming of this hypercall ended, so you'd have to pick it up and complete that work if you want to use it.
It's not part of the ARM stuff,
oops, my mistake, sorry.
Quoted 5 lines
the patch to add that OP is here:
http://lists.xen.org/archives/html/xen-devel/2013-07/msg02825.html
I guess I will need to resend that also.
Yes, I think you want to get the hcall locked down before you add anything to the kernel to use it...
Quoted 3 lines · Roger Pau Monne
The new
GNTTABOP_unmap_and_duplicateoperation doesn't zero the mapping passed innew_addr, allowing us to perform batch unmaps in p2m code without requiring the use of a multicall.
Should the grant maps should be batched in a similar way?
Quoted 14 lines
arch/x86/xen/p2m.c +14
-0+int m2p_remove_override_batch (struct page **pages, +struct gnttab_map_grant_ref *kmap_ops,int count) +{ +struct gnttab_unmap_and_duplicate *unmap_ops =NULL ; +int ret =0 , i; + +for (i =0 ; i < count; i++) { + ret =remove_page_override (pages[i]); +if (ret) + goto out; + } + +if (kmap_ops !=NULL ) { +struct page *scratch_page =get_balloon_scratch_page ();
arch/x86/xen/p2m.c int m2p_remove_override(struct page *page, · line 1083
Quoted 10 lines
arch/x86/xen/p2m.c +10
-0+unsigned long scratch_page_address = (unsigned long ) +__va (page_to_pfn (scratch_page) <<PAGE_SHIFT ); +int invcount =0 ; + + unmap_ops =kcalloc (count,sizeof (unmap_ops[0 ]),GFP_KERNEL ); +if (!unmap_ops) { +put_balloon_scratch_page (); + ret = -ENOMEM ; + goto out; + }
A memory allocation failure here looks bad as the grants will not be unmapped which will have an impact on the granter domain. Either: a) the unmap ops should be constructed in advance (excluding high mem pages at that point); or b) this should use a statically allocated per-cpu array of unmap ops with the batch split as necessary to fit into the static array.
David
Quoted 3 lines · Roger Pau Monne
The new
GNTTABOP_unmap_and_duplicateoperation doesn't zero the mapping passed innew_addr, allowing us to perform batch unmaps in p2m code without requiring the use of a multicall.
Should the grant maps should be batched in a similar way?
Quoted 14 lines
arch/x86/xen/p2m.c +14
-0+int m2p_remove_override_batch (struct page **pages, +struct gnttab_map_grant_ref *kmap_ops,int count) +{ +struct gnttab_unmap_and_duplicate *unmap_ops =NULL ; +int ret =0 , i; + +for (i =0 ; i < count; i++) { + ret =remove_page_override (pages[i]); +if (ret) + goto out; + } + +if (kmap_ops !=NULL ) { +struct page *scratch_page =get_balloon_scratch_page ();
arch/x86/xen/p2m.c int m2p_remove_override(struct page *page, · line 1083
Quoted 10 lines
arch/x86/xen/p2m.c +10
-0+unsigned long scratch_page_address = (unsigned long ) +__va (page_to_pfn (scratch_page) <<PAGE_SHIFT ); +int invcount =0 ; + + unmap_ops =kcalloc (count,sizeof (unmap_ops[0 ]),GFP_KERNEL ); +if (!unmap_ops) { +put_balloon_scratch_page (); + ret = -ENOMEM ; + goto out; + }
A memory allocation failure here looks bad as the grants will not be unmapped which will have an impact on the granter domain. Either: a) the unmap ops should be constructed in advance (excluding high mem pages at that point); or b) this should use a statically allocated per-cpu array of unmap ops with the batch split as necessary to fit into the static array.
David
Quoted 3 lines · Roger Pau Monne
The new
GNTTABOP_unmap_and_duplicateoperation doesn't zero the mapping passed innew_addr, allowing us to perform batch unmaps in p2m code without requiring the use of a multicall.
I have recently investigated some problems that were caused by a user space process using gntdev. It was unmapping page that still had outstanding I/O. This caused a number of failures:
1. Oopses due to swiotlb_bounce() attempting to memcpy() back to a page that now has a read-only mapping to a scratch page MFN.
2. Bad page errors due to the balloon page being freed by gntdev while the page count > 1 and the balloon driver setting page count to 1 and freeing the page.
I think we need to take a step back and look at the design of the gntdev device to make it handle misbehaved or crashing programs.
In particular, I think we need to use regular (non-ballooned) pages and restore their original direct mappings when grant unmapping. My initial thoughts are that this would require a GNTTABOP_unmap_and_replace variant that takes a GFN direct instead of a finding the GFN via a virtual address.
I think it is best to hold off on any optimization attempts here until we get the gntdev design right.
David
Quoted 3 lines · Roger Pau Monne
The new
GNTTABOP_unmap_and_duplicateoperation doesn't zero the mapping passed innew_addr, allowing us to perform batch unmaps in p2m code without requiring the use of a multicall.
I have recently investigated some problems that were caused by a user space process using gntdev. It was unmapping page that still had outstanding I/O. This caused a number of failures:
1. Oopses due to swiotlb_bounce() attempting to memcpy() back to a page that now has a read-only mapping to a scratch page MFN.
2. Bad page errors due to the balloon page being freed by gntdev while the page count > 1 and the balloon driver setting page count to 1 and freeing the page.
I think we need to take a step back and look at the design of the gntdev device to make it handle misbehaved or crashing programs.
In particular, I think we need to use regular (non-ballooned) pages and restore their original direct mappings when grant unmapping. My initial thoughts are that this would require a GNTTABOP_unmap_and_replace variant that takes a GFN direct instead of a finding the GFN via a virtual address.
I think it is best to hold off on any optimization attempts here until we get the gntdev design right.
David