KVM: xen: update shared_info and vcpu_info handling
Subsystems (6)
Paul Durrant
maintainerv5 cover KVM: xen: update shared_info and vcpu_info handling
From: Paul Durrant <[email protected]>
The following part of the original cover letter still applies...
"Currently we treat the shared_info page as guest memory and the VMM informs KVM of its location using a GFN. However it is not guest memory as such; it's an overlay page. So we pointlessly invalidate and re-cache a mapping to the *same page* of memory every time the guest requests that shared_info be mapped into its address space. Let's avoid doing that by modifying the pfncache code to allow activation using a fixed userspace HVA as well as a GPA."
However, this version of the series has dropped the other changes to try to handle the default vcpu_info location directly in KVM. With all the corner cases, it was getting sufficiently complex the functionality is better off staying in the VMM. So, instead of that code, two new patches have been added:
"xen: allow vcpu_info to be mapped by fixed HVA" is analogous to the "xen: allow shared_info to be mapped by fixed HVA" patch that has been present from the original version of the series and simply provides an attribute to that vcpu_info can be mapped using a fixed userspace HVA, which is desirable when using one embedded in the shared_info page (since we similarly avoid pointless cache invalidations).
"selftests / xen: re-map vcpu_info using HVA rather than GPA" is just a small addition to the 'xen_shinfo_test' selftest to swizzle the vcpu_info mapping to show that there's no functional change.
Paul Durrant (10):
KVM: pfncache: add a map helper function
KVM: pfncache: add a mark-dirty helper
KVM: pfncache: add a helper to get the gpa
KVM: pfncache: base offset check on khva rather than gpa
KVM: pfncache: allow a cache to be activated with a fixed (userspace)
HVA
KVM: xen: allow shared_info to be mapped by fixed HVA
KVM: xen: allow vcpu_info to be mapped by fixed HVA
KVM: selftests / xen: map shared_info using HVA rather than GFN
KVM: selftests / xen: re-map vcpu_info using HVA rather than GPA
KVM: xen: advertize the KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA capability
Documentation/virt/kvm/api.rst | 53 +++++-- arch/x86/kvm/x86.c | 5 +- arch/x86/kvm/xen.c | 89 ++++++++---- include/linux/kvm_host.h | 43 ++++++ include/linux/kvm_types.h | 3 +- include/uapi/linux/kvm.h | 9 +- .../selftests/kvm/x86_64/xen_shinfo_test.c | 59 ++++++-- virt/kvm/pfncache.c | 129 +++++++++++++----- 8 files changed, 302 insertions(+), 88 deletions(-)
---
Cc: "H. Peter Anvin" <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: David Woodhouse <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: Sean Christopherson <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: [email protected]
Signature · 1 line
2.39.2
Paul Durrant
maintainerPATCH v5 1/10 KVM: pfncache: add a map helper function
From: Paul Durrant <[email protected]>
We have an unmap helper but mapping is open-coded. Arguably this is fine because mapping is done in only one place, hva_to_pfn_retry(), but adding the helper does make that function more readable.
No functional change intended.
---
Cc: Sean Christopherson <[email protected]>
Cc: David Woodhouse <[email protected]>
Cc: Paolo Bonzini <[email protected]>
--- virt/kvm/pfncache.c | 43 +++++++++++++++++++++++++------------------ 1 file changed, 25 insertions(+), 18 deletions(-)
virt/kvm/pfncache.c +25 -18
@@ -96,17 +96,28 @@ bool kvm_gpc_check(struct gfn_to_pfn_cache *gpc, unsigned long len) }EXPORT_SYMBOL_GPL (kvm_gpc_check); -static void gpc_unmap_khva (kvm_pfn_t pfn,void *khva) +static void *gpc_map (kvm_pfn_t pfn) +{ +if (pfn_valid (pfn)) +return kmap (pfn_to_page (pfn)); +#ifdef CONFIG_HAS_IOMEM + else + return memremap(pfn_to_hpa(pfn), PAGE_SIZE, MEMREMAP_WB); +#endif +} + +static void gpc_unmap (kvm_pfn_t pfn,void *khva) {/* Unmap the old pfn/page if it was mapped before. */ -if (!is_error_noslot_pfn (pfn) && khva) { -if (pfn_valid (pfn)) -kunmap (pfn_to_page (pfn)); +if (is_error_noslot_pfn (pfn) || !khva) +return ; + +if (pfn_valid (pfn)) +kunmap (pfn_to_page (pfn));#ifdef CONFIG_HAS_IOMEM - else -memunmap (khva ); + else +memunmap (khva );#endif - } }static inline bool mmu_notifier_retry_cache (struct kvm *kvm,unsigned long mmu_seq)
@@ -175,7 +186,7 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc)* the existing mapping and didn't create a new one. */ if (new_khva != old_khva) -gpc_unmap_khva (new_pfn, new_khva); +gpc_unmap (new_pfn, new_khva);kvm_release_pfn_clean (new_pfn);
@@ -193,15 +204,11 @@ static kvm_pfn_t hva_to_pfn_retry(struct gfn_to_pfn_cache *gpc)* too must be done outside of gpc->lock! */ if (gpc->usage &KVM_HOST_USES_PFN ) { -if (new_pfn == gpc->pfn) { +if (new_pfn == gpc->pfn) new_khva = old_khva; - }else if (pfn_valid (new_pfn)) { - new_khva =kmap (pfn_to_page (new_pfn)); -#ifdef CONFIG_HAS_IOMEM - }else { - new_khva =memremap (pfn_to_hpa (new_pfn),PAGE_SIZE ,MEMREMAP_WB ); -#endif - } +else + new_khva =gpc_map (new_pfn); +if (!new_khva) {kvm_release_pfn_clean (new_pfn); goto out_error;
@@ -326,7 +333,7 @@ static int __kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, gpa_t gpa,mutex_unlock (&gpc->refresh_lock);if (unmap_old) -gpc_unmap_khva (old_pfn, old_khva); +gpc_unmap (old_pfn, old_khva);return ret; }
@@ -412,7 +419,7 @@ void kvm_gpc_deactivate(struct gfn_to_pfn_cache *gpc)list_del (&gpc->list);spin_unlock (&kvm->gpc_lock); -gpc_unmap_khva (old_pfn, old_khva); +gpc_unmap (old_pfn, old_khva); } }EXPORT_SYMBOL_GPL (kvm_gpc_deactivate);
Signature · 1 line
2.39.2
Paul Durrant
maintainerPATCH v5 2/10 KVM: pfncache: add a mark-dirty helper
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
At the moment pages are marked dirty by open-coded calls to mark_page_dirty_in_slot(), directly deferefencing the gpa and memslot from the cache. After a subsequent patch these may not always be set so add a helper now so that caller will protected from the need to know about this detail.
NOTE: Pages are now marked dirty while the cache lock is held. This is
to ensure that gpa and memslot are mutually consistent.
---
Cc: David Woodhouse <[email protected]>
Cc: Sean Christopherson <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Cc: [email protected]
--- arch/x86/kvm/x86.c | 2 +- arch/x86/kvm/xen.c | 13 ++++++------- include/linux/kvm_host.h | 7 +++++++ virt/kvm/pfncache.c | 6 ++++++ 4 files changed, 20 insertions(+), 8 deletions(-)
arch/x86/kvm/x86.c +1 -1
@@ -3137,7 +3137,7 @@ static void kvm_setup_guest_pvclock(struct kvm_vcpu *v, guest_hv_clock->version = ++vcpu->hv_clock.version; -mark_page_dirty_in_slot (v->kvm, gpc->memslot, gpc->gpa >>PAGE_SHIFT ); +kvm_gpc_mark_dirty (gpc);read_unlock_irqrestore (&gpc->lock, flags);trace_kvm_pvclock_update (v->vcpu_id, &vcpu->hv_clock);
arch/x86/kvm/xen.c +6 -7
@@ -430,14 +430,13 @@ static void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, bool atomic)smp_wmb (); } -if (user_len2) +if (user_len2) { +kvm_gpc_mark_dirty (gpc2);read_unlock (&gpc2->lock); + } +kvm_gpc_mark_dirty (gpc1);read_unlock_irqrestore (&gpc1->lock, flags); - -mark_page_dirty_in_slot (v->kvm, gpc1->memslot, gpc1->gpa >>PAGE_SHIFT ); -if (user_len2) -mark_page_dirty_in_slot (v->kvm, gpc2->memslot, gpc2->gpa >>PAGE_SHIFT ); }void kvm_xen_update_runstate (struct kvm_vcpu *v,int state)
@@ -543,13 +542,13 @@ void kvm_xen_inject_pending_events(struct kvm_vcpu *v) : "0" (evtchn_pending_sel32));WRITE_ONCE (vi->evtchn_upcall_pending,1 ); } + +kvm_gpc_mark_dirty (gpc);read_unlock_irqrestore (&gpc->lock, flags);/* For the per-vCPU lapic vector, deliver it as MSI. */ if (v->arch.xen.upcall_vector)kvm_xen_inject_vcpu_vector (v); - -mark_page_dirty_in_slot (v->kvm, gpc->memslot, gpc->gpa >>PAGE_SHIFT ); }int __kvm_xen_has_interrupt (struct kvm_vcpu *v)
include/linux/kvm_host.h +7 -0
@@ -1367,6 +1367,13 @@ int kvm_gpc_refresh(struct gfn_to_pfn_cache *gpc, unsigned long len);*/ void kvm_gpc_deactivate (struct gfn_to_pfn_cache *gpc); +/** +* kvm_gpc_mark_dirty - mark a cached page as dirty. +* +* @gpc: struct gfn_to_pfn_cache object. +*/ +void kvm_gpc_mark_dirty (struct gfn_to_pfn_cache *gpc); +void kvm_sigset_activate (struct kvm_vcpu *vcpu);void kvm_sigset_deactivate (struct kvm_vcpu *vcpu);
virt/kvm/pfncache.c +6 -0
@@ -386,6 +386,12 @@ int kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned long len) }EXPORT_SYMBOL_GPL (kvm_gpc_activate); +void kvm_gpc_mark_dirty (struct gfn_to_pfn_cache *gpc) +{ +mark_page_dirty_in_slot (gpc->kvm, gpc->memslot, gpc->gpa >>PAGE_SHIFT ); +} +EXPORT_SYMBOL_GPL (kvm_gpc_mark_dirty); +void kvm_gpc_deactivate (struct gfn_to_pfn_cache *gpc) {struct kvm *kvm = gpc->kvm;
Signature · 1 line
2.39.2
Paul Durrant
maintainerPATCH v5 3/10 KVM: pfncache: add a helper to get the gpa
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
A subsequent patch will rename this field since it will become overloaded. To avoid churn in places that currently retrieve the gpa, add a helper for that purpose now.
No functional change intended.
---
Cc: David Woodhouse <[email protected]>
Cc: Sean Christopherson <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Cc: [email protected]
--- arch/x86/kvm/xen.c | 15 ++++++++------- include/linux/kvm_host.h | 7 +++++++ virt/kvm/pfncache.c | 6 ++++++ 3 files changed, 21 insertions(+), 7 deletions(-)
arch/x86/kvm/xen.c +8 -7
@@ -261,8 +261,8 @@ static void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, bool atomic)* alignment (and the 32-bit ABI doesn't align the 64-bit integers * anyway, even if the overall struct had been 64-bit aligned). */ -if ((gpc1->gpa & ~PAGE_MASK ) + user_len >=PAGE_SIZE ) { - user_len1 =PAGE_SIZE - (gpc1->gpa & ~PAGE_MASK ); +if ((kvm_gpc_gpa (gpc1) & ~PAGE_MASK ) + user_len >=PAGE_SIZE ) { + user_len1 =PAGE_SIZE - (kvm_gpc_gpa (gpc1) & ~PAGE_MASK ); user_len2 = user_len - user_len1; }else { user_len1 = user_len;
@@ -343,7 +343,7 @@ static void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, bool atomic)* to the second page now because the guest changed to * 64-bit mode, the second GPC won't have been set up. */ -if (kvm_gpc_activate (gpc2, gpc1->gpa + user_len1, +if (kvm_gpc_activate (gpc2,kvm_gpc_gpa (gpc1) + user_len1, user_len2))return ;
@@ -677,7 +677,8 @@ int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data)case KVM_XEN_ATTR_TYPE_SHARED_INFO :if (kvm->arch.xen.shinfo_cache.active) - data->u.shared_info.gfn =gpa_to_gfn (kvm->arch.xen.shinfo_cache.gpa); + data->u.shared_info.gfn = +gpa_to_gfn (kvm_gpc_gpa (&kvm->arch.xen.shinfo_cache));else data->u.shared_info.gfn =KVM_XEN_INVALID_GFN ; r =0 ;
@@ -955,7 +956,7 @@ int kvm_xen_vcpu_get_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data)switch (data->type) {case KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO :if (vcpu->arch.xen.vcpu_info_cache.active) - data->u.gpa = vcpu->arch.xen.vcpu_info_cache.gpa; + data->u.gpa =kvm_gpc_gpa (&vcpu->arch.xen.vcpu_info_cache);else data->u.gpa =KVM_XEN_INVALID_GPA ; r =0 ;
@@ -963,7 +964,7 @@ int kvm_xen_vcpu_get_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data)case KVM_XEN_VCPU_ATTR_TYPE_VCPU_TIME_INFO :if (vcpu->arch.xen.vcpu_time_info_cache.active) - data->u.gpa = vcpu->arch.xen.vcpu_time_info_cache.gpa; + data->u.gpa =kvm_gpc_gpa (&vcpu->arch.xen.vcpu_time_info_cache);else data->u.gpa =KVM_XEN_INVALID_GPA ; r =0 ;
@@ -975,7 +976,7 @@ int kvm_xen_vcpu_get_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data)break ; }if (vcpu->arch.xen.runstate_cache.active) { - data->u.gpa = vcpu->arch.xen.runstate_cache.gpa; + data->u.gpa =kvm_gpc_gpa (&vcpu->arch.xen.runstate_cache); r =0 ; }break ;
include/linux/kvm_host.h +7 -0
@@ -1374,6 +1374,13 @@ void kvm_gpc_deactivate(struct gfn_to_pfn_cache *gpc);*/ void kvm_gpc_mark_dirty (struct gfn_to_pfn_cache *gpc); +/** +* kvm_gpc_gpa - retrieve the guest physical address of a cached mapping +* +* @gpc: struct gfn_to_pfn_cache object. +*/ +gpa_t kvm_gpc_gpa (struct gfn_to_pfn_cache *gpc); +void kvm_sigset_activate (struct kvm_vcpu *vcpu);void kvm_sigset_deactivate (struct kvm_vcpu *vcpu);
virt/kvm/pfncache.c +6 -0
@@ -386,6 +386,12 @@ int kvm_gpc_activate(struct gfn_to_pfn_cache *gpc, gpa_t gpa, unsigned long len) }EXPORT_SYMBOL_GPL (kvm_gpc_activate); +gpa_t kvm_gpc_gpa (struct gfn_to_pfn_cache *gpc) +{ +return gpc->gpa; +} +EXPORT_SYMBOL_GPL (kvm_gpc_gpa); +void kvm_gpc_mark_dirty (struct gfn_to_pfn_cache *gpc) {mark_page_dirty_in_slot (gpc->kvm, gpc->memslot, gpc->gpa >>PAGE_SHIFT );
Signature · 1 line
2.39.2
Paul Durrant
maintainerPATCH v5 4/10 KVM: pfncache: base offset check on khva rather than gpa
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
After a subsequent patch, the gpa may not always be set whereas khva will (as long as the cache valid flag is also set).
No functional change intended.
---
Cc: Sean Christopherson <[email protected]>
Cc: David Woodhouse <[email protected]>
Cc: Paolo Bonzini <[email protected]>
--- virt/kvm/pfncache.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
virt/kvm/pfncache.c +5 -2
@@ -83,15 +83,18 @@ bool kvm_gpc_check(struct gfn_to_pfn_cache *gpc, unsigned long len)if (!gpc->active)return false; -if ((gpc->gpa & ~PAGE_MASK ) + len >PAGE_SIZE ) +if (gpc->generation != slots->generation)return false; -if (gpc->generation != slots->generation ||kvm_is_error_hva (gpc->uhva)) +if (kvm_is_error_hva (gpc->uhva))return false;if (!gpc->valid)return false; +if (offset_in_page (gpc->khva) + len >PAGE_SIZE ) +return false; +return true; }EXPORT_SYMBOL_GPL (kvm_gpc_check);
Signature · 1 line
2.39.2
Paul Durrant
maintainerPATCH v5 5/10 KVM: pfncache: allow a cache to be activated with a fixed (userspace) HVA
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
Some cached pages may actually be overlays on guest memory that have a fixed HVA within the VMM. It's pointless to invalidate such cached mappings if the overlay is moved so allow a cache to be activated directly with the HVA to cater for such cases. A subsequent patch will make use of this facility.
---
Cc: Sean Christopherson <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: David Woodhouse <[email protected]>
--- include/linux/kvm_host.h | 29 ++++++++++++++++ include/linux/kvm_types.h | 3 +- virt/kvm/pfncache.c | 73 ++++++++++++++++++++++++++++----------- 3 files changed, 84 insertions(+), 21 deletions(-)
include/linux/kvm_host.h +29 -0
@@ -1321,6 +1321,22 @@ void kvm_gpc_init(struct gfn_to_pfn_cache *gpc, struct kvm *kvm,*/ int kvm_gpc_activate (struct gfn_to_pfn_cache *gpc,gpa_t gpa,unsigned long len); +/** +* kvm_gpc_activate_hva - prepare a cached kernel mapping and HPA for a given HVA. +* +* @gpc: struct gfn_to_pfn_cache object. +* @hva: userspace virtual address to map. +* @len: sanity check; the range being access must fit a single page. +* +* @return: 0 for success. +* -EINVAL for a mapping which would cross a page boundary. +* -EFAULT for an untranslatable guest physical address. +* +* The semantics of this function are the same as those of kvm_gpc_activate(). It +* merely bypasses a layer of address translation. +*/ +int kvm_gpc_activate_hva (struct gfn_to_pfn_cache *gpc,unsigned long hva,unsigned long len); +/** * kvm_gpc_check - check validity of a gfn_to_pfn_cache. *
@@ -1378,9 +1394,22 @@ void kvm_gpc_mark_dirty(struct gfn_to_pfn_cache *gpc);* kvm_gpc_gpa - retrieve the guest physical address of a cached mapping * * @gpc: struct gfn_to_pfn_cache object. +* +* @return: If the cache was activated with a fixed HVA then INVALID_GPA +* will be returned. */ gpa_t kvm_gpc_gpa (struct gfn_to_pfn_cache *gpc); +/** +* kvm_gpc_hva - retrieve the fixed host physical address of a cached mapping +* +* @gpc: struct gfn_to_pfn_cache object. +* +* @return: If the cache was activated with a guest physical address then +* 0 will be returned. +*/ +unsigned long kvm_gpc_hva (struct gfn_to_pfn_cache *gpc); +void kvm_sigset_activate (struct kvm_vcpu *vcpu);void kvm_sigset_deactivate (struct kvm_vcpu *vcpu);
include/linux/kvm_types.h +2 -1
@@ -64,7 +64,7 @@ struct gfn_to_hva_cache {struct gfn_to_pfn_cache {u64 generation; -gpa_t gpa; +u64 addr;unsigned long uhva;struct kvm_memory_slot *memslot;struct kvm *kvm;
@@ -77,6 +77,7 @@ struct gfn_to_pfn_cache {enum pfn_cache_usage usage;bool active;bool valid; +bool addr_is_gpa; };#ifdef KVM_ARCH_NR_OBJS_PER_MEMORY_CACHE
virt/kvm/pfncache.c +53 -20
133 lines not on this page · raw message
Signature · 1 line
2.39.2
Paul Durrant
maintainerPATCH v5 6/10 KVM: xen: allow shared_info to be mapped by fixed HVA
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
The shared_info page is not guest memory as such. It is a dedicated page allocated by the VMM and overlaid onto guest memory in a GFN chosen by the guest. The guest may even request that shared_info be moved from one GFN to another, but the HVA is never going to change. Thus it makes much more sense to map the shared_info page in kernel once using this fixed HVA. Hence add a new KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA attribute type for this purpose and a KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA flag to advertize its availability. Don't actually advertize it yet though. That will be done in a subsequent patch, which will also add tests for the new attribute type.
Also update the KVM API documentation with the new attribute and also fix it up to consistently refer to 'shared_info' (with the underscore).
NOTE: The change of the kvm_xen_hvm_attr shared_info from struct to union
is technically an ABI change but it's entirely compatible with existing users.
---
Cc: David Woodhouse <[email protected]>
Cc: Sean Christopherson <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Cc: [email protected]
v2:
- Define the new attribute and capability but don't advertize the capability yet.
- Add API documentation.
--- Documentation/virt/kvm/api.rst | 25 +++++++++++++++++++------ arch/x86/kvm/xen.c | 28 ++++++++++++++++++++++------ include/uapi/linux/kvm.h | 6 +++++- 3 files changed, 46 insertions(+), 13 deletions(-)
Documentation/virt/kvm/api.rst +19 -6
@@ -353,7 +353,7 @@ The bits in the dirty bitmap are cleared before the ioctl returns, unless KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 is enabled. For more information, see the description of the capability. -Note that the Xen shared info page, if configured, shall always be assumed +Note that the Xen shared_info page, if configured, shall always be assumed to be dirty. KVM will not explicitly mark it such.
@@ -5408,8 +5408,9 @@ KVM_PV_ASYNC_CLEANUP_PERFORM __u8 long_mode; __u8 vector; __u8 runstate_update_flag; - struct { + union { __u64 gfn; + __u64 hva; } shared_info; struct { __u32 send_port;
@@ -5437,10 +5438,10 @@ type values: KVM_XEN_ATTR_TYPE_LONG_MODE Sets the ABI mode of the VM to 32-bit or 64-bit (long mode). This - determines the layout of the shared info pages exposed to the VM. + determines the layout of the shared_info page exposed to the VM. KVM_XEN_ATTR_TYPE_SHARED_INFO - Sets the guest physical frame number at which the Xen "shared info" + Sets the guest physical frame number at which the Xen shared_info page resides. Note that although Xen places vcpu_info for the first 32 vCPUs in the shared_info page, KVM does not automatically do so and instead requires that KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO be used
@@ -5449,7 +5450,7 @@ KVM_XEN_ATTR_TYPE_SHARED_INFO not be aware of the Xen CPU id which is used as the index into the vcpu_info[] array, so may know the correct default location. - Note that the shared info page may be constantly written to by KVM; + Note that the shared_info page may be constantly written to by KVM; it contains the event channel bitmap used to deliver interrupts to a Xen guest, amongst other things. It is exempt from dirty tracking mechanisms — KVM will not explicitly mark the page as dirty each
@@ -5458,9 +5459,21 @@ KVM_XEN_ATTR_TYPE_SHARED_INFO any vCPU has been running or any event channel interrupts can be routed to the guest. - Setting the gfn to KVM_XEN_INVALID_GFN will disable the shared info + Setting the gfn to KVM_XEN_INVALID_GFN will disable the shared_info page. +KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA + If the KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA flag is also set in the + Xen capabilities, then this attribute may be used to set the + userspace address at which the shared_info page resides, which + will always be fixed in the VMM regardless of where it is mapped + in guest physical address space. This attribute should be used in + preference to KVM_XEN_ATTR_TYPE_SHARED_INFO as it avoids + unnecessary invalidation of an internal cache when the page is + re-mapped in guest physcial address space. + + Setting the hva to zero will disable the shared_info page. + KVM_XEN_ATTR_TYPE_UPCALL_VECTOR Sets the exception vector used to deliver Xen event channel upcalls. This is the HVM-wide vector injected directly by the hypervisor
arch/x86/kvm/xen.c +22 -6
@@ -34,24 +34,27 @@ static bool kvm_xen_hcall_evtchn_send(struct kvm_vcpu *vcpu, u64 param, u64 *r);DEFINE_STATIC_KEY_DEFERRED_FALSE (kvm_xen_enabled,HZ ); -static int kvm_xen_shared_info_init (struct kvm *kvm,gfn_t gfn) +static int kvm_xen_shared_info_init (struct kvm *kvm,u64 addr,bool addr_is_gfn) {struct gfn_to_pfn_cache *gpc = &kvm->arch.xen.shinfo_cache;struct pvclock_wall_clock *wc; -gpa_t gpa =gfn_to_gpa (gfn);u32 *wc_sec_hi;u32 wc_version;u64 wall_nsec;int ret =0 ;int idx =srcu_read_lock (&kvm->srcu); -if (gfn ==KVM_XEN_INVALID_GFN ) { +if ((addr_is_gfn && addr ==KVM_XEN_INVALID_GFN ) || + (!addr_is_gfn && addr ==0 )) {kvm_gpc_deactivate (gpc); goto out; } do { - ret = kvm_gpc_activate(gpc, gpa, PAGE_SIZE); + if (addr_is_gfn) + ret = kvm_gpc_activate(gpc, gfn_to_gpa(addr), PAGE_SIZE); + else + ret = kvm_gpc_activate_hva(gpc, addr, PAGE_SIZE); if (ret) goto out;
@@ -604,7 +607,6 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) {int r = -ENOENT ; -switch (data->type) {case KVM_XEN_ATTR_TYPE_LONG_MODE :if (!IS_ENABLED (CONFIG_64BIT ) && data->u.long_mode) {
@@ -619,7 +621,13 @@ int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data)case KVM_XEN_ATTR_TYPE_SHARED_INFO :mutex_lock (&kvm->arch.xen.xen_lock); - r =kvm_xen_shared_info_init (kvm, data->u.shared_info.gfn); + r =kvm_xen_shared_info_init (kvm, data->u.shared_info.gfn, true); +mutex_unlock (&kvm->arch.xen.xen_lock); +break ; + +case KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA : +mutex_lock (&kvm->arch.xen.xen_lock); + r =kvm_xen_shared_info_init (kvm, data->u.shared_info.hva, false);mutex_unlock (&kvm->arch.xen.xen_lock);break ;
@@ -684,6 +692,14 @@ int kvm_xen_hvm_get_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data) r =0 ;break ; +case KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA : +if (kvm->arch.xen.shinfo_cache.active) + data->u.shared_info.hva =kvm_gpc_hva (&kvm->arch.xen.shinfo_cache); +else + data->u.shared_info.hva =0 ; + r =0 ; +break ; +case KVM_XEN_ATTR_TYPE_UPCALL_VECTOR : data->u.vector = kvm->arch.xen.upcall_vector; r =0 ;
include/uapi/linux/kvm.h +5 -1
@@ -1282,6 +1282,7 @@ struct kvm_x86_mce {#define KVM_XEN_HVM_CONFIG_EVTCHN_2LEVEL (1 << 4)#define KVM_XEN_HVM_CONFIG_EVTCHN_SEND (1 << 5)#define KVM_XEN_HVM_CONFIG_RUNSTATE_UPDATE_FLAG (1 << 6) +#define KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA (1 << 7)struct kvm_xen_hvm_config {__u32 flags;
@@ -1793,9 +1794,10 @@ struct kvm_xen_hvm_attr {__u8 long_mode;__u8 vector;__u8 runstate_update_flag; -struct { +union {__u64 gfn;#define KVM_XEN_INVALID_GFN ((__u64)-1) +__u64 hva; } shared_info;struct {__u32 send_port;
@@ -1837,6 +1839,8 @@ struct kvm_xen_hvm_attr {#define KVM_XEN_ATTR_TYPE_XEN_VERSION 0x4/* Available with KVM_CAP_XEN_HVM / KVM_XEN_HVM_CONFIG_RUNSTATE_UPDATE_FLAG */ #define KVM_XEN_ATTR_TYPE_RUNSTATE_UPDATE_FLAG 0x5 +/* Available with KVM_CAP_XEN_HVM / KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA */ +#define KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA 0x6/* Per-vCPU Xen attributes */ #define KVM_XEN_VCPU_GET_ATTR _IOWR(KVMIO, 0xca, struct kvm_xen_vcpu_attr)
Signature · 1 line
2.39.2
David Woodhouse
maintainerRe: [PATCH v5 06/10] KVM: xen: allow shared_info to be mapped by fixed HVA
Quoted 7 lines · Paul Durrant
From: Paul Durrant <[email protected]>
The
shared_infopage is not guest memory as such. It is a dedicated page allocated by the VMM and overlaid onto guest memory in a GFN chosen by the guest. The guest may even request thatshared_infobe moved from one GFN to another, but the HVA is never going to change. Thus it makes much more sense to map theshared_infopage in kernel once using this fixed HVA.
The words "makes much more sense" are doing a *lot* of work there. :)
When heckling the cover letter in
https://lore.kernel.org/kvm/[email protected]/
I suggested that the explanation probably wants to make it into a commit message rather than just the cover letter which tends not to be preserved in the commit history. It's *this* commit which needs it, I think.
Re: [PATCH v5 06/10] KVM: xen: allow shared_info to be mapped by fixed HVA
Quoted 9 lines · David Woodhouse
Quoted 7 lines · Paul Durrant
From: Paul Durrant <[email protected]>
The
shared_infopage is not guest memory as such. It is a dedicated page allocated by the VMM and overlaid onto guest memory in a GFN chosen by the guest. The guest may even request thatshared_infobe moved from one GFN to another, but the HVA is never going to change. Thus it makes much more sense to map theshared_infopage in kernel once using this fixed HVA.
Quoted 8 lines
The words "makes much more sense" are doing a *lot* of work there. :)
When heckling the cover letter in
https://lore.kernel.org/kvm/[email protected]/
I suggested that the explanation probably wants to make it into a commit message rather than just the cover letter which tends not to be preserved in the commit history. It's *this* commit which needs it, I think.
Ok, I'll try to come up with some concise wording.
Paul
Paul Durrant
maintainerPATCH v5 7/10 KVM: xen: allow vcpu_info to be mapped by fixed HVA
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
If the guest does not explicitly set the GPA of vcpu_info structure in memory then, for guests with 32 vCPUs or fewer, the vcpu_info embedded in the shared_info page may be used. As described in a previous commit, the shared_info page is an overlay at a fixed HVA within the VMM, so in this case it also more optimal to activate the vcpu_info cache with a fixed HVA to avoid unnecessary invalidation if the guest memory layout is modified.
---
Cc: David Woodhouse <[email protected]>
Cc: Sean Christopherson <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Cc: [email protected]
v5:
- New in this version.
--- Documentation/virt/kvm/api.rst | 26 +++++++++++++++++++++----- arch/x86/kvm/xen.c | 33 +++++++++++++++++++++++++++------ include/uapi/linux/kvm.h | 3 +++ 3 files changed, 51 insertions(+), 11 deletions(-)
Documentation/virt/kvm/api.rst +21 -5
@@ -5444,11 +5444,12 @@ KVM_XEN_ATTR_TYPE_SHARED_INFO Sets the guest physical frame number at which the Xen shared_info page resides. Note that although Xen places vcpu_info for the first 32 vCPUs in the shared_info page, KVM does not automatically do so - and instead requires that KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO be used - explicitly even when the vcpu_info for a given vCPU resides at the - "default" location in the shared_info page. This is because KVM may - not be aware of the Xen CPU id which is used as the index into the - vcpu_info[] array, so may know the correct default location. + and instead requires that KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO or + KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO_HVA be used explicitly even when + the vcpu_info for a given vCPU resides at the "default" location + in the shared_info page. This is because KVM may not be aware of + the Xen CPU id which is used as the index into the vcpu_info[] + array, so may know the correct default location. Note that the shared_info page may be constantly written to by KVM; it contains the event channel bitmap used to deliver interrupts to
@@ -5570,6 +5571,21 @@ KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO on dirty logging. Setting the gpa to KVM_XEN_INVALID_GPA will disable the vcpu_info. +KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO_HVA + If the KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA flag is also set in the + Xen capabilities, then this attribute may be used to set the + userspace address of the vcpu_info for a given vCPU. It should + only be used when the vcpu_info resides at the "default" location + in the shared_info page. In this case it is safe to assume the + userspace address will not change, because the shared_info page is + an overlay on guest memory and remains at a fixed host address + regardless of where it is mapped in guest physical address space + and hence unnecessary invalidation of an internal cache may be + avoided if the guest memory layout is modified. + If the vcpu_info does not reside at the "default" location then + it is not guaranteed to remain at the same host address and + hence the aforementioned cache invalidation is required. + KVM_XEN_VCPU_ATTR_TYPE_VCPU_TIME_INFO Sets the guest physical address of an additional pvclock structure for a given vCPU. This is typically used for guest vsyscall support.
arch/x86/kvm/xen.c +27 -6
@@ -736,20 +736,33 @@ int kvm_xen_vcpu_set_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data)switch (data->type) {case KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO : +case KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO_HVA :/* No compat necessary here. */ BUILD_BUG_ON (sizeof (struct vcpu_info ) !=sizeof (struct compat_vcpu_info ));BUILD_BUG_ON (offsetof(struct vcpu_info , time) != offsetof(struct compat_vcpu_info , time)); -if (data->u.gpa ==KVM_XEN_INVALID_GPA ) { -kvm_gpc_deactivate (&vcpu->arch.xen.vcpu_info_cache); - r =0 ; -break ; +if (data->type ==KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO ) { +if (data->u.gpa ==KVM_XEN_INVALID_GPA ) { +kvm_gpc_deactivate (&vcpu->arch.xen.vcpu_info_cache); + r =0 ; +break ; + } + + r =kvm_gpc_activate (&vcpu->arch.xen.vcpu_info_cache, + data->u.gpa,sizeof (struct vcpu_info )); + }else { +if (data->u.hva ==0 ) { +kvm_gpc_deactivate (&vcpu->arch.xen.vcpu_info_cache); + r =0 ; +break ; + } + + r =kvm_gpc_activate_hva (&vcpu->arch.xen.vcpu_info_cache, + data->u.hva,sizeof (struct vcpu_info )); } - r =kvm_gpc_activate (&vcpu->arch.xen.vcpu_info_cache, - data->u.gpa,sizeof (struct vcpu_info ));if (!r)kvm_make_request (KVM_REQ_CLOCK_UPDATE , vcpu);
@@ -978,6 +991,14 @@ int kvm_xen_vcpu_get_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data) r =0 ;break ; +case KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO_HVA : +if (vcpu->arch.xen.vcpu_info_cache.active) + data->u.hva =kvm_gpc_hva (&vcpu->arch.xen.vcpu_info_cache); +else + data->u.hva =0 ; + r =0 ; +break ; +case KVM_XEN_VCPU_ATTR_TYPE_VCPU_TIME_INFO :if (vcpu->arch.xen.vcpu_time_info_cache.active) data->u.gpa =kvm_gpc_gpa (&vcpu->arch.xen.vcpu_time_info_cache);
include/uapi/linux/kvm.h +3 -0
@@ -1858,6 +1858,7 @@ struct kvm_xen_vcpu_attr {union {__u64 gpa;#define KVM_XEN_INVALID_GPA ((__u64)-1) +__u64 hva;__u64 pad[8 ];struct {__u64 state;
@@ -1888,6 +1889,8 @@ struct kvm_xen_vcpu_attr {#define KVM_XEN_VCPU_ATTR_TYPE_VCPU_ID 0x6#define KVM_XEN_VCPU_ATTR_TYPE_TIMER 0x7#define KVM_XEN_VCPU_ATTR_TYPE_UPCALL_VECTOR 0x8 +/* Available with KVM_CAP_XEN_HVM / KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA */ +#define KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO_HVA 0x9/* Secure Encrypted Virtualization command */ enum sev_cmd_id {
Signature · 1 line
2.39.2
David Woodhouse
maintainerRe: [PATCH v5 07/10] KVM: xen: allow vcpu_info to be mapped by fixed HVA
Quoted 11 lines · Paul Durrant
From: Paul Durrant <[email protected]>
If the guest does not explicitly set the GPA of
vcpu_infostructure in memory then, for guests with 32 vCPUs or fewer, thevcpu_infoembedded in theshared_infopage may be used. As described in a previous commit, theshared_infopage is an overlay at a fixed HVA within the VMM, so in this case it also more optimal to activate thevcpu_infocache with a fixed HVA to avoid unnecessary invalidation if the guest memory layout is modified.Signed-off-by Paul Durrant <[email protected]>
But it should *only* be defined as an HVA in the case where it's the one in the shinfo. Otherwise, it's defined by its GPA.
Which almost makes me want to see a sanity check that it precisely equals &shinfo->vcpu_info[vcpu->arch.xen.vcpu_id].
Which brings me back around the circle again to wonder why we don't just *default* to it.... you hate me, don't you?
Your previous set of patches did that, and it did end requiring that the VMM restore both VCPU_INFO and VCPU_ID for each vCPU *before* restoring the SHARED_INFO_HVA on resume, but wasn't that OK?
Re: [PATCH v5 07/10] KVM: xen: allow vcpu_info to be mapped by fixed HVA
Quoted 19 lines · David Woodhouse
Quoted 11 lines · Paul Durrant
From: Paul Durrant <[email protected]>
If the guest does not explicitly set the GPA of
vcpu_infostructure in memory then, for guests with 32 vCPUs or fewer, thevcpu_infoembedded in theshared_infopage may be used. As described in a previous commit, theshared_infopage is an overlay at a fixed HVA within the VMM, so in this case it also more optimal to activate thevcpu_infocache with a fixed HVA to avoid unnecessary invalidation if the guest memory layout is modified.Signed-off-by Paul Durrant <[email protected]>But it should *only* be defined as an HVA in the case where it's the one in the shinfo. Otherwise, it's defined by its GPA.
Which almost makes me want to see a sanity check that it precisely equals &shinfo->vcpu_info[vcpu->arch.xen.vcpu_id].
Quoted 7 lines
Which brings me back around the circle again to wonder why we don't just *default* to it.... you hate me, don't you?
Your previous set of patches did that, and it did end requiring that the VMM restore both
VCPU_INFOandVCPU_IDfor each vCPU *before* restoring theSHARED_INFO_HVAon resume, but wasn't that OK?
No. It was painful and overly complex, with too many corner cases that were making my brain hurt. Given the pain of the last week, leaving the default handling in the VMM is preferable. It means I don't need to impose rules about attribute ordering and hence get caught by needing
one thread to wait for another inside the VMM. It's better and more robust this way.
Paul
David Woodhouse
maintainerRe: [PATCH v5 07/10] KVM: xen: allow vcpu_info to be mapped by fixed HVA
Quoted 28 lines · Paul Durrant
Quoted 19 lines · David Woodhouse
Quoted 11 lines · Paul Durrant
From: Paul Durrant <[email protected]>
If the guest does not explicitly set the GPA of
vcpu_infostructure in memory then, for guests with 32 vCPUs or fewer, thevcpu_infoembedded in theshared_infopage may be used. As described in a previous commit, theshared_infopage is an overlay at a fixed HVA within the VMM, so in this case it also more optimal to activate thevcpu_infocache with a fixed HVA to avoid unnecessary invalidation if the guest memory layout is modified.Signed-off-by Paul Durrant <[email protected]>But it should *only* be defined as an HVA in the case where it's the one in the shinfo. Otherwise, it's defined by its GPA.
Which almost makes me want to see a sanity check that it precisely equals &shinfo->vcpu_info[vcpu->arch.xen.vcpu_id].
Quoted 7 lines
Which brings me back around the circle again to wonder why we don't just *default* to it.... you hate me, don't you?
Your previous set of patches did that, and it did end requiring that the VMM restore both
VCPU_INFOandVCPU_IDfor each vCPU *before* restoring theSHARED_INFO_HVAon resume, but wasn't that OK?
Quoted 6 lines
No. It was painful and overly complex, with too many corner cases that were making my brain hurt. Given the pain of the last week, leaving the default handling in the VMM is preferable. It means I don't need to impose rules about attribute ordering and hence get caught by needing one thread to wait for another inside the VMM. It's better and more robust this way.
Fair enough, I won't argue. That is fairly much the reason I ripped it out of João's original patch set in the first place :)
Re: [PATCH v5 07/10] KVM: xen: allow vcpu_info to be mapped by fixed HVA
In reply to Paul DurrantQuoted 11 lines · Paul Durrant
From: Paul Durrant <[email protected]>
If the guest does not explicitly set the GPA of
vcpu_infostructure in memory then, for guests with 32 vCPUs or fewer, thevcpu_infoembedded in theshared_infopage may be used. As described in a previous commit, theshared_infopage is an overlay at a fixed HVA within the VMM, so in this case it also more optimal to activate thevcpu_infocache with a fixed HVA to avoid unnecessary invalidation if the guest memory layout is modified.Signed-off-by Paul Durrant <[email protected]>
Paul Durrant
maintainerPATCH v5 8/10 KVM: selftests / xen: map shared_info using HVA rather than GFN
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
Using the HVA of the shared_info page is more efficient, so if the capability (KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA) is present use that method to do the mapping.
NOTE: Have the juggle_shinfo_state() thread map and unmap using both
GFN and HVA, to make sure the older mechanism is not broken.
---
Cc: Sean Christopherson <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: David Woodhouse <[email protected]>
v3:
- Re-work the juggle_shinfo_state() thread
v2:
- New in this version.
--- .../selftests/kvm/x86_64/xen_shinfo_test.c | 44 +++++++++++++++---- 1 file changed, 35 insertions(+), 9 deletions(-)
tools/testing/selftests/kvm/x86_64/xen_shinfo_test.c +35 -9
@@ -393,6 +393,7 @@ static int cmp_timespec(struct timespec *a, struct timespec *b)return 0 ; } +static struct shared_info *shinfo;static struct vcpu_info *vinfo;static struct kvm_vcpu *vcpu;
@@ -408,20 +409,38 @@ static void *juggle_shinfo_state(void *arg) {struct kvm_vm *vm = (struct kvm_vm *)arg; -struct kvm_xen_hvm_attr cache_activate = { +struct kvm_xen_hvm_attr cache_activate_gfn = { .type =KVM_XEN_ATTR_TYPE_SHARED_INFO , .u.shared_info.gfn =SHINFO_REGION_GPA /PAGE_SIZE }; -struct kvm_xen_hvm_attr cache_deactivate = { +struct kvm_xen_hvm_attr cache_deactivate_gfn = { .type =KVM_XEN_ATTR_TYPE_SHARED_INFO , .u.shared_info.gfn =KVM_XEN_INVALID_GFN }; +struct kvm_xen_hvm_attr cache_activate_hva = { + .type =KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA , + .u.shared_info.hva = (unsigned long )shinfo + }; + +struct kvm_xen_hvm_attr cache_deactivate_hva = { + .type =KVM_XEN_ATTR_TYPE_SHARED_INFO , + .u.shared_info.hva =0 + }; + +int xen_caps =kvm_check_cap (KVM_CAP_XEN_HVM ); +for (;;) { -__vm_ioctl (vm,KVM_XEN_HVM_SET_ATTR , &cache_activate); -__vm_ioctl (vm,KVM_XEN_HVM_SET_ATTR , &cache_deactivate); +__vm_ioctl (vm,KVM_XEN_HVM_SET_ATTR , &cache_activate_gfn);pthread_testcancel (); +__vm_ioctl (vm,KVM_XEN_HVM_SET_ATTR , &cache_deactivate_gfn); + +if (xen_caps &KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA ) { +__vm_ioctl (vm,KVM_XEN_HVM_SET_ATTR , &cache_activate_hva); +pthread_testcancel (); +__vm_ioctl (vm,KVM_XEN_HVM_SET_ATTR , &cache_deactivate_hva); + } }return NULL ;
@@ -446,6 +465,7 @@ int main(int argc, char *argv[])bool do_runstate_flag = !!(xen_caps &KVM_XEN_HVM_CONFIG_RUNSTATE_UPDATE_FLAG );bool do_eventfd_tests = !!(xen_caps &KVM_XEN_HVM_CONFIG_EVTCHN_2LEVEL );bool do_evtchn_tests = do_eventfd_tests && !!(xen_caps &KVM_XEN_HVM_CONFIG_EVTCHN_SEND ); +bool has_shinfo_hva = !!(xen_caps &KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA );clock_gettime (CLOCK_REALTIME , &min_ts);
@@ -456,7 +476,7 @@ int main(int argc, char *argv[]) SHINFO_REGION_GPA, SHINFO_REGION_SLOT, 3, 0);virt_map (vm,SHINFO_REGION_GVA ,SHINFO_REGION_GPA ,3 ); -struct shared_info *shinfo =addr_gpa2hva (vm,SHINFO_VADDR ); + shinfo =addr_gpa2hva (vm,SHINFO_VADDR );int zero_fd =open ("/dev/zero" ,O_RDONLY );TEST_ASSERT (zero_fd !=-1 ,"Failed to open /dev/zero" );
@@ -492,10 +512,16 @@ int main(int argc, char *argv[]) "Failed to read back RUNSTATE_UPDATE_FLAG attr"); } -struct kvm_xen_hvm_attr ha = { - .type =KVM_XEN_ATTR_TYPE_SHARED_INFO , - .u.shared_info.gfn =SHINFO_REGION_GPA /PAGE_SIZE , - }; +struct kvm_xen_hvm_attr ha = {}; + +if (has_shinfo_hva) { + ha.type =KVM_XEN_ATTR_TYPE_SHARED_INFO_HVA ; + ha.u.shared_info.hva = (unsigned long )shinfo; + }else { + ha.type =KVM_XEN_ATTR_TYPE_SHARED_INFO ; + ha.u.shared_info.gfn =SHINFO_ADDR /PAGE_SIZE ; + } +vm_ioctl (vm,KVM_XEN_HVM_SET_ATTR , &ha);/*
Signature · 1 line
2.39.2
Paul Durrant
maintainerPATCH v5 9/10 KVM: selftests / xen: re-map vcpu_info using HVA rather than GPA
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
If the relevant capability (KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA) is present then re-map vcpu_info using the HVA part way through the tests to make sure then there is no functional change.
---
Cc: Sean Christopherson <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: David Woodhouse <[email protected]>
v5:
- New in this version.
--- .../selftests/kvm/x86_64/xen_shinfo_test.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
tools/testing/selftests/kvm/x86_64/xen_shinfo_test.c +15 -0
@@ -66,6 +66,7 @@ enum {TEST_POLL_TIMEOUT ,TEST_POLL_MASKED ,TEST_POLL_WAKE , +SET_VCPU_INFO ,TEST_TIMER_PAST ,TEST_LOCKING_SEND_RACE , TEST_LOCKING_POLL_RACE,
@@ -325,6 +326,10 @@ static void guest_code(void)GUEST_SYNC (TEST_POLL_WAKE ); +/* Set the vcpu_info to point at exactly the place it already is to +* make sure the attribute is functional. */ +GUEST_SYNC (SET_VCPU_INFO ); +/* A timer wake an *unmasked* port which should wake us with an * actual interrupt, while we're polling on a different port. */ ports[0 ]++;
@@ -892,6 +897,16 @@ int main(int argc, char *argv[])alarm (1 );break ; +case SET_VCPU_INFO : +if (has_shinfo_hva) { +struct kvm_xen_vcpu_attr vih = { + .type =KVM_XEN_VCPU_ATTR_TYPE_VCPU_INFO_HVA , + .u.hva = (unsigned long )vinfo + }; +vcpu_ioctl (vcpu,KVM_XEN_VCPU_SET_ATTR , &vih); + } +break ; +case TEST_TIMER_PAST :TEST_ASSERT (!evtchn_irq_expected,"Expected event channel IRQ but it didn't happen" );
Signature · 1 line
2.39.2
Re: [PATCH v5 09/10] KVM: selftests / xen: re-map vcpu_info using HVA rather than GPA
Quoted 7 lines · Paul Durrant
From: Paul Durrant <[email protected]>
If the relevant capability (
KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA) is present then re-mapvcpu_infousing the HVA part way through the tests to make sure then there is no functional change.Signed-off-by Paul Durrant <[email protected]>
Paul Durrant
maintainerPATCH v5 10/10 KVM: xen: advertize the KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA capability
In reply to Paul DurrantFrom: Paul Durrant <[email protected]>
Now that all relevant kernel changes and selftests are in place, enable the new capability.
---
Cc: Sean Christopherson <[email protected]>
Cc: Paolo Bonzini <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Cc: David Woodhouse <[email protected]>
Cc: [email protected]
v2:
- New in this version.
--- arch/x86/kvm/x86.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/x86.c +2 -1
@@ -4531,7 +4531,8 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)KVM_XEN_HVM_CONFIG_INTERCEPT_HCALL |KVM_XEN_HVM_CONFIG_SHARED_INFO |KVM_XEN_HVM_CONFIG_EVTCHN_2LEVEL | -KVM_XEN_HVM_CONFIG_EVTCHN_SEND ; +KVM_XEN_HVM_CONFIG_EVTCHN_SEND | +KVM_XEN_HVM_CONFIG_SHARED_INFO_HVA ;if (sched_info_on ()) r |=KVM_XEN_HVM_CONFIG_RUNSTATE |KVM_XEN_HVM_CONFIG_RUNSTATE_UPDATE_FLAG ;
Signature · 1 line
2.39.2
David Woodhouse
maintainerQuoted 12 lines · Paul Durrant
From: Paul Durrant <[email protected]>
The following part of the original cover letter still applies...
"Currently we treat the
shared_infopage as guest memory and the VMM informs KVM of its location using a GFN. However it is not guest memory as such; it's an overlay page. So we pointlessly invalidate and re-cache a mapping to the *same page* of memory every time the guest requests thatshared_infobe mapped into its address space. Let's avoid doing that by modifying the pfncache code to allow activation using a fixed userspace HVA as well as a GPA."
Quoted 5 lines
However, this version of the series has dropped the other changes to try to handle the default
vcpu_infolocation directly in KVM. With all the corner cases, it was getting sufficiently complex the functionality is better off staying in the VMM. So, instead of that code, two new patches have been added:
I think there's key information missing from this cover letter (and since cover letters don't get preserved, it probably wants to end up in one of the commits too).
This isn't *just* an optimisation; it's not just that we're pointlessly invalidating and re-caching it. The problem is the time in *between* those two, because we don't have atomic memslot updates (qv).
If we have to break apart a large memslot which contains the shared_info GPA, then add back the two pieces and whatever we've overlaid in the middle which broke it in two... there are long periods of time when an interrupt might arrive and the shared_info GPA might just be *absent*.
Using the HVA for the shinfo page makes a whole bunch of sense since it's kind of supposed to be a xenheap page anyway and not defined by the guest address it may — or may NOT — be mapped at. But more to the point, using the HVA means that the kernel can continue to deliver event channel interrupts (e.g. timer virqs, MSI pirqs from passthrough devices, etc.) to it even when it *isn't* mapped.
We don't have the same problem for the vcpu_info because there's a per- vcpu *shadow* of evtchn_pending_sel for that very purpose, which the vCPU itself will OR into the real vcpu_info on the way into guest mode.
So since we have to stop all vCPUs before changing the memslots anyway, the events can gather in that evtchn_pending_sel and it all works out OK.
It would still be nice to have a way of atomically sticking an overlay page over the middle of an existing memslot and breaking it apart, but we can live without it. And even if we *did* get that, what you're doing here makes a lot of sense anyway.