KVM: x86: hyper-v: Fine-grained TLB flush + L2 TLB flush features
Subsystems (8)
Vitaly Kuznetsov
maintainerv11 cover KVM: x86: hyper-v: Fine-grained TLB flush + L2 TLB flush features
Changes since v10 (Sean):
- New patches added:
- "x86/hyperv: Move VMCB enlightenment definitions to hyperv-tlfs.h"
- "KVM: selftests: Move "struct hv_enlightenments" to x86_64/svm.h"
- "KVM: SVM: Add a proper field for Hyper-V VMCB enlightenments"
- 'x86/hyperv: KVM: Rename "hv_enlightenments" to "hv_vmcb_enlightenments"'
- 'KVM: VMX: Rename "vmx/evmcs.{ch}" to "vmx/hyperv.{ch}"'
- "KVM: x86: Move clearing of TLB_FLUSH_CURRENT to kvm_vcpu_flush_tlb_all()"
- "KVM: selftests: Drop helpers to read/write page table entries"
- "KVM: x86: Make kvm_hv_get_assist_page() return 0/-errno"
- Removed patches:
- "KVM: selftests: Export _vm_get_page_table_entry()"
- Main differences:
- Move Hyper-V TLB flushing out of kvm_service_local_tlb_flush_requests(). On SVM, Hyper-V TLB flush FIFO is emptied from svm_flush_tlb_current()
- Don't disable IRQs in hv_tlb_flush_enqueue().
- Don't call kvm_vcpu_flush_tlb_guest() from kvm_hv_vcpu_flush_tlb() but return -errno instead.
- Avoid unneded flushes in !EPT/!NPT cases.
- Optimize hv_is_vp_in_sparse_set().
- Move TLFS definitions to asm/hyperv-tlfs.h.
- Use u64 vals in Hyper-V PV TLB flush selftest + multiple smaler changes
- Typos, indentation, renames, ...
Original description:
Currently, KVM handles HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} requests by flushing the whole VPID and this is sub-optimal. This series introduces the required mechanism to make handling of these requests more fine-grained by flushing individual GVAs only (when requested). On this foundation, "Direct Virtual Flush" Hyper-V feature is implemented. The feature allows L0 to handle Hyper-V TLB flush hypercalls directly at L0 without the need to reflect the exit to L1. This has at least two
benefits: reflecting vmexit and the consequent vmenter are avoided + L0 has precise information whether the target vCPU is actually running (and thus requires a kick).
Sean Christopherson (7):
x86/hyperv: Move VMCB enlightenment definitions to hyperv-tlfs.h
KVM: selftests: Move "struct hv_enlightenments" to x86_64/svm.h
KVM: SVM: Add a proper field for Hyper-V VMCB enlightenments x86/hyperv: KVM: Rename "hv_enlightenments" to
"hv_vmcb_enlightenments"
KVM: x86: Move clearing of TLB_FLUSH_CURRENT to
kvm_vcpu_flush_tlb_all()
KVM: x86: hyper-v: Add helper to read hypercall data for array
KVM: selftests: Drop helpers to read/write page table entries
Vitaly Kuznetsov (39):
KVM: x86: Rename 'enable_direct_tlbflush' to 'enable_l2_tlb_flush'
KVM: VMX: Rename "vmx/evmcs.{ch}" to "vmx/hyperv.{ch}"
KVM: x86: hyper-v: Resurrect dedicated KVM_REQ_HV_TLB_FLUSH flag
KVM: x86: hyper-v: Introduce TLB flush fifo
KVM: x86: hyper-v: Handle HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} calls
gently
KVM: x86: hyper-v: Expose support for extended gva ranges for flush
hypercalls
KVM: x86: Prepare kvm_hv_flush_tlb() to handle L2's GPAs x86/hyperv: Introduce
HV_MAX_SPARSE_VCPU_BANKS/HV_VCPUS_PER_SPARSE_BANK constants
KVM: x86: hyper-v: Use
HV_MAX_SPARSE_VCPU_BANKS/HV_VCPUS_PER_SPARSE_BANK instead of raw '64'
KVM: x86: hyper-v: Don't use sparse_set_to_vcpu_mask() in
kvm_hv_send_ipi()
KVM: x86: hyper-v: Create a separate fifo for L2 TLB flush
KVM: x86: hyper-v: Use preallocated buffer in 'struct kvm_vcpu_hv'
instead of on-stack 'sparse_banks'
KVM: nVMX: Keep track of hv_vm_id/hv_vp_id when eVMCS is in use
KVM: nSVM: Keep track of Hyper-V hv_vm_id/hv_vp_id
KVM: x86: Introduce .hv_inject_synthetic_vmexit_post_tlb_flush()
nested hook
KVM: x86: hyper-v: Introduce kvm_hv_is_tlb_flush_hcall()
KVM: x86: hyper-v: L2 TLB flush
KVM: x86: hyper-v: Introduce fast guest_hv_cpuid_has_l2_tlb_flush()
check
KVM: nVMX: hyper-v: Cache VP assist page in 'struct kvm_vcpu_hv'
KVM: nVMX: hyper-v: Enable L2 TLB flush
KVM: x86: Make kvm_hv_get_assist_page() return 0/-errno
KVM: nSVM: hyper-v: Enable L2 TLB flush
KVM: x86: Expose Hyper-V L2 TLB flush feature
KVM: selftests: Better XMM read/write helpers
KVM: selftests: Move HYPERV_LINUX_OS_ID definition to a common header
KVM: selftests: Move the function doing Hyper-V hypercall to a common
header
KVM: selftests: Hyper-V PV IPI selftest
KVM: selftests: Fill in vm->vpages_mapped bitmap in virt_map() too
KVM: selftests: Export vm_vaddr_unused_gap() to make it possible to
request unmapped ranges
KVM: selftests: Hyper-V PV TLB flush selftest
KVM: selftests: Sync 'struct hv_enlightened_vmcs' definition with
hyperv-tlfs.h
KVM: selftests: Sync 'struct hv_vp_assist_page' definition with
hyperv-tlfs.h
KVM: selftests: Move Hyper-V VP assist page enablement out of evmcs.h
KVM: selftests: Split off load_evmcs() from load_vmcs()
KVM: selftests: Create a vendor independent helper to allocate Hyper-V
specific test pages
KVM: selftests: Allocate Hyper-V partition assist page
KVM: selftests: evmcs_test: Introduce L2 TLB flush test
KVM: selftests: hyperv_svm_test: Introduce L2 TLB flush test
KVM: selftests: Rename 'evmcs_test' to 'hyperv_evmcs'
arch/x86/include/asm/hyperv-tlfs.h | 37 +
arch/x86/include/asm/kvm-x86-ops.h | 2 +-
arch/x86/include/asm/kvm_host.h | 42 +-
arch/x86/include/asm/svm.h | 7 +-
arch/x86/kvm/Makefile | 5 +-
arch/x86/kvm/hyperv.c | 335 +++++++--
arch/x86/kvm/hyperv.h | 64 +-
arch/x86/kvm/svm/hyperv.c | 18 +
arch/x86/kvm/svm/hyperv.h | 50 +-
arch/x86/kvm/svm/nested.c | 49 +-
arch/x86/kvm/svm/svm.c | 7 +
arch/x86/kvm/svm/svm.h | 5 +-
arch/x86/kvm/svm/svm_onhyperv.c | 8 +-
arch/x86/kvm/svm/svm_onhyperv.h | 25 +-
arch/x86/kvm/trace.h | 21 +-
arch/x86/kvm/vmx/{evmcs.c => hyperv.c} | 45 +-
arch/x86/kvm/vmx/{evmcs.h => hyperv.h} | 12 +-
arch/x86/kvm/vmx/nested.c | 43 +-
arch/x86/kvm/vmx/vmx.c | 7 +-
arch/x86/kvm/vmx/vmx_ops.h | 2 +-
arch/x86/kvm/x86.c | 38 +-
arch/x86/kvm/x86.h | 1 +
include/asm-generic/hyperv-tlfs.h | 5 +
include/asm-generic/mshyperv.h | 11 +-
tools/testing/selftests/kvm/.gitignore | 4 +-
tools/testing/selftests/kvm/Makefile | 5 +-
.../selftests/kvm/include/kvm_util_base.h | 1 +
.../selftests/kvm/include/x86_64/evmcs.h | 48 +-
.../selftests/kvm/include/x86_64/hyperv.h | 102 +++
.../selftests/kvm/include/x86_64/processor.h | 76 +-
.../selftests/kvm/include/x86_64/svm.h | 26 +-
.../selftests/kvm/include/x86_64/vmx.h | 8 -
tools/testing/selftests/kvm/lib/kvm_util.c | 9 +-
.../testing/selftests/kvm/lib/x86_64/hyperv.c | 46 ++
.../selftests/kvm/lib/x86_64/processor.c | 21 +-
tools/testing/selftests/kvm/lib/x86_64/vmx.c | 45 +-
.../kvm/x86_64/emulator_error_test.c | 6 +-
.../x86_64/{evmcs_test.c => hyperv_evmcs.c} | 69 +-
.../selftests/kvm/x86_64/hyperv_features.c | 25 +-
.../testing/selftests/kvm/x86_64/hyperv_ipi.c | 330 +++++++++
.../selftests/kvm/x86_64/hyperv_svm_test.c | 86 ++-
.../selftests/kvm/x86_64/hyperv_tlb_flush.c | 689 ++++++++++++++++++
42 files changed, 2081 insertions(+), 354 deletions(-)
create mode 100644 arch/x86/kvm/svm/hyperv.c
rename arch/x86/kvm/vmx/{evmcs.c => hyperv.c} (95%)
rename arch/x86/kvm/vmx/{evmcs.h => hyperv.h} (95%)
create mode 100644 tools/testing/selftests/kvm/lib/x86_64/hyperv.c
rename tools/testing/selftests/kvm/x86_64/{evmcs_test.c => hyperv_evmcs.c} (73%)
create mode 100644 tools/testing/selftests/kvm/x86_64/hyperv_ipi.c
create mode 100644 tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.cSignature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 1/46 x86/hyperv: Move VMCB enlightenment definitions to hyperv-tlfs.h
From: Sean Christopherson <[email protected]>
Move Hyper-V's VMCB enlightenment definitions to the TLFS header; the definitions come directly from the TLFS[*], not from KVM.
No functional change intended.
[vitaly: rename VMCB_HV_ -> HV_VMCB_ to match the rest of hyperv-tlfs.h, keep svm/hyperv.h]
--- arch/x86/include/asm/hyperv-tlfs.h | 22 +++++++++++++++++++ arch/x86/kvm/svm/hyperv.h | 22 ------------------- arch/x86/kvm/svm/nested.c | 2 +- arch/x86/kvm/svm/svm_onhyperv.c | 2 +- arch/x86/kvm/svm/svm_onhyperv.h | 4 ++-- .../selftests/kvm/x86_64/hyperv_svm_test.c | 6 ++--- 6 files changed, 29 insertions(+), 29 deletions(-)
arch/x86/include/asm/hyperv-tlfs.h +22 -0
@@ -598,6 +598,28 @@ struct hv_enlightened_vmcs {#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL 0xFFFF +/* +* Hyper-V uses the software reserved 32 bytes in VMCB control area to expose +* SVM enlightenments to guests. +*/ +struct hv_enlightenments { + struct __packed hv_enlightenments_control { +u32 nested_flush_hypercall:1 ; +u32 msr_bitmap:1 ; +u32 enlightened_npt_tlb:1 ; +u32 reserved:29 ; + } __packed hv_enlightenments_control; +u32 hv_vp_id; +u64 hv_vm_id; +u64 partition_assist_page; +u64 reserved; +} __packed; + +/* +* Hyper-V uses the software reserved clean bit in VMCB. +*/ +#define HV_VMCB_NESTED_ENLIGHTENMENTS 31 +struct hv_partition_assist_pg {u32 tlb_lock_count; };
arch/x86/kvm/svm/hyperv.h +0 -22
@@ -10,26 +10,4 @@#include "../hyperv.h" -/* -* Hyper-V uses the software reserved 32 bytes in VMCB -* control area to expose SVM enlightenments to guests. -*/ -struct hv_enlightenments { - struct __packed hv_enlightenments_control { -u32 nested_flush_hypercall:1 ; -u32 msr_bitmap:1 ; -u32 enlightened_npt_tlb:1 ; -u32 reserved:29 ; - } __packed hv_enlightenments_control; -u32 hv_vp_id; -u64 hv_vm_id; -u64 partition_assist_page; -u64 reserved; -} __packed; - -/* -* Hyper-V uses the software reserved clean bit in VMCB -*/ -#define VMCB_HV_NESTED_ENLIGHTENMENTS VMCB_SW -#endif /* __ARCH_X86_KVM_SVM_HYPERV_H__ */
arch/x86/kvm/svm/nested.c +1 -1
@@ -194,7 +194,7 @@ static bool nested_svm_vmrun_msrpm(struct vcpu_svm *svm)if (!svm->nested.force_msr_bitmap_recalc &&kvm_hv_hypercall_enabled (&svm->vcpu) && hve->hv_enlightenments_control.msr_bitmap && - (svm->nested.ctl.clean &BIT (VMCB_HV_NESTED_ENLIGHTENMENTS ))) + (svm->nested.ctl.clean &BIT (HV_VMCB_NESTED_ENLIGHTENMENTS ))) goto set_msrpm_base_pa;if (!(vmcb12_is_intercept (&svm->nested.ctl,INTERCEPT_MSR_PROT )))
arch/x86/kvm/svm/svm_onhyperv.c +1 -1
@@ -32,7 +32,7 @@ int svm_hv_enable_direct_tlbflush(struct kvm_vcpu *vcpu) hve->hv_vm_id = (unsigned long )vcpu->kvm;if (!hve->hv_enlightenments_control.nested_flush_hypercall) { hve->hv_enlightenments_control.nested_flush_hypercall =1 ; -vmcb_mark_dirty (to_svm (vcpu)->vmcb,VMCB_HV_NESTED_ENLIGHTENMENTS ); +vmcb_mark_dirty (to_svm (vcpu)->vmcb,HV_VMCB_NESTED_ENLIGHTENMENTS ); }return 0 ;
arch/x86/kvm/svm/svm_onhyperv.h +2 -2
@@ -64,7 +64,7 @@ static inline void svm_hv_vmcb_dirty_nested_enlightenments( (struct hv_enlightenments *)vmcb->control.reserved_sw;if (hve->hv_enlightenments_control.msr_bitmap) -vmcb_mark_dirty (vmcb,VMCB_HV_NESTED_ENLIGHTENMENTS ); +vmcb_mark_dirty (vmcb,HV_VMCB_NESTED_ENLIGHTENMENTS ); } static inline void svm_hv_update_vp_id(struct vmcb *vmcb,
@@ -76,7 +76,7 @@ static inline void svm_hv_update_vp_id(struct vmcb *vmcb,if (hve->hv_vp_id != vp_index) { hve->hv_vp_id = vp_index; -vmcb_mark_dirty (vmcb,VMCB_HV_NESTED_ENLIGHTENMENTS ); +vmcb_mark_dirty (vmcb,HV_VMCB_NESTED_ENLIGHTENMENTS ); } }#else
tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c +3 -3
@@ -39,7 +39,7 @@ struct hv_enlightenments {/* * Hyper-V uses the software reserved clean bit in VMCB */ -#define VMCB_HV_NESTED_ENLIGHTENMENTS (1U << 31) +#define HV_VMCB_NESTED_ENLIGHTENMENTS (1U << 31)void l2_guest_code (void ) {
@@ -98,14 +98,14 @@ static void __attribute__((__flatten__)) guest_code(struct svm_test_data *svm)/* Intercept RDMSR 0xc0000101 without telling KVM about it */ set_bit (2 * (MSR_GS_BASE &0x1fff ), svm->msr +0x800 );/* Make sure HV_VMX_ENLIGHTENED_CLEAN_FIELD_MSR_BITMAP is set */ - vmcb->control.clean |=VMCB_HV_NESTED_ENLIGHTENMENTS ; + vmcb->control.clean |=HV_VMCB_NESTED_ENLIGHTENMENTS ;run_guest (vmcb, svm->vmcb_gpa);/* Make sure we don't see SVM_EXIT_MSR here so eMSR bitmap works */ GUEST_ASSERT (vmcb->control.exit_code ==SVM_EXIT_VMMCALL ); vmcb->save.rip +=3 ;/* vmcall */ /* Now tell KVM we've changed MSR-Bitmap */ - vmcb->control.clean &= ~VMCB_HV_NESTED_ENLIGHTENMENTS ; + vmcb->control.clean &= ~HV_VMCB_NESTED_ENLIGHTENMENTS ;run_guest (vmcb, svm->vmcb_gpa);GUEST_ASSERT (vmcb->control.exit_code ==SVM_EXIT_MSR ); vmcb->save.rip +=2 ;/* rdmsr */
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 2/46 KVM: selftests: Move "struct hv_enlightenments" to x86_64/svm.h
In reply to Vitaly KuznetsovFrom: Sean Christopherson <[email protected]>
Move Hyper-V's VMCB "struct hv_enlightenments" to the svm.h header so that the struct can be referenced in "struct vmcb_control_area". Alternatively, a dedicated header for SVM+Hyper-V could be added, a la x86_64/evmcs.h, but it doesn't appear that Hyper-V will end up needing a wholesale replacement for the VMCB.
No functional change intended.
--- .../testing/selftests/kvm/include/x86_64/svm.h | 17 +++++++++++++++++ .../selftests/kvm/x86_64/hyperv_svm_test.c | 18 ------------------ 2 files changed, 17 insertions(+), 18 deletions(-)
tools/testing/selftests/kvm/include/x86_64/svm.h +17 -0
@@ -58,6 +58,23 @@ enum { INTERCEPT_RDPRU, }; +struct hv_enlightenments { + struct __packed hv_enlightenments_control { +u32 nested_flush_hypercall:1 ; +u32 msr_bitmap:1 ; +u32 enlightened_npt_tlb:1 ; +u32 reserved:29 ; + } __packed hv_enlightenments_control; +u32 hv_vp_id; +u64 hv_vm_id; +u64 partition_assist_page; +u64 reserved; +} __packed; + +/* +* Hyper-V uses the software reserved clean bit in VMCB +*/ +#define HV_VMCB_NESTED_ENLIGHTENMENTS (1U << 31)struct __attribute__ ((__packed__ ))vmcb_control_area {u32 intercept_cr;
tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c +0 -18
@@ -23,24 +23,6 @@#define L2_GUEST_STACK_SIZE 256 -struct hv_enlightenments { - struct __packed hv_enlightenments_control { -u32 nested_flush_hypercall:1 ; -u32 msr_bitmap:1 ; -u32 enlightened_npt_tlb:1 ; -u32 reserved:29 ; - } __packed hv_enlightenments_control; -u32 hv_vp_id; -u64 hv_vm_id; -u64 partition_assist_page; -u64 reserved; -} __packed; - -/* -* Hyper-V uses the software reserved clean bit in VMCB -*/ -#define HV_VMCB_NESTED_ENLIGHTENMENTS (1U << 31) -void l2_guest_code (void ) {GUEST_SYNC (3 );
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 3/46 KVM: SVM: Add a proper field for Hyper-V VMCB enlightenments
In reply to Vitaly KuznetsovFrom: Sean Christopherson <[email protected]>
Add a union to provide hv_enlightenments side-by-side with the sw_reserved bytes that Hyper-V's enlightenments overlay. Casting sw_reserved everywhere is messy, confusing, and unnecessarily unsafe.
No functional change intended.
--- arch/x86/include/asm/svm.h | 7 ++++++- arch/x86/kvm/svm/nested.c | 9 ++++----- arch/x86/kvm/svm/svm.h | 5 ++++- arch/x86/kvm/svm/svm_onhyperv.c | 2 +- arch/x86/kvm/svm/svm_onhyperv.h | 15 +++++++-------- tools/testing/selftests/kvm/include/x86_64/svm.h | 5 ++++- .../selftests/kvm/x86_64/hyperv_svm_test.c | 3 +-- 7 files changed, 27 insertions(+), 19 deletions(-)
arch/x86/include/asm/svm.h +6 -1
@@ -5,6 +5,8 @@#include <uapi/asm/svm.h> #include <uapi/asm/kvm.h> +#include <asm/hyperv-tlfs.h> +/* * 32-bit intercept words in the VMCB Control Area, starting * at Byte offset 000h.
@@ -161,7 +163,10 @@ struct __attribute__ ((__packed__)) vmcb_control_area {* Offset 0x3e0, 32 bytes reserved * for use by hypervisor/software. */ -u8 reserved_sw[32 ]; +union { +struct hv_enlightenments hv_enlightenments; +u8 reserved_sw[32 ]; + }; };
arch/x86/kvm/svm/nested.c +4 -5
@@ -179,8 +179,7 @@ void recalc_intercepts(struct vcpu_svm *svm)*/ static bool nested_svm_vmrun_msrpm (struct vcpu_svm *svm) { -struct hv_enlightenments *hve = - (struct hv_enlightenments *)svm->nested.ctl.reserved_sw; +struct hv_enlightenments *hve = &svm->nested.ctl.hv_enlightenments;int i;/*
@@ -369,8 +368,8 @@ void __nested_copy_vmcb_control_to_cache(struct kvm_vcpu *vcpu,/* Hyper-V extensions (Enlightened VMCB) */ if (kvm_hv_hypercall_enabled (vcpu)) { to->clean = from->clean; -memcpy (to->reserved_sw, from->reserved_sw, -sizeof (struct hv_enlightenments )); +memcpy (&to->hv_enlightenments, &from->hv_enlightenments, +sizeof (to->hv_enlightenments)); } }
@@ -1479,7 +1478,7 @@ static void nested_copy_vmcb_cache_to_control(struct vmcb_control_area *dst, dst->virt_ext = from->virt_ext; dst->pause_filter_count = from->pause_filter_count; dst->pause_filter_thresh = from->pause_filter_thresh; -/* 'clean' and 'reserved_sw' are not changed by KVM */ +/* 'clean' and 'hv_enlightenments' are not changed by KVM */ } static int svm_get_nested_state(struct kvm_vcpu *vcpu,
arch/x86/kvm/svm/svm.h +4 -1
@@ -151,7 +151,10 @@ struct vmcb_ctrl_area_cached {u64 nested_cr3;u64 virt_ext;u32 clean; -u8 reserved_sw[32 ]; +union { +struct hv_enlightenments hv_enlightenments; +u8 reserved_sw[32 ]; + }; };struct svm_nested_state {
arch/x86/kvm/svm/svm_onhyperv.c +1 -1
@@ -26,7 +26,7 @@ int svm_hv_enable_direct_tlbflush(struct kvm_vcpu *vcpu)if (!*p_hv_pa_pg)return -ENOMEM ; - hve = (struct hv_enlightenments *)to_svm (vcpu)->vmcb->control.reserved_sw; + hve = &to_svm (vcpu)->vmcb->control.hv_enlightenments; hve->partition_assist_page =__pa (*p_hv_pa_pg); hve->hv_vm_id = (unsigned long )vcpu->kvm;
arch/x86/kvm/svm/svm_onhyperv.h +7 -8
@@ -17,8 +17,10 @@ int svm_hv_enable_direct_tlbflush(struct kvm_vcpu *vcpu);static inline void svm_hv_init_vmcb (struct vmcb *vmcb) { -struct hv_enlightenments *hve = - (struct hv_enlightenments *)vmcb->control.reserved_sw; +struct hv_enlightenments *hve = &vmcb->control.hv_enlightenments; + +BUILD_BUG_ON (sizeof (vmcb->control.hv_enlightenments) != +sizeof (vmcb->control.reserved_sw));if (npt_enabled && ms_hyperv.nested_features &HV_X64_NESTED_ENLIGHTENED_TLB )
@@ -60,18 +62,15 @@ static inline void svm_hv_vmcb_dirty_nested_enlightenments( struct kvm_vcpu *vcpu) {struct vmcb *vmcb =to_svm (vcpu)->vmcb; -struct hv_enlightenments *hve = - (struct hv_enlightenments *)vmcb->control.reserved_sw; +struct hv_enlightenments *hve = &vmcb->control.hv_enlightenments;if (hve->hv_enlightenments_control.msr_bitmap)vmcb_mark_dirty (vmcb,HV_VMCB_NESTED_ENLIGHTENMENTS ); } -static inline void svm_hv_update_vp_id (struct vmcb *vmcb, -struct kvm_vcpu *vcpu) +static inline void svm_hv_update_vp_id (struct vmcb *vmcb,struct kvm_vcpu *vcpu) { -struct hv_enlightenments *hve = - (struct hv_enlightenments *)vmcb->control.reserved_sw; +struct hv_enlightenments *hve = &vmcb->control.hv_enlightenments;u32 vp_index =kvm_hv_get_vpindex (vcpu);if (hve->hv_vp_id != vp_index) {
tools/testing/selftests/kvm/include/x86_64/svm.h +4 -1
@@ -123,7 +123,10 @@ struct __attribute__ ((__packed__)) vmcb_control_area {* Offset 0x3e0, 32 bytes reserved * for use by hypervisor/software. */ -u8 reserved_sw[32 ]; +union { +struct hv_enlightenments hv_enlightenments; +u8 reserved_sw[32 ]; + }; };
tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c +1 -2
@@ -46,8 +46,7 @@ static void __attribute__((__flatten__)) guest_code(struct svm_test_data *svm) {unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE ];struct vmcb *vmcb = svm->vmcb; -struct hv_enlightenments *hve = - (struct hv_enlightenments *)vmcb->control.reserved_sw; +struct hv_enlightenments *hve = &vmcb->control.hv_enlightenments;GUEST_SYNC (1 );
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 4/46 x86/hyperv: KVM: Rename "hv_enlightenments" to "hv_vmcb_enlightenments"
In reply to Vitaly KuznetsovFrom: Sean Christopherson <[email protected]>
Now that KVM isn't littered with "struct hv_enlightenments" casts, rename the struct to "hv_vmcb_enlightenments" to highlight the fact that the struct is specifically for SVM's VMCB.
No functional change intended.
--- arch/x86/include/asm/hyperv-tlfs.h | 2 +- arch/x86/include/asm/svm.h | 2 +- arch/x86/kvm/svm/nested.c | 2 +- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/svm/svm_onhyperv.c | 2 +- arch/x86/kvm/svm/svm_onhyperv.h | 6 +++--- tools/testing/selftests/kvm/include/x86_64/svm.h | 4 ++-- tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c | 2 +- 8 files changed, 11 insertions(+), 11 deletions(-)
arch/x86/include/asm/hyperv-tlfs.h +1 -1
@@ -602,7 +602,7 @@ struct hv_enlightened_vmcs {* Hyper-V uses the software reserved 32 bytes in VMCB control area to expose * SVM enlightenments to guests. */ -struct hv_enlightenments { +struct hv_vmcb_enlightenments { struct __packed hv_enlightenments_control { u32 nested_flush_hypercall:1; u32 msr_bitmap:1;
arch/x86/include/asm/svm.h +1 -1
@@ -164,7 +164,7 @@ struct __attribute__ ((__packed__)) vmcb_control_area {* for use by hypervisor/software. */ union { -struct hv_enlightenments hv_enlightenments; +struct hv_vmcb_enlightenments hv_enlightenments;u8 reserved_sw[32 ]; }; };
arch/x86/kvm/svm/nested.c +1 -1
@@ -179,7 +179,7 @@ void recalc_intercepts(struct vcpu_svm *svm)*/ static bool nested_svm_vmrun_msrpm (struct vcpu_svm *svm) { -struct hv_enlightenments *hve = &svm->nested.ctl.hv_enlightenments; +struct hv_vmcb_enlightenments *hve = &svm->nested.ctl.hv_enlightenments;int i;/*
arch/x86/kvm/svm/svm.h +1 -1
@@ -152,7 +152,7 @@ struct vmcb_ctrl_area_cached {u64 virt_ext;u32 clean;union { -struct hv_enlightenments hv_enlightenments; +struct hv_vmcb_enlightenments hv_enlightenments;u8 reserved_sw[32 ]; }; };
arch/x86/kvm/svm/svm_onhyperv.c +1 -1
@@ -16,7 +16,7 @@int svm_hv_enable_direct_tlbflush (struct kvm_vcpu *vcpu) { -struct hv_enlightenments *hve; +struct hv_vmcb_enlightenments *hve;struct hv_partition_assist_pg **p_hv_pa_pg = &to_kvm_hv (vcpu->kvm)->hv_pa_pg;
arch/x86/kvm/svm/svm_onhyperv.h +3 -3
@@ -17,7 +17,7 @@ int svm_hv_enable_direct_tlbflush(struct kvm_vcpu *vcpu);static inline void svm_hv_init_vmcb (struct vmcb *vmcb) { -struct hv_enlightenments *hve = &vmcb->control.hv_enlightenments; +struct hv_vmcb_enlightenments *hve = &vmcb->control.hv_enlightenments;BUILD_BUG_ON (sizeof (vmcb->control.hv_enlightenments) !=sizeof (vmcb->control.reserved_sw));
@@ -62,7 +62,7 @@ static inline void svm_hv_vmcb_dirty_nested_enlightenments( struct kvm_vcpu *vcpu) {struct vmcb *vmcb =to_svm (vcpu)->vmcb; -struct hv_enlightenments *hve = &vmcb->control.hv_enlightenments; +struct hv_vmcb_enlightenments *hve = &vmcb->control.hv_enlightenments;if (hve->hv_enlightenments_control.msr_bitmap)vmcb_mark_dirty (vmcb,HV_VMCB_NESTED_ENLIGHTENMENTS );
@@ -70,7 +70,7 @@ static inline void svm_hv_vmcb_dirty_nested_enlightenments(static inline void svm_hv_update_vp_id (struct vmcb *vmcb,struct kvm_vcpu *vcpu) { -struct hv_enlightenments *hve = &vmcb->control.hv_enlightenments; +struct hv_vmcb_enlightenments *hve = &vmcb->control.hv_enlightenments;u32 vp_index =kvm_hv_get_vpindex (vcpu);if (hve->hv_vp_id != vp_index) {
tools/testing/selftests/kvm/include/x86_64/svm.h +2 -2
@@ -58,7 +58,7 @@ enum { INTERCEPT_RDPRU, }; -struct hv_enlightenments { +struct hv_vmcb_enlightenments { struct __packed hv_enlightenments_control { u32 nested_flush_hypercall:1; u32 msr_bitmap:1;
@@ -124,7 +124,7 @@ struct __attribute__ ((__packed__)) vmcb_control_area {* for use by hypervisor/software. */ union { -struct hv_enlightenments hv_enlightenments; +struct hv_vmcb_enlightenments hv_enlightenments;u8 reserved_sw[32 ]; }; };
tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c +1 -1
@@ -46,7 +46,7 @@ static void __attribute__((__flatten__)) guest_code(struct svm_test_data *svm) {unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE ];struct vmcb *vmcb = svm->vmcb; -struct hv_enlightenments *hve = &vmcb->control.hv_enlightenments; +struct hv_vmcb_enlightenments *hve = &vmcb->control.hv_enlightenments;GUEST_SYNC (1 );
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 5/46 KVM: x86: Rename 'enable_direct_tlbflush' to 'enable_l2_tlb_flush'
In reply to Vitaly KuznetsovTo make terminology between Hyper-V-on-KVM and KVM-on-Hyper-V consistent, rename 'enable_direct_tlbflush' to 'enable_l2_tlb_flush'. The change eliminates the use of confusing 'direct' and adds the missing underscore.
No functional change.
--- arch/x86/include/asm/kvm-x86-ops.h | 2 +- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/svm/svm_onhyperv.c | 2 +- arch/x86/kvm/svm/svm_onhyperv.h | 6 +++--- arch/x86/kvm/vmx/vmx.c | 6 +++--- arch/x86/kvm/x86.c | 6 +++--- 6 files changed, 12 insertions(+), 12 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -1
@@ -123,7 +123,7 @@ KVM_X86_OP_OPTIONAL(guest_memory_reclaimed) KVM_X86_OP(get_msr_feature) KVM_X86_OP(can_emulate_instruction) KVM_X86_OP(apic_init_signal_blocked) -KVM_X86_OP_OPTIONAL(enable_direct_tlbflush) +KVM_X86_OP_OPTIONAL(enable_l2_tlb_flush) KVM_X86_OP_OPTIONAL(migrate_timers) KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr)
arch/x86/include/asm/kvm_host.h +1 -1
@@ -1624,7 +1624,7 @@ struct kvm_x86_ops { void *insn, int insn_len);bool (*apic_init_signal_blocked)(struct kvm_vcpu *vcpu); -int (*enable_direct_tlbflush)(struct kvm_vcpu *vcpu); +int (*enable_l2_tlb_flush)(struct kvm_vcpu *vcpu);void (*migrate_timers)(struct kvm_vcpu *vcpu);void (*msr_filter_changed)(struct kvm_vcpu *vcpu);
arch/x86/kvm/svm/svm_onhyperv.c +1 -1
@@ -14,7 +14,7 @@#include "kvm_onhyperv.h" #include "svm_onhyperv.h" -int svm_hv_enable_direct_tlbflush (struct kvm_vcpu *vcpu) +int svm_hv_enable_l2_tlb_flush (struct kvm_vcpu *vcpu) {struct hv_vmcb_enlightenments *hve; struct hv_partition_assist_pg **p_hv_pa_pg =
arch/x86/kvm/svm/svm_onhyperv.h +3 -3
@@ -13,7 +13,7 @@static struct kvm_x86_ops svm_x86_ops; -int svm_hv_enable_direct_tlbflush (struct kvm_vcpu *vcpu); +int svm_hv_enable_l2_tlb_flush (struct kvm_vcpu *vcpu);static inline void svm_hv_init_vmcb (struct vmcb *vmcb) {
@@ -53,8 +53,8 @@ static inline void svm_hv_hardware_setup(void) vp_ap->nested_control.features.directhypercall =1 ; } - svm_x86_ops.enable_direct_tlbflush = - svm_hv_enable_direct_tlbflush; + svm_x86_ops.enable_l2_tlb_flush = + svm_hv_enable_l2_tlb_flush; } }
arch/x86/kvm/vmx/vmx.c +3 -3
@@ -526,7 +526,7 @@ static unsigned long host_idt_base;static bool __read_mostly enlightened_vmcs = true;module_param (enlightened_vmcs, bool,0444 ); -static int hv_enable_direct_tlbflush (struct kvm_vcpu *vcpu) +static int hv_enable_l2_tlb_flush (struct kvm_vcpu *vcpu) {struct hv_enlightened_vmcs *evmcs; struct hv_partition_assist_pg **p_hv_pa_pg =
@@ -8479,8 +8479,8 @@ static int __init vmx_init(void) }if (ms_hyperv.nested_features &HV_X64_NESTED_DIRECT_FLUSH ) - vmx_x86_ops.enable_direct_tlbflush - = hv_enable_direct_tlbflush; + vmx_x86_ops.enable_l2_tlb_flush + = hv_enable_l2_tlb_flush; }else { enlightened_vmcs = false;
arch/x86/kvm/x86.c +3 -3
@@ -4467,7 +4467,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) kvm_x86_ops.nested_ops->get_state(NULL, NULL, 0) : 0;break ;case KVM_CAP_HYPERV_DIRECT_TLBFLUSH : - r = kvm_x86_ops.enable_direct_tlbflush !=NULL ; + r = kvm_x86_ops.enable_l2_tlb_flush !=NULL ;break ;case KVM_CAP_HYPERV_ENLIGHTENED_VMCS : r = kvm_x86_ops.nested_ops->enable_evmcs !=NULL ;
@@ -5483,10 +5483,10 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu, }return r;case KVM_CAP_HYPERV_DIRECT_TLBFLUSH : -if (!kvm_x86_ops.enable_direct_tlbflush) +if (!kvm_x86_ops.enable_l2_tlb_flush)return -ENOTTY ; -return static_call (kvm_x86_enable_direct_tlbflush)(vcpu); +return static_call (kvm_x86_enable_l2_tlb_flush)(vcpu);case KVM_CAP_HYPERV_ENFORCE_CPUID :return kvm_hv_set_enforce_cpuid (vcpu, cap->args[0 ]);
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 6/46 KVM: VMX: Rename "vmx/evmcs.{ch}" to "vmx/hyperv.{ch}"
In reply to Vitaly KuznetsovTo conform with SVM, rename VMX specific Hyper-V files from "evmcs.{ch}" to "hyperv.{ch}". While Enlightened VMCS is a lion's share of these files, some stuff (e.g. enlightened MSR bitmap, the upcoming Hyper-V L2 TLB flush, ...) goes beyond that.
---
arch/x86/kvm/Makefile | 2 +-
arch/x86/kvm/vmx/{evmcs.c => hyperv.c} | 3 +--
arch/x86/kvm/vmx/{evmcs.h => hyperv.h} | 8 +++++---
arch/x86/kvm/vmx/nested.c | 1 -
arch/x86/kvm/vmx/vmx.c | 1 -
arch/x86/kvm/vmx/vmx_ops.h | 2 +-
6 files changed, 8 insertions(+), 9 deletions(-)
rename arch/x86/kvm/vmx/{evmcs.c => hyperv.c} (99%)
rename arch/x86/kvm/vmx/{evmcs.h => hyperv.h} (98%)arch/x86/kvm/Makefile +1 -1
@@ -22,7 +22,7 @@ kvm-$(CONFIG_X86_64) += mmu/tdp_iter.o mmu/tdp_mmu.o kvm-$(CONFIG_KVM_XEN) += xen.o kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ - vmx/evmcs.o vmx/nested.o vmx/posted_intr.o + vmx/hyperv.o vmx/nested.o vmx/posted_intr.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.o
arch/x86/kvm/vmx/hyperv.c renamed from arch/x86/kvm/vmx/evmcs.c +1 -2
@@ -3,9 +3,8 @@#include <linux/errno.h> #include <linux/smp.h> -#include "../hyperv.h" #include "../cpuid.h" -#include "evmcs.h" +#include "hyperv.h" #include "vmcs.h" #include "vmx.h" #include "trace.h"
arch/x86/kvm/vmx/hyperv.h renamed from arch/x86/kvm/vmx/evmcs.h +5 -3
@@ -1,6 +1,6 @@/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef __KVM_X86_VMX_EVMCS_H -#define __KVM_X86_VMX_EVMCS_H +#ifndef __KVM_X86_VMX_HYPERV_H +#define __KVM_X86_VMX_HYPERV_H#include <linux/jump_label.h>
@@ -8,6 +8,8 @@#include <asm/mshyperv.h> #include <asm/vmx.h> +#include "../hyperv.h" +#include "capabilities.h" #include "vmcs.h" #include "vmcs12.h"
@@ -242,4 +244,4 @@ int nested_enable_evmcs(struct kvm_vcpu *vcpu,void nested_evmcs_filter_control_msr (struct kvm_vcpu *vcpu,u32 msr_index,u64 *pdata);int nested_evmcs_check_controls (struct vmcs12 *vmcs12); -#endif /* __KVM_X86_VMX_EVMCS_H */ +#endif /* __KVM_X86_VMX_HYPERV_H */
arch/x86/kvm/vmx/nested.c +0 -1
@@ -7,7 +7,6 @@#include <asm/mmu_context.h> #include "cpuid.h" -#include "evmcs.h" #include "hyperv.h" #include "mmu.h" #include "nested.h"
arch/x86/kvm/vmx/vmx.c +0 -1
@@ -51,7 +51,6 @@#include "capabilities.h" #include "cpuid.h" -#include "evmcs.h" #include "hyperv.h" #include "kvm_onhyperv.h" #include "irq.h"
arch/x86/kvm/vmx/vmx_ops.h +1 -1
@@ -6,7 +6,7 @@#include <asm/vmx.h> -#include "evmcs.h" +#include "hyperv.h" #include "vmcs.h" #include "../x86.h"
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 7/46 KVM: x86: Move clearing of TLB_FLUSH_CURRENT to kvm_vcpu_flush_tlb_all()
In reply to Vitaly KuznetsovFrom: Sean Christopherson <[email protected]>
Clear KVM_REQ_TLB_FLUSH_CURRENT in kvm_vcpu_flush_tlb_all() instead of in its sole caller that processes KVM_REQ_TLB_FLUSH. Regardless of why/when kvm_vcpu_flush_tlb_all() is called, flushing "all" TLB entries also flushes "current" TLB entries.
Ideally, there will never be another caller of kvm_vcpu_flush_tlb_all(), and moving the handling "requires" extra work to document the ordering requirement, but future Hyper-V paravirt TLB flushing support will add similar logic for flush "guest" (Hyper-V can flush a subset of "guest" entries). And in the Hyper-V case, KVM needs to do more than just clear the request, the queue of GPAs to flush also needs to purged, and doing all only in the request path is undesirable as kvm_vcpu_flush_tlb_guest() does have multiple callers (though it's unlikely KVM's paravirt TLB flush will coincide with Hyper-V's paravirt TLB flush).
Move the logic even though it adds extra "work" so that KVM will be consistent with how flush requests are processed when the Hyper-V support lands.
No functional change intended.
--- arch/x86/kvm/x86.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
arch/x86/kvm/x86.c +10 -4
@@ -3383,6 +3383,9 @@ static void kvm_vcpu_flush_tlb_all(struct kvm_vcpu *vcpu) { ++vcpu->stat.tlb_flush;static_call (kvm_x86_flush_tlb_all)(vcpu); + +/* Flushing all ASIDs flushes the current ASID... */ +kvm_clear_request (KVM_REQ_TLB_FLUSH_CURRENT , vcpu); }static void kvm_vcpu_flush_tlb_guest (struct kvm_vcpu *vcpu)
@@ -10464,12 +10467,15 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)kvm_mmu_sync_roots (vcpu);if (kvm_check_request (KVM_REQ_LOAD_MMU_PGD , vcpu))kvm_mmu_load_pgd (vcpu); -if (kvm_check_request (KVM_REQ_TLB_FLUSH , vcpu)) { + +/* +* Note, the order matters here, as flushing "all" TLB entries +* also flushes the "current" TLB entries, i.e. servicing the +* flush "all" will clear any request to flush "current". +*/ +if (kvm_check_request (KVM_REQ_TLB_FLUSH , vcpu))kvm_vcpu_flush_tlb_all (vcpu); -/* Flushing all ASIDs flushes the current ASID... */ -kvm_clear_request (KVM_REQ_TLB_FLUSH_CURRENT , vcpu); - }kvm_service_local_tlb_flush_requests (vcpu);if (kvm_check_request (KVM_REQ_REPORT_TPR_ACCESS , vcpu)) {
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 8/46 KVM: x86: hyper-v: Resurrect dedicated KVM_REQ_HV_TLB_FLUSH flag
In reply to Vitaly KuznetsovIn preparation to implementing fine-grained Hyper-V TLB flush and L2 TLB flush, resurrect dedicated KVM_REQ_HV_TLB_FLUSH request bit. As KVM_REQ_TLB_FLUSH_GUEST is a stronger operation, clear KVM_REQ_HV_TLB_FLUSH request in kvm_vcpu_flush_tlb_guest().
The flush itself is temporary handled by kvm_vcpu_flush_tlb_guest().
No functional change intended.
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/hyperv.c | 4 ++-- arch/x86/kvm/svm/svm.c | 7 +++++++ arch/x86/kvm/x86.c | 9 +++++++++ 4 files changed, 20 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -108,6 +108,8 @@ KVM_ARCH_REQ_FLAGS(30, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)#define KVM_REQ_MMU_FREE_OBSOLETE_ROOTS \ KVM_ARCH_REQ_FLAGS(31, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP) +#define KVM_REQ_HV_TLB_FLUSH \ + KVM_ARCH_REQ_FLAGS(32, KVM_REQUEST_WAIT | KVM_REQUEST_NO_WAKEUP)#define CR0_RESERVED_BITS \ (~(unsigned long)(X86_CR0_PE | X86_CR0_MP | X86_CR0_EM | X86_CR0_TS \
arch/x86/kvm/hyperv.c +2 -2
@@ -1870,11 +1870,11 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)* analyze it here, flush TLB regardless of the specified address space. */ if (all_cpus) { -kvm_make_all_cpus_request (kvm,KVM_REQ_TLB_FLUSH_GUEST ); +kvm_make_all_cpus_request (kvm,KVM_REQ_HV_TLB_FLUSH ); }else {sparse_set_to_vcpu_mask (kvm, sparse_banks, valid_bank_mask, vcpu_mask); -kvm_make_vcpus_request_mask (kvm,KVM_REQ_TLB_FLUSH_GUEST , vcpu_mask); +kvm_make_vcpus_request_mask (kvm,KVM_REQ_HV_TLB_FLUSH , vcpu_mask); } ret_success:
arch/x86/kvm/svm/svm.c +7 -0
@@ -3737,6 +3737,13 @@ static void svm_flush_tlb_current(struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu); +/* +* Unlike VMX, SVM doesn't provide a way to flush only NPT TLB entries. +* A TLB flush for the current ASID flushes both "host" and "guest" TLB +* entries, and thus is a superset of Hyper-V's fine grained flushing. +*/ +kvm_clear_request (KVM_REQ_HV_TLB_FLUSH , vcpu); +/* * Flush only the current ASID even if the TLB flush was invoked via * kvm_flush_remote_tlbs(). Although flushing remote TLBs requires all
arch/x86/kvm/x86.c +9 -0
@@ -3404,6 +3404,12 @@ static void kvm_vcpu_flush_tlb_guest(struct kvm_vcpu *vcpu) }static_call (kvm_x86_flush_tlb_guest)(vcpu); + +/* +* Flushing all "guest" TLB is always a superset of Hyper-V's fine +* grained flushing. +*/ +kvm_clear_request (KVM_REQ_HV_TLB_FLUSH , vcpu); }
@@ -10478,6 +10484,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)kvm_service_local_tlb_flush_requests (vcpu); +if (kvm_check_request (KVM_REQ_HV_TLB_FLUSH , vcpu)) +kvm_vcpu_flush_tlb_guest (vcpu); +if (kvm_check_request (KVM_REQ_REPORT_TPR_ACCESS , vcpu)) { vcpu->run->exit_reason =KVM_EXIT_TPR_ACCESS ; r =0 ;
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 9/46 KVM: x86: hyper-v: Introduce TLB flush fifo
In reply to Vitaly KuznetsovTo allow flushing individual GVAs instead of always flushing the whole VPID a per-vCPU structure to pass the requests is needed. Use standard 'kfifo' to queue two types of entries: individual GVA (GFN + up to 4095 following GFNs in the lower 12 bits) and 'flush all'.
The size of the fifo is arbitrarily set to '16'.
Note, kvm_hv_flush_tlb() only queues 'flush all' entries for now and kvm_hv_vcpu_flush_tlb() doesn't actually read the fifo just resets the queue before returning -EOPNOTSUPP (which triggers full TLB flush) so the functional change is very small but the infrastructure is prepared to handle individual GVA flush requests.
--- arch/x86/include/asm/kvm_host.h | 20 ++++++++++++++ arch/x86/kvm/hyperv.c | 47 +++++++++++++++++++++++++++++++++ arch/x86/kvm/hyperv.h | 15 +++++++++++ arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/x86.c | 13 ++++++--- arch/x86/kvm/x86.h | 1 + 6 files changed, 94 insertions(+), 4 deletions(-)
arch/x86/include/asm/kvm_host.h +20 -0
@@ -25,6 +25,7 @@#include <linux/clocksource.h> #include <linux/irqbypass.h> #include <linux/hyperv.h> +#include <linux/kfifo.h> #include <asm/apic.h> #include <asm/pvclock-abi.h>
@@ -599,6 +600,23 @@ struct kvm_vcpu_hv_synic {bool dont_zero_synic_pages; }; +/* The maximum number of entries on the TLB flush fifo. */ +#define KVM_HV_TLB_FLUSH_FIFO_SIZE (16) +/* +* Note: the following 'magic' entry is made up by KVM to avoid putting +* anything besides GVA on the TLB flush fifo. It is theoretically possible +* to observe a request to flush 4095 PFNs starting from 0xfffffffffffff000 +* which will look identical. KVM's action to 'flush everything' instead of +* flushing these particular addresses is, however, fully legitimate as +* flushing more than requested is always OK. +*/ +#define KVM_HV_TLB_FLUSHALL_ENTRY ((u64)-1) + +struct kvm_vcpu_hv_tlb_flush_fifo { +spinlock_t write_lock; + DECLARE_KFIFO(entries, u64, KVM_HV_TLB_FLUSH_FIFO_SIZE); +}; +/* Hyper-V per vcpu emulation context */ struct kvm_vcpu_hv {struct kvm_vcpu *vcpu;
@@ -620,6 +638,8 @@ struct kvm_vcpu_hv {u32 nested_eax;/* HYPERV_CPUID_NESTED_FEATURES.EAX */ u32 nested_ebx;/* HYPERV_CPUID_NESTED_FEATURES.EBX */ } cpuid_cache; + +struct kvm_vcpu_hv_tlb_flush_fifo tlb_flush_fifo; };/* Xen HVM per vcpu emulation context */
arch/x86/kvm/hyperv.c +47 -0
81 lines not on this page · raw message
arch/x86/kvm/hyperv.h +15 -0
@@ -151,4 +151,19 @@ int kvm_vm_ioctl_hv_eventfd(struct kvm *kvm, struct kvm_hyperv_eventfd *args);int kvm_get_hv_cpuid (struct kvm_vcpu *vcpu,struct kvm_cpuid2 *cpuid,struct kvm_cpuid_entry2 __user *entries); +static inline void kvm_hv_vcpu_purge_flush_tlb (struct kvm_vcpu *vcpu) +{ +struct kvm_vcpu_hv_tlb_flush_fifo *tlb_flush_fifo; +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); + +if (!hv_vcpu || !kvm_check_request (KVM_REQ_HV_TLB_FLUSH , vcpu)) +return ; + + tlb_flush_fifo = &hv_vcpu->tlb_flush_fifo; + +kfifo_reset_out (&tlb_flush_fifo->entries); +} + +int kvm_hv_vcpu_flush_tlb (struct kvm_vcpu *vcpu); +#endif
arch/x86/kvm/svm/svm.c +1 -1
@@ -3742,7 +3742,7 @@ static void svm_flush_tlb_current(struct kvm_vcpu *vcpu)* A TLB flush for the current ASID flushes both "host" and "guest" TLB * entries, and thus is a superset of Hyper-V's fine grained flushing. */ -kvm_clear_request (KVM_REQ_HV_TLB_FLUSH , vcpu); +kvm_hv_vcpu_purge_flush_tlb (vcpu);/* * Flush only the current ASID even if the TLB flush was invoked via
arch/x86/kvm/x86.c +10 -3
@@ -3388,7 +3388,7 @@ static void kvm_vcpu_flush_tlb_all(struct kvm_vcpu *vcpu)kvm_clear_request (KVM_REQ_TLB_FLUSH_CURRENT , vcpu); } -static void kvm_vcpu_flush_tlb_guest (struct kvm_vcpu *vcpu) +void kvm_vcpu_flush_tlb_guest (struct kvm_vcpu *vcpu) { ++vcpu->stat.tlb_flush;
@@ -3409,7 +3409,7 @@ static void kvm_vcpu_flush_tlb_guest(struct kvm_vcpu *vcpu)* Flushing all "guest" TLB is always a superset of Hyper-V's fine * grained flushing. */ -kvm_clear_request (KVM_REQ_HV_TLB_FLUSH , vcpu); +kvm_hv_vcpu_purge_flush_tlb (vcpu); }
@@ -10484,7 +10484,14 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)kvm_service_local_tlb_flush_requests (vcpu); -if (kvm_check_request (KVM_REQ_HV_TLB_FLUSH , vcpu)) +/* +* Fall back to a "full" guest flush if Hyper-V's precise +* flushing fails. Note, Hyper-V's flushing is per-vCPU, but +* the flushes are considered "remote" and not "local" because +* the requests can be initiated from other vCPUs. +*/ +if (kvm_check_request (KVM_REQ_HV_TLB_FLUSH , vcpu) && +kvm_hv_vcpu_flush_tlb (vcpu))kvm_vcpu_flush_tlb_guest (vcpu);if (kvm_check_request (KVM_REQ_REPORT_TPR_ACCESS , vcpu)) {
arch/x86/kvm/x86.h +1 -0
@@ -79,6 +79,7 @@ static inline unsigned int __shrink_ple_window(unsigned int val,#define MSR_IA32_CR_PAT_DEFAULT 0x0007040600070406ULL +void kvm_vcpu_flush_tlb_guest (struct kvm_vcpu *vcpu);void kvm_service_local_tlb_flush_requests (struct kvm_vcpu *vcpu);int kvm_check_nested_events (struct kvm_vcpu *vcpu);
Signature · 1 line
2.37.3
Sean Christopherson
maintainerRe: [PATCH v11 09/46] KVM: x86: hyper-v: Introduce TLB flush fifo
arch/x86/kvm/x86.c static void kvm_vcpu_flush_tlb_all(struct kvm_vcpu *vcpu) · line 3391
Quoted 10 lines · Vitaly Kuznetsov
arch/x86/kvm/x86.c +1
-1@@ -3388,7 +3388,7 @@ static void kvm_vcpu_flush_tlb_all(struct kvm_vcpu *vcpu)kvm_clear_request (KVM_REQ_TLB_FLUSH_CURRENT , vcpu); } -static void kvm_vcpu_flush_tlb_guest (struct kvm_vcpu *vcpu) +void kvm_vcpu_flush_tlb_guest (struct kvm_vcpu *vcpu)
kvm_vcpu_flush_tlb_guest() doesn't need to be exposed in this patch, i.e. can remain static. I think it can be restricted to x86.c even in the final form (will follow-up in later patch).
Quoted 33 lines
arch/x86/kvm/x86.c +9
-2{ ++vcpu->stat.tlb_flush;@@ -3409,7 +3409,7 @@ static void kvm_vcpu_flush_tlb_guest(struct kvm_vcpu *vcpu)* Flushing all "guest" TLB is always a superset of Hyper-V's fine * grained flushing. */ -kvm_clear_request (KVM_REQ_HV_TLB_FLUSH , vcpu); +kvm_hv_vcpu_purge_flush_tlb (vcpu); }@@ -10484,7 +10484,14 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)kvm_service_local_tlb_flush_requests (vcpu); -if (kvm_check_request (KVM_REQ_HV_TLB_FLUSH , vcpu)) +/* +* Fall back to a "full" guest flush if Hyper-V's precise +* flushing fails. Note, Hyper-V's flushing is per-vCPU, but +* the flushes are considered "remote" and not "local" because +* the requests can be initiated from other vCPUs. +*/ +if (kvm_check_request (KVM_REQ_HV_TLB_FLUSH , vcpu) && +kvm_hv_vcpu_flush_tlb (vcpu))kvm_vcpu_flush_tlb_guest (vcpu);if (kvm_check_request (KVM_REQ_REPORT_TPR_ACCESS , vcpu)) {arch/x86/kvm/x86.h +0
-0
arch/x86/kvm/x86.h static inline unsigned int __shrink_ple_window(unsigned int val, · line 82
Quoted 4 lines
arch/x86/kvm/x86.h +1
-0@@ -79,6 +79,7 @@ static inline unsigned int __shrink_ple_window(unsigned int val,#define MSR_IA32_CR_PAT_DEFAULT 0x0007040600070406ULL +void kvm_vcpu_flush_tlb_guest (struct kvm_vcpu *vcpu);
As above, this is unnecessary.
Quoted 6 lines
arch/x86/kvm/x86.h +0
-0void kvm_service_local_tlb_flush_requests (struct kvm_vcpu *vcpu);int kvm_check_nested_events (struct kvm_vcpu *vcpu);Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 10/46 KVM: x86: hyper-v: Add helper to read hypercall data for array
In reply to Vitaly KuznetsovFrom: Sean Christopherson <[email protected]>
Move the guts of kvm_get_sparse_vp_set() to a helper so that the code for reading a guest-provided array can be reused in the future, e.g. for getting a list of virtual addresses whose TLB entries need to be flushed.
Opportunisticaly swap the order of the data and XMM adjustment so that the XMM/gpa offsets are bundled together.
No functional change intended.
--- arch/x86/kvm/hyperv.c | 53 +++++++++++++++++++++++++++---------------- 1 file changed, 33 insertions(+), 20 deletions(-)
arch/x86/kvm/hyperv.c +33 -20
@@ -1753,38 +1753,51 @@ struct kvm_hv_hcall {sse128_t xmm[HV_HYPERCALL_MAX_XMM_REGISTERS ]; }; -static u64 kvm_get_sparse_vp_set (struct kvm *kvm,struct kvm_hv_hcall *hc, -int consumed_xmm_halves, -u64 *sparse_banks,gpa_t offset) -{ -u16 var_cnt; -int i; -if (hc->var_cnt >64 ) -return -EINVAL ; - -/* Ignore banks that cannot possibly contain a legal VP index. */ - var_cnt =min_t (u16, hc->var_cnt,KVM_HV_MAX_SPARSE_VCPU_SET_BITS ); +static int kvm_hv_get_hc_data (struct kvm *kvm,struct kvm_hv_hcall *hc, +u16 orig_cnt,u16 cnt_cap,u64 *data, +int consumed_xmm_halves,gpa_t offset) +{ +/* +* Preserve the original count when ignoring entries via a "cap", KVM +* still needs to validate the guest input (though the non-XMM path +* punts on the checks). +*/ +u16 cnt =min (orig_cnt, cnt_cap); +int i, j;if (hc->fast) {/* * Each XMM holds two sparse banks, but do not count halves that * have already been consumed for hypercall parameters. */ -if (hc->var_cnt >2 *HV_HYPERCALL_MAX_XMM_REGISTERS - consumed_xmm_halves) +if (orig_cnt >2 *HV_HYPERCALL_MAX_XMM_REGISTERS - consumed_xmm_halves)return HV_STATUS_INVALID_HYPERCALL_INPUT ; -for (i =0 ; i < var_cnt; i++) { -int j = i + consumed_xmm_halves; + +for (i =0 ; i < cnt; i++) { + j = i + consumed_xmm_halves;if (j %2 ) - sparse_banks[i] =sse128_hi (hc->xmm[j /2 ]); + data[i] =sse128_hi (hc->xmm[j /2 ]);else - sparse_banks[i] =sse128_lo (hc->xmm[j /2 ]); + data[i] =sse128_lo (hc->xmm[j /2 ]); }return 0 ; } -return kvm_read_guest (kvm, hc->ingpa + offset, sparse_banks, - var_cnt *sizeof (*sparse_banks)); +return kvm_read_guest (kvm, hc->ingpa + offset, data, + cnt *sizeof (*data)); +} + +static u64 kvm_get_sparse_vp_set (struct kvm *kvm,struct kvm_hv_hcall *hc, +u64 *sparse_banks,int consumed_xmm_halves, +gpa_t offset) +{ +if (hc->var_cnt >64 ) +return -EINVAL ; + +/* Cap var_cnt to ignore banks that cannot contain a legal VP index. */ +return kvm_hv_get_hc_data (kvm, hc, hc->var_cnt,KVM_HV_MAX_SPARSE_VCPU_SET_BITS , + sparse_banks, consumed_xmm_halves, offset); }static void hv_tlb_flush_enqueue (struct kvm_vcpu *vcpu)
@@ -1895,7 +1908,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)if (!hc->var_cnt) goto ret_success; -if (kvm_get_sparse_vp_set (kvm, hc,2 , sparse_banks, +if (kvm_get_sparse_vp_set (kvm, hc, sparse_banks,2 , offsetof(struct hv_tlb_flush_ex , hv_vp_set.bank_contents)))return HV_STATUS_INVALID_HYPERCALL_INPUT ;
@@ -2006,7 +2019,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)if (!hc->var_cnt) goto ret_success; -if (kvm_get_sparse_vp_set (kvm, hc,1 , sparse_banks, +if (kvm_get_sparse_vp_set (kvm, hc, sparse_banks,1 , offsetof(struct hv_send_ipi_ex , vp_set.bank_contents)))return HV_STATUS_INVALID_HYPERCALL_INPUT ;
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 11/46 KVM: x86: hyper-v: Handle HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} calls gently
In reply to Vitaly KuznetsovCurrently, HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} calls are handled the exact same way as HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE{,EX}: by flushing the whole VPID and this is sub-optimal. Switch to handling these requests with 'flush_tlb_gva()' hooks instead. Use the newly introduced TLB flush fifo to queue the requests.
--- arch/x86/kvm/hyperv.c | 100 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 87 insertions(+), 13 deletions(-)
arch/x86/kvm/hyperv.c +87 -13
@@ -1800,7 +1800,14 @@ static u64 kvm_get_sparse_vp_set(struct kvm *kvm, struct kvm_hv_hcall *hc, sparse_banks, consumed_xmm_halves, offset); } -static void hv_tlb_flush_enqueue (struct kvm_vcpu *vcpu) +static int kvm_hv_get_tlb_flush_entries (struct kvm *kvm,struct kvm_hv_hcall *hc,u64 entries[], +int consumed_xmm_halves,gpa_t offset) +{ +return kvm_hv_get_hc_data (kvm, hc, hc->rep_cnt, hc->rep_cnt, + entries, consumed_xmm_halves, offset); +} + +static void hv_tlb_flush_enqueue (struct kvm_vcpu *vcpu,u64 *entries,int count) {struct kvm_vcpu_hv_tlb_flush_fifo *tlb_flush_fifo;struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu);
@@ -1811,24 +1818,65 @@ static void hv_tlb_flush_enqueue(struct kvm_vcpu *vcpu) tlb_flush_fifo = &hv_vcpu->tlb_flush_fifo; -kfifo_in_spinlocked_noirqsave (&tlb_flush_fifo->entries, &flush_all_entry, -1 , &tlb_flush_fifo->write_lock); +spin_lock (&tlb_flush_fifo->write_lock); + +/* +* All entries should fit on the fifo leaving one free for 'flush all' +* entry in case another request comes in. In case there's not enough +* space, just put 'flush all' entry there. +*/ +if (count && entries && count <kfifo_avail (&tlb_flush_fifo->entries)) { +WARN_ON (kfifo_in (&tlb_flush_fifo->entries, entries, count) != count); + goto out_unlock; + } + +/* +* Note: full fifo always contains 'flush all' entry, no need to check the +* return value. +*/ +kfifo_in (&tlb_flush_fifo->entries, &flush_all_entry,1 ); + +out_unlock: +spin_unlock (&tlb_flush_fifo->write_lock); }int kvm_hv_vcpu_flush_tlb (struct kvm_vcpu *vcpu) {struct kvm_vcpu_hv_tlb_flush_fifo *tlb_flush_fifo;struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); +u64 entries[KVM_HV_TLB_FLUSH_FIFO_SIZE ]; +int i, j, count; +gva_t gva; -if (!hv_vcpu) +if (!tdp_enabled || !hv_vcpu)return -EINVAL ; tlb_flush_fifo = &hv_vcpu->tlb_flush_fifo; + count =kfifo_out (&tlb_flush_fifo->entries, entries,KVM_HV_TLB_FLUSH_FIFO_SIZE ); + +for (i =0 ; i < count; i++) { +if (entries[i] ==KVM_HV_TLB_FLUSHALL_ENTRY ) + goto out_flush_all; + +/* +* Lower 12 bits of 'address' encode the number of additional +* pages to flush. +*/ + gva = entries[i] &PAGE_MASK ; +for (j =0 ; j < (entries[i] & ~PAGE_MASK ) +1 ; j++) +static_call (kvm_x86_flush_tlb_gva)(vcpu, gva + j *PAGE_SIZE ); + + ++vcpu->stat.tlb_flush; + } +return 0 ; + +out_flush_all: +kvm_vcpu_flush_tlb_guest (vcpu);kfifo_reset_out (&tlb_flush_fifo->entries); -/* Precise flushing isn't implemented yet. */ -return -EOPNOTSUPP ; +/* Fall back to full flush. */ +return -ENOSPC ; }static u64 kvm_hv_flush_tlb (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc)
@@ -1837,11 +1885,21 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)struct hv_tlb_flush_ex flush_ex;struct hv_tlb_flush flush;DECLARE_BITMAP (vcpu_mask,KVM_MAX_VCPUS ); +/* +* Normally, there can be no more than 'KVM_HV_TLB_FLUSH_FIFO_SIZE' +* entries on the TLB flush fifo. The last entry, however, needs to be +* always left free for 'flush all' entry which gets placed when +* there is not enough space to put all the requested entries. +*/ +u64 __tlb_flush_entries[KVM_HV_TLB_FLUSH_FIFO_SIZE -1 ]; +u64 *tlb_flush_entries;u64 valid_bank_mask;u64 sparse_banks[KVM_HV_MAX_SPARSE_VCPU_SET_BITS ];struct kvm_vcpu *v;unsigned long i;bool all_cpus; +int consumed_xmm_halves =0 ; +gpa_t data_offset;/* * The Hyper-V TLFS doesn't allow more than 64 sparse banks, e.g. the
@@ -1857,10 +1915,12 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) flush.address_space = hc->ingpa; flush.flags = hc->outgpa; flush.processor_mask =sse128_lo (hc->xmm[0 ]); + consumed_xmm_halves =1 ; }else {if (unlikely (kvm_read_guest (kvm, hc->ingpa, &flush,sizeof (flush))))return HV_STATUS_INVALID_HYPERCALL_INPUT ; + data_offset =sizeof (flush); } trace_kvm_hv_flush_tlb(flush.processor_mask,
@@ -1884,10 +1944,12 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) flush_ex.flags = hc->outgpa;memcpy (&flush_ex.hv_vp_set, &hc->xmm[0 ],sizeof (hc->xmm[0 ])); + consumed_xmm_halves =2 ; }else {if (unlikely (kvm_read_guest (kvm, hc->ingpa, &flush_ex,sizeof (flush_ex))))return HV_STATUS_INVALID_HYPERCALL_INPUT ; + data_offset =sizeof (flush_ex); } trace_kvm_hv_flush_tlb_ex(flush_ex.hv_vp_set.valid_bank_mask,
@@ -1903,25 +1965,37 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)return HV_STATUS_INVALID_HYPERCALL_INPUT ;if (all_cpus) - goto do_flush; + goto read_flush_entries;if (!hc->var_cnt) goto ret_success; -if (kvm_get_sparse_vp_set (kvm, hc, sparse_banks,2 , - offsetof(struct hv_tlb_flush_ex , - hv_vp_set.bank_contents))) +if (kvm_get_sparse_vp_set (kvm, hc, sparse_banks, consumed_xmm_halves, + data_offset)) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; + data_offset += hc->var_cnt *sizeof (sparse_banks[0 ]); + consumed_xmm_halves += hc->var_cnt; + } + +read_flush_entries: +if (hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE || + hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX || + hc->rep_cnt >ARRAY_SIZE (__tlb_flush_entries)) { + tlb_flush_entries =NULL ; + }else { +if (kvm_hv_get_tlb_flush_entries (kvm, hc, __tlb_flush_entries, + consumed_xmm_halves, data_offset))return HV_STATUS_INVALID_HYPERCALL_INPUT ; + tlb_flush_entries = __tlb_flush_entries; } -do_flush:/* * vcpu->arch.cr3 may not be up-to-date for running vCPUs so we can't * analyze it here, flush TLB regardless of the specified address space. */ if (all_cpus) {kvm_for_each_vcpu (i, v, kvm) -hv_tlb_flush_enqueue (v); +hv_tlb_flush_enqueue (v, tlb_flush_entries, hc->rep_cnt);kvm_make_all_cpus_request (kvm,KVM_REQ_HV_TLB_FLUSH ); }else {
@@ -1931,7 +2005,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) v =kvm_get_vcpu (kvm, i);if (!v)continue ; -hv_tlb_flush_enqueue (v); +hv_tlb_flush_enqueue (v, tlb_flush_entries, hc->rep_cnt); }kvm_make_vcpus_request_mask (kvm,KVM_REQ_HV_TLB_FLUSH , vcpu_mask);
Signature · 1 line
2.37.3
Sean Christopherson
maintainerRe: [PATCH v11 11/46] KVM: x86: hyper-v: Handle HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} calls gently
Quoted 24 lines · Vitaly Kuznetsov
int
kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu)
{arch/x86/kvm/hyperv.c +14
-1struct kvm_vcpu_hv_tlb_flush_fifo *tlb_flush_fifo;struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); +u64 entries[KVM_HV_TLB_FLUSH_FIFO_SIZE ]; +int i, j, count; +gva_t gva; -if (!hv_vcpu) +if (!tdp_enabled || !hv_vcpu)return -EINVAL ; tlb_flush_fifo = &hv_vcpu->tlb_flush_fifo; + count =kfifo_out (&tlb_flush_fifo->entries, entries,KVM_HV_TLB_FLUSH_FIFO_SIZE ); + +for (i =0 ; i < count; i++) { +if (entries[i] ==KVM_HV_TLB_FLUSHALL_ENTRY ) + goto out_flush_all; + +/* +* Lower 12 bits of 'address' encode the number of additional +* pages to flush. +*/
arch/x86/kvm/hyperv.c static void hv_tlb_flush_enqueue(struct kvm_vcpu *vcpu) · line 1875
Quoted 10 lines
arch/x86/kvm/hyperv.c +10
-0+ gva = entries[i] &PAGE_MASK ; +for (j =0 ; j < (entries[i] & ~PAGE_MASK ) +1 ; j++) +static_call (kvm_x86_flush_tlb_gva)(vcpu, gva + j *PAGE_SIZE ); + + ++vcpu->stat.tlb_flush; + } +return 0 ; + +out_flush_all: +kvm_vcpu_flush_tlb_guest (vcpu);
No need to do kvm_vcpu_flush_tlb_guest() here, the caller is responsible for flushing on "failure", as indicated by -ENOSPC below.
Quoted 7 lines
arch/x86/kvm/hyperv.c +2
-2kfifo_reset_out (&tlb_flush_fifo->entries); -/* Precise flushing isn't implemented yet. */ -return -EOPNOTSUPP ; +/* Fall back to full flush. */ +return -ENOSPC ; }
Vitaly Kuznetsov
maintainerPATCH v11 12/46 KVM: x86: hyper-v: Expose support for extended gva ranges for flush hypercalls
In reply to Vitaly KuznetsovExtended GVA ranges support bit seems to indicate whether lower 12 bits of GVA can be used to specify up to 4095 additional consequent GVAs to flush. This is somewhat described in TLFS.
Previously, KVM was handling HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} requests by flushing the whole VPID so technically, extended GVA ranges were already supported. As such requests are handled more gently now, advertizing support for extended ranges starts making sense to reduce the size of TLB flush requests.
--- arch/x86/include/asm/hyperv-tlfs.h | 2 ++ arch/x86/kvm/hyperv.c | 1 + 2 files changed, 3 insertions(+)
arch/x86/include/asm/hyperv-tlfs.h +2 -0
@@ -61,6 +61,8 @@#define HV_FEATURE_GUEST_CRASH_MSR_AVAILABLE BIT(10)/* Support for debug MSRs available */ #define HV_FEATURE_DEBUG_MSRS_AVAILABLE BIT(11) +/* Support for extended gva ranges for flush hypercalls available */ +#define HV_FEATURE_EXT_GVA_RANGES_FLUSH BIT(14)/* * Support for returning hypercall output block via XMM * registers is available
arch/x86/kvm/hyperv.c +1 -0
@@ -2636,6 +2636,7 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid, ent->ebx |=HV_DEBUGGING ; ent->edx |=HV_X64_GUEST_DEBUGGING_AVAILABLE ; ent->edx |=HV_FEATURE_DEBUG_MSRS_AVAILABLE ; + ent->edx |=HV_FEATURE_EXT_GVA_RANGES_FLUSH ;/* * Direct Synthetic timers only make sense with in-kernel
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 13/46 KVM: x86: Prepare kvm_hv_flush_tlb() to handle L2's GPAs
In reply to Vitaly KuznetsovTo handle L2 TLB flush requests, KVM needs to translate the specified L2 GPA to L1 GPA to read hypercall arguments from there.
No functional change as KVM doesn't handle VMCALL/VMMCALL from L2 yet.
--- arch/x86/kvm/hyperv.c | 7 +++++++ 1 file changed, 7 insertions(+)
arch/x86/kvm/hyperv.c +7 -0
@@ -23,6 +23,7 @@#include "ioapic.h" #include "cpuid.h" #include "hyperv.h" +#include "mmu.h" #include "xen.h" #include <linux/cpu.h>
@@ -1909,6 +1910,12 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)*/ BUILD_BUG_ON (KVM_HV_MAX_SPARSE_VCPU_SET_BITS >64 ); +if (!hc->fast &&is_guest_mode (vcpu)) { + hc->ingpa =translate_nested_gpa (vcpu, hc->ingpa,0 ,NULL ); +if (unlikely (hc->ingpa ==INVALID_GPA )) +return HV_STATUS_INVALID_HYPERCALL_INPUT ; + } +if (hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST || hc->code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE ) {if (hc->fast) {
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 14/46 x86/hyperv: Introduce HV_MAX_SPARSE_VCPU_BANKS/HV_VCPUS_PER_SPARSE_BANK constants
In reply to Vitaly KuznetsovIt may not come clear from where the magical '64' value used in __cpumask_to_vpset() come from. Moreover, '64' means both the maximum sparse bank number as well as the number of vCPUs per bank. Add defines to make things clear. These defines are also going to be used by KVM.
No functional change.
--- include/asm-generic/hyperv-tlfs.h | 5 +++++ include/asm-generic/mshyperv.h | 11 ++++++----- 2 files changed, 11 insertions(+), 5 deletions(-)
include/asm-generic/hyperv-tlfs.h +5 -0
@@ -399,6 +399,11 @@ struct hv_vpset {u64 bank_contents[]; } __packed; +/* The maximum number of sparse vCPU banks which can be encoded by 'struct hv_vpset' */ +#define HV_MAX_SPARSE_VCPU_BANKS (64) +/* The number of vCPUs in one sparse bank */ +#define HV_VCPUS_PER_SPARSE_BANK (64) +/* HvCallSendSyntheticClusterIpi hypercall */ struct hv_send_ipi {u32 vector;
include/asm-generic/mshyperv.h +6 -5
@@ -214,9 +214,10 @@ static inline int __cpumask_to_vpset(struct hv_vpset *vpset, {int cpu, vcpu, vcpu_bank, vcpu_offset, nr_bank =1 ;int this_cpu =smp_processor_id (); +int max_vcpu_bank = hv_max_vp_index /HV_VCPUS_PER_SPARSE_BANK ; -/* valid_bank_mask can represent up to 64 banks */ -if (hv_max_vp_index /64 >=64 ) +/* vpset.valid_bank_mask can represent up to HV_MAX_SPARSE_VCPU_BANKS banks */ +if (max_vcpu_bank >=HV_MAX_SPARSE_VCPU_BANKS )return 0 ;/*
@@ -224,7 +225,7 @@ static inline int __cpumask_to_vpset(struct hv_vpset *vpset,* structs are not cleared between calls, we risk flushing unneeded * vCPUs otherwise. */ -for (vcpu_bank =0 ; vcpu_bank <= hv_max_vp_index /64 ; vcpu_bank++) +for (vcpu_bank =0 ; vcpu_bank <= max_vcpu_bank; vcpu_bank++) vpset->bank_contents[vcpu_bank] =0 ;/*
@@ -236,8 +237,8 @@ static inline int __cpumask_to_vpset(struct hv_vpset *vpset, vcpu =hv_cpu_number_to_vp_number (cpu);if (vcpu ==VP_INVAL )return -1 ; - vcpu_bank = vcpu /64 ; - vcpu_offset = vcpu %64 ; + vcpu_bank = vcpu /HV_VCPUS_PER_SPARSE_BANK ; + vcpu_offset = vcpu %HV_VCPUS_PER_SPARSE_BANK ;__set_bit (vcpu_offset, (unsigned long *) &vpset->bank_contents[vcpu_bank]);if (vcpu_bank >= nr_bank)
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 15/46 KVM: x86: hyper-v: Use HV_MAX_SPARSE_VCPU_BANKS/HV_VCPUS_PER_SPARSE_BANK instead of raw '64'
In reply to Vitaly KuznetsovIt may not be clear from where the '64' limit for the maximum sparse bank number comes from, use HV_MAX_SPARSE_VCPU_BANKS define instead. Use HV_VCPUS_PER_SPARSE_BANK in KVM_HV_MAX_SPARSE_VCPU_SET_BITS's definition. Opportunistically adjust the comment around BUILD_BUG_ON().
No functional change.
--- arch/x86/kvm/hyperv.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
arch/x86/kvm/hyperv.c +6 -7
@@ -40,7 +40,7 @@#include "irq.h" #include "fpu.h" -#define KVM_HV_MAX_SPARSE_VCPU_SET_BITS DIV_ROUND_UP(KVM_MAX_VCPUS, 64) +#define KVM_HV_MAX_SPARSE_VCPU_SET_BITS DIV_ROUND_UP(KVM_MAX_VCPUS, HV_VCPUS_PER_SPARSE_BANK)static void stimer_mark_pending (struct kvm_vcpu_hv_stimer *stimer,bool vcpu_kick);
@@ -1793,7 +1793,7 @@ static u64 kvm_get_sparse_vp_set(struct kvm *kvm, struct kvm_hv_hcall *hc, u64 *sparse_banks, int consumed_xmm_halves, gpa_t offset) { - if (hc->var_cnt > 64) + if (hc->var_cnt > HV_MAX_SPARSE_VCPU_BANKS) return -EINVAL;/* Cap var_cnt to ignore banks that cannot contain a legal VP index. */
@@ -1903,12 +1903,11 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)gpa_t data_offset;/* -* The Hyper-V TLFS doesn't allow more than 64 sparse banks, e.g. the -* valid mask is a u64. Fail the build if KVM's max allowed number of -* vCPUs (>4096) would exceed this limit, KVM will additional changes -* for Hyper-V support to avoid setting the guest up to fail. +* The Hyper-V TLFS doesn't allow more than HV_MAX_SPARSE_VCPU_BANKS +* sparse banks. Fail the build if KVM's max allowed number of +* vCPUs (>4096) exceeds this limit. */ -BUILD_BUG_ON (KVM_HV_MAX_SPARSE_VCPU_SET_BITS >64 ); +BUILD_BUG_ON (KVM_HV_MAX_SPARSE_VCPU_SET_BITS >HV_MAX_SPARSE_VCPU_BANKS );if (!hc->fast &&is_guest_mode (vcpu)) { hc->ingpa =translate_nested_gpa (vcpu, hc->ingpa,0 ,NULL );
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 16/46 KVM: x86: hyper-v: Don't use sparse_set_to_vcpu_mask() in kvm_hv_send_ipi()
In reply to Vitaly KuznetsovGet rid of on-stack allocation of vcpu_mask and optimize kvm_hv_send_ipi() for a smaller number of vCPUs in the request. When Hyper-V TLB flush is in use, HvSendSyntheticClusterIpi{,Ex} calls are not commonly used to send IPIs to a large number of vCPUs (and are rarely used in general).
Introduce hv_is_vp_in_sparse_set() to directly check if the specified VP_ID is present in sparse vCPU set.
--- arch/x86/kvm/hyperv.c | 40 +++++++++++++++++++++++++++++----------- 1 file changed, 29 insertions(+), 11 deletions(-)
arch/x86/kvm/hyperv.c +29 -11
@@ -1741,6 +1741,28 @@ static void sparse_set_to_vcpu_mask(struct kvm *kvm, u64 *sparse_banks, } } +static bool hv_is_vp_in_sparse_set (u32 vp_id,u64 valid_bank_mask,u64 sparse_banks[]) +{ +int valid_bit_nr = vp_id /HV_VCPUS_PER_SPARSE_BANK ; +unsigned long sbank; + +if (!test_bit (valid_bit_nr, (unsigned long *)&valid_bank_mask)) +return false; + +/* +* The index into the sparse bank is the number of preceding bits in +* the valid mask. Optimize for VMs with <64 vCPUs by skipping the +* fancy math if there can't possibly be preceding bits. +*/ +if (valid_bit_nr) + sbank =hweight64 (valid_bank_mask &GENMASK_ULL (valid_bit_nr -1 ,0 )); +else + sbank =0 ; + +return test_bit (vp_id %HV_VCPUS_PER_SPARSE_BANK , + (unsigned long *)&sparse_banks[sbank]); +} +struct kvm_hv_hcall {u64 param;u64 ingpa;
@@ -2023,8 +2045,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) ((u64 )hc->rep_cnt <<HV_HYPERCALL_REP_COMP_OFFSET ); } -static void kvm_send_ipi_to_many (struct kvm *kvm,u32 vector, -unsigned long *vcpu_bitmap) +static void kvm_hv_send_ipi_to_many (struct kvm *kvm,u32 vector, +u64 *sparse_banks,u64 valid_bank_mask) {struct kvm_lapic_irq irq = { .delivery_mode =APIC_DM_FIXED ,
@@ -2034,7 +2056,10 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector,unsigned long i;kvm_for_each_vcpu (i, vcpu, kvm) { -if (vcpu_bitmap && !test_bit (i, vcpu_bitmap)) +if (sparse_banks && + !hv_is_vp_in_sparse_set (kvm_hv_get_vpindex (vcpu), + valid_bank_mask, + sparse_banks))continue ;/* We fail only when APIC is disabled */
@@ -2047,7 +2072,6 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)struct kvm *kvm = vcpu->kvm;struct hv_send_ipi_ex send_ipi_ex;struct hv_send_ipi send_ipi; -DECLARE_BITMAP (vcpu_mask,KVM_MAX_VCPUS );u64 valid_bank_mask;u64 sparse_banks[KVM_HV_MAX_SPARSE_VCPU_SET_BITS ];u32 vector;
@@ -2109,13 +2133,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)if ((vector <HV_IPI_LOW_VECTOR ) || (vector >HV_IPI_HIGH_VECTOR ))return HV_STATUS_INVALID_HYPERCALL_INPUT ; -if (all_cpus) { -kvm_send_ipi_to_many (kvm, vector,NULL ); - }else { -sparse_set_to_vcpu_mask (kvm, sparse_banks, valid_bank_mask, vcpu_mask); - -kvm_send_ipi_to_many (kvm, vector, vcpu_mask); - } +kvm_hv_send_ipi_to_many (kvm, vector, all_cpus ?NULL : sparse_banks, valid_bank_mask); ret_success:return HV_STATUS_SUCCESS ;
Signature · 1 line
2.37.3
Sean Christopherson
maintainerRe: [PATCH v11 16/46] KVM: x86: hyper-v: Don't use sparse_set_to_vcpu_mask() in kvm_hv_send_ipi()
arch/x86/kvm/hyperv.c static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector, · line 2062
Quoted 9 lines · Vitaly Kuznetsov
arch/x86/kvm/hyperv.c +4
-1@@ -2034,7 +2056,10 @@ static void kvm_send_ipi_to_many(struct kvm *kvm, u32 vector,unsigned long i;kvm_for_each_vcpu (i, vcpu, kvm) { -if (vcpu_bitmap && !test_bit (i, vcpu_bitmap)) +if (sparse_banks && + !hv_is_vp_in_sparse_set (kvm_hv_get_vpindex (vcpu), + valid_bank_mask, + sparse_banks))
Nit, this fits on two lines and IMO is slightly easier on the eyes:
if (sparse_banks && !hv_is_vp_in_sparse_set(kvm_hv_get_vpindex(vcpu), valid_bank_mask, sparse_banks)) continue;
Quoted 13 lines
arch/x86/kvm/hyperv.c +0
-1continue ;/* We fail only when APIC is disabled */ @@ -2047,7 +2072,6 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)struct kvm *kvm = vcpu->kvm;struct hv_send_ipi_ex send_ipi_ex;struct hv_send_ipi send_ipi; -DECLARE_BITMAP (vcpu_mask,KVM_MAX_VCPUS );u64 valid_bank_mask;u64 sparse_banks[KVM_HV_MAX_SPARSE_VCPU_SET_BITS ];u32 vector;@@ -2109,13 +2133,7 @@ static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)if ((vector <HV_IPI_LOW_VECTOR ) || (vector >HV_IPI_HIGH_VECTOR ))
arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · line 2136
Quoted 10 lines
arch/x86/kvm/hyperv.c +1
-7return HV_STATUS_INVALID_HYPERCALL_INPUT ; -if (all_cpus) { -kvm_send_ipi_to_many (kvm, vector,NULL ); - }else { -sparse_set_to_vcpu_mask (kvm, sparse_banks, valid_bank_mask, vcpu_mask); - -kvm_send_ipi_to_many (kvm, vector, vcpu_mask); - } +kvm_hv_send_ipi_to_many (kvm, vector, all_cpus ?NULL : sparse_banks, valid_bank_mask);
Any objection to not using a ternary operator?
if (all_cpus) kvm_hv_send_ipi_to_many(kvm, vector, NULL, 0); else kvm_hv_send_ipi_to_many(kvm, vector, sparse_banks, valid_bank_mask);
Mostly because it's somewhat arbitrary that earlier code ensures valid_bank_mask is set in the all_cpus=true case, e.g. arguably KVM doesn't need to do the var_cnt sanity check in the all_cpus case:
all_cpus = send_ipi_ex.vp_set.format == HV_GENERIC_SET_ALL; if (all_cpus) goto check_and_send_ipi; valid_bank_mask = send_ipi_ex.vp_set.valid_bank_mask; if (hc->var_cnt != hweight64(valid_bank_mask)) return HV_STATUS_INVALID_HYPERCALL_INPUT; if (!hc->var_cnt) goto ret_success;
Vitaly Kuznetsov
maintainerRe: [PATCH v11 16/46] KVM: x86: hyper-v: Don't use sparse_set_to_vcpu_mask() in kvm_hv_send_ipi()
Quoted 1 line · Sean Christopherson
On Tue, Oct 04, 2022, Vitaly Kuznetsov wrote:
...
Quoted 12 lines
arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · line 2136
Quoted 9 lines
arch/x86/kvm/hyperv.c +1
-7-if (all_cpus) { -kvm_send_ipi_to_many (kvm, vector,NULL ); - }else { -sparse_set_to_vcpu_mask (kvm, sparse_banks, valid_bank_mask, vcpu_mask); - -kvm_send_ipi_to_many (kvm, vector, vcpu_mask); - } +kvm_hv_send_ipi_to_many (kvm, vector, all_cpus ?NULL : sparse_banks, valid_bank_mask);Any objection to not using a ternary operator?
Quoted 5 lines
if (all_cpus) kvm_hv_send_ipi_to_many(kvm, vector, NULL, 0); else kvm_hv_send_ipi_to_many(kvm, vector, sparse_banks, valid_bank_mask);
Not at all,
Quoted 4 lines
Mostly because it's somewhat arbitrary that earlier code ensures
valid_bank_maskis set in the all_cpus=true case, e.g. arguably KVM doesn't need to do thevar_cntsanity check in theall_cpuscase:
Quoted 11 lines
all_cpus = send_ipi_ex.vp_set.format == HV_GENERIC_SET_ALL; if (all_cpus) goto check_and_send_ipi; valid_bank_mask = send_ipi_ex.vp_set.valid_bank_mask; if (hc->var_cnt != hweight64(valid_bank_mask)) return HV_STATUS_INVALID_HYPERCALL_INPUT; if (!hc->var_cnt) goto ret_success;
I think 'var_cnt' (== hweight64(valid_bank_mask)) has to be checked in 'all_cpus' case, especially in kvm_hv_flush_tlb(): the code which reads TLB flush entries will read them from the wrong offset (data_offset/ consumed_xmm_halves) otherwise. The problem is less severe in kvm_hv_send_ipi() as there's no data after CPU banks.
At the bare minimum, "KVM: x86: hyper-v: Handle HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} calls gently" patch from this series will have to be adjusted. I *think* mandating var_cnt==0 in 'all_cpus' is OK but I don't recall such requirement from TLFS, maybe it's safer to just adjust 'data_offset'/'consumed_xmm_halves' even in 'all_cpus' case.
Let me do some tests...
Signature · 1 line
Vitaly
Vitaly Kuznetsov
maintainerRe: [PATCH v11 16/46] KVM: x86: hyper-v: Don't use sparse_set_to_vcpu_mask() in kvm_hv_send_ipi()
Quoted 48 lines · Vitaly Kuznetsov
Quoted 1 line · Sean Christopherson
On Tue, Oct 04, 2022, Vitaly Kuznetsov wrote:
...
Quoted 12 lines
arch/x86/kvm/hyperv.c static u64 kvm_hv_send_ipi(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) · line 2136
Quoted 9 lines
arch/x86/kvm/hyperv.c +1
-7-if (all_cpus) { -kvm_send_ipi_to_many (kvm, vector,NULL ); - }else { -sparse_set_to_vcpu_mask (kvm, sparse_banks, valid_bank_mask, vcpu_mask); - -kvm_send_ipi_to_many (kvm, vector, vcpu_mask); - } +kvm_hv_send_ipi_to_many (kvm, vector, all_cpus ?NULL : sparse_banks, valid_bank_mask);Any objection to not using a ternary operator?
Quoted 5 lines
if (all_cpus) kvm_hv_send_ipi_to_many(kvm, vector, NULL, 0); else kvm_hv_send_ipi_to_many(kvm, vector, sparse_banks, valid_bank_mask);Not at all,
Quoted 4 lines
Mostly because it's somewhat arbitrary that earlier code ensures
valid_bank_maskis set in the all_cpus=true case, e.g. arguably KVM doesn't need to do thevar_cntsanity check in theall_cpuscase:Quoted 11 lines
all_cpus = send_ipi_ex.vp_set.format == HV_GENERIC_SET_ALL; if (all_cpus) goto check_and_send_ipi; valid_bank_mask = send_ipi_ex.vp_set.valid_bank_mask; if (hc->var_cnt != hweight64(valid_bank_mask)) return HV_STATUS_INVALID_HYPERCALL_INPUT; if (!hc->var_cnt) goto ret_success;I think '
var_cnt' (==hweight64(valid_bank_mask)) has to be checked in 'all_cpus' case, especially inkvm_hv_flush_tlb(): the code which reads TLB flush entries will read them from the wrong offset (data_offset/consumed_xmm_halves) otherwise. The problem is less severe inkvm_hv_send_ipi()as there's no data after CPU banks.
Quoted 7 lines
At the bare minimum, "KVM: x86: hyper-v: Handle HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} calls gently" patch from this series will have to be adjusted. I *think* mandating var_cnt==0 in '
all_cpus' is OK but I don't recall such requirement from TLFS, maybe it's safer to just adjust 'data_offset'/'consumed_xmm_halves' even in 'all_cpus' case.Let me do some tests...
"We can neither confirm nor deny the existence of the problem". Windows guests seem to be smart enough to avoid using *_EX hypercalls altogether for "all cpus" case (as non-ex versions are good enough). Let's keep allowing non-zero var_cnt for 'all cpus' case for now and think about hardening it later...
Signature · 1 line
Vitaly
Sean Christopherson
maintainerRe: [PATCH v11 16/46] KVM: x86: hyper-v: Don't use sparse_set_to_vcpu_mask() in kvm_hv_send_ipi()
Quoted 31 lines · Vitaly Kuznetsov
Quoted 29 lines · Vitaly Kuznetsov
Quoted 4 lines · Sean Christopherson
Mostly because it's somewhat arbitrary that earlier code ensures
valid_bank_maskis set in the all_cpus=true case, e.g. arguably KVM doesn't need to do thevar_cntsanity check in theall_cpuscase:Quoted 11 lines
all_cpus = send_ipi_ex.vp_set.format == HV_GENERIC_SET_ALL; if (all_cpus) goto check_and_send_ipi; valid_bank_mask = send_ipi_ex.vp_set.valid_bank_mask; if (hc->var_cnt != hweight64(valid_bank_mask)) return HV_STATUS_INVALID_HYPERCALL_INPUT; if (!hc->var_cnt) goto ret_success;I think '
var_cnt' (==hweight64(valid_bank_mask)) has to be checked in 'all_cpus' case, especially inkvm_hv_flush_tlb(): the code which reads TLB flush entries will read them from the wrong offset (data_offset/consumed_xmm_halves) otherwise. The problem is less severe inkvm_hv_send_ipi()as there's no data after CPU banks.At the bare minimum, "KVM: x86: hyper-v: Handle HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST{,EX} calls gently" patch from this series will have to be adjusted. I *think* mandating var_cnt==0 in '
all_cpus' is OK but I don't recall such requirement from TLFS, maybe it's safer to just adjust 'data_offset'/'consumed_xmm_halves' even in 'all_cpus' case.
Quoted 6 lines
Quoted 1 line
Let me do some tests...
"We can neither confirm nor deny the existence of the problem". Windows guests seem to be smart enough to avoid using
*_EXhypercalls altogether for "all cpus" case (as non-ex versions are good enough). Let's keep allowing non-zerovar_cntfor 'all cpus' case for now
Sounds good.
Quoted 1 line
and think about hardening it later...
Eh, no need to add more work for ourselves. I wasn't thinking about hardening so much as slightly simplifying KVM code.
Vitaly Kuznetsov
maintainerPATCH v11 17/46 KVM: x86: hyper-v: Create a separate fifo for L2 TLB flush
In reply to Vitaly KuznetsovTo handle L2 TLB flush requests, KVM needs to use a separate fifo from regular (L1) Hyper-V TLB flush requests: e.g. when a request to flush something in L2 is made, the target vCPU can transition from L2 to L1, receive a request to flush a GVA for L1 and then try to enter L2 back. The first request needs to be processed at this point. Similarly, requests to flush GVAs in L1 must wait until L2 exits to L1.
No functional change as KVM doesn't handle L2 TLB flush requests from L2 yet.
--- arch/x86/include/asm/kvm_host.h | 8 +++++++- arch/x86/kvm/hyperv.c | 11 +++++++---- arch/x86/kvm/hyperv.h | 19 ++++++++++++++++--- 3 files changed, 30 insertions(+), 8 deletions(-)
arch/x86/include/asm/kvm_host.h +7 -1
@@ -612,6 +612,12 @@ struct kvm_vcpu_hv_synic {*/ #define KVM_HV_TLB_FLUSHALL_ENTRY ((u64)-1) +enum hv_tlb_flush_fifos { +HV_L1_TLB_FLUSH_FIFO , +HV_L2_TLB_FLUSH_FIFO , +HV_NR_TLB_FLUSH_FIFOS , +}; +struct kvm_vcpu_hv_tlb_flush_fifo {spinlock_t write_lock; DECLARE_KFIFO(entries, u64, KVM_HV_TLB_FLUSH_FIFO_SIZE);
@@ -639,7 +645,7 @@ struct kvm_vcpu_hv {u32 nested_ebx;/* HYPERV_CPUID_NESTED_FEATURES.EBX */ } cpuid_cache; -struct kvm_vcpu_hv_tlb_flush_fifo tlb_flush_fifo; +struct kvm_vcpu_hv_tlb_flush_fifo tlb_flush_fifo[HV_NR_TLB_FLUSH_FIFOS ]; };/* Xen HVM per vcpu emulation context */
arch/x86/kvm/hyperv.c +7 -4
@@ -956,8 +956,10 @@ int kvm_hv_vcpu_init(struct kvm_vcpu *vcpu) hv_vcpu->vp_index = vcpu->vcpu_idx; -INIT_KFIFO (hv_vcpu->tlb_flush_fifo.entries); -spin_lock_init (&hv_vcpu->tlb_flush_fifo.write_lock); +for (i =0 ; i <HV_NR_TLB_FLUSH_FIFOS ; i++) { +INIT_KFIFO (hv_vcpu->tlb_flush_fifo[i].entries); +spin_lock_init (&hv_vcpu->tlb_flush_fifo[i].write_lock); + }return 0 ; }
@@ -1839,7 +1841,8 @@ static void hv_tlb_flush_enqueue(struct kvm_vcpu *vcpu, u64 *entries, int count)if (!hv_vcpu)return ; - tlb_flush_fifo = &hv_vcpu->tlb_flush_fifo; +/* kvm_hv_flush_tlb() is not ready to handle requests for L2s yet */ + tlb_flush_fifo = &hv_vcpu->tlb_flush_fifo[HV_L1_TLB_FLUSH_FIFO ];spin_lock (&tlb_flush_fifo->write_lock);
@@ -1874,7 +1877,7 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu)if (!tdp_enabled || !hv_vcpu)return -EINVAL ; - tlb_flush_fifo = &hv_vcpu->tlb_flush_fifo; + tlb_flush_fifo =kvm_hv_get_tlb_flush_fifo (vcpu,is_guest_mode (vcpu)); count =kfifo_out (&tlb_flush_fifo->entries, entries,KVM_HV_TLB_FLUSH_FIFO_SIZE );
arch/x86/kvm/hyperv.h +16 -3
@@ -22,6 +22,7 @@#define __ARCH_X86_KVM_HYPERV_H__#include <linux/kvm_host.h> +#include "x86.h" /* "Hv#1" signature */ #define HYPERV_CPUID_SIGNATURE_EAX 0x31237648
@@ -151,15 +152,27 @@ int kvm_vm_ioctl_hv_eventfd(struct kvm *kvm, struct kvm_hyperv_eventfd *args);int kvm_get_hv_cpuid (struct kvm_vcpu *vcpu,struct kvm_cpuid2 *cpuid,struct kvm_cpuid_entry2 __user *entries); +static inline struct kvm_vcpu_hv_tlb_flush_fifo *kvm_hv_get_tlb_flush_fifo (struct kvm_vcpu *vcpu, +bool is_guest_mode) +{ +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); +int i = is_guest_mode ?HV_L2_TLB_FLUSH_FIFO : +HV_L1_TLB_FLUSH_FIFO ; + +/* KVM does not handle L2 TLB flush requests yet */ +WARN_ON_ONCE (i !=HV_L1_TLB_FLUSH_FIFO ); + +return &hv_vcpu->tlb_flush_fifo[i]; +} +static inline void kvm_hv_vcpu_purge_flush_tlb (struct kvm_vcpu *vcpu) {struct kvm_vcpu_hv_tlb_flush_fifo *tlb_flush_fifo; -struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); -if (!hv_vcpu || !kvm_check_request (KVM_REQ_HV_TLB_FLUSH , vcpu)) +if (!to_hv_vcpu (vcpu) || !kvm_check_request (KVM_REQ_HV_TLB_FLUSH , vcpu))return ; - tlb_flush_fifo = &hv_vcpu->tlb_flush_fifo; + tlb_flush_fifo =kvm_hv_get_tlb_flush_fifo (vcpu,is_guest_mode (vcpu));kfifo_reset_out (&tlb_flush_fifo->entries); }
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 18/46 KVM: x86: hyper-v: Use preallocated buffer in 'struct kvm_vcpu_hv' instead of on-stack 'sparse_banks'
In reply to Vitaly KuznetsovTo make kvm_hv_flush_tlb() ready to handle L2 TLB flush requests, KVM needs to allow for all 64 sparse vCPU banks regardless of KVM_MAX_VCPUs as L1 may use vCPU overcommit for L2. To avoid growing on-stack allocation, make 'sparse_banks' part of per-vCPU 'struct kvm_vcpu_hv' which is allocated dynamically.
Note: sparse_set_to_vcpu_mask() can't currently be used to handle L2 requests as KVM does not keep L2 VM_ID -> L2 VCPU_ID -> L1 vCPU mappings, i.e. its vp_bitmap array is still bounded by the number of L1 vCPUs and so can remain an on-stack allocation.
--- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/hyperv.c | 6 ++++-- 2 files changed, 7 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +3 -0
@@ -646,6 +646,9 @@ struct kvm_vcpu_hv { } cpuid_cache;struct kvm_vcpu_hv_tlb_flush_fifo tlb_flush_fifo[HV_NR_TLB_FLUSH_FIFOS ]; + +/* Preallocated buffer for handling hypercalls passing sparse vCPU set */ +u64 sparse_banks[HV_MAX_SPARSE_VCPU_BANKS ]; };/* Xen HVM per vcpu emulation context */
arch/x86/kvm/hyperv.c +4 -2
@@ -1907,6 +1907,8 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu)static u64 kvm_hv_flush_tlb (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) { +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); +u64 *sparse_banks = hv_vcpu->sparse_banks;struct kvm *kvm = vcpu->kvm;struct hv_tlb_flush_ex flush_ex;struct hv_tlb_flush flush;
@@ -1920,7 +1922,6 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)u64 __tlb_flush_entries[KVM_HV_TLB_FLUSH_FIFO_SIZE -1 ];u64 *tlb_flush_entries;u64 valid_bank_mask; -u64 sparse_banks[KVM_HV_MAX_SPARSE_VCPU_SET_BITS ];struct kvm_vcpu *v;unsigned long i;bool all_cpus;
@@ -2072,11 +2073,12 @@ static void kvm_hv_send_ipi_to_many(struct kvm *kvm, u32 vector,static u64 kvm_hv_send_ipi (struct kvm_vcpu *vcpu,struct kvm_hv_hcall *hc) { +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); +u64 *sparse_banks = hv_vcpu->sparse_banks;struct kvm *kvm = vcpu->kvm;struct hv_send_ipi_ex send_ipi_ex;struct hv_send_ipi send_ipi;u64 valid_bank_mask; -u64 sparse_banks[KVM_HV_MAX_SPARSE_VCPU_SET_BITS ];u32 vector;bool all_cpus;
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 19/46 KVM: nVMX: Keep track of hv_vm_id/hv_vp_id when eVMCS is in use
In reply to Vitaly KuznetsovTo handle L2 TLB flush requests, KVM needs to keep track of L2's VM_ID/ VP_IDs which are set by L1 hypervisor. 'Partition assist page' address is also needed to handle post-flush exit to L1 upon request.
--- arch/x86/include/asm/kvm_host.h | 6 ++++++ arch/x86/kvm/vmx/nested.c | 15 +++++++++++++++ 2 files changed, 21 insertions(+)
arch/x86/include/asm/kvm_host.h +6 -0
@@ -649,6 +649,12 @@ struct kvm_vcpu_hv {/* Preallocated buffer for handling hypercalls passing sparse vCPU set */ u64 sparse_banks[HV_MAX_SPARSE_VCPU_BANKS ]; + +struct { +u64 pa_page_gpa; +u64 vm_id; +u32 vp_id; + } nested; };/* Xen HVM per vcpu emulation context */
arch/x86/kvm/vmx/nested.c +15 -0
@@ -224,6 +224,7 @@ static void vmx_disable_shadow_vmcs(struct vcpu_vmx *vmx)static inline void nested_release_evmcs (struct kvm_vcpu *vcpu) { +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu);struct vcpu_vmx *vmx =to_vmx (vcpu);if (evmptr_is_valid (vmx->nested.hv_evmcs_vmptr)) {
@@ -232,6 +233,12 @@ static inline void nested_release_evmcs(struct kvm_vcpu *vcpu) } vmx->nested.hv_evmcs_vmptr =EVMPTR_INVALID ; + +if (hv_vcpu) { + hv_vcpu->nested.pa_page_gpa =INVALID_GPA ; + hv_vcpu->nested.vm_id =0 ; + hv_vcpu->nested.vp_id =0 ; + } } static void vmx_sync_vmcs_host_state(struct vcpu_vmx *vmx,
@@ -1556,11 +1563,19 @@ static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx, u32 hv_clean_fields {struct vmcs12 *vmcs12 = vmx->nested.cached_vmcs12;struct hv_enlightened_vmcs *evmcs = vmx->nested.hv_evmcs; +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (&vmx->vcpu);/* HV_VMX_ENLIGHTENED_CLEAN_FIELD_NONE */ vmcs12->tpr_threshold = evmcs->tpr_threshold; vmcs12->guest_rip = evmcs->guest_rip; +if (unlikely (!(hv_clean_fields & +HV_VMX_ENLIGHTENED_CLEAN_FIELD_ENLIGHTENMENTSCONTROL ))) { + hv_vcpu->nested.pa_page_gpa = evmcs->partition_assist_page; + hv_vcpu->nested.vm_id = evmcs->hv_vm_id; + hv_vcpu->nested.vp_id = evmcs->hv_vp_id; + } +if (unlikely (!(hv_clean_fields &HV_VMX_ENLIGHTENED_CLEAN_FIELD_GUEST_BASIC ))) { vmcs12->guest_rsp = evmcs->guest_rsp;
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 20/46 KVM: nSVM: Keep track of Hyper-V hv_vm_id/hv_vp_id
In reply to Vitaly KuznetsovSimilar to nSVM, KVM needs to know L2's VM_ID/VP_ID and Partition assist page address to handle L2 TLB flush requests.
--- arch/x86/kvm/svm/hyperv.h | 15 +++++++++++++++ arch/x86/kvm/svm/nested.c | 2 ++ 2 files changed, 17 insertions(+)
arch/x86/kvm/svm/hyperv.h +15 -0
@@ -9,5 +9,20 @@#include <asm/mshyperv.h> #include "../hyperv.h" +#include "svm.h" + +static inline void nested_svm_hv_update_vm_vp_ids (struct kvm_vcpu *vcpu) +{ +struct vcpu_svm *svm =to_svm (vcpu); +struct hv_vmcb_enlightenments *hve = &svm->nested.ctl.hv_enlightenments; +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); + +if (!hv_vcpu) +return ; + + hv_vcpu->nested.pa_page_gpa = hve->partition_assist_page; + hv_vcpu->nested.vm_id = hve->hv_vm_id; + hv_vcpu->nested.vp_id = hve->hv_vp_id; +}#endif /* __ARCH_X86_KVM_SVM_HYPERV_H__ */
arch/x86/kvm/svm/nested.c +2 -0
@@ -799,6 +799,8 @@ int enter_svm_guest_mode(struct kvm_vcpu *vcpu, u64 vmcb12_gpa,if (kvm_vcpu_apicv_active (vcpu))kvm_make_request (KVM_REQ_APICV_UPDATE , vcpu); +nested_svm_hv_update_vm_vp_ids (vcpu); +return 0 ; }
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 21/46 KVM: x86: Introduce .hv_inject_synthetic_vmexit_post_tlb_flush() nested hook
In reply to Vitaly KuznetsovHyper-V supports injecting synthetic L2->L1 exit after performing L2 TLB flush operation but the procedure is vendor specific. Introduce .hv_inject_synthetic_vmexit_post_tlb_flush nested hook for it.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/Makefile | 3 ++- arch/x86/kvm/svm/hyperv.c | 11 +++++++++++ arch/x86/kvm/svm/hyperv.h | 2 ++ arch/x86/kvm/svm/nested.c | 1 + arch/x86/kvm/vmx/hyperv.c | 4 ++++ arch/x86/kvm/vmx/hyperv.h | 1 + arch/x86/kvm/vmx/nested.c | 1 + 8 files changed, 23 insertions(+), 1 deletion(-) create mode 100644 arch/x86/kvm/svm/hyperv.c
arch/x86/include/asm/kvm_host.h +1 -0
@@ -1694,6 +1694,7 @@ struct kvm_x86_nested_ops {int (*enable_evmcs)(struct kvm_vcpu *vcpu,uint16_t *vmcs_version);uint16_t (*get_evmcs_version)(struct kvm_vcpu *vcpu); +void (*hv_inject_synthetic_vmexit_post_tlb_flush)(struct kvm_vcpu *vcpu); };struct kvm_x86_init_ops {
arch/x86/kvm/Makefile +2 -1
@@ -25,7 +25,8 @@ kvm-intel-y += vmx/vmx.o vmx/vmenter.o vmx/pmu_intel.o vmx/vmcs12.o \ vmx/hyperv.o vmx/nested.o vmx/posted_intr.o kvm-intel-$(CONFIG_X86_SGX_KVM) += vmx/sgx.o -kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o svm/sev.o +kvm-amd-y += svm/svm.o svm/vmenter.o svm/pmu.o svm/nested.o svm/avic.o \ + svm/sev.o svm/hyperv.o ifdef CONFIG_HYPERV kvm-amd-y += svm/svm_onhyperv.o
arch/x86/kvm/svm/hyperv.c new file +11 -0
@@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* AMD SVM specific code for Hyper-V on KVM. +* +* Copyright 2022 Red Hat, Inc. and/or its affiliates. +*/ +#include "hyperv.h" + +void svm_hv_inject_synthetic_vmexit_post_tlb_flush (struct kvm_vcpu *vcpu) +{ +}
arch/x86/kvm/svm/hyperv.h +2 -0
@@ -25,4 +25,6 @@ static inline void nested_svm_hv_update_vm_vp_ids(struct kvm_vcpu *vcpu) hv_vcpu->nested.vp_id = hve->hv_vp_id; } +void svm_hv_inject_synthetic_vmexit_post_tlb_flush (struct kvm_vcpu *vcpu); +#endif /* __ARCH_X86_KVM_SVM_HYPERV_H__ */
arch/x86/kvm/svm/nested.c +1 -0
@@ -1721,4 +1721,5 @@ struct kvm_x86_nested_ops svm_nested_ops = { .get_nested_state_pages = svm_get_nested_state_pages, .get_state = svm_get_nested_state, .set_state = svm_set_nested_state, + .hv_inject_synthetic_vmexit_post_tlb_flush = svm_hv_inject_synthetic_vmexit_post_tlb_flush, };
arch/x86/kvm/vmx/hyperv.c +4 -0
@@ -506,3 +506,7 @@ int nested_enable_evmcs(struct kvm_vcpu *vcpu,return 0 ; } + +void vmx_hv_inject_synthetic_vmexit_post_tlb_flush (struct kvm_vcpu *vcpu) +{ +}
arch/x86/kvm/vmx/hyperv.h +1 -0
@@ -243,5 +243,6 @@ int nested_enable_evmcs(struct kvm_vcpu *vcpu, uint16_t *vmcs_version);void nested_evmcs_filter_control_msr (struct kvm_vcpu *vcpu,u32 msr_index,u64 *pdata);int nested_evmcs_check_controls (struct vmcs12 *vmcs12); +void vmx_hv_inject_synthetic_vmexit_post_tlb_flush (struct kvm_vcpu *vcpu);#endif /* __KVM_X86_VMX_HYPERV_H */
arch/x86/kvm/vmx/nested.c +1 -0
@@ -6993,4 +6993,5 @@ struct kvm_x86_nested_ops vmx_nested_ops = { .write_log_dirty = nested_vmx_write_pml_buffer, .enable_evmcs = nested_enable_evmcs, .get_evmcs_version = nested_get_evmcs_version, + .hv_inject_synthetic_vmexit_post_tlb_flush = vmx_hv_inject_synthetic_vmexit_post_tlb_flush, };
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 22/46 KVM: x86: hyper-v: Introduce kvm_hv_is_tlb_flush_hcall()
In reply to Vitaly KuznetsovThe newly introduced helper checks whether vCPU is performing a Hyper-V TLB flush hypercall. This is required to filter out L2 TLB flush hypercalls for processing.
--- arch/x86/kvm/hyperv.h | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
arch/x86/kvm/hyperv.h +17 -0
@@ -177,6 +177,23 @@ static inline void kvm_hv_vcpu_purge_flush_tlb(struct kvm_vcpu *vcpu) kfifo_reset_out(&tlb_flush_fifo->entries); } +static inline bool kvm_hv_is_tlb_flush_hcall (struct kvm_vcpu *vcpu) +{ +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); +u16 code; + +if (!hv_vcpu) +return false; + + code =is_64_bit_hypercall (vcpu) ?kvm_rcx_read (vcpu) : +kvm_rax_read (vcpu); + +return (code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE || + code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST || + code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE_EX || + code ==HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX ); +} +int kvm_hv_vcpu_flush_tlb (struct kvm_vcpu *vcpu);#endif
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 23/46 KVM: x86: hyper-v: L2 TLB flush
In reply to Vitaly KuznetsovHandle L2 TLB flush requests by going through all vCPUs and checking whether there are vCPUs running the same VM_ID with a VP_ID specified in the requests. Perform synthetic exit to L2 upon finish.
Note, while checking VM_ID/VP_ID of running vCPUs seem to be a bit racy, we count on the fact that KVM flushes the whole L2 VPID upon transition. Also, KVM_REQ_HV_TLB_FLUSH request needs to be done upon transition between L1 and L2 to make sure all pending requests are always processed.
For the reference, Hyper-V TLFS refers to the feature as "Direct Virtual Flush".
Note, nVMX/nSVM code does not handle VMCALL/VMMCALL from L2 yet.
--- arch/x86/kvm/hyperv.c | 82 ++++++++++++++++++++++++++++++++++++------- arch/x86/kvm/hyperv.h | 3 -- arch/x86/kvm/trace.h | 21 ++++++----- 3 files changed, 82 insertions(+), 24 deletions(-)
arch/x86/kvm/hyperv.c +69 -13
@@ -34,6 +34,7 @@#include <linux/eventfd.h> #include <asm/apicdef.h> +#include <asm/mshyperv.h> #include <trace/events/kvm.h> #include "trace.h"
@@ -1832,18 +1833,16 @@ static int kvm_hv_get_tlb_flush_entries(struct kvm *kvm, struct kvm_hv_hcall *hc entries, consumed_xmm_halves, offset); } -static void hv_tlb_flush_enqueue (struct kvm_vcpu *vcpu,u64 *entries,int count) +static void hv_tlb_flush_enqueue (struct kvm_vcpu *vcpu, +struct kvm_vcpu_hv_tlb_flush_fifo *tlb_flush_fifo, +u64 *entries,int count) { -struct kvm_vcpu_hv_tlb_flush_fifo *tlb_flush_fifo;struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu);u64 flush_all_entry =KVM_HV_TLB_FLUSHALL_ENTRY ;if (!hv_vcpu)return ; -/* kvm_hv_flush_tlb() is not ready to handle requests for L2s yet */ - tlb_flush_fifo = &hv_vcpu->tlb_flush_fifo[HV_L1_TLB_FLUSH_FIFO ]; -spin_lock (&tlb_flush_fifo->write_lock);/*
@@ -1913,6 +1912,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)struct hv_tlb_flush_ex flush_ex;struct hv_tlb_flush flush;DECLARE_BITMAP (vcpu_mask,KVM_MAX_VCPUS ); +struct kvm_vcpu_hv_tlb_flush_fifo *tlb_flush_fifo;/* * Normally, there can be no more than 'KVM_HV_TLB_FLUSH_FIFO_SIZE' * entries on the TLB flush fifo. The last entry, however, needs to be
@@ -1956,7 +1956,8 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) }trace_kvm_hv_flush_tlb (flush.processor_mask, - flush.address_space, flush.flags); + flush.address_space, flush.flags, +is_guest_mode (vcpu)); valid_bank_mask =BIT_ULL (0 ); sparse_banks[0 ] = flush.processor_mask;
@@ -1987,7 +1988,7 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)trace_kvm_hv_flush_tlb_ex (flush_ex.hv_vp_set.valid_bank_mask, flush_ex.hv_vp_set.format, flush_ex.address_space, - flush_ex.flags); + flush_ex.flags,is_guest_mode (vcpu)); valid_bank_mask = flush_ex.hv_vp_set.valid_bank_mask; all_cpus = flush_ex.hv_vp_set.format !=
@@ -2025,19 +2026,57 @@ static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc)* vcpu->arch.cr3 may not be up-to-date for running vCPUs so we can't * analyze it here, flush TLB regardless of the specified address space. */ -if (all_cpus) { -kvm_for_each_vcpu (i, v, kvm) -hv_tlb_flush_enqueue (v, tlb_flush_entries, hc->rep_cnt); +if (all_cpus && !is_guest_mode (vcpu)) { +kvm_for_each_vcpu (i, v, kvm) { + tlb_flush_fifo =kvm_hv_get_tlb_flush_fifo (v, false); +hv_tlb_flush_enqueue (v, tlb_flush_fifo, + tlb_flush_entries, hc->rep_cnt); + }kvm_make_all_cpus_request (kvm,KVM_REQ_HV_TLB_FLUSH ); - }else { + }else if (!is_guest_mode (vcpu)) {sparse_set_to_vcpu_mask (kvm, sparse_banks, valid_bank_mask, vcpu_mask);for_each_set_bit (i, vcpu_mask,KVM_MAX_VCPUS ) { v =kvm_get_vcpu (kvm, i);if (!v)continue ; -hv_tlb_flush_enqueue (v, tlb_flush_entries, hc->rep_cnt); + tlb_flush_fifo =kvm_hv_get_tlb_flush_fifo (v, false); +hv_tlb_flush_enqueue (v, tlb_flush_fifo, + tlb_flush_entries, hc->rep_cnt); + } + +kvm_make_vcpus_request_mask (kvm,KVM_REQ_HV_TLB_FLUSH , vcpu_mask); + }else { +struct kvm_vcpu_hv *hv_v; + +bitmap_zero (vcpu_mask,KVM_MAX_VCPUS ); + +kvm_for_each_vcpu (i, v, kvm) { + hv_v =to_hv_vcpu (v); + +/* +* The following check races with nested vCPUs entering/exiting +* and/or migrating between L1's vCPUs, however the only case when +* KVM *must* flush the TLB is when the target L2 vCPU keeps +* running on the same L1 vCPU from the moment of the request until +* kvm_hv_flush_tlb() returns. TLB is fully flushed in all other +* cases, e.g. when the target L2 vCPU migrates to a different L1 +* vCPU or when the corresponding L1 vCPU temporary switches to a +* different L2 vCPU while the request is being processed. +*/ +if (!hv_v || hv_v->nested.vm_id != hv_vcpu->nested.vm_id) +continue ; + +if (!all_cpus && + !hv_is_vp_in_sparse_set (hv_v->nested.vp_id, valid_bank_mask, + sparse_banks)) +continue ; + +__set_bit (i, vcpu_mask); + tlb_flush_fifo =kvm_hv_get_tlb_flush_fifo (v, true); +hv_tlb_flush_enqueue (v, tlb_flush_fifo, + tlb_flush_entries, hc->rep_cnt); }kvm_make_vcpus_request_mask (kvm,KVM_REQ_HV_TLB_FLUSH , vcpu_mask);
@@ -2225,10 +2264,27 @@ static void kvm_hv_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result)static int kvm_hv_hypercall_complete (struct kvm_vcpu *vcpu,u64 result) { +int ret; +trace_kvm_hv_hypercall_done (result);kvm_hv_hypercall_set_result (vcpu, result); ++vcpu->stat.hypercalls; -return kvm_skip_emulated_instruction (vcpu); + ret =kvm_skip_emulated_instruction (vcpu); + +if (unlikely (hv_result_success (result) &&is_guest_mode (vcpu) + &&kvm_hv_is_tlb_flush_hcall (vcpu))) { +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); +u32 tlb_lock_count; + +if (unlikely (kvm_read_guest (vcpu->kvm, hv_vcpu->nested.pa_page_gpa, + &tlb_lock_count,sizeof (tlb_lock_count)))) +kvm_inject_gp (vcpu,0 ); + +if (tlb_lock_count) + kvm_x86_ops.nested_ops->hv_inject_synthetic_vmexit_post_tlb_flush (vcpu); + } + +return ret; }static int kvm_hv_hypercall_complete_userspace (struct kvm_vcpu *vcpu)
arch/x86/kvm/hyperv.h +0 -3
@@ -159,9 +159,6 @@ static inline struct kvm_vcpu_hv_tlb_flush_fifo *kvm_hv_get_tlb_flush_fifo(strucint i = is_guest_mode ?HV_L2_TLB_FLUSH_FIFO :HV_L1_TLB_FLUSH_FIFO ; -/* KVM does not handle L2 TLB flush requests yet */ -WARN_ON_ONCE (i !=HV_L1_TLB_FLUSH_FIFO ); -return &hv_vcpu->tlb_flush_fifo[i]; }
arch/x86/kvm/trace.h +13 -8
@@ -1547,38 +1547,41 @@ TRACE_EVENT(kvm_hv_timer_state,* Tracepoint for kvm_hv_flush_tlb. */ TRACE_EVENT (kvm_hv_flush_tlb, - TP_PROTO(u64 processor_mask, u64 address_space, u64 flags), -TP_ARGS (processor_mask, address_space, flags), + TP_PROTO(u64 processor_mask, u64 address_space, u64 flags, bool guest_mode), +TP_ARGS (processor_mask, address_space, flags, guest_mode),TP_STRUCT__entry ( __field(u64, processor_mask) __field(u64, address_space) __field(u64, flags) +__field (bool, guest_mode) ),TP_fast_assign ( __entry->processor_mask = processor_mask; __entry->address_space = address_space; __entry->flags = flags; + __entry->guest_mode = guest_mode; ), -TP_printk ("processor_mask 0x%llx address_space 0x%llx flags 0x%llx" , +TP_printk ("processor_mask 0x%llx address_space 0x%llx flags 0x%llx %s" , __entry->processor_mask, __entry->address_space, - __entry->flags) + __entry->flags, __entry->guest_mode ?"(L2)" :"" ) );/* * Tracepoint for kvm_hv_flush_tlb_ex. */ TRACE_EVENT(kvm_hv_flush_tlb_ex, - TP_PROTO(u64 valid_bank_mask, u64 format, u64 address_space, u64 flags), - TP_ARGS(valid_bank_mask, format, address_space, flags), + TP_PROTO(u64 valid_bank_mask, u64 format, u64 address_space, u64 flags, bool guest_mode), + TP_ARGS(valid_bank_mask, format, address_space, flags, guest_mode), TP_STRUCT__entry( __field(u64, valid_bank_mask) __field(u64, format) __field(u64, address_space) __field(u64, flags) + __field(bool, guest_mode) ), TP_fast_assign(
@@ -1586,12 +1589,14 @@ TRACE_EVENT(kvm_hv_flush_tlb_ex, __entry->format = format; __entry->address_space = address_space; __entry->flags = flags; + __entry->guest_mode = guest_mode; ),TP_printk ("valid_bank_mask 0x%llx format 0x%llx " -"address_space 0x%llx flags 0x%llx" , +"address_space 0x%llx flags 0x%llx %s" , __entry->valid_bank_mask, __entry->format, - __entry->address_space, __entry->flags) + __entry->address_space, __entry->flags, + __entry->guest_mode ?"(L2)" :"" ) );/*
Signature · 1 line
2.37.3
Sean Christopherson
maintainerRe: [PATCH v11 23/46] KVM: x86: hyper-v: L2 TLB flush
Quoted 4 lines · Vitaly Kuznetsov
arch/x86/kvm/hyperv.c +0
-0@@ -2225,10 +2264,27 @@ static void kvm_hv_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result)static int kvm_hv_hypercall_complete (struct kvm_vcpu *vcpu,u64 result) {
arch/x86/kvm/hyperv.c static void kvm_hv_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result) · line 2275
Quoted 10 lines
arch/x86/kvm/hyperv.c +6
-1+int ret; +trace_kvm_hv_hypercall_done (result);kvm_hv_hypercall_set_result (vcpu, result); ++vcpu->stat.hypercalls; -return kvm_skip_emulated_instruction (vcpu); + ret =kvm_skip_emulated_instruction (vcpu); + +if (unlikely (hv_result_success (result) &&is_guest_mode (vcpu) + &&kvm_hv_is_tlb_flush_hcall (vcpu))) {
"&&" goes on the previous line.
arch/x86/kvm/hyperv.c static void kvm_hv_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result) · line 2279
Quoted 4 lines
arch/x86/kvm/hyperv.c +4
-0+struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); +u32 tlb_lock_count; + + if (unlikely(kvm_read_guest(vcpu->kvm, hv_vcpu->nested.pa_page_gpa,
Nit, I'd say leave off the "unlikely", the hints almost never provide meaningful performance benefits, e.g. code generation is identical with and without the unlikely, and IMO the extra line length and parantheses depth makes the code harder to read.
arch/x86/kvm/hyperv.c static void kvm_hv_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result) · line 2281
Quoted 2 lines
arch/x86/kvm/hyperv.c +2
-0+ &tlb_lock_count, sizeof(tlb_lock_count)))) +kvm_inject_gp (vcpu,0 );
This will inject a #GP on the _next_ instruction. That seems wrong. And why #GP in the first place? E.g. if userspace yanks out the memslot, injecting #GP into the guest is less-than-ideal behavior.
What about reading tlb_lock_count before skipping the hypercall, e.g.
static int kvm_hv_hypercall_complete(struct kvm_vcpu *vcpu, u64 result)
{
u32 tlb_lock_count = 0;
int ret;
if (hv_result_success(result) && is_guest_mode(vcpu) &&
kvm_hv_is_tlb_flush_hcall(vcpu) &&
kvm_read_guest(vcpu->kvm, to_hv_vcpu(vcpu)->nested.pa_page_gpa,
&tlb_lock_count, sizeof(tlb_lock_count)))
result = HV_STATUS_INVALID_HYPERCALL_INPUT;
trace_kvm_hv_hypercall_done(result);
kvm_hv_hypercall_set_result(vcpu, result);
++vcpu->stat.hypercalls;
ret = kvm_skip_emulated_instruction(vcpu);
if (tlb_lock_count)
kvm_x86_ops.nested_ops->hv_inject_synthetic_vmexit_post_tlb_flush(vcpu);
return ret;
}Review static void kvm_hv_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result) · line 2283
Quoted 2 lines
(quoted diff) +2
-0+ + if (tlb_lock_count)
tlb_lock_count will be uninitialized if kvm_read_guest() fails.
arch/x86/kvm/hyperv.c static void kvm_hv_hypercall_set_result(struct kvm_vcpu *vcpu, u64 result) · line 2284
Quoted 1 line
arch/x86/kvm/hyperv.c +1
-0+ kvm_x86_ops.nested_ops->hv_inject_synthetic_vmexit_post_tlb_flush (vcpu);
Ugh, kvm_skip_emulated_instruction() is flawed. If skipping the emulated instruction fails, i.e. if EMULTYPE_SKIP emulation fails, then synthesizing a VM-Exit is technically wrong. But kvm_skip_emulated_instruction() also returns '0' for a KVM_EXIT_DEBUG, which happens after skipping the instruction.
Not worth handling here, e.g. nested_svm_vmrun() has the same "bug". Failure is effectively limited to old AMD CPUs, and userspace is likely goiing to kill the VM anyways.
Vitaly Kuznetsov
maintainerPATCH v11 24/46 KVM: x86: hyper-v: Introduce fast guest_hv_cpuid_has_l2_tlb_flush() check
In reply to Vitaly KuznetsovIntroduce a helper to quickly check if KVM needs to handle VMCALL/VMMCALL from L2 in L0 to process L2 TLB flush requests.
--- arch/x86/kvm/hyperv.h | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/x86/kvm/hyperv.h +8 -0
@@ -174,6 +174,14 @@ static inline void kvm_hv_vcpu_purge_flush_tlb(struct kvm_vcpu *vcpu)kfifo_reset_out (&tlb_flush_fifo->entries); } +static inline bool guest_hv_cpuid_has_l2_tlb_flush (struct kvm_vcpu *vcpu) +{ +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); + +return hv_vcpu && + (hv_vcpu->cpuid_cache.nested_eax &HV_X64_NESTED_DIRECT_FLUSH ); +} +static inline bool kvm_hv_is_tlb_flush_hcall (struct kvm_vcpu *vcpu) {struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu);
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 25/46 KVM: nVMX: hyper-v: Cache VP assist page in 'struct kvm_vcpu_hv'
In reply to Vitaly KuznetsovIn preparation to enabling L2 TLB flush, cache VP assist page in 'struct kvm_vcpu_hv'. While on it, rename nested_enlightened_vmentry() to nested_get_evmptr() and make it return eVMCS GPA directly.
No functional change intended.
--- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/hyperv.c | 10 ++++++---- arch/x86/kvm/hyperv.h | 3 +-- arch/x86/kvm/vmx/hyperv.c | 21 +++++++-------------- arch/x86/kvm/vmx/hyperv.h | 2 +- arch/x86/kvm/vmx/nested.c | 6 +++--- 6 files changed, 20 insertions(+), 24 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -650,6 +650,8 @@ struct kvm_vcpu_hv {/* Preallocated buffer for handling hypercalls passing sparse vCPU set */ u64 sparse_banks[HV_MAX_SPARSE_VCPU_BANKS ]; +struct hv_vp_assist_page vp_assist_page; +struct {u64 pa_page_gpa;u64 vm_id;
arch/x86/kvm/hyperv.c +6 -4
@@ -900,13 +900,15 @@ bool kvm_hv_assist_page_enabled(struct kvm_vcpu *vcpu) }EXPORT_SYMBOL_GPL (kvm_hv_assist_page_enabled); -bool kvm_hv_get_assist_page (struct kvm_vcpu *vcpu, -struct hv_vp_assist_page *assist_page) +bool kvm_hv_get_assist_page (struct kvm_vcpu *vcpu) { -if (!kvm_hv_assist_page_enabled (vcpu)) +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); + +if (!hv_vcpu || !kvm_hv_assist_page_enabled (vcpu))return false; +return !kvm_read_guest_cached (vcpu->kvm, &vcpu->arch.pv_eoi.data, - assist_page,sizeof (*assist_page)); + &hv_vcpu->vp_assist_page,sizeof (struct hv_vp_assist_page )); }EXPORT_SYMBOL_GPL (kvm_hv_get_assist_page);
arch/x86/kvm/hyperv.h +1 -2
@@ -108,8 +108,7 @@ int kvm_hv_activate_synic(struct kvm_vcpu *vcpu, bool dont_zero_synic_pages);void kvm_hv_vcpu_uninit (struct kvm_vcpu *vcpu);bool kvm_hv_assist_page_enabled (struct kvm_vcpu *vcpu); -bool kvm_hv_get_assist_page (struct kvm_vcpu *vcpu, -struct hv_vp_assist_page *assist_page); +bool kvm_hv_get_assist_page (struct kvm_vcpu *vcpu);static inline struct kvm_vcpu_hv_stimer *to_hv_stimer (struct kvm_vcpu *vcpu,int timer_index)
arch/x86/kvm/vmx/hyperv.c +7 -14
@@ -321,24 +321,17 @@ const struct evmcs_field vmcs_field_to_evmcs_1[] = { };const unsigned int nr_evmcs_1_fields =ARRAY_SIZE (vmcs_field_to_evmcs_1); -bool nested_enlightened_vmentry (struct kvm_vcpu *vcpu,u64 *evmcs_gpa) +u64 nested_get_evmptr (struct kvm_vcpu *vcpu) { -struct hv_vp_assist_page assist_page; - - *evmcs_gpa =-1ull ; - -if (unlikely (!kvm_hv_get_assist_page (vcpu, &assist_page))) -return false; - -if (unlikely (!assist_page.enlighten_vmentry)) -return false; +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); -if (unlikely (!evmptr_is_valid (assist_page.current_nested_vmcs))) -return false; +if (unlikely (!kvm_hv_get_assist_page (vcpu))) +return EVMPTR_INVALID ; - *evmcs_gpa = assist_page.current_nested_vmcs; +if (unlikely (!hv_vcpu->vp_assist_page.enlighten_vmentry)) +return EVMPTR_INVALID ; -return true; +return hv_vcpu->vp_assist_page.current_nested_vmcs; }uint16_t nested_get_evmcs_version (struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx/hyperv.h +1 -1
@@ -237,7 +237,7 @@ enum nested_evmptrld_status { EVMPTRLD_ERROR, }; -bool nested_enlightened_vmentry (struct kvm_vcpu *vcpu,u64 *evmcs_gpa); +u64 nested_get_evmptr (struct kvm_vcpu *vcpu);uint16_t nested_get_evmcs_version (struct kvm_vcpu *vcpu);int nested_enable_evmcs (struct kvm_vcpu *vcpu,uint16_t *vmcs_version);
arch/x86/kvm/vmx/nested.c +3 -3
@@ -1991,7 +1991,8 @@ static enum nested_evmptrld_status nested_vmx_handle_enlightened_vmptrld(if (likely (!guest_cpuid_has_evmcs (vcpu)))return EVMPTRLD_DISABLED ; -if (!nested_enlightened_vmentry (vcpu, &evmcs_gpa)) { + evmcs_gpa =nested_get_evmptr (vcpu); +if (!evmptr_is_valid (evmcs_gpa)) {nested_release_evmcs (vcpu);return EVMPTRLD_DISABLED ; }
@@ -5219,7 +5220,6 @@ static int handle_vmclear(struct kvm_vcpu *vcpu)struct vcpu_vmx *vmx =to_vmx (vcpu);u32 zero =0 ;gpa_t vmptr; -u64 evmcs_gpa;int r;if (!nested_vmx_check_permission (vcpu))
@@ -5245,7 +5245,7 @@ static int handle_vmclear(struct kvm_vcpu *vcpu)* vmx->nested.hv_evmcs but this shouldn't be a problem. */ if (likely (!guest_cpuid_has_evmcs (vcpu) || - !nested_enlightened_vmentry (vcpu, &evmcs_gpa))) { + !evmptr_is_valid (nested_get_evmptr (vcpu)))) {if (vmptr == vmx->nested.current_vmptr)nested_release_vmcs12 (vcpu);
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 26/46 KVM: nVMX: hyper-v: Enable L2 TLB flush
In reply to Vitaly KuznetsovEnable L2 TLB flush feature on nVMX when:
- Enlightened VMCS is in use.
- The feature flag is enabled in eVMCS.
- The feature flag is enabled in partition assist page.
Perform synthetic vmexit to L1 after processing TLB flush call upon request (HV_VMX_SYNTHETIC_EXIT_REASON_TRAP_AFTER_FLUSH).
Note: nested_evmcs_l2_tlb_flush_enabled() uses cached VP assist page copy which gets updated from nested_vmx_handle_enlightened_vmptrld(). This is also guaranteed to happen post migration with eVMCS backed L2 running.
--- arch/x86/include/asm/hyperv-tlfs.h | 9 +++++++++ arch/x86/kvm/vmx/hyperv.c | 17 +++++++++++++++++ arch/x86/kvm/vmx/hyperv.h | 1 + arch/x86/kvm/vmx/nested.c | 20 ++++++++++++++++++++ 4 files changed, 47 insertions(+)
arch/x86/include/asm/hyperv-tlfs.h +9 -0
@@ -600,6 +600,15 @@ struct hv_enlightened_vmcs {#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL 0xFFFF +/* +* Note, Hyper-V isn't actually stealing bit 28 from Intel, just abusing it by +* pairing it with architecturally impossible exit reasons. Bit 28 is set only +* on SMI exits to a SMI transfer monitor (STM) and if and only if a MTF VM-Exit +* is pending. I.e. it will never be set by hardware for non-SMI exits (there +* are only three), nor will it ever be set unless the VMM is an STM. +*/ +#define HV_VMX_SYNTHETIC_EXIT_REASON_TRAP_AFTER_FLUSH 0x10000031 +/* * Hyper-V uses the software reserved 32 bytes in VMCB control area to expose * SVM enlightenments to guests.
arch/x86/kvm/vmx/hyperv.c +17 -0
@@ -5,6 +5,7 @@#include "../cpuid.h" #include "hyperv.h" +#include "nested.h" #include "vmcs.h" #include "vmx.h" #include "trace.h"
@@ -500,6 +501,22 @@ int nested_enable_evmcs(struct kvm_vcpu *vcpu,return 0 ; } +bool nested_evmcs_l2_tlb_flush_enabled (struct kvm_vcpu *vcpu) +{ +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct hv_enlightened_vmcs *evmcs = vmx->nested.hv_evmcs; + +if (!hv_vcpu || !evmcs) +return false; + +if (!evmcs->hv_enlightenments_control.nested_flush_hypercall) +return false; + +return hv_vcpu->vp_assist_page.nested_control.features.directhypercall; +} +void vmx_hv_inject_synthetic_vmexit_post_tlb_flush (struct kvm_vcpu *vcpu) { +nested_vmx_vmexit (vcpu,HV_VMX_SYNTHETIC_EXIT_REASON_TRAP_AFTER_FLUSH ,0 ,0 ); }
arch/x86/kvm/vmx/hyperv.h +1 -0
@@ -243,6 +243,7 @@ int nested_enable_evmcs(struct kvm_vcpu *vcpu, uint16_t *vmcs_version);void nested_evmcs_filter_control_msr (struct kvm_vcpu *vcpu,u32 msr_index,u64 *pdata);int nested_evmcs_check_controls (struct vmcs12 *vmcs12); +bool nested_evmcs_l2_tlb_flush_enabled (struct kvm_vcpu *vcpu);void vmx_hv_inject_synthetic_vmexit_post_tlb_flush (struct kvm_vcpu *vcpu);#endif /* __KVM_X86_VMX_HYPERV_H */
arch/x86/kvm/vmx/nested.c +20 -0
@@ -1131,6 +1131,15 @@ static void nested_vmx_transition_tlb_flush(struct kvm_vcpu *vcpu, {struct vcpu_vmx *vmx =to_vmx (vcpu); +/* +* KVM_REQ_HV_TLB_FLUSH flushes entries from either L1's VP_ID or +* L2's VP_ID upon request from the guest. Make sure we check for +* pending entries in the right FIFO upon L1/L2 transition as these +* requests are put by other vCPUs asynchronously. +*/ +if (to_hv_vcpu (vcpu) && enable_ept) +kvm_make_request (KVM_REQ_HV_TLB_FLUSH , vcpu); +/* * If vmcs12 doesn't use VPID, L1 expects linear and combined mappings * for *all* contexts to be flushed on VM-Enter/VM-Exit, i.e. it's a
@@ -3266,6 +3275,12 @@ static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu)static bool vmx_get_nested_state_pages (struct kvm_vcpu *vcpu) { +/* +* Note: nested_get_evmcs_page() also updates 'vp_assist_page' copy +* in 'struct kvm_vcpu_hv' in case eVMCS is in use, this is mandatory +* to make nested_evmcs_l2_tlb_flush_enabled() work correctly post +* migration. +*/ if (!nested_get_evmcs_page (vcpu)) {pr_debug_ratelimited ("%s: enlightened vmptrld failed\n" , __func__);
@@ -6142,6 +6157,11 @@ static bool nested_vmx_l0_wants_exit(struct kvm_vcpu *vcpu,* Handle L2's bus locks in L0 directly. */ return true; +case EXIT_REASON_VMCALL : +/* Hyper-V L2 TLB flush hypercall is handled by L0 */ +return guest_hv_cpuid_has_l2_tlb_flush (vcpu) && +nested_evmcs_l2_tlb_flush_enabled (vcpu) && +kvm_hv_is_tlb_flush_hcall (vcpu);default :break ; }
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 27/46 KVM: x86: Make kvm_hv_get_assist_page() return 0/-errno
In reply to Vitaly KuznetsovConvert kvm_hv_get_assist_page() to return 'int' and propagate possible errors from kvm_read_guest_cached().
--- arch/x86/kvm/hyperv.c | 8 ++++---- arch/x86/kvm/hyperv.h | 2 +- arch/x86/kvm/vmx/hyperv.c | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-)
arch/x86/kvm/hyperv.c +4 -4
@@ -900,15 +900,15 @@ bool kvm_hv_assist_page_enabled(struct kvm_vcpu *vcpu) }EXPORT_SYMBOL_GPL (kvm_hv_assist_page_enabled); -bool kvm_hv_get_assist_page (struct kvm_vcpu *vcpu) +int kvm_hv_get_assist_page (struct kvm_vcpu *vcpu) {struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu);if (!hv_vcpu || !kvm_hv_assist_page_enabled (vcpu)) -return false; +return -EFAULT ; -return !kvm_read_guest_cached (vcpu->kvm, &vcpu->arch.pv_eoi.data, - &hv_vcpu->vp_assist_page,sizeof (struct hv_vp_assist_page )); +return kvm_read_guest_cached (vcpu->kvm, &vcpu->arch.pv_eoi.data, + &hv_vcpu->vp_assist_page,sizeof (struct hv_vp_assist_page )); }EXPORT_SYMBOL_GPL (kvm_hv_get_assist_page);
arch/x86/kvm/hyperv.h +1 -1
@@ -108,7 +108,7 @@ int kvm_hv_activate_synic(struct kvm_vcpu *vcpu, bool dont_zero_synic_pages);void kvm_hv_vcpu_uninit (struct kvm_vcpu *vcpu);bool kvm_hv_assist_page_enabled (struct kvm_vcpu *vcpu); -bool kvm_hv_get_assist_page (struct kvm_vcpu *vcpu); +int kvm_hv_get_assist_page (struct kvm_vcpu *vcpu);static inline struct kvm_vcpu_hv_stimer *to_hv_stimer (struct kvm_vcpu *vcpu,int timer_index)
arch/x86/kvm/vmx/hyperv.c +1 -1
@@ -326,7 +326,7 @@ u64 nested_get_evmptr(struct kvm_vcpu *vcpu) {struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); -if (unlikely (!kvm_hv_get_assist_page (vcpu))) +if (unlikely (kvm_hv_get_assist_page (vcpu)))return EVMPTR_INVALID ;if (unlikely (!hv_vcpu->vp_assist_page.enlighten_vmentry))
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 28/46 KVM: nSVM: hyper-v: Enable L2 TLB flush
In reply to Vitaly KuznetsovImplement Hyper-V L2 TLB flush for nSVM. The feature needs to be enabled both in extended 'nested controls' in VMCB and VP assist page. According to Hyper-V TLFS, synthetic vmexit to L1 is performed with
- HV_SVM_EXITCODE_ENL exit_code.
- HV_SVM_ENL_EXITCODE_TRAP_AFTER_FLUSH exit_info_1.
Note: VP assist page is cached in 'struct kvm_vcpu_hv' so recalc_intercepts() doesn't need to read from guest's memory. KVM needs to update the case upon each VMRUN and after svm_set_nested_state (svm_get_nested_state_pages()) to handle the case when the guest got migrated while L2 was running.
--- arch/x86/include/asm/hyperv-tlfs.h | 4 ++++ arch/x86/kvm/hyperv.h | 11 ++++++++++ arch/x86/kvm/svm/hyperv.c | 7 ++++++ arch/x86/kvm/svm/hyperv.h | 15 +++++++++++++ arch/x86/kvm/svm/nested.c | 35 ++++++++++++++++++++++++++++-- 5 files changed, 70 insertions(+), 2 deletions(-)
arch/x86/include/asm/hyperv-tlfs.h +4 -0
@@ -631,6 +631,10 @@ struct hv_vmcb_enlightenments {*/ #define HV_VMCB_NESTED_ENLIGHTENMENTS 31 +/* Synthetic VM-Exit */ +#define HV_SVM_EXITCODE_ENL 0xf0000000 +#define HV_SVM_ENL_EXITCODE_TRAP_AFTER_FLUSH (1) +struct hv_partition_assist_pg {u32 tlb_lock_count; };
arch/x86/kvm/hyperv.h +11 -0
@@ -198,6 +198,17 @@ static inline bool kvm_hv_is_tlb_flush_hcall(struct kvm_vcpu *vcpu) code == HVCALL_FLUSH_VIRTUAL_ADDRESS_LIST_EX); } +static inline int kvm_hv_verify_vp_assist (struct kvm_vcpu *vcpu) +{ +if (!to_hv_vcpu (vcpu)) +return 0 ; + +if (!kvm_hv_assist_page_enabled (vcpu)) +return 0 ; + +return kvm_hv_get_assist_page (vcpu); +} +int kvm_hv_vcpu_flush_tlb (struct kvm_vcpu *vcpu);#endif
arch/x86/kvm/svm/hyperv.c +7 -0
@@ -8,4 +8,11 @@void svm_hv_inject_synthetic_vmexit_post_tlb_flush (struct kvm_vcpu *vcpu) { +struct vcpu_svm *svm =to_svm (vcpu); + + svm->vmcb->control.exit_code =HV_SVM_EXITCODE_ENL ; + svm->vmcb->control.exit_code_hi =0 ; + svm->vmcb->control.exit_info_1 =HV_SVM_ENL_EXITCODE_TRAP_AFTER_FLUSH ; + svm->vmcb->control.exit_info_2 =0 ; +nested_svm_vmexit (svm); }
arch/x86/kvm/svm/hyperv.h +15 -0
@@ -25,6 +25,21 @@ static inline void nested_svm_hv_update_vm_vp_ids(struct kvm_vcpu *vcpu) hv_vcpu->nested.vp_id = hve->hv_vp_id; } +static inline bool nested_svm_l2_tlb_flush_enabled (struct kvm_vcpu *vcpu) +{ +struct vcpu_svm *svm =to_svm (vcpu); +struct hv_vmcb_enlightenments *hve = &svm->nested.ctl.hv_enlightenments; +struct kvm_vcpu_hv *hv_vcpu =to_hv_vcpu (vcpu); + +if (!hv_vcpu) +return false; + +if (!hve->hv_enlightenments_control.nested_flush_hypercall) +return false; + +return hv_vcpu->vp_assist_page.nested_control.features.directhypercall; +} +void svm_hv_inject_synthetic_vmexit_post_tlb_flush (struct kvm_vcpu *vcpu);#endif /* __ARCH_X86_KVM_SVM_HYPERV_H__ */
arch/x86/kvm/svm/nested.c +33 -2
@@ -149,8 +149,12 @@ void recalc_intercepts(struct vcpu_svm *svm)vmcb_clr_intercept (c,INTERCEPT_VINTR ); } -/* We don't want to see VMMCALLs from a nested guest */ -vmcb_clr_intercept (c,INTERCEPT_VMMCALL ); +/* +* We want to see VMMCALLs from a nested guest only when Hyper-V L2 TLB +* flush feature is enabled. +*/ +if (!nested_svm_l2_tlb_flush_enabled (&svm->vcpu)) +vmcb_clr_intercept (c,INTERCEPT_VMMCALL );for (i =0 ; i <MAX_INTERCEPT ; i++) c->intercepts[i] |= g->intercepts[i];
@@ -472,6 +476,15 @@ static void nested_save_pending_event_to_vmcb12(struct vcpu_svm *svm,static void nested_svm_transition_tlb_flush (struct kvm_vcpu *vcpu) { +/* +* KVM_REQ_HV_TLB_FLUSH flushes entries from either L1's VP_ID or +* L2's VP_ID upon request from the guest. Make sure we check for +* pending entries in the right FIFO upon L1/L2 transition as these +* requests are put by other vCPUs asynchronously. +*/ +if (to_hv_vcpu (vcpu) && npt_enabled) +kvm_make_request (KVM_REQ_HV_TLB_FLUSH , vcpu); +/* * TODO: optimize unconditional TLB flush/MMU sync. A partial list of * things to fix before this can be conditional:
@@ -823,6 +836,13 @@ int nested_svm_vmrun(struct kvm_vcpu *vcpu)return 1 ; } +/* This fails when VP assist page is enabled but the supplied GPA is bogus */ + ret =kvm_hv_verify_vp_assist (vcpu); +if (ret) { +kvm_inject_gp (vcpu,0 ); +return ret; + } + vmcb12_gpa = svm->vmcb->save.rax; ret =kvm_vcpu_map (vcpu,gpa_to_gfn (vmcb12_gpa), &map);if (ret == -EINVAL ) {
@@ -1412,6 +1432,7 @@ static int svm_check_nested_events(struct kvm_vcpu *vcpu)int nested_svm_exit_special (struct vcpu_svm *svm) {u32 exit_code = svm->vmcb->control.exit_code; +struct kvm_vcpu *vcpu = &svm->vcpu;switch (exit_code) {case SVM_EXIT_INTR :
@@ -1430,6 +1451,13 @@ int nested_svm_exit_special(struct vcpu_svm *svm)return NESTED_EXIT_HOST ;break ; } +case SVM_EXIT_VMMCALL : +/* Hyper-V L2 TLB flush hypercall is handled by L0 */ +if (guest_hv_cpuid_has_l2_tlb_flush (vcpu) && +nested_svm_l2_tlb_flush_enabled (vcpu) && +kvm_hv_is_tlb_flush_hcall (vcpu)) +return NESTED_EXIT_HOST ; +break ;default :break ; }
@@ -1710,6 +1738,9 @@ static bool svm_get_nested_state_pages(struct kvm_vcpu *vcpu)return false; } +if (kvm_hv_verify_vp_assist (vcpu)) +return false; +return true; }
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 29/46 KVM: x86: Expose Hyper-V L2 TLB flush feature
In reply to Vitaly KuznetsovWith both nSVM and nVMX implementations in place, KVM can now expose Hyper-V L2 TLB flush feature to userspace.
--- arch/x86/kvm/hyperv.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kvm/hyperv.c +1 -0
@@ -2767,6 +2767,7 @@ int kvm_get_hv_cpuid(struct kvm_vcpu *vcpu, struct kvm_cpuid2 *cpuid,case HYPERV_CPUID_NESTED_FEATURES : ent->eax = evmcs_ver; + ent->eax |=HV_X64_NESTED_DIRECT_FLUSH ; ent->eax |=HV_X64_NESTED_MSR_BITMAP ; ent->ebx |=HV_X64_NESTED_EVMCS1_PERF_GLOBAL_CTRL ;break ;
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 30/46 KVM: selftests: Better XMM read/write helpers
In reply to Vitaly Kuznetsovset_xmm()/get_xmm() helpers are fairly useless as they only read 64 bits from 128-bit registers. Moreover, these helpers are not used. Borrow _kvm_read_sse_reg()/_kvm_write_sse_reg() from KVM limiting them to XMM0-XMM8 for now.
--- .../selftests/kvm/include/x86_64/processor.h | 70 ++++++++++--------- 1 file changed, 36 insertions(+), 34 deletions(-)
tools/testing/selftests/kvm/include/x86_64/processor.h +36 -34
@@ -436,71 +436,73 @@ static inline bool this_cpu_has(struct kvm_x86_cpu_feature feature)return gprs[feature.reg] &BIT (feature.bit); } -#define SET_XMM (__var, __xmm) \ - asm volatile("movq %0, %%"#__xmm : : "r"(__var) : #__xmm) +typedef u32 __attribute__((vector_size(16))) sse128_t; +#define __sse128_u union { sse128_t vec; u64 as_u64[2]; u32 as_u32[4]; } +#define sse128_lo (x) ({ __sse128_u t; t.vec = x; t.as_u64[0]; }) +#define sse128_hi (x) ({ __sse128_u t; t.vec = x; t.as_u64[1]; }) -static inline void set_xmm (int n,unsigned long val) +static inline void read_sse_reg (int reg,sse128_t *data) { -switch (n) { +switch (reg) {case 0 : -SET_XMM (val, xmm0); + asm("movdqa %%xmm0, %0" :"=m" (*data));break ;case 1 : -SET_XMM (val, xmm1); + asm("movdqa %%xmm1, %0" :"=m" (*data));break ;case 2 : -SET_XMM (val, xmm2); + asm("movdqa %%xmm2, %0" :"=m" (*data));break ;case 3 : -SET_XMM (val, xmm3); + asm("movdqa %%xmm3, %0" :"=m" (*data));break ;case 4 : -SET_XMM (val, xmm4); + asm("movdqa %%xmm4, %0" :"=m" (*data));break ;case 5 : -SET_XMM (val, xmm5); + asm("movdqa %%xmm5, %0" :"=m" (*data));break ;case 6 : -SET_XMM (val, xmm6); + asm("movdqa %%xmm6, %0" :"=m" (*data));break ;case 7 : -SET_XMM (val, xmm7); + asm("movdqa %%xmm7, %0" :"=m" (*data));break ; +default : +BUG (); } } -#define GET_XMM (__xmm) \ -({ \ - unsigned long __val; \ - asm volatile("movq %%"#__xmm", %0" : "=r"(__val)); \ - __val; \ -}) - -static inline unsigned long get_xmm (int n) +static inline void write_sse_reg (int reg,const sse128_t *data) { -assert (n >=0 && n <=7 ); - -switch (n) { +switch (reg) {case 0 : -return GET_XMM (xmm0); + asm("movdqa %0, %%xmm0" : :"m" (*data)); +break ;case 1 : -return GET_XMM (xmm1); + asm("movdqa %0, %%xmm1" : :"m" (*data)); +break ;case 2 : -return GET_XMM (xmm2); + asm("movdqa %0, %%xmm2" : :"m" (*data)); +break ;case 3 : -return GET_XMM (xmm3); + asm("movdqa %0, %%xmm3" : :"m" (*data)); +break ;case 4 : -return GET_XMM (xmm4); + asm("movdqa %0, %%xmm4" : :"m" (*data)); +break ;case 5 : -return GET_XMM (xmm5); + asm("movdqa %0, %%xmm5" : :"m" (*data)); +break ;case 6 : -return GET_XMM (xmm6); + asm("movdqa %0, %%xmm6" : :"m" (*data)); +break ;case 7 : -return GET_XMM (xmm7); + asm("movdqa %0, %%xmm7" : :"m" (*data)); +break ; +default : +BUG (); } - -/* never reached */ -return 0 ; }static inline void cpu_relax (void )
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 31/46 KVM: selftests: Move HYPERV_LINUX_OS_ID definition to a common header
In reply to Vitaly KuznetsovHYPERV_LINUX_OS_ID needs to be written to HV_X64_MSR_GUEST_OS_ID by each Hyper-V specific selftest.
--- tools/testing/selftests/kvm/include/x86_64/hyperv.h | 3 +++ tools/testing/selftests/kvm/x86_64/hyperv_features.c | 6 ++---- 2 files changed, 5 insertions(+), 4 deletions(-)
tools/testing/selftests/kvm/include/x86_64/hyperv.h +3 -0
@@ -185,4 +185,7 @@/* hypercall options */ #define HV_HYPERCALL_FAST_BIT BIT(16) +/* Proper HV_X64_MSR_GUEST_OS_ID value */ +#define HYPERV_LINUX_OS_ID ((u64)0x8100 << 48) +#endif /* !SELFTEST_KVM_HYPERV_H */
tools/testing/selftests/kvm/x86_64/hyperv_features.c +2 -4
@@ -13,8 +13,6 @@#include "processor.h" #include "hyperv.h" -#define LINUX_OS_ID ((u64)0x8100 << 48) -static inline uint8_t hypercall (u64 control,vm_vaddr_t input_address,vm_vaddr_t output_address,uint64_t *hv_status) {
@@ -71,7 +69,7 @@ static void guest_hcall(vm_vaddr_t pgs_gpa, struct hcall_data *hcall)GUEST_ASSERT (hcall->control); -wrmsr (HV_X64_MSR_GUEST_OS_ID ,LINUX_OS_ID ); +wrmsr (HV_X64_MSR_GUEST_OS_ID ,HYPERV_LINUX_OS_ID );wrmsr (HV_X64_MSR_HYPERCALL , pgs_gpa);if (!(hcall->control &HV_HYPERCALL_FAST_BIT )) {
@@ -169,7 +167,7 @@ static void guest_test_msrs_access(void)*/ msr->idx =HV_X64_MSR_GUEST_OS_ID ; msr->write =1 ; - msr->write_val =LINUX_OS_ID ; + msr->write_val =HYPERV_LINUX_OS_ID ; msr->available =1 ;break ;case 3 :
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 32/46 KVM: selftests: Move the function doing Hyper-V hypercall to a common header
In reply to Vitaly KuznetsovAll Hyper-V specific tests issuing hypercalls need this.
--- .../selftests/kvm/include/x86_64/hyperv.h | 18 ++++++++++++++++++ .../selftests/kvm/x86_64/hyperv_features.c | 19 +------------------ 2 files changed, 19 insertions(+), 18 deletions(-)
tools/testing/selftests/kvm/include/x86_64/hyperv.h +18 -0
@@ -185,6 +185,24 @@/* hypercall options */ #define HV_HYPERCALL_FAST_BIT BIT(16) +static inline uint8_t hyperv_hypercall (u64 control,vm_vaddr_t input_address, +vm_vaddr_t output_address, +uint64_t *hv_status) +{ +uint8_t vector; + +/* Note both the hypercall and the "asm safe" clobber r9-r11. */ + asm volatile("mov %[output_address], %%r8\n\t" + KVM_ASM_SAFE("vmcall") + : "=a" (*hv_status), + "+c" (control), "+d" (input_address), + KVM_ASM_SAFE_OUTPUTS(vector) + : [output_address] "r"(output_address), + "a" (-EFAULT) + : "cc", "memory", "r8", KVM_ASM_SAFE_CLOBBERS); +return vector; +} +/* Proper HV_X64_MSR_GUEST_OS_ID value */ #define HYPERV_LINUX_OS_ID ((u64)0x8100 << 48)
tools/testing/selftests/kvm/x86_64/hyperv_features.c +1 -18
@@ -13,23 +13,6 @@#include "processor.h" #include "hyperv.h" -static inline uint8_t hypercall (u64 control,vm_vaddr_t input_address, -vm_vaddr_t output_address,uint64_t *hv_status) -{ -uint8_t vector; - -/* Note both the hypercall and the "asm safe" clobber r9-r11. */ - asm volatile("mov %[output_address], %%r8\n\t" - KVM_ASM_SAFE("vmcall") - : "=a" (*hv_status), - "+c" (control), "+d" (input_address), - KVM_ASM_SAFE_OUTPUTS(vector) - : [output_address] "r"(output_address), - "a" (-EFAULT) - : "cc", "memory", "r8", KVM_ASM_SAFE_CLOBBERS); -return vector; -} -struct msr_data {uint32_t idx;bool available;
@@ -79,7 +62,7 @@ static void guest_hcall(vm_vaddr_t pgs_gpa, struct hcall_data *hcall) input = output =0 ; } - vector =hypercall (hcall->control, input, output, &res); + vector =hyperv_hypercall (hcall->control, input, output, &res);if (hcall->ud_expected) {GUEST_ASSERT_2 (vector ==UD_VECTOR , hcall->control, vector); }else {
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 33/46 KVM: selftests: Hyper-V PV IPI selftest
In reply to Vitaly KuznetsovIntroduce a selftest for Hyper-V PV IPI hypercalls (HvCallSendSyntheticClusterIpi, HvCallSendSyntheticClusterIpiEx).
The test creates one 'sender' vCPU and two 'receiver' vCPU and then issues various combinations of send IPI hypercalls in both 'normal' and 'fast' (with XMM input where necessary) mode. Later, the test checks whether IPIs were delivered to the expected destination vCPU[s].
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/x86_64/hyperv.h | 35 +- .../selftests/kvm/x86_64/hyperv_features.c | 2 +- .../testing/selftests/kvm/x86_64/hyperv_ipi.c | 330 ++++++++++++++++++ 5 files changed, 365 insertions(+), 4 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86_64/hyperv_ipi.c
tools/testing/selftests/kvm/.gitignore +1 -0
@@ -24,6 +24,7 @@ /x86_64/hyperv_clock /x86_64/hyperv_cpuid /x86_64/hyperv_features +/x86_64/hyperv_ipi /x86_64/hyperv_svm_test /x86_64/max_vcpuid_cap_test /x86_64/mmio_warning_test
tools/testing/selftests/kvm/Makefile +1 -0
@@ -84,6 +84,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/fix_hypercall_test TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features +TEST_GEN_PROGS_x86_64 += x86_64/hyperv_ipi TEST_GEN_PROGS_x86_64 += x86_64/hyperv_svm_test TEST_GEN_PROGS_x86_64 += x86_64/kvm_clock_test TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test
tools/testing/selftests/kvm/include/x86_64/hyperv.h +32 -3
@@ -9,6 +9,8 @@#ifndef SELFTEST_KVM_HYPERV_H #define SELFTEST_KVM_HYPERV_H +#include "processor.h" +#define HYPERV_CPUID_VENDOR_AND_MAX_FUNCTIONS 0x40000000#define HYPERV_CPUID_INTERFACE 0x40000001#define HYPERV_CPUID_VERSION 0x40000002
@@ -184,10 +186,15 @@/* hypercall options */ #define HV_HYPERCALL_FAST_BIT BIT(16) +#define HV_HYPERCALL_VARHEAD_OFFSET 17 -static inline uint8_t hyperv_hypercall (u64 control,vm_vaddr_t input_address, -vm_vaddr_t output_address, -uint64_t *hv_status) +/* +* Issue a Hyper-V hypercall. Returns exception vector raised or 0, 'hv_status' +* is set to the hypercall status (if no exception occurred). +*/ +static inline uint8_t __hyperv_hypercall (u64 control,vm_vaddr_t input_address, +vm_vaddr_t output_address, +uint64_t *hv_status) {uint8_t vector;
@@ -203,6 +210,28 @@ static inline uint8_t hyperv_hypercall(u64 control, vm_vaddr_t input_address,return vector; } +/* Issue a Hyper-V hypercall and assert that it succeeded. */ +static inline void hyperv_hypercall (u64 control,vm_vaddr_t input_address, +vm_vaddr_t output_address) +{ +uint64_t hv_status; +uint8_t vector; + + vector =__hyperv_hypercall (control, input_address, output_address, &hv_status); + +GUEST_ASSERT (!vector); +GUEST_ASSERT ((hv_status &0xffff ) ==0 ); +} + +/* Write 'Fast' hypercall input 'data' to the first 'n_sse_regs' SSE regs */ +static inline void hyperv_write_xmm_input (void *data,int n_sse_regs) +{ +int i; + +for (i =0 ; i < n_sse_regs; i++) +write_sse_reg (i, (sse128_t *)(data +sizeof (sse128_t) * i)); +} +/* Proper HV_X64_MSR_GUEST_OS_ID value */ #define HYPERV_LINUX_OS_ID ((u64)0x8100 << 48)
tools/testing/selftests/kvm/x86_64/hyperv_features.c +1 -1
@@ -62,7 +62,7 @@ static void guest_hcall(vm_vaddr_t pgs_gpa, struct hcall_data *hcall) input = output =0 ; } - vector =hyperv_hypercall (hcall->control, input, output, &res); + vector =__hyperv_hypercall (hcall->control, input, output, &res);if (hcall->ud_expected) {GUEST_ASSERT_2 (vector ==UD_VECTOR , hcall->control, vector); }else {
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c new file +330 -0
330 lines not on this page · raw message
Signature · 1 line
2.37.3
Sean Christopherson
maintainerRe: [PATCH v11 33/46] KVM: selftests: Hyper-V PV IPI selftest
Quoted 14 lines · Vitaly Kuznetsov
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c new file +14
-0+static void *vcpu_thread (void *arg) +{ +struct kvm_vcpu *vcpu = (struct kvm_vcpu *)arg; +struct ucall uc; +int old; +int r; +unsigned int exit_reason; + + r =pthread_setcanceltype (PTHREAD_CANCEL_ASYNCHRONOUS , &old); +TEST_ASSERT (r ==0 , +"pthread_setcanceltype failed on vcpu_id=%u with errno=%d" , + vcpu->id, r); + +vcpu_run (vcpu);
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c line 231
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c new file +10
-0+ exit_reason = vcpu->run->exit_reason; + +TEST_ASSERT (exit_reason ==KVM_EXIT_IO , +"vCPU %u exited with unexpected exit reason %u-%s, expected KVM_EXIT_IO" , + vcpu->id, exit_reason,exit_reason_str (exit_reason)); + +if (get_ucall (vcpu, &uc) ==UCALL_ABORT ) { +TEST_ASSERT (false, +"vCPU %u exited with error: %s.\n" , + vcpu->id, (const char *)uc.args[0 ]);
REPORT_GUEST_ASSERT_N()?
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c line 243
Quoted 12 lines
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c new file +12
-0+ } + +return NULL ; +} + +static void cancel_join_vcpu_thread (pthread_t thread,struct kvm_vcpu *vcpu) +{ +void *retval; +int r; + + r =pthread_cancel (thread); + TEST_ASSERT(r == 0,
!r is generally preferred over "r == 0"
Review
Quoted 2 lines
(quoted diff) +2
-0+ "pthread_cancel on vcpu_id=%d failed with errno=%d", + vcpu->id, r);
Do you happen to know if errno is preserved? I.e. if TEST_ASSERT()'s print of errno will capture the right errno? If so, this and the pthread_join() assert can be:
TEST_ASSERT(!r, pthread_cancel() failed on vcpu_id=%d, vcpu->id);
Quoted 5 lines
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c new file +5
-0+ + r =pthread_join (thread, &retval); +TEST_ASSERT (r ==0 , +"pthread_join on vcpu_id=%d failed with errno=%d" , + vcpu->id, r);
...
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c line 297
Quoted 7 lines
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c new file +7
-0+ r =pthread_create (&threads[0 ],NULL , vcpu_thread, vcpu[1 ]); +TEST_ASSERT (r ==0 , +"pthread_create halter failed errno=%d" , errno); + + r =pthread_create (&threads[1 ],NULL , vcpu_thread, vcpu[2 ]); +TEST_ASSERT (r ==0 , +"pthread_create halter failed errno=%d" , errno);
Similar comments as above.
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c line 303
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c new file +6
-0+ +while (true) { + r =_vcpu_run (vcpu[0 ]); + exit_reason = vcpu[0 ]->run->exit_reason; + +TEST_ASSERT (!r,"vcpu_run failed: %d\n" , r);
Why use _vcpu_run() with a manual assert? Won't the vanilla vcpu_run() do what you want?
Vitaly Kuznetsov
maintainerRe: [PATCH v11 33/46] KVM: selftests: Hyper-V PV IPI selftest
Quoted 1 line · Sean Christopherson
On Tue, Oct 04, 2022, Vitaly Kuznetsov wrote:
...
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c line 243
Quoted 3 lines
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c new file +3
-0+ + r =pthread_cancel (thread); + TEST_ASSERT(r == 0,!r is generally preferred over "r == 0"
Quoted 9 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ "pthread_cancel on vcpu_id=%d failed with errno=%d", + vcpu->id, r);Do you happen to know if errno is preserved? I.e. if TEST_ASSERT()'s print of errno will capture the right errno? If so, this and the
pthread_join()assert can be:TEST_ASSERT(!r, pthread_cancel() failed on vcpu_id=%d, vcpu->id);
The example from 'man 3 pthread_cancel' makes me think errno is not set. 'man 3 errno' confirms that:
"
Note that the POSIX threads APIs do not set errno on error.
Instead, on failure they return an error number as the function result. These error numbers have the same meanings as the error numbers returned in errno by other APIs.
"
but nothing stops us from doing something like
#include <errno.h>
...
errno = pthread_cancel(thread);TEST_ASSERT(!errno, pthread_cancel() failed on vcpu_id=%d, vcpu->id);
I believe.
Signature · 1 line
Vitaly
Vitaly Kuznetsov
maintainerRe: [PATCH v11 33/46] KVM: selftests: Hyper-V PV IPI selftest
Quoted 34 lines · Vitaly Kuznetsov
Quoted 1 line · Sean Christopherson
On Tue, Oct 04, 2022, Vitaly Kuznetsov wrote:
...
Quoted 6 lines
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c line 243
Quoted 3 lines
tools/testing/selftests/kvm/x86_64/hyperv_ipi.c new file +3
-0+ + r =pthread_cancel (thread); + TEST_ASSERT(r == 0,!r is generally preferred over "r == 0"
Quoted 9 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ "pthread_cancel on vcpu_id=%d failed with errno=%d", + vcpu->id, r);Do you happen to know if errno is preserved? I.e. if TEST_ASSERT()'s print of errno will capture the right errno? If so, this and the
pthread_join()assert can be:TEST_ASSERT(!r, pthread_cancel() failed on vcpu_id=%d, vcpu->id);The example from 'man 3
pthread_cancel' makes me think errno is not set. 'man 3 errno' confirms that:"
Note that the POSIX threads APIs do not set errno on error.
Instead, on failure they return an error number as the function result. These error numbers have the same meanings as the error numbers returned in errno by other APIs.
"but nothing stops us from doing something like
Quoted 7 lines
#include <
errno.h>
...errno =
pthread_cancel(thread);TEST_ASSERT(!errno, pthread_cancel() failed on vcpu_id=%d, vcpu->id);I believe.
... only the fact that this won't be thread safe :-( i.e. if we also try setting 'errno' from vcpu_thread() (where the pattern for pthread_setcanceltype() is exactly the same), we will likely be reporting the wrong errno. I think it's better to keep reporting 'r' for now (and maybe think about pthread* wrappers later).
Signature · 1 line
Vitaly
Sean Christopherson
maintainerRe: [PATCH v11 33/46] KVM: selftests: Hyper-V PV IPI selftest
Quoted 7 lines · Vitaly Kuznetsov
Quoted 5 lines · Vitaly Kuznetsov
Quoted 4 lines · Sean Christopherson
Do you happen to know if errno is preserved? I.e. if TEST_ASSERT()'s print of errno will capture the right errno? If so, this and the
pthread_join()assert can be:
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
TEST_ASSERT(!r, pthread_cancel() failed on vcpu_id=%d, vcpu->id);The example from 'man 3
pthread_cancel' makes me think errno is not set. 'man 3 errno' confirms that:"
Note that the POSIX threads APIs do not set errno on error.
Ah, that's annoying.
Quoted 13 lines
Quoted 13 lines
Instead, on failure they return an error number as the function result. These error numbers have the same meanings as the error numbers returned in errno by other APIs.
"but nothing stops us from doing something like
#include <
errno.h>
...errno =
pthread_cancel(thread);TEST_ASSERT(!errno, pthread_cancel() failed on vcpu_id=%d, vcpu->id);
Quoted 6 lines
Quoted 1 line
I believe.
... only the fact that this won't be thread safe :-( i.e. if we also try setting 'errno' from
vcpu_thread()(where the pattern forpthread_setcanceltype()is exactly the same), we will likely be reporting the wrong errno.
errno is thread safe. From https://man7.org/linux/man-pages/man3/errno.3.html:
errno is defined by the ISO C standard to be a modifiable lvalue of type int, and must not be explicitly declared; errno may be a macro. errno is thread-local; setting it in one thread does not affect its value in any other thread.
Quoted 2 lines
I think it's better to keep reporting 'r' for now (and maybe think about pthread* wrappers later).
Yeah, definitely a future problem. pthread wrappers are a good idea, I doubt there's a single KVM selftest that wants to do anything but assert on failure.
Vitaly Kuznetsov
maintainerPATCH v11 34/46 KVM: selftests: Fill in vm->vpages_mapped bitmap in virt_map() too
In reply to Vitaly KuznetsovSimilar to vm_vaddr_alloc(), virt_map() needs to reflect the mapping in vm->vpages_mapped.
While on it, remove unneeded code wraping in vm_vaddr_alloc().
--- tools/testing/selftests/kvm/lib/kvm_util.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/lib/kvm_util.c +3 -2
@@ -1214,8 +1214,7 @@ vm_vaddr_t vm_vaddr_alloc(struct kvm_vm *vm, size_t sz, vm_vaddr_t vaddr_min)virt_pg_map (vm, vaddr, paddr); -sparsebit_set (vm->vpages_mapped, - vaddr >> vm->page_shift); +sparsebit_set (vm->vpages_mapped, vaddr >> vm->page_shift); }return vaddr_start;
@@ -1288,6 +1287,8 @@ void virt_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr,virt_pg_map (vm, vaddr, paddr); vaddr += page_size; paddr += page_size; + +sparsebit_set (vm->vpages_mapped, vaddr >> vm->page_shift); } }
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 35/46 KVM: selftests: Export vm_vaddr_unused_gap() to make it possible to request unmapped ranges
In reply to Vitaly KuznetsovCurrently, tests can only request a new vaddr range by using vm_vaddr_alloc()/vm_vaddr_alloc_page()/vm_vaddr_alloc_pages() but these functions allocate and map physical pages too. Make it possible to request unmapped range too.
--- tools/testing/selftests/kvm/include/kvm_util_base.h | 1 + tools/testing/selftests/kvm/lib/kvm_util.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/include/kvm_util_base.h +1 -0
@@ -379,6 +379,7 @@ void vm_mem_region_set_flags(struct kvm_vm *vm, uint32_t slot, uint32_t flags);void vm_mem_region_move (struct kvm_vm *vm,uint32_t slot,uint64_t new_gpa);void vm_mem_region_delete (struct kvm_vm *vm,uint32_t slot);struct kvm_vcpu *__vm_vcpu_add (struct kvm_vm *vm,uint32_t vcpu_id); +vm_vaddr_t vm_vaddr_unused_gap (struct kvm_vm *vm,size_t sz,vm_vaddr_t vaddr_min);vm_vaddr_t vm_vaddr_alloc (struct kvm_vm *vm,size_t sz,vm_vaddr_t vaddr_min);vm_vaddr_t vm_vaddr_alloc_pages (struct kvm_vm *vm,int nr_pages);vm_vaddr_t vm_vaddr_alloc_page (struct kvm_vm *vm);
tools/testing/selftests/kvm/lib/kvm_util.c +2 -2
@@ -1109,8 +1109,8 @@ struct kvm_vcpu *__vm_vcpu_add(struct kvm_vm *vm, uint32_t vcpu_id)* TEST_ASSERT failure occurs for invalid input or no area of at least * sz unallocated bytes >= vaddr_min is available. */ -static vm_vaddr_t vm_vaddr_unused_gap (struct kvm_vm *vm,size_t sz, -vm_vaddr_t vaddr_min) +vm_vaddr_t vm_vaddr_unused_gap (struct kvm_vm *vm,size_t sz, +vm_vaddr_t vaddr_min) {uint64_t pages = (sz + vm->page_size -1 ) >> vm->page_shift;
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 36/46 KVM: selftests: Drop helpers to read/write page table entries
In reply to Vitaly KuznetsovFrom: Sean Christopherson <[email protected]>
Drop vm_{g,s}et_page_table_entry() and instead expose the "inner" helper (was _vm_get_page_table_entry()) that returns a _pointer_ to the PTE, i.e. let tests directly modify PTEs instead of bouncing through helpers that just make life difficult.
Opportunsitically use BIT_ULL() in emulator_error_test, and use the MAXPHYADDR define to set the "rogue" GPA bit instead of open coding the same value.
No functional change intended.
--- .../selftests/kvm/include/x86_64/processor.h | 6 ++---- .../selftests/kvm/lib/x86_64/processor.c | 21 ++----------------- .../kvm/x86_64/emulator_error_test.c | 6 ++++-- 3 files changed, 8 insertions(+), 25 deletions(-)
tools/testing/selftests/kvm/include/x86_64/processor.h +2 -4
@@ -827,10 +827,8 @@ static inline uint8_t wrmsr_safe(uint32_t msr, uint64_t val)return kvm_asm_safe ("wrmsr" ,"a" (val &-1u ),"d" (val >>32 ),"c" (msr)); } -uint64_t vm_get_page_table_entry (struct kvm_vm *vm,struct kvm_vcpu *vcpu, -uint64_t vaddr); -void vm_set_page_table_entry (struct kvm_vm *vm,struct kvm_vcpu *vcpu, -uint64_t vaddr,uint64_t pte); +uint64_t *vm_get_page_table_entry (struct kvm_vm *vm,struct kvm_vcpu *vcpu, +uint64_t vaddr);uint64_t kvm_hypercall (uint64_t nr,uint64_t a0,uint64_t a1,uint64_t a2,uint64_t a3);
tools/testing/selftests/kvm/lib/x86_64/processor.c +2 -19
@@ -214,9 +214,8 @@ void virt_arch_pg_map(struct kvm_vm *vm, uint64_t vaddr, uint64_t paddr)__virt_pg_map (vm, vaddr, paddr,PG_LEVEL_4K ); } -static uint64_t *_vm_get_page_table_entry (struct kvm_vm *vm, -struct kvm_vcpu *vcpu, -uint64_t vaddr) +uint64_t *vm_get_page_table_entry (struct kvm_vm *vm,struct kvm_vcpu *vcpu, +uint64_t vaddr) {uint16_t index[4 ];uint64_t *pml4e, *pdpe, *pde;
@@ -286,22 +285,6 @@ static uint64_t *_vm_get_page_table_entry(struct kvm_vm *vm,return &pte[index[0 ]]; } -uint64_t vm_get_page_table_entry (struct kvm_vm *vm,struct kvm_vcpu *vcpu, -uint64_t vaddr) -{ -uint64_t *pte =_vm_get_page_table_entry (vm, vcpu, vaddr); - -return *(uint64_t *)pte; -} - -void vm_set_page_table_entry (struct kvm_vm *vm,struct kvm_vcpu *vcpu, -uint64_t vaddr,uint64_t pte) -{ -uint64_t *new_pte =_vm_get_page_table_entry (vm, vcpu, vaddr); - - *(uint64_t *)new_pte = pte; -} -void virt_arch_dump (FILE *stream,struct kvm_vm *vm,uint8_t indent) {uint64_t *pml4e, *pml4e_start;
tools/testing/selftests/kvm/x86_64/emulator_error_test.c +4 -2
@@ -152,8 +152,9 @@ int main(int argc, char *argv[]) {struct kvm_vcpu *vcpu;struct kvm_vm *vm; -uint64_t gpa, pte; +uint64_t *pte;uint64_t *hva; +uint64_t gpa;int rc;/* Tell stdout not to buffer its content */
@@ -178,8 +179,9 @@ int main(int argc, char *argv[])virt_map (vm,MEM_REGION_GVA ,MEM_REGION_GPA ,1 ); hva =addr_gpa2hva (vm,MEM_REGION_GPA );memset (hva,0 ,PAGE_SIZE ); + pte =vm_get_page_table_entry (vm, vcpu,MEM_REGION_GVA ); -vm_set_page_table_entry (vm, vcpu,MEM_REGION_GVA , pte | (1ull <<36 )); + *pte |=BIT_ULL (MAXPHYADDR );vcpu_run (vcpu);process_exit_on_emulation_error (vcpu);
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 37/46 KVM: selftests: Hyper-V PV TLB flush selftest
In reply to Vitaly KuznetsovIntroduce a selftest for Hyper-V PV TLB flush hypercalls (HvFlushVirtualAddressSpace/HvFlushVirtualAddressSpaceEx, HvFlushVirtualAddressList/HvFlushVirtualAddressListEx).
The test creates one 'sender' vCPU and two 'worker' vCPU which do busy loop reading from a certain GVA checking the observed value. Sender vCPU swaos the data page with another page filled with a different value. The expectation for workers is also altered. Without TLB flush on worker vCPUs, they may continue to observe old value. To guard against accidental TLB flushes for worker vCPUs the test is repeated 100 times.
Hyper-V TLB flush hypercalls are tested in both 'normal' and 'XMM fast' modes.
--- tools/testing/selftests/kvm/.gitignore | 1 + tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/x86_64/hyperv.h | 1 + .../selftests/kvm/x86_64/hyperv_tlb_flush.c | 689 ++++++++++++++++++ 4 files changed, 692 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c
tools/testing/selftests/kvm/.gitignore +1 -0
@@ -26,6 +26,7 @@ /x86_64/hyperv_features /x86_64/hyperv_ipi /x86_64/hyperv_svm_test +/x86_64/hyperv_tlb_flush /x86_64/max_vcpuid_cap_test /x86_64/mmio_warning_test /x86_64/monitor_mwait_test
tools/testing/selftests/kvm/Makefile +1 -0
@@ -86,6 +86,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features TEST_GEN_PROGS_x86_64 += x86_64/hyperv_ipi TEST_GEN_PROGS_x86_64 += x86_64/hyperv_svm_test +TEST_GEN_PROGS_x86_64 += x86_64/hyperv_tlb_flush TEST_GEN_PROGS_x86_64 += x86_64/kvm_clock_test TEST_GEN_PROGS_x86_64 += x86_64/kvm_pv_test TEST_GEN_PROGS_x86_64 += x86_64/mmio_warning_test
tools/testing/selftests/kvm/include/x86_64/hyperv.h +1 -0
@@ -187,6 +187,7 @@/* hypercall options */ #define HV_HYPERCALL_FAST_BIT BIT(16)#define HV_HYPERCALL_VARHEAD_OFFSET 17 +#define HV_HYPERCALL_REP_COMP_OFFSET 32/* * Issue a Hyper-V hypercall. Returns exception vector raised or 0, 'hv_status'
tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c new file +689 -0
689 lines not on this page · raw message
Signature · 1 line
2.37.3
Sean Christopherson
maintainerRe: [PATCH v11 37/46] KVM: selftests: Hyper-V PV TLB flush selftest
tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c line 550
Quoted 12 lines · Vitaly Kuznetsov
tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c new file +12
-0+static void *vcpu_thread (void *arg) +{ +struct kvm_vcpu *vcpu = (struct kvm_vcpu *)arg; +struct ucall uc; +int old; +int r; +unsigned int exit_reason; + + r =pthread_setcanceltype (PTHREAD_CANCEL_ASYNCHRONOUS , &old); +TEST_ASSERT (r ==0 , +"pthread_setcanceltype failed on vcpu_id=%u with errno=%d" , + vcpu->id, r);
If possible,
TEST_ASSERT(!r, pthread_setcanceltype() failed on vcpu_id=%u, vcpu->id);
tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c line 562
Quoted 12 lines
tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c new file +12
-0+ +vcpu_run (vcpu); + exit_reason = vcpu->run->exit_reason; + +TEST_ASSERT (exit_reason ==KVM_EXIT_IO , +"vCPU %u exited with unexpected exit reason %u-%s, expected KVM_EXIT_IO" , + vcpu->id, exit_reason,exit_reason_str (exit_reason)); + +if (get_ucall (vcpu, &uc) ==UCALL_ABORT ) { +TEST_ASSERT (false, +"vCPU %u exited with error: %s.\n" , + vcpu->id, (const char *)uc.args[0 ]);
REPORT_GUEST_ASSERT_N?
Quoted 9 lines
tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c new file +9
-0+ } + +return NULL ; +} + +static void cancel_join_vcpu_thread (pthread_t thread,struct kvm_vcpu *vcpu) +{ +void *retval; +int r;
tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c line 581
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c new file +10
-0+ + r =pthread_cancel (thread); +TEST_ASSERT (r ==0 , +"pthread_cancel on vcpu_id=%d failed with errno=%d" , + vcpu->id, r); + + r =pthread_join (thread, &retval); +TEST_ASSERT (r ==0 , +"pthread_join on vcpu_id=%d failed with errno=%d" , + vcpu->id, r);
Same assert/errno comment here.
tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c line 663
Quoted 5 lines
tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c new file +5
-0+while (true) { + r =_vcpu_run (vcpu[0 ]); + exit_reason = vcpu[0 ]->run->exit_reason; + +TEST_ASSERT (!r,"vcpu_run failed: %d" , r);
Just use vcpu_run().
Quoted 29 lines
tools/testing/selftests/kvm/x86_64/hyperv_tlb_flush.c new file +26
-0+TEST_ASSERT (exit_reason ==KVM_EXIT_IO , +"unexpected exit reason: %u (%s)" , + exit_reason,exit_reason_str (exit_reason)); + +switch (get_ucall (vcpu[0 ], &uc)) { +case UCALL_SYNC : +TEST_ASSERT (uc.args[1 ] == stage, +"Unexpected stage: %ld (%d expected)\n" , + uc.args[1 ], stage); +break ; +case UCALL_ABORT : +REPORT_GUEST_ASSERT (uc); +/* NOT REACHED */ +case UCALL_DONE : +return 0 ; + } + + stage++; + } + +cancel_join_vcpu_thread (threads[0 ], vcpu[1 ]); +cancel_join_vcpu_thread (threads[1 ], vcpu[2 ]); +kvm_vm_free (vm); + +return 0 ; +}Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 38/46 KVM: selftests: Sync 'struct hv_enlightened_vmcs' definition with hyperv-tlfs.h
In reply to Vitaly Kuznetsov'struct hv_enlightened_vmcs' definition in selftests is not '__packed' and so we rely on the compiler doing the right padding. This is not obvious so it seems beneficial to use the same definition as in kernel.
--- tools/testing/selftests/kvm/include/x86_64/evmcs.h | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
tools/testing/selftests/kvm/include/x86_64/evmcs.h +6 -4
@@ -41,6 +41,8 @@ struct hv_enlightened_vmcs {u16 host_gs_selector;u16 host_tr_selector; +u16 padding16_1; +u64 host_ia32_pat;u64 host_ia32_efer;
@@ -159,7 +161,7 @@ struct hv_enlightened_vmcs {u64 ept_pointer;u16 virtual_processor_id; -u16 padding16[3 ]; +u16 padding16_2[3 ];u64 padding64_2[5 ];u64 guest_physical_address;
@@ -195,13 +197,13 @@ struct hv_enlightened_vmcs {u64 guest_rip;u32 hv_clean_fields; -u32 hv_padding_32; +u32 padding32_1;u32 hv_synthetic_controls;struct {u32 nested_flush_hypercall:1 ;u32 msr_bitmap:1 ;u32 reserved:30 ; - } hv_enlightenments_control; + } __packed hv_enlightenments_control;u32 hv_vp_id;u32 padding32_2;u64 hv_vm_id;
@@ -222,7 +224,7 @@ struct hv_enlightened_vmcs {u64 host_ssp;u64 host_ia32_int_ssp_table_addr;u64 padding64_6; -}; +} __packed;#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_NONE 0#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_IO_BITMAP BIT(0)
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 39/46 KVM: selftests: Sync 'struct hv_vp_assist_page' definition with hyperv-tlfs.h
In reply to Vitaly Kuznetsov'struct hv_vp_assist_page' definition doesn't match TLFS. Also, define 'struct hv_nested_enlightenments_control' and use it instead of opaque '__u64'.
--- .../selftests/kvm/include/x86_64/evmcs.h | 22 ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/include/x86_64/evmcs.h +17 -5
@@ -20,14 +20,26 @@extern bool enable_evmcs; +struct hv_nested_enlightenments_control { +struct { +__u32 directhypercall:1 ; +__u32 reserved:31 ; + } features; +struct { +__u32 reserved; + } hypercallControls; +} __packed; + +/* Define virtual processor assist page structure. */ struct hv_vp_assist_page {__u32 apic_assist; -__u32 reserved; -__u64 vtl_control[2 ]; -__u64 nested_enlightenments_control[2 ]; -__u32 enlighten_vmentry; +__u32 reserved1; +__u64 vtl_control[3 ]; +struct hv_nested_enlightenments_control nested_control; +__u8 enlighten_vmentry; +__u8 reserved2[7 ];__u64 current_nested_vmcs; -}; +} __packed;struct hv_enlightened_vmcs {u32 revision_id;
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 40/46 KVM: selftests: Move Hyper-V VP assist page enablement out of evmcs.h
In reply to Vitaly KuznetsovHyper-V VP assist page is not eVMCS specific, it is also used for enlightened nSVM. Move the code to vendor neutral place.
--- tools/testing/selftests/kvm/Makefile | 1 + .../selftests/kvm/include/x86_64/evmcs.h | 40 +------------------ .../selftests/kvm/include/x86_64/hyperv.h | 31 ++++++++++++++ .../testing/selftests/kvm/lib/x86_64/hyperv.c | 21 ++++++++++ .../testing/selftests/kvm/x86_64/evmcs_test.c | 1 + 5 files changed, 56 insertions(+), 38 deletions(-) create mode 100644 tools/testing/selftests/kvm/lib/x86_64/hyperv.c
tools/testing/selftests/kvm/Makefile +1 -0
@@ -50,6 +50,7 @@ LIBKVM += lib/test_util.c LIBKVM_x86_64 += lib/x86_64/apic.c LIBKVM_x86_64 += lib/x86_64/handlers.S +LIBKVM_x86_64 += lib/x86_64/hyperv.c LIBKVM_x86_64 += lib/x86_64/perf_test_util.c LIBKVM_x86_64 += lib/x86_64/processor.c LIBKVM_x86_64 += lib/x86_64/svm.c
tools/testing/selftests/kvm/include/x86_64/evmcs.h +2 -38
@@ -10,6 +10,7 @@#define SELFTEST_KVM_EVMCS_H #include <stdint.h> +#include "hyperv.h" #include "vmx.h" #define u16 uint16_t
@@ -20,27 +21,6 @@extern bool enable_evmcs; -struct hv_nested_enlightenments_control { -struct { -__u32 directhypercall:1 ; -__u32 reserved:31 ; - } features; -struct { -__u32 reserved; - } hypercallControls; -} __packed; - -/* Define virtual processor assist page structure. */ -struct hv_vp_assist_page { -__u32 apic_assist; -__u32 reserved1; -__u64 vtl_control[3 ]; -struct hv_nested_enlightenments_control nested_control; -__u8 enlighten_vmentry; -__u8 reserved2[7 ]; -__u64 current_nested_vmcs; -} __packed; -struct hv_enlightened_vmcs {u32 revision_id;u32 abort;
@@ -257,29 +237,13 @@ struct hv_enlightened_vmcs {#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ENLIGHTENMENTSCONTROL BIT(15)#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL 0xFFFF -#define HV_X64_MSR_VP_ASSIST_PAGE 0x40000073 -#define HV_X64_MSR_VP_ASSIST_PAGE_ENABLE 0x00000001 -#define HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_SHIFT 12 -#define HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK \ - (~((1ull << HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_SHIFT) - 1)) -extern struct hv_enlightened_vmcs *current_evmcs; -extern struct hv_vp_assist_page *current_vp_assist;int vcpu_enable_evmcs (struct kvm_vcpu *vcpu); -static inline int enable_vp_assist (uint64_t vp_assist_pa,void *vp_assist) +static inline void evmcs_enable (void ) { -u64 val = (vp_assist_pa &HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK ) | -HV_X64_MSR_VP_ASSIST_PAGE_ENABLE ; - -wrmsr (HV_X64_MSR_VP_ASSIST_PAGE , val); - - current_vp_assist = vp_assist; - enable_evmcs = true; - -return 0 ; }static inline int evmcs_vmptrld (uint64_t vmcs_pa,void *vmcs)
tools/testing/selftests/kvm/include/x86_64/hyperv.h +31 -0
@@ -236,4 +236,35 @@ static inline void hyperv_write_xmm_input(void *data, int n_sse_regs)/* Proper HV_X64_MSR_GUEST_OS_ID value */ #define HYPERV_LINUX_OS_ID ((u64)0x8100 << 48) +#define HV_X64_MSR_VP_ASSIST_PAGE 0x40000073 +#define HV_X64_MSR_VP_ASSIST_PAGE_ENABLE 0x00000001 +#define HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_SHIFT 12 +#define HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK \ + (~((1ull << HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_SHIFT) - 1)) + +struct hv_nested_enlightenments_control { +struct { +__u32 directhypercall:1 ; +__u32 reserved:31 ; + } features; +struct { +__u32 reserved; + } hypercallControls; +} __packed; + +/* Define virtual processor assist page structure. */ +struct hv_vp_assist_page { +__u32 apic_assist; +__u32 reserved1; +__u64 vtl_control[3 ]; +struct hv_nested_enlightenments_control nested_control; +__u8 enlighten_vmentry; +__u8 reserved2[7 ]; +__u64 current_nested_vmcs; +} __packed; + +extern struct hv_vp_assist_page *current_vp_assist; + +int enable_vp_assist (uint64_t vp_assist_pa,void *vp_assist); +#endif /* !SELFTEST_KVM_HYPERV_H */
tools/testing/selftests/kvm/lib/x86_64/hyperv.c new file +21 -0
@@ -0,0 +1,21 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Hyper-V specific functions. +* +* Copyright (C) 2021, Red Hat Inc. +*/ +#include <stdint.h> +#include "processor.h" +#include "hyperv.h" + +int enable_vp_assist (uint64_t vp_assist_pa,void *vp_assist) +{ +uint64_t val = (vp_assist_pa &HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK ) | +HV_X64_MSR_VP_ASSIST_PAGE_ENABLE ; + +wrmsr (HV_X64_MSR_VP_ASSIST_PAGE , val); + + current_vp_assist = vp_assist; + +return 0 ; +}
tools/testing/selftests/kvm/x86_64/evmcs_test.c +1 -0
@@ -79,6 +79,7 @@ void guest_code(struct vmx_pages *vmx_pages)GUEST_SYNC (2 );enable_vp_assist (vmx_pages->vp_assist_gpa, vmx_pages->vp_assist); +evmcs_enable ();GUEST_ASSERT (vmx_pages->vmcs_gpa);GUEST_ASSERT (prepare_for_vmx_operation (vmx_pages));
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 41/46 KVM: selftests: Split off load_evmcs() from load_vmcs()
In reply to Vitaly KuznetsovIn preparation to putting Hyper-V specific test pages to a dedicated struct, move eVMCS load logic from load_vmcs(). Tests call load_vmcs() directly and the only one which needs 'enlightened' version is evmcs_test so there's not much gain in having this merged.
Temporary pass both GPA and HVA to load_evmcs().
--- .../selftests/kvm/include/x86_64/evmcs.h | 10 ++++++ tools/testing/selftests/kvm/lib/x86_64/vmx.c | 33 ++++++++----------- .../testing/selftests/kvm/x86_64/evmcs_test.c | 4 +-- 3 files changed, 25 insertions(+), 22 deletions(-)
tools/testing/selftests/kvm/include/x86_64/evmcs.h +10 -0
@@ -256,6 +256,16 @@ static inline int evmcs_vmptrld(uint64_t vmcs_pa, void *vmcs)return 0 ; } +static inline bool load_evmcs (uint64_t enlightened_vmcs_gpa,void *enlightened_vmcs) +{ +if (evmcs_vmptrld (enlightened_vmcs_gpa, enlightened_vmcs)) +return false; + + current_evmcs->revision_id =EVMCS_VERSION ; + +return true; +} +static inline int evmcs_vmptrst (uint64_t *value) { *value = current_vp_assist->current_nested_vmcs &
tools/testing/selftests/kvm/lib/x86_64/vmx.c +13 -20
@@ -169,26 +169,19 @@ bool prepare_for_vmx_operation(struct vmx_pages *vmx)bool load_vmcs (struct vmx_pages *vmx) { -if (!enable_evmcs) { -/* Load a VMCS. */ - *(uint32_t *)(vmx->vmcs) =vmcs_revision (); -if (vmclear (vmx->vmcs_gpa)) -return false; - -if (vmptrld (vmx->vmcs_gpa)) -return false; - -/* Setup shadow VMCS, do not load it yet. */ - *(uint32_t *)(vmx->shadow_vmcs) = -vmcs_revision () |0x80000000ul ; -if (vmclear (vmx->shadow_vmcs_gpa)) -return false; - }else { -if (evmcs_vmptrld (vmx->enlightened_vmcs_gpa, - vmx->enlightened_vmcs)) -return false; - current_evmcs->revision_id =EVMCS_VERSION ; - } +/* Load a VMCS. */ + *(uint32_t *)(vmx->vmcs) =vmcs_revision (); +if (vmclear (vmx->vmcs_gpa)) +return false; + +if (vmptrld (vmx->vmcs_gpa)) +return false; + +/* Setup shadow VMCS, do not load it yet. */ + *(uint32_t *)(vmx->shadow_vmcs) = +vmcs_revision () |0x80000000ul ; +if (vmclear (vmx->shadow_vmcs_gpa)) +return false;return true; }
tools/testing/selftests/kvm/x86_64/evmcs_test.c +2 -2
@@ -81,10 +81,10 @@ void guest_code(struct vmx_pages *vmx_pages)enable_vp_assist (vmx_pages->vp_assist_gpa, vmx_pages->vp_assist);evmcs_enable (); -GUEST_ASSERT (vmx_pages->vmcs_gpa);GUEST_ASSERT (prepare_for_vmx_operation (vmx_pages));GUEST_SYNC (3 ); -GUEST_ASSERT (load_vmcs (vmx_pages)); +GUEST_ASSERT (load_evmcs (vmx_pages->enlightened_vmcs_gpa, + vmx_pages->enlightened_vmcs));GUEST_ASSERT (vmptrstz () == vmx_pages->enlightened_vmcs_gpa);GUEST_SYNC (4 );
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 42/46 KVM: selftests: Create a vendor independent helper to allocate Hyper-V specific test pages
In reply to Vitaly KuznetsovThere's no need to pollute VMX and SVM code with Hyper-V specific stuff and allocate Hyper-V specific test pages for all test as only few really need them. Create a dedicated struct and an allocation helper.
--- .../selftests/kvm/include/x86_64/evmcs.h | 4 ++-- .../selftests/kvm/include/x86_64/hyperv.h | 15 +++++++++++++ .../selftests/kvm/include/x86_64/vmx.h | 8 ------- .../testing/selftests/kvm/lib/x86_64/hyperv.c | 20 +++++++++++++++++ tools/testing/selftests/kvm/lib/x86_64/vmx.c | 12 ---------- .../testing/selftests/kvm/x86_64/evmcs_test.c | 22 +++++++++---------- 6 files changed, 48 insertions(+), 33 deletions(-)
tools/testing/selftests/kvm/include/x86_64/evmcs.h +2 -2
@@ -256,9 +256,9 @@ static inline int evmcs_vmptrld(uint64_t vmcs_pa, void *vmcs)return 0 ; } -static inline bool load_evmcs (uint64_t enlightened_vmcs_gpa,void *enlightened_vmcs) +static inline bool load_evmcs (struct hyperv_test_pages *hv) { -if (evmcs_vmptrld (enlightened_vmcs_gpa, enlightened_vmcs)) +if (evmcs_vmptrld (hv->enlightened_vmcs_gpa, hv->enlightened_vmcs))return false; current_evmcs->revision_id =EVMCS_VERSION ;
tools/testing/selftests/kvm/include/x86_64/hyperv.h +15 -0
@@ -267,4 +267,19 @@ extern struct hv_vp_assist_page *current_vp_assist;int enable_vp_assist (uint64_t vp_assist_pa,void *vp_assist); +struct hyperv_test_pages { +/* VP assist page */ +void *vp_assist_hva; +uint64_t vp_assist_gpa; +void *vp_assist; + +/* Enlightened VMCS */ +void *enlightened_vmcs_hva; +uint64_t enlightened_vmcs_gpa; +void *enlightened_vmcs; +}; + +struct hyperv_test_pages *vcpu_alloc_hyperv_test_pages (struct kvm_vm *vm, +vm_vaddr_t *p_hv_pages_gva); +#endif /* !SELFTEST_KVM_HYPERV_H */
tools/testing/selftests/kvm/include/x86_64/vmx.h +0 -8
@@ -517,14 +517,6 @@ struct vmx_pages {uint64_t vmwrite_gpa;void *vmwrite; -void *vp_assist_hva; -uint64_t vp_assist_gpa; -void *vp_assist; - -void *enlightened_vmcs_hva; -uint64_t enlightened_vmcs_gpa; -void *enlightened_vmcs; -void *eptp_hva;uint64_t eptp_gpa;void *eptp;
tools/testing/selftests/kvm/lib/x86_64/hyperv.c +20 -0
@@ -8,6 +8,26 @@#include "processor.h" #include "hyperv.h" +struct hyperv_test_pages *vcpu_alloc_hyperv_test_pages (struct kvm_vm *vm, +vm_vaddr_t *p_hv_pages_gva) +{ +vm_vaddr_t hv_pages_gva =vm_vaddr_alloc_page (vm); +struct hyperv_test_pages *hv =addr_gva2hva (vm, hv_pages_gva); + +/* Setup of a region of guest memory for the VP Assist page. */ + hv->vp_assist = (void *)vm_vaddr_alloc_page (vm); + hv->vp_assist_hva =addr_gva2hva (vm, (uintptr_t )hv->vp_assist); + hv->vp_assist_gpa =addr_gva2gpa (vm, (uintptr_t )hv->vp_assist); + +/* Setup of a region of guest memory for the enlightened VMCS. */ + hv->enlightened_vmcs = (void *)vm_vaddr_alloc_page (vm); + hv->enlightened_vmcs_hva =addr_gva2hva (vm, (uintptr_t )hv->enlightened_vmcs); + hv->enlightened_vmcs_gpa =addr_gva2gpa (vm, (uintptr_t )hv->enlightened_vmcs); + + *p_hv_pages_gva = hv_pages_gva; +return hv; +} +int enable_vp_assist (uint64_t vp_assist_pa,void *vp_assist) { uint64_t val = (vp_assist_pa & HV_X64_MSR_VP_ASSIST_PAGE_ADDRESS_MASK) |
tools/testing/selftests/kvm/lib/x86_64/vmx.c +0 -12
@@ -107,18 +107,6 @@ vcpu_alloc_vmx(struct kvm_vm *vm, vm_vaddr_t *p_vmx_gva) vmx->vmwrite_gpa =addr_gva2gpa (vm, (uintptr_t )vmx->vmwrite);memset (vmx->vmwrite_hva,0 ,getpagesize ()); -/* Setup of a region of guest memory for the VP Assist page. */ - vmx->vp_assist = (void *)vm_vaddr_alloc_page (vm); - vmx->vp_assist_hva =addr_gva2hva (vm, (uintptr_t )vmx->vp_assist); - vmx->vp_assist_gpa =addr_gva2gpa (vm, (uintptr_t )vmx->vp_assist); - -/* Setup of a region of guest memory for the enlightened VMCS. */ - vmx->enlightened_vmcs = (void *)vm_vaddr_alloc_page (vm); - vmx->enlightened_vmcs_hva = -addr_gva2hva (vm, (uintptr_t )vmx->enlightened_vmcs); - vmx->enlightened_vmcs_gpa = -addr_gva2gpa (vm, (uintptr_t )vmx->enlightened_vmcs); - *p_vmx_gva = vmx_gva;return vmx; }
tools/testing/selftests/kvm/x86_64/evmcs_test.c +11 -11
@@ -68,7 +68,7 @@ void l2_guest_code(void)vmcall (); } -void guest_code (struct vmx_pages *vmx_pages) +void guest_code (struct vmx_pages *vmx_pages,struct hyperv_test_pages *hv_pages) {#define L2_GUEST_STACK_SIZE 64unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE ];
@@ -78,23 +78,22 @@ void guest_code(struct vmx_pages *vmx_pages)GUEST_SYNC (1 );GUEST_SYNC (2 ); -enable_vp_assist (vmx_pages->vp_assist_gpa, vmx_pages->vp_assist); +enable_vp_assist (hv_pages->vp_assist_gpa, hv_pages->vp_assist);evmcs_enable ();GUEST_ASSERT (prepare_for_vmx_operation (vmx_pages));GUEST_SYNC (3 ); -GUEST_ASSERT (load_evmcs (vmx_pages->enlightened_vmcs_gpa, - vmx_pages->enlightened_vmcs)); -GUEST_ASSERT (vmptrstz () == vmx_pages->enlightened_vmcs_gpa); +GUEST_ASSERT (load_evmcs (hv_pages)); +GUEST_ASSERT (vmptrstz () == hv_pages->enlightened_vmcs_gpa);GUEST_SYNC (4 ); -GUEST_ASSERT (vmptrstz () == vmx_pages->enlightened_vmcs_gpa); +GUEST_ASSERT (vmptrstz () == hv_pages->enlightened_vmcs_gpa);prepare_vmcs (vmx_pages, l2_guest_code, &l2_guest_stack[L2_GUEST_STACK_SIZE ]);GUEST_SYNC (5 ); -GUEST_ASSERT (vmptrstz () == vmx_pages->enlightened_vmcs_gpa); +GUEST_ASSERT (vmptrstz () == hv_pages->enlightened_vmcs_gpa); current_evmcs->revision_id =-1u ;GUEST_ASSERT (vmlaunch ()); current_evmcs->revision_id =EVMCS_VERSION ;
@@ -104,7 +103,7 @@ void guest_code(struct vmx_pages *vmx_pages) PIN_BASED_NMI_EXITING);GUEST_ASSERT (!vmlaunch ()); -GUEST_ASSERT (vmptrstz () == vmx_pages->enlightened_vmcs_gpa); +GUEST_ASSERT (vmptrstz () == hv_pages->enlightened_vmcs_gpa);/* * NMI forces L2->L1 exit, resuming L2 and hope that EVMCS is
@@ -152,7 +151,7 @@ void guest_code(struct vmx_pages *vmx_pages)GUEST_SYNC (11 );/* Try enlightened vmptrld with an incorrect GPA */ -evmcs_vmptrld (0xdeadbeef , vmx_pages->enlightened_vmcs); +evmcs_vmptrld (0xdeadbeef , hv_pages->enlightened_vmcs);GUEST_ASSERT (vmlaunch ());GUEST_ASSERT (ud_count ==1 );GUEST_DONE ();
@@ -199,7 +198,7 @@ static struct kvm_vcpu *save_restore_vm(struct kvm_vm *vm,int main (int argc,char *argv[]) { -vm_vaddr_t vmx_pages_gva =0 ; +vm_vaddr_t vmx_pages_gva =0 , hv_pages_gva =0 ;struct kvm_vcpu *vcpu;struct kvm_vm *vm;
@@ -217,7 +216,8 @@ int main(int argc, char *argv[])vcpu_enable_evmcs (vcpu);vcpu_alloc_vmx (vm, &vmx_pages_gva); -vcpu_args_set (vcpu,1 , vmx_pages_gva); +vcpu_alloc_hyperv_test_pages (vm, &hv_pages_gva); +vcpu_args_set (vcpu,2 , vmx_pages_gva, hv_pages_gva);vm_init_descriptor_tables (vm);vcpu_init_descriptor_tables (vcpu);
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 43/46 KVM: selftests: Allocate Hyper-V partition assist page
In reply to Vitaly KuznetsovIn preparation to testing Hyper-V L2 TLB flush hypercalls, allocate so-called Partition assist page.
--- tools/testing/selftests/kvm/include/x86_64/hyperv.h | 5 +++++ tools/testing/selftests/kvm/lib/x86_64/hyperv.c | 5 +++++ 2 files changed, 10 insertions(+)
tools/testing/selftests/kvm/include/x86_64/hyperv.h +5 -0
@@ -273,6 +273,11 @@ struct hyperv_test_pages {uint64_t vp_assist_gpa;void *vp_assist; +/* Partition assist page */ +void *partition_assist_hva; +uint64_t partition_assist_gpa; +void *partition_assist; +/* Enlightened VMCS */ void *enlightened_vmcs_hva;uint64_t enlightened_vmcs_gpa;
tools/testing/selftests/kvm/lib/x86_64/hyperv.c +5 -0
@@ -19,6 +19,11 @@ struct hyperv_test_pages *vcpu_alloc_hyperv_test_pages(struct kvm_vm *vm, hv->vp_assist_hva =addr_gva2hva (vm, (uintptr_t )hv->vp_assist); hv->vp_assist_gpa =addr_gva2gpa (vm, (uintptr_t )hv->vp_assist); +/* Setup of a region of guest memory for the partition assist page. */ + hv->partition_assist = (void *)vm_vaddr_alloc_page (vm); + hv->partition_assist_hva =addr_gva2hva (vm, (uintptr_t )hv->partition_assist); + hv->partition_assist_gpa =addr_gva2gpa (vm, (uintptr_t )hv->partition_assist); +/* Setup of a region of guest memory for the enlightened VMCS. */ hv->enlightened_vmcs = (void *)vm_vaddr_alloc_page (vm); hv->enlightened_vmcs_hva =addr_gva2hva (vm, (uintptr_t )hv->enlightened_vmcs);
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 44/46 KVM: selftests: evmcs_test: Introduce L2 TLB flush test
In reply to Vitaly KuznetsovEnable Hyper-V L2 TLB flush and check that Hyper-V TLB flush hypercalls from L2 don't exit to L1 unless 'TlbLockCount' is set in the Partition assist page.
--- .../selftests/kvm/include/x86_64/evmcs.h | 2 + .../testing/selftests/kvm/x86_64/evmcs_test.c | 50 ++++++++++++++++++- 2 files changed, 50 insertions(+), 2 deletions(-)
tools/testing/selftests/kvm/include/x86_64/evmcs.h +2 -0
@@ -237,6 +237,8 @@ struct hv_enlightened_vmcs {#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ENLIGHTENMENTSCONTROL BIT(15)#define HV_VMX_ENLIGHTENED_CLEAN_FIELD_ALL 0xFFFF +#define HV_VMX_SYNTHETIC_EXIT_REASON_TRAP_AFTER_FLUSH 0x10000031 +extern struct hv_enlightened_vmcs *current_evmcs;int vcpu_enable_evmcs (struct kvm_vcpu *vcpu);
tools/testing/selftests/kvm/x86_64/evmcs_test.c +48 -2
105 lines not on this page · raw message
Signature · 1 line
2.37.3
Sean Christopherson
maintainerRe: [PATCH v11 44/46] KVM: selftests: evmcs_test: Introduce L2 TLB flush test
tools/testing/selftests/kvm/x86_64/evmcs_test.c void l2_guest_code(void) · line 72
Quoted 7 lines · Vitaly Kuznetsov
tools/testing/selftests/kvm/x86_64/evmcs_test.c +3
-0@@ -64,15 +67,33 @@ void l2_guest_code(void)vmcall ();rdmsr_gs_base ();/* intercepted */ +/* L2 TLB flush tests */ +hyperv_hypercall (HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE |HV_HYPERCALL_FAST_BIT ,0x0 , +HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES |HV_FLUSH_ALL_PROCESSORS );
Alignment is funky.
tools/testing/selftests/kvm/x86_64/evmcs_test.c void l2_guest_code(void) · line 76
Quoted 4 lines
tools/testing/selftests/kvm/x86_64/evmcs_test.c +4
-0+rdmsr_fs_base (); +/* +* Note: hypercall status (RAX) is not preserved correctly by L1 after +* synthetic vmexit, use unchecked version.
Nice...
Quoted 27 lines
tools/testing/selftests/kvm/x86_64/evmcs_test.c +12
-1+*/ +__hyperv_hypercall (HVCALL_FLUSH_VIRTUAL_ADDRESS_SPACE |HV_HYPERCALL_FAST_BIT ,0x0 , +HV_FLUSH_ALL_VIRTUAL_ADDRESS_SPACES |HV_FLUSH_ALL_PROCESSORS , + &unused); +/* Make sure we're no issuing Hyper-V TLB flush call again */ + __asm__ __volatile__ ("mov $0xdeadbeef, %rcx" ); +/* Done, exit to L1 and never come back. */ vmcall (); } -void guest_code (struct vmx_pages *vmx_pages,struct hyperv_test_pages *hv_pages) +void guest_code (struct vmx_pages *vmx_pages,struct hyperv_test_pages *hv_pages, +vm_vaddr_t hv_hcall_page_gpa) {#define L2_GUEST_STACK_SIZE 64unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE ]; +wrmsr (HV_X64_MSR_GUEST_OS_ID ,HYPERV_LINUX_OS_ID ); +wrmsr (HV_X64_MSR_HYPERCALL , hv_hcall_page_gpa); +x2apic_enable ();GUEST_SYNC (1 );@@ -102,6 +123,14 @@ void guest_code(struct vmx_pages *vmx_pages, struct hyperv_test_pages *hv_pages)vmwrite (PIN_BASED_VM_EXEC_CONTROL ,vmreadz (PIN_BASED_VM_EXEC_CONTROL ) |PIN_BASED_NMI_EXITING );
tools/testing/selftests/kvm/x86_64/evmcs_test.c void guest_code(struct vmx_pages *vmx_pages, struct hyperv_test_pages *hv_pages) · line 134
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/evmcs_test.c +8
-0+/* L2 TLB flush setup */ + current_evmcs->partition_assist_page = hv_pages->partition_assist_gpa; + current_evmcs->hv_enlightenments_control.nested_flush_hypercall =1 ; + current_evmcs->hv_vm_id =1 ; + current_evmcs->hv_vp_id =1 ; + current_vp_assist->nested_control.features.directhypercall =1 ; + *(u32 *)(hv_pages->partition_assist) =0 ; +GUEST_ASSERT (!vmlaunch ());
Pre-existing code, but would it make sense to add an assert here to verify L2 exited due to an NMI? Feel free to ignore this for now if it's not straightforward, this series is plenty big :-)
Vitaly Kuznetsov
maintainerRe: [PATCH v11 44/46] KVM: selftests: evmcs_test: Introduce L2 TLB flush test
Quoted 1 line · Sean Christopherson
On Tue, Oct 04, 2022, Vitaly Kuznetsov wrote:
...
Quoted 15 lines
tools/testing/selftests/kvm/x86_64/evmcs_test.c void guest_code(struct vmx_pages *vmx_pages, struct hyperv_test_pages *hv_pages) · line 134
Quoted 10 lines
tools/testing/selftests/kvm/x86_64/evmcs_test.c +8
-0+/* L2 TLB flush setup */ + current_evmcs->partition_assist_page = hv_pages->partition_assist_gpa; + current_evmcs->hv_enlightenments_control.nested_flush_hypercall =1 ; + current_evmcs->hv_vm_id =1 ; + current_evmcs->hv_vp_id =1 ; + current_vp_assist->nested_control.features.directhypercall =1 ; + *(u32 *)(hv_pages->partition_assist) =0 ; +GUEST_ASSERT (!vmlaunch ());Pre-existing code, but would it make sense to add an assert here to verify L2 exited due to an NMI? Feel free to ignore this for now if it's not straightforward, this series is plenty big :-)
Well, simple
+ GUEST_ASSERT_EQ(vmreadz(VM_EXIT_REASON), EXIT_REASON_EXCEPTION_NMI);
+ GUEST_ASSERT_EQ((vmreadz(VM_EXIT_INTR_INFO) & 0xff), NMI_VECTOR);
seems to be working (passing) so why not :-)
Signature · 1 line
Vitaly
Vitaly Kuznetsov
maintainerPATCH v11 45/46 KVM: selftests: hyperv_svm_test: Introduce L2 TLB flush test
In reply to Vitaly KuznetsovEnable Hyper-V L2 TLB flush and check that Hyper-V TLB flush hypercalls from L2 don't exit to L1 unless 'TlbLockCount' is set in the Partition assist page.
--- .../selftests/kvm/include/x86_64/svm.h | 4 ++ .../selftests/kvm/x86_64/hyperv_svm_test.c | 61 +++++++++++++++++-- 2 files changed, 60 insertions(+), 5 deletions(-)
tools/testing/selftests/kvm/include/x86_64/svm.h +4 -0
@@ -76,6 +76,10 @@ struct hv_vmcb_enlightenments {*/ #define HV_VMCB_NESTED_ENLIGHTENMENTS (1U << 31) +/* Synthetic VM-Exit */ +#define HV_SVM_EXITCODE_ENL 0xf0000000 +#define HV_SVM_ENL_EXITCODE_TRAP_AFTER_FLUSH (1) +struct __attribute__ ((__packed__ ))vmcb_control_area {u32 intercept_cr;u32 intercept_dr;
tools/testing/selftests/kvm/x86_64/hyperv_svm_test.c +56 -5
107 lines not on this page · raw message
Signature · 1 line
2.37.3
Vitaly Kuznetsov
maintainerPATCH v11 46/46 KVM: selftests: Rename 'evmcs_test' to 'hyperv_evmcs'
In reply to Vitaly KuznetsovConform to the rest of Hyper-V emulation selftests which have 'hyperv' prefix. Get rid of '_test' suffix as well as the purpose of this code is fairly obvious.
---
tools/testing/selftests/kvm/.gitignore | 2 +-
tools/testing/selftests/kvm/Makefile | 2 +-
.../selftests/kvm/x86_64/{evmcs_test.c => hyperv_evmcs.c} | 0
3 files changed, 2 insertions(+), 2 deletions(-)
rename tools/testing/selftests/kvm/x86_64/{evmcs_test.c => hyperv_evmcs.c} (100%)tools/testing/selftests/kvm/.gitignore +1 -1
@@ -15,7 +15,6 @@ /x86_64/cpuid_test /x86_64/cr4_cpuid_sync_test /x86_64/debug_regs -/x86_64/evmcs_test /x86_64/emulator_error_test /x86_64/fix_hypercall_test /x86_64/get_msr_index_features
@@ -23,6 +22,7 @@ /x86_64/kvm_pv_test /x86_64/hyperv_clock /x86_64/hyperv_cpuid +/x86_64/hyperv_evmcs /x86_64/hyperv_features /x86_64/hyperv_ipi /x86_64/hyperv_svm_test
tools/testing/selftests/kvm/Makefile +1 -1
@@ -79,11 +79,11 @@ TEST_PROGS_x86_64 += x86_64/nx_huge_pages_test.sh TEST_GEN_PROGS_x86_64 = x86_64/cpuid_test TEST_GEN_PROGS_x86_64 += x86_64/cr4_cpuid_sync_test TEST_GEN_PROGS_x86_64 += x86_64/get_msr_index_features -TEST_GEN_PROGS_x86_64 += x86_64/evmcs_test TEST_GEN_PROGS_x86_64 += x86_64/emulator_error_test TEST_GEN_PROGS_x86_64 += x86_64/fix_hypercall_test TEST_GEN_PROGS_x86_64 += x86_64/hyperv_clock TEST_GEN_PROGS_x86_64 += x86_64/hyperv_cpuid +TEST_GEN_PROGS_x86_64 += x86_64/hyperv_evmcs TEST_GEN_PROGS_x86_64 += x86_64/hyperv_features TEST_GEN_PROGS_x86_64 += x86_64/hyperv_ipi TEST_GEN_PROGS_x86_64 += x86_64/hyperv_svm_test
tools/testing/selftests/kvm/x86_64/hyperv_evmcs.c renamed from tools/testing/selftests/kvm/x86_64/evmcs_test.c +0 -0
Signature · 1 line
2.37.3
Sean Christopherson
maintainerQuoted 15 lines · Vitaly Kuznetsov
Changes since v10 (Sean):
- New patches added:
- "x86/hyperv: Move VMCB enlightenment definitions tohyperv-tlfs.h"
- "KVM: selftests: Move "structhv_enlightenments" tox86_64/svm.h"
- "KVM: SVM: Add a proper field for Hyper-V VMCB enlightenments"
- 'x86/hyperv: KVM: Rename "hv_enlightenments" to "hv_vmcb_enlightenments"'
- 'KVM: VMX: Rename "vmx/evmcs.{ch}" to "vmx/hyperv.{ch}"'
- "KVM: x86: Move clearing ofTLB_FLUSH_CURRENTtokvm_vcpu_flush_tlb_all()"
- "KVM: selftests: Drop helpers to read/write page table entries"
- "KVM: x86: Makekvm_hv_get_assist_page()return 0/-errno"
- Removed patches:
- "KVM: selftests: Export_vm_get_page_table_entry()"
- Main differences:
- Move Hyper-V TLB flushing out ofkvm_service_local_tlb_flush_requests(). On SVM, Hyper-V TLB flush FIFO is emptied fromsvm_flush_tlb_current()
Quoted 8 lines
- Don't disable IRQs in
hv_tlb_flush_enqueue().
- Don't callkvm_vcpu_flush_tlb_guest()fromkvm_hv_vcpu_flush_tlb()but return -errno instead.
- Avoid unneded flushes in !EPT/!NPT cases.
- Optimizehv_is_vp_in_sparse_set().
- Move TLFS definitions toasm/hyperv-tlfs.h.
- Use u64 vals in Hyper-V PV TLB flush selftest + multiple smaler changes
- Typos, indentation, renames, ...
Some nits throughout, but nothing major. Everything could be fixed up when applying, but if it's not too much trouble I'd prefer a v11, the potential changes to kvm_hv_hypercall_complete() aren't completely trivial.
Vitaly Kuznetsov
maintainerQuoted 25 lines · Sean Christopherson
Quoted 15 lines · Vitaly Kuznetsov
Changes since v10 (Sean):
- New patches added:
- "x86/hyperv: Move VMCB enlightenment definitions tohyperv-tlfs.h"
- "KVM: selftests: Move "structhv_enlightenments" tox86_64/svm.h"
- "KVM: SVM: Add a proper field for Hyper-V VMCB enlightenments"
- 'x86/hyperv: KVM: Rename "hv_enlightenments" to "hv_vmcb_enlightenments"'
- 'KVM: VMX: Rename "vmx/evmcs.{ch}" to "vmx/hyperv.{ch}"'
- "KVM: x86: Move clearing ofTLB_FLUSH_CURRENTtokvm_vcpu_flush_tlb_all()"
- "KVM: selftests: Drop helpers to read/write page table entries"
- "KVM: x86: Makekvm_hv_get_assist_page()return 0/-errno"
- Removed patches:
- "KVM: selftests: Export_vm_get_page_table_entry()"
- Main differences:
- Move Hyper-V TLB flushing out ofkvm_service_local_tlb_flush_requests(). On SVM, Hyper-V TLB flush FIFO is emptied fromsvm_flush_tlb_current()Quoted 8 lines
- Don't disable IRQs in
hv_tlb_flush_enqueue().
- Don't callkvm_vcpu_flush_tlb_guest()fromkvm_hv_vcpu_flush_tlb()but return -errno instead.
- Avoid unneded flushes in !EPT/!NPT cases.
- Optimizehv_is_vp_in_sparse_set().
- Move TLFS definitions toasm/hyperv-tlfs.h.
- Use u64 vals in Hyper-V PV TLB flush selftest + multiple smaler changes
- Typos, indentation, renames, ...
Quoted 3 lines
Some nits throughout, but nothing major. Everything could be fixed up when applying, but if it's not too much trouble I'd prefer a v11, the potential changes to
kvm_hv_hypercall_complete()aren't completely trivial.
Thanks for the review! Let me do v12 to address your comments, I plan to do it tomorrow.
Signature · 1 line
Vitaly