KVM: x86: Provide a capability to disable APERF/MPERF read intercepts
Subsystems (5)
v4 cover KVM: x86: Provide a capability to disable APERF/MPERF read intercepts
Allow a guest to read IA32_APERF and IA32_MPERF, so that it can determine the effective frequency multiplier for the physical LPU.
Commit b51700632e0e ("KVM: X86: Provide a capability to disable cstate msr read intercepts") allowed the userspace VMM to grant a guest read access to four core C-state residency MSRs. Do the same for IA32_APERF and IA32_MPERF.
While this isn't sufficient to claim support for CPUID.6:ECX.APERFMPERF[bit 0], it may suffice in a sufficiently restricted environment (i.e. vCPUs pinned to LPUs, no TSC multiplier, and no suspend/resume).
v1 -> v2: Add {IA32_APERF,IA32_MPERF} to vmx_possible_passthrough_msrs[] v2 -> v3: Add a selftest
v3 -> v4: Collect all disabled_exit flags in a u64 [Sean]
Improve documentation [Sean]
Add pin_task_to_one_cpu() to kvm selftests library [Sean]
Jim Mattson (3):
KVM: x86: Replace growing set of *_in_guest bools with a u64
KVM: x86: Provide a capability to disable APERF/MPERF read intercepts
KVM: selftests: Test behavior of KVM_X86_DISABLE_EXITS_APERFMPERF
Documentation/virt/kvm/api.rst | 23 +++ arch/x86/include/asm/kvm_host.h | 5 +- arch/x86/kvm/svm/svm.c | 9 +- arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/vmx/vmx.c | 8 +- arch/x86/kvm/vmx/vmx.h | 2 +- arch/x86/kvm/x86.c | 16 ++- arch/x86/kvm/x86.h | 18 ++- include/uapi/linux/kvm.h | 1 + tools/include/uapi/linux/kvm.h | 1 + tools/testing/selftests/kvm/Makefile.kvm | 1 + .../testing/selftests/kvm/include/kvm_util.h | 2 + tools/testing/selftests/kvm/lib/kvm_util.c | 17 +++ .../selftests/kvm/x86/aperfmperf_test.c | 132 ++++++++++++++++++ 14 files changed, 220 insertions(+), 17 deletions(-) create mode 100644 tools/testing/selftests/kvm/x86/aperfmperf_test.c
Signature · 1 line
2.49.0.1204.g71687c7c1d-goog
PATCH v4 1/3 KVM: x86: Replace growing set of *_in_guest bools with a u64
Store each "disabled exit" boolean in a single bit rather than a byte.
No functional change intended.
--- arch/x86/include/asm/kvm_host.h | 5 +---- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- arch/x86/kvm/x86.c | 8 ++++---- arch/x86/kvm/x86.h | 13 +++++++++---- 5 files changed, 16 insertions(+), 14 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -4
@@ -1390,10 +1390,7 @@ struct kvm_arch {gpa_t wall_clock; -bool mwait_in_guest; -bool hlt_in_guest; -bool pause_in_guest; -bool cstate_in_guest; +u64 disabled_exits;unsigned long irq_sources_bitmap;s64 kvmclock_offset;
arch/x86/kvm/svm/svm.c +1 -1
@@ -5102,7 +5102,7 @@ static int svm_vm_init(struct kvm *kvm) }if (!pause_filter_count || !pause_filter_thresh) - kvm->arch.pause_in_guest = true; +kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_PAUSE );if (enable_apicv) {int ret =avic_vm_init (kvm);
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -7619,7 +7619,7 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)int vmx_vm_init (struct kvm *kvm) {if (!ple_gap) - kvm->arch.pause_in_guest = true; +kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_PAUSE );if (boot_cpu_has (X86_BUG_L1TF ) && enable_ept) {switch (l1tf_mitigation) {
arch/x86/kvm/x86.c +4 -4
@@ -6605,13 +6605,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,pr_warn_once (SMT_RSB_MSG );if (cap->args[0 ] &KVM_X86_DISABLE_EXITS_PAUSE ) - kvm->arch.pause_in_guest = true; +kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_PAUSE );if (cap->args[0 ] &KVM_X86_DISABLE_EXITS_MWAIT ) - kvm->arch.mwait_in_guest = true; +kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_MWAIT );if (cap->args[0 ] &KVM_X86_DISABLE_EXITS_HLT ) - kvm->arch.hlt_in_guest = true; +kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_HLT );if (cap->args[0 ] &KVM_X86_DISABLE_EXITS_CSTATE ) - kvm->arch.cstate_in_guest = true; +kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_CSTATE ); r =0 ; disable_exits_unlock:mutex_unlock (&kvm->lock);
arch/x86/kvm/x86.h +9 -4
@@ -481,24 +481,29 @@ static inline u64 nsec_to_cycles(struct kvm_vcpu *vcpu, u64 nsec) __rem; \ }) +static inline void kvm_disable_exits (struct kvm *kvm,u64 mask) +{ + kvm->arch.disabled_exits |= mask; +} +static inline bool kvm_mwait_in_guest (struct kvm *kvm) { -return kvm->arch.mwait_in_guest; +return kvm->arch.disabled_exits &KVM_X86_DISABLE_EXITS_MWAIT ; }static inline bool kvm_hlt_in_guest (struct kvm *kvm) { -return kvm->arch.hlt_in_guest; +return kvm->arch.disabled_exits &KVM_X86_DISABLE_EXITS_HLT ; }static inline bool kvm_pause_in_guest (struct kvm *kvm) { -return kvm->arch.pause_in_guest; +return kvm->arch.disabled_exits &KVM_X86_DISABLE_EXITS_PAUSE ; }static inline bool kvm_cstate_in_guest (struct kvm *kvm) { -return kvm->arch.cstate_in_guest; +return kvm->arch.disabled_exits &KVM_X86_DISABLE_EXITS_CSTATE ; }static inline bool kvm_notify_vmexit_enabled (struct kvm *kvm)
Signature · 1 line
2.49.0.1204.g71687c7c1d-goog
Sean Christopherson
maintainerRe: [PATCH v4 1/3] KVM: x86: Replace growing set of *_in_guest bools with a u64
Quoted 12 lines · Jim Mattson
arch/x86/kvm/x86.c +1
-2@@ -6605,13 +6605,13 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,pr_warn_once (SMT_RSB_MSG );if (cap->args[0 ] &KVM_X86_DISABLE_EXITS_PAUSE ) - kvm->arch.pause_in_guest = true; +kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_PAUSE );if (cap->args[0 ] &KVM_X86_DISABLE_EXITS_MWAIT ) - kvm->arch.mwait_in_guest = true;
arch/x86/kvm/x86.c int kvm_vm_ioctl_enable_cap(struct kvm *kvm, · line 6617
Quoted 10 lines
arch/x86/kvm/x86.c +3
-2+kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_MWAIT );if (cap->args[0 ] &KVM_X86_DISABLE_EXITS_HLT ) - kvm->arch.hlt_in_guest = true; +kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_HLT );if (cap->args[0 ] &KVM_X86_DISABLE_EXITS_CSTATE ) - kvm->arch.cstate_in_guest = true; +kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_CSTATE ); r =0 ; disable_exits_unlock:mutex_unlock (&kvm->lock);
Can't this simply be? The set of capabilities to disable has already been vetted, so I don't see any reason to manually process each flag.
mutex_lock(&kvm->lock); if (kvm->created_vcpus) goto disable_exits_unlock;
#define SMT_RSB_MSG "This processor is affected by the Cross-Thread Return Predictions vulnerability. " \ "KVM_CAP_X86_DISABLE_EXITS should only be used with SMT disabled or trusted guests." if (!mitigate_smt_rsb && boot_cpu_has_bug(X86_BUG_SMT_RSB) && cpu_smt_possible() && (cap->args[0] & ~(KVM_X86_DISABLE_EXITS_PAUSE | KVM_X86_DISABLE_EXITS_APERFMPERF))) pr_warn_once(SMT_RSB_MSG); kvm_disable_exits(kvm, cap->args[0]); r = 0; disable_exits_unlock: mutex_unlock(&kvm->lock); break;
Re: [PATCH v4 1/3] KVM: x86: Replace growing set of *_in_guest bools with a u64
Quoted 2 lines · Sean Christopherson
Can't this simply be? The set of capabilities to disable has already been vetted, so I don't see any reason to manually process each flag.
I love it! Thank you.
PATCH v4 2/3 KVM: x86: Provide a capability to disable APERF/MPERF read intercepts
In reply to Jim MattsonAllow a guest to read the physical IA32_APERF and IA32_MPERF MSRs without interception.
The IA32_APERF and IA32_MPERF MSRs are not virtualized. Writes are not handled at all. The MSR values are not zeroed on vCPU creation, saved on suspend, or restored on resume. No accommodation is made for processor migration or for sharing a logical processor with other tasks. No adjustments are made for non-unit TSC multipliers. The MSRs do not account for time the same way as the comparable PMU events, whether the PMU is virtualized by the traditional emulation method or the new mediated pass-through approach.
Nonetheless, in a properly constrained environment, this capability can be combined with a guest CPUID table that advertises support for CPUID.6:ECX.APERFMPERF[bit 0] to induce a Linux guest to report the effective physical CPU frequency in /proc/cpuinfo. Moreover, there is no performance cost for this capability.
--- Documentation/virt/kvm/api.rst | 23 +++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 7 +++++++ arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/vmx/vmx.c | 6 ++++++ arch/x86/kvm/vmx/vmx.h | 2 +- arch/x86/kvm/x86.c | 8 +++++++- arch/x86/kvm/x86.h | 5 +++++ include/uapi/linux/kvm.h | 1 + tools/include/uapi/linux/kvm.h | 1 + 9 files changed, 52 insertions(+), 3 deletions(-)
Documentation/virt/kvm/api.rst +23 -0
@@ -7780,6 +7780,7 @@ Valid bits in args[0] are:: #define KVM_X86_DISABLE_EXITS_HLT (1 << 1) #define KVM_X86_DISABLE_EXITS_PAUSE (1 << 2) #define KVM_X86_DISABLE_EXITS_CSTATE (1 << 3) + #define KVM_X86_DISABLE_EXITS_APERFMPERF (1 << 4) Enabling this capability on a VM provides userspace with a way to no longer intercept some instructions for improved latency in some
@@ -7790,6 +7791,28 @@ all such vmexits. Do not enable KVM_FEATURE_PV_UNHALT if you disable HLT exits. +Virtualizing the ``IA32_APERF`` and ``IA32_MPERF`` MSRs requires more +than just disabling APERF/MPERF exits. While both Intel and AMD +document strict usage conditions for these MSRs--emphasizing that only +the ratio of their deltas over a time interval (T0 to T1) is +architecturally defined--simply passing through the MSRs can still +produce an incorrect ratio. + +This erroneous ratio can occur if, between T0 and T1: + +1. The vCPU thread migrates between logical processors. +2. Live migration or suspend/resume operations take place. +3. Another task shares the vCPU's logical processor. +4. C-states lower thean C0 are emulated (e.g., via HLT interception). +5. The guest TSC frequency doesn't match the host TSC frequency. + +Due to these complexities, KVM does not automatically associate this +passthrough capability with the guest CPUID bit, +``CPUID.6:ECX.APERFMPERF[bit 0]``. Userspace VMMs that deem this +mechanism adequate for virtualizing the ``IA32_APERF`` and +``IA32_MPERF`` MSRs must set the guest CPUID bit explicitly. + + 7.14 KVM_CAP_S390_HPAGE_1M --------------------------
arch/x86/kvm/svm/svm.c +7 -0
@@ -112,6 +112,8 @@ static const struct svm_direct_access_msrs { { .index = MSR_IA32_CR_PAT, .always = false }, { .index = MSR_AMD64_SEV_ES_GHCB, .always = true }, { .index = MSR_TSC_AUX, .always = false }, + { .index = MSR_IA32_APERF, .always = false }, + { .index = MSR_IA32_MPERF, .always = false }, { .index = X2APIC_MSR(APIC_ID), .always = false }, { .index = X2APIC_MSR(APIC_LVR), .always = false }, { .index = X2APIC_MSR(APIC_TASKPRI), .always = false },
@@ -1357,6 +1359,11 @@ static void init_vmcb(struct kvm_vcpu *vcpu)if (boot_cpu_has (X86_FEATURE_V_SPEC_CTRL ))set_msr_interception (vcpu, svm->msrpm,MSR_IA32_SPEC_CTRL ,1 ,1 ); +if (kvm_aperfmperf_in_guest (vcpu->kvm)) { +set_msr_interception (vcpu, svm->msrpm,MSR_IA32_APERF ,1 ,0 ); +set_msr_interception (vcpu, svm->msrpm,MSR_IA32_MPERF ,1 ,0 ); + } +if (kvm_vcpu_apicv_active (vcpu))avic_init_vmcb (svm, vmcb);
arch/x86/kvm/svm/svm.h +1 -1
@@ -44,7 +44,7 @@ static inline struct page *__sme_pa_to_page(unsigned long pa)#define IOPM_SIZE PAGE_SIZE * 3#define MSRPM_SIZE PAGE_SIZE * 2 -#define MAX_DIRECT_ACCESS_MSRS 48 +#define MAX_DIRECT_ACCESS_MSRS 50#define MSRPM_OFFSETS 32extern u32 msrpm_offsets[MSRPM_OFFSETS ]__read_mostly ;extern bool npt_enabled;
arch/x86/kvm/vmx/vmx.c +6 -0
@@ -188,6 +188,8 @@ static u32 vmx_possible_passthrough_msrs[MAX_POSSIBLE_PASSTHROUGH_MSRS] = {MSR_CORE_C3_RESIDENCY ,MSR_CORE_C6_RESIDENCY ,MSR_CORE_C7_RESIDENCY , +MSR_IA32_APERF , + MSR_IA32_MPERF, };/*
@@ -7569,6 +7571,10 @@ int vmx_vcpu_create(struct kvm_vcpu *vcpu)vmx_disable_intercept_for_msr (vcpu,MSR_CORE_C6_RESIDENCY ,MSR_TYPE_R );vmx_disable_intercept_for_msr (vcpu,MSR_CORE_C7_RESIDENCY ,MSR_TYPE_R ); } +if (kvm_aperfmperf_in_guest (vcpu->kvm)) { +vmx_disable_intercept_for_msr (vcpu,MSR_IA32_APERF ,MSR_TYPE_R ); +vmx_disable_intercept_for_msr (vcpu,MSR_IA32_MPERF ,MSR_TYPE_R ); + } vmx->loaded_vmcs = &vmx->vmcs01;
arch/x86/kvm/vmx/vmx.h +1 -1
@@ -297,7 +297,7 @@ struct vcpu_vmx {struct lbr_desc lbr_desc;/* Save desired MSR intercept (read: pass-through) state */ -#define MAX_POSSIBLE_PASSTHROUGH_MSRS 16 +#define MAX_POSSIBLE_PASSTHROUGH_MSRS 18struct { DECLARE_BITMAP(read, MAX_POSSIBLE_PASSTHROUGH_MSRS); DECLARE_BITMAP(write, MAX_POSSIBLE_PASSTHROUGH_MSRS);
arch/x86/kvm/x86.c +7 -1
@@ -4574,6 +4574,9 @@ static u64 kvm_get_allowed_disable_exits(void) {u64 r =KVM_X86_DISABLE_EXITS_PAUSE ; +if (boot_cpu_has (X86_FEATURE_APERFMPERF )) + r |=KVM_X86_DISABLE_EXITS_APERFMPERF ; +if (!mitigate_smt_rsb) { r |=KVM_X86_DISABLE_EXITS_HLT |KVM_X86_DISABLE_EXITS_CSTATE ;
@@ -6601,7 +6604,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,if (!mitigate_smt_rsb &&boot_cpu_has_bug (X86_BUG_SMT_RSB ) &&cpu_smt_possible () && - (cap->args[0 ] & ~KVM_X86_DISABLE_EXITS_PAUSE )) + (cap->args[0 ] & ~(KVM_X86_DISABLE_EXITS_PAUSE | +KVM_X86_DISABLE_EXITS_APERFMPERF )))pr_warn_once (SMT_RSB_MSG );if (cap->args[0 ] &KVM_X86_DISABLE_EXITS_PAUSE )
@@ -6612,6 +6616,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_HLT );if (cap->args[0 ] &KVM_X86_DISABLE_EXITS_CSTATE )kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_CSTATE ); +if (cap->args[0 ] &KVM_X86_DISABLE_EXITS_APERFMPERF ) +kvm_disable_exits (kvm,KVM_X86_DISABLE_EXITS_APERFMPERF ); r =0 ; disable_exits_unlock:mutex_unlock (&kvm->lock);
arch/x86/kvm/x86.h +5 -0
@@ -506,6 +506,11 @@ static inline bool kvm_cstate_in_guest(struct kvm *kvm)return kvm->arch.disabled_exits &KVM_X86_DISABLE_EXITS_CSTATE ; } +static inline bool kvm_aperfmperf_in_guest (struct kvm *kvm) +{ +return kvm->arch.disabled_exits &KVM_X86_DISABLE_EXITS_APERFMPERF ; +} +static inline bool kvm_notify_vmexit_enabled (struct kvm *kvm) {return kvm->arch.notify_vmexit_flags &KVM_X86_NOTIFY_VMEXIT_ENABLED ;
include/uapi/linux/kvm.h +1 -0
@@ -618,6 +618,7 @@ struct kvm_ioeventfd {#define KVM_X86_DISABLE_EXITS_HLT (1 << 1)#define KVM_X86_DISABLE_EXITS_PAUSE (1 << 2)#define KVM_X86_DISABLE_EXITS_CSTATE (1 << 3) +#define KVM_X86_DISABLE_EXITS_APERFMPERF (1 << 4)/* for KVM_ENABLE_CAP */ struct kvm_enable_cap {
tools/include/uapi/linux/kvm.h +1 -0
@@ -617,6 +617,7 @@ struct kvm_ioeventfd {#define KVM_X86_DISABLE_EXITS_HLT (1 << 1)#define KVM_X86_DISABLE_EXITS_PAUSE (1 << 2)#define KVM_X86_DISABLE_EXITS_CSTATE (1 << 3) +#define KVM_X86_DISABLE_EXITS_APERFMPERF (1 << 4)/* for KVM_ENABLE_CAP */ struct kvm_enable_cap {
Signature · 1 line
2.49.0.1204.g71687c7c1d-goog
Sean Christopherson
maintainerQuoted 19 lines · Jim Mattson
Allow a guest to read the physical
IA32_APERFandIA32_MPERFMSRs without interception.The
IA32_APERFandIA32_MPERFMSRs are not virtualized. Writes are not handled at all. The MSR values are not zeroed on vCPU creation, saved on suspend, or restored on resume. No accommodation is made for processor migration or for sharing a logical processor with other tasks. No adjustments are made for non-unit TSC multipliers. The MSRs do not account for time the same way as the comparable PMU events, whether the PMU is virtualized by the traditional emulation method or the new mediated pass-through approach.Nonetheless, in a properly constrained environment, this capability can be combined with a guest CPUID table that advertises support for CPUID.6:ECX.APERFMPERF[bit 0] to induce a Linux guest to report the effective physical CPU frequency in
/proc/cpuinfo. Moreover, there is no performance cost for this capability.Signed-off-by Jim Mattson <[email protected]>
Quoted 11 lines
--- Documentation/virt/kvm/api.rst | 23 +++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 7 +++++++ arch/x86/kvm/svm/svm.h | 2 +- arch/x86/kvm/vmx/vmx.c | 6 ++++++ arch/x86/kvm/vmx/vmx.h | 2 +- arch/x86/kvm/x86.c | 8 +++++++- arch/x86/kvm/x86.h | 5 +++++ include/uapi/linux/kvm.h | 1 + tools/include/uapi/linux/kvm.h | 1 + 9 files changed, 52 insertions(+), 3 deletions(-)
This needs to be rebased on top of the MSR interception rework, which I've now pushed to kvm-x86 next. Luckily, it's quite painless. Compile tested only at this point (about to throw it onto metal).
I'd be happy to post a v5 on your behalf (pending your thoughts on my feedback to patch 1), unless you want the honors. The fixup is a wee bit more than I'm comfortable doing on-the-fly.
--- Documentation/virt/kvm/api.rst | 23 +++++++++++++++++++++++ arch/x86/kvm/svm/svm.c | 5 +++++ arch/x86/kvm/vmx/vmx.c | 4 ++++ arch/x86/kvm/x86.c | 6 +++++- arch/x86/kvm/x86.h | 5 +++++ include/uapi/linux/kvm.h | 1 + tools/include/uapi/linux/kvm.h | 1 + 7 files changed, 44 insertions(+), 1 deletion(-)
Documentation/virt/kvm/api.rst +23 -0
@@ -7844,6 +7844,7 @@ Valid bits in args[0] are:: #define KVM_X86_DISABLE_EXITS_HLT (1 << 1) #define KVM_X86_DISABLE_EXITS_PAUSE (1 << 2) #define KVM_X86_DISABLE_EXITS_CSTATE (1 << 3) + #define KVM_X86_DISABLE_EXITS_APERFMPERF (1 << 4) Enabling this capability on a VM provides userspace with a way to no longer intercept some instructions for improved latency in some
@@ -7854,6 +7855,28 @@ all such vmexits. Do not enable KVM_FEATURE_PV_UNHALT if you disable HLT exits. +Virtualizing the ``IA32_APERF`` and ``IA32_MPERF`` MSRs requires more +than just disabling APERF/MPERF exits. While both Intel and AMD +document strict usage conditions for these MSRs--emphasizing that only +the ratio of their deltas over a time interval (T0 to T1) is +architecturally defined--simply passing through the MSRs can still +produce an incorrect ratio. + +This erroneous ratio can occur if, between T0 and T1: + +1. The vCPU thread migrates between logical processors. +2. Live migration or suspend/resume operations take place. +3. Another task shares the vCPU's logical processor. +4. C-states lower thean C0 are emulated (e.g., via HLT interception). +5. The guest TSC frequency doesn't match the host TSC frequency. + +Due to these complexities, KVM does not automatically associate this +passthrough capability with the guest CPUID bit, +``CPUID.6:ECX.APERFMPERF[bit 0]``. Userspace VMMs that deem this +mechanism adequate for virtualizing the ``IA32_APERF`` and +``IA32_MPERF`` MSRs must set the guest CPUID bit explicitly. + + 7.14 KVM_CAP_S390_HPAGE_1M --------------------------
arch/x86/kvm/svm/svm.c +5 -0
@@ -854,6 +854,11 @@ static void svm_recalc_msr_intercepts(struct kvm_vcpu *vcpu)svm_set_intercept_for_msr (vcpu,MSR_IA32_SYSENTER_ESP ,MSR_TYPE_RW ,guest_cpuid_is_intel_compatible (vcpu)); +if (kvm_aperfmperf_in_guest (vcpu->kvm)) { +svm_disable_intercept_for_msr (vcpu,MSR_IA32_APERF ,MSR_TYPE_R ); +svm_disable_intercept_for_msr (vcpu,MSR_IA32_MPERF ,MSR_TYPE_R ); + } +if (sev_es_guest (vcpu->kvm))sev_es_recalc_msr_intercepts (vcpu);
arch/x86/kvm/vmx/vmx.c +4 -0
@@ -4099,6 +4099,10 @@ void vmx_recalc_msr_intercepts(struct kvm_vcpu *vcpu)vmx_disable_intercept_for_msr (vcpu,MSR_CORE_C6_RESIDENCY ,MSR_TYPE_R );vmx_disable_intercept_for_msr (vcpu,MSR_CORE_C7_RESIDENCY ,MSR_TYPE_R ); } +if (kvm_aperfmperf_in_guest (vcpu->kvm)) { +vmx_disable_intercept_for_msr (vcpu,MSR_IA32_APERF ,MSR_TYPE_R ); +vmx_disable_intercept_for_msr (vcpu,MSR_IA32_MPERF ,MSR_TYPE_R ); + }/* PT MSRs can be passed through iff PT is exposed to the guest. */ if (vmx_pt_mode_is_host_guest ())
arch/x86/kvm/x86.c +5 -1
@@ -4580,6 +4580,9 @@ static u64 kvm_get_allowed_disable_exits(void) {u64 r =KVM_X86_DISABLE_EXITS_PAUSE ; +if (boot_cpu_has (X86_FEATURE_APERFMPERF )) + r |=KVM_X86_DISABLE_EXITS_APERFMPERF ; +if (!mitigate_smt_rsb) { r |=KVM_X86_DISABLE_EXITS_HLT |KVM_X86_DISABLE_EXITS_CSTATE ;
@@ -6478,7 +6481,8 @@ int kvm_vm_ioctl_enable_cap(struct kvm *kvm,if (!mitigate_smt_rsb &&boot_cpu_has_bug (X86_BUG_SMT_RSB ) &&cpu_smt_possible () && - (cap->args[0 ] & ~KVM_X86_DISABLE_EXITS_PAUSE )) + (cap->args[0 ] & ~(KVM_X86_DISABLE_EXITS_PAUSE | +KVM_X86_DISABLE_EXITS_APERFMPERF )))pr_warn_once (SMT_RSB_MSG );kvm_disable_exits (kvm, cap->args[0 ]);
arch/x86/kvm/x86.h +5 -0
@@ -546,6 +546,11 @@ static inline bool kvm_cstate_in_guest(struct kvm *kvm)return kvm->arch.disabled_exits &KVM_X86_DISABLE_EXITS_CSTATE ; } +static inline bool kvm_aperfmperf_in_guest (struct kvm *kvm) +{ +return kvm->arch.disabled_exits &KVM_X86_DISABLE_EXITS_APERFMPERF ; +} +static inline bool kvm_notify_vmexit_enabled (struct kvm *kvm) {return kvm->arch.notify_vmexit_flags &KVM_X86_NOTIFY_VMEXIT_ENABLED ;
include/uapi/linux/kvm.h +1 -0
@@ -644,6 +644,7 @@ struct kvm_ioeventfd {#define KVM_X86_DISABLE_EXITS_HLT (1 << 1)#define KVM_X86_DISABLE_EXITS_PAUSE (1 << 2)#define KVM_X86_DISABLE_EXITS_CSTATE (1 << 3) +#define KVM_X86_DISABLE_EXITS_APERFMPERF (1 << 4)/* for KVM_ENABLE_CAP */ struct kvm_enable_cap {
tools/include/uapi/linux/kvm.h +1 -0
@@ -618,6 +618,7 @@ struct kvm_ioeventfd {#define KVM_X86_DISABLE_EXITS_HLT (1 << 1)#define KVM_X86_DISABLE_EXITS_PAUSE (1 << 2)#define KVM_X86_DISABLE_EXITS_CSTATE (1 << 3) +#define KVM_X86_DISABLE_EXITS_APERFMPERF (1 << 4)/* for KVM_ENABLE_CAP */ struct kvm_enable_cap {
--
Quoted 9 lines · Sean Christopherson
This needs to be rebased on top of the MSR interception rework, which I've now pushed to kvm-x86 next. Luckily, it's quite painless. Compile tested only at this point (about to throw it onto metal).
I'd be happy to post a v5 on your behalf (pending your thoughts on my feedback to patch 1), unless you want the honors. The fixup is a wee bit more than I'm comfortable doing on-the-fly.
Please do.
Sean Christopherson
maintainerQuoted 14 lines · Jim Mattson
(quoted diff) +10
-0@@ -7790,6 +7791,28 @@ all such vmexits. Do not enable KVM_FEATURE_PV_UNHALT if you disable HLT exits. +Virtualizing the ``IA32_APERF`` and ``IA32_MPERF`` MSRs requires more +than just disabling APERF/MPERF exits. While both Intel and AMD +document strict usage conditions for these MSRs--emphasizing that only +the ratio of their deltas over a time interval (T0 to T1) is +architecturally defined--simply passing through the MSRs can still +produce an incorrect ratio. + +This erroneous ratio can occur if, between T0 and T1: + +1. The vCPU thread migrates between logical processors.
Review all such vmexits. · line 7813
Quoted 10 lines
(quoted diff) +10
-0+2. Live migration or suspend/resume operations take place. +3. Another task shares the vCPU's logical processor. +4. C-states lower thean C0 are emulated (e.g., via HLT interception). +5. The guest TSC frequency doesn't match the host TSC frequency. + +Due to these complexities, KVM does not automatically associate this +passthrough capability with the guest CPUID bit, +``CPUID.6:ECX.APERFMPERF[bit 0]``. Userspace VMMs that deem this +mechanism adequate for virtualizing the ``IA32_APERF`` and +``IA32_MPERF`` MSRs must set the guest CPUID bit explicitly.
Question: what do we want to do about nested? Due to differences between SVM and VMX at the time you posted your patches, this series _as posted_ will do nested passthrough for SVM, but not VMX (before the MSR rework, SVM auto-merged bitmaps for all MSRs in svm_direct_access_msrs).
As I've got it locally applied, neither SVM nor VMX will do passthrough to L2. I'm leaning toward allowing full passthrough, because (a) it's easy, (b) I can't think of any reason not to, and (c) SVM's semi-auto-merging logic means we could *unintentinally* do full passthrough in the future, in the unlikely event that KVM added passthrough support for an MSR in the same chunk as APERF and MPERF.
This would be the extent of the changes (I think, haven't tested yet).
arch/x86/kvm/svm/nested.c +3 -1
@@ -194,7 +194,7 @@ void recalc_intercepts(struct vcpu_svm *svm)* Hardcode the capacity of the array based on the maximum number of _offsets_. * MSRs are batched together, so there are fewer offsets than MSRs. */ -static int nested_svm_msrpm_merge_offsets[6 ]__ro_after_init ; +static int nested_svm_msrpm_merge_offsets[7 ]__ro_after_init ;static int nested_svm_nr_msrpm_merge_offsets__ro_after_init ;typedef unsigned long nsvm_msrpm_merge_t ;
@@ -216,6 +216,8 @@ int __init nested_svm_init_msrpm_merge_offsets(void)MSR_IA32_SPEC_CTRL ,MSR_IA32_PRED_CMD ,MSR_IA32_FLUSH_CMD , +MSR_IA32_APERF , +MSR_IA32_MPERF ,MSR_IA32_LASTBRANCHFROMIP ,MSR_IA32_LASTBRANCHTOIP , MSR_IA32_LASTINTFROMIP,
arch/x86/kvm/vmx/nested.c +6 -0
@@ -715,6 +715,12 @@ static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu,nested_vmx_set_intercept_for_msr (vmx, msr_bitmap_l1, msr_bitmap_l0,MSR_IA32_FLUSH_CMD ,MSR_TYPE_W ); +nested_vmx_set_intercept_for_msr (vmx, msr_bitmap_l1, msr_bitmap_l0, +MSR_IA32_APERF ,MSR_TYPE_R ); + +nested_vmx_set_intercept_for_msr (vmx, msr_bitmap_l1, msr_bitmap_l0, +MSR_IA32_MPERF ,MSR_TYPE_R ); +kvm_vcpu_unmap (vcpu, &map); vmx->nested.force_msr_bitmap_recalc = false;
Quoted 11 lines · Sean Christopherson
Question: what do we want to do about nested? Due to differences between SVM and VMX at the time you posted your patches, this series
_asposted_will do nested passthrough for SVM, but not VMX (before the MSR rework, SVM auto-merged bitmaps for all MSRs insvm_direct_access_msrs).As I've got it locally applied, neither SVM nor VMX will do passthrough to L2. I'm leaning toward allowing full passthrough, because (a) it's easy, (b) I can't think of any reason not to, and (c) SVM's semi-auto-merging logic means we could *unintentinally* do full passthrough in the future, in the unlikely event that KVM added passthrough support for an MSR in the same chunk as APERF and MPERF.
I think full passthrough makes sense.
PATCH v4 3/3 KVM: selftests: Test behavior of KVM_X86_DISABLE_EXITS_APERFMPERF
In reply to Jim MattsonFor a VCPU thread pinned to a single LPU, verify that interleaved host and guest reads of IA32_[AM]PERF return strictly increasing values when APERFMPERF exiting is disabled.
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../testing/selftests/kvm/include/kvm_util.h | 2 + tools/testing/selftests/kvm/lib/kvm_util.c | 17 +++ .../selftests/kvm/x86/aperfmperf_test.c | 132 ++++++++++++++++++ 4 files changed, 152 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/aperfmperf_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -131,6 +131,7 @@ TEST_GEN_PROGS_x86 += x86/amx_test TEST_GEN_PROGS_x86 += x86/max_vcpuid_cap_test TEST_GEN_PROGS_x86 += x86/triple_fault_event_test TEST_GEN_PROGS_x86 += x86/recalc_apic_map_test +TEST_GEN_PROGS_x86 += x86/aperfmperf_test TEST_GEN_PROGS_x86 += access_tracking_perf_test TEST_GEN_PROGS_x86 += coalesced_io_test TEST_GEN_PROGS_x86 += dirty_log_perf_test
tools/testing/selftests/kvm/include/kvm_util.h +2 -0
@@ -1158,4 +1158,6 @@ bool vm_is_gpa_protected(struct kvm_vm *vm, vm_paddr_t paddr);uint32_t guest_get_vcpuid (void ); +int pin_task_to_one_cpu (void ); +#endif /* SELFTEST_KVM_UTIL_H */
tools/testing/selftests/kvm/lib/kvm_util.c +17 -0
@@ -10,6 +10,7 @@#include "ucall_common.h" #include <assert.h> +#include <pthread.h> #include <sched.h> #include <sys/mman.h> #include <sys/resource.h>
@@ -2321,3 +2322,19 @@ bool vm_is_gpa_protected(struct kvm_vm *vm, vm_paddr_t paddr) pg = paddr >> vm->page_shift;return sparsebit_is_set (region->protected_phy_pages, pg); } + +int pin_task_to_one_cpu (void ) +{ +int cpu =sched_getcpu (); +cpu_set_t cpuset; +int rc; + +CPU_ZERO (&cpuset); +CPU_SET (cpu, &cpuset); + + rc =pthread_setaffinity_np (pthread_self (),sizeof (cpuset), &cpuset); +TEST_ASSERT (rc ==0 ,"%s: Can't set thread affinity" , __func__); + +return cpu; +} +
tools/testing/selftests/kvm/x86/aperfmperf_test.c new file +132 -0
132 lines not on this page · raw message
Signature · 1 line
2.49.0.1204.g71687c7c1d-goog
Re: [PATCH v4 3/3] KVM: selftests: Test behavior of KVM_X86_DISABLE_EXITS_APERFMPERF
Quoted 183 lines · Jim Mattson
For a VCPU thread pinned to a single LPU, verify that interleaved host and guest reads of IA32_[AM]PERF return strictly increasing values when APERFMPERF exiting is disabled.
Signed-off-by Jim Mattson <[email protected]>--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../testing/selftests/kvm/include/kvm_util.h | 2 + tools/testing/selftests/kvm/lib/kvm_util.c | 17 +++ .../selftests/kvm/x86/aperfmperf_test.c | 132 ++++++++++++++++++ 4 files changed, 152 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/aperfmperf_test.ctools/testing/selftests/kvm/Makefile.kvm +1
-0@@ -131,6 +131,7 @@ TEST_GEN_PROGS_x86 += x86/amx_test TEST_GEN_PROGS_x86 += x86/max_vcpuid_cap_test TEST_GEN_PROGS_x86 += x86/triple_fault_event_test TEST_GEN_PROGS_x86 += x86/recalc_apic_map_test +TEST_GEN_PROGS_x86 += x86/aperfmperf_test TEST_GEN_PROGS_x86 += access_tracking_perf_test TEST_GEN_PROGS_x86 += coalesced_io_test TEST_GEN_PROGS_x86 += dirty_log_perf_testtools/testing/selftests/kvm/include/kvm_util.h +2
-0@@ -1158,4 +1158,6 @@ bool vm_is_gpa_protected(struct kvm_vm *vm, vm_paddr_t paddr);uint32_t guest_get_vcpuid (void ); +int pin_task_to_one_cpu (void ); +#endif /* SELFTEST_KVM_UTIL_H */ tools/testing/selftests/kvm/lib/kvm_util.c +17
-0@@ -10,6 +10,7 @@#include "ucall_common.h" #include <assert.h> +#include <pthread.h> #include <sched.h> #include <sys/mman.h> #include <sys/resource.h> @@ -2321,3 +2322,19 @@ bool vm_is_gpa_protected(struct kvm_vm *vm, vm_paddr_t paddr) pg = paddr >> vm->page_shift;return sparsebit_is_set (region->protected_phy_pages, pg); } + +int pin_task_to_one_cpu (void ) +{ +int cpu =sched_getcpu (); +cpu_set_t cpuset; +int rc; + +CPU_ZERO (&cpuset); +CPU_SET (cpu, &cpuset); + + rc =pthread_setaffinity_np (pthread_self (),sizeof (cpuset), &cpuset); +TEST_ASSERT (rc ==0 ,"%s: Can't set thread affinity" , __func__); + +return cpu; +} +tools/testing/selftests/kvm/x86/aperfmperf_test.c new file +109
-0@@ -0,0 +1,132 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Test for KVM_X86_DISABLE_EXITS_APERFMPERF +* +* Copyright (C) 2025, Google LLC. +* +* Test the ability to disable VM-exits for rdmsr of IA32_APERF and +* IA32_MPERF. When these VM-exits are disabled, reads of these MSRs +* return the host's values. +* +* Note: Requires read access to /dev/cpu/<lpu>/msr to read host MSRs. +*/ + +#include <fcntl.h> +#include <limits.h> +#include <stdbool.h> +#include <stdio.h> +#include <stdint.h> +#include <unistd.h> +#include <asm/msr-index.h> + +#include "kvm_util.h" +#include "processor.h" +#include "test_util.h" + +#define NUM_ITERATIONS 100 + +static int open_dev_msr (int cpu) +{ +char path[PATH_MAX ]; + +snprintf (path,sizeof (path),"/dev/cpu/%d/msr" , cpu); +return open_path_or_exit (path,O_RDONLY ); +} + +static uint64_t read_dev_msr (int msr_fd,uint32_t msr) +{ +uint64_t data; +ssize_t rc; + + rc =pread (msr_fd, &data,sizeof (data), msr); +TEST_ASSERT (rc ==sizeof (data),"Read of MSR 0x%x failed" , msr); + +return data; +} + +static void guest_code (void ) +{ +int i; + +for (i =0 ; i <NUM_ITERATIONS ; i++) +GUEST_SYNC2 (rdmsr (MSR_IA32_APERF ),rdmsr (MSR_IA32_MPERF )); + +GUEST_DONE (); +} + +int main (int argc,char *argv[]) +{ +uint64_t host_aperf_before, host_mperf_before; +struct kvm_vcpu *vcpu; +struct kvm_vm *vm; +int msr_fd; +int cpu; +int i; + + cpu =pin_task_to_one_cpu (); + + msr_fd =open_dev_msr (cpu); + +/* +* This test requires a non-standard VM initialization, because +* KVM_ENABLE_CAP cannot be used on a VM file descriptor after +* a VCPU has been created. +*/ + vm =vm_create (1 ); + +TEST_REQUIRE (vm_check_cap (vm,KVM_CAP_X86_DISABLE_EXITS ) & +KVM_X86_DISABLE_EXITS_APERFMPERF ); + +vm_enable_cap (vm,KVM_CAP_X86_DISABLE_EXITS , +KVM_X86_DISABLE_EXITS_APERFMPERF ); + + vcpu =vm_vcpu_add (vm,0 , guest_code); + + host_aperf_before =read_dev_msr (msr_fd,MSR_IA32_APERF ); + host_mperf_before =read_dev_msr (msr_fd,MSR_IA32_MPERF ); + +for (i =0 ; i <NUM_ITERATIONS ; i++) { +uint64_t host_aperf_after, host_mperf_after; +uint64_t guest_aperf, guest_mperf; +struct ucall uc; + +vcpu_run (vcpu); +TEST_ASSERT_KVM_EXIT_REASON (vcpu,KVM_EXIT_IO ); + +switch (get_ucall (vcpu, &uc)) { +case UCALL_DONE : +break ; +case UCALL_ABORT : +REPORT_GUEST_ASSERT (uc); +case UCALL_SYNC : + guest_aperf = uc.args[0 ]; + guest_mperf = uc.args[1 ]; + + host_aperf_after =read_dev_msr (msr_fd,MSR_IA32_APERF ); + host_mperf_after =read_dev_msr (msr_fd,MSR_IA32_MPERF ); + + TEST_ASSERT(host_aperf_before < guest_aperf, + "APERF: host_before (0x%" PRIx64 ") >= guest (0x%" PRIx64 ")",
tools/testing/selftests/kvm/x86/aperfmperf_test.c line 119
Quoted 10 lines
tools/testing/selftests/kvm/x86/aperfmperf_test.c new file +10
-0+ host_aperf_before, guest_aperf); +TEST_ASSERT (guest_aperf < host_aperf_after, +"APERF: guest (0x%" PRIx64") >= host_after (0x%" PRIx64")" , + guest_aperf, host_aperf_after); +TEST_ASSERT (host_mperf_before < guest_mperf, +"MPERF: host_before (0x%" PRIx64") >= guest (0x%" PRIx64")" , + host_mperf_before, guest_mperf); +TEST_ASSERT (guest_mperf < host_mperf_after, +"MPERF: guest (0x%" PRIx64") >= host_after (0x%" PRIx64")" , + guest_mperf, host_mperf_after);
Should we consider the possible overflow case of these 2 MSRs although it could be extremely rare? Thanks.
Quoted 13 lines
tools/testing/selftests/kvm/x86/aperfmperf_test.c new file +13
-0+ + host_aperf_before = host_aperf_after; + host_mperf_before = host_mperf_after; + +break ; + } + } + +kvm_vm_free (vm); +close (msr_fd); + +return 0 ; +}
Re: [PATCH v4 3/3] KVM: selftests: Test behavior of KVM_X86_DISABLE_EXITS_APERFMPERF
Quoted 9 lines · Dapeng
Quoted 3 lines · Jim Mattson
For a VCPU thread pinned to a single LPU, verify that interleaved host and guest reads of IA32_[AM]PERF return strictly increasing values when APERFMPERF exiting is disabled.
Should we consider the possible overflow case of these 2 MSRs although it could be extremely rare? Thanks.
Unless someone moves the MSRs forward, at current frequencies, the machine will have to be up for more than 100 years. I'll be long dead by then.
Note that frequency invariant scheduling doesn't accommodate overflow either. If the MSRs overflow, frequency invariant scheduling is disabled.
Re: [PATCH v4 3/3] KVM: selftests: Test behavior of KVM_X86_DISABLE_EXITS_APERFMPERF
Quoted 11 lines · Jim Mattson
Quoted 7 lines · Dapeng
Quoted 3 lines · Jim Mattson
For a VCPU thread pinned to a single LPU, verify that interleaved host and guest reads of IA32_[AM]PERF return strictly increasing values when APERFMPERF exiting is disabled.
Should we consider the possible overflow case of these 2 MSRs although it could be extremely rare? Thanks.
Unless someone moves the MSRs forward, at current frequencies, the machine will have to be up for more than 100 years. I'll be long dead by then.
😂
Quoted 4 lines
Note that frequency invariant scheduling doesn't accommodate overflow either. If the MSRs overflow, frequency invariant scheduling is disabled.
Agree.
Sean Christopherson
maintainerRe: [PATCH v4 3/3] KVM: selftests: Test behavior of KVM_X86_DISABLE_EXITS_APERFMPERF
In reply to Jim MattsonQuoted 21 lines · Jim Mattson
tools/testing/selftests/kvm/lib/kvm_util.c +6
-0@@ -10,6 +10,7 @@#include "ucall_common.h" #include <assert.h> +#include <pthread.h> #include <sched.h> #include <sys/mman.h> #include <sys/resource.h> @@ -2321,3 +2322,19 @@ bool vm_is_gpa_protected(struct kvm_vm *vm, vm_paddr_t paddr) pg = paddr >> vm->page_shift;return sparsebit_is_set (region->protected_phy_pages, pg); } + +int pin_task_to_one_cpu (void ) +{ +int cpu =sched_getcpu (); +cpu_set_t cpuset;
tools/testing/selftests/kvm/lib/kvm_util.c bool vm_is_gpa_protected(struct kvm_vm *vm, vm_paddr_t paddr) · line 2339
Quoted 10 lines
tools/testing/selftests/kvm/lib/kvm_util.c +10
-0+int rc; + +CPU_ZERO (&cpuset); +CPU_SET (cpu, &cpuset); + + rc =pthread_setaffinity_np (pthread_self (),sizeof (cpuset), &cpuset); +TEST_ASSERT (rc ==0 ,"%s: Can't set thread affinity" , __func__); + +return cpu; +}
There's already kvm_pin_this_task_to_pcpu(), which is *almost* what is needed here. I'll slot in a patch in v5 to expand that into a set of APIs, along with a patch to convert the low hanging fruit (arch_timer tests).