KVM: X86: Emulate APERF/MPERF to report actual VCPU frequency
PATCH 1/1 KVM: X86: Emulate APERF/MPERF to report actual VCPU frequency
The aperf/mperf are used to report current CPU frequency after 7d5905dc14a "x86 / CPU: Always show current CPU frequency in /proc/cpuinfo". But guest kernel always reports a fixed VCPU frequency in the /proc/cpuinfo, which may confuse users especially when turbo is enabled on the host.
Emulate guest APERF/MPERF capability based their values on the host.
--- arch/x86/include/asm/kvm_host.h | 12 ++++++ arch/x86/kvm/cpuid.c | 8 +++- arch/x86/kvm/x86.c | 76 ++++++++++++++++++++++++++++++++- 3 files changed, 94 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +12 -0
@@ -539,6 +539,16 @@ struct kvm_vcpu_hv_stimer {bool msg_pending; }; +/* vCPU thermal and power context */ +struct kvm_vcpu_hwp { +/* Hardware Coordination Feedback Capability (Presence of APERF/MPERF) */ +bool hw_coord_fb_cap; +/* MPERF increases with a fixed frequency */ +u64 mperf; +/* APERF increases with the current/actual frequency */ +u64 aperf; +}; +/* Hyper-V synthetic interrupt controller (SynIC)*/ struct kvm_vcpu_hv_synic {u64 version;
@@ -829,6 +839,8 @@ struct kvm_vcpu_arch {/* AMD MSRC001_0015 Hardware Configuration */ u64 msr_hwcr; + +struct kvm_vcpu_hwp hwp; };struct kvm_lpage_info {
arch/x86/kvm/cpuid.c +7 -1
@@ -78,6 +78,11 @@ int kvm_update_cpuid(struct kvm_vcpu *vcpu) apic->lapic_timer.timer_mode_mask =1 <<17 ; } + best =kvm_find_cpuid_entry (vcpu,0x6 ,0 ); +if (best && best->function ==0x6 && +boot_cpu_has (X86_FEATURE_APERFMPERF ) && (best->ecx &0x1 )) + vcpu->arch.hwp.hw_coord_fb_cap = true; + best =kvm_find_cpuid_entry (vcpu,7 ,0 );if (best &&boot_cpu_has (X86_FEATURE_PKU ) && best->function ==0x7 ) cpuid_entry_change(best, X86_FEATURE_OSPKE,
@@ -561,7 +566,8 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function)case 6 :/* Thermal management */ entry->eax =0x4 ;/* allow ARAT */ entry->ebx =0 ; - entry->ecx =0 ; +/* allow aperf/mperf to report the true VCPU frequency. */ + entry->ecx =boot_cpu_has (X86_FEATURE_APERFMPERF ) ?0x1 :0 ; entry->edx =0 ;break ;/* function 7 has additional index. */
arch/x86/kvm/x86.c +75 -1
114 lines not on this page · raw message
Signature · 1 line
2.21.3
Quoted 13 lines · Like Xu
The aperf/mperf are used to report current CPU frequency after
7d5905dc14a"x86 / CPU: Always show current CPU frequency in/proc/cpuinfo". But guest kernel always reports a fixed VCPU frequency in the/proc/cpuinfo, which may confuse users especially when turbo is enabled on the host.Emulate guest APERF/MPERF capability based their values on the host.
Co-developed-by Li RongQing <[email protected]>Signed-off-by Li RongQing <[email protected]>Reviewed-by Chai Wen <[email protected]>Reviewed-by Jia Lina <[email protected]>Signed-off-by Like Xu <[email protected]>---
...
Quoted 14 lines
arch/x86/kvm/x86.c +2
-1@@ -8312,7 +8376,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)dm_request_for_irq_injection (vcpu) &&kvm_cpu_accept_dm_intr (vcpu);fastpath_t exit_fastpath; - +u64 enter_mperf =0 , enter_aperf =0 , exit_mperf =0 , exit_aperf =0 ;bool req_immediate_exit = false;if (kvm_request_pending (vcpu)) {@@ -8516,8 +8580,17 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) vcpu->arch.switch_db_regs &= ~KVM_DEBUGREG_RELOAD ; } +if (unlikely (vcpu->arch.hwp.hw_coord_fb_cap))
arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · line 8593
Quoted 10 lines
arch/x86/kvm/x86.c +8
-0+get_host_amperf (&enter_mperf, &enter_aperf); + exit_fastpath = kvm_x86_ops.run (vcpu); +if (unlikely (vcpu->arch.hwp.hw_coord_fb_cap)) { +get_host_amperf (&exit_mperf, &exit_aperf); +vcpu_update_amperf (vcpu,get_amperf_delta (enter_aperf, exit_aperf), +get_amperf_delta (enter_mperf, exit_mperf)); + } +
Is there an alternative approach that doesn't require 4 RDMSRs on every VMX round trip? That's literally more expensive than VM-Enter + VM-Exit combined.
E.g. what about adding KVM_X86_DISABLE_EXITS_APERF_MPERF and exposing the MSRs for read when that capability is enabled?
Quoted 47 lines · Sean Christopherson
Quoted 13 lines · Like Xu
The aperf/mperf are used to report current CPU frequency after
7d5905dc14a"x86 / CPU: Always show current CPU frequency in/proc/cpuinfo". But guest kernel always reports a fixed VCPU frequency in the/proc/cpuinfo, which may confuse users especially when turbo is enabled on the host.Emulate guest APERF/MPERF capability based their values on the host.
Co-developed-by Li RongQing <[email protected]>Signed-off-by Li RongQing <[email protected]>Reviewed-by Chai Wen <[email protected]>Reviewed-by Jia Lina <[email protected]>Signed-off-by Like Xu <[email protected]>---
...
Quoted 14 lines
arch/x86/kvm/x86.c +2
-1@@ -8312,7 +8376,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)dm_request_for_irq_injection (vcpu) &&kvm_cpu_accept_dm_intr (vcpu);fastpath_t exit_fastpath; - +u64 enter_mperf =0 , enter_aperf =0 , exit_mperf =0 , exit_aperf =0 ;bool req_immediate_exit = false;if (kvm_request_pending (vcpu)) {@@ -8516,8 +8580,17 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) vcpu->arch.switch_db_regs &= ~KVM_DEBUGREG_RELOAD ; } +if (unlikely (vcpu->arch.hwp.hw_coord_fb_cap))arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · line 8593
Quoted 10 lines
arch/x86/kvm/x86.c +8
-0+get_host_amperf (&enter_mperf, &enter_aperf); + exit_fastpath = kvm_x86_ops.run (vcpu); +if (unlikely (vcpu->arch.hwp.hw_coord_fb_cap)) { +get_host_amperf (&exit_mperf, &exit_aperf); +vcpu_update_amperf (vcpu,get_amperf_delta (enter_aperf, exit_aperf), +get_amperf_delta (enter_mperf, exit_mperf)); + } +Is there an alternative approach that doesn't require 4 RDMSRs on every VMX round trip? That's literally more expensive than VM-Enter + VM-Exit combined.
Quoted 2 lines
E.g. what about adding
KVM_X86_DISABLE_EXITS_APERF_MPERFand exposing the MSRs for read when that capability is enabled?
When would you load the hardware MSRs with the guest/host values?
Quoted 49 lines · Jim Mattson
Quoted 47 lines · Sean Christopherson
Quoted 13 lines · Like Xu
The aperf/mperf are used to report current CPU frequency after
7d5905dc14a"x86 / CPU: Always show current CPU frequency in/proc/cpuinfo". But guest kernel always reports a fixed VCPU frequency in the/proc/cpuinfo, which may confuse users especially when turbo is enabled on the host.Emulate guest APERF/MPERF capability based their values on the host.
Co-developed-by Li RongQing <[email protected]>Signed-off-by Li RongQing <[email protected]>Reviewed-by Chai Wen <[email protected]>Reviewed-by Jia Lina <[email protected]>Signed-off-by Like Xu <[email protected]>---
...
Quoted 14 lines
arch/x86/kvm/x86.c +2
-1@@ -8312,7 +8376,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)dm_request_for_irq_injection (vcpu) &&kvm_cpu_accept_dm_intr (vcpu);fastpath_t exit_fastpath; - +u64 enter_mperf =0 , enter_aperf =0 , exit_mperf =0 , exit_aperf =0 ;bool req_immediate_exit = false;if (kvm_request_pending (vcpu)) {@@ -8516,8 +8580,17 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) vcpu->arch.switch_db_regs &= ~KVM_DEBUGREG_RELOAD ; } +if (unlikely (vcpu->arch.hwp.hw_coord_fb_cap))arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · line 8593
Quoted 10 lines
arch/x86/kvm/x86.c +8
-0+get_host_amperf (&enter_mperf, &enter_aperf); + exit_fastpath = kvm_x86_ops.run (vcpu); +if (unlikely (vcpu->arch.hwp.hw_coord_fb_cap)) { +get_host_amperf (&exit_mperf, &exit_aperf); +vcpu_update_amperf (vcpu,get_amperf_delta (enter_aperf, exit_aperf), +get_amperf_delta (enter_mperf, exit_mperf)); + } +Is there an alternative approach that doesn't require 4 RDMSRs on every VMX round trip? That's literally more expensive than VM-Enter + VM-Exit combined.
Quoted 4 lines
Quoted 2 lines
E.g. what about adding
KVM_X86_DISABLE_EXITS_APERF_MPERFand exposing the MSRs for read when that capability is enabled?When would you load the hardware MSRs with the guest/host values?
Ugh, I was thinking the MSRs were read-only.
Doesn't this also interact with TSC scaling?
Quoted 51 lines · Sean Christopherson
Quoted 49 lines · Jim Mattson
Quoted 47 lines · Sean Christopherson
Quoted 13 lines · Like Xu
The aperf/mperf are used to report current CPU frequency after
7d5905dc14a"x86 / CPU: Always show current CPU frequency in/proc/cpuinfo". But guest kernel always reports a fixed VCPU frequency in the/proc/cpuinfo, which may confuse users especially when turbo is enabled on the host.Emulate guest APERF/MPERF capability based their values on the host.
Co-developed-by Li RongQing <[email protected]>Signed-off-by Li RongQing <[email protected]>Reviewed-by Chai Wen <[email protected]>Reviewed-by Jia Lina <[email protected]>Signed-off-by Like Xu <[email protected]>---
...
Quoted 14 lines
arch/x86/kvm/x86.c +2
-1@@ -8312,7 +8376,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)dm_request_for_irq_injection (vcpu) &&kvm_cpu_accept_dm_intr (vcpu);fastpath_t exit_fastpath; - +u64 enter_mperf =0 , enter_aperf =0 , exit_mperf =0 , exit_aperf =0 ;bool req_immediate_exit = false;if (kvm_request_pending (vcpu)) {@@ -8516,8 +8580,17 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) vcpu->arch.switch_db_regs &= ~KVM_DEBUGREG_RELOAD ; } +if (unlikely (vcpu->arch.hwp.hw_coord_fb_cap))arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · line 8593
Quoted 10 lines
arch/x86/kvm/x86.c +8
-0+get_host_amperf (&enter_mperf, &enter_aperf); + exit_fastpath = kvm_x86_ops.run (vcpu); +if (unlikely (vcpu->arch.hwp.hw_coord_fb_cap)) { +get_host_amperf (&exit_mperf, &exit_aperf); +vcpu_update_amperf (vcpu,get_amperf_delta (enter_aperf, exit_aperf), +get_amperf_delta (enter_mperf, exit_mperf)); + } +Is there an alternative approach that doesn't require 4 RDMSRs on every VMX round trip? That's literally more expensive than VM-Enter + VM-Exit combined.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
E.g. what about adding
KVM_X86_DISABLE_EXITS_APERF_MPERFand exposing the MSRs for read when that capability is enabled?When would you load the hardware MSRs with the guest/host values?
Ugh, I was thinking the MSRs were read-only.
EVen if they were read-only, they should power on to zero, and they will most likely not be zero when a guest powers on.
Quoted 1 line
Doesn't this also interact with TSC scaling?
Yes, it should!
Quoted 42 lines · Jim Mattson
Quoted 40 lines · Sean Christopherson
Quoted 38 lines · Jim Mattson
Quoted 36 lines · Sean Christopherson
Quoted 13 lines · Like Xu
The aperf/mperf are used to report current CPU frequency after
7d5905dc14a"x86 / CPU: Always show current CPU frequency in/proc/cpuinfo". But guest kernel always reports a fixed VCPU frequency in the/proc/cpuinfo, which may confuse users especially when turbo is enabled on the host.Emulate guest APERF/MPERF capability based their values on the host.
Co-developed-by Li RongQing <[email protected]>Signed-off-by Li RongQing <[email protected]>Reviewed-by Chai Wen <[email protected]>Reviewed-by Jia Lina <[email protected]>Signed-off-by Like Xu <[email protected]>---
...
Quoted 18 lines
arch/x86/kvm/x86.c +4
-1@@ -8312,7 +8376,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)dm_request_for_irq_injection (vcpu) &&kvm_cpu_accept_dm_intr (vcpu);fastpath_t exit_fastpath; - +u64 enter_mperf =0 , enter_aperf =0 , exit_mperf =0 , exit_aperf =0 ;bool req_immediate_exit = false;if (kvm_request_pending (vcpu)) {@@ -8516,8 +8580,17 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) vcpu->arch.switch_db_regs &= ~KVM_DEBUGREG_RELOAD ; } +if (unlikely (vcpu->arch.hwp.hw_coord_fb_cap)) +get_host_amperf (&enter_mperf, &enter_aperf); + exit_fastpath = kvm_x86_ops.run (vcpu);
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
arch/x86/kvm/x86.c static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · line 8593
Quoted 6 lines
arch/x86/kvm/x86.c +6
-0+if (unlikely (vcpu->arch.hwp.hw_coord_fb_cap)) { +get_host_amperf (&exit_mperf, &exit_aperf); +vcpu_update_amperf (vcpu,get_amperf_delta (enter_aperf, exit_aperf), +get_amperf_delta (enter_mperf, exit_mperf)); + } +Is there an alternative approach that doesn't require 4 RDMSRs on every VMX round trip? That's literally more expensive than VM-Enter + VM-Exit combined.
It looks like we have quite a few users who are expecting this feature in different scenarios.
I will add a fast path for RO usage and a slow path if the guest tries to change the AMPERF values.
Quoted 10 lines
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
E.g. what about adding
KVM_X86_DISABLE_EXITS_APERF_MPERFand exposing the MSRs for read when that capability is enabled?When would you load the hardware MSRs with the guest/host values?
Ugh, I was thinking the MSRs were read-only.
EVen if they were read-only, they should power on to zero, and they will most likely not be zero when a guest powers on.
Can we assume that "not zero when the guest is on" will not harm any guests ?
Quoted 4 lines
Quoted 2 lines
Doesn't this also interact with TSC scaling?
Yes, it should!
We have too much of a historical burden on TSC emulations.
For practical reasons, what if we only expose the AMPERF cap if the host/guest has both CONSTANT_TSC and NONSTOP_TSC ?
One more design concern, I wonder if it is *safe* for the guest to read amperf on pCPU[x] the first time and on pCPU[y] the next time.
Any input ?
Thanks,
Like Xu