KVM: LAPIC: Per vCPU control over kvm_can_post_timer_interrupt
PATCH v2 1/1 KVM: LAPIC: Per vCPU control over kvm_can_post_timer_interrupt
From: Aili Yao <[email protected]>
When we isolate some pyhiscal cores, We may not use them for kvm guests, We may use them for other purposes like DPDK, or we can make some kvm guests isolated and some not, the global judgement pi_inject_timer is not enough; We may make wrong decisions:
In such a scenario, the guests without isolated cores will not be permitted to use vmx preemption timer, and tscdeadline fastpath also be disabled, both will lead to performance penalty.
So check whether the vcpu->cpu is isolated, if not, don't post timer interrupt.
And when qemu enable -cpu-pm feature for guests, all the available disable_exit will be set, including mwait,halt,pause,cstate, when this operation succeed, hlt_in_guest,pause_in_guest,cstate_in_guest will all be definitly set true with one special case, mwait_in_guest, this feature's enablement is depended on the HOST cpu feature support;
When cpu-pm is successfully enabled, and hlt_in_guest is true and mwait_in_guest is false, the guest cant't use Monitor/Mwait instruction for idle operation, instead, the guest may use halt for that purpose, as we have enable the cpu-pm feature and hlt_in_guest is true, we will also minimize the guest exit; For such a scenario, Monitor/Mwait instruction support is totally disabled, the guest has no way to use Mwait to exit from non-root mode;
For cpu-pm feature, hlt_in_guest and others except mwait_in_guest will be a good hint for it. So replace it with hlt_in_guest.
--- arch/x86/kvm/lapic.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
arch/x86/kvm/lapic.c +4 -2
@@ -34,6 +34,7 @@#include <asm/delay.h> #include <linux/atomic.h> #include <linux/jump_label.h> +#include <linux/sched/isolation.h> #include "kvm_cache_regs.h" #include "irq.h" #include "ioapic.h"
@@ -113,13 +114,14 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu) && + !housekeeping_cpu (vcpu->cpu,HK_FLAG_TIMER ); }bool kvm_can_use_hv_timer (struct kvm_vcpu *vcpu) {return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest (vcpu->kvm) || + && !(kvm_hlt_in_guest (vcpu->kvm) ||kvm_can_post_timer_interrupt (vcpu)); }EXPORT_SYMBOL_GPL (kvm_can_use_hv_timer);
Signature · 1 line
2.25.1
Sean Christopherson
maintainerQuoted 10 lines · Aili Yao
When cpu-pm is successfully enabled, and
hlt_in_guestis true andmwait_in_guestis false, the guest cant't use Monitor/Mwait instruction for idle operation, instead, the guest may use halt for that purpose, as we have enable the cpu-pm feature andhlt_in_guestis true, we will also minimize the guest exit; For such a scenario, Monitor/Mwait instruction support is totally disabled, the guest has no way to use Mwait to exit from non-root mode;For cpu-pm feature,
hlt_in_guestand others exceptmwait_in_guestwill be a good hint for it. So replace it withhlt_in_guest.
This should be a separate patch from the housekeeping_cpu() check, if we add the housekeeping check.
Quoted 18 lines
Signed-off-by Aili Yao <[email protected]>--- arch/x86/kvm/lapic.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)arch/x86/kvm/lapic.c +1
-0@@ -34,6 +34,7 @@#include <asm/delay.h> #include <linux/atomic.h> #include <linux/jump_label.h> +#include <linux/sched/isolation.h> #include "kvm_cache_regs.h" #include "irq.h" #include "ioapic.h"
arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 118
Quoted 6 lines
arch/x86/kvm/lapic.c +2
-1@@ -113,13 +114,14 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu) && + !housekeeping_cpu (vcpu->cpu,HK_FLAG_TIMER );
Why not check kvm_{hlt,mwait}_in_guest()? IIUC, non-housekeeping CPUs don't _have_ to be associated 1:1 with a vCPU, in which case posting the timer is unlikely to be a performance win even though the target isn't a housekeeping CPU.
And wouldn't exposing HLT/MWAIT to a vCPU that's on a housekeeping CPU be a bogus configuration?
arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 124
Quoted 7 lines
arch/x86/kvm/lapic.c +1
-1}bool kvm_can_use_hv_timer (struct kvm_vcpu *vcpu) {return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest(vcpu->kvm) || + && !(kvm_hlt_in_guest(vcpu->kvm) ||
This is incorrect, the HLT vs. MWAIT isn't purely a posting interrupts thing. The VMX preemption timer counts down in C0, C1, and C2, but not deeper sleep states. HLT is always C1, thus it's safe to use the VMX preemption timer even if the guest can execute HLT without exiting.
The timer isn't compatible with MWAIT because it stops counting in C3 (or lower), i.e. the guest can cause the timer to stop counting.
arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 127
Quoted 4 lines
arch/x86/kvm/lapic.c +0
-0kvm_can_post_timer_interrupt(vcpu)); }EXPORT_SYMBOL_GPL (kvm_can_use_hv_timer);
Splicing in Wanpeng's version to try and merge the two threads:
Quoted 9 lines · Wanpeng Li
--- arch/x86/kvm/lapic.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)arch/x86/kvm/lapic.c +0
-0
arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 116
Quoted 5 lines
arch/x86/kvm/lapic.c +1
-1@@ -113,14 +113,13 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_mwait_in_guest (vcpu->kvm) &&kvm_vcpu_apicv_active (vcpu);
As Aili's changelog pointed out, MWAIT may not be advertised to the guest.
So I think we want this? With a non-functional, opinionated refactoring of kvm_can_use_hv_timer() because I'm terrible at reading !(a || b).
arch/x86/kvm/lapic.c +15 -4
@@ -113,14 +113,25 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu) && + (kvm_mwait_in_guest (vcpu) ||kvm_hlt_in_guest (vcpu)); }bool kvm_can_use_hv_timer (struct kvm_vcpu *vcpu) { -return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest (vcpu->kvm) || -kvm_can_post_timer_interrupt (vcpu)); +/* +* Don't use the hypervisor timer, a.k.a. VMX Preemption Timer, if the +* guest can execute MWAIT without exiting as the timer will stop +* counting if the core enters C3 or lower. HLT in the guest is ok as +* HLT is effectively C1 and the timer counts in C0, C1, and C2. +* +* Don't use the hypervisor timer if KVM can post a timer interrupt to +* the guest since posted the timer avoids taking an extra a VM-Exit +* when the timer expires. +*/ +return kvm_x86_ops.set_hv_timer && + !kvm_mwait_in_guest (vcpu->kvm) && + !kvm_can_post_timer_interrupt(vcpu)); }EXPORT_SYMBOL_GPL (kvm_can_use_hv_timer);
Quoted 34 lines · On Tue, 7 Dec 2021 23:23:03 +0000 Sean Christopherson <[email protected]> wrote:
Quoted 10 lines · Aili Yao
When cpu-pm is successfully enabled, and
hlt_in_guestis true andmwait_in_guestis false, the guest cant't use Monitor/Mwait instruction for idle operation, instead, the guest may use halt for that purpose, as we have enable the cpu-pm feature andhlt_in_guestis true, we will also minimize the guest exit; For such a scenario, Monitor/Mwait instruction support is totally disabled, the guest has no way to use Mwait to exit from non-root mode;For cpu-pm feature,
hlt_in_guestand others exceptmwait_in_guestwill be a good hint for it. So replace it withhlt_in_guest.This should be a separate patch from the
housekeeping_cpu()check, if we add the housekeeping check.Quoted 19 lines
Signed-off-by Aili Yao <[email protected]>--- arch/x86/kvm/lapic.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)arch/x86/kvm/lapic.c +1
-0@@ -34,6 +34,7 @@#include <asm/delay.h> #include <linux/atomic.h> #include <linux/jump_label.h> +#include <linux/sched/isolation.h> #include "kvm_cache_regs.h" #include "irq.h" #include "ioapic.h" @@ -113,13 +114,14 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)
Quoted 9 lines
arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)
Quoted 5 lines
arch/x86/kvm/lapic.c +2
-1static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu) && + !housekeeping_cpu (vcpu->cpu,HK_FLAG_TIMER );Why not check kvm_{hlt,mwait}_in_guest()? IIUC, non-housekeeping CPUs don't _have_ to be associated 1:1 with a vCPU, in which case posting the timer is unlikely to be a performance win even though the target isn't a housekeeping CPU.
Yes, non-housekeeping CPUs can be assigned to multi vCPUs, I don't think it's a common configuration; But this can happen.
Quoted 2 lines
And wouldn't exposing HLT/MWAIT to a vCPU that's on a housekeeping CPU be a bogus configuration?
Agree, it's a bogus configuration and not suppose to like this, but this can happen;
It seems we can't cover all the abnormal cases in a single line. So I think just checking for most right configurations is needed.
Quoted 14 lines
arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)
Quoted 7 lines
arch/x86/kvm/lapic.c +1
-1}bool kvm_can_use_hv_timer (struct kvm_vcpu *vcpu) {return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest(vcpu->kvm) || + && !(kvm_hlt_in_guest(vcpu->kvm) ||This is incorrect, the HLT vs. MWAIT isn't purely a posting interrupts thing. The VMX preemption timer counts down in C0, C1, and C2, but not deeper sleep states. HLT is always C1, thus it's safe to use the VMX preemption timer even if the guest can execute HLT without exiting.
The timer isn't compatible with MWAIT because it stops counting in C3 (or lower), i.e. the guest can cause the timer to stop counting.
Thanks for your pointer, now i know this.
Quoted 34 lines
arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 136
Quoted 5 lines
arch/x86/kvm/lapic.c +0
-0kvm_can_post_timer_interrupt(vcpu)); }EXPORT_SYMBOL_GPL (kvm_can_use_hv_timer);Splicing in Wanpeng's version to try and merge the two threads:
Quoted 9 lines · Wanpeng Li
--- arch/x86/kvm/lapic.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)arch/x86/kvm/lapic.c +0
-0arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 116
Quoted 5 lines
arch/x86/kvm/lapic.c +1
-1@@ -113,14 +113,13 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_mwait_in_guest (vcpu->kvm) &&kvm_vcpu_apicv_active (vcpu);As Aili's changelog pointed out, MWAIT may not be advertised to the guest.
So I think we want this? With a non-functional, opinionated refactoring of
kvm_can_use_hv_timer()because I'm terrible at reading !(a || b).arch/x86/kvm/lapic.c +0
-0
arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 118
Quoted 7 lines
arch/x86/kvm/lapic.c +2
-1@@ -113,14 +113,25 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu) && + (kvm_mwait_in_guest (vcpu) ||kvm_hlt_in_guest (vcpu)); }
I think only kvm_hlt_in_guest() check is enough here, as for current code, if kvm_mwait_in_guest() is true, kvm_hlt_in_guest must be ture, if kvm_mwait_in_guest() is false, kvm_hlt_in_guest() could also be true.
Quoted 11 lines
arch/x86/kvm/lapic.c +6
-3bool kvm_can_use_hv_timer (struct kvm_vcpu *vcpu) { -return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest (vcpu->kvm) || -kvm_can_post_timer_interrupt (vcpu)); +/* +* Don't use the hypervisor timer, a.k.a. VMX Preemption Timer, if the +* guest can execute MWAIT without exiting as the timer will stop +* counting if the core enters C3 or lower. HLT in the guest is ok as +* HLT is effectively C1 and the timer counts in C0, C1, and C2. +*
arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 137
Quoted 10 lines
arch/x86/kvm/lapic.c +7
-0+* Don't use the hypervisor timer if KVM can post a timer interrupt to +* the guest since posted the timer avoids taking an extra a VM-Exit +* when the timer expires. +*/ +return kvm_x86_ops.set_hv_timer && + !kvm_mwait_in_guest (vcpu->kvm) && + !kvm_can_post_timer_interrupt(vcpu)); }EXPORT_SYMBOL_GPL (kvm_can_use_hv_timer);
I think this modification covers most used configurations and it's right. Thanks!
--Aili Yao
Quoted 19 lines · On Tue, 7 Dec 2021 23:23:03 +0000 Sean Christopherson <[email protected]> wrote:
static bool
kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu){arch/x86/kvm/lapic.c +7
-4-return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu) && + (kvm_mwait_in_guest (vcpu) ||kvm_hlt_in_guest (vcpu)); }bool kvm_can_use_hv_timer (struct kvm_vcpu *vcpu) { -return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest (vcpu->kvm) || -kvm_can_post_timer_interrupt (vcpu)); +/* +* Don't use the hypervisor timer, a.k.a. VMX Preemption Timer, if the +* guest can execute MWAIT without exiting as the timer will stop +* counting if the core enters C3 or lower. HLT in the guest is ok as +* HLT is effectively C1 and the timer counts in C0, C1, and C2.
arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 136
Quoted 10 lines
arch/x86/kvm/lapic.c +8
-0+* +* Don't use the hypervisor timer if KVM can post a timer interrupt to +* the guest since posted the timer avoids taking an extra a VM-Exit +* when the timer expires. +*/ +return kvm_x86_ops.set_hv_timer && + !kvm_mwait_in_guest (vcpu->kvm) && + !kvm_can_post_timer_interrupt(vcpu)); }EXPORT_SYMBOL_GPL (kvm_can_use_hv_timer);
Sorry, I am little confused here now:
if kvm_can_post_timer_interrupt(vcpu) return true(cpu-pm enabled), then the kvm_can_use_hv_timer will always be false; if kvm_can_post_timer_interrupt(vcpu) return false(cpu-pm disable),then kvm_mwait_in_guest(vcpu->kvm) can't be true ether; It seems we don't need kvm_mwait_in_guest(vcpu->kvm) here?
Sorry, I am just a little confused and not too sure about this, if anything wrong, just ignore it.
Thanks!
Sean Christopherson
maintainerQuoted 33 lines · Aili Yao
Quoted 19 lines · On Tue, 7 Dec 2021 23:23:03 +0000 Sean Christopherson <[email protected]> wrote:
static bool
kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu){arch/x86/kvm/lapic.c +7
-4-return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu) && + (kvm_mwait_in_guest (vcpu) ||kvm_hlt_in_guest (vcpu)); }bool kvm_can_use_hv_timer (struct kvm_vcpu *vcpu) { -return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest (vcpu->kvm) || -kvm_can_post_timer_interrupt (vcpu)); +/* +* Don't use the hypervisor timer, a.k.a. VMX Preemption Timer, if the +* guest can execute MWAIT without exiting as the timer will stop +* counting if the core enters C3 or lower. HLT in the guest is ok as +* HLT is effectively C1 and the timer counts in C0, C1, and C2. arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 136
Quoted 10 lines
arch/x86/kvm/lapic.c +8
-0+* +* Don't use the hypervisor timer if KVM can post a timer interrupt to +* the guest since posted the timer avoids taking an extra a VM-Exit +* when the timer expires. +*/ +return kvm_x86_ops.set_hv_timer && + !kvm_mwait_in_guest (vcpu->kvm) && + !kvm_can_post_timer_interrupt(vcpu)); }EXPORT_SYMBOL_GPL (kvm_can_use_hv_timer);
Quoted 4 lines
Sorry, I am little confused here now:
ifkvm_can_post_timer_interrupt(vcpu)return true(cpu-pm enabled), then thekvm_can_use_hv_timerwill always be false; ifkvm_can_post_timer_interrupt(vcpu)return false(cpu-pm disable),thenkvm_mwait_in_guest(vcpu->kvm)can't be true ether; It seems we don't needkvm_mwait_in_guest(vcpu->kvm)here?
We do, it's to prevent the guest from enter C3+ and stopping the VMX preemption timer, e.g. if either kvm_vcpu_apicv_active() or pi_inject_timer evaluates false.
Quoted 34 lines · On Thu, 9 Dec 2021 18:20:57 +0000 Sean Christopherson <[email protected]> wrote:
Quoted 33 lines · Aili Yao
Quoted 29 lines · On Tue, 7 Dec 2021 23:23:03 +0000 Sean Christopherson <[email protected]> wrote:
static bool
kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu){arch/x86/kvm/lapic.c +15
-4-return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu) && + (kvm_mwait_in_guest (vcpu) ||kvm_hlt_in_guest (vcpu)); }bool kvm_can_use_hv_timer (struct kvm_vcpu *vcpu) { -return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest (vcpu->kvm) || -kvm_can_post_timer_interrupt (vcpu)); +/* +* Don't use the hypervisor timer, a.k.a. VMX Preemption Timer, if the +* guest can execute MWAIT without exiting as the timer will stop +* counting if the core enters C3 or lower. HLT in the guest is ok as +* HLT is effectively C1 and the timer counts in C0, C1, and C2. +* +* Don't use the hypervisor timer if KVM can post a timer interrupt to +* the guest since posted the timer avoids taking an extra a VM-Exit +* when the timer expires. +*/ +return kvm_x86_ops.set_hv_timer && + !kvm_mwait_in_guest (vcpu->kvm) && + !kvm_can_post_timer_interrupt(vcpu)); }EXPORT_SYMBOL_GPL (kvm_can_use_hv_timer);
Quoted 7 lines
Quoted 4 lines
Sorry, I am little confused here now:
ifkvm_can_post_timer_interrupt(vcpu)return true(cpu-pm enabled), then thekvm_can_use_hv_timerwill always be false; ifkvm_can_post_timer_interrupt(vcpu)return false(cpu-pm disable),thenkvm_mwait_in_guest(vcpu->kvm)can't be true ether; It seems we don't needkvm_mwait_in_guest(vcpu->kvm)here?We do, it's to prevent the guest from enter C3+ and stopping the VMX preemption timer, e.g. if either
kvm_vcpu_apicv_active()orpi_inject_timerevaluates false.
Great thanks for your explanation!
Now i am clear!
--Aili Yao
Quoted 45 lines · On Tue, 7 Dec 2021 23:23:03 +0000 Sean Christopherson <[email protected]> wrote:
Quoted 9 lines · Wanpeng Li
--- arch/x86/kvm/lapic.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)arch/x86/kvm/lapic.c +0
-0arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 116
Quoted 5 lines
arch/x86/kvm/lapic.c +1
-1@@ -113,14 +113,13 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_mwait_in_guest (vcpu->kvm) &&kvm_vcpu_apicv_active (vcpu);As Aili's changelog pointed out, MWAIT may not be advertised to the guest.
So I think we want this? With a non-functional, opinionated refactoring of
kvm_can_use_hv_timer()because I'm terrible at reading !(a || b).arch/x86/kvm/lapic.c +8
-4@@ -113,14 +113,25 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu) && + (kvm_mwait_in_guest (vcpu) ||kvm_hlt_in_guest (vcpu)); }bool kvm_can_use_hv_timer (struct kvm_vcpu *vcpu) { -return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest (vcpu->kvm) || -kvm_can_post_timer_interrupt (vcpu)); +/* +* Don't use the hypervisor timer, a.k.a. VMX Preemption Timer, if the +* guest can execute MWAIT without exiting as the timer will stop +* counting if the core enters C3 or lower. HLT in the guest is ok as +* HLT is effectively C1 and the timer counts in C0, C1, and C2. +*
arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 137
Quoted 10 lines
arch/x86/kvm/lapic.c +7
-0+* Don't use the hypervisor timer if KVM can post a timer interrupt to +* the guest since posted the timer avoids taking an extra a VM-Exit +* when the timer expires. +*/ +return kvm_x86_ops.set_hv_timer && + !kvm_mwait_in_guest (vcpu->kvm) && + !kvm_can_post_timer_interrupt(vcpu)); }EXPORT_SYMBOL_GPL (kvm_can_use_hv_timer);
It seems Sean and Wanpeng are busy with some other more important issues;
So Please let me try to merge Sean, Wanpeng's ideas and suggestions together,also including my opinions into one possible approach and get it reviewed, Only if others are OK with this;
I will post a new patch for this later today or tomorrow.
Thanks!
--Aili Yao
Sean Christopherson
maintainerQuoted 62 lines · Aili Yao
Quoted 45 lines · On Tue, 7 Dec 2021 23:23:03 +0000 Sean Christopherson <[email protected]> wrote:
Quoted 9 lines · Wanpeng Li
--- arch/x86/kvm/lapic.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)arch/x86/kvm/lapic.c +0
-0arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 116
Quoted 5 lines
arch/x86/kvm/lapic.c +1
-1@@ -113,14 +113,13 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_mwait_in_guest (vcpu->kvm) &&kvm_vcpu_apicv_active (vcpu);As Aili's changelog pointed out, MWAIT may not be advertised to the guest.
So I think we want this? With a non-functional, opinionated refactoring of
kvm_can_use_hv_timer()because I'm terrible at reading !(a || b).arch/x86/kvm/lapic.c +8
-4@@ -113,14 +113,25 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu) && + (kvm_mwait_in_guest (vcpu) ||kvm_hlt_in_guest (vcpu)); }bool kvm_can_use_hv_timer (struct kvm_vcpu *vcpu) { -return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest (vcpu->kvm) || -kvm_can_post_timer_interrupt (vcpu)); +/* +* Don't use the hypervisor timer, a.k.a. VMX Preemption Timer, if the +* guest can execute MWAIT without exiting as the timer will stop +* counting if the core enters C3 or lower. HLT in the guest is ok as +* HLT is effectively C1 and the timer counts in C0, C1, and C2. +* arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 137
Quoted 10 lines
arch/x86/kvm/lapic.c +7
-0+* Don't use the hypervisor timer if KVM can post a timer interrupt to +* the guest since posted the timer avoids taking an extra a VM-Exit +* when the timer expires. +*/ +return kvm_x86_ops.set_hv_timer && + !kvm_mwait_in_guest (vcpu->kvm) && + !kvm_can_post_timer_interrupt(vcpu)); }EXPORT_SYMBOL_GPL (kvm_can_use_hv_timer);It seems Sean and Wanpeng are busy with some other more important issues;
So Please let me try to merge Sean, Wanpeng's ideas and suggestions together,also including my opinions into one possible approach and get it reviewed, Only if others are OK with this;
Quoted 1 line
I will post a new patch for this later today or tomorrow.
Sorry, I was waiting for someone to say "this works", but never actually said as much.
Does the above change address your use case? If not, what's missing?
Quoted 63 lines · On Thu, 16 Dec 2021 15:45:57 +0000 Sean Christopherson <[email protected]>…
Quoted 62 lines · Aili Yao
Quoted 45 lines · On Tue, 7 Dec 2021 23:23:03 +0000 Sean Christopherson <[email protected]> wrote:
Quoted 9 lines · Wanpeng Li
--- arch/x86/kvm/lapic.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)arch/x86/kvm/lapic.c +0
-0arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 116
Quoted 5 lines
arch/x86/kvm/lapic.c +1
-1@@ -113,14 +113,13 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_mwait_in_guest (vcpu->kvm) &&kvm_vcpu_apicv_active (vcpu);As Aili's changelog pointed out, MWAIT may not be advertised to the guest.
So I think we want this? With a non-functional, opinionated refactoring of
kvm_can_use_hv_timer()because I'm terrible at reading !(a || b).arch/x86/kvm/lapic.c +8
-4@@ -113,14 +113,25 @@ static inline u32 kvm_x2apic_id(struct kvm_lapic *apic)static bool kvm_can_post_timer_interrupt (struct kvm_vcpu *vcpu) { -return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu); +return pi_inject_timer &&kvm_vcpu_apicv_active (vcpu) && + (kvm_mwait_in_guest (vcpu) ||kvm_hlt_in_guest (vcpu)); }bool kvm_can_use_hv_timer (struct kvm_vcpu *vcpu) { -return kvm_x86_ops.set_hv_timer - && !(kvm_mwait_in_guest (vcpu->kvm) || -kvm_can_post_timer_interrupt (vcpu)); +/* +* Don't use the hypervisor timer, a.k.a. VMX Preemption Timer, if the +* guest can execute MWAIT without exiting as the timer will stop +* counting if the core enters C3 or lower. HLT in the guest is ok as +* HLT is effectively C1 and the timer counts in C0, C1, and C2. +* arch/x86/kvm/lapic.c static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) · line 137
Quoted 10 lines
arch/x86/kvm/lapic.c +7
-0+* Don't use the hypervisor timer if KVM can post a timer interrupt to +* the guest since posted the timer avoids taking an extra a VM-Exit +* when the timer expires. +*/ +return kvm_x86_ops.set_hv_timer && + !kvm_mwait_in_guest (vcpu->kvm) && + !kvm_can_post_timer_interrupt(vcpu)); }EXPORT_SYMBOL_GPL (kvm_can_use_hv_timer);It seems Sean and Wanpeng are busy with some other more important issues;
So Please let me try to merge Sean, Wanpeng's ideas and suggestions together,also including my opinions into one possible approach and get it reviewed, Only if others are OK with this;
Quoted 6 lines
Quoted 1 line
I will post a new patch for this later today or tomorrow.
Sorry, I was waiting for someone to say "this works", but never actually said as much.
Does the above change address your use case? If not, what's missing?
After a little modifications, This works in my test.
static bool kvm_can_post_timer_interrupt(struct kvm_vcpu *vcpu) {
- return pi_inject_timer && kvm_vcpu_apicv_active(vcpu);
+ return pi_inject_timer && kvm_vcpu_apicv_active(vcpu) &&
+ (kvm_mwait_in_guest(vcpu->kvm) || kvm_hlt_in_guest(vcpu->kvm));
}
and also you can delete or keep kvm_mwait_in_guest() check;
Thanks!