KVM: collect sporadic patches
Subsystems (4)
v2 cover KVM: collect sporadic patches
Collect sporadic patches for easy apply.
Wanpeng Li (9):
KVM: LAPIC: Return 0 when getting the tscdeadline timer if the lapic
is hw disabled
KVM: LAPIC: Guarantee the timer is in tsc-deadline mode when setting
KVM: LAPIC: Fix updating DFR missing apic map recalculation
KVM: VMX: Don't freeze guest when event delivery causes an APIC-access
exit
KVM: LAPIC: Narrow down the kick target vCPU
KVM: LAPIC: Reduce world switch latency caused by timer_advance_ns
KVM: SVM: Get rid of handle_fastpath_set_msr_irqoff()
KVM: SVM: Move svm_complete_interrupts() into svm_vcpu_run()
KVM: SVM: Reenable handle_fastpath_set_msr_irqoff() after
complete_interrupts()
arch/x86/kvm/lapic.c | 36 ++++++++++++++++++++---------------- arch/x86/kvm/svm/svm.c | 17 +++++++++-------- arch/x86/kvm/vmx/vmx.c | 5 ++--- arch/x86/kvm/x86.c | 6 ------ arch/x86/kvm/x86.h | 1 - 5 files changed, 31 insertions(+), 34 deletions(-)
Signature · 1 line
2.7.4
PATCH v2 1/9 KVM: LAPIC: Return 0 when getting the tscdeadline timer if the lapic is hw disabled
From: Wanpeng Li <[email protected]>
Return 0 when getting the tscdeadline timer if the lapic is hw disabled.
--- arch/x86/kvm/lapic.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
arch/x86/kvm/lapic.c +1 -2
@@ -2183,8 +2183,7 @@ u64 kvm_get_lapic_tscdeadline_msr(struct kvm_vcpu *vcpu) {struct kvm_lapic *apic = vcpu->arch.apic; -if (!lapic_in_kernel (vcpu) || - !apic_lvtt_tscdeadline (apic)) +if (!kvm_apic_present (vcpu) || !apic_lvtt_tscdeadline (apic))return 0 ;return apic->lapic_timer.tscdeadline;
Signature · 1 line
2.7.4
PATCH v2 2/9 KVM: LAPIC: Guarantee the timer is in tsc-deadline mode when setting
In reply to Wanpeng LiFrom: Wanpeng Li <[email protected]>
Check apic_lvtt_tscdeadline() mode directly instead of apic_lvtt_oneshot() and apic_lvtt_period() to guarantee the timer is in tsc-deadline mode when wrmsr MSR_IA32_TSCDEADLINE.
--- arch/x86/kvm/lapic.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
arch/x86/kvm/lapic.c +1 -2
@@ -2193,8 +2193,7 @@ void kvm_set_lapic_tscdeadline_msr(struct kvm_vcpu *vcpu, u64 data) {struct kvm_lapic *apic = vcpu->arch.apic; -if (!kvm_apic_present (vcpu) ||apic_lvtt_oneshot (apic) || -apic_lvtt_period (apic)) +if (!kvm_apic_present (vcpu) || !apic_lvtt_tscdeadline (apic))return ;hrtimer_cancel (&apic->lapic_timer.timer);
Signature · 1 line
2.7.4
PATCH v2 3/9 KVM: LAPIC: Fix updating DFR missing apic map recalculation
In reply to Wanpeng LiFrom: Wanpeng Li <[email protected]>
There is missing apic map recalculation after updating DFR, if it is INIT RESET, in x2apic mode, local apic is software enabled before. This patch fix it by introducing the function kvm_apic_set_dfr() to be called in INIT RESET handling path.
--- arch/x86/kvm/lapic.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
arch/x86/kvm/lapic.c +10 -5
@@ -310,6 +310,12 @@ static inline void kvm_apic_set_ldr(struct kvm_lapic *apic, u32 id)atomic_set_release (&apic->vcpu->kvm->arch.apic_map_dirty,DIRTY ); } +static inline void kvm_apic_set_dfr (struct kvm_lapic *apic,u32 val) +{ +kvm_lapic_set_reg (apic,APIC_DFR , val); +atomic_set_release (&apic->vcpu->kvm->arch.apic_map_dirty,DIRTY ); +} +static inline u32 kvm_apic_calc_x2apic_ldr (u32 id) {return ((id >>4 ) <<16 ) | (1 << (id &0xf ));
@@ -1984,10 +1990,9 @@ int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val)break ;case APIC_DFR : -if (!apic_x2apic_mode (apic)) { -kvm_lapic_set_reg (apic,APIC_DFR , val |0x0FFFFFFF ); -atomic_set_release (&apic->vcpu->kvm->arch.apic_map_dirty,DIRTY ); - }else +if (!apic_x2apic_mode (apic)) +kvm_apic_set_dfr (apic, val |0x0FFFFFFF ); +else ret =1 ;break ;
@@ -2301,7 +2306,7 @@ void kvm_lapic_reset(struct kvm_vcpu *vcpu, bool init_event) SET_APIC_DELIVERY_MODE(0, APIC_MODE_EXTINT));apic_manage_nmi_watchdog (apic,kvm_lapic_get_reg (apic,APIC_LVT0 )); -kvm_lapic_set_reg (apic,APIC_DFR ,0xffffffffU ); +kvm_apic_set_dfr (apic,0xffffffffU );apic_set_spiv (apic,0xff );kvm_lapic_set_reg (apic,APIC_TASKPRI ,0 );if (!apic_x2apic_mode (apic))
Signature · 1 line
2.7.4
PATCH v2 4/9 KVM: VMX: Don't freeze guest when event delivery causes an APIC-access exit
In reply to Wanpeng LiFrom: Wanpeng Li <[email protected]>
According to SDM 27.2.4, Event delivery causes an APIC-access VM exit. Don't report internal error and freeze guest when event delivery causes an APIC-access exit, it is handleable and the event will be re-injected during the next vmentry.
--- arch/x86/kvm/vmx/vmx.c | 1 + 1 file changed, 1 insertion(+)
arch/x86/kvm/vmx/vmx.c +1 -0
@@ -6054,6 +6054,7 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) (exit_reason !=EXIT_REASON_EXCEPTION_NMI && exit_reason !=EXIT_REASON_EPT_VIOLATION && exit_reason !=EXIT_REASON_PML_FULL && + exit_reason !=EXIT_REASON_APIC_ACCESS && exit_reason != EXIT_REASON_TASK_SWITCH)) { vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR; vcpu->run->internal.suberror =KVM_INTERNAL_ERROR_DELIVERY_EV ;
Signature · 1 line
2.7.4
Re: [PATCH v2 4/9] KVM: VMX: Don't freeze guest when event delivery causes an APIC-access exit
Quoted 9 lines · Wanpeng Li
From: Wanpeng Li <[email protected]>
According to SDM 27.2.4, Event delivery causes an APIC-access VM exit. Don't report internal error and freeze guest when event delivery causes an APIC-access exit, it is handleable and the event will be re-injected during the next vmentry.
Signed-off-by Wanpeng Li <[email protected]>
I'm curious if you have a test case for this.
PATCH v2 5/9 KVM: LAPIC: Narrow down the kick target vCPU
In reply to Wanpeng LiFrom: Wanpeng Li <[email protected]>
The kick after setting KVM_REQ_PENDING_TIMER is used to handle the timer fires on a different pCPU which vCPU is running on, this kick is expensive since memory barrier, rcu, preemption disable/enable operations. We don't need this kick when injecting already-expired timer, we also should call out the VMX preemption timer case, which also passes from_timer_fn=false but doesn't need a kick because kvm_lapic_expired_hv_timer() is called from the target vCPU.
--- arch/x86/kvm/lapic.c | 4 +++- arch/x86/kvm/x86.c | 6 ------ arch/x86/kvm/x86.h | 1 - 3 files changed, 3 insertions(+), 8 deletions(-)
arch/x86/kvm/lapic.c +3 -1
@@ -1642,7 +1642,9 @@ static void apic_timer_expired(struct kvm_lapic *apic, bool from_timer_fn) }atomic_inc (&apic->lapic_timer.pending); -kvm_set_pending_timer (vcpu); +kvm_make_request (KVM_REQ_PENDING_TIMER , vcpu); +if (from_timer_fn) +kvm_vcpu_kick (vcpu); }static void start_sw_tscdeadline (struct kvm_lapic *apic)
arch/x86/kvm/x86.c +0 -6
@@ -1774,12 +1774,6 @@ static s64 get_kvmclock_base_ns(void) }#endif -void kvm_set_pending_timer (struct kvm_vcpu *vcpu) -{ -kvm_make_request (KVM_REQ_PENDING_TIMER , vcpu); -kvm_vcpu_kick (vcpu); -} -static void kvm_write_wall_clock (struct kvm *kvm,gpa_t wall_clock) {int version;
arch/x86/kvm/x86.h +0 -1
@@ -246,7 +246,6 @@ static inline bool kvm_vcpu_latch_init(struct kvm_vcpu *vcpu)return is_smm (vcpu) || kvm_x86_ops.apic_init_signal_blocked (vcpu); } -void kvm_set_pending_timer (struct kvm_vcpu *vcpu);void kvm_inject_realmode_interrupt (struct kvm_vcpu *vcpu,int irq,int inc_eip);void kvm_write_tsc (struct kvm_vcpu *vcpu,struct msr_data *msr);
Signature · 1 line
2.7.4
Paolo Bonzini
maintainerRe: [PATCH v2 5/9] KVM: LAPIC: Narrow down the kick target vCPU
Quoted 23 lines · Wanpeng Li
From: Wanpeng Li <[email protected]>
The kick after setting
KVM_REQ_PENDING_TIMERis used to handle the timer fires on a different pCPU which vCPU is running on, this kick is expensive since memory barrier, rcu, preemption disable/enable operations. We don't need this kick when injecting already-expired timer, we also should call out the VMX preemption timer case, which also passes from_timer_fn=false but doesn't need a kick becausekvm_lapic_expired_hv_timer()is called from the target vCPU.Reviewed-by Sean Christopherson <[email protected]>Signed-off-by Wanpeng Li <[email protected]>--- arch/x86/kvm/lapic.c | 4 +++- arch/x86/kvm/x86.c | 6 ------ arch/x86/kvm/x86.h | 1 - 3 files changed, 3 insertions(+), 8 deletions(-)arch/x86/kvm/lapic.c +0
-0
arch/x86/kvm/lapic.c static void apic_timer_expired(struct kvm_lapic *apic, bool from_timer_fn) · line 1648
Quoted 8 lines
arch/x86/kvm/lapic.c +3
-1@@ -1642,7 +1642,9 @@ static void apic_timer_expired(struct kvm_lapic *apic, bool from_timer_fn) }atomic_inc (&apic->lapic_timer.pending); -kvm_set_pending_timer (vcpu); +kvm_make_request (KVM_REQ_PENDING_TIMER , vcpu); +if (from_timer_fn) +kvm_vcpu_kick (vcpu); }
This only matters for !APICv, but it's nice anyway.
Paolo
Quoted 31 lines
arch/x86/kvm/lapic.c +0
-0static void start_sw_tscdeadline (struct kvm_lapic *apic)arch/x86/kvm/x86.c +0
-6@@ -1774,12 +1774,6 @@ static s64 get_kvmclock_base_ns(void) }#endif -void kvm_set_pending_timer (struct kvm_vcpu *vcpu) -{ -kvm_make_request (KVM_REQ_PENDING_TIMER , vcpu); -kvm_vcpu_kick (vcpu); -} -static void kvm_write_wall_clock (struct kvm *kvm,gpa_t wall_clock) {int version;arch/x86/kvm/x86.h +0
-1@@ -246,7 +246,6 @@ static inline bool kvm_vcpu_latch_init(struct kvm_vcpu *vcpu)return is_smm (vcpu) || kvm_x86_ops.apic_init_signal_blocked (vcpu); } -void kvm_set_pending_timer (struct kvm_vcpu *vcpu);void kvm_inject_realmode_interrupt (struct kvm_vcpu *vcpu,int irq,int inc_eip);void kvm_write_tsc (struct kvm_vcpu *vcpu,struct msr_data *msr);
PATCH v2 6/9 KVM: LAPIC: Reduce world switch latency caused by timer_advance_ns
In reply to Wanpeng LiFrom: Wanpeng Li <[email protected]>
All the checks in lapic_timer_int_injected(), __kvm_wait_lapic_expire(), and these function calls waste cpu cycles when the timer mode is not tscdeadline. We can observe ~1.3% world switch time overhead by kvm-unit-tests/vmexit.flat vmcall testing on AMD server. This patch reduces the world switch latency caused by timer_advance_ns feature when the timer mode is not tscdeadline by simpling move the check against apic->lapic_timer.expired_tscdeadline much earlier.
--- arch/x86/kvm/lapic.c | 11 +++++------ arch/x86/kvm/svm/svm.c | 4 +--- arch/x86/kvm/vmx/vmx.c | 4 +--- 3 files changed, 7 insertions(+), 12 deletions(-)
arch/x86/kvm/lapic.c +5 -6
@@ -1582,9 +1582,6 @@ static void __kvm_wait_lapic_expire(struct kvm_vcpu *vcpu)struct kvm_lapic *apic = vcpu->arch.apic;u64 guest_tsc, tsc_deadline; -if (apic->lapic_timer.expired_tscdeadline ==0 ) -return ; - tsc_deadline = apic->lapic_timer.expired_tscdeadline; apic->lapic_timer.expired_tscdeadline =0 ; guest_tsc =kvm_read_l1_tsc (vcpu,rdtsc ());
@@ -1599,7 +1596,10 @@ static void __kvm_wait_lapic_expire(struct kvm_vcpu *vcpu)void kvm_wait_lapic_expire (struct kvm_vcpu *vcpu) { -if (lapic_timer_int_injected (vcpu)) +if (lapic_in_kernel (vcpu) && + vcpu->arch.apic->lapic_timer.expired_tscdeadline && + vcpu->arch.apic->lapic_timer.timer_advance_ns && +lapic_timer_int_injected (vcpu))__kvm_wait_lapic_expire (vcpu); }EXPORT_SYMBOL_GPL (kvm_wait_lapic_expire);
@@ -1635,8 +1635,7 @@ static void apic_timer_expired(struct kvm_lapic *apic, bool from_timer_fn) }if (kvm_use_posted_timer_interrupt (apic->vcpu)) { -if (apic->lapic_timer.timer_advance_ns) -__kvm_wait_lapic_expire (vcpu); +kvm_wait_lapic_expire (vcpu);kvm_apic_inject_pending_timer_irqs (apic);return ; }
arch/x86/kvm/svm/svm.c +1 -3
@@ -3456,9 +3456,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)clgi ();kvm_load_guest_xsave_state (vcpu); -if (lapic_in_kernel (vcpu) && - vcpu->arch.apic->lapic_timer.timer_advance_ns) -kvm_wait_lapic_expire (vcpu); +kvm_wait_lapic_expire (vcpu);/* * If this vCPU has touched SPEC_CTRL, restore the guest's value if
arch/x86/kvm/vmx/vmx.c +1 -3
@@ -6800,9 +6800,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu)if (enable_preemption_timer)vmx_update_hv_timer (vcpu); -if (lapic_in_kernel (vcpu) && - vcpu->arch.apic->lapic_timer.timer_advance_ns) -kvm_wait_lapic_expire (vcpu); +kvm_wait_lapic_expire (vcpu);/* * If this vCPU has touched SPEC_CTRL, restore the guest's value if
Signature · 1 line
2.7.4
PATCH v2 7/9 KVM: SVM: Get rid of handle_fastpath_set_msr_irqoff()
In reply to Wanpeng LiFrom: Wanpeng Li <[email protected]>
Analysis from Sean:
| svm->next_rip is reset in svm_vcpu_run() only after calling | svm_exit_handlers_fastpath(), which will cause SVM's | skip_emulated_instruction() to write a stale RIP.
Let's get rid of handle_fastpath_set_msr_irqoff() in svm_exit_handlers_fastpath() to have a quick fix.
--- arch/x86/kvm/svm/svm.c | 5 ----- 1 file changed, 5 deletions(-)
arch/x86/kvm/svm/svm.c +0 -5
@@ -3349,11 +3349,6 @@ static void svm_cancel_injection(struct kvm_vcpu *vcpu)static fastpath_t svm_exit_handlers_fastpath (struct kvm_vcpu *vcpu) { -if (!is_guest_mode (vcpu) && -to_svm (vcpu)->vmcb->control.exit_code ==SVM_EXIT_MSR && -to_svm (vcpu)->vmcb->control.exit_info_1) -return handle_fastpath_set_msr_irqoff (vcpu); -return EXIT_FASTPATH_NONE ; }
Signature · 1 line
2.7.4
PATCH v2 8/9 KVM: SVM: Move svm_complete_interrupts() into svm_vcpu_run()
In reply to Wanpeng LiFrom: Wanpeng Li <[email protected]>
Moving svm_complete_interrupts() into svm_vcpu_run() which can align VMX and SVM with respect to completing interrupts.
--- arch/x86/kvm/svm/svm.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
arch/x86/kvm/svm/svm.c +1 -2
@@ -2938,8 +2938,6 @@ static int handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath)if (npt_enabled) vcpu->arch.cr3 = svm->vmcb->save.cr3; -svm_complete_interrupts (svm); -if (is_guest_mode (vcpu)) {int vmexit;
@@ -3530,6 +3528,7 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) SVM_EXIT_EXCP_BASE + MC_VECTOR))svm_handle_mce (svm); +svm_complete_interrupts (svm);vmcb_mark_all_clean (svm->vmcb);return exit_fastpath; }
Signature · 1 line
2.7.4
PATCH v2 9/9 KVM: SVM: Reenable handle_fastpath_set_msr_irqoff() after complete_interrupts()
In reply to Wanpeng LiFrom: Wanpeng Li <[email protected]>
Moving the call to svm_exit_handlers_fastpath() after svm_complete_interrupts() since svm_complete_interrupts() consumes rip and reenable the function handle_fastpath_set_msr_irqoff() call in svm_exit_handlers_fastpath().
--- arch/x86/kvm/svm/svm.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/svm.c +10 -1
@@ -3347,6 +3347,10 @@ static void svm_cancel_injection(struct kvm_vcpu *vcpu)static fastpath_t svm_exit_handlers_fastpath (struct kvm_vcpu *vcpu) { +if (to_svm (vcpu)->vmcb->control.exit_code ==SVM_EXIT_MSR && +to_svm (vcpu)->vmcb->control.exit_info_1) +return handle_fastpath_set_msr_irqoff (vcpu); +return EXIT_FASTPATH_NONE ; }
@@ -3495,7 +3499,6 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)stgi ();/* Any pending NMI will happen here */ - exit_fastpath =svm_exit_handlers_fastpath (vcpu);if (unlikely (svm->vmcb->control.exit_code ==SVM_EXIT_NMI ))kvm_after_interrupt (&svm->vcpu);
@@ -3530,6 +3533,12 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu)svm_complete_interrupts (svm);vmcb_mark_all_clean (svm->vmcb); + +if (is_guest_mode (vcpu)) +return EXIT_FASTPATH_NONE ; + + exit_fastpath =svm_exit_handlers_fastpath (vcpu); +return exit_fastpath; }
Signature · 1 line
2.7.4
Quoted 16 lines · Wanpeng Li
Collect sporadic patches for easy apply.
Wanpeng Li (9):
KVM: LAPIC: Return 0 when getting the tscdeadline timer if the lapic
is hw disabled
KVM: LAPIC: Guarantee the timer is in tsc-deadline mode when setting
KVM: LAPIC: Fix updating DFR missing apic map recalculation
KVM: VMX: Don't freeze guest when event delivery causes an APIC-access
exit
KVM: LAPIC: Narrow down the kick target vCPU
KVM: LAPIC: Reduce world switch latency caused bytimer_advance_ns
KVM: SVM: Get rid ofhandle_fastpath_set_msr_irqoff()
KVM: SVM: Movesvm_complete_interrupts()intosvm_vcpu_run()
KVM: SVM: Reenablehandle_fastpath_set_msr_irqoff()after
complete_interrupts()
Quoted 7 lines
arch/x86/kvm/lapic.c | 36 ++++++++++++++++++++---------------- arch/x86/kvm/svm/svm.c | 17 +++++++++-------- arch/x86/kvm/vmx/vmx.c | 5 ++--- arch/x86/kvm/x86.c | 6 ------ arch/x86/kvm/x86.h | 1 - 5 files changed, 31 insertions(+), 34 deletions(-)
Queued what was left, thanks.
Paolo