KVM updates for the 2.6.31 merge window (batch 3/3)
cover KVM updates for the 2.6.31 merge window (batch 3/3)
Third batch of the 2.6.31 merge window submission. Please review.
Andre Przywara (1):
KVM: SVM: Fix cross vendor migration issue in segment segment
descriptor
Avi Kivity (4):
KVM: Fix cpuid feature misreporting
KVM: Add AMD cpuid bit: cr8_legacy, abm, misaligned sse, sse4, 3dnow
prefetch
x86: Add cpu features MOVBE and POPCNT
KVM: Update cpuid 1.ecx reporting
Carsten Otte (4):
KVM: s390: Fix memory slot versus run - v3
KVM: s390: Unlink vcpu on destroy - v2
KVM: s390: Sanity check on validity intercept
KVM: s390: Verify memory in kvm run
Chris Wright (1):
KVM: Trivial format fix in setup_routing_entry()
Christian Borntraeger (2):
KVM: s390: use hrtimer for clock wakeup from idle - v2
KVM: s390: optimize float int lock: spin_lock_bh --> spin_lock
Glauber Costa (3):
KVM: fix apic_debug instances
KVM: Replace ->drop_interrupt_shadow() by ->set_interrupt_shadow()
KVM: Deal with interrupt shadow state for emulated instructions
Gleb Natapov (19):
KVM: Remove inject_pending_vectors() callback
KVM: Remove kvm_push_irq()
KVM: sync_lapic_to_cr8() should always sync cr8 to V_TPR
KVM: Do not report TPR write to userspace if new value bigger or
equal to a previous one.
KVM: Get rid of arch.interrupt_window_open & arch.nmi_window_open
KVM: SVM: Add NMI injection support
KVM: Fix userspace IRQ chip migration
KVM: Get rid of get_irq() callback
KVM: SVM: Don't reinject event that caused a task switch
KVM: Unprotect a page if #PF happens during NMI injection.
KVM: Do not allow interrupt injection from userspace if there is a
pending event.
KVM: Remove irq_pending bitmap
KVM: skip_emulated_instruction() decode instruction if size is not
known
KVM: Do not re-execute INTn instruction.
KVM: Always request IRQ/NMI window if an interrupt is pending
KVM: inject NMI after IRET from a previous NMI, not before.
KVM: Do not migrate pending software interrupts.
KVM: Disable CR8 intercept if tpr patching is active
KVM: Move "exit due to NMI" handling into vmx_complete_interrupts()
Jan Blunck (1):
KVM: Wake up waitqueue before calling get_cpu()
Jan Kiszka (1):
KVM: Drop request_nmi from stats
Marcelo Tosatti (5):
KVM: use smp_send_reschedule in kvm_vcpu_kick
KVM: protect assigned dev workqueue, int handler and irq acker
KVM: MMU: protect kvm_mmu_change_mmu_pages with mmu_lock
KVM: take mmu_lock when updating a deleted slot
KVM: x86: check for cr3 validity in mmu_alloc_roots
Robert P. J. Day (1):
KVM: Expand on "help" info to specify kvm intel and amd module names
Sheng Yang (3):
KVM: Replace get_mt_mask_shift with get_mt_mask
KVM: Enable snooping control for supported hardware
KVM: VMX: Disable VMX when system shutdown
arch/ia64/include/asm/kvm_host.h | 1 + arch/ia64/kernel/irq_ia64.c | 3 + arch/ia64/kvm/kvm-ia64.c | 22 +-- arch/s390/include/asm/kvm_host.h | 5 +- arch/s390/kvm/intercept.c | 28 +++-- arch/s390/kvm/interrupt.c | 53 ++++--- arch/s390/kvm/kvm-s390.c | 63 +++++++-- arch/s390/kvm/kvm-s390.h | 4 +- arch/s390/kvm/priv.c | 4 +- arch/s390/kvm/sigp.c | 16 +- arch/x86/include/asm/cpufeature.h | 2 + arch/x86/include/asm/kvm_host.h | 30 +++-- arch/x86/include/asm/kvm_x86_emulate.h | 6 + arch/x86/kernel/smp.c | 3 + arch/x86/kvm/Kconfig | 6 + arch/x86/kvm/irq.c | 4 +- arch/x86/kvm/lapic.c | 4 +- arch/x86/kvm/mmu.c | 50 ++++--- arch/x86/kvm/svm.c | 248 ++++++++++++++++++++----------- arch/x86/kvm/vmx.c | 260 ++++++++++++++++---------------- arch/x86/kvm/x86.c | 247 ++++++++++++++++++------------ arch/x86/kvm/x86.h | 22 +-- arch/x86/kvm/x86_emulate.c | 20 +++ include/linux/kvm_host.h | 6 +- virt/kvm/iommu.c | 27 +++- virt/kvm/irq_comm.c | 2 +- virt/kvm/kvm_main.c | 33 +++-- 27 files changed, 721 insertions(+), 448 deletions(-)
PATCH 1/45 KVM: Remove inject_pending_vectors() callback
From: Gleb Natapov <[email protected]>
It is the same as inject_pending_irq() for VMX/SVM now.
--- arch/x86/include/asm/kvm_host.h | 2 -- arch/x86/kvm/svm.c | 1 - arch/x86/kvm/vmx.c | 1 - arch/x86/kvm/x86.c | 4 +--- 4 files changed, 1 insertions(+), 7 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -2
@@ -517,8 +517,6 @@ struct kvm_x86_ops {void (*queue_exception)(struct kvm_vcpu *vcpu,unsigned nr,bool has_error_code,u32 error_code);void (*inject_pending_irq)(struct kvm_vcpu *vcpu,struct kvm_run *run); -void (*inject_pending_vectors)(struct kvm_vcpu *vcpu, -struct kvm_run *run);int (*interrupt_allowed)(struct kvm_vcpu *vcpu);int (*set_tss_addr)(struct kvm *kvm,unsigned int addr);int (*get_tdp_level)(void );
arch/x86/kvm/svm.c +0 -1
@@ -2655,7 +2655,6 @@ static struct kvm_x86_ops svm_x86_ops = { .set_irq = svm_set_irq, .queue_exception = svm_queue_exception, .inject_pending_irq = svm_intr_assist, - .inject_pending_vectors = svm_intr_assist, .interrupt_allowed = svm_interrupt_allowed, .set_tss_addr = svm_set_tss_addr,
arch/x86/kvm/vmx.c +0 -1
@@ -3693,7 +3693,6 @@ static struct kvm_x86_ops vmx_x86_ops = { .set_irq = vmx_inject_irq, .queue_exception = vmx_queue_exception, .inject_pending_irq = vmx_intr_assist, - .inject_pending_vectors = vmx_intr_assist, .interrupt_allowed = vmx_interrupt_allowed, .set_tss_addr = vmx_set_tss_addr, .get_tdp_level = get_ept_level,
arch/x86/kvm/x86.c +1 -3
@@ -3167,10 +3167,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)if (vcpu->arch.exception.pending)__queue_exception (vcpu); -else if (irqchip_in_kernel (vcpu->kvm)) - kvm_x86_ops->inject_pending_irq (vcpu, kvm_run);else - kvm_x86_ops->inject_pending_vectors (vcpu, kvm_run); + kvm_x86_ops->inject_pending_irq (vcpu, kvm_run);kvm_lapic_sync_to_vapic (vcpu);
Signature · 1 line
1.6.0.6
PATCH 2/45 KVM: Remove kvm_push_irq()
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
No longer used.
--- arch/x86/kvm/x86.h | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-)
arch/x86/kvm/x86.h +0 -7
@@ -30,11 +30,4 @@ static inline u8 kvm_pop_irq(struct kvm_vcpu *vcpu)clear_bit (word_index, &vcpu->arch.irq_summary); return irq; } - -static inline void kvm_push_irq (struct kvm_vcpu *vcpu,u8 irq) -{ -set_bit (irq, vcpu->arch.irq_pending); -set_bit (irq /BITS_PER_LONG , &vcpu->arch.irq_summary); -} -#endif
Signature · 1 line
1.6.0.6
PATCH 3/45 KVM: sync_lapic_to_cr8() should always sync cr8 to V_TPR
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
Even if IRQ chip is in userspace.
--- arch/x86/kvm/svm.c | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-)
arch/x86/kvm/svm.c +1 -4
@@ -2348,7 +2348,7 @@ static inline void sync_cr8_to_lapic(struct kvm_vcpu *vcpu)if (!(svm->vmcb->control.intercept_cr_write &INTERCEPT_CR8_MASK )) {int cr8 = svm->vmcb->control.int_ctl &V_TPR_MASK ; -kvm_lapic_set_tpr (vcpu, cr8); +kvm_set_cr8 (vcpu, cr8); } }
@@ -2357,9 +2357,6 @@ static inline void sync_lapic_to_cr8(struct kvm_vcpu *vcpu)struct vcpu_svm *svm =to_svm (vcpu);u64 cr8; -if (!irqchip_in_kernel (vcpu->kvm)) -return ; - cr8 =kvm_get_cr8 (vcpu); svm->vmcb->control.int_ctl &= ~V_TPR_MASK ; svm->vmcb->control.int_ctl |= cr8 &V_TPR_MASK ;
Signature · 1 line
1.6.0.6
PATCH 4/45 KVM: Do not report TPR write to userspace if new value bigger or equal to a previous one.
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
Saves many exits to userspace in a case of IRQ chip in userspace.
--- arch/x86/kvm/svm.c | 4 ++++ arch/x86/kvm/vmx.c | 19 ++++++++++++------- 2 files changed, 16 insertions(+), 7 deletions(-)
arch/x86/kvm/svm.c +4 -0
@@ -1862,9 +1862,13 @@ static int emulate_on_interception(struct vcpu_svm *svm,static int cr8_write_interception (struct vcpu_svm *svm,struct kvm_run *kvm_run) { +u8 cr8_prev =kvm_get_cr8 (&svm->vcpu); +/* instruction emulation calls kvm_set_cr8() */ emulate_instruction (&svm->vcpu,NULL ,0 ,0 ,0 );if (irqchip_in_kernel (svm->vcpu.kvm))return 1 ; +if (cr8_prev <=kvm_get_cr8 (&svm->vcpu)) +return 1 ; kvm_run->exit_reason =KVM_EXIT_SET_TPR ;return 0 ; }
arch/x86/kvm/vmx.c +12 -7
@@ -2724,13 +2724,18 @@ static int handle_cr(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)kvm_set_cr4 (vcpu,kvm_register_read (vcpu, reg));skip_emulated_instruction (vcpu);return 1 ; -case 8 : -kvm_set_cr8 (vcpu,kvm_register_read (vcpu, reg)); -skip_emulated_instruction (vcpu); -if (irqchip_in_kernel (vcpu->kvm)) -return 1 ; - kvm_run->exit_reason =KVM_EXIT_SET_TPR ; -return 0 ; +case 8 : { +u8 cr8_prev =kvm_get_cr8 (vcpu); +u8 cr8 =kvm_register_read (vcpu, reg); +kvm_set_cr8 (vcpu, cr8); +skip_emulated_instruction (vcpu); +if (irqchip_in_kernel (vcpu->kvm)) +return 1 ; +if (cr8_prev <= cr8) +return 1 ; + kvm_run->exit_reason =KVM_EXIT_SET_TPR ; +return 0 ; + } };break ;case 2 :/* clts */
Signature · 1 line
1.6.0.6
PATCH 5/45 KVM: Get rid of arch.interrupt_window_open & arch.nmi_window_open
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
They are recalculated before each use anyway.
--- arch/x86/include/asm/kvm_host.h | 2 -- arch/x86/kvm/svm.c | 6 +----- arch/x86/kvm/vmx.c | 35 +++++++++++------------------------ 3 files changed, 12 insertions(+), 31 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -2
@@ -266,7 +266,6 @@ struct kvm_mmu {struct kvm_vcpu_arch {u64 host_tsc; -int interrupt_window_open;unsigned long irq_summary;/* bit vector: 1 per word in irq_pending */ DECLARE_BITMAP(irq_pending, KVM_NR_INTERRUPTS);/*
@@ -360,7 +359,6 @@ struct kvm_vcpu_arch {bool nmi_pending;bool nmi_injected; -bool nmi_window_open;struct mtrr_state_type mtrr_state;u32 pat;
arch/x86/kvm/svm.c +1 -5
@@ -216,8 +216,6 @@ static void skip_emulated_instruction(struct kvm_vcpu *vcpu)kvm_rip_write (vcpu, svm->next_rip); svm->vmcb->control.int_state &= ~SVM_INTERRUPT_SHADOW_MASK ; - - vcpu->arch.interrupt_window_open = (svm->vcpu.arch.hflags &HF_GIF_MASK ); }static int has_svm (void )
@@ -2305,7 +2303,7 @@ static void svm_intr_inject(struct kvm_vcpu *vcpu)/* try to inject new event if pending */ if (kvm_cpu_has_interrupt (vcpu)) { -if (vcpu->arch.interrupt_window_open) { +if (svm_interrupt_allowed (vcpu)) {kvm_queue_interrupt (vcpu,kvm_cpu_get_interrupt (vcpu));svm_queue_irq (to_svm (vcpu), vcpu->arch.interrupt.nr); }
@@ -2321,8 +2319,6 @@ static void svm_intr_assist(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)if (nested_svm_intr (svm)) goto out; - svm->vcpu.arch.interrupt_window_open =svm_interrupt_allowed (vcpu); -svm_intr_inject (vcpu);if (kvm_cpu_has_interrupt (vcpu) || req_int_win)
arch/x86/kvm/vmx.c +11 -24
@@ -753,7 +753,6 @@ static void skip_emulated_instruction(struct kvm_vcpu *vcpu)if (interruptibility &3 )vmcs_write32 (GUEST_INTERRUPTIBILITY_INFO , interruptibility & ~3 ); - vcpu->arch.interrupt_window_open =1 ; } static void vmx_queue_exception(struct kvm_vcpu *vcpu, unsigned nr,
@@ -2482,27 +2481,21 @@ static void vmx_inject_nmi(struct kvm_vcpu *vcpu) INTR_TYPE_NMI_INTR | INTR_INFO_VALID_MASK | NMI_VECTOR); } -static void vmx_update_window_states (struct kvm_vcpu *vcpu) +static int vmx_nmi_allowed (struct kvm_vcpu *vcpu) { -u32 guest_intr =vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ); - - vcpu->arch.nmi_window_open = - !(guest_intr & (GUEST_INTR_STATE_STI | -GUEST_INTR_STATE_MOV_SS | -GUEST_INTR_STATE_NMI ));if (!cpu_has_virtual_nmis () &&to_vmx (vcpu)->soft_vnmi_blocked) - vcpu->arch.nmi_window_open =0 ; +return 0 ; - vcpu->arch.interrupt_window_open = - ((vmcs_readl (GUEST_RFLAGS ) &X86_EFLAGS_IF ) && - !(guest_intr & (GUEST_INTR_STATE_STI | -GUEST_INTR_STATE_MOV_SS ))); +return !(vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ) & + (GUEST_INTR_STATE_STI |GUEST_INTR_STATE_MOV_SS | +GUEST_INTR_STATE_NMI )); }static int vmx_interrupt_allowed (struct kvm_vcpu *vcpu) { -vmx_update_window_states (vcpu); -return vcpu->arch.interrupt_window_open; +return (vmcs_readl (GUEST_RFLAGS ) &X86_EFLAGS_IF ) && + !(vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ) & + (GUEST_INTR_STATE_STI |GUEST_INTR_STATE_MOV_SS )); }static int vmx_set_tss_addr (struct kvm *kvm,unsigned int addr)
@@ -3194,9 +3187,8 @@ static int vmx_handle_exit(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) __func__, vectoring_info, exit_reason);if (unlikely (!cpu_has_virtual_nmis () && vmx->soft_vnmi_blocked)) { -if (vcpu->arch.interrupt_window_open) { +if (vmx_interrupt_allowed (vcpu)) { vmx->soft_vnmi_blocked =0 ; - vcpu->arch.nmi_window_open =1 ; }else if (vmx->vnmi_blocked_time >1000000000LL && vcpu->arch.nmi_pending) {/*
@@ -3209,7 +3201,6 @@ static int vmx_handle_exit(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)"state on VCPU %d after 1 s timeout\n" , __func__, vcpu->vcpu_id); vmx->soft_vnmi_blocked =0 ; - vmx->vcpu.arch.nmi_window_open =1 ; } }
@@ -3324,13 +3315,13 @@ static void vmx_intr_inject(struct kvm_vcpu *vcpu)/* try to inject new event if pending */ if (vcpu->arch.nmi_pending) { -if (vcpu->arch.nmi_window_open) { +if (vmx_nmi_allowed (vcpu)) { vcpu->arch.nmi_pending = false; vcpu->arch.nmi_injected = true;vmx_inject_nmi (vcpu); } }else if (kvm_cpu_has_interrupt (vcpu)) { -if (vcpu->arch.interrupt_window_open) { +if (vmx_interrupt_allowed (vcpu)) {kvm_queue_interrupt (vcpu,kvm_cpu_get_interrupt (vcpu));vmx_inject_irq (vcpu, vcpu->arch.interrupt.nr); }
@@ -3344,8 +3335,6 @@ static void vmx_intr_assist(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)update_tpr_threshold (vcpu); -vmx_update_window_states (vcpu); -if (vcpu->guest_debug &KVM_GUESTDBG_SINGLESTEP ) vmcs_clear_bits(GUEST_INTERRUPTIBILITY_INFO, GUEST_INTR_STATE_STI |
@@ -3518,8 +3507,6 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)if (vmx->rmode.irq.pending)fixup_rmode_irq (vmx); -vmx_update_window_states (vcpu); - asm("mov %0, %%ds; mov %0, %%es" : :"r" (__USER_DS)); vmx->launched =1 ;
Signature · 1 line
1.6.0.6
PATCH 6/45 KVM: SVM: Add NMI injection support
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
--- arch/x86/include/asm/kvm_host.h | 8 +++- arch/x86/kvm/svm.c | 96 ++++++++++++++++++++------------------- arch/x86/kvm/vmx.c | 79 ++++++++------------------------ arch/x86/kvm/x86.c | 71 ++++++++++++++++++++++++++++- 4 files changed, 145 insertions(+), 109 deletions(-)
arch/x86/include/asm/kvm_host.h +7 -1
@@ -512,10 +512,15 @@ struct kvm_x86_ops { unsigned char *hypercall_addr);int (*get_irq)(struct kvm_vcpu *vcpu);void (*set_irq)(struct kvm_vcpu *vcpu,int vec); +void (*set_nmi)(struct kvm_vcpu *vcpu);void (*queue_exception)(struct kvm_vcpu *vcpu,unsigned nr,bool has_error_code,u32 error_code); -void (*inject_pending_irq)(struct kvm_vcpu *vcpu,struct kvm_run *run);int (*interrupt_allowed)(struct kvm_vcpu *vcpu); +int (*nmi_allowed)(struct kvm_vcpu *vcpu); +void (*enable_nmi_window)(struct kvm_vcpu *vcpu); +void (*enable_irq_window)(struct kvm_vcpu *vcpu); +void (*update_cr8_intercept)(struct kvm_vcpu *vcpu,int tpr,int irr); +void (*drop_interrupt_shadow)(struct kvm_vcpu *vcpu);int (*set_tss_addr)(struct kvm *kvm,unsigned int addr);int (*get_tdp_level)(void );int (*get_mt_mask_shift)(void );
@@ -763,6 +768,7 @@ enum {#define HF_GIF_MASK (1 << 0)#define HF_HIF_MASK (1 << 1)#define HF_VINTR_MASK (1 << 2) +#define HF_NMI_MASK (1 << 3)/* * Hardware virtualization extension instructions may fault if a
arch/x86/kvm/svm.c +50 -46
160 lines not on this page · raw message
arch/x86/kvm/vmx.c +19 -60
116 lines not on this page · raw message
arch/x86/kvm/x86.c +69 -2
84 lines not on this page · raw message
Signature · 1 line
1.6.0.6
PATCH 7/45 KVM: Fix userspace IRQ chip migration
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
Re-put pending IRQ vector into interrupt_bitmap before migration. Otherwise it will be lost if migration happens in the wrong time.
--- arch/x86/kvm/x86.c | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-)
arch/x86/kvm/x86.c +6 -6
@@ -3562,17 +3562,17 @@ int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu, sregs->efer = vcpu->arch.shadow_efer; sregs->apic_base =kvm_get_apic_base (vcpu); -if (irqchip_in_kernel (vcpu->kvm)) { +if (irqchip_in_kernel (vcpu->kvm))memset (sregs->interrupt_bitmap,0 ,sizeof sregs->interrupt_bitmap); - pending_vec = kvm_x86_ops->get_irq (vcpu); -if (pending_vec >=0 ) -set_bit (pending_vec, - (unsigned long *)sregs->interrupt_bitmap); - }else +else memcpy (sregs->interrupt_bitmap, vcpu->arch.irq_pending,sizeof sregs->interrupt_bitmap); + pending_vec = kvm_x86_ops->get_irq (vcpu); +if (pending_vec >=0 ) +set_bit (pending_vec, (unsigned long *)sregs->interrupt_bitmap); +vcpu_put (vcpu);return 0 ;
Signature · 1 line
1.6.0.6
PATCH 8/45 KVM: Get rid of get_irq() callback
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
It just returns pending IRQ vector from the queue for VMX/SVM. Get IRQ directly from the queue before migration and put it back after.
--- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/svm.c | 8 -------- arch/x86/kvm/vmx.c | 8 -------- arch/x86/kvm/x86.c | 12 +++++------- 4 files changed, 5 insertions(+), 24 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -1
@@ -510,7 +510,6 @@ struct kvm_x86_ops {void (*skip_emulated_instruction)(struct kvm_vcpu *vcpu);void (*patch_hypercall)(struct kvm_vcpu *vcpu,unsigned char *hypercall_addr); -int (*get_irq)(struct kvm_vcpu *vcpu);void (*set_irq)(struct kvm_vcpu *vcpu,int vec);void (*set_nmi)(struct kvm_vcpu *vcpu); void (*queue_exception)(struct kvm_vcpu *vcpu, unsigned nr,
arch/x86/kvm/svm.c +0 -8
@@ -966,13 +966,6 @@ static int svm_guest_debug(struct kvm_vcpu *vcpu, struct kvm_guest_debug *dbg)return 0 ; } -static int svm_get_irq (struct kvm_vcpu *vcpu) -{ -if (!vcpu->arch.interrupt.pending) -return -1 ; -return vcpu->arch.interrupt.nr; -} -static void load_host_msrs (struct kvm_vcpu *vcpu) {#ifdef CONFIG_X86_64
@@ -2647,7 +2640,6 @@ static struct kvm_x86_ops svm_x86_ops = { .handle_exit = handle_exit, .skip_emulated_instruction = skip_emulated_instruction, .patch_hypercall = svm_patch_hypercall, - .get_irq = svm_get_irq, .set_irq = svm_set_irq, .set_nmi = svm_inject_nmi, .queue_exception = svm_queue_exception,
arch/x86/kvm/vmx.c +0 -8
@@ -1045,13 +1045,6 @@ static int set_guest_debug(struct kvm_vcpu *vcpu, struct kvm_guest_debug *dbg)return 0 ; } -static int vmx_get_irq (struct kvm_vcpu *vcpu) -{ -if (!vcpu->arch.interrupt.pending) -return -1 ; -return vcpu->arch.interrupt.nr; -} -static __init int cpu_has_kvm_support (void ) {return cpu_has_vmx ();
@@ -3634,7 +3627,6 @@ static struct kvm_x86_ops vmx_x86_ops = { .handle_exit = vmx_handle_exit, .skip_emulated_instruction = skip_emulated_instruction, .patch_hypercall = vmx_patch_hypercall, - .get_irq = vmx_get_irq, .set_irq = vmx_inject_irq, .set_nmi = vmx_inject_nmi, .queue_exception = vmx_queue_exception,
arch/x86/kvm/x86.c +5 -7
@@ -3532,7 +3532,6 @@ int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) {struct descriptor_table dt; -int pending_vec;vcpu_load (vcpu);
@@ -3569,9 +3568,9 @@ int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu,memcpy (sregs->interrupt_bitmap, vcpu->arch.irq_pending,sizeof sregs->interrupt_bitmap); - pending_vec = kvm_x86_ops->get_irq (vcpu); -if (pending_vec >=0 ) -set_bit (pending_vec, (unsigned long *)sregs->interrupt_bitmap); +if (vcpu->arch.interrupt.pending) +set_bit (vcpu->arch.interrupt.nr, + (unsigned long *)sregs->interrupt_bitmap);vcpu_put (vcpu);
@@ -4093,9 +4092,8 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu, max_bits);/* Only pending external irq is handled here */ if (pending_vec < max_bits) { - kvm_x86_ops->set_irq (vcpu, pending_vec); -pr_debug ("Set back pending irq %d\n" , - pending_vec); +kvm_queue_interrupt (vcpu, pending_vec); +pr_debug ("Set back pending irq %d\n" , pending_vec); }kvm_pic_clear_isr_ack (vcpu->kvm); }
Signature · 1 line
1.6.0.6
PATCH 9/45 KVM: Wake up waitqueue before calling get_cpu()
In reply to Avi KivityFrom: Jan Blunck <[email protected]>
This moves the get_cpu() call down to be called after we wake up the waiters. Therefore the waitqueue locks can safely be rt mutex.
--- arch/x86/kvm/x86.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
arch/x86/kvm/x86.c +2 -1
@@ -4565,7 +4565,7 @@ static void vcpu_kick_intr(void *info)void kvm_vcpu_kick (struct kvm_vcpu *vcpu) {int ipi_pcpu = vcpu->cpu; -int cpu =get_cpu (); +int cpu;if (waitqueue_active (&vcpu->wq)) {wake_up_interruptible (&vcpu->wq);
@@ -4575,6 +4575,7 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)* We may be called synchronously with irqs disabled in guest mode, * So need not to call smp_call_function_single() in that case. */ + cpu =get_cpu ();if (vcpu->guest_mode && vcpu->cpu != cpu)smp_call_function_single (ipi_pcpu, vcpu_kick_intr, vcpu,0 );put_cpu ();
Signature · 1 line
1.6.0.6
PATCH 10/45 KVM: Replace get_mt_mask_shift with get_mt_mask
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
Shadow_mt_mask is out of date, now it have only been used as a flag to indicate if TDP enabled. Get rid of it and use tdp_enabled instead.
Also put memory type logical in kvm_x86_ops->get_mt_mask().
--- arch/x86/include/asm/kvm_host.h | 5 +++-- arch/x86/kvm/mmu.c | 21 ++++++--------------- arch/x86/kvm/svm.c | 4 ++-- arch/x86/kvm/vmx.c | 17 ++++++++++++----- arch/x86/kvm/x86.c | 2 +- 5 files changed, 24 insertions(+), 25 deletions(-)
arch/x86/include/asm/kvm_host.h +3 -2
@@ -522,7 +522,7 @@ struct kvm_x86_ops {void (*drop_interrupt_shadow)(struct kvm_vcpu *vcpu);int (*set_tss_addr)(struct kvm *kvm,unsigned int addr);int (*get_tdp_level)(void ); -int (*get_mt_mask_shift)(void ); +u64 (*get_mt_mask)(struct kvm_vcpu *vcpu,gfn_t gfn,bool is_mmio); };extern struct kvm_x86_ops *kvm_x86_ops;
@@ -536,7 +536,7 @@ int kvm_mmu_setup(struct kvm_vcpu *vcpu);void kvm_mmu_set_nonpresent_ptes (u64 trap_pte,u64 notrap_pte);void kvm_mmu_set_base_ptes (u64 base_pte);void kvm_mmu_set_mask_ptes (u64 user_mask,u64 accessed_mask, -u64 dirty_mask,u64 nx_mask,u64 x_mask,u64 mt_mask); +u64 dirty_mask,u64 nx_mask,u64 x_mask);int kvm_mmu_reset_context (struct kvm_vcpu *vcpu);void kvm_mmu_slot_remove_write_access (struct kvm *kvm,int slot);
@@ -550,6 +550,7 @@ int emulator_write_phys(struct kvm_vcpu *vcpu, gpa_t gpa, const void *val, int bytes);int kvm_pv_mmu_op (struct kvm_vcpu *vcpu,unsigned long bytes,gpa_t addr,unsigned long *ret); +u8 kvm_get_guest_memory_type (struct kvm_vcpu *vcpu,gfn_t gfn);extern bool tdp_enabled;
arch/x86/kvm/mmu.c +6 -15
@@ -178,7 +178,6 @@ static u64 __read_mostly shadow_x_mask; /* mutual exclusive with nx_mask */static u64 __read_mostly shadow_user_mask;static u64 __read_mostly shadow_accessed_mask;static u64 __read_mostly shadow_dirty_mask; -static u64 __read_mostly shadow_mt_mask;static inline u64 rsvd_bits (int s,int e) {
@@ -199,14 +198,13 @@ void kvm_mmu_set_base_ptes(u64 base_pte)EXPORT_SYMBOL_GPL (kvm_mmu_set_base_ptes);void kvm_mmu_set_mask_ptes (u64 user_mask,u64 accessed_mask, -u64 dirty_mask,u64 nx_mask,u64 x_mask,u64 mt_mask) +u64 dirty_mask,u64 nx_mask,u64 x_mask) { shadow_user_mask = user_mask; shadow_accessed_mask = accessed_mask; shadow_dirty_mask = dirty_mask; shadow_nx_mask = nx_mask; shadow_x_mask = x_mask; - shadow_mt_mask = mt_mask; }EXPORT_SYMBOL_GPL (kvm_mmu_set_mask_ptes);
@@ -1608,7 +1606,7 @@ static int get_mtrr_type(struct mtrr_state_type *mtrr_state,return mtrr_state->def_type; } -static u8 get_memory_type (struct kvm_vcpu *vcpu,gfn_t gfn) +u8 kvm_get_guest_memory_type (struct kvm_vcpu *vcpu,gfn_t gfn) {u8 mtrr;
@@ -1618,6 +1616,7 @@ static u8 get_memory_type(struct kvm_vcpu *vcpu, gfn_t gfn) mtrr =MTRR_TYPE_WRBACK ;return mtrr; } +EXPORT_SYMBOL_GPL (kvm_get_guest_memory_type);static int kvm_unsync_page (struct kvm_vcpu *vcpu,struct kvm_mmu_page *sp) {
@@ -1670,7 +1669,6 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *shadow_pte, {u64 spte;int ret =0 ; -u64 mt_mask = shadow_mt_mask;/* * We don't set the accessed bit, since we sometimes want to see
@@ -1690,16 +1688,9 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *shadow_pte, spte |= shadow_user_mask;if (largepage) spte |=PT_PAGE_SIZE_MASK ; -if (mt_mask) { -if (!kvm_is_mmio_pfn (pfn)) { - mt_mask =get_memory_type (vcpu, gfn) << - kvm_x86_ops->get_mt_mask_shift (); - mt_mask |=VMX_EPT_IGMT_BIT ; - }else - mt_mask =MTRR_TYPE_UNCACHABLE << - kvm_x86_ops->get_mt_mask_shift (); - spte |= mt_mask; - } +if (tdp_enabled) + spte |= kvm_x86_ops->get_mt_mask (vcpu, gfn, +kvm_is_mmio_pfn (pfn)); spte |= (u64 )pfn <<PAGE_SHIFT ;
arch/x86/kvm/svm.c +2 -2
@@ -2589,7 +2589,7 @@ static int get_npt_level(void)#endif } -static int svm_get_mt_mask_shift (void ) +static u64 svm_get_mt_mask (struct kvm_vcpu *vcpu,gfn_t gfn,bool is_mmio) {return 0 ; }
@@ -2652,7 +2652,7 @@ static struct kvm_x86_ops svm_x86_ops = { .set_tss_addr = svm_set_tss_addr, .get_tdp_level = get_npt_level, - .get_mt_mask_shift = svm_get_mt_mask_shift, + .get_mt_mask = svm_get_mt_mask, };static int __init svm_init (void )
arch/x86/kvm/vmx.c +12 -5
@@ -3577,9 +3577,17 @@ static int get_ept_level(void)return VMX_EPT_DEFAULT_GAW +1 ; } -static int vmx_get_mt_mask_shift (void ) +static u64 vmx_get_mt_mask (struct kvm_vcpu *vcpu,gfn_t gfn,bool is_mmio) { -return VMX_EPT_MT_EPTE_SHIFT ; +u64 ret; + +if (is_mmio) + ret =MTRR_TYPE_UNCACHABLE <<VMX_EPT_MT_EPTE_SHIFT ; +else + ret = (kvm_get_guest_memory_type (vcpu, gfn) << +VMX_EPT_MT_EPTE_SHIFT ) |VMX_EPT_IGMT_BIT ; + +return ret; }static struct kvm_x86_ops vmx_x86_ops = {
@@ -3639,7 +3647,7 @@ static struct kvm_x86_ops vmx_x86_ops = { .set_tss_addr = vmx_set_tss_addr, .get_tdp_level = get_ept_level, - .get_mt_mask_shift = vmx_get_mt_mask_shift, + .get_mt_mask = vmx_get_mt_mask, };static int __init vmx_init (void )
@@ -3698,8 +3706,7 @@ static int __init vmx_init(void)kvm_mmu_set_base_ptes (VMX_EPT_READABLE_MASK |VMX_EPT_WRITABLE_MASK );kvm_mmu_set_mask_ptes (0ull ,0ull ,0ull ,0ull , -VMX_EPT_EXECUTABLE_MASK , -VMX_EPT_DEFAULT_MT <<VMX_EPT_MT_EPTE_SHIFT ); +VMX_EPT_EXECUTABLE_MASK );kvm_enable_tdp (); }else kvm_disable_tdp ();
arch/x86/kvm/x86.c +1 -1
@@ -2768,7 +2768,7 @@ int kvm_arch_init(void *opaque)kvm_mmu_set_nonpresent_ptes (0ull ,0ull );kvm_mmu_set_base_ptes (PT_PRESENT_MASK );kvm_mmu_set_mask_ptes (PT_USER_MASK ,PT_ACCESSED_MASK , -PT_DIRTY_MASK ,PT64_NX_MASK ,0 ,0 ); +PT_DIRTY_MASK ,PT64_NX_MASK ,0 );for_each_possible_cpu (cpu)per_cpu (cpu_tsc_khz, cpu) = tsc_khz;
Signature · 1 line
1.6.0.6
PATCH 11/45 KVM: Enable snooping control for supported hardware
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
Memory aliases with different memory type is a problem for guest. For the guest without assigned device, the memory type of guest memory would always been the same as host(WB); but for the assigned device, some part of memory may be used as DMA and then set to uncacheable memory type(UC/WC), which would be a conflict of host memory type then be a potential issue.
Snooping control can guarantee the cache correctness of memory go through the DMA engine of VT-d.
[avi: fix build on ia64]
--- arch/ia64/include/asm/kvm_host.h | 1 + arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/vmx.c | 19 +++++++++++++++++-- include/linux/kvm_host.h | 3 +++ virt/kvm/iommu.c | 27 ++++++++++++++++++++++++--- 5 files changed, 46 insertions(+), 5 deletions(-)
arch/ia64/include/asm/kvm_host.h +1 -0
@@ -474,6 +474,7 @@ struct kvm_arch {struct list_head assigned_dev_head;struct iommu_domain *iommu_domain; +int iommu_flags;struct hlist_head irq_ack_notifier_list;unsigned long irq_sources_bitmap;
arch/x86/include/asm/kvm_host.h +1 -0
@@ -393,6 +393,7 @@ struct kvm_arch{struct list_head active_mmu_pages;struct list_head assigned_dev_head;struct iommu_domain *iommu_domain; +int iommu_flags;struct kvm_pic *vpic;struct kvm_ioapic *vioapic;struct kvm_pit *vpit;
arch/x86/kvm/vmx.c +17 -2
@@ -3581,11 +3581,26 @@ static u64 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio) {u64 ret; +/* For VT-d and EPT combination +* 1. MMIO: always map as UC +* 2. EPT with VT-d: +* a. VT-d without snooping control feature: can't guarantee the +* result, try to trust guest. +* b. VT-d with snooping control feature: snooping control feature of +* VT-d engine can guarantee the cache correctness. Just set it +* to WB to keep consistent with host. So the same as item 3. +* 3. EPT without VT-d: always map as WB and set IGMT=1 to keep +* consistent with host MTRR +*/ if (is_mmio) ret =MTRR_TYPE_UNCACHABLE <<VMX_EPT_MT_EPTE_SHIFT ; +else if (vcpu->kvm->arch.iommu_domain && + !(vcpu->kvm->arch.iommu_flags &KVM_IOMMU_CACHE_COHERENCY )) + ret =kvm_get_guest_memory_type (vcpu, gfn) << +VMX_EPT_MT_EPTE_SHIFT ;else - ret = (kvm_get_guest_memory_type (vcpu, gfn) << -VMX_EPT_MT_EPTE_SHIFT ) |VMX_EPT_IGMT_BIT ; + ret = (MTRR_TYPE_WRBACK <<VMX_EPT_MT_EPTE_SHIFT ) + |VMX_EPT_IGMT_BIT ;return ret; }
include/linux/kvm_host.h +3 -0
@@ -367,6 +367,9 @@ void kvm_unregister_irq_ack_notifier(struct kvm_irq_ack_notifier *kian);int kvm_request_irq_source_id (struct kvm *kvm);void kvm_free_irq_source_id (struct kvm *kvm,int irq_source_id); +/* For vcpu->arch.iommu_flags */ +#define KVM_IOMMU_CACHE_COHERENCY 0x1 +#ifdef CONFIG_IOMMU_API int kvm_iommu_map_pages (struct kvm *kvm,gfn_t base_gfn,unsigned long npages);
virt/kvm/iommu.c +24 -3
@@ -39,11 +39,16 @@ int kvm_iommu_map_pages(struct kvm *kvm,pfn_t pfn;int i, r =0 ;struct iommu_domain *domain = kvm->arch.iommu_domain; +int flags;/* check if iommu exists and in use */ if (!domain)return 0 ; + flags =IOMMU_READ |IOMMU_WRITE ; +if (kvm->arch.iommu_flags &KVM_IOMMU_CACHE_COHERENCY ) + flags |=IOMMU_CACHE ; +for (i =0 ; i < npages; i++) {/* check if already mapped */ if (iommu_iova_to_phys (domain,gfn_to_gpa (gfn)))
@@ -53,8 +58,7 @@ int kvm_iommu_map_pages(struct kvm *kvm, r =iommu_map_range (domain,gfn_to_gpa (gfn),pfn_to_hpa (pfn), -PAGE_SIZE , -IOMMU_READ |IOMMU_WRITE ); +PAGE_SIZE , flags);if (r) {printk (KERN_ERR "kvm_iommu_map_address:" "iommu failed to map pfn=%lx\n" , pfn);
@@ -88,7 +92,7 @@ int kvm_assign_device(struct kvm *kvm, {struct pci_dev *pdev =NULL ;struct iommu_domain *domain = kvm->arch.iommu_domain; -int r; +int r, last_flags;/* check if iommu exists and in use */ if (!domain)
@@ -107,12 +111,29 @@ int kvm_assign_device(struct kvm *kvm,return r; } + last_flags = kvm->arch.iommu_flags; +if (iommu_domain_has_cap (kvm->arch.iommu_domain, +IOMMU_CAP_CACHE_COHERENCY )) + kvm->arch.iommu_flags |=KVM_IOMMU_CACHE_COHERENCY ; + +/* Check if need to update IOMMU page table for guest memory */ +if ((last_flags ^ kvm->arch.iommu_flags) == +KVM_IOMMU_CACHE_COHERENCY ) { +kvm_iommu_unmap_memslots (kvm); + r =kvm_iommu_map_memslots (kvm); +if (r) + goto out_unmap; + } +printk (KERN_DEBUG "assign device: host bdf = %x:%x:%x\n" , assigned_dev->host_busnr,PCI_SLOT (assigned_dev->host_devfn),PCI_FUNC (assigned_dev->host_devfn));return 0 ; +out_unmap: +kvm_iommu_unmap_memslots (kvm); +return r; } int kvm_deassign_device(struct kvm *kvm,
Signature · 1 line
1.6.0.6
PATCH 12/45 KVM: VMX: Disable VMX when system shutdown
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
Intel TXT(Trusted Execution Technology) required VMX off for all cpu to work when system shutdown.
--- virt/kvm/kvm_main.c | 18 +++++++++--------- 1 files changed, 9 insertions(+), 9 deletions(-)
virt/kvm/kvm_main.c +9 -9
@@ -2391,15 +2391,15 @@ EXPORT_SYMBOL_GPL(kvm_handle_fault_on_reboot);static int kvm_reboot (struct notifier_block *notifier,unsigned long val,void *v) { -if (val ==SYS_RESTART ) { -/* -* Some (well, at least mine) BIOSes hang on reboot if -* in vmx root mode. -*/ -printk (KERN_INFO "kvm: exiting hardware virtualization\n" ); - kvm_rebooting = true; -on_each_cpu (hardware_disable,NULL ,1 ); - } +/* +* Some (well, at least mine) BIOSes hang on reboot if +* in vmx root mode. +* +* And Intel TXT required VMX off for all cpu when system shutdown. +*/ +printk (KERN_INFO "kvm: exiting hardware virtualization\n" ); + kvm_rebooting = true; +on_each_cpu (hardware_disable,NULL ,1 );return NOTIFY_OK ; }
Signature · 1 line
1.6.0.6
PATCH 13/45 KVM: Trivial format fix in setup_routing_entry()
In reply to Avi KivityFrom: Chris Wright <[email protected]>
Remove extra tab.
--- virt/kvm/irq_comm.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
virt/kvm/irq_comm.c +1 -1
@@ -271,7 +271,7 @@ static int setup_routing_entry(struct kvm_kernel_irq_routing_entry *e, delta =8 ;break ;case KVM_IRQCHIP_IOAPIC : - e->set = kvm_set_ioapic_irq; + e->set = kvm_set_ioapic_irq;break ;default : goto out;
Signature · 1 line
1.6.0.6
PATCH 14/45 KVM: fix apic_debug instances
In reply to Avi KivityFrom: Glauber Costa <[email protected]>
Apparently nobody turned this on in a while... setting apic_debug to something compilable, generates some errors. This patch fixes it.
--- arch/x86/kvm/lapic.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/lapic.c +2 -2
@@ -445,7 +445,7 @@ static void apic_send_ipi(struct kvm_lapic *apic)apic_debug ("icr_high 0x%x, icr_low 0x%x, " "short_hand 0x%x, dest 0x%x, trig_mode 0x%x, level 0x%x, " "dest_mode 0x%x, delivery_mode 0x%x, vector 0x%x\n" , - icr_high, icr_low, irq.shorthand, irq.dest, + icr_high, icr_low, irq.shorthand, irq.dest_id, irq.trig_mode, irq.level, irq.dest_mode, irq.delivery_mode, irq.vector);
@@ -560,7 +560,7 @@ static void update_divide_count(struct kvm_lapic *apic) apic->divide_count =0x1 << (tmp2 &0x7 );apic_debug ("timer divide count is 0x%x\n" , - apic->lapic_timer.divide_count); + apic->divide_count); }static void start_apic_timer (struct kvm_lapic *apic)
Signature · 1 line
1.6.0.6
PATCH 15/45 KVM: SVM: Fix cross vendor migration issue in segment segment descriptor
In reply to Avi KivityFrom: Andre Przywara <[email protected]>
On AMD CPUs sometimes the DB bit in the stack segment descriptor is left as 1, although the whole segment has been made unusable. Clear it here to pass an Intel VMX entry check when cross vendor migrating.
--- arch/x86/kvm/svm.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-)
arch/x86/kvm/svm.c +9 -0
@@ -803,6 +803,15 @@ static void svm_get_segment(struct kvm_vcpu *vcpu,if (!var->unusable) var->type |=0x1 ;break ; +case VCPU_SREG_SS : +/* On AMD CPUs sometimes the DB bit in the segment +* descriptor is left as 1, although the whole segment has +* been made unusable. Clear it here to pass an Intel VMX +* entry check when cross vendor migrating. +*/ +if (var->unusable) + var->db =0 ; +break ; } }
Signature · 1 line
1.6.0.6
PATCH 16/45 KVM: SVM: Don't reinject event that caused a task switch
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
If a task switch caused by an event remove it from the event queue. VMX already does that.
--- arch/x86/kvm/svm.c | 23 +++++++++++++++++++++-- 1 files changed, 21 insertions(+), 2 deletions(-)
arch/x86/kvm/svm.c +21 -2
@@ -1811,6 +1811,10 @@ static int task_switch_interception(struct vcpu_svm *svm,int int_type = svm->vmcb->control.exit_int_info &SVM_EXITINTINFO_TYPE_MASK ;int int_vec = svm->vmcb->control.exit_int_info &SVM_EVTINJ_VEC_MASK ; +uint32_t type = + svm->vmcb->control.exit_int_info &SVM_EXITINTINFO_TYPE_MASK ; +uint32_t idt_v = + svm->vmcb->control.exit_int_info &SVM_EXITINTINFO_VALID ; tss_selector = (u16 )svm->vmcb->control.exit_info_1;
@@ -1820,11 +1824,26 @@ static int task_switch_interception(struct vcpu_svm *svm,else if (svm->vmcb->control.exit_info_2 & (1ULL <<SVM_EXITINFOSHIFT_TS_REASON_JMP )) reason =TASK_SWITCH_JMP ; -else if (svm->vmcb->control.exit_int_info &SVM_EXITINTINFO_VALID ) +else if (idt_v) reason =TASK_SWITCH_GATE ;else reason =TASK_SWITCH_CALL ; +if (reason ==TASK_SWITCH_GATE ) { +switch (type) { +case SVM_EXITINTINFO_TYPE_NMI : + svm->vcpu.arch.nmi_injected = false; +break ; +case SVM_EXITINTINFO_TYPE_EXEPT : +kvm_clear_exception_queue (&svm->vcpu); +break ; +case SVM_EXITINTINFO_TYPE_INTR : +kvm_clear_interrupt_queue (&svm->vcpu); +break ; +default : +break ; + } + } if (reason != TASK_SWITCH_GATE || int_type == SVM_EXITINTINFO_TYPE_SOFT ||
@@ -2203,7 +2222,7 @@ static int handle_exit(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)if (is_external_interrupt (svm->vmcb->control.exit_int_info) && exit_code !=SVM_EXIT_EXCP_BASE +PF_VECTOR && - exit_code !=SVM_EXIT_NPF ) + exit_code !=SVM_EXIT_NPF && exit_code !=SVM_EXIT_TASK_SWITCH ) printk(KERN_ERR "%s: unexpected exit_ini_info 0x%x " "exit_code 0x%x\n", __func__, svm->vmcb->control.exit_int_info,
Signature · 1 line
1.6.0.6
PATCH 17/45 KVM: Drop request_nmi from stats
In reply to Avi KivityFrom: Jan Kiszka <[email protected]>
The stats entry request_nmi is no longer used as the related user space interface was dropped. So clean it up.
--- arch/x86/include/asm/kvm_host.h | 1 - arch/x86/kvm/x86.c | 1 - 2 files changed, 0 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -1
@@ -441,7 +441,6 @@ struct kvm_vcpu_stat {u32 halt_exits;u32 halt_wakeup;u32 request_irq_exits; -u32 request_nmi_exits;u32 irq_exits;u32 host_state_reload;u32 efer_reload;
arch/x86/kvm/x86.c +0 -1
@@ -91,7 +91,6 @@ struct kvm_stats_debugfs_item debugfs_entries[] = { { "halt_wakeup", VCPU_STAT(halt_wakeup) }, { "hypercalls", VCPU_STAT(hypercalls) }, { "request_irq", VCPU_STAT(request_irq_exits) }, - { "request_nmi", VCPU_STAT(request_nmi_exits) }, { "irq_exits", VCPU_STAT(irq_exits) }, { "host_state_reload", VCPU_STAT(host_state_reload) }, { "efer_reload", VCPU_STAT(efer_reload) },
Signature · 1 line
1.6.0.6
PATCH 18/45 KVM: Fix cpuid feature misreporting
In reply to Avi KivityMTRR, PAT, MCE, and MCA are all supported (to some extent) but not reported. Vista requires these features, so if userspace relies on kernel cpuid reporting, it loses support for Vista.
--- arch/x86/kvm/x86.c | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +5 -2
@@ -1250,9 +1250,12 @@ static void do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,bit (X86_FEATURE_VME ) |bit (X86_FEATURE_DE ) |bit (X86_FEATURE_PSE ) |bit (X86_FEATURE_TSC ) |bit (X86_FEATURE_MSR ) |bit (X86_FEATURE_PAE ) | +bit (X86_FEATURE_MCE ) |bit (X86_FEATURE_CX8 ) |bit (X86_FEATURE_APIC ) | -bit (X86_FEATURE_SEP ) |bit (X86_FEATURE_PGE ) | -bit (X86_FEATURE_CMOV ) |bit (X86_FEATURE_PSE36 ) | +bit (X86_FEATURE_SEP ) |bit (X86_FEATURE_MTRR ) | +bit (X86_FEATURE_PGE ) |bit (X86_FEATURE_MCA ) | +bit (X86_FEATURE_CMOV ) |bit (X86_FEATURE_PAT ) | +bit (X86_FEATURE_PSE36 ) |bit (X86_FEATURE_CLFLSH ) |bit (X86_FEATURE_MMX ) |bit (X86_FEATURE_FXSR ) |bit (X86_FEATURE_XMM ) |bit (X86_FEATURE_XMM2 ) |bit (X86_FEATURE_SELFSNOOP );
Signature · 1 line
1.6.0.6
PATCH 19/45 KVM: Add AMD cpuid bit: cr8_legacy, abm, misaligned sse, sse4, 3dnow prefetch
In reply to Avi Kivity--- arch/x86/kvm/x86.c | 71 +++++++++++++++++++++++++++------------------------ 1 files changed, 38 insertions(+), 33 deletions(-)
arch/x86/kvm/x86.c +38 -33
@@ -1243,44 +1243,47 @@ static void do_cpuid_1_ent(struct kvm_cpuid_entry2 *entry, u32 function, entry->flags =0 ; } +#define F (x) bit(X86_FEATURE_##x) +static void do_cpuid_ent (struct kvm_cpuid_entry2 *entry,u32 function,u32 index,int *nent,int maxnent) { -const u32 kvm_supported_word0_x86_features =bit (X86_FEATURE_FPU ) | -bit (X86_FEATURE_VME ) |bit (X86_FEATURE_DE ) | -bit (X86_FEATURE_PSE ) |bit (X86_FEATURE_TSC ) | -bit (X86_FEATURE_MSR ) |bit (X86_FEATURE_PAE ) | -bit (X86_FEATURE_MCE ) | -bit (X86_FEATURE_CX8 ) |bit (X86_FEATURE_APIC ) | -bit (X86_FEATURE_SEP ) |bit (X86_FEATURE_MTRR ) | -bit (X86_FEATURE_PGE ) |bit (X86_FEATURE_MCA ) | -bit (X86_FEATURE_CMOV ) |bit (X86_FEATURE_PAT ) | -bit (X86_FEATURE_PSE36 ) | -bit (X86_FEATURE_CLFLSH ) |bit (X86_FEATURE_MMX ) | -bit (X86_FEATURE_FXSR ) |bit (X86_FEATURE_XMM ) | -bit (X86_FEATURE_XMM2 ) |bit (X86_FEATURE_SELFSNOOP ); -const u32 kvm_supported_word1_x86_features =bit (X86_FEATURE_FPU ) | -bit (X86_FEATURE_VME ) |bit (X86_FEATURE_DE ) | -bit (X86_FEATURE_PSE ) |bit (X86_FEATURE_TSC ) | -bit (X86_FEATURE_MSR ) |bit (X86_FEATURE_PAE ) | -bit (X86_FEATURE_CX8 ) |bit (X86_FEATURE_APIC ) | -bit (X86_FEATURE_PGE ) | -bit (X86_FEATURE_CMOV ) |bit (X86_FEATURE_PSE36 ) | -bit (X86_FEATURE_MMX ) |bit (X86_FEATURE_FXSR ) | -bit (X86_FEATURE_SYSCALL ) | - (is_efer_nx () ?bit (X86_FEATURE_NX ) :0 ) | +unsigned f_nx =is_efer_nx () ?F (NX ) :0 ;#ifdef CONFIG_X86_64 -bit (X86_FEATURE_LM ) | +unsigned f_lm =F (LM ); +#else +unsigned f_lm =0 ;#endif -bit (X86_FEATURE_FXSR_OPT ) | -bit (X86_FEATURE_MMXEXT ) | -bit (X86_FEATURE_3DNOWEXT ) | -bit (X86_FEATURE_3DNOW ); -const u32 kvm_supported_word3_x86_features = -bit (X86_FEATURE_XMM3 ) |bit (X86_FEATURE_CX16 ); + +/* cpuid 1.edx */ +const u32 kvm_supported_word0_x86_features = +F (FPU ) |F (VME ) |F (DE ) |F (PSE ) | +F (TSC ) |F (MSR ) |F (PAE ) |F (MCE ) | +F (CX8 ) |F (APIC ) |0 /* Reserved */ |F (SEP ) | +F (MTRR ) |F (PGE ) |F (MCA ) |F (CMOV ) | +F (PAT ) |F (PSE36 ) |0 /* PSN */ |F (CLFLSH ) | +0 /* Reserved, DS, ACPI */ |F (MMX ) | +F (FXSR ) |F (XMM ) |F (XMM2 ) |F (SELFSNOOP ) | +0 /* HTT, TM, Reserved, PBE */ ; +/* cpuid 0x80000001.edx */ +const u32 kvm_supported_word1_x86_features = +F (FPU ) |F (VME ) |F (DE ) |F (PSE ) | +F (TSC ) |F (MSR ) |F (PAE ) |F (MCE ) | +F (CX8 ) |F (APIC ) |0 /* Reserved */ |F (SYSCALL ) | +F (MTRR ) |F (PGE ) |F (MCA ) |F (CMOV ) | +F (PAT ) |F (PSE36 ) |0 /* Reserved */ | + f_nx |0 /* Reserved */ |F (MMXEXT ) |F (MMX ) | +F (FXSR ) |F (FXSR_OPT ) |0 /* GBPAGES */ |0 /* RDTSCP */ | + 0 /* Reserved */ | f_lm | F(3DNOWEXT) | F(3DNOW); +/* cpuid 1.ecx */ +const u32 kvm_supported_word4_x86_features = +F (XMM3 ) |F (CX16 ); +/* cpuid 0x80000001.ecx */ const u32 kvm_supported_word6_x86_features = -bit (X86_FEATURE_LAHF_LM ) |bit (X86_FEATURE_CMP_LEGACY ) | -bit (X86_FEATURE_SVM ); +F (LAHF_LM ) |F (CMP_LEGACY ) |F (SVM ) |0 /* ExtApicSpace */ | +F (CR8_LEGACY ) |F (ABM ) |F (SSE4A ) |F (MISALIGNSSE ) | + F(3DNOWPREFETCH) | 0 /* OSVW */ | 0 /* IBS */ | F(SSE5) | +0 /* SKINIT */ |0 /* WDT */ ;/* all calls to cpuid_count() should be made on the same cpu */ get_cpu ();
@@ -1293,7 +1296,7 @@ static void do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,break ;case 1 : entry->edx &= kvm_supported_word0_x86_features; - entry->ecx &= kvm_supported_word3_x86_features; + entry->ecx &= kvm_supported_word4_x86_features;break ;/* function 2 entries are STATEFUL. That is, repeated cpuid commands * may return different values. This forces us to get_cpu() before
@@ -1355,6 +1358,8 @@ static void do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,put_cpu (); } +#undef F +static int kvm_dev_ioctl_get_supported_cpuid (struct kvm_cpuid2 *cpuid,struct kvm_cpuid_entry2 __user *entries) {
Signature · 1 line
1.6.0.6
PATCH 20/45 x86: Add cpu features MOVBE and POPCNT
In reply to Avi KivityAdd cpu feature bit support for the MOVBE and POPCNT instructions.
--- arch/x86/include/asm/cpufeature.h | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/include/asm/cpufeature.h +2 -0
@@ -115,6 +115,8 @@#define X86_FEATURE_XMM4_1 (4*32+19)/* "sse4_1" SSE-4.1 */ #define X86_FEATURE_XMM4_2 (4*32+20)/* "sse4_2" SSE-4.2 */ #define X86_FEATURE_X2APIC (4*32+21)/* x2APIC */ +#define X86_FEATURE_MOVBE (4*32+22)/* MOVBE instruction */ +#define X86_FEATURE_POPCNT (4*32+23)/* POPCNT instruction */ #define X86_FEATURE_AES (4*32+25)/* AES instructions */ #define X86_FEATURE_XSAVE (4*32+26)/* XSAVE/XRSTOR/XSETBV/XGETBV */ #define X86_FEATURE_OSXSAVE (4*32+27)/* "" XSAVE enabled in the OS */
Signature · 1 line
1.6.0.6
PATCH 21/45 KVM: Update cpuid 1.ecx reporting
In reply to Avi Kivity--- arch/x86/kvm/x86.c | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-)
arch/x86/kvm/x86.c +7 -1
@@ -1277,7 +1277,13 @@ static void do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function, 0 /* Reserved */ | f_lm | F(3DNOWEXT) | F(3DNOW);/* cpuid 1.ecx */ const u32 kvm_supported_word4_x86_features = -F (XMM3 ) |F (CX16 ); +F (XMM3 ) |0 /* Reserved, DTES64, MONITOR */ | +0 /* DS-CPL, VMX, SMX, EST */ | +0 /* TM2 */ |F (SSSE3 ) |0 /* CNXT-ID */ |0 /* Reserved */ | +0 /* Reserved */ |F (CX16 ) |0 /* xTPR Update, PDCM */ | +0 /* Reserved, DCA */ |F (XMM4_1 ) | +F (XMM4_2 ) |0 /* x2APIC */ |F (MOVBE ) |F (POPCNT ) | +0 /* Reserved, XSAVE, OSXSAVE */ ;/* cpuid 0x80000001.ecx */ const u32 kvm_supported_word6_x86_features = F(LAHF_LM) | F(CMP_LEGACY) | F(SVM) | 0 /* ExtApicSpace */ |
Signature · 1 line
1.6.0.6
PATCH 22/45 KVM: use smp_send_reschedule in kvm_vcpu_kick
In reply to Avi KivityFrom: Marcelo Tosatti <[email protected]>
KVM uses a function call IPI to cause the exit of a guest running on a physical cpu. For virtual interrupt notification there is no need to wait on IPI receival, or to execute any function.
This is exactly what the reschedule IPI does, without the overhead of function IPI. So use it instead of smp_call_function_single in kvm_vcpu_kick.
Also change the "guest_mode" variable to a bit in vcpu->requests, and use that to collapse multiple IPI's that would be issued between the first one and zeroing of guest mode.
This allows kvm_vcpu_kick to called with interrupts disabled.
--- arch/ia64/kernel/irq_ia64.c | 3 +++ arch/ia64/kvm/kvm-ia64.c | 22 ++++++++-------------- arch/x86/kernel/smp.c | 3 +++ arch/x86/kvm/x86.c | 36 +++++++++++------------------------- include/linux/kvm_host.h | 2 +- 5 files changed, 26 insertions(+), 40 deletions(-)
arch/ia64/kernel/irq_ia64.c +3 -0
@@ -610,6 +610,9 @@ static struct irqaction ipi_irqaction = { .name = "IPI" }; +/* +* KVM uses this interrupt to force a cpu out of guest mode +*/ static struct irqaction resched_irqaction = { .handler = dummy_handler, .flags =IRQF_DISABLED ,
arch/ia64/kvm/kvm-ia64.c +8 -14
@@ -668,7 +668,7 @@ again: host_ctx =kvm_get_host_context (vcpu); guest_ctx =kvm_get_guest_context (vcpu); - vcpu->guest_mode =1 ; +clear_bit (KVM_REQ_KICK , &vcpu->requests); r =kvm_vcpu_pre_transition (vcpu);if (r <0 )
@@ -685,7 +685,7 @@ again:kvm_vcpu_post_transition (vcpu); vcpu->arch.launched =1 ; - vcpu->guest_mode =0 ; +set_bit (KVM_REQ_KICK , &vcpu->requests);local_irq_enable ();/*
@@ -1879,24 +1879,18 @@ void kvm_arch_hardware_unsetup(void) { } -static void vcpu_kick_intr (void *info) -{ -#ifdef DEBUG -struct kvm_vcpu *vcpu = (struct kvm_vcpu *)info; -printk (KERN_DEBUG "vcpu_kick_intr %p \n" , vcpu); -#endif -} -void kvm_vcpu_kick (struct kvm_vcpu *vcpu) { -int ipi_pcpu = vcpu->cpu; -int cpu =get_cpu (); +int me; +int cpu = vcpu->cpu;if (waitqueue_active (&vcpu->wq))wake_up_interruptible (&vcpu->wq); -if (vcpu->guest_mode && cpu != ipi_pcpu) -smp_call_function_single (ipi_pcpu, vcpu_kick_intr, vcpu,0 ); + me =get_cpu (); +if (cpu != me && (unsigned ) cpu < nr_cpu_ids &&cpu_online (cpu)) +if (!test_and_set_bit (KVM_REQ_KICK , &vcpu->requests)) +smp_send_reschedule (cpu);put_cpu (); }
arch/x86/kernel/smp.c +3 -0
@@ -172,6 +172,9 @@ void smp_reschedule_interrupt(struct pt_regs *regs) {ack_APIC_irq ();inc_irq_stat (irq_resched_count); +/* +* KVM uses this interrupt to force a cpu out of guest mode +*/ }void smp_call_function_interrupt (struct pt_regs *regs)
arch/x86/kvm/x86.c +11 -25
@@ -3226,6 +3226,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)local_irq_disable (); +clear_bit (KVM_REQ_KICK , &vcpu->requests); +smp_mb__after_clear_bit (); +if (vcpu->requests ||need_resched () ||signal_pending (current)) {local_irq_enable ();preempt_enable ();
@@ -3233,13 +3236,6 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) goto out; } - vcpu->guest_mode =1 ; -/* -* Make sure that guest_mode assignment won't happen after -* testing the pending IRQ vector bitmap. -*/ -smp_wmb (); -if (vcpu->arch.exception.pending)__queue_exception (vcpu);else
@@ -3284,7 +3280,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)set_debugreg (vcpu->arch.host_dr6,6 );set_debugreg (vcpu->arch.host_dr7,7 ); - vcpu->guest_mode =0 ; +set_bit (KVM_REQ_KICK , &vcpu->requests);local_irq_enable (); ++vcpu->stat.exits;
@@ -4567,30 +4563,20 @@ int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu) || vcpu->arch.nmi_pending; } -static void vcpu_kick_intr (void *info) -{ -#ifdef DEBUG -struct kvm_vcpu *vcpu = (struct kvm_vcpu *)info; -printk (KERN_DEBUG "vcpu_kick_intr %p \n" , vcpu); -#endif -} -void kvm_vcpu_kick (struct kvm_vcpu *vcpu) { -int ipi_pcpu = vcpu->cpu; -int cpu; +int me; +int cpu = vcpu->cpu;if (waitqueue_active (&vcpu->wq)) {wake_up_interruptible (&vcpu->wq); ++vcpu->stat.halt_wakeup; } -/* -* We may be called synchronously with irqs disabled in guest mode, -* So need not to call smp_call_function_single() in that case. -*/ - cpu =get_cpu (); -if (vcpu->guest_mode && vcpu->cpu != cpu) -smp_call_function_single (ipi_pcpu, vcpu_kick_intr, vcpu,0 ); + + me =get_cpu (); +if (cpu != me && (unsigned )cpu < nr_cpu_ids &&cpu_online (cpu)) +if (!test_and_set_bit (KVM_REQ_KICK , &vcpu->requests)) +smp_send_reschedule (cpu);put_cpu (); }
include/linux/kvm_host.h +1 -1
@@ -38,6 +38,7 @@#define KVM_REQ_UNHALT 6#define KVM_REQ_MMU_SYNC 7#define KVM_REQ_KVMCLOCK_UPDATE 8 +#define KVM_REQ_KICK 9#define KVM_USERSPACE_IRQ_SOURCE_ID 0
@@ -72,7 +73,6 @@ struct kvm_vcpu {struct mutex mutex;int cpu;struct kvm_run *run; -int guest_mode;unsigned long requests;unsigned long guest_debug;int fpu_active;
Signature · 1 line
1.6.0.6
PATCH 23/45 KVM: protect assigned dev workqueue, int handler and irq acker
In reply to Avi KivityFrom: Marcelo Tosatti <[email protected]>
kvm_assigned_dev_ack_irq is vulnerable to a race condition with the interrupt handler function. It does:
if (dev->host_irq_disabled) {
enable_irq(dev->host_irq);
dev->host_irq_disabled = false;
}If an interrupt triggers before the host->dev_irq_disabled assignment, it will disable the interrupt and set dev->host_irq_disabled to true.
On return to kvm_assigned_dev_ack_irq, dev->host_irq_disabled is set to false, and the next kvm_assigned_dev_ack_irq call will fail to reenable it.
Other than that, having the interrupt handler and work handlers run in parallel sounds like asking for trouble (could not spot any obvious problem, but better not have to, its fragile).
--- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 13 ++++++++++++- 2 files changed, 13 insertions(+), 1 deletions(-)
include/linux/kvm_host.h +1 -0
@@ -345,6 +345,7 @@ struct kvm_assigned_dev_kernel {int flags;struct pci_dev *dev;struct kvm *kvm; +spinlock_t assigned_dev_lock; };struct kvm_irq_mask_notifier {
virt/kvm/kvm_main.c +12 -1
@@ -42,6 +42,7 @@#include <linux/mman.h> #include <linux/swap.h> #include <linux/bitops.h> +#include <linux/spinlock.h> #include <asm/processor.h> #include <asm/io.h>
@@ -130,6 +131,7 @@ static void kvm_assigned_dev_interrupt_work_handler(struct work_struct *work)* finer-grained lock, update this */ mutex_lock (&kvm->lock); +spin_lock_irq (&assigned_dev->assigned_dev_lock);if (assigned_dev->irq_requested_type &KVM_DEV_IRQ_HOST_MSIX ) {struct kvm_guest_msix_entry *guest_entries = assigned_dev->guest_msix_entries;
@@ -156,18 +158,21 @@ static void kvm_assigned_dev_interrupt_work_handler(struct work_struct *work) } } +spin_unlock_irq (&assigned_dev->assigned_dev_lock);mutex_unlock (&assigned_dev->kvm->lock); }static irqreturn_t kvm_assigned_dev_intr (int irq,void *dev_id) { +unsigned long flags;struct kvm_assigned_dev_kernel *assigned_dev = (struct kvm_assigned_dev_kernel *) dev_id; +spin_lock_irqsave (&assigned_dev->assigned_dev_lock, flags);if (assigned_dev->irq_requested_type &KVM_DEV_IRQ_HOST_MSIX ) {int index =find_index_from_host_irq (assigned_dev, irq);if (index <0 ) -return IRQ_HANDLED ; + goto out; assigned_dev->guest_msix_entries[index].flags |=KVM_ASSIGNED_MSIX_PENDING ; }
@@ -177,6 +182,8 @@ static irqreturn_t kvm_assigned_dev_intr(int irq, void *dev_id)disable_irq_nosync (irq); assigned_dev->host_irq_disabled = true; +out: +spin_unlock_irqrestore (&assigned_dev->assigned_dev_lock, flags);return IRQ_HANDLED ; }
@@ -184,6 +191,7 @@ static irqreturn_t kvm_assigned_dev_intr(int irq, void *dev_id)static void kvm_assigned_dev_ack_irq (struct kvm_irq_ack_notifier *kian) {struct kvm_assigned_dev_kernel *dev; +unsigned long flags;if (kian->gsi ==-1 )return ;
@@ -196,10 +204,12 @@ static void kvm_assigned_dev_ack_irq(struct kvm_irq_ack_notifier *kian)/* The guest irq may be shared so this ack may be * from another device. */ +spin_lock_irqsave (&dev->assigned_dev_lock, flags);if (dev->host_irq_disabled) {enable_irq (dev->host_irq); dev->host_irq_disabled = false; } +spin_unlock_irqrestore (&dev->assigned_dev_lock, flags); } static void deassign_guest_irq(struct kvm *kvm,
@@ -615,6 +625,7 @@ static int kvm_vm_ioctl_assign_device(struct kvm *kvm, match->host_devfn = assigned_dev->devfn; match->flags = assigned_dev->flags; match->dev = dev; +spin_lock_init (&match->assigned_dev_lock); match->irq_source_id =-1 ; match->kvm = kvm; match->ack_notifier.irq_acked = kvm_assigned_dev_ack_irq;
Signature · 1 line
1.6.0.6
PATCH 24/45 KVM: Replace ->drop_interrupt_shadow() by ->set_interrupt_shadow()
In reply to Avi KivityFrom: Glauber Costa <[email protected]>
This patch replaces drop_interrupt_shadow with the more general set_interrupt_shadow, that can either drop or raise it, depending on its parameter. It also adds ->get_interrupt_shadow() for future use.
--- arch/x86/include/asm/kvm_host.h | 3 +- arch/x86/include/asm/kvm_x86_emulate.h | 3 ++ arch/x86/kvm/svm.c | 32 +++++++++++++++----- arch/x86/kvm/vmx.c | 49 +++++++++++++++++++++---------- arch/x86/kvm/x86.c | 2 +- 5 files changed, 63 insertions(+), 26 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -1
@@ -508,6 +508,8 @@ struct kvm_x86_ops {void (*run)(struct kvm_vcpu *vcpu,struct kvm_run *run);int (*handle_exit)(struct kvm_run *run,struct kvm_vcpu *vcpu);void (*skip_emulated_instruction)(struct kvm_vcpu *vcpu); +void (*set_interrupt_shadow)(struct kvm_vcpu *vcpu,int mask); +u32 (*get_interrupt_shadow)(struct kvm_vcpu *vcpu,int mask);void (*patch_hypercall)(struct kvm_vcpu *vcpu,unsigned char *hypercall_addr);void (*set_irq)(struct kvm_vcpu *vcpu,int vec);
@@ -519,7 +521,6 @@ struct kvm_x86_ops {void (*enable_nmi_window)(struct kvm_vcpu *vcpu);void (*enable_irq_window)(struct kvm_vcpu *vcpu);void (*update_cr8_intercept)(struct kvm_vcpu *vcpu,int tpr,int irr); -void (*drop_interrupt_shadow)(struct kvm_vcpu *vcpu);int (*set_tss_addr)(struct kvm *kvm,unsigned int addr);int (*get_tdp_level)(void );u64 (*get_mt_mask)(struct kvm_vcpu *vcpu,gfn_t gfn,bool is_mmio);
arch/x86/include/asm/kvm_x86_emulate.h +3 -0
@@ -143,6 +143,9 @@ struct decode_cache {struct fetch_cache fetch; }; +#define X86_SHADOW_INT_MOV_SS 1 +#define X86_SHADOW_INT_STI 2 +struct x86_emulate_ctxt {/* Register state before/after emulation. */ struct kvm_vcpu *vcpu;
arch/x86/kvm/svm.c +24 -8
@@ -202,6 +202,27 @@ static int is_external_interrupt(u32 info)return info == (SVM_EVTINJ_VALID |SVM_EVTINJ_TYPE_INTR ); } +static u32 svm_get_interrupt_shadow (struct kvm_vcpu *vcpu,int mask) +{ +struct vcpu_svm *svm =to_svm (vcpu); +u32 ret =0 ; + +if (svm->vmcb->control.int_state &SVM_INTERRUPT_SHADOW_MASK ) + ret |=X86_SHADOW_INT_STI |X86_SHADOW_INT_MOV_SS ; +return ret & mask; +} + +static void svm_set_interrupt_shadow (struct kvm_vcpu *vcpu,int mask) +{ +struct vcpu_svm *svm =to_svm (vcpu); + +if (mask ==0 ) + svm->vmcb->control.int_state &= ~SVM_INTERRUPT_SHADOW_MASK ; +else + svm->vmcb->control.int_state |=SVM_INTERRUPT_SHADOW_MASK ; + +} +static void skip_emulated_instruction (struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu);
@@ -215,7 +236,7 @@ static void skip_emulated_instruction(struct kvm_vcpu *vcpu) __func__, kvm_rip_read(vcpu), svm->next_rip);kvm_rip_write (vcpu, svm->next_rip); - svm->vmcb->control.int_state &= ~SVM_INTERRUPT_SHADOW_MASK ; +svm_set_interrupt_shadow (vcpu,0 ); }static int has_svm (void )
@@ -2259,12 +2280,6 @@ static void pre_svm_run(struct vcpu_svm *svm)new_asid (svm, svm_data); } -static void svm_drop_interrupt_shadow (struct kvm_vcpu *vcpu) -{ -struct vcpu_svm *svm =to_svm (vcpu); - svm->vmcb->control.int_state &= ~SVM_INTERRUPT_SHADOW_MASK ; -} -static void svm_inject_nmi (struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu);
@@ -2667,6 +2682,8 @@ static struct kvm_x86_ops svm_x86_ops = { .run = svm_vcpu_run, .handle_exit = handle_exit, .skip_emulated_instruction = skip_emulated_instruction, + .set_interrupt_shadow = svm_set_interrupt_shadow, + .get_interrupt_shadow = svm_get_interrupt_shadow, .patch_hypercall = svm_patch_hypercall, .set_irq = svm_set_irq, .set_nmi = svm_inject_nmi,
@@ -2676,7 +2693,6 @@ static struct kvm_x86_ops svm_x86_ops = { .enable_nmi_window = enable_nmi_window, .enable_irq_window = enable_irq_window, .update_cr8_intercept = update_cr8_intercept, - .drop_interrupt_shadow = svm_drop_interrupt_shadow, .set_tss_addr = svm_set_tss_addr, .get_tdp_level = get_npt_level,
arch/x86/kvm/vmx.c +33 -16
81 lines not on this page · raw message
arch/x86/kvm/x86.c +1 -1
@@ -3174,7 +3174,7 @@ static void inject_pending_irq(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) kvm_run->request_interrupt_window;if (vcpu->guest_debug &KVM_GUESTDBG_SINGLESTEP ) - kvm_x86_ops->drop_interrupt_shadow (vcpu); + kvm_x86_ops->set_interrupt_shadow (vcpu,0 );inject_irq (vcpu);
Signature · 1 line
1.6.0.6
PATCH 25/45 KVM: Deal with interrupt shadow state for emulated instructions
In reply to Avi KivityFrom: Glauber Costa <[email protected]>
We currently unblock shadow interrupt state when we skip an instruction, but failing to do so when we actually emulate one. This blocks interrupts in key instruction blocks, in particular sti; hlt; sequences
If the instruction emulated is an sti, we have to block shadow interrupts. The same goes for mov ss. pop ss also needs it, but we don't currently emulate it.
Without this patch, I cannot boot gpxe option roms at vmx machines. This is described at https://bugzilla.redhat.com/show_bug.cgi?id=494469
--- arch/x86/include/asm/kvm_x86_emulate.h | 3 +++ arch/x86/kvm/x86.c | 6 +++++- arch/x86/kvm/x86_emulate.c | 20 ++++++++++++++++++++ 3 files changed, 28 insertions(+), 1 deletions(-)
arch/x86/include/asm/kvm_x86_emulate.h +3 -0
@@ -155,6 +155,9 @@ struct x86_emulate_ctxt {int mode;u32 cs_base; +/* interruptibility state, as a result of execution of STI or MOV SS */ +int interruptibility; +/* decode cache */ struct decode_cache decode; };
arch/x86/kvm/x86.c +5 -1
@@ -2375,7 +2375,7 @@ int emulate_instruction(struct kvm_vcpu *vcpu,u16 error_code, int emulation_type) { - int r; + int r, shadow_mask;struct decode_cache *c;kvm_clear_exception_queue (vcpu);
@@ -2429,6 +2429,10 @@ int emulate_instruction(struct kvm_vcpu *vcpu, } r =x86_emulate_insn (&vcpu->arch.emulate_ctxt, &emulate_ops); + shadow_mask = vcpu->arch.emulate_ctxt.interruptibility; + +if (r ==0 ) + kvm_x86_ops->set_interrupt_shadow (vcpu, shadow_mask);if (vcpu->arch.pio.string)return EMULATE_DO_MMIO ;
arch/x86/kvm/x86_emulate.c +20 -0
@@ -1361,6 +1361,20 @@ static inline int writeback(struct x86_emulate_ctxt *ctxt,return 0 ; } +void toggle_interruptibility (struct x86_emulate_ctxt *ctxt,u32 mask) +{ +u32 int_shadow = kvm_x86_ops->get_interrupt_shadow (ctxt->vcpu, mask); +/* +* an sti; sti; sequence only disable interrupts for the first +* instruction. So, if the last instruction, be it emulated or +* not, left the system with the INT_STI flag enabled, it +* means that the last instruction is an sti. We should not +* leave the flag on in this case. The same goes for mov ss +*/ +if (!(int_shadow & mask)) + ctxt->interruptibility = mask; +} +int x86_emulate_insn (struct x86_emulate_ctxt *ctxt,struct x86_emulate_ops *ops) {
@@ -1372,6 +1386,8 @@ x86_emulate_insn(struct x86_emulate_ctxt *ctxt, struct x86_emulate_ops *ops)int io_dir_in;int rc =0 ; + ctxt->interruptibility =0 ; +/* Shadow copy of register state. Committed on successful emulation. * NOTE: we can copy them from vcpu as x86_decode_insn() doesn't * modify them.
@@ -1618,6 +1634,9 @@ special_insn:int err; sel = c->src.val; +if (c->modrm_reg ==VCPU_SREG_SS ) +toggle_interruptibility (ctxt,X86_SHADOW_INT_MOV_SS ); +if (c->modrm_reg <=5 ) { type_bits = (c->modrm_reg ==1 ) ?9 :1 ; err = kvm_load_segment_descriptor(ctxt->vcpu, sel,
@@ -1847,6 +1866,7 @@ special_insn: c->dst.type =OP_NONE ;/* Disable writeback. */ break ;case 0xfb :/* sti */ +toggle_interruptibility (ctxt,X86_SHADOW_INT_STI ); ctxt->eflags |=X86_EFLAGS_IF ; c->dst.type =OP_NONE ;/* Disable writeback. */ break ;
Signature · 1 line
1.6.0.6
PATCH 26/45 KVM: MMU: protect kvm_mmu_change_mmu_pages with mmu_lock
In reply to Avi KivityFrom: Marcelo Tosatti <[email protected]>
kvm_handle_hva, called by MMU notifiers, manipulates mmu data only with the protection of mmu_lock.
Update kvm_mmu_change_mmu_pages callers to take mmu_lock, thus protecting against kvm_handle_hva.
--- arch/x86/kvm/mmu.c | 2 -- arch/x86/kvm/x86.c | 6 ++++++ 2 files changed, 6 insertions(+), 2 deletions(-)
arch/x86/kvm/mmu.c +0 -2
@@ -2723,7 +2723,6 @@ void kvm_mmu_slot_remove_write_access(struct kvm *kvm, int slot) {struct kvm_mmu_page *sp; -spin_lock (&kvm->mmu_lock);list_for_each_entry (sp, &kvm->arch.active_mmu_pages, link) {int i;u64 *pt;
@@ -2738,7 +2737,6 @@ void kvm_mmu_slot_remove_write_access(struct kvm *kvm, int slot) pt[i] &= ~PT_WRITABLE_MASK ; }kvm_flush_remote_tlbs (kvm); -spin_unlock (&kvm->mmu_lock); }void kvm_mmu_zap_all (struct kvm *kvm)
arch/x86/kvm/x86.c +6 -0
@@ -1621,10 +1621,12 @@ static int kvm_vm_ioctl_set_nr_mmu_pages(struct kvm *kvm,return -EINVAL ;down_write (&kvm->slots_lock); +spin_lock (&kvm->mmu_lock);kvm_mmu_change_mmu_pages (kvm, kvm_nr_mmu_pages); kvm->arch.n_requested_mmu_pages = kvm_nr_mmu_pages; +spin_unlock (&kvm->mmu_lock);up_write (&kvm->slots_lock);return 0 ; }
@@ -1800,7 +1802,9 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm,/* If nothing is dirty, don't bother messing with page tables. */ if (is_dirty) { +spin_lock (&kvm->mmu_lock);kvm_mmu_slot_remove_write_access (kvm, log->slot); +spin_unlock (&kvm->mmu_lock);kvm_flush_remote_tlbs (kvm); memslot = &kvm->memslots[log->slot]; n =ALIGN (memslot->npages,BITS_PER_LONG ) /8 ;
@@ -4544,12 +4548,14 @@ int kvm_arch_set_memory_region(struct kvm *kvm, } } +spin_lock (&kvm->mmu_lock);if (!kvm->arch.n_requested_mmu_pages) {unsigned int nr_mmu_pages =kvm_mmu_calculate_mmu_pages (kvm);kvm_mmu_change_mmu_pages (kvm, nr_mmu_pages); }kvm_mmu_slot_remove_write_access (kvm, mem->slot); +spin_unlock (&kvm->mmu_lock);kvm_flush_remote_tlbs (kvm);return 0 ;
Signature · 1 line
1.6.0.6
PATCH 27/45 KVM: take mmu_lock when updating a deleted slot
In reply to Avi KivityFrom: Marcelo Tosatti <[email protected]>
kvm_handle_hva relies on mmu_lock protection to safely access the memslot structures.
--- virt/kvm/kvm_main.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
virt/kvm/kvm_main.c +2 -0
@@ -1210,8 +1210,10 @@ int __kvm_set_memory_region(struct kvm *kvm,kvm_free_physmem_slot (&old, npages ? &new :NULL );/* Slot deletion case: we have to update the current slot */ +spin_lock (&kvm->mmu_lock);if (!npages) *memslot = old; +spin_unlock (&kvm->mmu_lock);#ifdef CONFIG_DMAR /* map the pages in iommu page table */ r =kvm_iommu_map_pages (kvm, base_gfn, npages);
Signature · 1 line
1.6.0.6
PATCH 28/45 KVM: x86: check for cr3 validity in mmu_alloc_roots
In reply to Avi KivityFrom: Marcelo Tosatti <[email protected]>
Verify the cr3 address stored in vcpu->arch.cr3 points to an existant memslot. If not, inject a triple fault.
--- arch/x86/kvm/mmu.c | 27 +++++++++++++++++++++++---- arch/x86/kvm/x86.c | 1 + 2 files changed, 24 insertions(+), 4 deletions(-)
arch/x86/kvm/mmu.c +23 -4
@@ -1912,7 +1912,19 @@ static void mmu_free_roots(struct kvm_vcpu *vcpu) vcpu->arch.mmu.root_hpa =INVALID_PAGE ; } -static void mmu_alloc_roots (struct kvm_vcpu *vcpu) +static int mmu_check_root (struct kvm_vcpu *vcpu,gfn_t root_gfn) +{ +int ret =0 ; + +if (!kvm_is_visible_gfn (vcpu->kvm, root_gfn)) { +set_bit (KVM_REQ_TRIPLE_FAULT , &vcpu->requests); + ret =1 ; + } + +return ret; +} + +static int mmu_alloc_roots (struct kvm_vcpu *vcpu) {int i;gfn_t root_gfn;
@@ -1927,13 +1939,15 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu)ASSERT (!VALID_PAGE (root));if (tdp_enabled) direct =1 ; +if (mmu_check_root (vcpu, root_gfn)) +return 1 ; sp =kvm_mmu_get_page (vcpu, root_gfn,0 ,PT64_ROOT_LEVEL , direct,ACC_ALL ,NULL ); root =__pa (sp->spt); ++sp->root_count; vcpu->arch.mmu.root_hpa = root; -return ; +return 0 ; } direct = !is_paging (vcpu);if (tdp_enabled)
@@ -1950,6 +1964,8 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu) root_gfn = vcpu->arch.pdptrs[i] >>PAGE_SHIFT ; }else if (vcpu->arch.mmu.root_level ==0 ) root_gfn =0 ; +if (mmu_check_root (vcpu, root_gfn)) +return 1 ; sp =kvm_mmu_get_page (vcpu, root_gfn, i <<30 ,PT32_ROOT_LEVEL , direct,ACC_ALL ,NULL );
@@ -1958,6 +1974,7 @@ static void mmu_alloc_roots(struct kvm_vcpu *vcpu) vcpu->arch.mmu.pae_root[i] = root |PT_PRESENT_MASK ; } vcpu->arch.mmu.root_hpa =__pa (vcpu->arch.mmu.pae_root); +return 0 ; }static void mmu_sync_roots (struct kvm_vcpu *vcpu)
@@ -1976,7 +1993,7 @@ static void mmu_sync_roots(struct kvm_vcpu *vcpu)for (i =0 ; i <4 ; ++i) {hpa_t root = vcpu->arch.mmu.pae_root[i]; -if (root) { +if (root &&VALID_PAGE (root)) { root &=PT64_BASE_ADDR_MASK ; sp =page_header (root);mmu_sync_children (vcpu, sp);
@@ -2311,9 +2328,11 @@ int kvm_mmu_load(struct kvm_vcpu *vcpu) goto out;spin_lock (&vcpu->kvm->mmu_lock);kvm_mmu_free_some_pages (vcpu); -mmu_alloc_roots (vcpu); + r =mmu_alloc_roots (vcpu);mmu_sync_roots (vcpu);spin_unlock (&vcpu->kvm->mmu_lock); +if (r) + goto out; kvm_x86_ops->set_cr3 (vcpu, vcpu->arch.mmu.root_hpa);kvm_mmu_flush_tlb (vcpu); out:
arch/x86/kvm/x86.c +1 -0
@@ -4564,6 +4564,7 @@ int kvm_arch_set_memory_region(struct kvm *kvm,void kvm_arch_flush_shadow (struct kvm *kvm) {kvm_mmu_zap_all (kvm); +kvm_reload_remote_mmus (kvm); }int kvm_arch_vcpu_runnable (struct kvm_vcpu *vcpu)
Signature · 1 line
1.6.0.6
PATCH 29/45 KVM: Expand on "help" info to specify kvm intel and amd module names
In reply to Avi KivityFrom: Robert P. J. Day <[email protected]>
--- arch/x86/kvm/Kconfig | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
arch/x86/kvm/Kconfig +6 -0
@@ -50,6 +50,9 @@ config KVM_INTEL Provides support for KVM on Intel processors equipped with the VT extensions. + To compile this as a module, choose M here: the module + will be called kvm-intel. + config KVM_AMD tristate "KVM for AMD processors support" depends on KVM
@@ -57,6 +60,9 @@ config KVM_AMD Provides support for KVM on AMD processors equipped with the AMD-V (SVM) extensions. + To compile this as a module, choose M here: the module + will be called kvm-amd. + config KVM_TRACE bool "KVM trace support" depends on KVM && SYSFS
Signature · 1 line
1.6.0.6
PATCH 30/45 KVM: s390: Fix memory slot versus run - v3
In reply to Avi KivityFrom: Carsten Otte <[email protected]>
This patch fixes an incorrectness in the kvm backend for s390. In case virtual cpus are being created before the corresponding memory slot is being registered, we need to update the sie control blocks for the virtual cpus.
*updates in v3*
In consideration of the s390 memslot constraints locking was changed to trylock. These locks should never be held, as vcpu's can't run without the single memslot we just assign when running this code. To ensure this never deadlocks in case other code changes the code uses trylocks and bail out if it can't get all locks.
Additionally most of the discussed special conditions for s390 like only one memslot and no user_alloc are now checked for validity in kvm_arch_set_memory_region.
--- arch/s390/kvm/kvm-s390.c | 36 +++++++++++++++++++++++++++++++----- 1 files changed, 31 insertions(+), 5 deletions(-)
arch/s390/kvm/kvm-s390.c +31 -5
@@ -657,6 +657,8 @@ int kvm_arch_set_memory_region(struct kvm *kvm, struct kvm_memory_slot old, int user_alloc) { +int i; +/* A few sanity checks. We can have exactly one memory slot which has to start at guest virtual zero and which has to be located at a page boundary in userland and which has to end at a page boundary.
@@ -664,7 +666,7 @@ int kvm_arch_set_memory_region(struct kvm *kvm,vmas. It is okay to mmap() and munmap() stuff in this slot after doing this call at any time */ -if (mem->slot) +if (mem->slot || kvm->arch.guest_memsize)return -EINVAL ;if (mem->guest_phys_addr)
@@ -676,15 +678,39 @@ int kvm_arch_set_memory_region(struct kvm *kvm,if (mem->memory_size & (PAGE_SIZE -1 ))return -EINVAL ; +if (!user_alloc) +return -EINVAL ; + +/* lock all vcpus */ +for (i =0 ; i <KVM_MAX_VCPUS ; ++i) { +if (!kvm->vcpus[i]) +continue ; +if (!mutex_trylock (&kvm->vcpus[i]->mutex)) + goto fail_out; + } + kvm->arch.guest_origin = mem->userspace_addr; kvm->arch.guest_memsize = mem->memory_size; -/* FIXME: we do want to interrupt running CPUs and update their memory -configuration now to avoid race conditions. But hey, changing the -memory layout while virtual CPUs are running is usually bad -programming practice. */ +/* update sie control blocks, and unlock all vcpus */ +for (i =0 ; i <KVM_MAX_VCPUS ; ++i) { +if (kvm->vcpus[i]) { + kvm->vcpus[i]->arch.sie_block->gmsor = + kvm->arch.guest_origin; + kvm->vcpus[i]->arch.sie_block->gmslm = + kvm->arch.guest_memsize + + kvm->arch.guest_origin + +VIRTIODESCSPACE -1ul ; +mutex_unlock (&kvm->vcpus[i]->mutex); + } + }return 0 ; + +fail_out: +for (; i >=0 ; i--) +mutex_unlock (&kvm->vcpus[i]->mutex); +return -EINVAL ; }void kvm_arch_flush_shadow (struct kvm *kvm)
Signature · 1 line
1.6.0.6
PATCH 31/45 KVM: s390: use hrtimer for clock wakeup from idle - v2
In reply to Avi KivityFrom: Christian Borntraeger <[email protected]>
This patch reworks the s390 clock comparator wakeup to hrtimer. The clock comparator is a per-cpu value that is compared against the TOD clock. If ckc <= TOD an external interrupt 1004 is triggered. Since the clock comparator and the TOD clock have a much higher resolution than jiffies we should use hrtimers to trigger the wakeup. This speeds up guest nanosleep for small values.
Since hrtimers callbacks run in hard-irq context, I added a tasklet to do the actual work with enabled interrupts.
--- arch/s390/include/asm/kvm_host.h | 5 ++++- arch/s390/kvm/interrupt.c | 33 +++++++++++++++++++++++---------- arch/s390/kvm/kvm-s390.c | 7 +++++-- arch/s390/kvm/kvm-s390.h | 4 +++- 4 files changed, 35 insertions(+), 14 deletions(-)
arch/s390/include/asm/kvm_host.h +4 -1
@@ -13,6 +13,8 @@#ifndef ASM_KVM_HOST_H #define ASM_KVM_HOST_H +#include <linux/hrtimer.h> +#include <linux/interrupt.h> #include <linux/kvm_host.h> #include <asm/debug.h> #include <asm/cpuid.h>
@@ -210,7 +212,8 @@ struct kvm_vcpu_arch {s390_fp_regs guest_fpregs;unsigned int guest_acrs[NUM_ACRS ];struct kvm_s390_local_interrupt local_int; -struct timer_list ckc_timer; +struct hrtimer ckc_timer; +struct tasklet_struct tasklet;union {cpuid_t cpu_id;u64 stidp_data;
arch/s390/kvm/interrupt.c +23 -10
@@ -12,6 +12,8 @@#include <asm/lowcore.h> #include <asm/uaccess.h> +#include <linux/hrtimer.h> +#include <linux/interrupt.h> #include <linux/kvm_host.h> #include <linux/signal.h> #include "kvm-s390.h"
@@ -361,12 +363,10 @@ int kvm_s390_handle_wait(struct kvm_vcpu *vcpu)return 0 ; } - sltime = (vcpu->arch.sie_block->ckc - now) / (0xf4240000ul /HZ ) +1 ; + sltime = ((vcpu->arch.sie_block->ckc - now)*125 )>>9 ; - vcpu->arch.ckc_timer.expires = jiffies + sltime; - -add_timer (&vcpu->arch.ckc_timer); -VCPU_EVENT (vcpu,5 ,"enabled wait timer:%llx jiffies" , sltime); +hrtimer_start (&vcpu->arch.ckc_timer,ktime_set (0 , sltime) ,HRTIMER_MODE_REL ); +VCPU_EVENT (vcpu,5 ,"enabled wait via clock comparator: %llx ns" , sltime); no_timer:spin_lock_bh (&vcpu->arch.local_int.float_int->lock);spin_lock_bh (&vcpu->arch.local_int.lock);
@@ -389,21 +389,34 @@ no_timer:remove_wait_queue (&vcpu->wq, &wait);spin_unlock_bh (&vcpu->arch.local_int.lock);spin_unlock_bh (&vcpu->arch.local_int.float_int->lock); -del_timer (&vcpu->arch.ckc_timer); +hrtimer_try_to_cancel (&vcpu->arch.ckc_timer);return 0 ; } -void kvm_s390_idle_wakeup (unsigned long data) +void kvm_s390_tasklet (unsigned long parm) { -struct kvm_vcpu *vcpu = (struct kvm_vcpu *)data; +struct kvm_vcpu *vcpu = (struct kvm_vcpu *) parm; -spin_lock_bh (&vcpu->arch.local_int.lock); +spin_lock (&vcpu->arch.local_int.lock); vcpu->arch.local_int.timer_due =1 ;if (waitqueue_active (&vcpu->arch.local_int.wq))wake_up_interruptible (&vcpu->arch.local_int.wq); -spin_unlock_bh (&vcpu->arch.local_int.lock); +spin_unlock (&vcpu->arch.local_int.lock); } +/* +* low level hrtimer wake routine. Because this runs in hardirq context +* we schedule a tasklet to do the real work. +*/ +enum hrtimer_restart kvm_s390_idle_wakeup (struct hrtimer *timer) +{ +struct kvm_vcpu *vcpu; + + vcpu = container_of(timer, struct kvm_vcpu, arch.ckc_timer); +tasklet_schedule (&vcpu->arch.tasklet); + +return HRTIMER_NORESTART ; +}void kvm_s390_deliver_pending_interrupts (struct kvm_vcpu *vcpu) {
arch/s390/kvm/kvm-s390.c +5 -2
@@ -15,6 +15,7 @@#include <linux/compiler.h> #include <linux/err.h> #include <linux/fs.h> +#include <linux/hrtimer.h> #include <linux/init.h> #include <linux/kvm.h> #include <linux/kvm_host.h>
@@ -283,8 +284,10 @@ int kvm_arch_vcpu_setup(struct kvm_vcpu *vcpu) vcpu->arch.sie_block->gmsor = vcpu->kvm->arch.guest_origin; vcpu->arch.sie_block->ecb =2 ; vcpu->arch.sie_block->eca =0xC1002001U ; -setup_timer (&vcpu->arch.ckc_timer, kvm_s390_idle_wakeup, - (unsigned long ) vcpu); +hrtimer_init (&vcpu->arch.ckc_timer,CLOCK_REALTIME ,HRTIMER_MODE_ABS ); +tasklet_init (&vcpu->arch.tasklet, kvm_s390_tasklet, + (unsigned long ) vcpu); + vcpu->arch.ckc_timer.function = kvm_s390_idle_wakeup;get_cpu_id (&vcpu->arch.cpu_id); vcpu->arch.cpu_id.version =0xff ;return 0 ;
arch/s390/kvm/kvm-s390.h +3 -1
@@ -14,6 +14,7 @@#ifndef ARCH_S390_KVM_S390_H #define ARCH_S390_KVM_S390_H +#include <linux/hrtimer.h> #include <linux/kvm.h> #include <linux/kvm_host.h>
@@ -41,7 +42,8 @@ static inline int __cpu_is_stopped(struct kvm_vcpu *vcpu) }int kvm_s390_handle_wait (struct kvm_vcpu *vcpu); -void kvm_s390_idle_wakeup (unsigned long data); +enum hrtimer_restart kvm_s390_idle_wakeup (struct hrtimer *timer); +void kvm_s390_tasklet (unsigned long parm);void kvm_s390_deliver_pending_interrupts (struct kvm_vcpu *vcpu);int kvm_s390_inject_vm (struct kvm *kvm,struct kvm_s390_interrupt *s390int);
Signature · 1 line
1.6.0.6
PATCH 32/45 KVM: s390: optimize float int lock: spin_lock_bh --> spin_lock
In reply to Avi KivityFrom: Christian Borntraeger <[email protected]>
The floating interrupt lock is only taken in process context. We can replace all spin_lock_bh with standard spin_lock calls.
--- arch/s390/kvm/interrupt.c | 20 ++++++++++---------- arch/s390/kvm/kvm-s390.c | 4 ++-- arch/s390/kvm/priv.c | 4 ++-- arch/s390/kvm/sigp.c | 16 ++++++++-------- 4 files changed, 22 insertions(+), 22 deletions(-)
arch/s390/kvm/interrupt.c +10 -10
@@ -301,13 +301,13 @@ int kvm_cpu_has_interrupt(struct kvm_vcpu *vcpu) }if ((!rc) &&atomic_read (&fi->active)) { -spin_lock_bh (&fi->lock); +spin_lock (&fi->lock);list_for_each_entry (inti, &fi->list, list)if (__interrupt_is_deliverable (vcpu, inti)) { rc =1 ;break ; } -spin_unlock_bh (&fi->lock); +spin_unlock (&fi->lock); } if ((!rc) && (vcpu->arch.sie_block->ckc <
@@ -368,7 +368,7 @@ int kvm_s390_handle_wait(struct kvm_vcpu *vcpu)hrtimer_start (&vcpu->arch.ckc_timer,ktime_set (0 , sltime) ,HRTIMER_MODE_REL );VCPU_EVENT (vcpu,5 ,"enabled wait via clock comparator: %llx ns" , sltime); no_timer: -spin_lock_bh (&vcpu->arch.local_int.float_int->lock); +spin_lock (&vcpu->arch.local_int.float_int->lock);spin_lock_bh (&vcpu->arch.local_int.lock);add_wait_queue (&vcpu->arch.local_int.wq, &wait); while (list_empty(&vcpu->arch.local_int.list) &&
@@ -377,18 +377,18 @@ no_timer: !signal_pending(current)) {set_current_state (TASK_INTERRUPTIBLE );spin_unlock_bh (&vcpu->arch.local_int.lock); -spin_unlock_bh (&vcpu->arch.local_int.float_int->lock); +spin_unlock (&vcpu->arch.local_int.float_int->lock);vcpu_put (vcpu);schedule ();vcpu_load (vcpu); -spin_lock_bh (&vcpu->arch.local_int.float_int->lock); +spin_lock (&vcpu->arch.local_int.float_int->lock);spin_lock_bh (&vcpu->arch.local_int.lock); }__unset_cpu_idle (vcpu);__set_current_state (TASK_RUNNING );remove_wait_queue (&vcpu->wq, &wait);spin_unlock_bh (&vcpu->arch.local_int.lock); -spin_unlock_bh (&vcpu->arch.local_int.float_int->lock); +spin_unlock (&vcpu->arch.local_int.float_int->lock);hrtimer_try_to_cancel (&vcpu->arch.ckc_timer);return 0 ; }
@@ -455,7 +455,7 @@ void kvm_s390_deliver_pending_interrupts(struct kvm_vcpu *vcpu)if (atomic_read (&fi->active)) { do { deliver = 0; - spin_lock_bh(&fi->lock); + spin_lock(&fi->lock); list_for_each_entry_safe(inti, n, &fi->list, list) { if (__interrupt_is_deliverable(vcpu, inti)) { list_del(&inti->list);
@@ -466,7 +466,7 @@ void kvm_s390_deliver_pending_interrupts(struct kvm_vcpu *vcpu) }if (list_empty (&fi->list))atomic_set (&fi->active,0 ); -spin_unlock_bh (&fi->lock); +spin_unlock (&fi->lock);if (deliver) {__do_deliver_interrupt (vcpu, inti);kfree (inti);
@@ -531,7 +531,7 @@ int kvm_s390_inject_vm(struct kvm *kvm,mutex_lock (&kvm->lock); fi = &kvm->arch.float_int; -spin_lock_bh (&fi->lock); +spin_lock (&fi->lock);list_add_tail (&inti->list, &fi->list);atomic_set (&fi->active,1 ); sigcpu =find_first_bit (fi->idle_mask,KVM_MAX_VCPUS );
@@ -548,7 +548,7 @@ int kvm_s390_inject_vm(struct kvm *kvm,if (waitqueue_active (&li->wq))wake_up_interruptible (&li->wq);spin_unlock_bh (&li->lock); -spin_unlock_bh (&fi->lock); +spin_unlock (&fi->lock);mutex_unlock (&kvm->lock);return 0 ; }
arch/s390/kvm/kvm-s390.c +2 -2
@@ -318,11 +318,11 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm,spin_lock_init (&vcpu->arch.local_int.lock);INIT_LIST_HEAD (&vcpu->arch.local_int.list); vcpu->arch.local_int.float_int = &kvm->arch.float_int; -spin_lock_bh (&kvm->arch.float_int.lock); +spin_lock (&kvm->arch.float_int.lock); kvm->arch.float_int.local_int[id] = &vcpu->arch.local_int;init_waitqueue_head (&vcpu->arch.local_int.wq); vcpu->arch.local_int.cpuflags = &vcpu->arch.sie_block->cpuflags; -spin_unlock_bh (&kvm->arch.float_int.lock); +spin_unlock (&kvm->arch.float_int.lock); rc =kvm_vcpu_init (vcpu, kvm, id);if (rc)
arch/s390/kvm/priv.c +2 -2
@@ -204,11 +204,11 @@ static void handle_stsi_3_2_2(struct kvm_vcpu *vcpu, struct sysinfo_3_2_2 *mem)int cpus =0 ;int n; -spin_lock_bh (&fi->lock); +spin_lock (&fi->lock);for (n =0 ; n <KVM_MAX_VCPUS ; n++)if (fi->local_int[n]) cpus++; -spin_unlock_bh (&fi->lock); +spin_unlock (&fi->lock);/* deal with other level 3 hypervisors */ if (stsi (mem,3 ,2 ,2 ) == -ENOSYS )
arch/s390/kvm/sigp.c +8 -8
@@ -52,7 +52,7 @@ static int __sigp_sense(struct kvm_vcpu *vcpu, u16 cpu_addr,if (cpu_addr >=KVM_MAX_VCPUS )return 3 ;/* not operational */ -spin_lock_bh (&fi->lock); +spin_lock (&fi->lock);if (fi->local_int[cpu_addr] ==NULL ) rc =3 ;/* not operational */ else if (atomic_read(fi->local_int[cpu_addr]->cpuflags)
@@ -64,7 +64,7 @@ static int __sigp_sense(struct kvm_vcpu *vcpu, u16 cpu_addr, *reg |=SIGP_STAT_STOPPED ; rc =1 ;/* status stored */ } -spin_unlock_bh (&fi->lock); +spin_unlock (&fi->lock);VCPU_EVENT (vcpu,4 ,"sensed status of cpu %x rc %x" , cpu_addr, rc);return rc;
@@ -86,7 +86,7 @@ static int __sigp_emergency(struct kvm_vcpu *vcpu, u16 cpu_addr) inti->type =KVM_S390_INT_EMERGENCY ; -spin_lock_bh (&fi->lock); +spin_lock (&fi->lock); li = fi->local_int[cpu_addr];if (li ==NULL ) { rc =3 ;/* not operational */
@@ -102,7 +102,7 @@ static int __sigp_emergency(struct kvm_vcpu *vcpu, u16 cpu_addr)spin_unlock_bh (&li->lock); rc =0 ;/* order accepted */ unlock: -spin_unlock_bh (&fi->lock); +spin_unlock (&fi->lock);VCPU_EVENT (vcpu,4 ,"sent sigp emerg to cpu %x" , cpu_addr);return rc; }
@@ -123,7 +123,7 @@ static int __sigp_stop(struct kvm_vcpu *vcpu, u16 cpu_addr, int store) inti->type =KVM_S390_SIGP_STOP ; -spin_lock_bh (&fi->lock); +spin_lock (&fi->lock); li = fi->local_int[cpu_addr];if (li ==NULL ) { rc =3 ;/* not operational */
@@ -142,7 +142,7 @@ static int __sigp_stop(struct kvm_vcpu *vcpu, u16 cpu_addr, int store)spin_unlock_bh (&li->lock); rc =0 ;/* order accepted */ unlock: -spin_unlock_bh (&fi->lock); +spin_unlock (&fi->lock);VCPU_EVENT (vcpu,4 ,"sent sigp stop to cpu %x" , cpu_addr);return rc; }
@@ -188,7 +188,7 @@ static int __sigp_set_prefix(struct kvm_vcpu *vcpu, u16 cpu_addr, u32 address,if (!inti)return 2 ;/* busy */ -spin_lock_bh (&fi->lock); +spin_lock (&fi->lock); li = fi->local_int[cpu_addr];if ((cpu_addr >=KVM_MAX_VCPUS ) || (li ==NULL )) {
@@ -220,7 +220,7 @@ static int __sigp_set_prefix(struct kvm_vcpu *vcpu, u16 cpu_addr, u32 address, out_li:spin_unlock_bh (&li->lock); out_fi: -spin_unlock_bh (&fi->lock); +spin_unlock (&fi->lock);return rc; }
Signature · 1 line
1.6.0.6
PATCH 33/45 KVM: s390: Unlink vcpu on destroy - v2
In reply to Avi KivityFrom: Carsten Otte <[email protected]>
This patch makes sure we do unlink a vcpu's sie control block from the system control area in kvm_arch_vcpu_destroy. This prevents illegal accesses to the sie control block from other virtual cpus after free.
--- arch/s390/kvm/kvm-s390.c | 10 ++++++++-- 1 files changed, 8 insertions(+), 2 deletions(-)
arch/s390/kvm/kvm-s390.c +8 -2
@@ -196,6 +196,10 @@ out_nokvm:void kvm_arch_vcpu_destroy (struct kvm_vcpu *vcpu) {VCPU_EVENT (vcpu,3 ,"%s" ,"free cpu" ); +if (vcpu->kvm->arch.sca->cpu[vcpu->vcpu_id].sda == + (__u64 ) vcpu->arch.sie_block) + vcpu->kvm->arch.sca->cpu[vcpu->vcpu_id].sda =0 ; +smp_mb ();free_page ((unsigned long )(vcpu->arch.sie_block));kvm_vcpu_uninit (vcpu);kfree (vcpu);
@@ -310,8 +314,10 @@ struct kvm_vcpu *kvm_arch_vcpu_create(struct kvm *kvm, vcpu->arch.sie_block->icpua = id;BUG_ON (!kvm->arch.sca); -BUG_ON (kvm->arch.sca->cpu[id].sda); - kvm->arch.sca->cpu[id].sda = (__u64 ) vcpu->arch.sie_block; +if (!kvm->arch.sca->cpu[id].sda) + kvm->arch.sca->cpu[id].sda = (__u64 ) vcpu->arch.sie_block; +else +BUG_ON (!kvm->vcpus[id]);/* vcpu does already exist */ vcpu->arch.sie_block->scaoh = (__u32)(((__u64 )kvm->arch.sca) >>32 ); vcpu->arch.sie_block->scaol = (__u32 )(__u64 )kvm->arch.sca;
Signature · 1 line
1.6.0.6
PATCH 34/45 KVM: s390: Sanity check on validity intercept
In reply to Avi KivityFrom: Carsten Otte <[email protected]>
This patch adds a sanity check for the content of the guest prefix register content before faulting in the cpu lowcore that it refers to. The guest might end up in an endless loop where SIE complains about missing lowcore with incorrect content of the prefix register without this fix.
--- arch/s390/kvm/intercept.c | 28 ++++++++++++++++++---------- 1 files changed, 18 insertions(+), 10 deletions(-)
arch/s390/kvm/intercept.c +18 -10
@@ -154,17 +154,25 @@ static int handle_stop(struct kvm_vcpu *vcpu)static int handle_validity (struct kvm_vcpu *vcpu) {int viwhy = vcpu->arch.sie_block->ipb >>16 ; +int rc; + vcpu->stat.exit_validity++; -if (viwhy ==0x37 ) { -fault_in_pages_writeable ((char __user *) - vcpu->kvm->arch.guest_origin + - vcpu->arch.sie_block->prefix, -PAGE_SIZE ); -return 0 ; - } -VCPU_EVENT (vcpu,2 ,"unhandled validity intercept code %d" , - viwhy); -return -ENOTSUPP ; +if ((viwhy ==0x37 ) && (vcpu->arch.sie_block->prefix + <= vcpu->kvm->arch.guest_memsize -2 *PAGE_SIZE )){ + rc =fault_in_pages_writeable ((char __user *) + vcpu->kvm->arch.guest_origin + + vcpu->arch.sie_block->prefix, +2 *PAGE_SIZE ); +if (rc) +/* user will receive sigsegv, exit to user */ + rc = -ENOTSUPP ; + }else + rc = -ENOTSUPP ; + +if (rc) +VCPU_EVENT (vcpu,2 ,"unhandled validity intercept code %d" , + viwhy); +return rc; }static int handle_instruction (struct kvm_vcpu *vcpu)
Signature · 1 line
1.6.0.6
PATCH 35/45 KVM: s390: Verify memory in kvm run
In reply to Avi KivityFrom: Carsten Otte <[email protected]>
This check verifies that the guest we're trying to run in KVM_RUN has some memory assigned to it. It enters an endless exception loop if this is not the case.
--- arch/s390/kvm/kvm-s390.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
arch/s390/kvm/kvm-s390.c +6 -0
@@ -487,6 +487,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)vcpu_load (vcpu); +/* verify, that memory has been registered */ +if (!vcpu->kvm->arch.guest_memsize) { +vcpu_put (vcpu); +return -EINVAL ; + } +if (vcpu->sigset_active)sigprocmask (SIG_SETMASK , &vcpu->sigset, &sigsaved);
Signature · 1 line
1.6.0.6
PATCH 36/45 KVM: Unprotect a page if #PF happens during NMI injection.
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
It is done for exception and interrupt already.
--- arch/x86/kvm/svm.c | 3 +-- arch/x86/kvm/vmx.c | 2 +- arch/x86/kvm/x86.h | 6 ++++++ 3 files changed, 8 insertions(+), 3 deletions(-)
arch/x86/kvm/svm.c +1 -2
@@ -1122,8 +1122,7 @@ static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)if (npt_enabled)svm_flush_tlb (&svm->vcpu);else { -if (svm->vcpu.arch.interrupt.pending || - svm->vcpu.arch.exception.pending) +if (kvm_event_needs_reinjection (&svm->vcpu))kvm_mmu_unprotect_page_virt (&svm->vcpu, fault_address); }return kvm_mmu_page_fault (&svm->vcpu, fault_address, error_code);
arch/x86/kvm/vmx.c +1 -1
@@ -2615,7 +2615,7 @@ static int handle_exception(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) cr2 =vmcs_readl (EXIT_QUALIFICATION );KVMTRACE_3D (PAGE_FAULT , vcpu, error_code, (u32 )cr2, (u32)((u64 )cr2 >>32 ), handler); -if (vcpu->arch.interrupt.pending || vcpu->arch.exception.pending) +if (kvm_event_needs_reinjection (vcpu))kvm_mmu_unprotect_page_virt (vcpu, cr2);return kvm_mmu_page_fault (vcpu, cr2, error_code); }
arch/x86/kvm/x86.h +6 -0
@@ -30,4 +30,10 @@ static inline u8 kvm_pop_irq(struct kvm_vcpu *vcpu)clear_bit (word_index, &vcpu->arch.irq_summary); return irq; } + +static inline bool kvm_event_needs_reinjection (struct kvm_vcpu *vcpu) +{ +return vcpu->arch.exception.pending || vcpu->arch.interrupt.pending || + vcpu->arch.nmi_injected; +}#endif
Signature · 1 line
1.6.0.6
PATCH 37/45 KVM: Do not allow interrupt injection from userspace if there is a pending event.
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
The exception will immediately close the interrupt window.
--- arch/x86/kvm/x86.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-)
arch/x86/kvm/x86.c +3 -2
@@ -3101,8 +3101,9 @@ static void post_kvm_run_save(struct kvm_vcpu *vcpu, kvm_run->ready_for_interrupt_injection =1 ; else kvm_run->ready_for_interrupt_injection = - (kvm_arch_interrupt_allowed (vcpu) && - !kvm_cpu_has_interrupt (vcpu)); +kvm_arch_interrupt_allowed (vcpu) && + !kvm_cpu_has_interrupt (vcpu) && + !kvm_event_needs_reinjection (vcpu); }static void vapic_enter (struct kvm_vcpu *vcpu)
Signature · 1 line
1.6.0.6
PATCH 38/45 KVM: Remove irq_pending bitmap
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
Only one interrupt vector can be injected from userspace irqchip at any given time so no need to store it in a bitmap. Put it into interrupt queue directly.
--- arch/x86/include/asm/kvm_host.h | 2 -- arch/x86/kvm/irq.c | 4 ++-- arch/x86/kvm/x86.c | 38 +++++++++++--------------------------- arch/x86/kvm/x86.h | 12 ------------ 4 files changed, 13 insertions(+), 43 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -2
@@ -266,8 +266,6 @@ struct kvm_mmu {struct kvm_vcpu_arch {u64 host_tsc; -unsigned long irq_summary;/* bit vector: 1 per word in irq_pending */ - DECLARE_BITMAP(irq_pending, KVM_NR_INTERRUPTS);/* * rip and regs accesses must go through * kvm_{register,rip}_{read,write} functions.
arch/x86/kvm/irq.c +2 -2
@@ -50,7 +50,7 @@ int kvm_cpu_has_interrupt(struct kvm_vcpu *v)struct kvm_pic *s;if (!irqchip_in_kernel (v->kvm)) -return v->arch.irq_summary; +return v->arch.interrupt.pending;if (kvm_apic_has_interrupt (v) ==-1 ) {/* LAPIC */ if (kvm_apic_accept_pic_intr (v)) {
@@ -72,7 +72,7 @@ int kvm_cpu_get_interrupt(struct kvm_vcpu *v)int vector;if (!irqchip_in_kernel (v->kvm)) -return kvm_pop_irq (v); +return v->arch.interrupt.nr; vector =kvm_get_apic_interrupt (v);/* APIC */ if (vector ==-1 ) {
arch/x86/kvm/x86.c +11 -27
@@ -1437,8 +1437,7 @@ static int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,return -ENXIO ;vcpu_load (vcpu); -set_bit (irq->irq, vcpu->arch.irq_pending); -set_bit (irq->irq /BITS_PER_LONG , &vcpu->arch.irq_summary); +kvm_queue_interrupt (vcpu, irq->irq);vcpu_put (vcpu);
@@ -3579,12 +3578,7 @@ int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu, sregs->efer = vcpu->arch.shadow_efer; sregs->apic_base =kvm_get_apic_base (vcpu); -if (irqchip_in_kernel (vcpu->kvm)) -memset (sregs->interrupt_bitmap,0 , -sizeof sregs->interrupt_bitmap); -else -memcpy (sregs->interrupt_bitmap, vcpu->arch.irq_pending, -sizeof sregs->interrupt_bitmap); +memset (sregs->interrupt_bitmap,0 ,sizeof sregs->interrupt_bitmap);if (vcpu->arch.interrupt.pending) set_bit(vcpu->arch.interrupt.nr,
@@ -4054,7 +4048,7 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) {int mmu_reset_needed =0 ; -int i, pending_vec, max_bits; +int pending_vec, max_bits;struct descriptor_table dt;vcpu_load (vcpu);
@@ -4096,24 +4090,14 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu,if (mmu_reset_needed)kvm_mmu_reset_context (vcpu); -if (!irqchip_in_kernel (vcpu->kvm)) { -memcpy (vcpu->arch.irq_pending, sregs->interrupt_bitmap, -sizeof vcpu->arch.irq_pending); - vcpu->arch.irq_summary =0 ; -for (i =0 ; i <ARRAY_SIZE (vcpu->arch.irq_pending); ++i) -if (vcpu->arch.irq_pending[i]) -__set_bit (i, &vcpu->arch.irq_summary); - }else { - max_bits = (sizeof sregs->interrupt_bitmap) <<3 ; - pending_vec =find_first_bit ( - (const unsigned long *)sregs->interrupt_bitmap, - max_bits); -/* Only pending external irq is handled here */ -if (pending_vec < max_bits) { -kvm_queue_interrupt (vcpu, pending_vec); -pr_debug ("Set back pending irq %d\n" , pending_vec); - } -kvm_pic_clear_isr_ack (vcpu->kvm); + max_bits = (sizeof sregs->interrupt_bitmap) <<3 ; + pending_vec =find_first_bit ( + (const unsigned long *)sregs->interrupt_bitmap, max_bits); +if (pending_vec < max_bits) { +kvm_queue_interrupt (vcpu, pending_vec); +pr_debug ("Set back pending irq %d\n" , pending_vec); +if (irqchip_in_kernel (vcpu->kvm)) +kvm_pic_clear_isr_ack (vcpu->kvm); }kvm_set_segment (vcpu, &sregs->cs,VCPU_SREG_CS );
arch/x86/kvm/x86.h +0 -12
@@ -19,18 +19,6 @@ static inline void kvm_clear_interrupt_queue(struct kvm_vcpu *vcpu) vcpu->arch.interrupt.pending = false; } -static inline u8 kvm_pop_irq (struct kvm_vcpu *vcpu) -{ -int word_index =__ffs (vcpu->arch.irq_summary); -int bit_index =__ffs (vcpu->arch.irq_pending[word_index]); -int irq = word_index *BITS_PER_LONG + bit_index; - -clear_bit (bit_index, &vcpu->arch.irq_pending[word_index]); -if (!vcpu->arch.irq_pending[word_index]) -clear_bit (word_index, &vcpu->arch.irq_summary); -return irq; -} -static inline bool kvm_event_needs_reinjection (struct kvm_vcpu *vcpu) { return vcpu->arch.exception.pending || vcpu->arch.interrupt.pending ||
Signature · 1 line
1.6.0.6
PATCH 39/45 KVM: skip_emulated_instruction() decode instruction if size is not known
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
--- arch/x86/kvm/svm.c | 11 +++++------ 1 files changed, 5 insertions(+), 6 deletions(-)
arch/x86/kvm/svm.c +5 -6
@@ -228,7 +228,9 @@ static void skip_emulated_instruction(struct kvm_vcpu *vcpu)struct vcpu_svm *svm =to_svm (vcpu);if (!svm->next_rip) { -printk (KERN_DEBUG "%s: NOP\n" , __func__); +if (emulate_instruction (vcpu, vcpu->run,0 ,0 ,EMULTYPE_SKIP ) != +EMULATE_DONE ) +printk (KERN_DEBUG "%s: NOP\n" , __func__);return ; }if (svm->next_rip -kvm_rip_read (vcpu) >MAX_INST_SIZE )
@@ -1868,11 +1870,8 @@ static int task_switch_interception(struct vcpu_svm *svm,if (reason !=TASK_SWITCH_GATE || int_type ==SVM_EXITINTINFO_TYPE_SOFT || (int_type ==SVM_EXITINTINFO_TYPE_EXEPT && - (int_vec ==OF_VECTOR || int_vec ==BP_VECTOR ))) { -if (emulate_instruction (&svm->vcpu, kvm_run,0 ,0 , -EMULTYPE_SKIP ) !=EMULATE_DONE ) -return 0 ; - } + (int_vec ==OF_VECTOR || int_vec ==BP_VECTOR ))) +skip_emulated_instruction (&svm->vcpu);return kvm_task_switch (&svm->vcpu, tss_selector, reason); }
Signature · 1 line
1.6.0.6
PATCH 40/45 KVM: Do not re-execute INTn instruction.
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
Re-inject event instead. This is what Intel suggest. Also use correct instruction length when re-injecting soft fault/interrupt.
--- arch/x86/include/asm/kvm_host.h | 5 ++++- arch/x86/kvm/svm.c | 8 ++++---- arch/x86/kvm/vmx.c | 32 +++++++++++++++++++++++++------- arch/x86/kvm/x86.c | 11 ++++++----- arch/x86/kvm/x86.h | 9 ++++++++- 5 files changed, 47 insertions(+), 18 deletions(-)
arch/x86/include/asm/kvm_host.h +4 -1
@@ -319,6 +319,8 @@ struct kvm_vcpu_arch {struct kvm_pio_request pio;void *pio_data; +u8 event_exit_inst_len; +struct kvm_queued_exception {bool pending;bool has_error_code;
@@ -328,6 +330,7 @@ struct kvm_vcpu_arch {struct kvm_queued_interrupt {bool pending; +bool soft;u8 nr; } interrupt;
@@ -510,7 +513,7 @@ struct kvm_x86_ops {u32 (*get_interrupt_shadow)(struct kvm_vcpu *vcpu,int mask);void (*patch_hypercall)(struct kvm_vcpu *vcpu,unsigned char *hypercall_addr); -void (*set_irq)(struct kvm_vcpu *vcpu,int vec); +void (*set_irq)(struct kvm_vcpu *vcpu);void (*set_nmi)(struct kvm_vcpu *vcpu);void (*queue_exception)(struct kvm_vcpu *vcpu,unsigned nr,bool has_error_code,u32 error_code);
arch/x86/kvm/svm.c +4 -4
@@ -2310,13 +2310,13 @@ static void svm_queue_irq(struct kvm_vcpu *vcpu, unsigned nr)SVM_EVTINJ_VALID |SVM_EVTINJ_TYPE_INTR ; } -static void svm_set_irq (struct kvm_vcpu *vcpu,int irq) +static void svm_set_irq (struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu);nested_svm_intr (svm); -svm_queue_irq (vcpu, irq); +svm_queue_irq (vcpu, vcpu->arch.interrupt.nr); }static void update_cr8_intercept (struct kvm_vcpu *vcpu,int tpr,int irr)
@@ -2418,7 +2418,7 @@ static void svm_complete_interrupts(struct vcpu_svm *svm)case SVM_EXITINTINFO_TYPE_EXEPT :/* In case of software exception do not reinject an exception vector, but re-execute and instruction instead */ -if (vector ==BP_VECTOR || vector ==OF_VECTOR ) +if (kvm_exception_is_soft (vector))break ;if (exitintinfo &SVM_EXITINTINFO_VALID_ERR ) {u32 err = svm->vmcb->control.exit_int_info_err;
@@ -2428,7 +2428,7 @@ static void svm_complete_interrupts(struct vcpu_svm *svm)kvm_queue_exception (&svm->vcpu, vector);break ;case SVM_EXITINTINFO_TYPE_INTR : -kvm_queue_interrupt (&svm->vcpu, vector); +kvm_queue_interrupt (&svm->vcpu, vector, false);break ;default :break ;
arch/x86/kvm/vmx.c +25 -7
@@ -801,8 +801,9 @@ static void vmx_queue_exception(struct kvm_vcpu *vcpu, unsigned nr,return ; } -if (nr ==BP_VECTOR || nr ==OF_VECTOR ) { -vmcs_write32 (VM_ENTRY_INSTRUCTION_LEN ,1 ); +if (kvm_exception_is_soft (nr)) { +vmcs_write32 (VM_ENTRY_INSTRUCTION_LEN , + vmx->vcpu.arch.event_exit_inst_len); intr_info |=INTR_TYPE_SOFT_EXCEPTION ; }else intr_info |=INTR_TYPE_HARD_EXCEPTION ;
@@ -2445,9 +2446,11 @@ static void enable_nmi_window(struct kvm_vcpu *vcpu)vmcs_write32 (CPU_BASED_VM_EXEC_CONTROL , cpu_based_vm_exec_control); } -static void vmx_inject_irq (struct kvm_vcpu *vcpu,int irq) +static void vmx_inject_irq (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +uint32_t intr; +int irq = vcpu->arch.interrupt.nr;KVMTRACE_1D (INJ_VIRQ , vcpu, (u32 )irq, handler);
@@ -2462,8 +2465,14 @@ static void vmx_inject_irq(struct kvm_vcpu *vcpu, int irq)kvm_rip_write (vcpu, vmx->rmode.irq.rip -1 );return ; } -vmcs_write32 (VM_ENTRY_INTR_INFO_FIELD , - irq |INTR_TYPE_EXT_INTR |INTR_INFO_VALID_MASK ); + intr = irq |INTR_INFO_VALID_MASK ; +if (vcpu->arch.interrupt.soft) { + intr |=INTR_TYPE_SOFT_INTR ; +vmcs_write32 (VM_ENTRY_INSTRUCTION_LEN , + vmx->vcpu.arch.event_exit_inst_len); + }else + intr |=INTR_TYPE_EXT_INTR ; +vmcs_write32 (VM_ENTRY_INTR_INFO_FIELD , intr); }static void vmx_inject_nmi (struct kvm_vcpu *vcpu)
@@ -3024,6 +3033,7 @@ static int handle_task_switch(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) GUEST_INTR_STATE_NMI);break ;case INTR_TYPE_EXT_INTR : +case INTR_TYPE_SOFT_INTR :kvm_clear_interrupt_queue (vcpu);break ;case INTR_TYPE_HARD_EXCEPTION :
@@ -3295,16 +3305,24 @@ static void vmx_complete_interrupts(struct vcpu_vmx *vmx)vmcs_clear_bits (GUEST_INTERRUPTIBILITY_INFO ,GUEST_INTR_STATE_NMI );break ; -case INTR_TYPE_HARD_EXCEPTION :case INTR_TYPE_SOFT_EXCEPTION : + vmx->vcpu.arch.event_exit_inst_len = +vmcs_read32 (VM_EXIT_INSTRUCTION_LEN ); +/* fall through */ +case INTR_TYPE_HARD_EXCEPTION :if (idt_vectoring_info &VECTORING_INFO_DELIVER_CODE_MASK ) {u32 err =vmcs_read32 (IDT_VECTORING_ERROR_CODE );kvm_queue_exception_e (&vmx->vcpu, vector, err); }else kvm_queue_exception (&vmx->vcpu, vector);break ; +case INTR_TYPE_SOFT_INTR : + vmx->vcpu.arch.event_exit_inst_len = +vmcs_read32 (VM_EXIT_INSTRUCTION_LEN ); +/* fall through */ case INTR_TYPE_EXT_INTR : -kvm_queue_interrupt (&vmx->vcpu, vector); +kvm_queue_interrupt (&vmx->vcpu, vector, + type ==INTR_TYPE_SOFT_INTR );break ;default :break ;
arch/x86/kvm/x86.c +6 -5
@@ -1437,7 +1437,7 @@ static int kvm_vcpu_ioctl_interrupt(struct kvm_vcpu *vcpu,return -ENXIO ;vcpu_load (vcpu); -kvm_queue_interrupt (vcpu, irq->irq); +kvm_queue_interrupt (vcpu, irq->irq, false);vcpu_put (vcpu);
@@ -3157,7 +3157,7 @@ static void inject_irq(struct kvm_vcpu *vcpu) }if (vcpu->arch.interrupt.pending) { - kvm_x86_ops->set_irq (vcpu, vcpu->arch.interrupt.nr); + kvm_x86_ops->set_irq (vcpu);return ; }
@@ -3170,8 +3170,9 @@ static void inject_irq(struct kvm_vcpu *vcpu) } }else if (kvm_cpu_has_interrupt (vcpu)) {if (kvm_x86_ops->interrupt_allowed (vcpu)) { -kvm_queue_interrupt (vcpu,kvm_cpu_get_interrupt (vcpu)); - kvm_x86_ops->set_irq (vcpu, vcpu->arch.interrupt.nr); +kvm_queue_interrupt (vcpu,kvm_cpu_get_interrupt (vcpu), + false); + kvm_x86_ops->set_irq (vcpu); } } }
@@ -4094,7 +4095,7 @@ int kvm_arch_vcpu_ioctl_set_sregs(struct kvm_vcpu *vcpu, pending_vec =find_first_bit ( (const unsigned long *)sregs->interrupt_bitmap, max_bits);if (pending_vec < max_bits) { -kvm_queue_interrupt (vcpu, pending_vec); +kvm_queue_interrupt (vcpu, pending_vec, false);pr_debug ("Set back pending irq %d\n" , pending_vec);if (irqchip_in_kernel (vcpu->kvm))kvm_pic_clear_isr_ack (vcpu->kvm);
arch/x86/kvm/x86.h +8 -1
@@ -8,9 +8,11 @@ static inline void kvm_clear_exception_queue(struct kvm_vcpu *vcpu) vcpu->arch.exception.pending = false; } -static inline void kvm_queue_interrupt (struct kvm_vcpu *vcpu,u8 vector) +static inline void kvm_queue_interrupt (struct kvm_vcpu *vcpu,u8 vector, +bool soft) { vcpu->arch.interrupt.pending = true; + vcpu->arch.interrupt.soft = soft; vcpu->arch.interrupt.nr = vector; }
@@ -24,4 +26,9 @@ static inline bool kvm_event_needs_reinjection(struct kvm_vcpu *vcpu)return vcpu->arch.exception.pending || vcpu->arch.interrupt.pending || vcpu->arch.nmi_injected; } + +static inline bool kvm_exception_is_soft (unsigned int nr) +{ +return (nr ==BP_VECTOR ) || (nr ==OF_VECTOR ); +}#endif
Signature · 1 line
1.6.0.6
PATCH 41/45 KVM: Always request IRQ/NMI window if an interrupt is pending
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
Currently they are not requested if there is pending exception.
--- arch/x86/kvm/x86.c | 30 ++++++++++++------------------ 1 files changed, 12 insertions(+), 18 deletions(-)
arch/x86/kvm/x86.c +12 -18
@@ -3148,8 +3148,11 @@ static void update_cr8_intercept(struct kvm_vcpu *vcpu) kvm_x86_ops->update_cr8_intercept (vcpu, tpr, max_irr); } -static void inject_irq (struct kvm_vcpu *vcpu) +static void inject_pending_irq (struct kvm_vcpu *vcpu,struct kvm_run *kvm_run) { +if (vcpu->guest_debug &KVM_GUESTDBG_SINGLESTEP ) + kvm_x86_ops->set_interrupt_shadow (vcpu,0 ); +/* try to reinject previous events if any */ if (vcpu->arch.nmi_injected) { kvm_x86_ops->set_nmi (vcpu);
@@ -3177,26 +3180,11 @@ static void inject_irq(struct kvm_vcpu *vcpu) } } -static void inject_pending_irq (struct kvm_vcpu *vcpu,struct kvm_run *kvm_run) -{ -bool req_int_win = !irqchip_in_kernel (vcpu->kvm) && - kvm_run->request_interrupt_window; - -if (vcpu->guest_debug &KVM_GUESTDBG_SINGLESTEP ) - kvm_x86_ops->set_interrupt_shadow (vcpu,0 ); - -inject_irq (vcpu); - -/* enable NMI/IRQ window open exits if needed */ -if (vcpu->arch.nmi_pending) - kvm_x86_ops->enable_nmi_window (vcpu); -else if (kvm_cpu_has_interrupt (vcpu) || req_int_win) - kvm_x86_ops->enable_irq_window (vcpu); -} -static int vcpu_enter_guest (struct kvm_vcpu *vcpu,struct kvm_run *kvm_run) {int r; +bool req_int_win = !irqchip_in_kernel (vcpu->kvm) && + kvm_run->request_interrupt_window;if (vcpu->requests)if (test_and_clear_bit (KVM_REQ_MMU_RELOAD , &vcpu->requests))
@@ -3250,6 +3238,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)else inject_pending_irq (vcpu, kvm_run); +/* enable NMI/IRQ window open exits if needed */ +if (vcpu->arch.nmi_pending) + kvm_x86_ops->enable_nmi_window (vcpu); +else if (kvm_cpu_has_interrupt (vcpu) || req_int_win) + kvm_x86_ops->enable_irq_window (vcpu); +if (kvm_lapic_enabled (vcpu)) {if (!vcpu->arch.apic->vapic_addr)update_cr8_intercept (vcpu);
Signature · 1 line
1.6.0.6
PATCH 42/45 KVM: inject NMI after IRET from a previous NMI, not before.
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
If NMI is received during handling of another NMI it should be injected immediately after IRET from previous NMI handler, but SVM intercept IRET before instruction execution so we can't inject pending NMI at this point and there is not way to request exit when NMI window opens. This patch fix SVM code to open NMI window after IRET by single stepping over IRET instruction.
--- arch/x86/include/asm/kvm_host.h | 2 + arch/x86/kvm/svm.c | 62 +++++++++++++++++++++++++++++++------- 2 files changed, 52 insertions(+), 12 deletions(-)
arch/x86/include/asm/kvm_host.h +2 -0
@@ -358,6 +358,7 @@ struct kvm_vcpu_arch {unsigned int time_offset;struct page *time_page; +bool singlestep;/* guest is single stepped by KVM */ bool nmi_pending;bool nmi_injected;
@@ -771,6 +772,7 @@ enum {#define HF_HIF_MASK (1 << 1)#define HF_VINTR_MASK (1 << 2)#define HF_NMI_MASK (1 << 3) +#define HF_IRET_MASK (1 << 4)/* * Hardware virtualization extension instructions may fault if a
arch/x86/kvm/svm.c +50 -12
106 lines not on this page · raw message
Signature · 1 line
1.6.0.6
PATCH 43/45 KVM: Do not migrate pending software interrupts.
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
INTn will be re-executed after migration. If we wanted to migrate pending software interrupt we would need to migrate interrupt type and instruction length too, but we do not have all required info on SVM, so SVM->VMX migration would need to re-execute INTn anyway. To make it simple never migrate pending soft interrupt.
--- arch/x86/kvm/x86.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/kvm/x86.c +1 -1
@@ -3575,7 +3575,7 @@ int kvm_arch_vcpu_ioctl_get_sregs(struct kvm_vcpu *vcpu,memset (sregs->interrupt_bitmap,0 ,sizeof sregs->interrupt_bitmap); -if (vcpu->arch.interrupt.pending) +if (vcpu->arch.interrupt.pending && !vcpu->arch.interrupt.soft)set_bit (vcpu->arch.interrupt.nr, (unsigned long *)sregs->interrupt_bitmap);
Signature · 1 line
1.6.0.6
PATCH 44/45 KVM: Disable CR8 intercept if tpr patching is active
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
--- arch/x86/kvm/x86.c | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-)
arch/x86/kvm/x86.c +6 -5
@@ -3138,7 +3138,10 @@ static void update_cr8_intercept(struct kvm_vcpu *vcpu)if (!kvm_x86_ops->update_cr8_intercept)return ; - max_irr =kvm_lapic_find_highest_irr (vcpu); +if (!vcpu->arch.apic->vapic_addr) + max_irr =kvm_lapic_find_highest_irr (vcpu); +else + max_irr =-1 ;if (max_irr !=-1 ) max_irr >>=4 ;
@@ -3245,10 +3248,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) kvm_x86_ops->enable_irq_window (vcpu);if (kvm_lapic_enabled (vcpu)) { -if (!vcpu->arch.apic->vapic_addr) -update_cr8_intercept (vcpu); -else -kvm_lapic_sync_to_vapic (vcpu); +update_cr8_intercept (vcpu); +kvm_lapic_sync_to_vapic (vcpu); }up_read (&vcpu->kvm->slots_lock);
Signature · 1 line
1.6.0.6
PATCH 45/45 KVM: Move "exit due to NMI" handling into vmx_complete_interrupts()
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
To save us one reading of VM_EXIT_INTR_INFO.
--- arch/x86/kvm/vmx.c | 21 ++++++++++----------- 1 files changed, 10 insertions(+), 11 deletions(-)
arch/x86/kvm/vmx.c +10 -11
@@ -3261,8 +3261,17 @@ static void vmx_complete_interrupts(struct vcpu_vmx *vmx)int type;bool idtv_info_valid; - idtv_info_valid = idt_vectoring_info &VECTORING_INFO_VALID_MASK ; exit_intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); + +/* We need to handle NMIs before interrupts are enabled */ +if ((exit_intr_info &INTR_INFO_INTR_TYPE_MASK ) ==INTR_TYPE_NMI_INTR && + (exit_intr_info &INTR_INFO_VALID_MASK )) { +KVMTRACE_0D (NMI , &vmx->vcpu, handler); + asm("int $2" ); + } + + idtv_info_valid = idt_vectoring_info &VECTORING_INFO_VALID_MASK ; +if (cpu_has_virtual_nmis ()) { unblock_nmi = (exit_intr_info &INTR_INFO_UNBLOCK_NMI ) !=0 ; vector = exit_intr_info &INTR_INFO_VECTOR_MASK ;
@@ -3363,7 +3372,6 @@ static void fixup_rmode_irq(struct vcpu_vmx *vmx)static void vmx_vcpu_run (struct kvm_vcpu *vcpu,struct kvm_run *kvm_run) {struct vcpu_vmx *vmx =to_vmx (vcpu); -u32 intr_info;/* Record the guest's net vcpu time for enforced NMI injections. */ if (unlikely (!cpu_has_virtual_nmis () && vmx->soft_vnmi_blocked))
@@ -3490,15 +3498,6 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) asm("mov %0, %%ds; mov %0, %%es" : :"r" (__USER_DS)); vmx->launched =1 ; - intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); - -/* We need to handle NMIs before interrupts are enabled */ -if ((intr_info &INTR_INFO_INTR_TYPE_MASK ) ==INTR_TYPE_NMI_INTR && - (intr_info &INTR_INFO_VALID_MASK )) { -KVMTRACE_0D (NMI , vcpu, handler); - asm("int $2" ); - } -vmx_complete_interrupts (vmx); }
Signature · 1 line
1.6.0.6