KVM updates for 2.6.32 merge window (4/4)
Subsystems (9)
cover KVM updates for 2.6.32 merge window (4/4)
Fourth and final batch of the 2.6.32 KVM patch queue.
Amit Shah (2):
KVM: ignore reads to perfctr msrs
Documentation: Update KVM list email address
Anthony Liguori (1):
KVM: When switching to a vm8086 task, load segments as 16-bit
Avi Kivity (10):
KVM: VMX: Optimize vmx_get_cpl()
x86: Export kmap_atomic_to_page()
KVM: SVM: Drop tlb flush workaround in npt
KVM: Move #endif KVM_CAP_IRQ_ROUTING to correct place
KVM: VMX: Adjust rflags if in real mode emulation
KVM: Rename x86_emulate.c to emulate.c
KVM: Add __KERNEL__ guards to exported headers
KVM: Add missing #include
KVM: Protect update_cr8_intercept() when running without an apic
KVM: Document KVM_CAP_IRQCHIP
Bartlomiej Zolnierkiewicz (1):
KVM: remove superfluous NULL pointer check in
kvm_inject_pit_timer_irqs()
Gleb Natapov (4):
KVM: Call kvm_vcpu_kick() inside pic spinlock
KVM: Call ack notifiers from PIC when guest OS acks an IRQ.
KVM: Replace pic_lock()/pic_unlock() with direct call to spinlock
functions
KVM: Update cr8 intercept when APIC TPR is changed by userspace
Izik Eidus (1):
KVM: MMU: make __kvm_mmu_free_some_pages handle empty list
Jan Kiszka (1):
KVM: x86: Disallow hypercalls for guest callers in rings > 0
Joerg Roedel (21):
KVM: SVM: add helper functions for global interrupt flag
KVM: SVM: optimize nested #vmexit
KVM: SVM: optimize nested vmrun
KVM: SVM: copy only necessary parts of the control area on
vmrun/vmexit
KVM: SVM: complete interrupts after handling nested exits
KVM: SVM: move nested svm state into seperate struct
KVM: SVM: cache nested intercepts
KVM: SVM: consolidate nested_svm_exit_handled
KVM: SVM: do nested vmexit in nested_svm_exit_handled
KVM: SVM: simplify nested_svm_check_exception
KVM: SVM: get rid of nested_svm_vmexit_real
KVM: SVM: clean up nested_svm_exit_handled_msr
KVM: SVM: clean up nestec vmload/vmsave paths
KVM: SVM: clean up nested vmrun path
KVM: SVM: remove nested_svm_do and helper functions
KVM: SVM: handle errors in vmrun emulation path appropriatly
KVM: SVM: move special nested exit handling to separate function
KVM: SVM: remove unnecessary is_nested check from svm_cpu_run
KVM: SVM: move nested_svm_intr main logic out of if-clause
KVM: SVM: check for nested VINTR flag in svm_interrupt_allowed
KVM: SVM: enable nested svm by default
Marcelo Tosatti (1):
KVM: MMU: fix bogus alloc_mmu_pages assignment
Michael S. Tsirkin (1):
KVM: export kvm_para.h
Mikhail Ershov (1):
KVM: Use kvm_{read,write}_guest_virt() to read and write segment
descriptors
Mohammed Gamal (1):
KVM: x86 emulator: Add adc and sbb missing decoder flags
Roel Kluin (1):
KVM: fix EFER read buffer overflow
Sheng Yang (1):
KVM: VMX: Fix EPT with WP bit change during paging
Documentation/ioctl/ioctl-number.txt | 2 +-
Documentation/kvm/api.txt | 76 +++
arch/ia64/include/asm/kvm_para.h | 4 +
arch/s390/include/asm/kvm_para.h | 4 +
.../asm/{kvm_x86_emulate.h => kvm_emulate.h} | 0
arch/x86/include/asm/kvm_host.h | 2 +-
arch/x86/include/asm/kvm_para.h | 2 +
arch/x86/kvm/Makefile | 2 +-
arch/x86/kvm/{x86_emulate.c => emulate.c} | 8 +-
arch/x86/kvm/i8254.c | 2 +-
arch/x86/kvm/i8259.c | 64 +--
arch/x86/kvm/irq.h | 1 -
arch/x86/kvm/mmu.c | 11 +-
arch/x86/kvm/svm.c | 650 +++++++++++---------
arch/x86/kvm/vmx.c | 25 +-
arch/x86/kvm/x86.c | 35 +-
arch/x86/mm/highmem_32.c | 1 +
include/asm-generic/Kbuild.asm | 5 +
include/linux/Kbuild | 4 +
include/linux/kvm_para.h | 1 +
virt/kvm/kvm_main.c | 2 +-
21 files changed, 525 insertions(+), 376 deletions(-)
rename arch/x86/include/asm/{kvm_x86_emulate.h => kvm_emulate.h} (100%)
rename arch/x86/kvm/{x86_emulate.c => emulate.c} (99%)PATCH 1/47 KVM: remove superfluous NULL pointer check in kvm_inject_pit_timer_irqs()
From: Bartlomiej Zolnierkiewicz <[email protected]>
This takes care of the following entries from Dan's list:
arch/x86/kvm/i8254.c +714 kvm_inject_pit_timer_irqs(6) warning: variable derefenced in initializer 'vcpu' arch/x86/kvm/i8254.c +714 kvm_inject_pit_timer_irqs(6) warning: variable derefenced before check 'vcpu'
Reported-by Dan Carpenter <[email protected]>
Cc Julia Lawall <[email protected]>
Signed-off-by Bartlomiej Zolnierkiewicz <[email protected]>
Acked-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/i8254.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/kvm/i8254.c +1 -1
@@ -713,7 +713,7 @@ void kvm_inject_pit_timer_irqs(struct kvm_vcpu *vcpu)struct kvm *kvm = vcpu->kvm;struct kvm_kpit_state *ps; -if (vcpu && pit) { +if (pit) {int inject =0 ; ps = &pit->pit_state;
Signature · 1 line
1.6.4.1
PATCH 2/47 KVM: MMU: make __kvm_mmu_free_some_pages handle empty list
In reply to Avi KivityFrom: Izik Eidus <[email protected]>
First check if the list is empty before attempting to look at list entries.
Signed-off-by Izik Eidus <[email protected]>
Signed-off-by Marcelo Tosatti <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/mmu.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
arch/x86/kvm/mmu.c +2 -1
@@ -2705,7 +2705,8 @@ EXPORT_SYMBOL_GPL(kvm_mmu_unprotect_page_virt);void __kvm_mmu_free_some_pages (struct kvm_vcpu *vcpu) { -while (vcpu->kvm->arch.n_free_mmu_pages <KVM_REFILL_PAGES ) { +while (vcpu->kvm->arch.n_free_mmu_pages <KVM_REFILL_PAGES && + !list_empty (&vcpu->kvm->arch.active_mmu_pages)) {struct kvm_mmu_page *sp; sp = container_of(vcpu->kvm->arch.active_mmu_pages.prev,
Signature · 1 line
1.6.4.1
PATCH 3/47 KVM: MMU: fix bogus alloc_mmu_pages assignment
In reply to Avi KivityFrom: Marcelo Tosatti <[email protected]>
Remove the bogus n_free_mmu_pages assignment from alloc_mmu_pages.
It breaks accounting of mmu pages, since n_free_mmu_pages is modified but the real number of pages remains the same.
Signed-off-by Marcelo Tosatti <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/mmu.c | 8 -------- 1 files changed, 0 insertions(+), 8 deletions(-)
arch/x86/kvm/mmu.c +0 -8
@@ -2786,14 +2786,6 @@ static int alloc_mmu_pages(struct kvm_vcpu *vcpu)ASSERT (vcpu); -spin_lock (&vcpu->kvm->mmu_lock); -if (vcpu->kvm->arch.n_requested_mmu_pages) - vcpu->kvm->arch.n_free_mmu_pages = - vcpu->kvm->arch.n_requested_mmu_pages; -else - vcpu->kvm->arch.n_free_mmu_pages = - vcpu->kvm->arch.n_alloc_mmu_pages; -spin_unlock (&vcpu->kvm->mmu_lock);/* * When emulating 32-bit mode, cr3 is only 32 bits even on x86_64. * Therefore we need to allocate shadow page tables in the first
Signature · 1 line
1.6.4.1
PATCH 4/47 KVM: x86: Disallow hypercalls for guest callers in rings > 0
In reply to Avi KivityFrom: Jan Kiszka <[email protected]>
So far unprivileged guest callers running in ring 3 can issue, e.g., MMU hypercalls. Normally, such callers cannot provide any hand-crafted MMU command structure as it has to be passed by its physical address, but they can still crash the guest kernel by passing random addresses.
To close the hole, this patch considers hypercalls valid only if issued from guest ring 0. This may still be relaxed on a per-hypercall base in the future once required.
Signed-off-by Jan Kiszka <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/x86.c | 6 ++++++ include/linux/kvm_para.h | 1 + 2 files changed, 7 insertions(+), 0 deletions(-)
arch/x86/kvm/x86.c +6 -0
@@ -3213,6 +3213,11 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) a3 &=0xFFFFFFFF ; } +if (kvm_x86_ops->get_cpl (vcpu) !=0 ) { + ret = -KVM_EPERM ; + goto out; + } +switch (nr) {case KVM_HC_VAPIC_POLL_IRQ : ret =0 ;
@@ -3224,6 +3229,7 @@ int kvm_emulate_hypercall(struct kvm_vcpu *vcpu) ret = -KVM_ENOSYS ;break ; } +out:kvm_register_write (vcpu,VCPU_REGS_RAX , ret); ++vcpu->stat.hypercalls;return r;
include/linux/kvm_para.h +1 -0
@@ -13,6 +13,7 @@#define KVM_ENOSYS 1000#define KVM_EFAULT EFAULT#define KVM_E2BIG E2BIG +#define KVM_EPERM EPERM#define KVM_HC_VAPIC_POLL_IRQ 1#define KVM_HC_MMU_OP 2
Signature · 1 line
1.6.4.1
Re: [PATCH 04/47] KVM: x86: Disallow hypercalls for guest callers in rings > 0
Hi!
Quoted 10 lines · Avi Kivity
From: Jan Kiszka <[email protected]>
So far unprivileged guest callers running in ring 3 can issue, e.g., MMU hypercalls. Normally, such callers cannot provide any hand-crafted MMU command structure as it has to be passed by its physical address, but they can still crash the guest kernel by passing random addresses.
To close the hole, this patch considers hypercalls valid only if issued from guest ring 0. This may still be relaxed on a per-hypercall base in the future once required.
Does kvm-72 (used by Debian and Ubuntu in stable releases) have the problem? If yes, would the approach in this fix also work there?
Thanks,
Jan
Re: [PATCH 04/47] KVM: x86: Disallow hypercalls for guest callers in rings > 0
Quoted 11 lines · Jan Lübbe
Hi!
Quoted 7 lines · Avi Kivity
From: Jan Kiszka<[email protected]>
So far unprivileged guest callers running in ring 3 can issue, e.g., MMU hypercalls. Normally, such callers cannot provide any hand-crafted MMU command structure as it has to be passed by its physical address, but they can still crash the guest kernel by passing random addresses.
Quoted 8 lines
Quoted 4 lines
To close the hole, this patch considers hypercalls valid only if issued from guest ring 0. This may still be relaxed on a per-hypercall base in the future once required.
Does kvm-72 (used by Debian and Ubuntu in stable releases) have the problem? If yes, would the approach in this fix also work there?
Probably yes to both.
Signature · 1 line
error compiling committee.c: too many arguments to function
PATCH 5/47 KVM: VMX: Optimize vmx_get_cpl()
In reply to Avi KivityInstead of calling vmx_get_segment() (which reads a whole bunch of vmcs fields), read only the cs selector which contains the cpl.
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/vmx.c | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-)
arch/x86/kvm/vmx.c +1 -4
@@ -1773,16 +1773,13 @@ static void vmx_get_segment(struct kvm_vcpu *vcpu,static int vmx_get_cpl (struct kvm_vcpu *vcpu) { -struct kvm_segment kvm_seg; -if (!(vcpu->arch.cr0 &X86_CR0_PE ))/* if real mode */ return 0 ;if (vmx_get_rflags (vcpu) &X86_EFLAGS_VM )/* if virtual 8086 */ return 3 ; -vmx_get_segment (vcpu, &kvm_seg,VCPU_SREG_CS ); -return kvm_seg.selector &3 ; +return vmcs_read16 (GUEST_CS_SELECTOR ) &3 ; }static u32 vmx_segment_access_rights (struct kvm_segment *var)
Signature · 1 line
1.6.4.1
Re: [PATCH 05/47] KVM: VMX: Optimize vmx_get_cpl()
Op 26-08-09 12:29, Avi Kivity schreef:
Quoted 4 lines
Instead of calling
vmx_get_segment()(which reads a whole bunch of vmcs fields), read only the cs selector which contains the cpl.Signed-off-by Avi Kivity <[email protected]>
Can't we also optimise cs_ss_rpl_check()? (Please review, untested.)
------------- >8 ---------------------- 8< -------------------- Instead of calling vmx_get_segment() (which reads a whole bunch of vmcs fields), read only the cs/ss selectors which contains the rpls.
Signed-off-by Roel Kluin <[email protected]>
---
arch/x86/kvm/vmx.c +2 -7
@@ -1932,13 +1932,8 @@ static bool ldtr_valid(struct kvm_vcpu *vcpu)static bool cs_ss_rpl_check (struct kvm_vcpu *vcpu) { -struct kvm_segment cs, ss; - -vmx_get_segment (vcpu, &cs,VCPU_SREG_CS ); -vmx_get_segment (vcpu, &ss,VCPU_SREG_SS ); - -return ((cs.selector &SELECTOR_RPL_MASK ) == - (ss.selector &SELECTOR_RPL_MASK )); +return ((vmcs_read16 (GUEST_CS_SELECTOR ) &SELECTOR_RPL_MASK ) == + (vmcs_read16 (GUEST_SS_SELECTOR ) &SELECTOR_RPL_MASK )); }/*
Re: [PATCH 05/47] KVM: VMX: Optimize vmx_get_cpl()
Quoted 5 lines · Roel Kluin
Op 26-08-09 12:29, Avi Kivity schreef:
Quoted 3 lines
Instead of calling
vmx_get_segment()(which reads a whole bunch of vmcs fields), read only the cs selector which contains the cpl.
Quoted 9 lines
Quoted 2 lines
Signed-off-by Avi Kivity<[email protected]>Can't we also optimise
cs_ss_rpl_check()? (Please review, untested.)------------->8 ---------------------- 8< -------------------- Instead of calling
vmx_get_segment()(which reads a whole bunch of vmcs fields), read only the cs/ss selectors which contains the rpls.
It's really a slowpath, so I prefer not to touch it. We're likely to start caching guest segment fields soon, so the less code that reads them directly, the better.
Quoted 23 lines
Signed-off-by Roel Kluin<[email protected]>---
arch/x86/kvm/vmx.c +2
-7@@ -1932,13 +1932,8 @@ static bool ldtr_valid(struct kvm_vcpu *vcpu)static bool cs_ss_rpl_check (struct kvm_vcpu *vcpu) { -struct kvm_segment cs, ss; - -vmx_get_segment (vcpu,&cs,VCPU_SREG_CS ); -vmx_get_segment (vcpu,&ss,VCPU_SREG_SS ); - -return ((cs.selector&SELECTOR_RPL_MASK ) == - (ss.selector&SELECTOR_RPL_MASK )); +return ((vmcs_read16 (GUEST_CS_SELECTOR )&SELECTOR_RPL_MASK ) == + (vmcs_read16 (GUEST_SS_SELECTOR )&SELECTOR_RPL_MASK )); }/*
Signature · 1 line
error compiling committee.c: too many arguments to function
PATCH 6/47 KVM: ignore reads to perfctr msrs
In reply to Avi KivityFrom: Amit Shah <[email protected]>
We ignore writes to the perfctr msrs. Ignore reads as well.
Kaspersky antivirus crashes Windows guests if it can't read these MSRs.
Signed-off-by Amit Shah <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/x86.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
arch/x86/kvm/x86.c +3 -0
@@ -1048,9 +1048,12 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)case MSR_K8_SYSCFG :case MSR_K7_HWCR :case MSR_VM_HSAVE_PA : +case MSR_P6_PERFCTR0 : +case MSR_P6_PERFCTR1 :case MSR_P6_EVNTSEL0 :case MSR_P6_EVNTSEL1 :case MSR_K7_EVNTSEL0 : +case MSR_K7_PERFCTR0 :case MSR_K8_INT_PENDING_MSG :case MSR_AMD64_NB_CFG :case MSR_FAM10H_MMIO_CONF_BASE :
Signature · 1 line
1.6.4.1
PATCH 7/47 KVM: fix EFER read buffer overflow
In reply to Avi KivityFrom: Roel Kluin <[email protected]>
Check whether index is within bounds before grabbing the element.
Signed-off-by Roel Kluin <[email protected]>
Cc Avi Kivity <[email protected]>
Signed-off-by Andrew Morton <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/vmx.c | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx.c +5 -2
@@ -571,12 +571,15 @@ static void reload_tss(void)static void load_transition_efer (struct vcpu_vmx *vmx) {int efer_offset = vmx->msr_offset_efer; -u64 host_efer = vmx->host_msrs[efer_offset].data; -u64 guest_efer = vmx->guest_msrs[efer_offset].data; +u64 host_efer; +u64 guest_efer;u64 ignore_bits;if (efer_offset <0 )return ; + host_efer = vmx->host_msrs[efer_offset].data; + guest_efer = vmx->guest_msrs[efer_offset].data; +/* * NX is emulated; LMA and LME handled by hardware; SCE meaninless * outside long mode
Signature · 1 line
1.6.4.1
PATCH 8/47 KVM: Call kvm_vcpu_kick() inside pic spinlock
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
d5ecfdd25 moved it out because back than it was impossible to call it inside spinlock. This restriction no longer exists.
Signed-off-by Gleb Natapov <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/i8259.c | 10 +--------- arch/x86/kvm/irq.h | 1 - 2 files changed, 1 insertions(+), 10 deletions(-)
arch/x86/kvm/i8259.c +1 -9
@@ -43,11 +43,9 @@ static void pic_unlock(struct kvm_pic *s) {struct kvm *kvm = s->kvm;unsigned acks = s->pending_acks; -bool wakeup = s->wakeup_needed;struct kvm_vcpu *vcpu; s->pending_acks =0 ; - s->wakeup_needed = false;spin_unlock (&s->lock);
@@ -56,12 +54,6 @@ static void pic_unlock(struct kvm_pic *s) __ffs(acks)); acks &= acks -1 ; } - -if (wakeup) { - vcpu = s->kvm->bsp_vcpu; -if (vcpu) -kvm_vcpu_kick (vcpu); - } }static void pic_clear_isr (struct kvm_kpic_state *s,int irq)
@@ -527,7 +519,7 @@ static void pic_irq_request(void *opaque, int level) s->output = level;if (vcpu && level && (s->pics[0 ].isr_ack & (1 << irq))) { s->pics[0 ].isr_ack &= ~(1 << irq); - s->wakeup_needed = true; +kvm_vcpu_kick (vcpu); } }
arch/x86/kvm/irq.h +0 -1
@@ -63,7 +63,6 @@ struct kvm_kpic_state {struct kvm_pic {spinlock_t lock; -bool wakeup_needed;unsigned pending_acks;struct kvm *kvm;struct kvm_kpic_state pics[2 ];/* 0 is master pic, 1 is slave pic */
Signature · 1 line
1.6.4.1
PATCH 9/47 KVM: Call ack notifiers from PIC when guest OS acks an IRQ.
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
Currently they are called when irq vector is been delivered. Calling ack notifiers at this point is wrong. Device assignment ack notifier enables host interrupts, but guest not yet had a chance to clear interrupt condition in a device.
Signed-off-by Gleb Natapov <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/i8259.c | 18 ++++-------------- 1 files changed, 4 insertions(+), 14 deletions(-)
arch/x86/kvm/i8259.c +4 -14
@@ -41,25 +41,16 @@ static void pic_lock(struct kvm_pic *s)static void pic_unlock (struct kvm_pic *s) __releases(&s->lock) { -struct kvm *kvm = s->kvm; -unsigned acks = s->pending_acks; -struct kvm_vcpu *vcpu; - - s->pending_acks =0 ; -spin_unlock (&s->lock); - -while (acks) { -kvm_notify_acked_irq (kvm,SELECT_PIC (__ffs (acks)), -__ffs (acks)); - acks &= acks -1 ; - } }static void pic_clear_isr (struct kvm_kpic_state *s,int irq) { s->isr &= ~(1 << irq); s->isr_ack |= (1 << irq); +if (s != &s->pics_state->pics[0 ]) + irq +=8 ; +kvm_notify_acked_irq (s->pics_state->kvm,SELECT_PIC (irq), irq); }void kvm_pic_clear_isr_ack (struct kvm *kvm)
@@ -240,7 +231,6 @@ int kvm_pic_read_irq(struct kvm *kvm) }pic_update_irq (s);pic_unlock (s); -kvm_notify_acked_irq (kvm,SELECT_PIC (irq), irq);return intno; }
@@ -260,7 +250,7 @@ void kvm_pic_reset(struct kvm_kpic_state *s)if (vcpu0 &&kvm_apic_accept_pic_intr (vcpu0))if (s->irr & (1 << irq) || s->isr & (1 << irq)) { n = irq + irqbase; - s->pics_state->pending_acks |=1 << n; +kvm_notify_acked_irq (kvm,SELECT_PIC (n), n); } } s->last_irr =0 ;
Signature · 1 line
1.6.4.1
PATCH 10/47 KVM: Replace pic_lock()/pic_unlock() with direct call to spinlock functions
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
They are not doing anything else now.
Signed-off-by Gleb Natapov <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/i8259.c | 36 ++++++++++++------------------------ 1 files changed, 12 insertions(+), 24 deletions(-)
arch/x86/kvm/i8259.c +12 -24
@@ -32,18 +32,6 @@#include <linux/kvm_host.h> #include "trace.h" -static void pic_lock (struct kvm_pic *s) - __acquires(&s->lock) -{ -spin_lock (&s->lock); -} - -static void pic_unlock (struct kvm_pic *s) - __releases(&s->lock) -{ -spin_unlock (&s->lock); -} -static void pic_clear_isr (struct kvm_kpic_state *s,int irq) { s->isr &= ~(1 << irq);
@@ -56,10 +44,10 @@ static void pic_clear_isr(struct kvm_kpic_state *s, int irq)void kvm_pic_clear_isr_ack (struct kvm *kvm) {struct kvm_pic *s =pic_irqchip (kvm); -pic_lock (s); +spin_lock (&s->lock); s->pics[0 ].isr_ack =0xff ; s->pics[1 ].isr_ack =0xff ; -pic_unlock (s); +spin_unlock (&s->lock); }/*
@@ -160,9 +148,9 @@ static void pic_update_irq(struct kvm_pic *s)void kvm_pic_update_irq (struct kvm_pic *s) { -pic_lock (s); +spin_lock (&s->lock);pic_update_irq (s); -pic_unlock (s); +spin_unlock (&s->lock); }int kvm_pic_set_irq (void *opaque,int irq,int level)
@@ -170,14 +158,14 @@ int kvm_pic_set_irq(void *opaque, int irq, int level)struct kvm_pic *s = opaque;int ret =-1 ; -pic_lock (s); +spin_lock (&s->lock);if (irq >=0 && irq <PIC_NUM_PINS ) { ret =pic_set_irq1 (&s->pics[irq >>3 ], irq &7 , level);pic_update_irq (s);trace_kvm_pic_set_irq (irq >>3 , irq &7 , s->pics[irq >>3 ].elcr, s->pics[irq >>3 ].imr, ret ==0 ); } -pic_unlock (s); +spin_unlock (&s->lock);return ret; }
@@ -205,7 +193,7 @@ int kvm_pic_read_irq(struct kvm *kvm)int irq, irq2, intno;struct kvm_pic *s =pic_irqchip (kvm); -pic_lock (s); +spin_lock (&s->lock); irq =pic_get_irq (&s->pics[0 ]);if (irq >=0 ) {pic_intack (&s->pics[0 ], irq);
@@ -230,7 +218,7 @@ int kvm_pic_read_irq(struct kvm *kvm) intno = s->pics[0 ].irq_base + irq; }pic_update_irq (s); -pic_unlock (s); +spin_unlock (&s->lock);return intno; }
@@ -448,7 +436,7 @@ static int picdev_write(struct kvm_io_device *this,printk (KERN_ERR "PIC: non byte write\n" );return 0 ; } -pic_lock (s); +spin_lock (&s->lock);switch (addr) {case 0x20 :case 0x21 :
@@ -461,7 +449,7 @@ static int picdev_write(struct kvm_io_device *this,elcr_ioport_write (&s->pics[addr &1 ], addr, data);break ; } -pic_unlock (s); +spin_unlock (&s->lock);return 0 ; }
@@ -478,7 +466,7 @@ static int picdev_read(struct kvm_io_device *this,printk (KERN_ERR "PIC: non byte read\n" );return 0 ; } -pic_lock (s); +spin_lock (&s->lock);switch (addr) {case 0x20 :case 0x21 :
@@ -492,7 +480,7 @@ static int picdev_read(struct kvm_io_device *this,break ; } *(unsigned char *)val = data; -pic_unlock (s); +spin_unlock (&s->lock);return 0 ; }
Signature · 1 line
1.6.4.1
PATCH 11/47 x86: Export kmap_atomic_to_page()
In reply to Avi KivityNeeded by KVM.
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/mm/highmem_32.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-)
arch/x86/mm/highmem_32.c +1 -0
@@ -104,6 +104,7 @@ EXPORT_SYMBOL(kunmap);EXPORT_SYMBOL (kmap_atomic);EXPORT_SYMBOL (kunmap_atomic);EXPORT_SYMBOL (kmap_atomic_prot); +EXPORT_SYMBOL (kmap_atomic_to_page);void __init set_highmem_pages_init (void ) {
Signature · 1 line
1.6.4.1
PATCH 12/47 KVM: SVM: add helper functions for global interrupt flag
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
This patch makes the code easier to read when it comes to setting, clearing and checking the status of the virtualized global interrupt flag for the VCPU.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 33 +++++++++++++++++++++++++-------- 1 files changed, 25 insertions(+), 8 deletions(-)
arch/x86/kvm/svm.c +25 -8
@@ -129,6 +129,21 @@ static inline bool is_nested(struct vcpu_svm *svm)return svm->nested_vmcb; } +static inline void enable_gif (struct vcpu_svm *svm) +{ + svm->vcpu.arch.hflags |=HF_GIF_MASK ; +} + +static inline void disable_gif (struct vcpu_svm *svm) +{ + svm->vcpu.arch.hflags &= ~HF_GIF_MASK ; +} + +static inline bool gif_set (struct vcpu_svm *svm) +{ +return !!(svm->vcpu.arch.hflags &HF_GIF_MASK ); +} +static unsigned long iopm_base;struct kvm_ldttss_desc {
@@ -621,7 +636,9 @@ static void init_vmcb(struct vcpu_svm *svm)force_new_asid (&svm->vcpu); svm->nested_vmcb =0 ; - svm->vcpu.arch.hflags =HF_GIF_MASK ; + svm->vcpu.arch.hflags =0 ; + +enable_gif (svm); }static int svm_vcpu_reset (struct kvm_vcpu *vcpu)
@@ -1629,7 +1646,7 @@ static int nested_svm_vmexit_real(struct vcpu_svm *svm, void *arg1, svm->vmcb->save.cpl =0 ; svm->vmcb->control.exit_int_info =0 ; - svm->vcpu.arch.hflags &= ~HF_GIF_MASK ; +disable_gif (svm);/* Exit nested SVM mode */ svm->nested_vmcb =0 ;
@@ -1761,7 +1778,7 @@ static int nested_svm_vmrun(struct vcpu_svm *svm, void *arg1, svm->vmcb->control.event_inj = nested_vmcb->control.event_inj; svm->vmcb->control.event_inj_err = nested_vmcb->control.event_inj_err; - svm->vcpu.arch.hflags |=HF_GIF_MASK ; +enable_gif (svm);return 0 ; }
@@ -1850,7 +1867,7 @@ static int stgi_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) svm->next_rip =kvm_rip_read (&svm->vcpu) +3 ;skip_emulated_instruction (&svm->vcpu); - svm->vcpu.arch.hflags |=HF_GIF_MASK ; +enable_gif (svm);return 1 ; }
@@ -1863,7 +1880,7 @@ static int clgi_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) svm->next_rip =kvm_rip_read (&svm->vcpu) +3 ;skip_emulated_instruction (&svm->vcpu); - svm->vcpu.arch.hflags &= ~HF_GIF_MASK ; +disable_gif (svm);/* After a CLGI no interrupts should come */ svm_clear_vintr (svm);
@@ -2352,7 +2369,7 @@ static void svm_set_irq(struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu); -BUG_ON (!(svm->vcpu.arch.hflags &HF_GIF_MASK )); +BUG_ON (!(gif_set (svm))); svm->vmcb->control.event_inj = vcpu->arch.interrupt.nr |SVM_EVTINJ_VALID |SVM_EVTINJ_TYPE_INTR ;
@@ -2383,7 +2400,7 @@ static int svm_interrupt_allowed(struct kvm_vcpu *vcpu)struct vmcb *vmcb = svm->vmcb;return (vmcb->save.rflags &X86_EFLAGS_IF ) && !(vmcb->control.int_state &SVM_INTERRUPT_SHADOW_MASK ) && - (svm->vcpu.arch.hflags &HF_GIF_MASK ) && +gif_set (svm) && !is_nested (svm); }
@@ -2398,7 +2415,7 @@ static void enable_irq_window(struct kvm_vcpu *vcpu)* GIF becomes 1, because that's a separate STGI/VMRUN intercept. * The next time we get that intercept, this function will be * called again though and we'll get the vintr intercept. */ -if (svm->vcpu.arch.hflags &HF_GIF_MASK ) { +if (gif_set (svm)) {svm_set_vintr (svm);svm_inject_irq (svm,0x0 ); }
Signature · 1 line
1.6.4.1
PATCH 13/47 KVM: SVM: optimize nested #vmexit
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
It is more efficient to copy only the relevant parts of the vmcb back to the nested vmcb when we emulate an vmexit.
Signed-off-by Joerg Roedel <[email protected]>
Acked-by Alexander Graf <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 68 +++++++++++++++++++++++++-------------------------- 1 files changed, 33 insertions(+), 35 deletions(-)
arch/x86/kvm/svm.c +33 -35
@@ -1572,53 +1572,52 @@ static int nested_svm_vmexit_real(struct vcpu_svm *svm, void *arg1, {struct vmcb *nested_vmcb = (struct vmcb *)arg1;struct vmcb *hsave = svm->hsave; -u64 nested_save[] = { nested_vmcb->save.cr0, - nested_vmcb->save.cr3, - nested_vmcb->save.cr4, - nested_vmcb->save.efer, - nested_vmcb->control.intercept_cr_read, - nested_vmcb->control.intercept_cr_write, - nested_vmcb->control.intercept_dr_read, - nested_vmcb->control.intercept_dr_write, - nested_vmcb->control.intercept_exceptions, - nested_vmcb->control.intercept, - nested_vmcb->control.msrpm_base_pa, - nested_vmcb->control.iopm_base_pa, - nested_vmcb->control.tsc_offset }; +struct vmcb *vmcb = svm->vmcb;/* Give the current vmcb to the guest */ -memcpy (nested_vmcb, svm->vmcb,sizeof (struct vmcb )); - nested_vmcb->save.cr0 = nested_save[0 ]; -if (!npt_enabled) - nested_vmcb->save.cr3 = nested_save[1 ]; - nested_vmcb->save.cr4 = nested_save[2 ]; - nested_vmcb->save.efer = nested_save[3 ]; - nested_vmcb->control.intercept_cr_read = nested_save[4 ]; - nested_vmcb->control.intercept_cr_write = nested_save[5 ]; - nested_vmcb->control.intercept_dr_read = nested_save[6 ]; - nested_vmcb->control.intercept_dr_write = nested_save[7 ]; - nested_vmcb->control.intercept_exceptions = nested_save[8 ]; - nested_vmcb->control.intercept = nested_save[9 ]; - nested_vmcb->control.msrpm_base_pa = nested_save[10 ]; - nested_vmcb->control.iopm_base_pa = nested_save[11 ]; - nested_vmcb->control.tsc_offset = nested_save[12 ]; +disable_gif (svm); + + nested_vmcb->save.es = vmcb->save.es; + nested_vmcb->save.cs = vmcb->save.cs; + nested_vmcb->save.ss = vmcb->save.ss; + nested_vmcb->save.ds = vmcb->save.ds; + nested_vmcb->save.gdtr = vmcb->save.gdtr; + nested_vmcb->save.idtr = vmcb->save.idtr; +if (npt_enabled) + nested_vmcb->save.cr3 = vmcb->save.cr3; + nested_vmcb->save.cr2 = vmcb->save.cr2; + nested_vmcb->save.rflags = vmcb->save.rflags; + nested_vmcb->save.rip = vmcb->save.rip; + nested_vmcb->save.rsp = vmcb->save.rsp; + nested_vmcb->save.rax = vmcb->save.rax; + nested_vmcb->save.dr7 = vmcb->save.dr7; + nested_vmcb->save.dr6 = vmcb->save.dr6; + nested_vmcb->save.cpl = vmcb->save.cpl; + + nested_vmcb->control.int_ctl = vmcb->control.int_ctl; + nested_vmcb->control.int_vector = vmcb->control.int_vector; + nested_vmcb->control.int_state = vmcb->control.int_state; + nested_vmcb->control.exit_code = vmcb->control.exit_code; + nested_vmcb->control.exit_code_hi = vmcb->control.exit_code_hi; + nested_vmcb->control.exit_info_1 = vmcb->control.exit_info_1; + nested_vmcb->control.exit_info_2 = vmcb->control.exit_info_2; + nested_vmcb->control.exit_int_info = vmcb->control.exit_int_info; + nested_vmcb->control.exit_int_info_err = vmcb->control.exit_int_info_err; + nested_vmcb->control.tlb_ctl =0 ; + nested_vmcb->control.event_inj =0 ; + nested_vmcb->control.event_inj_err =0 ;/* We always set V_INTR_MASKING and remember the old value in hflags */ if (!(svm->vcpu.arch.hflags &HF_VINTR_MASK )) nested_vmcb->control.int_ctl &= ~V_INTR_MASKING_MASK ; -if ((nested_vmcb->control.int_ctl &V_IRQ_MASK ) && - (nested_vmcb->control.int_vector)) { -nsvm_printk ("WARNING: IRQ 0x%x still enabled on #VMEXIT\n" , - nested_vmcb->control.int_vector); - } -/* Restore the original control entries */ svm->vmcb->control = hsave->control;/* Kill any pending exceptions */ if (svm->vcpu.arch.exception.pending == true)nsvm_printk ("WARNING: Pending Exception\n" ); +kvm_clear_exception_queue (&svm->vcpu);kvm_clear_interrupt_queue (&svm->vcpu);
@@ -1646,7 +1645,6 @@ static int nested_svm_vmexit_real(struct vcpu_svm *svm, void *arg1, svm->vmcb->save.cpl =0 ; svm->vmcb->control.exit_int_info =0 ; -disable_gif (svm);/* Exit nested SVM mode */ svm->nested_vmcb =0 ;
Signature · 1 line
1.6.4.1
PATCH 14/47 KVM: SVM: optimize nested vmrun
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
Only copy the necessary parts of the vmcb save area on vmrun and save precious time.
Signed-off-by Joerg Roedel <[email protected]>
Acked-by Alexander Graf <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 28 +++++++++++++++++++++------- 1 files changed, 21 insertions(+), 7 deletions(-)
arch/x86/kvm/svm.c +21 -7
@@ -1681,6 +1681,7 @@ static int nested_svm_vmrun(struct vcpu_svm *svm, void *arg1, {struct vmcb *nested_vmcb = (struct vmcb *)arg1;struct vmcb *hsave = svm->hsave; +struct vmcb *vmcb = svm->vmcb;/* nested_vmcb is our indicator if nested SVM is activated */ svm->nested_vmcb = svm->vmcb->save.rax;
@@ -1691,12 +1692,25 @@ static int nested_svm_vmrun(struct vcpu_svm *svm, void *arg1,/* Save the old vmcb, so we don't need to pick what we save, but can restore everything when a VMEXIT occurs */ -memcpy (hsave, svm->vmcb,sizeof (struct vmcb )); -/* We need to remember the original CR3 in the SPT case */ -if (!npt_enabled) - hsave->save.cr3 = svm->vcpu.arch.cr3; - hsave->save.cr4 = svm->vcpu.arch.cr4; - hsave->save.rip = svm->next_rip; + hsave->save.es = vmcb->save.es; + hsave->save.cs = vmcb->save.cs; + hsave->save.ss = vmcb->save.ss; + hsave->save.ds = vmcb->save.ds; + hsave->save.gdtr = vmcb->save.gdtr; + hsave->save.idtr = vmcb->save.idtr; + hsave->save.efer = svm->vcpu.arch.shadow_efer; + hsave->save.cr0 = svm->vcpu.arch.cr0; + hsave->save.cr4 = svm->vcpu.arch.cr4; + hsave->save.rflags = vmcb->save.rflags; + hsave->save.rip = svm->next_rip; + hsave->save.rsp = vmcb->save.rsp; + hsave->save.rax = vmcb->save.rax; +if (npt_enabled) + hsave->save.cr3 = vmcb->save.cr3; +else + hsave->save.cr3 = svm->vcpu.arch.cr3; + + hsave->control = vmcb->control;if (svm->vmcb->save.rflags &X86_EFLAGS_IF ) svm->vcpu.arch.hflags |=HF_HIF_MASK ;
@@ -1721,7 +1735,7 @@ static int nested_svm_vmrun(struct vcpu_svm *svm, void *arg1,kvm_set_cr3 (&svm->vcpu, nested_vmcb->save.cr3);kvm_mmu_reset_context (&svm->vcpu); } - svm->vmcb->save.cr2 = nested_vmcb->save.cr2; + svm->vmcb->save.cr2 = svm->vcpu.arch.cr2 = nested_vmcb->save.cr2;kvm_register_write (&svm->vcpu,VCPU_REGS_RAX , nested_vmcb->save.rax);kvm_register_write (&svm->vcpu,VCPU_REGS_RSP , nested_vmcb->save.rsp);kvm_register_write (&svm->vcpu,VCPU_REGS_RIP , nested_vmcb->save.rip);
Signature · 1 line
1.6.4.1
PATCH 15/47 KVM: SVM: copy only necessary parts of the control area on vmrun/vmexit
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
The vmcb control area contains more then 800 bytes of reserved fields which are unnecessarily copied. Fix this by introducing a copy function which only copies the relevant part and saves time.
Signed-off-by Joerg Roedel <[email protected]>
Acked-by Alexander Graf <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 36 ++++++++++++++++++++++++++++++++++-- 1 files changed, 34 insertions(+), 2 deletions(-)
arch/x86/kvm/svm.c +34 -2
@@ -1567,6 +1567,38 @@ static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override) nested_svm_exit_handled_real); } +static inline void copy_vmcb_control_area (struct vmcb *dst_vmcb,struct vmcb *from_vmcb) +{ +struct vmcb_control_area *dst = &dst_vmcb->control; +struct vmcb_control_area *from = &from_vmcb->control; + + dst->intercept_cr_read = from->intercept_cr_read; + dst->intercept_cr_write = from->intercept_cr_write; + dst->intercept_dr_read = from->intercept_dr_read; + dst->intercept_dr_write = from->intercept_dr_write; + dst->intercept_exceptions = from->intercept_exceptions; + dst->intercept = from->intercept; + dst->iopm_base_pa = from->iopm_base_pa; + dst->msrpm_base_pa = from->msrpm_base_pa; + dst->tsc_offset = from->tsc_offset; + dst->asid = from->asid; + dst->tlb_ctl = from->tlb_ctl; + dst->int_ctl = from->int_ctl; + dst->int_vector = from->int_vector; + dst->int_state = from->int_state; + dst->exit_code = from->exit_code; + dst->exit_code_hi = from->exit_code_hi; + dst->exit_info_1 = from->exit_info_1; + dst->exit_info_2 = from->exit_info_2; + dst->exit_int_info = from->exit_int_info; + dst->exit_int_info_err = from->exit_int_info_err; + dst->nested_ctl = from->nested_ctl; + dst->event_inj = from->event_inj; + dst->event_inj_err = from->event_inj_err; + dst->nested_cr3 = from->nested_cr3; + dst->lbr_ctl = from->lbr_ctl; +} +static int nested_svm_vmexit_real (struct vcpu_svm *svm,void *arg1,void *arg2,void *opaque) {
@@ -1612,7 +1644,7 @@ static int nested_svm_vmexit_real(struct vcpu_svm *svm, void *arg1, nested_vmcb->control.int_ctl &= ~V_INTR_MASKING_MASK ;/* Restore the original control entries */ - svm->vmcb->control = hsave->control; +copy_vmcb_control_area (vmcb, hsave);/* Kill any pending exceptions */ if (svm->vcpu.arch.exception.pending == true)
@@ -1710,7 +1742,7 @@ static int nested_svm_vmrun(struct vcpu_svm *svm, void *arg1,else hsave->save.cr3 = svm->vcpu.arch.cr3; - hsave->control = vmcb->control; +copy_vmcb_control_area (hsave, vmcb);if (svm->vmcb->save.rflags &X86_EFLAGS_IF ) svm->vcpu.arch.hflags |=HF_HIF_MASK ;
Signature · 1 line
1.6.4.1
PATCH 16/47 KVM: SVM: complete interrupts after handling nested exits
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
The interrupt completion code must run after nested exits are handled because not injected interrupts or exceptions may be handled by the l1 guest first.
Signed-off-by Joerg Roedel <[email protected]>
Acked-by Alexander Graf <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-)
arch/x86/kvm/svm.c +3 -2
@@ -111,6 +111,7 @@ static int nested = 0;module_param (nested, int,S_IRUGO );static void svm_flush_tlb (struct kvm_vcpu *vcpu); +static void svm_complete_interrupts (struct vcpu_svm *svm);static int nested_svm_exit_handled (struct vcpu_svm *svm,bool kvm_override);static int nested_svm_vmexit (struct vcpu_svm *svm);
@@ -2324,6 +2325,8 @@ static int handle_exit(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu) } } +svm_complete_interrupts (svm); +if (npt_enabled) {int mmu_reload =0 ;if ((vcpu->arch.cr0 ^ svm->vmcb->save.cr0) &X86_CR0_PG ) {
@@ -2690,8 +2693,6 @@ static void svm_vcpu_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) vcpu->arch.regs_avail &= ~(1 <<VCPU_EXREG_PDPTR ); vcpu->arch.regs_dirty &= ~(1 <<VCPU_EXREG_PDPTR ); } - -svm_complete_interrupts (svm); }#undef R
Signature · 1 line
1.6.4.1
PATCH 17/47 KVM: SVM: move nested svm state into seperate struct
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
This makes it more clear for which purpose these members in the vcpu_svm exist.
Signed-off-by Joerg Roedel <[email protected]>
Acked-by Alexander Graf <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 62 +++++++++++++++++++++++++++------------------------ 1 files changed, 33 insertions(+), 29 deletions(-)
arch/x86/kvm/svm.c +33 -29
@@ -70,6 +70,18 @@ static const u32 host_save_user_msrs[] = {struct kvm_vcpu ; +struct nested_state { +struct vmcb *hsave; +u64 hsave_msr; +u64 vmcb; + +/* These are the merged vectors */ +u32 *msrpm; + +/* gpa pointers to the real vectors */ +u64 vmcb_msrpm; +}; +struct vcpu_svm {struct kvm_vcpu vcpu;struct vmcb *vmcb;
@@ -85,16 +97,8 @@ struct vcpu_svm {u64 host_gs_base;u32 *msrpm; -struct vmcb *hsave; -u64 hsave_msr; - -u64 nested_vmcb; -/* These are the merged vectors */ -u32 *nested_msrpm; - -/* gpa pointers to the real vectors */ -u64 nested_vmcb_msrpm; +struct nested_state nested; };/* enable NPT for AMD64 and X86 with PAE */
@@ -127,7 +131,7 @@ static inline struct vcpu_svm *to_svm(struct kvm_vcpu *vcpu)static inline bool is_nested (struct vcpu_svm *svm) { -return svm->nested_vmcb; +return svm->nested.vmcb; }static inline void enable_gif (struct vcpu_svm *svm)
@@ -636,7 +640,7 @@ static void init_vmcb(struct vcpu_svm *svm) }force_new_asid (&svm->vcpu); - svm->nested_vmcb =0 ; + svm->nested.vmcb =0 ; svm->vcpu.arch.hflags =0 ;enable_gif (svm);
@@ -699,9 +703,9 @@ static struct kvm_vcpu *svm_create_vcpu(struct kvm *kvm, unsigned int id) hsave_page =alloc_page (GFP_KERNEL );if (!hsave_page) goto uninit; - svm->hsave =page_address (hsave_page); + svm->nested.hsave =page_address (hsave_page); - svm->nested_msrpm =page_address (nested_msrpm_pages); + svm->nested.msrpm =page_address (nested_msrpm_pages); svm->vmcb =page_address (page);clear_page (svm->vmcb);
@@ -731,8 +735,8 @@ static void svm_free_vcpu(struct kvm_vcpu *vcpu)__free_page (pfn_to_page (svm->vmcb_pa >>PAGE_SHIFT ));__free_pages (virt_to_page (svm->msrpm),MSRPM_ALLOC_ORDER ); -__free_page (virt_to_page (svm->hsave)); -__free_pages (virt_to_page (svm->nested_msrpm),MSRPM_ALLOC_ORDER ); +__free_page (virt_to_page (svm->nested.hsave)); +__free_pages (virt_to_page (svm->nested.msrpm),MSRPM_ALLOC_ORDER );kvm_vcpu_uninit (vcpu);kmem_cache_free (kvm_vcpu_cache, svm); }
@@ -1558,13 +1562,13 @@ static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override)switch (svm->vmcb->control.exit_code) {case SVM_EXIT_MSR : -return nested_svm_do (svm, svm->nested_vmcb, - svm->nested_vmcb_msrpm,NULL , +return nested_svm_do (svm, svm->nested.vmcb, + svm->nested.vmcb_msrpm,NULL , nested_svm_exit_handled_msr);default :break ; } -return nested_svm_do (svm, svm->nested_vmcb,0 , &k, +return nested_svm_do (svm, svm->nested.vmcb,0 , &k, nested_svm_exit_handled_real); }
@@ -1604,7 +1608,7 @@ static int nested_svm_vmexit_real(struct vcpu_svm *svm, void *arg1, void *arg2, void *opaque) { struct vmcb *nested_vmcb = (struct vmcb *)arg1; -struct vmcb *hsave = svm->hsave; +struct vmcb *hsave = svm->nested.hsave;struct vmcb *vmcb = svm->vmcb;/* Give the current vmcb to the guest */
@@ -1679,7 +1683,7 @@ static int nested_svm_vmexit_real(struct vcpu_svm *svm, void *arg1, svm->vmcb->control.exit_int_info =0 ;/* Exit nested SVM mode */ - svm->nested_vmcb =0 ; + svm->nested.vmcb =0 ;return 0 ; }
@@ -1687,7 +1691,7 @@ static int nested_svm_vmexit_real(struct vcpu_svm *svm, void *arg1,static int nested_svm_vmexit (struct vcpu_svm *svm) {nsvm_printk ("VMexit\n" ); -if (nested_svm_do (svm, svm->nested_vmcb,0 , +if (nested_svm_do (svm, svm->nested.vmcb,0 ,NULL , nested_svm_vmexit_real))return 1 ;
@@ -1703,8 +1707,8 @@ static int nested_svm_vmrun_msrpm(struct vcpu_svm *svm, void *arg1,int i;u32 *nested_msrpm = (u32 *)arg1;for (i=0 ; i<PAGE_SIZE * (1 <<MSRPM_ALLOC_ORDER ) /4 ; i++) - svm->nested_msrpm[i] = svm->msrpm[i] | nested_msrpm[i]; - svm->vmcb->control.msrpm_base_pa =__pa (svm->nested_msrpm); + svm->nested.msrpm[i] = svm->msrpm[i] | nested_msrpm[i]; + svm->vmcb->control.msrpm_base_pa =__pa (svm->nested.msrpm);return 0 ; }
@@ -1713,11 +1717,11 @@ static int nested_svm_vmrun(struct vcpu_svm *svm, void *arg1, void *arg2, void *opaque) { struct vmcb *nested_vmcb = (struct vmcb *)arg1; -struct vmcb *hsave = svm->hsave; +struct vmcb *hsave = svm->nested.hsave;struct vmcb *vmcb = svm->vmcb;/* nested_vmcb is our indicator if nested SVM is activated */ - svm->nested_vmcb = svm->vmcb->save.rax; + svm->nested.vmcb = svm->vmcb->save.rax;/* Clear internal status */ kvm_clear_exception_queue (&svm->vcpu);
@@ -1795,7 +1799,7 @@ static int nested_svm_vmrun(struct vcpu_svm *svm, void *arg1, svm->vmcb->control.intercept |= nested_vmcb->control.intercept; - svm->nested_vmcb_msrpm = nested_vmcb->control.msrpm_base_pa; + svm->nested.vmcb_msrpm = nested_vmcb->control.msrpm_base_pa;force_new_asid (&svm->vcpu); svm->vmcb->control.exit_int_info = nested_vmcb->control.exit_int_info;
@@ -1897,7 +1901,7 @@ static int vmrun_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) NULL, nested_svm_vmrun))return 1 ; -if (nested_svm_do (svm, svm->nested_vmcb_msrpm,0 , +if (nested_svm_do (svm, svm->nested.vmcb_msrpm,0 ,NULL , nested_svm_vmrun_msrpm))return 1 ;
@@ -2107,7 +2111,7 @@ static int svm_get_msr(struct kvm_vcpu *vcpu, unsigned ecx, u64 *data) *data = svm->vmcb->save.last_excp_to;break ;case MSR_VM_HSAVE_PA : - *data = svm->hsave_msr; + *data = svm->nested.hsave_msr;break ;case MSR_VM_CR : *data =0 ;
@@ -2195,7 +2199,7 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, unsigned ecx, u64 data)svm_disable_lbrv (svm);break ;case MSR_VM_HSAVE_PA : - svm->hsave_msr = data; + svm->nested.hsave_msr = data;break ;case MSR_VM_CR :case MSR_VM_IGNNE :
Signature · 1 line
1.6.4.1
PATCH 18/47 KVM: SVM: cache nested intercepts
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
When the nested intercepts are cached we don't need to call get_user_pages and/or map the nested vmcb on every nested #vmexit to check who will handle the intercept.
Further this patch aligns the emulated svm behavior better to real hardware.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 30 +++++++++++++++++++++++------- 1 files changed, 23 insertions(+), 7 deletions(-)
arch/x86/kvm/svm.c +23 -7
@@ -80,6 +80,15 @@ struct nested_state {/* gpa pointers to the real vectors */ u64 vmcb_msrpm; + +/* cache for intercepts of the guest */ +u16 intercept_cr_read; +u16 intercept_cr_write; +u16 intercept_dr_read; +u16 intercept_dr_write; +u32 intercept_exceptions; +u64 intercept; + };struct vcpu_svm {
@@ -1452,7 +1461,6 @@ static int nested_svm_exit_handled_real(struct vcpu_svm *svm,void *arg2, void *opaque) { - struct vmcb *nested_vmcb = (struct vmcb *)arg1; bool kvm_overrides = *(bool *)opaque;u32 exit_code = svm->vmcb->control.exit_code;
@@ -1479,38 +1487,38 @@ static int nested_svm_exit_handled_real(struct vcpu_svm *svm,switch (exit_code) { case SVM_EXIT_READ_CR0 ... SVM_EXIT_READ_CR8: {u32 cr_bits =1 << (exit_code -SVM_EXIT_READ_CR0 ); -if (nested_vmcb->control.intercept_cr_read & cr_bits) +if (svm->nested.intercept_cr_read & cr_bits)return 1 ;break ; } case SVM_EXIT_WRITE_CR0 ... SVM_EXIT_WRITE_CR8: {u32 cr_bits =1 << (exit_code -SVM_EXIT_WRITE_CR0 ); -if (nested_vmcb->control.intercept_cr_write & cr_bits) +if (svm->nested.intercept_cr_write & cr_bits)return 1 ;break ; } case SVM_EXIT_READ_DR0 ... SVM_EXIT_READ_DR7: {u32 dr_bits =1 << (exit_code -SVM_EXIT_READ_DR0 ); -if (nested_vmcb->control.intercept_dr_read & dr_bits) +if (svm->nested.intercept_dr_read & dr_bits)return 1 ;break ; } case SVM_EXIT_WRITE_DR0 ... SVM_EXIT_WRITE_DR7: {u32 dr_bits =1 << (exit_code -SVM_EXIT_WRITE_DR0 ); -if (nested_vmcb->control.intercept_dr_write & dr_bits) +if (svm->nested.intercept_dr_write & dr_bits)return 1 ;break ; } case SVM_EXIT_EXCP_BASE ... SVM_EXIT_EXCP_BASE + 0x1f: {u32 excp_bits =1 << (exit_code -SVM_EXIT_EXCP_BASE ); -if (nested_vmcb->control.intercept_exceptions & excp_bits) +if (svm->nested.intercept_exceptions & excp_bits)return 1 ;break ; }default : {u64 exit_bits =1ULL << (exit_code -SVM_EXIT_INTR );nsvm_printk ("exit code: 0x%x\n" , exit_code); -if (nested_vmcb->control.intercept & exit_bits) +if (svm->nested.intercept & exit_bits)return 1 ; } }
@@ -1801,6 +1809,14 @@ static int nested_svm_vmrun(struct vcpu_svm *svm, void *arg1, svm->nested.vmcb_msrpm = nested_vmcb->control.msrpm_base_pa; +/* cache intercepts */ + svm->nested.intercept_cr_read = nested_vmcb->control.intercept_cr_read; + svm->nested.intercept_cr_write = nested_vmcb->control.intercept_cr_write; + svm->nested.intercept_dr_read = nested_vmcb->control.intercept_dr_read; + svm->nested.intercept_dr_write = nested_vmcb->control.intercept_dr_write; + svm->nested.intercept_exceptions = nested_vmcb->control.intercept_exceptions; + svm->nested.intercept = nested_vmcb->control.intercept; +force_new_asid (&svm->vcpu); svm->vmcb->control.exit_int_info = nested_vmcb->control.exit_int_info; svm->vmcb->control.exit_int_info_err = nested_vmcb->control.exit_int_info_err;
Signature · 1 line
1.6.4.1
PATCH 19/47 KVM: SVM: consolidate nested_svm_exit_handled
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
When caching guest intercepts there is no need anymore for the nested_svm_exit_handled_real function. So move its code into nested_svm_exit_handled.
Signed-off-by Joerg Roedel <[email protected]>
Acked-by Alexander Graf <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 109 +++++++++++++++++++++++---------------------------- 1 files changed, 49 insertions(+), 60 deletions(-)
arch/x86/kvm/svm.c +49 -60
@@ -1456,15 +1456,58 @@ static int nested_svm_do(struct vcpu_svm *svm,return retval; } -static int nested_svm_exit_handled_real (struct vcpu_svm *svm, -void *arg1, -void *arg2, -void *opaque) +static int nested_svm_exit_handled_msr (struct vcpu_svm *svm, +void *arg1,void *arg2, +void *opaque) +{ +struct vmcb *nested_vmcb = (struct vmcb *)arg1; +u8 *msrpm = (u8 *)arg2; +u32 t0, t1; +u32 msr = svm->vcpu.arch.regs[VCPU_REGS_RCX ]; +u32 param = svm->vmcb->control.exit_info_1 &1 ; + +if (!(nested_vmcb->control.intercept & (1ULL <<INTERCEPT_MSR_PROT ))) +return 0 ; + +switch (msr) { + case 0 ... 0x1fff: + t0 = (msr *2 ) %8 ; + t1 = msr /8 ; +break ; + case 0xc0000000 ... 0xc0001fff: + t0 = (8192 + msr -0xc0000000 ) *2 ; + t1 = (t0 /8 ); + t0 %=8 ; +break ; + case 0xc0010000 ... 0xc0011fff: + t0 = (16384 + msr -0xc0010000 ) *2 ; + t1 = (t0 /8 ); + t0 %=8 ; +break ; +default : +return 1 ; +break ; + } +if (msrpm[t1] & ((1 << param) << t0)) +return 1 ; + +return 0 ; +} + +static int nested_svm_exit_handled (struct vcpu_svm *svm,bool kvm_override) { -bool kvm_overrides = *(bool *)opaque;u32 exit_code = svm->vmcb->control.exit_code; -if (kvm_overrides) { +switch (svm->vmcb->control.exit_code) { +case SVM_EXIT_MSR : +return nested_svm_do (svm, svm->nested.vmcb, + svm->nested.vmcb_msrpm,NULL , + nested_svm_exit_handled_msr); +default : +break ; + } + +if (kvm_override) {switch (exit_code) {case SVM_EXIT_INTR :case SVM_EXIT_NMI :
@@ -1526,60 +1569,6 @@ static int nested_svm_exit_handled_real(struct vcpu_svm *svm,return 0 ; } -static int nested_svm_exit_handled_msr (struct vcpu_svm *svm, -void *arg1,void *arg2, -void *opaque) -{ -struct vmcb *nested_vmcb = (struct vmcb *)arg1; -u8 *msrpm = (u8 *)arg2; -u32 t0, t1; -u32 msr = svm->vcpu.arch.regs[VCPU_REGS_RCX ]; -u32 param = svm->vmcb->control.exit_info_1 &1 ; - -if (!(nested_vmcb->control.intercept & (1ULL <<INTERCEPT_MSR_PROT ))) -return 0 ; - -switch (msr) { - case 0 ... 0x1fff: - t0 = (msr *2 ) %8 ; - t1 = msr /8 ; -break ; - case 0xc0000000 ... 0xc0001fff: - t0 = (8192 + msr -0xc0000000 ) *2 ; - t1 = (t0 /8 ); - t0 %=8 ; -break ; - case 0xc0010000 ... 0xc0011fff: - t0 = (16384 + msr -0xc0010000 ) *2 ; - t1 = (t0 /8 ); - t0 %=8 ; -break ; -default : -return 1 ; -break ; - } -if (msrpm[t1] & ((1 << param) << t0)) -return 1 ; - -return 0 ; -} - -static int nested_svm_exit_handled (struct vcpu_svm *svm,bool kvm_override) -{ -bool k = kvm_override; - -switch (svm->vmcb->control.exit_code) { -case SVM_EXIT_MSR : -return nested_svm_do (svm, svm->nested.vmcb, - svm->nested.vmcb_msrpm,NULL , - nested_svm_exit_handled_msr); -default :break ; - } - -return nested_svm_do (svm, svm->nested.vmcb,0 , &k, - nested_svm_exit_handled_real); -} -static inline void copy_vmcb_control_area (struct vmcb *dst_vmcb,struct vmcb *from_vmcb) {struct vmcb_control_area *dst = &dst_vmcb->control;
Signature · 1 line
1.6.4.1
PATCH 20/47 KVM: SVM: do nested vmexit in nested_svm_exit_handled
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
If this function returns true a nested vmexit is required. Move that vmexit into the nested_svm_exit_handled function. This also simplifies the handling of nested #pf intercepts in this function.
Signed-off-by Joerg Roedel <[email protected]>
Acked-by Alexander Graf <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 42 +++++++++++++++++++----------------------- 1 files changed, 19 insertions(+), 23 deletions(-)
arch/x86/kvm/svm.c +19 -23
@@ -1366,8 +1366,6 @@ static int nested_svm_check_exception(struct vcpu_svm *svm, unsigned nr, svm->vmcb->control.exit_info_2 = svm->vcpu.arch.cr2;if (nested_svm_exit_handled (svm, false)) {nsvm_printk ("VMexit -> EXCP 0x%x\n" , nr); - -nested_svm_vmexit (svm);return 1 ; } }
@@ -1388,7 +1386,6 @@ static inline int nested_svm_intr(struct vcpu_svm *svm)if (nested_svm_exit_handled (svm, false)) {nsvm_printk ("VMexit -> INTR\n" ); -nested_svm_vmexit (svm);return 1 ; } }
@@ -1497,15 +1494,7 @@ static int nested_svm_exit_handled_msr(struct vcpu_svm *svm,static int nested_svm_exit_handled (struct vcpu_svm *svm,bool kvm_override) {u32 exit_code = svm->vmcb->control.exit_code; - -switch (svm->vmcb->control.exit_code) { -case SVM_EXIT_MSR : -return nested_svm_do (svm, svm->nested.vmcb, - svm->nested.vmcb_msrpm,NULL , - nested_svm_exit_handled_msr); -default : -break ; - } +bool vmexit = false;if (kvm_override) {switch (exit_code) {
@@ -1528,45 +1517,55 @@ static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override) }switch (exit_code) { +case SVM_EXIT_MSR : +if (nested_svm_do (svm, svm->nested.vmcb, svm->nested.vmcb_msrpm, +NULL , nested_svm_exit_handled_msr)) + vmexit = true; +break ; case SVM_EXIT_READ_CR0 ... SVM_EXIT_READ_CR8: {u32 cr_bits =1 << (exit_code -SVM_EXIT_READ_CR0 );if (svm->nested.intercept_cr_read & cr_bits) -return 1 ; + vmexit = true;break ; } case SVM_EXIT_WRITE_CR0 ... SVM_EXIT_WRITE_CR8: {u32 cr_bits =1 << (exit_code -SVM_EXIT_WRITE_CR0 );if (svm->nested.intercept_cr_write & cr_bits) -return 1 ; + vmexit = true;break ; } case SVM_EXIT_READ_DR0 ... SVM_EXIT_READ_DR7: {u32 dr_bits =1 << (exit_code -SVM_EXIT_READ_DR0 );if (svm->nested.intercept_dr_read & dr_bits) -return 1 ; + vmexit = true;break ; } case SVM_EXIT_WRITE_DR0 ... SVM_EXIT_WRITE_DR7: {u32 dr_bits =1 << (exit_code -SVM_EXIT_WRITE_DR0 );if (svm->nested.intercept_dr_write & dr_bits) -return 1 ; + vmexit = true;break ; } case SVM_EXIT_EXCP_BASE ... SVM_EXIT_EXCP_BASE + 0x1f: {u32 excp_bits =1 << (exit_code -SVM_EXIT_EXCP_BASE );if (svm->nested.intercept_exceptions & excp_bits) -return 1 ; + vmexit = true;break ; }default : {u64 exit_bits =1ULL << (exit_code -SVM_EXIT_INTR );nsvm_printk ("exit code: 0x%x\n" , exit_code);if (svm->nested.intercept & exit_bits) -return 1 ; + vmexit = true; } } -return 0 ; +if (vmexit) { +nsvm_printk ("#VMEXIT reason=%04x\n" , exit_code); +nested_svm_vmexit (svm); + } + +return vmexit; }static inline void copy_vmcb_control_area (struct vmcb *dst_vmcb,struct vmcb *from_vmcb)
@@ -2327,11 +2326,8 @@ static int handle_exit(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)nsvm_printk ("nested handle_exit: 0x%x | 0x%lx | 0x%lx | 0x%lx\n" , exit_code, svm->vmcb->control.exit_info_1, svm->vmcb->control.exit_info_2, svm->vmcb->save.rip); -if (nested_svm_exit_handled (svm, true)) { -nested_svm_vmexit (svm); -nsvm_printk ("-> #VMEXIT\n" ); +if (nested_svm_exit_handled (svm, true))return 1 ; - } }svm_complete_interrupts (svm);
Signature · 1 line
1.6.4.1
PATCH 21/47 KVM: SVM: simplify nested_svm_check_exception
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
Makes the code of this function more readable by removing on indentation level for the core logic.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 19 ++++++++----------- 1 files changed, 8 insertions(+), 11 deletions(-)
arch/x86/kvm/svm.c +8 -11
@@ -1359,18 +1359,15 @@ static int nested_svm_check_permissions(struct vcpu_svm *svm)static int nested_svm_check_exception (struct vcpu_svm *svm,unsigned nr,bool has_error_code,u32 error_code) { -if (is_nested (svm)) { - svm->vmcb->control.exit_code =SVM_EXIT_EXCP_BASE + nr; - svm->vmcb->control.exit_code_hi =0 ; - svm->vmcb->control.exit_info_1 = error_code; - svm->vmcb->control.exit_info_2 = svm->vcpu.arch.cr2; -if (nested_svm_exit_handled (svm, false)) { -nsvm_printk ("VMexit -> EXCP 0x%x\n" , nr); -return 1 ; - } - } +if (!is_nested (svm)) +return 0 ; -return 0 ; + svm->vmcb->control.exit_code =SVM_EXIT_EXCP_BASE + nr; + svm->vmcb->control.exit_code_hi =0 ; + svm->vmcb->control.exit_info_1 = error_code; + svm->vmcb->control.exit_info_2 = svm->vcpu.arch.cr2; + +return nested_svm_exit_handled (svm, false); }static inline int nested_svm_intr (struct vcpu_svm *svm)
Signature · 1 line
1.6.4.1
PATCH 22/47 KVM: SVM: get rid of nested_svm_vmexit_real
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
This patch is the starting point of removing nested_svm_do from the nested svm code. The nested_svm_do function basically maps two guest physical pages to host virtual addresses and calls a passed function on it. This function pointer code flow is hard to read and not the best technical solution here.
As a side effect this patch indroduces the nested_svm_[un]map helper functions.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 52 ++++++++++++++++++++++++++++++++++++++++------------ 1 files changed, 40 insertions(+), 12 deletions(-)
arch/x86/kvm/svm.c +40 -12
@@ -1390,6 +1390,39 @@ static inline int nested_svm_intr(struct vcpu_svm *svm)return 0 ; } +static void *nested_svm_map (struct vcpu_svm *svm,u64 gpa,enum km_type idx) +{ +struct page *page; + +down_read (¤t->mm->mmap_sem); + page =gfn_to_page (svm->vcpu.kvm, gpa >>PAGE_SHIFT ); +up_read (¤t->mm->mmap_sem); + +if (is_error_page (page)) + goto error; + +return kmap_atomic (page, idx); + +error: +kvm_release_page_clean (page); +kvm_inject_gp (&svm->vcpu,0 ); + +return NULL ; +} + +static void nested_svm_unmap (void *addr,enum km_type idx) +{ +struct page *page; + +if (!addr) +return ; + + page =kmap_atomic_to_page (addr); + +kunmap_atomic (addr, idx); +kvm_release_page_dirty (page); +} +static struct page *nested_svm_get_page (struct vcpu_svm *svm,u64 gpa) {struct page *page;
@@ -1597,13 +1630,16 @@ static inline void copy_vmcb_control_area(struct vmcb *dst_vmcb, struct vmcb *fr dst->lbr_ctl = from->lbr_ctl; } -static int nested_svm_vmexit_real (struct vcpu_svm *svm,void *arg1, -void *arg2,void *opaque) +static int nested_svm_vmexit (struct vcpu_svm *svm) { -struct vmcb *nested_vmcb = (struct vmcb *)arg1; +struct vmcb *nested_vmcb;struct vmcb *hsave = svm->nested.hsave;struct vmcb *vmcb = svm->vmcb; + nested_vmcb =nested_svm_map (svm, svm->nested.vmcb,KM_USER0 ); +if (!nested_vmcb) +return 1 ; +/* Give the current vmcb to the guest */ disable_gif (svm);
@@ -1678,15 +1714,7 @@ static int nested_svm_vmexit_real(struct vcpu_svm *svm, void *arg1,/* Exit nested SVM mode */ svm->nested.vmcb =0 ; -return 0 ; -} - -static int nested_svm_vmexit (struct vcpu_svm *svm) -{ -nsvm_printk ("VMexit\n" ); -if (nested_svm_do (svm, svm->nested.vmcb,0 , -NULL , nested_svm_vmexit_real)) -return 1 ; +nested_svm_unmap (nested_vmcb,KM_USER0 );kvm_mmu_reset_context (&svm->vcpu);kvm_mmu_load (&svm->vcpu);
Signature · 1 line
1.6.4.1
PATCH 23/47 KVM: SVM: clean up nested_svm_exit_handled_msr
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
This patch changes nested svm to call nested_svm_exit_handled_msr directly and not through nested_svm_do.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 37 ++++++++++++++++++++++--------------- 1 files changed, 22 insertions(+), 15 deletions(-)
arch/x86/kvm/svm.c +22 -15
@@ -1483,15 +1483,20 @@ static int nested_svm_do(struct vcpu_svm *svm,return retval; } -static int nested_svm_exit_handled_msr (struct vcpu_svm *svm, -void *arg1,void *arg2, -void *opaque) +static bool nested_svm_exit_handled_msr (struct vcpu_svm *svm) { -struct vmcb *nested_vmcb = (struct vmcb *)arg1; -u8 *msrpm = (u8 *)arg2; -u32 t0, t1; -u32 msr = svm->vcpu.arch.regs[VCPU_REGS_RCX ];u32 param = svm->vmcb->control.exit_info_1 &1 ; +u32 msr = svm->vcpu.arch.regs[VCPU_REGS_RCX ]; +struct vmcb *nested_vmcb; +bool ret = false; +u32 t0, t1; +u8 *msrpm; + + nested_vmcb =nested_svm_map (svm, svm->nested.vmcb,KM_USER0 ); + msrpm =nested_svm_map (svm, svm->nested.vmcb_msrpm,KM_USER1 ); + +if (!nested_vmcb || !msrpm) + goto out;if (!(nested_vmcb->control.intercept & (1ULL <<INTERCEPT_MSR_PROT )))return 0 ;
@@ -1512,13 +1517,17 @@ static int nested_svm_exit_handled_msr(struct vcpu_svm *svm, t0 %=8 ;break ;default : -return 1 ; -break ; + ret = true; + goto out; } -if (msrpm[t1] & ((1 << param) << t0)) -return 1 ; -return 0 ; + ret = msrpm[t1] & ((1 << param) << t0); + +out: +nested_svm_unmap (nested_vmcb,KM_USER0 ); +nested_svm_unmap (msrpm,KM_USER1 ); + +return ret; }static int nested_svm_exit_handled (struct vcpu_svm *svm,bool kvm_override)
@@ -1548,9 +1557,7 @@ static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override)switch (exit_code) {case SVM_EXIT_MSR : -if (nested_svm_do (svm, svm->nested.vmcb, svm->nested.vmcb_msrpm, -NULL , nested_svm_exit_handled_msr)) - vmexit = true; + vmexit =nested_svm_exit_handled_msr (svm);break ; case SVM_EXIT_READ_CR0 ... SVM_EXIT_READ_CR8: {u32 cr_bits =1 << (exit_code -SVM_EXIT_READ_CR0 );
Signature · 1 line
1.6.4.1
PATCH 24/47 KVM: SVM: clean up nestec vmload/vmsave paths
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
This patch removes the usage of nested_svm_do from the vmload and vmsave emulation code paths.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 36 +++++++++++++++++------------------- 1 files changed, 17 insertions(+), 19 deletions(-)
arch/x86/kvm/svm.c +17 -19
@@ -128,8 +128,6 @@ static void svm_complete_interrupts(struct vcpu_svm *svm);static int nested_svm_exit_handled (struct vcpu_svm *svm,bool kvm_override);static int nested_svm_vmexit (struct vcpu_svm *svm); -static int nested_svm_vmsave (struct vcpu_svm *svm,void *nested_vmcb, -void *arg2,void *opaque);static int nested_svm_check_exception (struct vcpu_svm *svm,unsigned nr,bool has_error_code,u32 error_code);
@@ -1868,7 +1866,7 @@ static int nested_svm_vmrun(struct vcpu_svm *svm, void *arg1,return 0 ; } -static int nested_svm_vmloadsave (struct vmcb *from_vmcb,struct vmcb *to_vmcb) +static void nested_svm_vmloadsave (struct vmcb *from_vmcb,struct vmcb *to_vmcb) { to_vmcb->save.fs = from_vmcb->save.fs; to_vmcb->save.gs = from_vmcb->save.gs;
@@ -1882,44 +1880,44 @@ static int nested_svm_vmloadsave(struct vmcb *from_vmcb, struct vmcb *to_vmcb) to_vmcb->save.sysenter_cs = from_vmcb->save.sysenter_cs; to_vmcb->save.sysenter_esp = from_vmcb->save.sysenter_esp; to_vmcb->save.sysenter_eip = from_vmcb->save.sysenter_eip; - -return 1 ; -} - -static int nested_svm_vmload (struct vcpu_svm *svm,void *nested_vmcb, -void *arg2,void *opaque) -{ -return nested_svm_vmloadsave ((struct vmcb *)nested_vmcb, svm->vmcb); -} - -static int nested_svm_vmsave (struct vcpu_svm *svm,void *nested_vmcb, -void *arg2,void *opaque) -{ -return nested_svm_vmloadsave (svm->vmcb, (struct vmcb *)nested_vmcb); }static int vmload_interception (struct vcpu_svm *svm,struct kvm_run *kvm_run) { +struct vmcb *nested_vmcb; +if (nested_svm_check_permissions (svm))return 1 ; svm->next_rip =kvm_rip_read (&svm->vcpu) +3 ;skip_emulated_instruction (&svm->vcpu); -nested_svm_do (svm, svm->vmcb->save.rax,0 ,NULL , nested_svm_vmload); + nested_vmcb =nested_svm_map (svm, svm->vmcb->save.rax,KM_USER0 ); +if (!nested_vmcb) +return 1 ; + +nested_svm_vmloadsave (nested_vmcb, svm->vmcb); +nested_svm_unmap (nested_vmcb,KM_USER0 );return 1 ; }static int vmsave_interception (struct vcpu_svm *svm,struct kvm_run *kvm_run) { +struct vmcb *nested_vmcb; +if (nested_svm_check_permissions (svm))return 1 ; svm->next_rip =kvm_rip_read (&svm->vcpu) +3 ;skip_emulated_instruction (&svm->vcpu); -nested_svm_do (svm, svm->vmcb->save.rax,0 ,NULL , nested_svm_vmsave); + nested_vmcb =nested_svm_map (svm, svm->vmcb->save.rax,KM_USER0 ); +if (!nested_vmcb) +return 1 ; + +nested_svm_vmloadsave (svm->vmcb, nested_vmcb); +nested_svm_unmap (nested_vmcb,KM_USER0 );return 1 ; }
Signature · 1 line
1.6.4.1
PATCH 25/47 KVM: SVM: clean up nested vmrun path
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
This patch removes the usage of nested_svm_do from the vmrun emulation path.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 34 ++++++++++++++++++++++------------ 1 files changed, 22 insertions(+), 12 deletions(-)
arch/x86/kvm/svm.c +22 -12
@@ -1727,25 +1727,35 @@ static int nested_svm_vmexit(struct vcpu_svm *svm)return 0 ; } -static int nested_svm_vmrun_msrpm (struct vcpu_svm *svm,void *arg1, -void *arg2,void *opaque) +static bool nested_svm_vmrun_msrpm (struct vcpu_svm *svm) { +u32 *nested_msrpm;int i; -u32 *nested_msrpm = (u32 *)arg1; + + nested_msrpm =nested_svm_map (svm, svm->nested.vmcb_msrpm,KM_USER0 ); +if (!nested_msrpm) +return false; +for (i=0 ; i<PAGE_SIZE * (1 <<MSRPM_ALLOC_ORDER ) /4 ; i++) svm->nested.msrpm[i] = svm->msrpm[i] | nested_msrpm[i]; + svm->vmcb->control.msrpm_base_pa =__pa (svm->nested.msrpm); -return 0 ; +nested_svm_unmap (nested_msrpm,KM_USER0 ); + +return true; } -static int nested_svm_vmrun (struct vcpu_svm *svm,void *arg1, -void *arg2,void *opaque) +static bool nested_svm_vmrun (struct vcpu_svm *svm) { -struct vmcb *nested_vmcb = (struct vmcb *)arg1; +struct vmcb *nested_vmcb;struct vmcb *hsave = svm->nested.hsave;struct vmcb *vmcb = svm->vmcb; + nested_vmcb =nested_svm_map (svm, svm->vmcb->save.rax,KM_USER0 ); +if (!nested_vmcb) +return false; +/* nested_vmcb is our indicator if nested SVM is activated */ svm->nested.vmcb = svm->vmcb->save.rax;
@@ -1861,9 +1871,11 @@ static int nested_svm_vmrun(struct vcpu_svm *svm, void *arg1, svm->vmcb->control.event_inj = nested_vmcb->control.event_inj; svm->vmcb->control.event_inj_err = nested_vmcb->control.event_inj_err; +nested_svm_unmap (nested_vmcb,KM_USER0 ); +enable_gif (svm); -return 0 ; +return true; }static void nested_svm_vmloadsave (struct vmcb *from_vmcb,struct vmcb *to_vmcb)
@@ -1931,12 +1943,10 @@ static int vmrun_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) svm->next_rip =kvm_rip_read (&svm->vcpu) +3 ;skip_emulated_instruction (&svm->vcpu); -if (nested_svm_do (svm, svm->vmcb->save.rax,0 , -NULL , nested_svm_vmrun)) +if (!nested_svm_vmrun (svm))return 1 ; -if (nested_svm_do (svm, svm->nested.vmcb_msrpm,0 , -NULL , nested_svm_vmrun_msrpm)) +if (!nested_svm_vmrun_msrpm (svm))return 1 ;return 1 ;
Signature · 1 line
1.6.4.1
PATCH 26/47 KVM: SVM: remove nested_svm_do and helper functions
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
This function is not longer required. So remove it.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 60 ---------------------------------------------------- 1 files changed, 0 insertions(+), 60 deletions(-)
arch/x86/kvm/svm.c +0 -60
@@ -1421,66 +1421,6 @@ static void nested_svm_unmap(void *addr, enum km_type idx)kvm_release_page_dirty (page); } -static struct page *nested_svm_get_page (struct vcpu_svm *svm,u64 gpa) -{ -struct page *page; - -down_read (¤t->mm->mmap_sem); - page =gfn_to_page (svm->vcpu.kvm, gpa >>PAGE_SHIFT ); -up_read (¤t->mm->mmap_sem); - -if (is_error_page (page)) { -printk (KERN_INFO "%s: could not find page at 0x%llx\n" , - __func__, gpa); -kvm_release_page_clean (page); -kvm_inject_gp (&svm->vcpu,0 ); -return NULL ; - } -return page; -} - -static int nested_svm_do (struct vcpu_svm *svm, -u64 arg1_gpa,u64 arg2_gpa,void *opaque, -int (*handler)(struct vcpu_svm *svm, -void *arg1, -void *arg2, -void *opaque)) -{ -struct page *arg1_page; -struct page *arg2_page =NULL ; -void *arg1; -void *arg2 =NULL ; -int retval; - - arg1_page =nested_svm_get_page (svm, arg1_gpa); -if (arg1_page ==NULL ) -return 1 ; - -if (arg2_gpa) { - arg2_page =nested_svm_get_page (svm, arg2_gpa); -if (arg2_page ==NULL ) { -kvm_release_page_clean (arg1_page); -return 1 ; - } - } - - arg1 =kmap_atomic (arg1_page,KM_USER0 ); -if (arg2_gpa) - arg2 =kmap_atomic (arg2_page,KM_USER1 ); - - retval =handler (svm, arg1, arg2, opaque); - -kunmap_atomic (arg1,KM_USER0 ); -if (arg2_gpa) -kunmap_atomic (arg2,KM_USER1 ); - -kvm_release_page_dirty (arg1_page); -if (arg2_gpa) -kvm_release_page_dirty (arg2_page); - -return retval; -} -static bool nested_svm_exit_handled_msr (struct vcpu_svm *svm) {u32 param = svm->vmcb->control.exit_info_1 &1 ;
Signature · 1 line
1.6.4.1
PATCH 27/47 KVM: SVM: handle errors in vmrun emulation path appropriatly
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
If nested svm fails to load the msrpm the vmrun succeeds with the old msrpm which is not correct. This patch changes the logic to roll back to host mode in case the msrpm cannot be loaded.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 14 +++++++++++++- 1 files changed, 13 insertions(+), 1 deletions(-)
arch/x86/kvm/svm.c +13 -1
@@ -1877,6 +1877,7 @@ static int vmsave_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)static int vmrun_interception (struct vcpu_svm *svm,struct kvm_run *kvm_run) {nsvm_printk ("VMrun\n" ); +if (nested_svm_check_permissions (svm))return 1 ;
@@ -1887,7 +1888,18 @@ static int vmrun_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)return 1 ;if (!nested_svm_vmrun_msrpm (svm)) -return 1 ; + goto failed; + +return 1 ; + +failed: + + svm->vmcb->control.exit_code =SVM_EXIT_ERR ; + svm->vmcb->control.exit_code_hi =0 ; + svm->vmcb->control.exit_info_1 =0 ; + svm->vmcb->control.exit_info_2 =0 ; + +nested_svm_vmexit (svm);return 1 ; }
Signature · 1 line
1.6.4.1
PATCH 28/47 KVM: SVM: move special nested exit handling to separate function
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
This patch moves the handling for special nested vmexits like #pf to a separate function. This makes the kvm_override parameter obsolete and makes the code more readable.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 80 ++++++++++++++++++++++++++++++++------------------- 1 files changed, 50 insertions(+), 30 deletions(-)
arch/x86/kvm/svm.c +50 -30
@@ -47,6 +47,10 @@ MODULE_LICENSE("GPL");#define SVM_FEATURE_LBRV (1 << 1)#define SVM_FEATURE_SVML (1 << 2) +#define NESTED_EXIT_HOST 0/* Exit handled on host level */ +#define NESTED_EXIT_DONE 1/* Exit caused nested vmexit */ +#define NESTED_EXIT_CONTINUE 2/* Further checks needed */ +#define DEBUGCTL_RESERVED_BITS (~(0x3fULL))/* Turn on to get debugging output*/
@@ -126,7 +130,7 @@ module_param(nested, int, S_IRUGO);static void svm_flush_tlb (struct kvm_vcpu *vcpu);static void svm_complete_interrupts (struct vcpu_svm *svm); -static int nested_svm_exit_handled (struct vcpu_svm *svm,bool kvm_override); +static int nested_svm_exit_handled (struct vcpu_svm *svm);static int nested_svm_vmexit (struct vcpu_svm *svm);static int nested_svm_check_exception (struct vcpu_svm *svm,unsigned nr,bool has_error_code,u32 error_code);
@@ -1365,7 +1369,7 @@ static int nested_svm_check_exception(struct vcpu_svm *svm, unsigned nr, svm->vmcb->control.exit_info_1 = error_code; svm->vmcb->control.exit_info_2 = svm->vcpu.arch.cr2; -return nested_svm_exit_handled (svm, false); +return nested_svm_exit_handled (svm); }static inline int nested_svm_intr (struct vcpu_svm *svm)
@@ -1379,7 +1383,7 @@ static inline int nested_svm_intr(struct vcpu_svm *svm) svm->vmcb->control.exit_code =SVM_EXIT_INTR ; -if (nested_svm_exit_handled (svm, false)) { +if (nested_svm_exit_handled (svm)) {nsvm_printk ("VMexit -> INTR\n" );return 1 ; }
@@ -1468,31 +1472,39 @@ out:return ret; } -static int nested_svm_exit_handled (struct vcpu_svm *svm,bool kvm_override) +static int nested_svm_exit_special (struct vcpu_svm *svm) {u32 exit_code = svm->vmcb->control.exit_code; -bool vmexit = false; -if (kvm_override) { -switch (exit_code) { -case SVM_EXIT_INTR : -case SVM_EXIT_NMI : -return 0 ; +switch (exit_code) { +case SVM_EXIT_INTR : +case SVM_EXIT_NMI : +return NESTED_EXIT_HOST ;/* For now we are always handling NPFs when using them */ -case SVM_EXIT_NPF : -if (npt_enabled) -return 0 ; -break ; -/* When we're shadowing, trap PFs */ -case SVM_EXIT_EXCP_BASE +PF_VECTOR : -if (!npt_enabled) -return 0 ; -break ; -default : -break ; - } +case SVM_EXIT_NPF : +if (npt_enabled) +return NESTED_EXIT_HOST ; +break ; +/* When we're shadowing, trap PFs */ +case SVM_EXIT_EXCP_BASE +PF_VECTOR : +if (!npt_enabled) +return NESTED_EXIT_HOST ; +break ; +default : +break ; } +return NESTED_EXIT_CONTINUE ; +} + +/* +* If this function returns true, this #vmexit was already handled +*/ +static int nested_svm_exit_handled (struct vcpu_svm *svm) +{ +u32 exit_code = svm->vmcb->control.exit_code; +int vmexit =NESTED_EXIT_HOST ; +switch (exit_code) {case SVM_EXIT_MSR : vmexit =nested_svm_exit_handled_msr (svm);
@@ -1500,42 +1512,42 @@ static int nested_svm_exit_handled(struct vcpu_svm *svm, bool kvm_override) case SVM_EXIT_READ_CR0 ... SVM_EXIT_READ_CR8: {u32 cr_bits =1 << (exit_code -SVM_EXIT_READ_CR0 );if (svm->nested.intercept_cr_read & cr_bits) - vmexit = true; + vmexit =NESTED_EXIT_DONE ;break ; } case SVM_EXIT_WRITE_CR0 ... SVM_EXIT_WRITE_CR8: {u32 cr_bits =1 << (exit_code -SVM_EXIT_WRITE_CR0 );if (svm->nested.intercept_cr_write & cr_bits) - vmexit = true; + vmexit =NESTED_EXIT_DONE ;break ; } case SVM_EXIT_READ_DR0 ... SVM_EXIT_READ_DR7: {u32 dr_bits =1 << (exit_code -SVM_EXIT_READ_DR0 );if (svm->nested.intercept_dr_read & dr_bits) - vmexit = true; + vmexit =NESTED_EXIT_DONE ;break ; } case SVM_EXIT_WRITE_DR0 ... SVM_EXIT_WRITE_DR7: {u32 dr_bits =1 << (exit_code -SVM_EXIT_WRITE_DR0 );if (svm->nested.intercept_dr_write & dr_bits) - vmexit = true; + vmexit =NESTED_EXIT_DONE ;break ; } case SVM_EXIT_EXCP_BASE ... SVM_EXIT_EXCP_BASE + 0x1f: {u32 excp_bits =1 << (exit_code -SVM_EXIT_EXCP_BASE );if (svm->nested.intercept_exceptions & excp_bits) - vmexit = true; + vmexit =NESTED_EXIT_DONE ;break ; }default : {u64 exit_bits =1ULL << (exit_code -SVM_EXIT_INTR );nsvm_printk ("exit code: 0x%x\n" , exit_code);if (svm->nested.intercept & exit_bits) - vmexit = true; + vmexit =NESTED_EXIT_DONE ; } } -if (vmexit) { +if (vmexit ==NESTED_EXIT_DONE ) {nsvm_printk ("#VMEXIT reason=%04x\n" , exit_code);nested_svm_vmexit (svm); }
@@ -2315,10 +2327,18 @@ static int handle_exit(struct kvm_run *kvm_run, struct kvm_vcpu *vcpu)trace_kvm_exit (exit_code, svm->vmcb->save.rip);if (is_nested (svm)) { +int vmexit; +nsvm_printk ("nested handle_exit: 0x%x | 0x%lx | 0x%lx | 0x%lx\n" , exit_code, svm->vmcb->control.exit_info_1, svm->vmcb->control.exit_info_2, svm->vmcb->save.rip); -if (nested_svm_exit_handled (svm, true)) + + vmexit =nested_svm_exit_special (svm); + +if (vmexit ==NESTED_EXIT_CONTINUE ) + vmexit =nested_svm_exit_handled (svm); + +if (vmexit ==NESTED_EXIT_DONE )return 1 ; }
Signature · 1 line
1.6.4.1
PATCH 29/47 KVM: SVM: remove unnecessary is_nested check from svm_cpu_run
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
This check is not necessary. We have to sync the vcpu->arch.cr2 always back to the VMCB. This patch remove the is_nested check.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-)
arch/x86/kvm/svm.c +1 -2
@@ -2605,8 +2605,7 @@ static void svm_vcpu_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) fs_selector =kvm_read_fs (); gs_selector =kvm_read_gs (); ldt_selector =kvm_read_ldt (); -if (!is_nested (svm)) - svm->vmcb->save.cr2 = vcpu->arch.cr2; + svm->vmcb->save.cr2 = vcpu->arch.cr2;/* required for live migration with NPT */ if (npt_enabled) svm->vmcb->save.cr3 = vcpu->arch.cr3;
Signature · 1 line
1.6.4.1
PATCH 30/47 KVM: SVM: move nested_svm_intr main logic out of if-clause
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
This patch removes one indentation level from nested_svm_intr and makes the logic more readable.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 21 +++++++++++---------- 1 files changed, 11 insertions(+), 10 deletions(-)
arch/x86/kvm/svm.c +11 -10
@@ -1374,19 +1374,20 @@ static int nested_svm_check_exception(struct vcpu_svm *svm, unsigned nr,static inline int nested_svm_intr (struct vcpu_svm *svm) { -if (is_nested (svm)) { -if (!(svm->vcpu.arch.hflags &HF_VINTR_MASK )) -return 0 ; +if (!is_nested (svm)) +return 0 ; -if (!(svm->vcpu.arch.hflags &HF_HIF_MASK )) -return 0 ; +if (!(svm->vcpu.arch.hflags &HF_VINTR_MASK )) +return 0 ; - svm->vmcb->control.exit_code =SVM_EXIT_INTR ; +if (!(svm->vcpu.arch.hflags &HF_HIF_MASK )) +return 0 ; -if (nested_svm_exit_handled (svm)) { -nsvm_printk ("VMexit -> INTR\n" ); -return 1 ; - } + svm->vmcb->control.exit_code =SVM_EXIT_INTR ; + +if (nested_svm_exit_handled (svm)) { +nsvm_printk ("VMexit -> INTR\n" ); +return 1 ; }return 0 ;
Signature · 1 line
1.6.4.1
PATCH 31/47 KVM: SVM: check for nested VINTR flag in svm_interrupt_allowed
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
Not checking for this flag breaks any nested hypervisor that does not set VINTR. So fix it with this patch.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/kvm/svm.c +1 -1
@@ -2466,7 +2466,7 @@ static int svm_interrupt_allowed(struct kvm_vcpu *vcpu)return (vmcb->save.rflags &X86_EFLAGS_IF ) && !(vmcb->control.int_state &SVM_INTERRUPT_SHADOW_MASK ) &&gif_set (svm) && - !is_nested (svm); + !(is_nested (svm) && (svm->vcpu.arch.hflags &HF_VINTR_MASK )); }static void enable_irq_window (struct kvm_vcpu *vcpu)
Signature · 1 line
1.6.4.1
PATCH 32/47 KVM: SVM: enable nested svm by default
In reply to Avi KivityFrom: Joerg Roedel <[email protected]>
Nested SVM is (in my experience) stable enough to be enabled by default. So omit the requirement to pass a module parameter.
Signed-off-by Joerg Roedel <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/kvm/svm.c +1 -1
@@ -124,7 +124,7 @@ static int npt = 1;module_param (npt, int,S_IRUGO ); -static int nested =0 ; +static int nested =1 ;module_param (nested, int,S_IRUGO );static void svm_flush_tlb (struct kvm_vcpu *vcpu);
Signature · 1 line
1.6.4.1
PATCH 33/47 KVM: Update cr8 intercept when APIC TPR is changed by userspace
In reply to Avi KivityFrom: Gleb Natapov <[email protected]>
Since on vcpu entry we do it only if apic is enabled we should do it when TPR is changed while apic is disabled. This happens when windows resets HW without setting TPR to zero.
Signed-off-by Gleb Natapov <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/x86.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/kvm/x86.c +2 -0
@@ -77,6 +77,7 @@ static u64 __read_mostly efer_reserved_bits = 0xfffffffffffffffeULL;#define VM_STAT (x) offsetof(struct kvm, stat.x), KVM_STAT_VM#define VCPU_STAT (x) offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPU +static void update_cr8_intercept (struct kvm_vcpu *vcpu);static int kvm_dev_ioctl_get_supported_cpuid (struct kvm_cpuid2 *cpuid,struct kvm_cpuid_entry2 __user *entries);
@@ -1629,6 +1630,7 @@ static int kvm_vcpu_ioctl_set_lapic(struct kvm_vcpu *vcpu,vcpu_load (vcpu);memcpy (vcpu->arch.apic->regs, s->regs,sizeof *s);kvm_apic_post_state_restore (vcpu); +update_cr8_intercept (vcpu);vcpu_put (vcpu);return 0 ;
Signature · 1 line
1.6.4.1
PATCH 34/47 KVM: SVM: Drop tlb flush workaround in npt
In reply to Avi KivityIt is no longer possible to reproduce the problem any more, so presumably it has been fixed.
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/svm.c | 13 ++----------- 1 files changed, 2 insertions(+), 11 deletions(-)
arch/x86/kvm/svm.c +2 -11
@@ -1187,17 +1187,8 @@ static int pf_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run) error_code = svm->vmcb->control.exit_info_1;trace_kvm_page_fault (fault_address, error_code); -/* -* FIXME: Tis shouldn't be necessary here, but there is a flush -* missing in the MMU code. Until we find this bug, flush the -* complete TLB here on an NPF -*/ -if (npt_enabled) -svm_flush_tlb (&svm->vcpu); -else { -if (kvm_event_needs_reinjection (&svm->vcpu)) -kvm_mmu_unprotect_page_virt (&svm->vcpu, fault_address); - } +if (!npt_enabled &&kvm_event_needs_reinjection (&svm->vcpu)) +kvm_mmu_unprotect_page_virt (&svm->vcpu, fault_address);return kvm_mmu_page_fault (&svm->vcpu, fault_address, error_code); }
Signature · 1 line
1.6.4.1
PATCH 35/47 KVM: Move #endif KVM_CAP_IRQ_ROUTING to correct place
In reply to Avi KivityThe symbol only controls irq routing, not MSI-X.
Signed-off-by Avi Kivity <[email protected]>
--- virt/kvm/kvm_main.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
virt/kvm/kvm_main.c +1 -1
@@ -2236,6 +2236,7 @@ static long kvm_vm_ioctl(struct file *filp,vfree (entries);break ; } +#endif /* KVM_CAP_IRQ_ROUTING */ #ifdef __KVM_HAVE_MSIXcase KVM_ASSIGN_SET_MSIX_NR : {struct kvm_assigned_msix_nr entry_nr;
@@ -2258,7 +2259,6 @@ static long kvm_vm_ioctl(struct file *filp, break; }#endif -#endif /* KVM_CAP_IRQ_ROUTING */ case KVM_IRQFD : {struct kvm_irqfd data;
Signature · 1 line
1.6.4.1
PATCH 36/47 KVM: VMX: Adjust rflags if in real mode emulation
In reply to Avi KivityWe set rflags.vm86 when virtualizing real mode to do through vm8086 mode; so we need to take it out again when reading rflags.
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/vmx.c | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-)
arch/x86/kvm/vmx.c +6 -1
@@ -781,7 +781,12 @@ static void vmx_fpu_deactivate(struct kvm_vcpu *vcpu)static unsigned long vmx_get_rflags (struct kvm_vcpu *vcpu) { -return vmcs_readl (GUEST_RFLAGS ); +unsigned long rflags; + + rflags =vmcs_readl (GUEST_RFLAGS ); +if (to_vmx (vcpu)->rmode.vm86_active) + rflags &= ~(unsigned long )(X86_EFLAGS_IOPL |X86_EFLAGS_VM ); +return rflags; }static void vmx_set_rflags (struct kvm_vcpu *vcpu,unsigned long rflags)
Signature · 1 line
1.6.4.1
PATCH 37/47 KVM: When switching to a vm8086 task, load segments as 16-bit
In reply to Avi KivityFrom: Anthony Liguori <[email protected]>
According to 16.2.5 in the SDM, eflags.vm in the tss is consulted before loading and new segments. If eflags.vm == 1, then the segments are treated as 16-bit segments. The LDTR and TR are not normally available in vm86 mode so if they happen to somehow get loaded, they need to be treated as 32-bit segments.
This fixes an invalid vmentry failure in a custom OS that was happening after a task switch into vm8086 mode. Since the segments were being mistakenly treated as 32-bit, we loaded garbage state.
Signed-off-by Anthony Liguori <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/x86.c | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-)
arch/x86/kvm/x86.c +8 -1
@@ -4101,12 +4101,19 @@ static int kvm_load_realmode_segment(struct kvm_vcpu *vcpu, u16 selector, int sereturn 0 ; } +static int is_vm86_segment (struct kvm_vcpu *vcpu,int seg) +{ +return (seg !=VCPU_SREG_LDTR ) && + (seg !=VCPU_SREG_TR ) && + (kvm_x86_ops->get_rflags (vcpu) &X86_EFLAGS_VM ); +} +int kvm_load_segment_descriptor (struct kvm_vcpu *vcpu,u16 selector,int type_bits,int seg) {struct kvm_segment kvm_seg; -if (!(vcpu->arch.cr0 &X86_CR0_PE )) +if (is_vm86_segment (vcpu, seg) || !(vcpu->arch.cr0 &X86_CR0_PE ))return kvm_load_realmode_segment (vcpu, selector, seg);if (load_segment_descriptor_to_kvm_desct (vcpu, selector, &kvm_seg))return 1 ;
Signature · 1 line
1.6.4.1
PATCH 38/47 KVM: Rename x86_emulate.c to emulate.c
In reply to Avi KivityWe're in arch/x86, what could we possibly be emulating?
Signed-off-by Avi Kivity <[email protected]>
---
.../asm/{kvm_x86_emulate.h => kvm_emulate.h} | 0
arch/x86/include/asm/kvm_host.h | 2 +-
arch/x86/kvm/Makefile | 2 +-
arch/x86/kvm/{x86_emulate.c => emulate.c} | 4 ++--
arch/x86/kvm/x86.c | 2 +-
5 files changed, 5 insertions(+), 5 deletions(-)
rename arch/x86/include/asm/{kvm_x86_emulate.h => kvm_emulate.h} (100%)
rename arch/x86/kvm/{x86_emulate.c => emulate.c} (99%)arch/x86/include/asm/kvm_emulate.h renamed from arch/x86/include/asm/kvm_x86_emulate.h +0 -0
arch/x86/include/asm/kvm_host.h +1 -1
@@ -138,7 +138,7 @@ enum { VCPU_SREG_LDTR, }; -#include <asm/kvm_x86_emulate.h> +#include <asm/kvm_emulate.h> #define KVM_NR_MEM_OBJS 40
arch/x86/kvm/Makefile +1 -1
@@ -9,7 +9,7 @@ kvm-y += $(addprefix ../../../virt/kvm/, kvm_main.o ioapic.o \ coalesced_mmio.o irq_comm.o eventfd.o) kvm-$(CONFIG_IOMMU_API) += $(addprefix ../../../virt/kvm/, iommu.o) -kvm-y += x86.o mmu.o x86_emulate.o i8259.o irq.o lapic.o \ +kvm-y += x86.o mmu.o emulate.o i8259.o irq.o lapic.o \ i8254.o timer.o kvm-intel-y += vmx.o kvm-amd-y += svm.o
arch/x86/kvm/emulate.c renamed from arch/x86/kvm/x86_emulate.c +2 -2
@@ -1,5 +1,5 @@/****************************************************************************** -* x86_emulate.c +* emulate.c * * Generic x86 (32-bit and 64-bit) instruction decoder and emulator. *
@@ -30,7 +30,7 @@ #define DPRINTF(x...) do {} while (0)#endif #include <linux/module.h> -#include <asm/kvm_x86_emulate.h> +#include <asm/kvm_emulate.h> #include "mmu.h" /* for is_long_mode() */
arch/x86/kvm/x86.c +1 -1
@@ -2759,7 +2759,7 @@ int emulate_instruction(struct kvm_vcpu *vcpu,kvm_clear_exception_queue (vcpu); vcpu->arch.mmio_fault_cr2 = cr2;/* -* TODO: fix x86_emulate.c to use guest_read/write_register +* TODO: fix emulate.c to use guest_read/write_register * instead of direct ->regs accesses, can save hundred cycles * on Intel for instructions that don't read/change RSP, for * for example.
Signature · 1 line
1.6.4.1
PATCH 39/47 Documentation: Update KVM list email address
In reply to Avi KivityFrom: Amit Shah <[email protected]>
The KVM list moved to vger.kernel.org last year
Signed-off-by Amit Shah <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- Documentation/ioctl/ioctl-number.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
Documentation/ioctl/ioctl-number.txt +1 -1
@@ -191,7 +191,7 @@ Code Seq# Include File Comments 0xAD 00 Netfilter device in development: <mailto:[email protected]> 0xAE all linux/kvm.h Kernel-based Virtual Machine - <mailto:[email protected]> + <mailto:[email protected]> 0xB0 all RATIO devices in development: <mailto:[email protected]> 0xB1 00-1F PPPoX <mailto:[email protected]>
Signature · 1 line
1.6.4.1
PATCH 40/47 KVM: export kvm_para.h
In reply to Avi KivityFrom: Michael S. Tsirkin <[email protected]>
kvm_para.h contains userspace interface and so
should be exported.
Signed-off-by Michael S. Tsirkin <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- include/asm-generic/Kbuild.asm | 5 +++++ include/linux/Kbuild | 4 ++++ 2 files changed, 9 insertions(+), 0 deletions(-)
include/asm-generic/Kbuild.asm +5 -0
@@ -3,6 +3,11 @@ ifneq ($(wildcard $(srctree)/arch/$(SRCARCH)/include/asm/kvm.h \ header-y += kvm.h endif +ifneq ($(wildcard $(srctree)/arch/$(SRCARCH)/include/asm/kvm_para.h \ + $(srctree)/include/asm-$(SRCARCH)/kvm_para.h),) +header-y += kvm_para.h +endif + ifneq ($(wildcard $(srctree)/arch/$(SRCARCH)/include/asm/a.out.h \ $(srctree)/include/asm-$(SRCARCH)/a.out.h),) unifdef-y += a.out.h
include/linux/Kbuild +4 -0
@@ -268,6 +268,10 @@ ifneq ($(wildcard $(srctree)/arch/$(SRCARCH)/include/asm/kvm.h \ $(srctree)/include/asm-$(SRCARCH)/kvm.h),) unifdef-y += kvm.h endif +ifneq ($(wildcard $(srctree)/arch/$(SRCARCH)/include/asm/kvm_para.h \ + $(srctree)/include/asm-$(SRCARCH)/kvm_para.h),) +unifdef-y += kvm_para.h +endif unifdef-y += llc.h unifdef-y += loop.h unifdef-y += lp.h
Signature · 1 line
1.6.4.1
PATCH 41/47 KVM: Add __KERNEL__ guards to exported headers
In reply to Avi KivitySigned-off-by Avi Kivity <[email protected]>
--- arch/ia64/include/asm/kvm_para.h | 4 ++++ arch/s390/include/asm/kvm_para.h | 4 ++++ 2 files changed, 8 insertions(+), 0 deletions(-)
arch/ia64/include/asm/kvm_para.h +4 -0
@@ -19,9 +19,13 @@* */ +#ifdef __KERNEL__ +static inline unsigned int kvm_arch_para_features (void ) {return 0 ; }#endif + +#endif
arch/s390/include/asm/kvm_para.h +4 -0
@@ -13,6 +13,8 @@#ifndef __S390_KVM_PARA_H#define __S390_KVM_PARA_H +#ifdef __KERNEL__ +/* * Hypercalls for KVM on s390. The calling convention is similar to the * s390 ABI, so we use R2-R6 for parameters 1-5. In addition we use R1
@@ -147,4 +149,6 @@ static inline unsigned int kvm_arch_para_features(void)return 0 ; } +#endif +#endif /* __S390_KVM_PARA_H */
Signature · 1 line
1.6.4.1
PATCH 42/47 KVM: Add missing #include
In reply to Avi KivitySigned-off-by Avi Kivity <[email protected]>
--- arch/x86/include/asm/kvm_para.h | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/include/asm/kvm_para.h +2 -0
@@ -1,6 +1,8 @@#ifndef _ASM_X86_KVM_PARA_H#define _ASM_X86_KVM_PARA_H +#include <linux/types.h> +/* This CPUID returns the signature 'KVMKVMKVM' in ebx, ecx, and edx. It * should be used to determine that a VM is running under KVM. */
Signature · 1 line
1.6.4.1
PATCH 43/47 KVM: x86 emulator: Add adc and sbb missing decoder flags
In reply to Avi KivityFrom: Mohammed Gamal <[email protected]>
Add missing decoder flags for adc and sbb instructions (opcodes 0x14-0x15, 0x1c-0x1d)
Signed-off-by Mohammed Gamal <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/emulate.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/emulate.c +2 -2
@@ -100,11 +100,11 @@ static u32 opcode_table[256] = {/* 0x10 - 0x17 */ ByteOp | DstMem | SrcReg | ModRM, DstMem | SrcReg | ModRM, ByteOp | DstReg | SrcMem | ModRM, DstReg | SrcMem | ModRM, -0 ,0 ,0 ,0 , + ByteOp | DstAcc | SrcImm, DstAcc | SrcImm,0 ,0 ,/* 0x18 - 0x1F */ ByteOp | DstMem | SrcReg | ModRM, DstMem | SrcReg | ModRM, ByteOp | DstReg | SrcMem | ModRM, DstReg | SrcMem | ModRM, -0 ,0 ,0 ,0 , + ByteOp | DstAcc | SrcImm, DstAcc | SrcImm,0 ,0 ,/* 0x20 - 0x27 */ ByteOp | DstMem | SrcReg | ModRM, DstMem | SrcReg | ModRM, ByteOp | DstReg | SrcMem | ModRM, DstReg | SrcMem | ModRM,
Signature · 1 line
1.6.4.1
PATCH 44/47 KVM: Use kvm_{read,write}_guest_virt() to read and write segment descriptors
In reply to Avi KivityFrom: Mikhail Ershov <[email protected]>
Segment descriptors tables can be placed on two non-contiguous pages. This patch makes reading segment descriptors by linear address.
Signed-off-by Mikhail Ershov <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/x86.c | 10 ++-------- 1 files changed, 2 insertions(+), 8 deletions(-)
arch/x86/kvm/x86.c +2 -8
@@ -4021,7 +4021,6 @@ static void get_segment_descriptor_dtable(struct kvm_vcpu *vcpu,static int load_guest_segment_descriptor (struct kvm_vcpu *vcpu,u16 selector,struct desc_struct *seg_desc) { -gpa_t gpa;struct descriptor_table dtable;u16 index = selector >>3 ;
@@ -4031,16 +4030,13 @@ static int load_guest_segment_descriptor(struct kvm_vcpu *vcpu, u16 selector,kvm_queue_exception_e (vcpu,GP_VECTOR , selector &0xfffc );return 1 ; } - gpa = vcpu->arch.mmu.gva_to_gpa (vcpu, dtable.base); - gpa += index *8 ; -return kvm_read_guest (vcpu->kvm, gpa, seg_desc,8 ); +return kvm_read_guest_virt (dtable.base + index*8 , seg_desc,sizeof (*seg_desc), vcpu); }/* allowed just for 8 bytes segments */ static int save_guest_segment_descriptor (struct kvm_vcpu *vcpu,u16 selector,struct desc_struct *seg_desc) { -gpa_t gpa;struct descriptor_table dtable;u16 index = selector >>3 ;
@@ -4048,9 +4044,7 @@ static int save_guest_segment_descriptor(struct kvm_vcpu *vcpu, u16 selector,if (dtable.limit < index *8 +7 )return 1 ; - gpa = vcpu->arch.mmu.gva_to_gpa (vcpu, dtable.base); - gpa += index *8 ; -return kvm_write_guest (vcpu->kvm, gpa, seg_desc,8 ); +return kvm_write_guest_virt (dtable.base + index*8 , seg_desc,sizeof (*seg_desc), vcpu); } static u32 get_tss_base_addr(struct kvm_vcpu *vcpu,
Signature · 1 line
1.6.4.1
PATCH 45/47 KVM: VMX: Fix EPT with WP bit change during paging
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
QNX update WP bit when paging enabled, which is not covered yet. This one fix QNX boot with EPT.
Signed-off-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/vmx.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx.c +3 -3
@@ -1642,7 +1642,6 @@ static void ept_update_paging_mode_cr0(unsigned long *hw_cr0, CPU_BASED_CR3_STORE_EXITING)); vcpu->arch.cr0 = cr0;vmx_set_cr4 (vcpu, vcpu->arch.cr4); - *hw_cr0 &= ~X86_CR0_WP ; }else if (!is_paging (vcpu)) {/* From nonpaging to paging */ vmcs_write32(CPU_BASED_VM_EXEC_CONTROL,
@@ -1651,9 +1650,10 @@ static void ept_update_paging_mode_cr0(unsigned long *hw_cr0, CPU_BASED_CR3_STORE_EXITING)); vcpu->arch.cr0 = cr0;vmx_set_cr4 (vcpu, vcpu->arch.cr4); -if (!(vcpu->arch.cr0 &X86_CR0_WP )) - *hw_cr0 &= ~X86_CR0_WP ; } + +if (!(cr0 &X86_CR0_WP )) + *hw_cr0 &= ~X86_CR0_WP ; } static void ept_update_paging_mode_cr4(unsigned long *hw_cr4,
Signature · 1 line
1.6.4.1
PATCH 46/47 KVM: Protect update_cr8_intercept() when running without an apic
In reply to Avi Kivityupdate_cr8_intercept() can be triggered from userspace while there is no apic present.
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/x86.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
arch/x86/kvm/x86.c +3 -0
@@ -3500,6 +3500,9 @@ static void update_cr8_intercept(struct kvm_vcpu *vcpu)if (!kvm_x86_ops->update_cr8_intercept)return ; +if (!vcpu->arch.apic) +return ; +if (!vcpu->arch.apic->vapic_addr) max_irr =kvm_lapic_find_highest_irr (vcpu);else
Signature · 1 line
1.6.4.1
PATCH 47/47 KVM: Document KVM_CAP_IRQCHIP
In reply to Avi KivitySigned-off-by Avi Kivity <[email protected]>
--- Documentation/kvm/api.txt | 76 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 76 insertions(+), 0 deletions(-)
Documentation/kvm/api.txt +76 -0
@@ -517,6 +517,82 @@ struct kvm_fpu { __u32 pad2; }; +4.23 KVM_CREATE_IRQCHIP + +Capability: KVM_CAP_IRQCHIP +Architectures: x86, ia64 +Type: vm ioctl +Parameters: none +Returns: 0 on success, -1 on error + +Creates an interrupt controller model in the kernel. On x86, creates a virtual +ioapic, a virtual PIC (two PICs, nested), and sets up future vcpus to have a +local APIC. IRQ routing for GSIs 0-15 is set to both PIC and IOAPIC; GSI 16-23 +only go to the IOAPIC. On ia64, a IOSAPIC is created. + +4.24 KVM_IRQ_LINE + +Capability: KVM_CAP_IRQCHIP +Architectures: x86, ia64 +Type: vm ioctl +Parameters: struct kvm_irq_level +Returns: 0 on success, -1 on error + +Sets the level of a GSI input to the interrupt controller model in the kernel. +Requires that an interrupt controller model has been previously created with +KVM_CREATE_IRQCHIP. Note that edge-triggered interrupts require the level +to be set to 1 and then back to 0. + +struct kvm_irq_level { + union { + __u32 irq; /* GSI */ + __s32 status; /* not used for KVM_IRQ_LEVEL */ + }; + __u32 level; /* 0 or 1 */ +}; + +4.25 KVM_GET_IRQCHIP + +Capability: KVM_CAP_IRQCHIP +Architectures: x86, ia64 +Type: vm ioctl +Parameters: struct kvm_irqchip (in/out) +Returns: 0 on success, -1 on error + +Reads the state of a kernel interrupt controller created with +KVM_CREATE_IRQCHIP into a buffer provided by the caller. + +struct kvm_irqchip { + __u32 chip_id; /* 0 = PIC1, 1 = PIC2, 2 = IOAPIC */ + __u32 pad; + union { + char dummy[512]; /* reserving space */ + struct kvm_pic_state pic; + struct kvm_ioapic_state ioapic; + } chip; +}; + +4.26 KVM_SET_IRQCHIP + +Capability: KVM_CAP_IRQCHIP +Architectures: x86, ia64 +Type: vm ioctl +Parameters: struct kvm_irqchip (in) +Returns: 0 on success, -1 on error + +Sets the state of a kernel interrupt controller created with +KVM_CREATE_IRQCHIP from a buffer provided by the caller. + +struct kvm_irqchip { + __u32 chip_id; /* 0 = PIC1, 1 = PIC2, 2 = IOAPIC */ + __u32 pad; + union { + char dummy[512]; /* reserving space */ + struct kvm_pic_state pic; + struct kvm_ioapic_state ioapic; + } chip; +}; + 5. The kvm_run structure Application code obtains a pointer to the kvm_run structure by
Signature · 1 line
1.6.4.1