KVM Updates for 2.6.29 (part 2 of 3)
cover KVM Updates for 2.6.29 (part 2 of 3)
This is the second batch of the 2.6.29 merge window queue.
Avi Kivity (2):
Merge branch 'x86/crashdump' of
git://git.kernel.org/.../tip/linux-2.6-tip
KVM: VMX: Conditionally request interrupt window after injecting irq
Eduardo Habkost (12):
KVM: VMX: move vmx.h to include/asm
KVM: SVM: move svm.h to include/asm
KVM: VMX: move ASM_VMX_* definitions from asm/kvm_host.h to asm/vmx.h
KVM: VMX: move cpu_has_kvm_support() to an inline on asm/virtext.h
x86: asm/virtext.h: add cpu_vmxoff() inline function
KVM: VMX: extract kvm_cpu_vmxoff() from hardware_disable()
x86: cpu_emergency_vmxoff() function
KVM: SVM: move has_svm() code to asm/virtext.h
KVM: SVM: move svm_hardware_disable() code to asm/virtext.h
x86: cpu_emergency_svm_disable() function
kdump: forcibly disable VMX and SVM on machine_crash_shutdown()
x86: disable VMX on all CPUs on reboot
Glauber Costa (1):
x86: KVM guest: sign kvmclock as paravirt
Hollis Blanchard (11):
KVM: ppc: adjust vcpu types to support 64-bit cores
KVM: ppc: fix set regs to take care of msr change
KVM: ppc: optimize kvm stat handling
KVM: ppc: optimize find first bit
KVM: ppc: optimize irq delivery path
KVM: ppc: improve trap emulation
KVM: ensure that memslot userspace addresses are page-aligned
KVM: ppc: fix Kconfig constraints
KVM: ppc: use MMUCR accessor to obtain TID
KVM: ppc: use prefetchable mappings for guest memory
KVM: ppc: fix userspace mapping invalidation on context switch
Marcelo Tosatti (1):
KVM: MMU: optimize set_spte for page sync
Nitin A Kamble (2):
KVM: Fix cpuid leaf 0xb loop termination
KVM: Fix cpuid iteration on multiple leaves per eac
Sheng Yang (10):
KVM: Fix kernel allocated memory slot
KVM: Move ack notifier register and IRQ sourcd ID request
KVM: Separate update irq to a single function
KVM: Replace irq_requested with more generic irq_requested_type
KVM: Clean up assigned_device_update_irq
KVM: Add fields for MSI device assignment
KVM: Export ioapic_get_delivery_bitmask
KVM: Add assigned_device_msi_dispatch()
KVM: Enable MSI for device assignment
KVM: MSI to INTx translate
Xiantao Zhang (5):
KVM: ia64: Remove some macro definitions in asm-offsets.c.
KVM: ia64: Define printk function for kvm-intel module
KVM: ia64: Add some debug points to provide crash infomation
KVM: ia64: Add handler for crashed vmm
KVM: ia64: Clean up vmm_ivt.S using tab to indent every line
arch/ia64/include/asm/kvm_host.h | 4 +
arch/ia64/kvm/Makefile | 2 +-
arch/ia64/kvm/asm-offsets.c | 11 +-
arch/ia64/kvm/kvm-ia64.c | 8 +
arch/ia64/kvm/kvm_lib.c | 15 +
arch/ia64/kvm/mmio.c | 38 +-
arch/ia64/kvm/process.c | 29 +-
arch/ia64/kvm/vcpu.c | 71 ++-
arch/ia64/kvm/vcpu.h | 5 +-
arch/ia64/kvm/vmm.c | 29 +
arch/ia64/kvm/vmm_ivt.S | 1469 +++++++++++++++++-----------------
arch/powerpc/include/asm/kvm_44x.h | 2 +
arch/powerpc/include/asm/kvm_host.h | 50 +-
arch/powerpc/include/asm/kvm_ppc.h | 3 -
arch/powerpc/kvm/44x_emulate.c | 109 ++-
arch/powerpc/kvm/44x_tlb.c | 42 +-
arch/powerpc/kvm/Kconfig | 18 +-
arch/powerpc/kvm/booke.c | 221 ++----
arch/powerpc/kvm/booke.h | 18 +
arch/powerpc/kvm/emulate.c | 6 +-
arch/powerpc/kvm/powerpc.c | 4 +-
arch/x86/include/asm/kexec.h | 31 +-
arch/x86/include/asm/kvm_host.h | 12 -
arch/x86/include/asm/reboot.h | 5 +
arch/x86/{kvm => include/asm}/svm.h | 0
arch/x86/include/asm/virtext.h | 132 +++
arch/x86/{kvm => include/asm}/vmx.h | 15 +
arch/x86/kernel/crash.c | 86 +--
arch/x86/kernel/kvmclock.c | 2 +
arch/x86/kernel/machine_kexec_32.c | 104 ++-
arch/x86/kernel/reboot.c | 157 ++++-
arch/x86/kernel/relocate_kernel_32.S | 115 ---
arch/x86/kvm/kvm_svm.h | 2 +-
arch/x86/kvm/mmu.c | 11 +-
arch/x86/kvm/svm.c | 25 +-
arch/x86/kvm/vmx.c | 21 +-
arch/x86/kvm/x86.c | 5 +-
include/linux/kexec.h | 4 +
include/linux/kvm.h | 10 +
include/linux/kvm_host.h | 8 +-
virt/kvm/ioapic.c | 7 +-
virt/kvm/ioapic.h | 2 +
virt/kvm/kvm_main.c | 255 +++++-
43 files changed, 1834 insertions(+), 1329 deletions(-)
create mode 100644 arch/ia64/kvm/kvm_lib.c
rename arch/x86/{kvm => include/asm}/svm.h (100%)
create mode 100644 arch/x86/include/asm/virtext.h
rename arch/x86/{kvm => include/asm}/vmx.h (95%)PATCH 1/44 KVM: ppc: adjust vcpu types to support 64-bit cores
From: Hollis Blanchard <[email protected]>
However, some of these fields could be split into separate per-core structures in the future.
Signed-off-by Hollis Blanchard <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/powerpc/include/asm/kvm_host.h | 50 +++++++++++++++++----------------- arch/powerpc/kvm/booke.c | 10 +++--- arch/powerpc/kvm/emulate.c | 2 +- arch/powerpc/kvm/powerpc.c | 4 +- 4 files changed, 33 insertions(+), 33 deletions(-)
arch/powerpc/include/asm/kvm_host.h +25 -25
@@ -84,32 +84,32 @@ struct kvm_vcpu_arch {u32 host_msr;u64 fpr[32 ]; -u32 gpr[32 ]; +ulong gpr[32 ]; -u32 pc; +ulong pc;u32 cr; -u32 ctr; -u32 lr; -u32 xer; +ulong ctr; +ulong lr; +ulong xer; -u32 msr; +ulong msr;u32 mmucr; -u32 sprg0; -u32 sprg1; -u32 sprg2; -u32 sprg3; -u32 sprg4; -u32 sprg5; -u32 sprg6; -u32 sprg7; -u32 srr0; -u32 srr1; -u32 csrr0; -u32 csrr1; -u32 dsrr0; -u32 dsrr1; -u32 dear; -u32 esr; +ulong sprg0; +ulong sprg1; +ulong sprg2; +ulong sprg3; +ulong sprg4; +ulong sprg5; +ulong sprg6; +ulong sprg7; +ulong srr0; +ulong srr1; +ulong csrr0; +ulong csrr1; +ulong dsrr0; +ulong dsrr1; +ulong dear; +ulong esr;u32 dec;u32 decar;u32 tbl;
@@ -117,7 +117,7 @@ struct kvm_vcpu_arch {u32 tcr;u32 tsr;u32 ivor[16 ]; -u32 ivpr; +ulong ivpr;u32 pir;u32 shadow_pid;
@@ -131,8 +131,8 @@ struct kvm_vcpu_arch {u32 dbcr1;u32 last_inst; -u32 fault_dear; -u32 fault_esr; +ulong fault_dear; +ulong fault_esr;gpa_t paddr_accessed;u8 io_gpr;/* GPR used as IO source/target */
arch/powerpc/kvm/booke.c +5 -5
@@ -120,14 +120,14 @@ void kvmppc_dump_vcpu(struct kvm_vcpu *vcpu) {int i; -printk ("pc: %08x msr: %08x\n" , vcpu->arch.pc, vcpu->arch.msr); -printk ("lr: %08x ctr: %08x\n" , vcpu->arch.lr, vcpu->arch.ctr); -printk ("srr0: %08x srr1: %08x\n" , vcpu->arch.srr0, vcpu->arch.srr1); +printk ("pc: %08lx msr: %08lx\n" , vcpu->arch.pc, vcpu->arch.msr); +printk ("lr: %08lx ctr: %08lx\n" , vcpu->arch.lr, vcpu->arch.ctr); +printk ("srr0: %08lx srr1: %08lx\n" , vcpu->arch.srr0, vcpu->arch.srr1);printk ("exceptions: %08lx\n" , vcpu->arch.pending_exceptions);for (i =0 ; i <32 ; i +=4 ) { - printk("gpr%02d: %08x %08x %08x %08x\n", i, + printk("gpr%02d: %08lx %08lx %08lx %08lx\n", i, vcpu->arch.gpr[i], vcpu->arch.gpr[i+1], vcpu->arch.gpr[i+2],
@@ -305,7 +305,7 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,break ;case EMULATE_FAIL :/* XXX Deliver Program interrupt to guest. */ -printk (KERN_CRIT "%s: emulation at %x failed (%08x)\n" , +printk (KERN_CRIT "%s: emulation at %lx failed (%08x)\n" , __func__, vcpu->arch.pc, vcpu->arch.last_inst);/* For debugging, encode the failing instruction and * report it to userspace. */
arch/powerpc/kvm/emulate.c +1 -1
@@ -380,7 +380,7 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu) } } -KVMTRACE_3D (PPC_INSTR , vcpu, inst, vcpu->arch.pc, emulated, entryexit); +KVMTRACE_3D (PPC_INSTR , vcpu, inst, (int )vcpu->arch.pc, emulated, entryexit);if (advance) vcpu->arch.pc +=4 ;/* Advance past emulated instruction. */
arch/powerpc/kvm/powerpc.c +2 -2
@@ -250,14 +250,14 @@ int kvm_arch_vcpu_ioctl_debug_guest(struct kvm_vcpu *vcpu,static void kvmppc_complete_dcr_load (struct kvm_vcpu *vcpu,struct kvm_run *run) { -u32 *gpr = &vcpu->arch.gpr[vcpu->arch.io_gpr]; +ulong *gpr = &vcpu->arch.gpr[vcpu->arch.io_gpr]; *gpr = run->dcr.data; }static void kvmppc_complete_mmio_load (struct kvm_vcpu *vcpu,struct kvm_run *run) { -u32 *gpr = &vcpu->arch.gpr[vcpu->arch.io_gpr]; +ulong *gpr = &vcpu->arch.gpr[vcpu->arch.io_gpr];if (run->mmio.len >sizeof (*gpr)) {printk (KERN_ERR "bad MMIO length: %d\n" , run->mmio.len);
Signature · 1 line
1.6.0.3
PATCH 2/44 KVM: ppc: fix set regs to take care of msr change
In reply to Avi KivityFrom: Hollis Blanchard <[email protected]>
When changing some msr bits e.g. problem state we need to take special care of that. We call the function in our mtmsr emulation (not needed for wrtee[i]), but we don't call kvmppc_set_msr if we change msr via set_regs ioctl.
It's a corner case we never hit so far, but I assume it should be kvmppc_set_msr in our arch set regs function (I found it because it is also a corner case when using pv support which would miss the update otherwise).
Signed-off-by Christian Ehrhardt <[email protected]>
Signed-off-by Hollis Blanchard <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/powerpc/kvm/booke.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/powerpc/kvm/booke.c +1 -1
@@ -528,7 +528,7 @@ int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) vcpu->arch.ctr = regs->ctr; vcpu->arch.lr = regs->lr; vcpu->arch.xer = regs->xer; - vcpu->arch.msr = regs->msr; +kvmppc_set_msr (vcpu, regs->msr); vcpu->arch.srr0 = regs->srr0; vcpu->arch.srr1 = regs->srr1; vcpu->arch.sprg0 = regs->sprg0;
Signature · 1 line
1.6.0.3
PATCH 3/44 KVM: ppc: optimize kvm stat handling
In reply to Avi KivityFrom: Hollis Blanchard <[email protected]>
Currently we use an unnecessary if&switch to detect some cases. To be honest we don't need the ligh_exits counter anyway, because we can calculate it out of others. Sum_exits can also be calculated, so we can remove that too.
MMIO, DCR and INTR can be counted on other places without these additional control structures (The INTR case was never hit anyway).
The handling of BOOKE_INTERRUPT_EXTERNAL/BOOKE_INTERRUPT_DECREMENTER is similar, but we can avoid the additional if when copying 3 lines of code. I thought about a goto there to prevent duplicate lines, but rewriting three lines should be better style than a goto cross switch/case statements (its also not enough code to justify a new inline function).
Signed-off-by Christian Ehrhardt <[email protected]>
Signed-off-by Hollis Blanchard <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/powerpc/kvm/booke.c | 32 +++++++++----------------------- 1 files changed, 9 insertions(+), 23 deletions(-)
arch/powerpc/kvm/booke.c +9 -23
@@ -38,11 +38,9 @@ unsigned long kvmppc_booke_handlers;#define VCPU_STAT (x) offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPUstruct kvm_stats_debugfs_item debugfs_entries[] = { - {"exits" ,VCPU_STAT (sum_exits) }, {"mmio" ,VCPU_STAT (mmio_exits) }, {"dcr" ,VCPU_STAT (dcr_exits) }, {"sig" ,VCPU_STAT (signal_exits) }, - {"light" ,VCPU_STAT (light_exits) }, {"itlb_r" ,VCPU_STAT (itlb_real_miss_exits) }, {"itlb_v" ,VCPU_STAT (itlb_virt_miss_exits) }, {"dtlb_r" ,VCPU_STAT (dtlb_real_miss_exits) },
@@ -263,6 +261,12 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,break ;case BOOKE_INTERRUPT_EXTERNAL : + vcpu->stat.ext_intr_exits++; +if (need_resched ()) +cond_resched (); + r =RESUME_GUEST ; +break ; +case BOOKE_INTERRUPT_DECREMENTER :/* Since we switched IVPR back to the host's value, the host * handled this interrupt the moment we enabled interrupts.
@@ -272,12 +276,9 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,* we do reschedule the host will fault over it. Perhaps we * should politely restore the host's entries to minimize * misses before ceding control. */ + vcpu->stat.dec_exits++;if (need_resched ())cond_resched (); -if (exit_nr ==BOOKE_INTERRUPT_DECREMENTER ) - vcpu->stat.dec_exits++; -else - vcpu->stat.ext_intr_exits++; r =RESUME_GUEST ;break ;
@@ -301,6 +302,7 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,break ;case EMULATE_DO_DCR : run->exit_reason =KVM_EXIT_DCR ; + vcpu->stat.dcr_exits++; r =RESUME_HOST ;break ;case EMULATE_FAIL :
@@ -379,6 +381,7 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,/* Guest has mapped and accessed a page which is not * actually RAM. */ r =kvmppc_emulate_mmio (run, vcpu); + vcpu->stat.mmio_exits++; }break ;
@@ -445,8 +448,6 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,kvmppc_core_deliver_interrupts (vcpu); -/* Do some exit accounting. */ - vcpu->stat.sum_exits++;if (!(r &RESUME_HOST )) {/* To avoid clobbering exit_reason, only check for signals if * we aren't already exiting to userspace for some other
@@ -454,22 +455,7 @@ int kvmppc_handle_exit(struct kvm_run *run, struct kvm_vcpu *vcpu,if (signal_pending (current)) { run->exit_reason =KVM_EXIT_INTR ; r = (-EINTR <<2 ) |RESUME_HOST | (r &RESUME_FLAG_NV ); - vcpu->stat.signal_exits++; - }else { - vcpu->stat.light_exits++; - } - }else { -switch (run->exit_reason) { -case KVM_EXIT_MMIO : - vcpu->stat.mmio_exits++; -break ; -case KVM_EXIT_DCR : - vcpu->stat.dcr_exits++; -break ; -case KVM_EXIT_INTR : - vcpu->stat.signal_exits++; -break ; } }
Signature · 1 line
1.6.0.3
PATCH 4/44 KVM: ppc: optimize find first bit
In reply to Avi KivityFrom: Hollis Blanchard <[email protected]>
Since we use a unsigned long here anyway we can use the optimized __ffs.
Signed-off-by Christian Ehrhardt <[email protected]>
Signed-off-by Hollis Blanchard <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/powerpc/kvm/booke.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/powerpc/kvm/booke.c +1 -1
@@ -222,7 +222,7 @@ void kvmppc_core_deliver_interrupts(struct kvm_vcpu *vcpu)unsigned int exception;unsigned int priority; - priority =find_first_bit (pending,BITS_PER_BYTE *sizeof (*pending)); + priority =__ffs (*pending);while (priority <=BOOKE_MAX_INTERRUPT ) { exception = priority_exception[priority];if (kvmppc_can_deliver_interrupt (vcpu, exception)) {
Signature · 1 line
1.6.0.3
PATCH 5/44 KVM: ppc: optimize irq delivery path
In reply to Avi KivityFrom: Hollis Blanchard <[email protected]>
In kvmppc_deliver_interrupt is just one case left in the switch and it is a rare one (less than 8%) when looking at the exit numbers. Therefore we can at least drop the switch/case and if an if. I inserted an unlikely too, but that's open for discussion.
In kvmppc_can_deliver_interrupt all frequent cases are in the default case. I know compilers are smart but we can make it easier for them. By writing down all options and removing the default case combined with the fact that ithe values are constants 0..15 should allow the compiler to write an easy jump table.
Modifying kvmppc_can_deliver_interrupt pointed me to the fact that gcc seems to be unable to reduce priority_exception[x] to a build time constant. Therefore I changed the usage of the translation arrays in the interrupt delivery path completely. It is now using priority without translation to irq on the full irq delivery path.
To be able to do that ivpr regs are stored by their priority now.
Additionally the decision made in kvmppc_can_deliver_interrupt is already sufficient to get the value of interrupt_msr_mask[x]. Therefore we can replace the 16x4byte array used here with a single 4byte variable (might still be one miss, but the chance to find this in cache should be better than the right entry of the whole array).
Signed-off-by Christian Ehrhardt <[email protected]>
Signed-off-by Hollis Blanchard <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/powerpc/include/asm/kvm_ppc.h | 3 - arch/powerpc/kvm/44x_emulate.c | 100 ++++++++++++++------- arch/powerpc/kvm/booke.c | 175 +++++++++++------------------------ arch/powerpc/kvm/booke.h | 18 ++++ 4 files changed, 140 insertions(+), 156 deletions(-)
arch/powerpc/include/asm/kvm_ppc.h +0 -3
@@ -36,9 +36,6 @@ enum emulation_result { EMULATE_FAIL, /* can't emulate this instruction */ }; -extern const unsigned char exception_priority[]; -extern const unsigned char priority_exception[]; -extern int __kvmppc_vcpu_run (struct kvm_run *kvm_run,struct kvm_vcpu *vcpu);extern char kvmppc_handlers_start[];extern unsigned long kvmppc_handler_len;
arch/powerpc/kvm/44x_emulate.c +67 -33
143 lines not on this page · raw message
arch/powerpc/kvm/booke.c +55 -120
277 lines not on this page · raw message
arch/powerpc/kvm/booke.h +18 -0
@@ -23,6 +23,24 @@#include <linux/types.h> #include <linux/kvm_host.h> +/* interrupt priortity ordering */ +#define BOOKE_IRQPRIO_DATA_STORAGE 0 +#define BOOKE_IRQPRIO_INST_STORAGE 1 +#define BOOKE_IRQPRIO_ALIGNMENT 2 +#define BOOKE_IRQPRIO_PROGRAM 3 +#define BOOKE_IRQPRIO_FP_UNAVAIL 4 +#define BOOKE_IRQPRIO_SYSCALL 5 +#define BOOKE_IRQPRIO_AP_UNAVAIL 6 +#define BOOKE_IRQPRIO_DTLB_MISS 7 +#define BOOKE_IRQPRIO_ITLB_MISS 8 +#define BOOKE_IRQPRIO_MACHINE_CHECK 9 +#define BOOKE_IRQPRIO_DEBUG 10 +#define BOOKE_IRQPRIO_CRITICAL 11 +#define BOOKE_IRQPRIO_WATCHDOG 12 +#define BOOKE_IRQPRIO_EXTERNAL 13 +#define BOOKE_IRQPRIO_FIT 14 +#define BOOKE_IRQPRIO_DECREMENTER 15 +/* Helper function for "full" MSR writes. No need to call this if only EE is * changing. */ static inline void kvmppc_set_msr (struct kvm_vcpu *vcpu,u32 new_msr)
Signature · 1 line
1.6.0.3
PATCH 6/44 KVM: ppc: improve trap emulation
In reply to Avi KivityFrom: Hollis Blanchard <[email protected]>
set ESR[PTR] when emulating a guest trap. This allows Linux guests to properly handle WARN_ON() (i.e. detect that it's a non-fatal trap).
Also remove debugging printk in trap emulation.
Signed-off-by Hollis Blanchard <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/powerpc/kvm/emulate.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
arch/powerpc/kvm/emulate.c +2 -2
@@ -74,8 +74,8 @@ int kvmppc_emulate_instruction(struct kvm_run *run, struct kvm_vcpu *vcpu)int advance =1 ;switch (get_op (inst)) { -case 3 :/* trap */ -printk ("trap!\n" ); +case 3 :/* trap */ + vcpu->arch.esr |=ESR_PTR ;kvmppc_core_queue_program (vcpu); advance =0 ;break ;
Signature · 1 line
1.6.0.3
PATCH 7/44 KVM: Fix cpuid leaf 0xb loop termination
In reply to Avi KivityFrom: Nitin A Kamble <[email protected]>
For cpuid leaf 0xb the bits 8-15 in ECX register define the end of counting
leaf. The previous code was using bits 0-7 for this purpose, which is
a bug.
Signed-off-by Nitin A Kamble <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/x86.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/x86/kvm/x86.c +1 -1
@@ -1276,7 +1276,7 @@ static void do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function, entry->flags |=KVM_CPUID_FLAG_SIGNIFCANT_INDEX ;/* read more entries until level_type is zero */ for (i =1 ; *nent < maxnent; ++i) { - level_type = entry[i -1 ].ecx &0xff ; + level_type = entry[i -1 ].ecx &0xff00 ;if (!level_type)break ;do_cpuid_1_ent (&entry[i], function, i);
Signature · 1 line
1.6.0.3
PATCH 8/44 KVM: Fix cpuid iteration on multiple leaves per eac
In reply to Avi KivityFrom: Nitin A Kamble <[email protected]>
The code to traverse the cpuid data array list for counting type of leaves is currently broken.
This patches fixes the 2 things in it.
1. Set the 1st counting entry's flag KVM_CPUID_FLAG_STATE_READ_NEXT. Without it the code will never find a valid entry.
2. Also the stop condition in the for loop while looking for the next unflaged entry is broken. It needs to stop when it find one matching entry; and in the case of count of 1, it will be the same entry found in this iteration.
Signed-off-by Nitin A Kamble <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/x86.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
arch/x86/kvm/x86.c +2 -1
@@ -1246,6 +1246,7 @@ static void do_cpuid_ent(struct kvm_cpuid_entry2 *entry, u32 function,int t, times = entry->eax &0xff ; entry->flags |=KVM_CPUID_FLAG_STATEFUL_FUNC ; + entry->flags |=KVM_CPUID_FLAG_STATE_READ_NEXT ;for (t =1 ; t < times && *nent < maxnent; ++t) {do_cpuid_1_ent (&entry[t], function,0 ); entry[t].flags |=KVM_CPUID_FLAG_STATEFUL_FUNC ;
@@ -2801,7 +2802,7 @@ static int move_to_next_stateful_cpuid_entry(struct kvm_vcpu *vcpu, int i) e->flags &= ~KVM_CPUID_FLAG_STATE_READ_NEXT ;/* when no next entry is found, the current entry[i] is reselected */ -for (j = i +1 ; j == i; j = (j +1 ) % nent) { +for (j = i +1 ; ; j = (j +1 ) % nent) {struct kvm_cpuid_entry2 *ej = &vcpu->arch.cpuid_entries[j];if (ej->function == e->function) { ej->flags |=KVM_CPUID_FLAG_STATE_READ_NEXT ;
Signature · 1 line
1.6.0.3
PATCH 9/44 KVM: ensure that memslot userspace addresses are page-aligned
In reply to Avi KivityFrom: Hollis Blanchard <[email protected]>
Bad page translation and silent guest failure ensue if the userspace address is not page-aligned. I hit this problem using large (host) pages with qemu, because qemu currently has a hardcoded 4096-byte alignment for guest memory allocations.
Signed-off-by Hollis Blanchard <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- virt/kvm/kvm_main.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
virt/kvm/kvm_main.c +2 -0
@@ -715,6 +715,8 @@ int __kvm_set_memory_region(struct kvm *kvm, goto out;if (mem->guest_phys_addr & (PAGE_SIZE -1 )) goto out; +if (mem->userspace_addr & (PAGE_SIZE -1 )) + goto out;if (mem->slot >=KVM_MEMORY_SLOTS +KVM_PRIVATE_MEM_SLOTS ) goto out;if (mem->guest_phys_addr + mem->memory_size < mem->guest_phys_addr)
Signature · 1 line
1.6.0.3
PATCH 10/44 KVM: ppc: fix Kconfig constraints
In reply to Avi KivityFrom: Hollis Blanchard <[email protected]>
Make sure that CONFIG_KVM cannot be selected without processor support (currently, 440 is the only processor implementation available).
Signed-off-by Hollis Blanchard <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/powerpc/kvm/Kconfig | 18 ++++++++---------- 1 files changed, 8 insertions(+), 10 deletions(-)
arch/powerpc/kvm/Kconfig +8 -10
@@ -15,25 +15,23 @@ menuconfig VIRTUALIZATION if VIRTUALIZATION config KVM - bool "Kernel-based Virtual Machine (KVM) support" - depends on EXPERIMENTAL + bool select PREEMPT_NOTIFIERS select ANON_INODES + +config KVM_440 + bool "KVM support for PowerPC 440 processors" + depends on EXPERIMENTAL && 44x + select KVM ---help--- - Support hosting virtualized guest machines. You will also - need to select one or more of the processor modules below. + Support running unmodified 440 guest kernels in virtual machines on + 440 host processors. This module provides access to the hardware capabilities through a character device node named /dev/kvm. If unsure, say N. -config KVM_440 - bool "KVM support for PowerPC 440 processors" - depends on KVM && 44x - ---help--- - KVM can run unmodified 440 guest kernels on 440 host processors. - config KVM_TRACE bool "KVM trace support" depends on KVM && MARKERS && SYSFS
Signature · 1 line
1.6.0.3
PATCH 11/44 KVM: ia64: Remove some macro definitions in asm-offsets.c.
In reply to Avi KivityFrom: Xiantao Zhang <[email protected]>
Use kernel's corresponding macro instead.
Signed-off-by Xiantao Zhang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/ia64/kvm/asm-offsets.c | 11 +---------- 1 files changed, 1 insertions(+), 10 deletions(-)
arch/ia64/kvm/asm-offsets.c +1 -10
@@ -24,19 +24,10 @@#include <linux/autoconf.h> #include <linux/kvm_host.h> +#include <linux/kbuild.h> #include "vcpu.h" -#define task_struct kvm_vcpu - -#define DEFINE (sym, val) \ - asm volatile("\n->" #sym " (%0) " #val : : "i" (val)) - -#define BLANK () asm volatile("\n->" : :) - -#define OFFSET (_sym, _str, _mem) \ - DEFINE(_sym, offsetof(_str, _mem)); -void foo (void ) {DEFINE (VMM_TASK_SIZE ,sizeof (struct kvm_vcpu ));
Signature · 1 line
1.6.0.3
PATCH 12/44 KVM: Fix kernel allocated memory slot
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
Commit 7fd49de9773fdcb7b75e823b21c1c5dc1e218c14 "KVM: ensure that memslot userspace addresses are page-aligned" broke kernel space allocated memory slot, for the userspace_addr is invalid.
Signed-off-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- virt/kvm/kvm_main.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
virt/kvm/kvm_main.c +1 -1
@@ -715,7 +715,7 @@ int __kvm_set_memory_region(struct kvm *kvm, goto out;if (mem->guest_phys_addr & (PAGE_SIZE -1 )) goto out; -if (mem->userspace_addr & (PAGE_SIZE -1 )) +if (user_alloc && (mem->userspace_addr & (PAGE_SIZE -1 ))) goto out;if (mem->slot >=KVM_MEMORY_SLOTS +KVM_PRIVATE_MEM_SLOTS ) goto out;
Signature · 1 line
1.6.0.3
PATCH 13/44 KVM: ppc: use MMUCR accessor to obtain TID
In reply to Avi KivityFrom: Hollis Blanchard <[email protected]>
We have an accessor; might as well use it.
Signed-off-by Hollis Blanchard <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/powerpc/kvm/44x_tlb.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
arch/powerpc/kvm/44x_tlb.c +1 -1
@@ -339,7 +339,7 @@ int kvmppc_44x_emul_tlbwe(struct kvm_vcpu *vcpu, u8 ra, u8 rs, u8 ws)switch (ws) {case PPC44x_TLB_PAGEID: - tlbe->tid = vcpu->arch.mmucr &0xff ; + tlbe->tid =get_mmucr_stid (vcpu); tlbe->word0 = vcpu->arch.gpr[rs];break ;
Signature · 1 line
1.6.0.3
PATCH 14/44 KVM: ppc: use prefetchable mappings for guest memory
In reply to Avi KivityFrom: Hollis Blanchard <[email protected]>
Bare metal Linux on 440 can "overmap" RAM in the kernel linear map, so that it can use large (256MB) mappings even if memory isn't a multiple of 256MB. To prevent the hardware prefetcher from loading from an invalid physical address through that mapping, it's marked Guarded.
However, KVM must ensure that all guest mappings are backed by real physical RAM (since a deliberate access through a guarded mapping could still cause a machine check). Accordingly, we don't need to make our mappings guarded, so let's allow prefetching as the designers intended.
Curiously this patch didn't affect performance at all on the quick test I tried, but it's clearly the right thing to do anyways and may improve other workloads.
Signed-off-by Hollis Blanchard <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/powerpc/kvm/44x_tlb.c | 9 +++++++-- 1 files changed, 7 insertions(+), 2 deletions(-)
arch/powerpc/kvm/44x_tlb.c +7 -2
@@ -28,6 +28,8 @@#include "44x_tlb.h" +#define PPC44x_TLB_UATTR_MASK \ + (PPC44x_TLB_U0|PPC44x_TLB_U1|PPC44x_TLB_U2|PPC44x_TLB_U3)#define PPC44x_TLB_USER_PERM_MASK (PPC44x_TLB_UX|PPC44x_TLB_UR|PPC44x_TLB_UW)#define PPC44x_TLB_SUPER_PERM_MASK (PPC44x_TLB_SX|PPC44x_TLB_SR|PPC44x_TLB_SW)
@@ -63,8 +65,8 @@ void kvmppc_dump_tlbs(struct kvm_vcpu *vcpu)static u32 kvmppc_44x_tlb_shadow_attrib (u32 attrib,int usermode) { -/* Mask off reserved bits. */ - attrib &= PPC44x_TLB_PERM_MASK|PPC44x_TLB_ATTR_MASK; +/* We only care about the guest's permission and user bits. */ + attrib &= PPC44x_TLB_PERM_MASK|PPC44x_TLB_UATTR_MASK;if (!usermode) {/* Guest is in supervisor mode, so we need to translate guest
@@ -76,6 +78,9 @@ static u32 kvmppc_44x_tlb_shadow_attrib(u32 attrib, int usermode)/* Make sure host can always access this memory. */ attrib |= PPC44x_TLB_SX|PPC44x_TLB_SR|PPC44x_TLB_SW; +/* WIMGE = 0b00100 */ + attrib |= PPC44x_TLB_M; +return attrib; }
Signature · 1 line
1.6.0.3
PATCH 15/44 KVM: ppc: fix userspace mapping invalidation on context switch
In reply to Avi KivityFrom: Hollis Blanchard <[email protected]>
We used to defer invalidating userspace TLB entries until jumping out of the kernel. This was causing MMU weirdness most easily triggered by using a pipe in the guest, e.g. "dmesg | tail". I believe the problem was that after the guest kernel changed the PID (part of context switch), the old process's mappings were still present, and so copy_to_user() on the "return to new process" path ended up using stale mappings.
Testing with large pages (64K) exposed the problem, probably because with 4K pages, pressure on the TLB faulted all process A's mappings out before the guest kernel could insert any for process B.
Signed-off-by Hollis Blanchard <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/powerpc/include/asm/kvm_44x.h | 2 ++ arch/powerpc/kvm/44x_emulate.c | 9 +-------- arch/powerpc/kvm/44x_tlb.c | 31 +++++++++++++++++-------------- 3 files changed, 20 insertions(+), 22 deletions(-)
arch/powerpc/include/asm/kvm_44x.h +2 -0
@@ -44,4 +44,6 @@ static inline struct kvmppc_vcpu_44x *to_44x(struct kvm_vcpu *vcpu) return container_of(vcpu, struct kvmppc_vcpu_44x, vcpu); } +void kvmppc_set_pid (struct kvm_vcpu *vcpu,u32 new_pid); +#endif /* __ASM_44X_H__ */
arch/powerpc/kvm/44x_emulate.c +1 -8
@@ -21,6 +21,7 @@#include <asm/dcr.h> #include <asm/dcr-regs.h> #include <asm/disassemble.h> +#include <asm/kvm_44x.h> #include "booke.h" #include "44x_tlb.h"
@@ -38,14 +39,6 @@#define XOP_ICCCI 966#define XOP_TLBWE 978 -static inline void kvmppc_set_pid (struct kvm_vcpu *vcpu,u32 new_pid) -{ -if (vcpu->arch.pid != new_pid) { - vcpu->arch.pid = new_pid; - vcpu->arch.swap_pid =1 ; - } -} -static void kvmppc_emul_rfi (struct kvm_vcpu *vcpu) { vcpu->arch.pc = vcpu->arch.srr0;
arch/powerpc/kvm/44x_tlb.c +17 -14
@@ -268,31 +268,34 @@ static void kvmppc_mmu_invalidate(struct kvm_vcpu *vcpu, gva_t eaddr, } } -/* Invalidate all mappings on the privilege switch after PID has been changed. -* The guest always runs with PID=1, so we must clear the entire TLB when -* switching address spaces. */ void kvmppc_mmu_priv_switch (struct kvm_vcpu *vcpu,int usermode) { + vcpu->arch.shadow_pid = !usermode; +} + +void kvmppc_set_pid (struct kvm_vcpu *vcpu,u32 new_pid) +{struct kvmppc_vcpu_44x *vcpu_44x =to_44x (vcpu);int i; -if (vcpu->arch.swap_pid) { -/* XXX Replace loop with fancy data structures. */ -for (i =0 ; i <= tlb_44x_hwater; i++) { -struct kvmppc_44x_tlbe *stlbe = &vcpu_44x->shadow_tlb[i]; +if (unlikely (vcpu->arch.pid == new_pid)) +return ; + + vcpu->arch.pid = new_pid; + +/* Guest userspace runs with TID=0 mappings and PID=0, to make sure it +* can't access guest kernel mappings (TID=1). When we switch to a new +* guest PID, which will also use host PID=0, we must discard the old guest +* userspace mappings. */ +for (i =0 ; i <ARRAY_SIZE (vcpu_44x->shadow_tlb); i++) { +struct kvmppc_44x_tlbe *stlbe = &vcpu_44x->shadow_tlb[i]; -/* Future optimization: clear only userspace mappings. */ +if (get_tlb_tid (stlbe) ==0 ) {kvmppc_44x_shadow_release (vcpu, i); stlbe->word0 =0 ;kvmppc_tlbe_set_modified (vcpu, i); -KVMTRACE_5D (STLB_INVAL , vcpu, i, - stlbe->tid, stlbe->word0, stlbe->word1, - stlbe->word2, handler); } - vcpu->arch.swap_pid =0 ; } - - vcpu->arch.shadow_pid = !usermode; } static int tlbe_is_host_safe(const struct kvm_vcpu *vcpu,
Signature · 1 line
1.6.0.3
PATCH 16/44 Merge branch 'x86/crashdump' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
In reply to Avi Kivity--- arch/x86/include/asm/kexec.h | 31 +++++----- arch/x86/include/asm/reboot.h | 5 ++ arch/x86/kernel/crash.c | 70 +++------------------ arch/x86/kernel/machine_kexec_32.c | 104 ++++++++++++++++++++++++------- arch/x86/kernel/reboot.c | 95 ++++++++++++++++++++++++++++ arch/x86/kernel/relocate_kernel_32.S | 115 ---------------------------------- include/linux/kexec.h | 4 + 7 files changed, 210 insertions(+), 214 deletions(-)
[This is a merge from the x86 forest]
PATCH 17/44 KVM: VMX: move vmx.h to include/asm
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
vmx.h will be used by core code that is independent of KVM, so I am moving it outside the arch/x86/kvm directory.
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
---
arch/x86/{kvm => include/asm}/vmx.h | 0
arch/x86/kvm/mmu.c | 2 +-
arch/x86/kvm/vmx.c | 2 +-
3 files changed, 2 insertions(+), 2 deletions(-)
rename arch/x86/{kvm => include/asm}/vmx.h (100%)arch/x86/include/asm/vmx.h renamed from arch/x86/kvm/vmx.h +0 -0
arch/x86/kvm/mmu.c +1 -1
@@ -17,7 +17,6 @@* */ -#include "vmx.h" #include "mmu.h" #include <linux/kvm_host.h>
@@ -33,6 +32,7 @@#include <asm/page.h> #include <asm/cmpxchg.h> #include <asm/io.h> +#include <asm/vmx.h> /* * When setting this variable to true it enables Two-Dimensional-Paging
arch/x86/kvm/vmx.c +1 -1
@@ -16,7 +16,6 @@*/ #include "irq.h" -#include "vmx.h" #include "mmu.h" #include <linux/kvm_host.h>
@@ -31,6 +30,7 @@#include <asm/io.h> #include <asm/desc.h> +#include <asm/vmx.h> #define __ex (x) __kvm_handle_fault_on_reboot(x)
Signature · 1 line
1.6.0.3
PATCH 18/44 KVM: SVM: move svm.h to include/asm
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
svm.h will be used by core code that is independent of KVM, so I am moving it outside the arch/x86/kvm directory.
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
---
arch/x86/{kvm => include/asm}/svm.h | 0
arch/x86/kvm/kvm_svm.h | 2 +-
2 files changed, 1 insertions(+), 1 deletions(-)
rename arch/x86/{kvm => include/asm}/svm.h (100%)arch/x86/include/asm/svm.h renamed from arch/x86/kvm/svm.h +0 -0
arch/x86/kvm/kvm_svm.h +1 -1
@@ -7,7 +7,7 @@#include <linux/kvm_host.h> #include <asm/msr.h> -#include "svm.h" +#include <asm/svm.h> static const u32 host_save_user_msrs[] = { #ifdef CONFIG_X86_64
Signature · 1 line
1.6.0.3
PATCH 19/44 KVM: VMX: move ASM_VMX_* definitions from asm/kvm_host.h to asm/vmx.h
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
Those definitions will be used by code outside KVM, so move it outside of a KVM-specific source file.
Those definitions are used only on kvm/vmx.c, that already includes asm/vmx.h, so they can be moved safely.
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/include/asm/kvm_host.h | 12 ------------ arch/x86/include/asm/vmx.h | 15 +++++++++++++++ 2 files changed, 15 insertions(+), 12 deletions(-)
arch/x86/include/asm/kvm_host.h +0 -12
@@ -714,18 +714,6 @@ static inline void kvm_inject_gp(struct kvm_vcpu *vcpu, u32 error_code)kvm_queue_exception_e (vcpu,GP_VECTOR , error_code); } -#define ASM_VMX_VMCLEAR_RAX ".byte 0x66, 0x0f, 0xc7, 0x30" -#define ASM_VMX_VMLAUNCH ".byte 0x0f, 0x01, 0xc2" -#define ASM_VMX_VMRESUME ".byte 0x0f, 0x01, 0xc3" -#define ASM_VMX_VMPTRLD_RAX ".byte 0x0f, 0xc7, 0x30" -#define ASM_VMX_VMREAD_RDX_RAX ".byte 0x0f, 0x78, 0xd0" -#define ASM_VMX_VMWRITE_RAX_RDX ".byte 0x0f, 0x79, 0xd0" -#define ASM_VMX_VMWRITE_RSP_RDX ".byte 0x0f, 0x79, 0xd4" -#define ASM_VMX_VMXOFF ".byte 0x0f, 0x01, 0xc4" -#define ASM_VMX_VMXON_RAX ".byte 0xf3, 0x0f, 0xc7, 0x30" -#define ASM_VMX_INVEPT ".byte 0x66, 0x0f, 0x38, 0x80, 0x08" -#define ASM_VMX_INVVPID ".byte 0x66, 0x0f, 0x38, 0x81, 0x08" -#define MSR_IA32_TIME_STAMP_COUNTER 0x010#define TSS_IOPB_BASE_OFFSET 0x66
arch/x86/include/asm/vmx.h +15 -0
@@ -364,4 +364,19 @@ enum vmcs_field {#define VMX_EPT_IDENTITY_PAGETABLE_ADDR 0xfffbc000ul + +#define ASM_VMX_VMCLEAR_RAX ".byte 0x66, 0x0f, 0xc7, 0x30" +#define ASM_VMX_VMLAUNCH ".byte 0x0f, 0x01, 0xc2" +#define ASM_VMX_VMRESUME ".byte 0x0f, 0x01, 0xc3" +#define ASM_VMX_VMPTRLD_RAX ".byte 0x0f, 0xc7, 0x30" +#define ASM_VMX_VMREAD_RDX_RAX ".byte 0x0f, 0x78, 0xd0" +#define ASM_VMX_VMWRITE_RAX_RDX ".byte 0x0f, 0x79, 0xd0" +#define ASM_VMX_VMWRITE_RSP_RDX ".byte 0x0f, 0x79, 0xd4" +#define ASM_VMX_VMXOFF ".byte 0x0f, 0x01, 0xc4" +#define ASM_VMX_VMXON_RAX ".byte 0xf3, 0x0f, 0xc7, 0x30" +#define ASM_VMX_INVEPT ".byte 0x66, 0x0f, 0x38, 0x80, 0x08" +#define ASM_VMX_INVVPID ".byte 0x66, 0x0f, 0x38, 0x81, 0x08" + + +#endif
Signature · 1 line
1.6.0.3
PATCH 20/44 KVM: VMX: move cpu_has_kvm_support() to an inline on asm/virtext.h
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
It will be used by core code on kdump and reboot, to disable vmx if needed.
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/include/asm/virtext.h | 31 +++++++++++++++++++++++++++++++ arch/x86/kvm/vmx.c | 4 ++-- 2 files changed, 33 insertions(+), 2 deletions(-) create mode 100644 arch/x86/include/asm/virtext.h
arch/x86/include/asm/virtext.h new file +31 -0
@@ -0,0 +1,31 @@ +/* CPU virtualization extensions handling +* +* This should carry the code for handling CPU virtualization extensions +* that needs to live in the kernel core. +* +* Author: Eduardo Habkost <[email protected]> +* +* Copyright (C) 2008, Red Hat Inc. +* +* Contains code from KVM, Copyright (C) 2006 Qumranet, Inc. +* +* This work is licensed under the terms of the GNU GPL, version 2. See +* the COPYING file in the top-level directory. +*/ +#ifndef _ASM_X86_VIRTEX_H +#define _ASM_X86_VIRTEX_H + +#include <asm/processor.h> +#include <asm/system.h> + +/* +* VMX functions: +*/ + +static inline int cpu_has_vmx (void ) +{ +unsigned long ecx =cpuid_ecx (1 ); +return test_bit (5 , &ecx);/* CPUID.1:ECX.VMX[bit 5] -> VT */ +} + +#endif /* _ASM_X86_VIRTEX_H */
arch/x86/kvm/vmx.c +2 -2
@@ -31,6 +31,7 @@#include <asm/io.h> #include <asm/desc.h> #include <asm/vmx.h> +#include <asm/virtext.h> #define __ex (x) __kvm_handle_fault_on_reboot(x)
@@ -1044,8 +1045,7 @@ static int vmx_get_irq(struct kvm_vcpu *vcpu)static __init int cpu_has_kvm_support (void ) { -unsigned long ecx =cpuid_ecx (1 ); -return test_bit (5 , &ecx);/* CPUID.1:ECX.VMX[bit 5] -> VT */ +return cpu_has_vmx (); }static __init int vmx_disabled_by_bios (void )
Signature · 1 line
1.6.0.3
PATCH 21/44 x86: asm/virtext.h: add cpu_vmxoff() inline function
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
Unfortunately we can't use exactly the same code from vmx hardware_disable(), because the KVM function uses the __kvm_handle_fault_on_reboot() tricks.
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/include/asm/virtext.h | 15 +++++++++++++++ 1 files changed, 15 insertions(+), 0 deletions(-)
arch/x86/include/asm/virtext.h +15 -0
@@ -18,6 +18,8 @@#include <asm/processor.h> #include <asm/system.h> +#include <asm/vmx.h> +/* * VMX functions: */
@@ -28,4 +30,17 @@ static inline int cpu_has_vmx(void) return test_bit(5, &ecx); /* CPUID.1:ECX.VMX[bit 5] -> VT */ } + +/** Disable VMX on the current CPU +* +* vmxoff causes a undefined-opcode exception if vmxon was not run +* on the CPU previously. Only call this function if you know VMX +* is enabled. +*/ +static inline void cpu_vmxoff (void ) +{ + asm volatile (ASM_VMX_VMXOFF : : : "cc"); +write_cr4 (read_cr4 () & ~X86_CR4_VMXE ); +} +#endif /* _ASM_X86_VIRTEX_H */
Signature · 1 line
1.6.0.3
PATCH 22/44 KVM: VMX: extract kvm_cpu_vmxoff() from hardware_disable()
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
Along with some comments on why it is different from the core cpu_vmxoff() function.
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/vmx.c | 13 +++++++++++-- 1 files changed, 11 insertions(+), 2 deletions(-)
arch/x86/kvm/vmx.c +11 -2
@@ -1091,13 +1091,22 @@ static void vmclear_local_vcpus(void)__vcpu_clear (vmx); } -static void hardware_disable (void *garbage) + +/* Just like cpu_vmxoff(), but with the __kvm_handle_fault_on_reboot() +* tricks. +*/ +static void kvm_cpu_vmxoff (void ) { -vmclear_local_vcpus (); asm volatile (__ex(ASM_VMX_VMXOFF) : : : "cc");write_cr4 (read_cr4 () & ~X86_CR4_VMXE ); } +static void hardware_disable (void *garbage) +{ +vmclear_local_vcpus (); +kvm_cpu_vmxoff (); +} +static __init int adjust_vmx_controls (u32 ctl_min,u32 ctl_opt,u32 msr,u32 *result) {
Signature · 1 line
1.6.0.3
PATCH 23/44 x86: cpu_emergency_vmxoff() function
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
Add cpu_emergency_vmxoff() and its friends: cpu_vmx_enabled() and __cpu_emergency_vmxoff().
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/include/asm/virtext.h | 23 +++++++++++++++++++++++ 1 files changed, 23 insertions(+), 0 deletions(-)
arch/x86/include/asm/virtext.h +23 -0
@@ -43,4 +43,27 @@ static inline void cpu_vmxoff(void) write_cr4(read_cr4() & ~X86_CR4_VMXE); } +static inline int cpu_vmx_enabled (void ) +{ +return read_cr4 () &X86_CR4_VMXE ; +} + +/** Disable VMX if it is enabled on the current CPU +* +* You shouldn't call this if cpu_has_vmx() returns 0. +*/ +static inline void __cpu_emergency_vmxoff (void ) +{ +if (cpu_vmx_enabled ()) +cpu_vmxoff (); +} + +/** Disable VMX if it is supported and enabled on the current CPU +*/ +static inline void cpu_emergency_vmxoff (void ) +{ +if (cpu_has_vmx ()) +__cpu_emergency_vmxoff (); +} +#endif /* _ASM_X86_VIRTEX_H */
Signature · 1 line
1.6.0.3
PATCH 24/44 KVM: SVM: move has_svm() code to asm/virtext.h
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
Use a trick to keep the printk()s on has_svm() working as before. gcc will take care of not generating code for the 'msg' stuff when the function is called with a NULL msg argument.
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/include/asm/virtext.h | 41 ++++++++++++++++++++++++++++++++++++++++ arch/x86/kvm/svm.c | 19 ++++------------- 2 files changed, 46 insertions(+), 14 deletions(-)
arch/x86/include/asm/virtext.h +41 -0
@@ -19,6 +19,7 @@#include <asm/system.h> #include <asm/vmx.h> +#include <asm/svm.h> /* * VMX functions:
@@ -66,4 +67,44 @@ static inline void cpu_emergency_vmxoff(void) __cpu_emergency_vmxoff(); } + + + +/* +* SVM functions: +*/ + +/** Check if the CPU has SVM support +* +* You can use the 'msg' arg to get a message describing the problem, +* if the function returns zero. Simply pass NULL if you are not interested +* on the messages; gcc should take care of not generating code for +* the messages on this case. +*/ +static inline int cpu_has_svm (const char **msg) +{ +uint32_t eax, ebx, ecx, edx; + +if (boot_cpu_data.x86_vendor !=X86_VENDOR_AMD ) { +if (msg) + *msg ="not amd" ; +return 0 ; + } + +cpuid (0x80000000 , &eax, &ebx, &ecx, &edx); +if (eax <SVM_CPUID_FUNC ) { +if (msg) + *msg ="can't execute cpuid_8000000a" ; +return 0 ; + } + +cpuid (0x80000001 , &eax, &ebx, &ecx, &edx); +if (!(ecx & (1 <<SVM_CPUID_FEATURE_SHIFT ))) { +if (msg) + *msg ="svm not available" ; +return 0 ; + } +return 1 ; +} +#endif /* _ASM_X86_VIRTEX_H */
arch/x86/kvm/svm.c +5 -14
@@ -28,6 +28,8 @@#include <asm/desc.h> +#include <asm/virtext.h> +#define __ex (x) __kvm_handle_fault_on_reboot(x)MODULE_AUTHOR ("Qumranet" );
@@ -245,24 +247,13 @@ static void skip_emulated_instruction(struct kvm_vcpu *vcpu)static int has_svm (void ) { -uint32_t eax, ebx, ecx, edx; - -if (boot_cpu_data.x86_vendor !=X86_VENDOR_AMD ) { -printk (KERN_INFO "has_svm: not amd\n" ); -return 0 ; - } +const char *msg; -cpuid (0x80000000 , &eax, &ebx, &ecx, &edx); -if (eax <SVM_CPUID_FUNC ) { -printk (KERN_INFO "has_svm: can't execute cpuid_8000000a\n" ); +if (!cpu_has_svm (&msg)) { +printk (KERN_INFO "has_svn: %s\n" , msg);return 0 ; } -cpuid (0x80000001 , &eax, &ebx, &ecx, &edx); -if (!(ecx & (1 <<SVM_CPUID_FEATURE_SHIFT ))) { -printk (KERN_DEBUG "has_svm: svm not available\n" ); -return 0 ; - }return 1 ; }
Signature · 1 line
1.6.0.3
PATCH 25/44 KVM: SVM: move svm_hardware_disable() code to asm/virtext.h
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
Create cpu_svm_disable() function.
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/include/asm/virtext.h | 14 ++++++++++++++ arch/x86/kvm/svm.c | 6 +----- 2 files changed, 15 insertions(+), 5 deletions(-)
arch/x86/include/asm/virtext.h +14 -0
@@ -107,4 +107,18 @@ static inline int cpu_has_svm(const char **msg) return 1; } + +/** Disable SVM on the current CPU +* +* You should call this only if cpu_has_svm() returned true. +*/ +static inline void cpu_svm_disable (void ) +{ +uint64_t efer; + +wrmsrl (MSR_VM_HSAVE_PA ,0 ); +rdmsrl (MSR_EFER , efer); +wrmsrl (MSR_EFER , efer & ~MSR_EFER_SVME_MASK ); +} +#endif /* _ASM_X86_VIRTEX_H */
arch/x86/kvm/svm.c +1 -5
@@ -259,11 +259,7 @@ static int has_svm(void)static void svm_hardware_disable (void *garbage) { -uint64_t efer; - -wrmsrl (MSR_VM_HSAVE_PA ,0 ); -rdmsrl (MSR_EFER , efer); -wrmsrl (MSR_EFER , efer & ~MSR_EFER_SVME_MASK ); +cpu_svm_disable (); }static void svm_hardware_enable (void *garbage)
Signature · 1 line
1.6.0.3
PATCH 26/44 x86: cpu_emergency_svm_disable() function
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
This function can be used by the reboot or kdump code to forcibly disable SVM on the CPU.
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/include/asm/virtext.h | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-)
arch/x86/include/asm/virtext.h +8 -0
@@ -121,4 +121,12 @@ static inline void cpu_svm_disable(void) wrmsrl(MSR_EFER, efer & ~MSR_EFER_SVME_MASK); } +/** Makes sure SVM is disabled, if it is supported on the CPU +*/ +static inline void cpu_emergency_svm_disable (void ) +{ +if (cpu_has_svm (NULL )) +cpu_svm_disable (); +} +#endif /* _ASM_X86_VIRTEX_H */
Signature · 1 line
1.6.0.3
PATCH 27/44 kdump: forcibly disable VMX and SVM on machine_crash_shutdown()
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
We need to disable virtualization extensions on all CPUs before booting the kdump kernel, otherwise the kdump kernel booting will fail, and rebooting after the kdump kernel did its task may also fail.
We do it using cpu_emergency_vmxoff() and cpu_emergency_svm_disable(), that should always work, because those functions check if the CPUs support SVM or VMX before doing their tasks.
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kernel/crash.c | 18 ++++++++++++++++++ 1 files changed, 18 insertions(+), 0 deletions(-)
arch/x86/kernel/crash.c +18 -0
@@ -26,6 +26,7 @@#include <linux/kdebug.h> #include <asm/smp.h> #include <asm/reboot.h> +#include <asm/virtext.h> #include <mach_ipi.h>
@@ -49,6 +50,15 @@ static void kdump_nmi_callback(int cpu, struct die_args *args)#endif crash_save_cpu (regs, cpu); +/* Disable VMX or SVM if needed. +* +* We need to disable virtualization on all CPUs. +* Having VMX or SVM enabled on any CPU may break rebooting +* after the kdump kernel has finished its task. +*/ +cpu_emergency_vmxoff (); +cpu_emergency_svm_disable (); +disable_local_APIC (); }
@@ -80,6 +90,14 @@ void native_machine_crash_shutdown(struct pt_regs *regs)local_irq_disable ();kdump_nmi_shootdown_cpus (); + +/* Booting kdump kernel with VMX or SVM enabled won't work, +* because (among other limitations) we can't disable paging +* with the virt flags. +*/ +cpu_emergency_vmxoff (); +cpu_emergency_svm_disable (); +lapic_shutdown ();#if defined(CONFIG_X86_IO_APIC )disable_IO_APIC ();
Signature · 1 line
1.6.0.3
PATCH 28/44 x86: disable VMX on all CPUs on reboot
In reply to Avi KivityFrom: Eduardo Habkost <[email protected]>
On emergency_restart, we may need to use an NMI to disable virtualization on all CPUs. We do that using nmi_shootdown_cpus() if VMX is enabled.
Note: With this patch, we will run the NMI stuff only when the CPU where emergency_restart() was called has VMX enabled. This should work on most cases because KVM enables VMX on all CPUs, but we may miss the small window where KVM is doing that. Also, I don't know if all code using VMX out there always enable VMX on all CPUs like KVM does. We have two other alternatives for that:
a) Have an API that all code that enables VMX on any CPU should use to tell the kernel core that it is going to enable VMX on the CPUs.
b) Always call nmi_shootdown_cpus() if the CPU supports VMX. This is a bit intrusive and more risky, as it would run nmi_shootdown_cpus() on emergency_reboot() even on systems where virtualization is never enabled.
Finding a proper point to hook the nmi_shootdown_cpus() call isn't trivial, as the non-emergency machine_restart() (that doesn't need the NMI tricks) uses machine_emergency_restart() directly.
The solution to make this work without adding a new function or argument to machine_ops was setting a 'reboot_emergency' flag that tells if native_machine_emergency_restart() needs to do the virt cleanup or not.
Signed-off-by Eduardo Habkost <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kernel/reboot.c | 62 ++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 60 insertions(+), 2 deletions(-)
arch/x86/kernel/reboot.c +60 -2
@@ -12,6 +12,7 @@#include <asm/proto.h> #include <asm/reboot_fixups.h> #include <asm/reboot.h> +#include <asm/virtext.h> #ifdef CONFIG_X86_32 # include <linux/dmi.h>
@@ -39,6 +40,12 @@ int reboot_force;static int reboot_cpu =-1 ;#endif +/* This is set if we need to go through the 'emergency' path. +* When machine_emergency_restart() is called, we may be on +* an inconsistent state and won't be able to do a clean cleanup +*/ +static int reboot_emergency; +/* reboot=b[ios] | s[mp] | t[riple] | k[bd] | e[fi] [, [w]arm | [c]old] warm Don't set the cold reboot flag cold Set the cold reboot flag
@@ -363,6 +370,48 @@ static inline void kb_wait(void) } } +static void vmxoff_nmi (int cpu,struct die_args *args) +{ +cpu_emergency_vmxoff (); +} + +/* Use NMIs as IPIs to tell all CPUs to disable virtualization +*/ +static void emergency_vmx_disable_all (void ) +{ +/* Just make sure we won't change CPUs while doing this */ +local_irq_disable (); + +/* We need to disable VMX on all CPUs before rebooting, otherwise +* we risk hanging up the machine, because the CPU ignore INIT +* signals when VMX is enabled. +* +* We can't take any locks and we may be on an inconsistent +* state, so we use NMIs as IPIs to tell the other CPUs to disable +* VMX and halt. +* +* For safety, we will avoid running the nmi_shootdown_cpus() +* stuff unnecessarily, but we don't have a way to check +* if other CPUs have VMX enabled. So we will call it only if the +* CPU we are running on has VMX enabled. +* +* We will miss cases where VMX is not enabled on all CPUs. This +* shouldn't do much harm because KVM always enable VMX on all +* CPUs anyway. But we can miss it on the small window where KVM +* is still enabling VMX. +*/ +if (cpu_has_vmx () &&cpu_vmx_enabled ()) { +/* Disable VMX on this CPU. +*/ +cpu_vmxoff (); + +/* Halt and disable VMX on the other CPUs */ +nmi_shootdown_cpus (vmxoff_nmi); + + } +} + +void __attribute__((weak))mach_reboot_fixups (void ) { }
@@ -371,6 +420,9 @@ static void native_machine_emergency_restart(void) {int i; +if (reboot_emergency) +emergency_vmx_disable_all (); +/* Tell the BIOS if we want cold or warm reboot */ *((unsigned short *)__va (0x472 )) = reboot_mode;
@@ -462,13 +514,19 @@ void native_machine_shutdown(void)#endif } +static void __machine_emergency_restart (int emergency) +{ + reboot_emergency = emergency; + machine_ops.emergency_restart (); +} +static void native_machine_restart (char *__unused) {printk ("machine restart\n" );if (!reboot_force)machine_shutdown (); -machine_emergency_restart (); +__machine_emergency_restart (0 ); }static void native_machine_halt (void )
@@ -507,7 +565,7 @@ void machine_shutdown(void)void machine_emergency_restart (void ) { - machine_ops.emergency_restart (); +__machine_emergency_restart (1 ); }void machine_restart (char *cmd)
Signature · 1 line
1.6.0.3
PATCH 29/44 KVM: ia64: Define printk function for kvm-intel module
In reply to Avi KivityFrom: Xiantao Zhang <[email protected]>
kvm-intel module is relocated to an isolated address space with kernel, so it can't call host kernel's printk for debug purpose. In the module, we implement the printk to output debug info of vmm.
Signed-off-by Xiantao Zhang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/ia64/include/asm/kvm_host.h | 4 ++++ arch/ia64/kvm/Makefile | 2 +- arch/ia64/kvm/kvm-ia64.c | 8 ++++++++ arch/ia64/kvm/kvm_lib.c | 15 +++++++++++++++ arch/ia64/kvm/vmm.c | 26 ++++++++++++++++++++++++++ 5 files changed, 54 insertions(+), 1 deletions(-) create mode 100644 arch/ia64/kvm/kvm_lib.c
arch/ia64/include/asm/kvm_host.h +4 -0
@@ -39,6 +39,7 @@#define EXIT_REASON_EXTERNAL_INTERRUPT 6#define EXIT_REASON_IPI 7#define EXIT_REASON_PTC_G 8 +#define EXIT_REASON_DEBUG 20/*Define vmm address space and vm data space.*/ #define KVM_VMM_SIZE (__IA64_UL_CONST(16)<<20)
@@ -126,6 +127,8 @@ KVM_MEM_DIRTY_LOG_SIZE) / sizeof(struct kvm_vcpu_data)#define KVM_MAX_MEM_SIZE (KVM_P2M_SIZE >> 3 << PAGE_SHIFT) +#define VMM_LOG_LEN 256 +#include <linux/types.h> #include <linux/mm.h> #include <linux/kvm.h>
@@ -437,6 +440,7 @@ struct kvm_vcpu_arch {unsigned long opcode;unsigned long cause; +char log_buf[VMM_LOG_LEN ];union context host;union context guest; };
arch/ia64/kvm/Makefile +1 -1
@@ -60,7 +60,7 @@ obj-$(CONFIG_KVM) += kvm.o CFLAGS_vcpu.o += -mfixed-range=f2-f5,f12-f127 kvm-intel-objs = vmm.o vmm_ivt.o trampoline.o vcpu.o optvfault.o mmio.o \ - vtlb.o process.o + vtlb.o process.o kvm_lib.o #Add link memcpy and memset to avoid possible structure assignment error kvm-intel-objs += memcpy.o memset.o obj-$(CONFIG_KVM_INTEL) += kvm-intel.o
arch/ia64/kvm/kvm-ia64.c +8 -0
@@ -474,6 +474,13 @@ static int handle_external_interrupt(struct kvm_vcpu *vcpu,return 1 ; } +static int handle_vcpu_debug (struct kvm_vcpu *vcpu, +struct kvm_run *kvm_run) +{ +printk ("VMM: %s" , vcpu->arch.log_buf); +return 1 ; +} +static int (*kvm_vti_exit_handlers[])(struct kvm_vcpu *vcpu,struct kvm_run *kvm_run) = { [EXIT_REASON_VM_PANIC ] = handle_vm_error,
@@ -485,6 +492,7 @@ static int (*kvm_vti_exit_handlers[])(struct kvm_vcpu *vcpu, [EXIT_REASON_EXTERNAL_INTERRUPT] = handle_external_interrupt, [EXIT_REASON_IPI] = handle_ipi, [EXIT_REASON_PTC_G] = handle_global_purge, + [EXIT_REASON_DEBUG] = handle_vcpu_debug, };
arch/ia64/kvm/kvm_lib.c new file +15 -0
@@ -0,0 +1,15 @@ +/* +* kvm_lib.c: Compile some libraries for kvm-intel module. +* +* Just include kernel's library, and disable symbols export. +* Copyright (C) 2008, Intel Corporation. +* Xiantao Zhang ([email protected]) +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License version 2 as +* published by the Free Software Foundation. +* +*/ +#undef CONFIG_MODULES +#include "../../../lib/vsprintf.c" +#include "../../../lib/ctype.c"
arch/ia64/kvm/vmm.c +26 -0
@@ -62,5 +62,31 @@ void vmm_spin_unlock(spinlock_t *lock) {_vmm_raw_spin_unlock (lock); } + +static void vcpu_debug_exit (struct kvm_vcpu *vcpu) +{ +struct exit_ctl_data *p = &vcpu->arch.exit_data; +long psr; + +local_irq_save (psr); + p->exit_reason =EXIT_REASON_DEBUG ; +vmm_transition (vcpu); +local_irq_restore (psr); +} + +asmlinkage int printk(const char *fmt, ...) +{ +struct kvm_vcpu *vcpu = current_vcpu; +va_list args; +int r; + +memset (vcpu->arch.log_buf,0 ,VMM_LOG_LEN ); +va_start (args, fmt); + r =vsnprintf (vcpu->arch.log_buf,VMM_LOG_LEN , fmt, args); +va_end (args); +vcpu_debug_exit (vcpu); +return r; +} +module_init (kvm_vmm_init)module_exit (kvm_vmm_exit)
Signature · 1 line
1.6.0.3
PATCH 30/44 KVM: ia64: Add some debug points to provide crash infomation
In reply to Avi KivityFrom: Xiantao Zhang <[email protected]>
Use printk infrastructure to print out some debug info once VM crashes.
Signed-off-by Xiantao Zhang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/ia64/kvm/mmio.c | 38 ++++++++++--------------- arch/ia64/kvm/process.c | 9 +++-- arch/ia64/kvm/vcpu.c | 71 +++++++++++++++++++++++++++++++++++++++++++--- arch/ia64/kvm/vcpu.h | 2 +- arch/ia64/kvm/vmm.c | 1 + 5 files changed, 88 insertions(+), 33 deletions(-)
arch/ia64/kvm/mmio.c +15 -23
@@ -66,31 +66,25 @@ void lsapic_write(struct kvm_vcpu *v, unsigned long addr,switch (addr) {case PIB_OFST_INTA : -/*panic_domain(NULL, "Undefined write on PIB INTA\n");*/ -panic_vm (v); +panic_vm (v,"Undefined write on PIB INTA\n" );break ;case PIB_OFST_XTP :if (length ==1 ) {vlsapic_write_xtp (v, val); }else { -/*panic_domain(NULL, -"Undefined write on PIB XTP\n");*/ -panic_vm (v); +panic_vm (v,"Undefined write on PIB XTP\n" ); }break ;default :if (PIB_LOW_HALF (addr)) { -/*lower half */ +/*Lower half */ if (length !=8 ) -/*panic_domain(NULL, -"Can't LHF write with size %ld!\n", -length);*/ -panic_vm (v); +panic_vm (v,"Can't LHF write with size %ld!\n" , + length);else vlsapic_write_ipi (v, addr, val); - }else {/* upper half -printk("IPI-UHF write %lx\n",addr);*/ -panic_vm (v); + }else {/*Upper half */ +panic_vm (v,"IPI-UHF write %lx\n" , addr); }break ; }
@@ -108,22 +102,18 @@ unsigned long lsapic_read(struct kvm_vcpu *v, unsigned long addr,if (length ==1 )/* 1 byte load */ ;/* There is no i8259, there is no INTA access*/ else -/*panic_domain(NULL,"Undefined read on PIB INTA\n"); */ -panic_vm (v); +panic_vm (v,"Undefined read on PIB INTA\n" );break ;case PIB_OFST_XTP :if (length ==1 ) { result =VLSAPIC_XTP (v); -/* printk("read xtp %lx\n", result); */ }else { -/*panic_domain(NULL, -"Undefined read on PIB XTP\n");*/ -panic_vm (v); +panic_vm (v,"Undefined read on PIB XTP\n" ); }break ;default : -panic_vm (v); +panic_vm (v,"Undefined addr access for lsapic!\n" );break ; }return result;
@@ -162,7 +152,7 @@ static void mmio_access(struct kvm_vcpu *vcpu, u64 src_pa, u64 *dest,/* it's necessary to ensure zero extending */ *dest = p->u.ioreq.data & (~0UL >> (64 -(s*8 ))); }else -panic_vm (vcpu); +panic_vm (vcpu,"Unhandled mmio access returned!\n" ); out:local_irq_restore (psr);return ;
@@ -324,7 +314,9 @@ void emulate_io_inst(struct kvm_vcpu *vcpu, u64 padr, u64 ma)return ; }else { inst_type =-1 ; -panic_vm (vcpu); +panic_vm (vcpu,"Unsupported MMIO access instruction! \ +Bunld[0]=0x%lx, Bundle[1]=0x%lx\n" , + bundle.i64[0 ], bundle.i64[1 ]); } size =1 << size;
@@ -335,7 +327,7 @@ void emulate_io_inst(struct kvm_vcpu *vcpu, u64 padr, u64 ma)if (inst_type ==SL_INTEGER )vcpu_set_gr (vcpu, inst.M1.r1, data,0 );else -panic_vm (vcpu); +panic_vm (vcpu,"Unsupported instruction type!\n" ); }vcpu_increment_iip (vcpu);
arch/ia64/kvm/process.c +5 -4
@@ -527,7 +527,8 @@ void reflect_interruption(u64 ifa, u64 isr, u64 iim, vector = vec2off[vec];if (!(vpsr &IA64_PSR_IC ) && (vector !=IA64_DATA_NESTED_TLB_VECTOR )) { -panic_vm (vcpu); +panic_vm (vcpu,"Interruption with vector :0x%lx occurs " +"with psr.ic = 0\n" , vector);return ; }
@@ -586,7 +587,7 @@ static void set_pal_call_result(struct kvm_vcpu *vcpu)vcpu_set_gr (vcpu,10 , p->u.pal_data.ret.v1,0 );vcpu_set_gr (vcpu,11 , p->u.pal_data.ret.v2,0 ); }else -panic_vm (vcpu); +panic_vm (vcpu,"Mis-set for exit reason!\n" ); }static void set_sal_call_data (struct kvm_vcpu *vcpu)
@@ -614,7 +615,7 @@ static void set_sal_call_result(struct kvm_vcpu *vcpu)vcpu_set_gr (vcpu,10 , p->u.sal_data.ret.r10,0 );vcpu_set_gr (vcpu,11 , p->u.sal_data.ret.r11,0 ); }else -panic_vm (vcpu); +panic_vm (vcpu,"Mis-set for exit reason!\n" ); } void kvm_ia64_handle_break(unsigned long ifa, struct kvm_pt_regs *regs,
@@ -680,7 +681,7 @@ static void generate_exirq(struct kvm_vcpu *vcpu) vpsr =VCPU (vcpu, vpsr); isr = vpsr &IA64_PSR_RI ;if (!(vpsr &IA64_PSR_IC )) -panic_vm (vcpu); +panic_vm (vcpu,"Trying to inject one IRQ with psr.ic=0\n" );reflect_interruption (0 , isr,0 ,12 , regs);/* EXT IRQ */ }
arch/ia64/kvm/vcpu.c +66 -5
90 lines not on this page · raw message
arch/ia64/kvm/vcpu.h +1 -1
@@ -737,7 +737,7 @@ void kvm_init_vtlb(struct kvm_vcpu *v);void kvm_init_vhpt (struct kvm_vcpu *v);void thash_init (struct thash_cb *hcb,u64 sz); -void panic_vm (struct kvm_vcpu *v); +void panic_vm (struct kvm_vcpu *v,const char *fmt, ...);extern u64 ia64_call_vsa (u64 proc,u64 arg1,u64 arg2,u64 arg3,u64 arg4,u64 arg5,u64 arg6,u64 arg7);
arch/ia64/kvm/vmm.c +1 -0
@@ -20,6 +20,7 @@*/ +#include <linux/kernel.h> #include <linux/module.h> #include <asm/fpswa.h>
Signature · 1 line
1.6.0.3
PATCH 31/44 KVM: ia64: Add handler for crashed vmm
In reply to Avi KivityFrom: Xiantao Zhang <[email protected]>
Since vmm runs in an isolated address space and it is just a copy of host's kvm-intel module, so once vmm crashes, we just crash all guests running on it instead of crashing whole kernel.
Signed-off-by Xiantao Zhang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/ia64/kvm/process.c | 20 ++++++++++++++++++++ arch/ia64/kvm/vcpu.h | 3 +++ arch/ia64/kvm/vmm.c | 2 ++ arch/ia64/kvm/vmm_ivt.S | 31 +++++++++++++++++++------------ 4 files changed, 44 insertions(+), 12 deletions(-)
arch/ia64/kvm/process.c +20 -0
@@ -942,8 +942,20 @@ static void vcpu_do_resume(struct kvm_vcpu *vcpu)ia64_set_pta (vcpu->arch.vhpt.pta.val); } +static void vmm_sanity_check (struct kvm_vcpu *vcpu) +{ +struct exit_ctl_data *p = &vcpu->arch.exit_data; + +if (!vmm_sanity && p->exit_reason !=EXIT_REASON_DEBUG ) { +panic_vm (vcpu,"Failed to do vmm sanity check," +"it maybe caused by crashed vmm!!\n\n" ); + } +} +static void kvm_do_resume_op (struct kvm_vcpu *vcpu) { +vmm_sanity_check (vcpu);/*Guarantee vcpu runing on healthy vmm!*/ +if (test_and_clear_bit (KVM_REQ_RESUME , &vcpu->requests)) {vcpu_do_resume (vcpu);return ;
@@ -969,3 +981,11 @@ void vmm_transition(struct kvm_vcpu *vcpu) 1, 0, 0, 0, 0, 0);kvm_do_resume_op (vcpu); } + +void vmm_panic_handler (u64 vec) +{ +struct kvm_vcpu *vcpu = current_vcpu; + vmm_sanity =0 ; +panic_vm (vcpu,"Unexpected interruption occurs in VMM, vector:0x%lx\n" , + vec2off[vec]); +}
arch/ia64/kvm/vcpu.h +3 -0
@@ -741,5 +741,8 @@ void panic_vm(struct kvm_vcpu *v, const char *fmt, ...);extern u64 ia64_call_vsa (u64 proc,u64 arg1,u64 arg2,u64 arg3,u64 arg4,u64 arg5,u64 arg6,u64 arg7); + +extern long vmm_sanity; +#endif #endif /* __VCPU_H__ */
arch/ia64/kvm/vmm.c +2 -0
@@ -32,6 +32,8 @@ MODULE_LICENSE("GPL");extern char kvm_ia64_ivt;extern fpswa_interface_t *vmm_fpswa_interface; +long vmm_sanity =1 ; +struct kvm_vmm_info vmm_info = { .module =THIS_MODULE , .vmm_entry = vmm_entry,
arch/ia64/kvm/vmm_ivt.S +19 -12
@@ -70,14 +70,12 @@ # define PSR_DEFAULT_BITS 0 #endif - #define KVM_FAULT(n) \ kvm_fault_##n:; \ mov r19=n;; \ - br.sptk.many kvm_fault_##n; \ + br.sptk.many kvm_vmm_panic; \ ;; \ - #define KVM_REFLECT(n) \ mov r31=pr; \ mov r19=n; /* prepare to save predicates */ \
@@ -85,17 +83,26 @@ ;; \ tbit.z p6,p7=r29,IA64_PSR_VM_BIT; \ (p7)br.sptk.many kvm_dispatch_reflection; \ - br.sptk.many kvm_panic; \ - + br.sptk.many kvm_vmm_panic; \ -GLOBAL_ENTRY(kvm_panic) - br.sptk.many kvm_panic +GLOBAL_ENTRY(kvm_vmm_panic) + KVM_SAVE_MIN_WITH_COVER_R19 + alloc r14=ar.pfs,0,0,1,0 + mov out0=r15 + adds r3=8,r2 // set up second base pointer ;; -END(kvm_panic) - - - - + ssm psr.ic + ;; + srlz.i // guarantee that interruption collection is on + ;; + //(p15) ssm psr.i // restore psr.i + addl r14=@gprel(ia64_leave_hypervisor),gp + ;; + KVM_SAVE_REST + mov rp=r14 + ;; + br.call.sptk.many b6=vmm_panic_handler; +END(kvm_vmm_panic) .section .text.ivt,"ax"
Signature · 1 line
1.6.0.3
PATCH 32/44 KVM: ia64: Clean up vmm_ivt.S using tab to indent every line
In reply to Avi KivityFrom: Xiantao Zhang <[email protected]>
Using tab for indentation for vmm_ivt.S.
Signed-off-by Xiantao Zhang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/ia64/kvm/vmm_ivt.S | 1470 +++++++++++++++++++++++------------------------ 1 files changed, 729 insertions(+), 741 deletions(-)
arch/ia64/kvm/vmm_ivt.S +729 -741
@@ -1,5 +1,5 @@ /* - * /ia64/kvm_ivt.S + * arch/ia64/kvm/vmm_ivt.S * * Copyright (C) 1998-2001, 2003 Hewlett-Packard Co * Stephane Eranian <[email protected]>
@@ -71,37 +71,37 @@ #endif #define KVM_FAULT(n) \ - kvm_fault_##n:; \ - mov r19=n;; \ - br.sptk.many kvm_vmm_panic; \ - ;; \ + kvm_fault_##n:; \ + mov r19=n;; \ + br.sptk.many kvm_vmm_panic; \ + ;; \ #define KVM_REFLECT(n) \ - mov r31=pr; \ - mov r19=n; /* prepare to save predicates */ \ - mov r29=cr.ipsr; \ - ;; \ - tbit.z p6,p7=r29,IA64_PSR_VM_BIT; \ -(p7)br.sptk.many kvm_dispatch_reflection; \ - br.sptk.many kvm_vmm_panic; \ + mov r31=pr; \ + mov r19=n; /* prepare to save predicates */ \ + mov r29=cr.ipsr; \ + ;; \ + tbit.z p6,p7=r29,IA64_PSR_VM_BIT; \ +(p7) br.sptk.many kvm_dispatch_reflection; \ + br.sptk.many kvm_vmm_panic; \ GLOBAL_ENTRY(kvm_vmm_panic) - KVM_SAVE_MIN_WITH_COVER_R19 - alloc r14=ar.pfs,0,0,1,0 - mov out0=r15 - adds r3=8,r2 // set up second base pointer - ;; - ssm psr.ic - ;; - srlz.i // guarantee that interruption collection is on - ;; - //(p15) ssm psr.i // restore psr.i - addl r14=@gprel(ia64_leave_hypervisor),gp - ;; - KVM_SAVE_REST - mov rp=r14 - ;; - br.call.sptk.many b6=vmm_panic_handler; + KVM_SAVE_MIN_WITH_COVER_R19 + alloc r14=ar.pfs,0,0,1,0 + mov out0=r15 + adds r3=8,r2 // set up second base pointer + ;; + ssm psr.ic + ;; + srlz.i // guarantee that interruption collection is on + ;; + //(p15) ssm psr.i // restore psr.i + addl r14=@gprel(ia64_leave_hypervisor),gp + ;; + KVM_SAVE_REST + mov rp=r14 + ;; + br.call.sptk.many b6=vmm_panic_handler; END(kvm_vmm_panic) .section .text.ivt,"ax"
@@ -112,308 +112,307 @@ kvm_ia64_ivt: /////////////////////////////////////////////////////////////// // 0x0000 Entry 0 (size 64 bundles) VHPT Translation (8,20,47) ENTRY(kvm_vhpt_miss) - KVM_FAULT(0) + KVM_FAULT(0) END(kvm_vhpt_miss) - .org kvm_ia64_ivt+0x400 //////////////////////////////////////////////////////////////// // 0x0400 Entry 1 (size 64 bundles) ITLB (21) ENTRY(kvm_itlb_miss) - mov r31 = pr - mov r29=cr.ipsr; - ;; - tbit.z p6,p7=r29,IA64_PSR_VM_BIT; - (p6) br.sptk kvm_alt_itlb_miss - mov r19 = 1 - br.sptk kvm_itlb_miss_dispatch - KVM_FAULT(1); + mov r31 = pr + mov r29=cr.ipsr; + ;; + tbit.z p6,p7=r29,IA64_PSR_VM_BIT; +(p6) br.sptk kvm_alt_itlb_miss + mov r19 = 1 + br.sptk kvm_itlb_miss_dispatch + KVM_FAULT(1); END(kvm_itlb_miss) .org kvm_ia64_ivt+0x0800 ////////////////////////////////////////////////////////////////// // 0x0800 Entry 2 (size 64 bundles) DTLB (9,48) ENTRY(kvm_dtlb_miss) - mov r31 = pr - mov r29=cr.ipsr; - ;; - tbit.z p6,p7=r29,IA64_PSR_VM_BIT; -(p6)br.sptk kvm_alt_dtlb_miss - br.sptk kvm_dtlb_miss_dispatch + mov r31 = pr + mov r29=cr.ipsr; + ;; + tbit.z p6,p7=r29,IA64_PSR_VM_BIT; +(p6) br.sptk kvm_alt_dtlb_miss + br.sptk kvm_dtlb_miss_dispatch END(kvm_dtlb_miss) .org kvm_ia64_ivt+0x0c00 //////////////////////////////////////////////////////////////////// // 0x0c00 Entry 3 (size 64 bundles) Alt ITLB (19) ENTRY(kvm_alt_itlb_miss) - mov r16=cr.ifa // get address that caused the TLB miss - ;; - movl r17=PAGE_KERNEL - mov r24=cr.ipsr - movl r19=(((1 << IA64_MAX_PHYS_BITS) - 1) & ~0xfff) - ;; - and r19=r19,r16 // clear ed, reserved bits, and PTE control bits - ;; - or r19=r17,r19 // insert PTE control bits into r19 - ;; - movl r20=IA64_GRANULE_SHIFT<<2 - ;; - mov cr.itir=r20 - ;; - itc.i r19 // insert the TLB entry - mov pr=r31,-1 - rfi + mov r16=cr.ifa // get address that caused the TLB miss + ;; + movl r17=PAGE_KERNEL + mov r24=cr.ipsr + movl r19=(((1 << IA64_MAX_PHYS_BITS) - 1) & ~0xfff) + ;; + and r19=r19,r16 // clear ed, reserved bits, and PTE control bits + ;; + or r19=r17,r19 // insert PTE control bits into r19 + ;; + movl r20=IA64_GRANULE_SHIFT<<2 + ;; + mov cr.itir=r20 + ;; + itc.i r19 // insert the TLB entry + mov pr=r31,-1 + rfi END(kvm_alt_itlb_miss) .org kvm_ia64_ivt+0x1000 ///////////////////////////////////////////////////////////////////// // 0x1000 Entry 4 (size 64 bundles) Alt DTLB (7,46) ENTRY(kvm_alt_dtlb_miss) - mov r16=cr.ifa // get address that caused the TLB miss - ;; - movl r17=PAGE_KERNEL - movl r19=(((1 << IA64_MAX_PHYS_BITS) - 1) & ~0xfff) - mov r24=cr.ipsr - ;; - and r19=r19,r16 // clear ed, reserved bits, and PTE control bits - ;; - or r19=r19,r17 // insert PTE control bits into r19 - ;; - movl r20=IA64_GRANULE_SHIFT<<2 - ;; - mov cr.itir=r20 - ;; - itc.d r19 // insert the TLB entry - mov pr=r31,-1 - rfi + mov r16=cr.ifa // get address that caused the TLB miss + ;; + movl r17=PAGE_KERNEL + movl r19=(((1 << IA64_MAX_PHYS_BITS) - 1) & ~0xfff) + mov r24=cr.ipsr + ;; + and r19=r19,r16 // clear ed, reserved bits, and PTE control bits + ;; + or r19=r19,r17 // insert PTE control bits into r19 + ;; + movl r20=IA64_GRANULE_SHIFT<<2 + ;; + mov cr.itir=r20 + ;; + itc.d r19 // insert the TLB entry + mov pr=r31,-1 + rfi END(kvm_alt_dtlb_miss) .org kvm_ia64_ivt+0x1400 ////////////////////////////////////////////////////////////////////// // 0x1400 Entry 5 (size 64 bundles) Data nested TLB (6,45) ENTRY(kvm_nested_dtlb_miss) - KVM_FAULT(5) + KVM_FAULT(5) END(kvm_nested_dtlb_miss) .org kvm_ia64_ivt+0x1800 ///////////////////////////////////////////////////////////////////// // 0x1800 Entry 6 (size 64 bundles) Instruction Key Miss (24) ENTRY(kvm_ikey_miss) - KVM_REFLECT(6) + KVM_REFLECT(6) END(kvm_ikey_miss) .org kvm_ia64_ivt+0x1c00 ///////////////////////////////////////////////////////////////////// // 0x1c00 Entry 7 (size 64 bundles) Data Key Miss (12,51) ENTRY(kvm_dkey_miss) - KVM_REFLECT(7) + KVM_REFLECT(7) END(kvm_dkey_miss) .org kvm_ia64_ivt+0x2000 //////////////////////////////////////////////////////////////////// // 0x2000 Entry 8 (size 64 bundles) Dirty-bit (54) ENTRY(kvm_dirty_bit) - KVM_REFLECT(8) + KVM_REFLECT(8) END(kvm_dirty_bit) .org kvm_ia64_ivt+0x2400 //////////////////////////////////////////////////////////////////// // 0x2400 Entry 9 (size 64 bundles) Instruction Access-bit (27) ENTRY(kvm_iaccess_bit) - KVM_REFLECT(9) + KVM_REFLECT(9) END(kvm_iaccess_bit) .org kvm_ia64_ivt+0x2800 /////////////////////////////////////////////////////////////////// // 0x2800 Entry 10 (size 64 bundles) Data Access-bit (15,55) ENTRY(kvm_daccess_bit) - KVM_REFLECT(10) + KVM_REFLECT(10) END(kvm_daccess_bit) .org kvm_ia64_ivt+0x2c00 ///////////////////////////////////////////////////////////////// // 0x2c00 Entry 11 (size 64 bundles) Break instruction (33) ENTRY(kvm_break_fault) - mov r31=pr - mov r19=11 - mov r29=cr.ipsr - ;; - KVM_SAVE_MIN_WITH_COVER_R19 - ;; - alloc r14=ar.pfs,0,0,4,0 // now it's safe (must be first in insn group!) - mov out0=cr.ifa - mov out2=cr.isr // FIXME: pity to make this slow access twice - mov out3=cr.iim // FIXME: pity to make this slow access twice - adds r3=8,r2 // set up second base pointer - ;; - ssm psr.ic - ;; - srlz.i // guarantee that interruption collection is on - ;; - //(p15)ssm psr.i // restore psr.i - addl r14=@gprel(ia64_leave_hypervisor),gp - ;; - KVM_SAVE_REST - mov rp=r14 - ;; - adds out1=16,sp - br.call.sptk.many b6=kvm_ia64_handle_break - ;; + mov r31=pr + mov r19=11 + mov r29=cr.ipsr + ;; + KVM_SAVE_MIN_WITH_COVER_R19 + ;; + alloc r14=ar.pfs,0,0,4,0 //(must be first in insn group!) + mov out0=cr.ifa + mov out2=cr.isr // FIXME: pity to make this slow access twice + mov out3=cr.iim // FIXME: pity to make this slow access twice + adds r3=8,r2 // set up second base pointer + ;; + ssm psr.ic + ;; + srlz.i // guarantee that interruption collection is on + ;; + //(p15)ssm psr.i // restore psr.i + addl r14=@gprel(ia64_leave_hypervisor),gp + ;; + KVM_SAVE_REST + mov rp=r14 + ;; + adds out1=16,sp + br.call.sptk.many b6=kvm_ia64_handle_break + ;; END(kvm_break_fault) .org kvm_ia64_ivt+0x3000 ///////////////////////////////////////////////////////////////// // 0x3000 Entry 12 (size 64 bundles) External Interrupt (4) ENTRY(kvm_interrupt) - mov r31=pr // prepare to save predicates - mov r19=12 - mov r29=cr.ipsr - ;; - tbit.z p6,p7=r29,IA64_PSR_VM_BIT - tbit.z p0,p15=r29,IA64_PSR_I_BIT - ;; -(p7) br.sptk kvm_dispatch_interrupt - ;; - mov r27=ar.rsc /* M */ - mov r20=r1 /* A */ - mov r25=ar.unat /* M */ - mov r26=ar.pfs /* I */ - mov r28=cr.iip /* M */ - cover /* B (or nothing) */ - ;; - mov r1=sp - ;; - invala /* M */ - mov r30=cr.ifs - ;; - addl r1=-VMM_PT_REGS_SIZE,r1 - ;; - adds r17=2*L1_CACHE_BYTES,r1 /* really: biggest cache-line size */ - adds r16=PT(CR_IPSR),r1 - ;; - lfetch.fault.excl.nt1 [r17],L1_CACHE_BYTES - st8 [r16]=r29 /* save cr.ipsr */ - ;; - lfetch.fault.excl.nt1 [r17] - mov r29=b0 - ;; - adds r16=PT(R8),r1 /* initialize first base pointer */ - adds r17=PT(R9),r1 /* initialize second base pointer */ - mov r18=r0 /* make sure r18 isn't NaT */ - ;; + mov r31=pr // prepare to save predicates + mov r19=12 + mov r29=cr.ipsr + ;; + tbit.z p6,p7=r29,IA64_PSR_VM_BIT + tbit.z p0,p15=r29,IA64_PSR_I_BIT + ;; +(p7) br.sptk kvm_dispatch_interrupt + ;; + mov r27=ar.rsc /* M */ + mov r20=r1 /* A */ + mov r25=ar.unat /* M */ + mov r26=ar.pfs /* I */ + mov r28=cr.iip /* M */ + cover /* B (or nothing) */ + ;; + mov r1=sp + ;; + invala /* M */ + mov r30=cr.ifs + ;; + addl r1=-VMM_PT_REGS_SIZE,r1 + ;; + adds r17=2*L1_CACHE_BYTES,r1 /* really: biggest cache-line size */ + adds r16=PT(CR_IPSR),r1 + ;; + lfetch.fault.excl.nt1 [r17],L1_CACHE_BYTES + st8 [r16]=r29 /* save cr.ipsr */ + ;; + lfetch.fault.excl.nt1 [r17] + mov r29=b0 + ;; + adds r16=PT(R8),r1 /* initialize first base pointer */ + adds r17=PT(R9),r1 /* initialize second base pointer */ + mov r18=r0 /* make sure r18 isn't NaT */ + ;; .mem.offset 0,0; st8.spill [r16]=r8,16 .mem.offset 8,0; st8.spill [r17]=r9,16 ;; .mem.offset 0,0; st8.spill [r16]=r10,24 .mem.offset 8,0; st8.spill [r17]=r11,24 ;; - st8 [r16]=r28,16 /* save cr.iip */ - st8 [r17]=r30,16 /* save cr.ifs */ - mov r8=ar.fpsr /* M */ - mov r9=ar.csd - mov r10=ar.ssd - movl r11=FPSR_DEFAULT /* L-unit */ - ;; - st8 [r16]=r25,16 /* save ar.unat */ - st8 [r17]=r26,16 /* save ar.pfs */ - shl r18=r18,16 /* compute ar.rsc to be used for "loadrs" */ - ;; - st8 [r16]=r27,16 /* save ar.rsc */ - adds r17=16,r17 /* skip over ar_rnat field */ - ;; - st8 [r17]=r31,16 /* save predicates */ - adds r16=16,r16 /* skip over ar_bspstore field */ - ;; - st8 [r16]=r29,16 /* save b0 */ - st8 [r17]=r18,16 /* save ar.rsc value for "loadrs" */ - ;; + st8 [r16]=r28,16 /* save cr.iip */ + st8 [r17]=r30,16 /* save cr.ifs */ + mov r8=ar.fpsr /* M */ + mov r9=ar.csd + mov r10=ar.ssd + movl r11=FPSR_DEFAULT /* L-unit */ + ;; + st8 [r16]=r25,16 /* save ar.unat */ + st8 [r17]=r26,16 /* save ar.pfs */ + shl r18=r18,16 /* compute ar.rsc to be used for "loadrs" */ + ;; + st8 [r16]=r27,16 /* save ar.rsc */ + adds r17=16,r17 /* skip over ar_rnat field */ + ;; + st8 [r17]=r31,16 /* save predicates */ + adds r16=16,r16 /* skip over ar_bspstore field */ + ;; + st8 [r16]=r29,16 /* save b0 */ + st8 [r17]=r18,16 /* save ar.rsc value for "loadrs" */ + ;; .mem.offset 0,0; st8.spill [r16]=r20,16 /* save original r1 */ .mem.offset 8,0; st8.spill [r17]=r12,16 - adds r12=-16,r1 - /* switch to kernel memory stack (with 16 bytes of scratch) */ - ;; + adds r12=-16,r1 + /* switch to kernel memory stack (with 16 bytes of scratch) */ + ;; .mem.offset 0,0; st8.spill [r16]=r13,16 .mem.offset 8,0; st8.spill [r17]=r8,16 /* save ar.fpsr */ - ;; + ;; .mem.offset 0,0; st8.spill [r16]=r15,16 .mem.offset 8,0; st8.spill [r17]=r14,16 - dep r14=-1,r0,60,4 - ;; + dep r14=-1,r0,60,4 + ;; .mem.offset 0,0; st8.spill [r16]=r2,16 .mem.offset 8,0; st8.spill [r17]=r3,16 - adds r2=VMM_PT_REGS_R16_OFFSET,r1 - adds r14 = VMM_VCPU_GP_OFFSET,r13 - ;; - mov r8=ar.ccv - ld8 r14 = [r14] - ;; - mov r1=r14 /* establish kernel global pointer */ - ;; \ - bsw.1 - ;; - alloc r14=ar.pfs,0,0,1,0 // must be first in an insn group - mov out0=r13 - ;; - ssm psr.ic - ;; - srlz.i - ;; - //(p15) ssm psr.i - adds r3=8,r2 // set up second base pointer for SAVE_REST - srlz.i // ensure everybody knows psr.ic is back on - ;; + adds r2=VMM_PT_REGS_R16_OFFSET,r1 + adds r14 = VMM_VCPU_GP_OFFSET,r13 + ;; + mov r8=ar.ccv + ld8 r14 = [r14] + ;; + mov r1=r14 /* establish kernel global pointer */ + ;; \ + bsw.1 + ;; + alloc r14=ar.pfs,0,0,1,0 // must be first in an insn group + mov out0=r13 + ;; + ssm psr.ic + ;; + srlz.i + ;; + //(p15) ssm psr.i + adds r3=8,r2 // set up second base pointer for SAVE_REST + srlz.i // ensure everybody knows psr.ic is back on + ;; .mem.offset 0,0; st8.spill [r2]=r16,16 .mem.offset 8,0; st8.spill [r3]=r17,16 - ;; + ;; .mem.offset 0,0; st8.spill [r2]=r18,16 .mem.offset 8,0; st8.spill [r3]=r19,16 - ;; + ;; .mem.offset 0,0; st8.spill [r2]=r20,16 .mem.offset 8,0; st8.spill [r3]=r21,16 - mov r18=b6 - ;; + mov r18=b6 + ;; .mem.offset 0,0; st8.spill [r2]=r22,16 .mem.offset 8,0; st8.spill [r3]=r23,16 - mov r19=b7 - ;; + mov r19=b7 + ;; .mem.offset 0,0; st8.spill [r2]=r24,16 .mem.offset 8,0; st8.spill [r3]=r25,16 - ;; + ;; .mem.offset 0,0; st8.spill [r2]=r26,16 .mem.offset 8,0; st8.spill [r3]=r27,16 - ;; + ;; .mem.offset 0,0; st8.spill [r2]=r28,16 .mem.offset 8,0; st8.spill [r3]=r29,16 - ;; + ;; .mem.offset 0,0; st8.spill [r2]=r30,16 .mem.offset 8,0; st8.spill [r3]=r31,32 - ;; - mov ar.fpsr=r11 /* M-unit */ - st8 [r2]=r8,8 /* ar.ccv */ - adds r24=PT(B6)-PT(F7),r3 - ;; - stf.spill [r2]=f6,32 - stf.spill [r3]=f7,32 - ;; - stf.spill [r2]=f8,32 - stf.spill [r3]=f9,32 - ;; - stf.spill [r2]=f10 - stf.spill [r3]=f11 - adds r25=PT(B7)-PT(F11),r3 - ;; - st8 [r24]=r18,16 /* b6 */ - st8 [r25]=r19,16 /* b7 */ - ;; - st8 [r24]=r9 /* ar.csd */ - st8 [r25]=r10 /* ar.ssd */ - ;; - srlz.d // make sure we see the effect of cr.ivr - addl r14=@gprel(ia64_leave_nested),gp - ;; - mov rp=r14 - br.call.sptk.many b6=kvm_ia64_handle_irq - ;; + ;; + mov ar.fpsr=r11 /* M-unit */ + st8 [r2]=r8,8 /* ar.ccv */ + adds r24=PT(B6)-PT(F7),r3 + ;; + stf.spill [r2]=f6,32 + stf.spill [r3]=f7,32 + ;; + stf.spill [r2]=f8,32 + stf.spill [r3]=f9,32 + ;; + stf.spill [r2]=f10 + stf.spill [r3]=f11 + adds r25=PT(B7)-PT(F11),r3 + ;; + st8 [r24]=r18,16 /* b6 */ + st8 [r25]=r19,16 /* b7 */ + ;; + st8 [r24]=r9 /* ar.csd */ + st8 [r25]=r10 /* ar.ssd */ + ;; + srlz.d // make sure we see the effect of cr.ivr + addl r14=@gprel(ia64_leave_nested),gp + ;; + mov rp=r14 + br.call.sptk.many b6=kvm_ia64_handle_irq + ;; END(kvm_interrupt) .global kvm_dispatch_vexirq
@@ -421,387 +420,385 @@ END(kvm_interrupt) ////////////////////////////////////////////////////////////////////// // 0x3400 Entry 13 (size 64 bundles) Reserved ENTRY(kvm_virtual_exirq) - mov r31=pr - mov r19=13 - mov r30 =r0 - ;; + mov r31=pr + mov r19=13 + mov r30 =r0 + ;; kvm_dispatch_vexirq: - cmp.eq p6,p0 = 1,r30 - ;; -(p6)add r29 = VMM_VCPU_SAVED_GP_OFFSET,r21 - ;; -(p6)ld8 r1 = [r29] - ;; - KVM_SAVE_MIN_WITH_COVER_R19 - alloc r14=ar.pfs,0,0,1,0 - mov out0=r13 - - ssm psr.ic - ;; - srlz.i // guarantee that interruption collection is on - ;; - //(p15) ssm psr.i // restore psr.i - adds r3=8,r2 // set up second base pointer - ;; - KVM_SAVE_REST - addl r14=@gprel(ia64_leave_hypervisor),gp - ;; - mov rp=r14 - br.call.sptk.many b6=kvm_vexirq + cmp.eq p6,p0 = 1,r30 + ;; +(p6) add r29 = VMM_VCPU_SAVED_GP_OFFSET,r21 + ;; +(p6) ld8 r1 = [r29] + ;; + KVM_SAVE_MIN_WITH_COVER_R19 + alloc r14=ar.pfs,0,0,1,0 + mov out0=r13 + + ssm psr.ic + ;; + srlz.i // guarantee that interruption collection is on + ;; + //(p15) ssm psr.i // restore psr.i + adds r3=8,r2 // set up second base pointer + ;; + KVM_SAVE_REST + addl r14=@gprel(ia64_leave_hypervisor),gp + ;; + mov rp=r14 + br.call.sptk.many b6=kvm_vexirq END(kvm_virtual_exirq) .org kvm_ia64_ivt+0x3800 ///////////////////////////////////////////////////////////////////// // 0x3800 Entry 14 (size 64 bundles) Reserved - KVM_FAULT(14) - // this code segment is from 2.6.16.13 - + KVM_FAULT(14) + // this code segment is from 2.6.16.13 .org kvm_ia64_ivt+0x3c00 /////////////////////////////////////////////////////////////////////// // 0x3c00 Entry 15 (size 64 bundles) Reserved - KVM_FAULT(15) - + KVM_FAULT(15) .org kvm_ia64_ivt+0x4000 /////////////////////////////////////////////////////////////////////// // 0x4000 Entry 16 (size 64 bundles) Reserved - KVM_FAULT(16) + KVM_FAULT(16) .org kvm_ia64_ivt+0x4400 ////////////////////////////////////////////////////////////////////// // 0x4400 Entry 17 (size 64 bundles) Reserved - KVM_FAULT(17) + KVM_FAULT(17) .org kvm_ia64_ivt+0x4800 ////////////////////////////////////////////////////////////////////// // 0x4800 Entry 18 (size 64 bundles) Reserved - KVM_FAULT(18) + KVM_FAULT(18) .org kvm_ia64_ivt+0x4c00 ////////////////////////////////////////////////////////////////////// // 0x4c00 Entry 19 (size 64 bundles) Reserved - KVM_FAULT(19) + KVM_FAULT(19) .org kvm_ia64_ivt+0x5000 ////////////////////////////////////////////////////////////////////// // 0x5000 Entry 20 (size 16 bundles) Page Not Present ENTRY(kvm_page_not_present) - KVM_REFLECT(20) + KVM_REFLECT(20) END(kvm_page_not_present) .org kvm_ia64_ivt+0x5100 /////////////////////////////////////////////////////////////////////// // 0x5100 Entry 21 (size 16 bundles) Key Permission vector ENTRY(kvm_key_permission) - KVM_REFLECT(21) + KVM_REFLECT(21) END(kvm_key_permission) .org kvm_ia64_ivt+0x5200 ////////////////////////////////////////////////////////////////////// // 0x5200 Entry 22 (size 16 bundles) Instruction Access Rights (26) ENTRY(kvm_iaccess_rights) - KVM_REFLECT(22) + KVM_REFLECT(22) END(kvm_iaccess_rights) .org kvm_ia64_ivt+0x5300 ////////////////////////////////////////////////////////////////////// // 0x5300 Entry 23 (size 16 bundles) Data Access Rights (14,53) ENTRY(kvm_daccess_rights) - KVM_REFLECT(23) + KVM_REFLECT(23) END(kvm_daccess_rights) .org kvm_ia64_ivt+0x5400 ///////////////////////////////////////////////////////////////////// // 0x5400 Entry 24 (size 16 bundles) General Exception (5,32,34,36,38,39) ENTRY(kvm_general_exception) - KVM_REFLECT(24) - KVM_FAULT(24) + KVM_REFLECT(24) + KVM_FAULT(24) END(kvm_general_exception) .org kvm_ia64_ivt+0x5500 ////////////////////////////////////////////////////////////////////// // 0x5500 Entry 25 (size 16 bundles) Disabled FP-Register (35) ENTRY(kvm_disabled_fp_reg) - KVM_REFLECT(25) + KVM_REFLECT(25) END(kvm_disabled_fp_reg) .org kvm_ia64_ivt+0x5600 //////////////////////////////////////////////////////////////////// // 0x5600 Entry 26 (size 16 bundles) Nat Consumption (11,23,37,50) ENTRY(kvm_nat_consumption) - KVM_REFLECT(26) + KVM_REFLECT(26) END(kvm_nat_consumption) .org kvm_ia64_ivt+0x5700 ///////////////////////////////////////////////////////////////////// // 0x5700 Entry 27 (size 16 bundles) Speculation (40) ENTRY(kvm_speculation_vector) - KVM_REFLECT(27) + KVM_REFLECT(27) END(kvm_speculation_vector) .org kvm_ia64_ivt+0x5800 ///////////////////////////////////////////////////////////////////// // 0x5800 Entry 28 (size 16 bundles) Reserved - KVM_FAULT(28) + KVM_FAULT(28) .org kvm_ia64_ivt+0x5900 /////////////////////////////////////////////////////////////////// // 0x5900 Entry 29 (size 16 bundles) Debug (16,28,56) ENTRY(kvm_debug_vector) - KVM_FAULT(29) + KVM_FAULT(29) END(kvm_debug_vector) .org kvm_ia64_ivt+0x5a00 /////////////////////////////////////////////////////////////// // 0x5a00 Entry 30 (size 16 bundles) Unaligned Reference (57) ENTRY(kvm_unaligned_access) - KVM_REFLECT(30) + KVM_REFLECT(30) END(kvm_unaligned_access) .org kvm_ia64_ivt+0x5b00 ////////////////////////////////////////////////////////////////////// // 0x5b00 Entry 31 (size 16 bundles) Unsupported Data Reference (57) ENTRY(kvm_unsupported_data_reference) - KVM_REFLECT(31) + KVM_REFLECT(31) END(kvm_unsupported_data_reference) .org kvm_ia64_ivt+0x5c00 //////////////////////////////////////////////////////////////////// // 0x5c00 Entry 32 (size 16 bundles) Floating Point FAULT (65) ENTRY(kvm_floating_point_fault) - KVM_REFLECT(32) + KVM_REFLECT(32) END(kvm_floating_point_fault) .org kvm_ia64_ivt+0x5d00 ///////////////////////////////////////////////////////////////////// // 0x5d00 Entry 33 (size 16 bundles) Floating Point Trap (66) ENTRY(kvm_floating_point_trap) - KVM_REFLECT(33) + KVM_REFLECT(33) END(kvm_floating_point_trap) .org kvm_ia64_ivt+0x5e00 ////////////////////////////////////////////////////////////////////// // 0x5e00 Entry 34 (size 16 bundles) Lower Privilege Transfer Trap (66) ENTRY(kvm_lower_privilege_trap) - KVM_REFLECT(34) + KVM_REFLECT(34) END(kvm_lower_privilege_trap) .org kvm_ia64_ivt+0x5f00 ////////////////////////////////////////////////////////////////////// // 0x5f00 Entry 35 (size 16 bundles) Taken Branch Trap (68) ENTRY(kvm_taken_branch_trap) - KVM_REFLECT(35) + KVM_REFLECT(35) END(kvm_taken_branch_trap) .org kvm_ia64_ivt+0x6000 //////////////////////////////////////////////////////////////////// // 0x6000 Entry 36 (size 16 bundles) Single Step Trap (69) ENTRY(kvm_single_step_trap) - KVM_REFLECT(36) + KVM_REFLECT(36) END(kvm_single_step_trap) .global kvm_virtualization_fault_back .org kvm_ia64_ivt+0x6100 ///////////////////////////////////////////////////////////////////// // 0x6100 Entry 37 (size 16 bundles) Virtualization Fault ENTRY(kvm_virtualization_fault) - mov r31=pr - adds r16 = VMM_VCPU_SAVED_GP_OFFSET,r21 - ;; - st8 [r16] = r1 - adds r17 = VMM_VCPU_GP_OFFSET, r21 - ;; - ld8 r1 = [r17] - cmp.eq p6,p0=EVENT_MOV_FROM_AR,r24 - cmp.eq p7,p0=EVENT_MOV_FROM_RR,r24 - cmp.eq p8,p0=EVENT_MOV_TO_RR,r24 - cmp.eq p9,p0=EVENT_RSM,r24 - cmp.eq p10,p0=EVENT_SSM,r24 - cmp.eq p11,p0=EVENT_MOV_TO_PSR,r24 - cmp.eq p12,p0=EVENT_THASH,r24 - (p6) br.dptk.many kvm_asm_mov_from_ar - (p7) br.dptk.many kvm_asm_mov_from_rr - (p8) br.dptk.many kvm_asm_mov_to_rr - (p9) br.dptk.many kvm_asm_rsm - (p10) br.dptk.many kvm_asm_ssm - (p11) br.dptk.many kvm_asm_mov_to_psr - (p12) br.dptk.many kvm_asm_thash - ;; + mov r31=pr + adds r16 = VMM_VCPU_SAVED_GP_OFFSET,r21 + ;; + st8 [r16] = r1 + adds r17 = VMM_VCPU_GP_OFFSET, r21 + ;; + ld8 r1 = [r17] + cmp.eq p6,p0=EVENT_MOV_FROM_AR,r24 + cmp.eq p7,p0=EVENT_MOV_FROM_RR,r24 + cmp.eq p8,p0=EVENT_MOV_TO_RR,r24 + cmp.eq p9,p0=EVENT_RSM,r24 + cmp.eq p10,p0=EVENT_SSM,r24 + cmp.eq p11,p0=EVENT_MOV_TO_PSR,r24 + cmp.eq p12,p0=EVENT_THASH,r24 +(p6) br.dptk.many kvm_asm_mov_from_ar +(p7) br.dptk.many kvm_asm_mov_from_rr +(p8) br.dptk.many kvm_asm_mov_to_rr +(p9) br.dptk.many kvm_asm_rsm +(p10) br.dptk.many kvm_asm_ssm +(p11) br.dptk.many kvm_asm_mov_to_psr +(p12) br.dptk.many kvm_asm_thash + ;; kvm_virtualization_fault_back: - adds r16 = VMM_VCPU_SAVED_GP_OFFSET,r21 - ;; - ld8 r1 = [r16] - ;; - mov r19=37 - adds r16 = VMM_VCPU_CAUSE_OFFSET,r21 - adds r17 = VMM_VCPU_OPCODE_OFFSET,r21 - ;; - st8 [r16] = r24 - st8 [r17] = r25 - ;; - cmp.ne p6,p0=EVENT_RFI, r24 - (p6) br.sptk kvm_dispatch_virtualization_fault - ;; - adds r18=VMM_VPD_BASE_OFFSET,r21 - ;; - ld8 r18=[r18] - ;; - adds r18=VMM_VPD_VIFS_OFFSET,r18 - ;; - ld8 r18=[r18] - ;; - tbit.z p6,p0=r18,63 - (p6) br.sptk kvm_dispatch_virtualization_fault - ;; - //if vifs.v=1 desert current register frame - alloc r18=ar.pfs,0,0,0,0 - br.sptk kvm_dispatch_virtualization_fault + adds r16 = VMM_VCPU_SAVED_GP_OFFSET,r21 + ;; + ld8 r1 = [r16] + ;; + mov r19=37 + adds r16 = VMM_VCPU_CAUSE_OFFSET,r21 + adds r17 = VMM_VCPU_OPCODE_OFFSET,r21 + ;; + st8 [r16] = r24 + st8 [r17] = r25 + ;; + cmp.ne p6,p0=EVENT_RFI, r24 +(p6) br.sptk kvm_dispatch_virtualization_fault + ;; + adds r18=VMM_VPD_BASE_OFFSET,r21 + ;; + ld8 r18=[r18] + ;; + adds r18=VMM_VPD_VIFS_OFFSET,r18 + ;; + ld8 r18=[r18] + ;; + tbit.z p6,p0=r18,63 +(p6) br.sptk kvm_dispatch_virtualization_fault + ;; +//if vifs.v=1 desert current register frame + alloc r18=ar.pfs,0,0,0,0 + br.sptk kvm_dispatch_virtualization_fault END(kvm_virtualization_fault) .org kvm_ia64_ivt+0x6200 ////////////////////////////////////////////////////////////// // 0x6200 Entry 38 (size 16 bundles) Reserved - KVM_FAULT(38) + KVM_FAULT(38) .org kvm_ia64_ivt+0x6300 ///////////////////////////////////////////////////////////////// // 0x6300 Entry 39 (size 16 bundles) Reserved - KVM_FAULT(39) + KVM_FAULT(39) .org kvm_ia64_ivt+0x6400 ///////////////////////////////////////////////////////////////// // 0x6400 Entry 40 (size 16 bundles) Reserved - KVM_FAULT(40) + KVM_FAULT(40) .org kvm_ia64_ivt+0x6500 ////////////////////////////////////////////////////////////////// // 0x6500 Entry 41 (size 16 bundles) Reserved - KVM_FAULT(41) + KVM_FAULT(41) .org kvm_ia64_ivt+0x6600 ////////////////////////////////////////////////////////////////// // 0x6600 Entry 42 (size 16 bundles) Reserved - KVM_FAULT(42) + KVM_FAULT(42) .org kvm_ia64_ivt+0x6700 ////////////////////////////////////////////////////////////////// // 0x6700 Entry 43 (size 16 bundles) Reserved - KVM_FAULT(43) + KVM_FAULT(43) .org kvm_ia64_ivt+0x6800 ////////////////////////////////////////////////////////////////// // 0x6800 Entry 44 (size 16 bundles) Reserved - KVM_FAULT(44) + KVM_FAULT(44) .org kvm_ia64_ivt+0x6900 /////////////////////////////////////////////////////////////////// // 0x6900 Entry 45 (size 16 bundles) IA-32 Exeception //(17,18,29,41,42,43,44,58,60,61,62,72,73,75,76,77) ENTRY(kvm_ia32_exception) - KVM_FAULT(45) + KVM_FAULT(45) END(kvm_ia32_exception) .org kvm_ia64_ivt+0x6a00 //////////////////////////////////////////////////////////////////// // 0x6a00 Entry 46 (size 16 bundles) IA-32 Intercept (30,31,59,70,71) ENTRY(kvm_ia32_intercept) - KVM_FAULT(47) + KVM_FAULT(47) END(kvm_ia32_intercept) .org kvm_ia64_ivt+0x6c00 ///////////////////////////////////////////////////////////////////// // 0x6c00 Entry 48 (size 16 bundles) Reserved - KVM_FAULT(48) + KVM_FAULT(48) .org kvm_ia64_ivt+0x6d00 ////////////////////////////////////////////////////////////////////// // 0x6d00 Entry 49 (size 16 bundles) Reserved - KVM_FAULT(49) + KVM_FAULT(49) .org kvm_ia64_ivt+0x6e00 ////////////////////////////////////////////////////////////////////// // 0x6e00 Entry 50 (size 16 bundles) Reserved - KVM_FAULT(50) + KVM_FAULT(50) .org kvm_ia64_ivt+0x6f00 ///////////////////////////////////////////////////////////////////// // 0x6f00 Entry 51 (size 16 bundles) Reserved - KVM_FAULT(52) + KVM_FAULT(52) .org kvm_ia64_ivt+0x7100 //////////////////////////////////////////////////////////////////// // 0x7100 Entry 53 (size 16 bundles) Reserved - KVM_FAULT(53) + KVM_FAULT(53) .org kvm_ia64_ivt+0x7200 ///////////////////////////////////////////////////////////////////// // 0x7200 Entry 54 (size 16 bundles) Reserved - KVM_FAULT(54) + KVM_FAULT(54) .org kvm_ia64_ivt+0x7300 //////////////////////////////////////////////////////////////////// // 0x7300 Entry 55 (size 16 bundles) Reserved - KVM_FAULT(55) + KVM_FAULT(55) .org kvm_ia64_ivt+0x7400 //////////////////////////////////////////////////////////////////// // 0x7400 Entry 56 (size 16 bundles) Reserved - KVM_FAULT(56) + KVM_FAULT(56) .org kvm_ia64_ivt+0x7500 ///////////////////////////////////////////////////////////////////// // 0x7500 Entry 57 (size 16 bundles) Reserved - KVM_FAULT(57) + KVM_FAULT(57) .org kvm_ia64_ivt+0x7600 ///////////////////////////////////////////////////////////////////// // 0x7600 Entry 58 (size 16 bundles) Reserved - KVM_FAULT(58) + KVM_FAULT(58) .org kvm_ia64_ivt+0x7700 //////////////////////////////////////////////////////////////////// // 0x7700 Entry 59 (size 16 bundles) Reserved - KVM_FAULT(59) + KVM_FAULT(59) .org kvm_ia64_ivt+0x7800 //////////////////////////////////////////////////////////////////// // 0x7800 Entry 60 (size 16 bundles) Reserved - KVM_FAULT(60) + KVM_FAULT(60) .org kvm_ia64_ivt+0x7900 ///////////////////////////////////////////////////////////////////// // 0x7900 Entry 61 (size 16 bundles) Reserved - KVM_FAULT(61) + KVM_FAULT(61) .org kvm_ia64_ivt+0x7a00 ///////////////////////////////////////////////////////////////////// // 0x7a00 Entry 62 (size 16 bundles) Reserved - KVM_FAULT(62) + KVM_FAULT(62) .org kvm_ia64_ivt+0x7b00 ///////////////////////////////////////////////////////////////////// // 0x7b00 Entry 63 (size 16 bundles) Reserved - KVM_FAULT(63) + KVM_FAULT(63) .org kvm_ia64_ivt+0x7c00 //////////////////////////////////////////////////////////////////// // 0x7c00 Entry 64 (size 16 bundles) Reserved - KVM_FAULT(64) + KVM_FAULT(64) .org kvm_ia64_ivt+0x7d00 ///////////////////////////////////////////////////////////////////// // 0x7d00 Entry 65 (size 16 bundles) Reserved - KVM_FAULT(65) + KVM_FAULT(65) .org kvm_ia64_ivt+0x7e00 ///////////////////////////////////////////////////////////////////// // 0x7e00 Entry 66 (size 16 bundles) Reserved - KVM_FAULT(66) + KVM_FAULT(66) .org kvm_ia64_ivt+0x7f00 //////////////////////////////////////////////////////////////////// // 0x7f00 Entry 67 (size 16 bundles) Reserved - KVM_FAULT(67) + KVM_FAULT(67) .org kvm_ia64_ivt+0x8000 // There is no particular reason for this code to be here, other than that
@@ -811,132 +808,128 @@ END(kvm_ia32_intercept) ENTRY(kvm_dtlb_miss_dispatch) - mov r19 = 2 - KVM_SAVE_MIN_WITH_COVER_R19 - alloc r14=ar.pfs,0,0,3,0 - mov out0=cr.ifa - mov out1=r15 - adds r3=8,r2 // set up second base pointer - ;; - ssm psr.ic - ;; - srlz.i // guarantee that interruption collection is on - ;; - //(p15) ssm psr.i // restore psr.i - addl r14=@gprel(ia64_leave_hypervisor_prepare),gp - ;; - KVM_SAVE_REST - KVM_SAVE_EXTRA - mov rp=r14 - ;; - adds out2=16,r12 - br.call.sptk.many b6=kvm_page_fault + mov r19 = 2 + KVM_SAVE_MIN_WITH_COVER_R19 + alloc r14=ar.pfs,0,0,3,0 + mov out0=cr.ifa + mov out1=r15 + adds r3=8,r2 // set up second base pointer + ;; + ssm psr.ic + ;; + srlz.i // guarantee that interruption collection is on + ;; + //(p15) ssm psr.i // restore psr.i + addl r14=@gprel(ia64_leave_hypervisor_prepare),gp + ;; + KVM_SAVE_REST + KVM_SAVE_EXTRA + mov rp=r14 + ;; + adds out2=16,r12 + br.call.sptk.many b6=kvm_page_fault END(kvm_dtlb_miss_dispatch) ENTRY(kvm_itlb_miss_dispatch) - KVM_SAVE_MIN_WITH_COVER_R19 - alloc r14=ar.pfs,0,0,3,0 - mov out0=cr.ifa - mov out1=r15 - adds r3=8,r2 // set up second base pointer - ;; - ssm psr.ic - ;; - srlz.i // guarantee that interruption collection is on - ;; - //(p15) ssm psr.i // restore psr.i - addl r14=@gprel(ia64_leave_hypervisor),gp - ;; - KVM_SAVE_REST - mov rp=r14 - ;; - adds out2=16,r12 - br.call.sptk.many b6=kvm_page_fault + KVM_SAVE_MIN_WITH_COVER_R19 + alloc r14=ar.pfs,0,0,3,0 + mov out0=cr.ifa + mov out1=r15 + adds r3=8,r2 // set up second base pointer + ;; + ssm psr.ic + ;; + srlz.i // guarantee that interruption collection is on + ;; + //(p15) ssm psr.i // restore psr.i + addl r14=@gprel(ia64_leave_hypervisor),gp + ;; + KVM_SAVE_REST + mov rp=r14 + ;; + adds out2=16,r12 + br.call.sptk.many b6=kvm_page_fault END(kvm_itlb_miss_dispatch) ENTRY(kvm_dispatch_reflection) - /* - * Input: - * psr.ic: off - * r19: intr type (offset into ivt, see ia64_int.h) - * r31: contains saved predicates (pr) - */ - KVM_SAVE_MIN_WITH_COVER_R19 - alloc r14=ar.pfs,0,0,5,0 - mov out0=cr.ifa - mov out1=cr.isr - mov out2=cr.iim - mov out3=r15 - adds r3=8,r2 // set up second base pointer - ;; - ssm psr.ic - ;; - srlz.i // guarantee that interruption collection is on - ;; - //(p15) ssm psr.i // restore psr.i - addl r14=@gprel(ia64_leave_hypervisor),gp - ;; - KVM_SAVE_REST - mov rp=r14 - ;; - adds out4=16,r12 - br.call.sptk.many b6=reflect_interruption +/* + * Input: + * psr.ic: off + * r19: intr type (offset into ivt, see ia64_int.h) + * r31: contains saved predicates (pr) + */ + KVM_SAVE_MIN_WITH_COVER_R19 + alloc r14=ar.pfs,0,0,5,0 + mov out0=cr.ifa + mov out1=cr.isr + mov out2=cr.iim + mov out3=r15 + adds r3=8,r2 // set up second base pointer + ;; + ssm psr.ic + ;; + srlz.i // guarantee that interruption collection is on + ;; + //(p15) ssm psr.i // restore psr.i + addl r14=@gprel(ia64_leave_hypervisor),gp + ;; + KVM_SAVE_REST + mov rp=r14 + ;; + adds out4=16,r12 + br.call.sptk.many b6=reflect_interruption END(kvm_dispatch_reflection) ENTRY(kvm_dispatch_virtualization_fault) - adds r16 = VMM_VCPU_CAUSE_OFFSET,r21 - adds r17 = VMM_VCPU_OPCODE_OFFSET,r21 - ;; - st8 [r16] = r24 - st8 [r17] = r25 - ;; - KVM_SAVE_MIN_WITH_COVER_R19 - ;; - alloc r14=ar.pfs,0,0,2,0 // now it's safe (must be first in insn group!) - mov out0=r13 //vcpu - adds r3=8,r2 // set up second base pointer - ;; - ssm psr.ic - ;; - srlz.i // guarantee that interruption collection is on - ;; - //(p15) ssm psr.i // restore psr.i - addl r14=@gprel(ia64_leave_hypervisor_prepare),gp - ;; - KVM_SAVE_REST - KVM_SAVE_EXTRA - mov rp=r14 - ;; - adds out1=16,sp //regs - br.call.sptk.many b6=kvm_emulate + adds r16 = VMM_VCPU_CAUSE_OFFSET,r21 + adds r17 = VMM_VCPU_OPCODE_OFFSET,r21 + ;; + st8 [r16] = r24 + st8 [r17] = r25 + ;; + KVM_SAVE_MIN_WITH_COVER_R19 + ;; + alloc r14=ar.pfs,0,0,2,0 // (must be first in insn group!) + mov out0=r13 //vcpu + adds r3=8,r2 // set up second base pointer + ;; + ssm psr.ic + ;; + srlz.i // guarantee that interruption collection is on + ;; + //(p15) ssm psr.i // restore psr.i + addl r14=@gprel(ia64_leave_hypervisor_prepare),gp + ;; + KVM_SAVE_REST + KVM_SAVE_EXTRA + mov rp=r14 + ;; + adds out1=16,sp //regs + br.call.sptk.many b6=kvm_emulate END(kvm_dispatch_virtualization_fault) ENTRY(kvm_dispatch_interrupt) - KVM_SAVE_MIN_WITH_COVER_R19 // uses r31; defines r2 and r3 - ;; - alloc r14=ar.pfs,0,0,1,0 // must be first in an insn group - //mov out0=cr.ivr // pass cr.ivr as first arg - adds r3=8,r2 // set up second base pointer for SAVE_REST - ;; - ssm psr.ic - ;; - srlz.i - ;; - //(p15) ssm psr.i - addl r14=@gprel(ia64_leave_hypervisor),gp - ;; - KVM_SAVE_REST - mov rp=r14 - ;; - mov out0=r13 // pass pointer to pt_regs as second arg - br.call.sptk.many b6=kvm_ia64_handle_irq + KVM_SAVE_MIN_WITH_COVER_R19 // uses r31; defines r2 and r3 + ;; + alloc r14=ar.pfs,0,0,1,0 // must be first in an insn group + adds r3=8,r2 // set up second base pointer for SAVE_REST + ;; + ssm psr.ic + ;; + srlz.i + ;; + //(p15) ssm psr.i + addl r14=@gprel(ia64_leave_hypervisor),gp + ;; + KVM_SAVE_REST + mov rp=r14 + ;; + mov out0=r13 // pass pointer to pt_regs as second arg + br.call.sptk.many b6=kvm_ia64_handle_irq END(kvm_dispatch_interrupt) - - - GLOBAL_ENTRY(ia64_leave_nested) rsm psr.i ;;
@@ -1015,7 +1008,7 @@ GLOBAL_ENTRY(ia64_leave_nested) ;; ldf.fill f11=[r2] // mov r18=r13 -// mov r21=r13 +// mov r21=r13 adds r16=PT(CR_IPSR)+16,r12 adds r17=PT(CR_IIP)+16,r12 ;;
@@ -1065,138 +1058,135 @@ GLOBAL_ENTRY(ia64_leave_nested) rfi END(ia64_leave_nested) - - GLOBAL_ENTRY(ia64_leave_hypervisor_prepare) - /* - * work.need_resched etc. mustn't get changed - *by this CPU before it returns to - ;; - * user- or fsys-mode, hence we disable interrupts early on: - */ - adds r2 = PT(R4)+16,r12 - adds r3 = PT(R5)+16,r12 - adds r8 = PT(EML_UNAT)+16,r12 - ;; - ld8 r8 = [r8] - ;; - mov ar.unat=r8 - ;; - ld8.fill r4=[r2],16 //load r4 - ld8.fill r5=[r3],16 //load r5 - ;; - ld8.fill r6=[r2] //load r6 - ld8.fill r7=[r3] //load r7 - ;; +/* + * work.need_resched etc. mustn't get changed + *by this CPU before it returns to + * user- or fsys-mode, hence we disable interrupts early on: + */ + adds r2 = PT(R4)+16,r12 + adds r3 = PT(R5)+16,r12 + adds r8 = PT(EML_UNAT)+16,r12 + ;; + ld8 r8 = [r8] + ;; + mov ar.unat=r8 + ;; + ld8.fill r4=[r2],16 //load r4 + ld8.fill r5=[r3],16 //load r5 + ;; + ld8.fill r6=[r2] //load r6 + ld8.fill r7=[r3] //load r7 + ;; END(ia64_leave_hypervisor_prepare) //fall through GLOBAL_ENTRY(ia64_leave_hypervisor) - rsm psr.i - ;; - br.call.sptk.many b0=leave_hypervisor_tail - ;; - adds r20=PT(PR)+16,r12 - adds r8=PT(EML_UNAT)+16,r12 - ;; - ld8 r8=[r8] - ;; - mov ar.unat=r8 - ;; - lfetch [r20],PT(CR_IPSR)-PT(PR) - adds r2 = PT(B6)+16,r12 - adds r3 = PT(B7)+16,r12 - ;; - lfetch [r20] - ;; - ld8 r24=[r2],16 /* B6 */ - ld8 r25=[r3],16 /* B7 */ - ;; - ld8 r26=[r2],16 /* ar_csd */ - ld8 r27=[r3],16 /* ar_ssd */ - mov b6 = r24 - ;; - ld8.fill r8=[r2],16 - ld8.fill r9=[r3],16 - mov b7 = r25 - ;; - mov ar.csd = r26 - mov ar.ssd = r27 - ;; - ld8.fill r10=[r2],PT(R15)-PT(R10) - ld8.fill r11=[r3],PT(R14)-PT(R11) - ;; - ld8.fill r15=[r2],PT(R16)-PT(R15) - ld8.fill r14=[r3],PT(R17)-PT(R14) - ;; - ld8.fill r16=[r2],16 - ld8.fill r17=[r3],16 - ;; - ld8.fill r18=[r2],16 - ld8.fill r19=[r3],16 - ;; - ld8.fill r20=[r2],16 - ld8.fill r21=[r3],16 - ;; - ld8.fill r22=[r2],16 - ld8.fill r23=[r3],16 - ;; - ld8.fill r24=[r2],16 - ld8.fill r25=[r3],16 - ;; - ld8.fill r26=[r2],16 - ld8.fill r27=[r3],16 - ;; - ld8.fill r28=[r2],16 - ld8.fill r29=[r3],16 - ;; - ld8.fill r30=[r2],PT(F6)-PT(R30) - ld8.fill r31=[r3],PT(F7)-PT(R31) - ;; - rsm psr.i | psr.ic - // initiate turning off of interrupt and interruption collection - invala // invalidate ALAT - ;; - srlz.i // ensure interruption collection is off - ;; - bsw.0 - ;; - adds r16 = PT(CR_IPSR)+16,r12 - adds r17 = PT(CR_IIP)+16,r12 - mov r21=r13 // get current - ;; - ld8 r31=[r16],16 // load cr.ipsr - ld8 r30=[r17],16 // load cr.iip - ;; - ld8 r29=[r16],16 // load cr.ifs - ld8 r28=[r17],16 // load ar.unat - ;; - ld8 r27=[r16],16 // load ar.pfs - ld8 r26=[r17],16 // load ar.rsc - ;; - ld8 r25=[r16],16 // load ar.rnat - ld8 r24=[r17],16 // load ar.bspstore - ;; - ld8 r23=[r16],16 // load predicates - ld8 r22=[r17],16 // load b0 - ;; - ld8 r20=[r16],16 // load ar.rsc value for "loadrs" - ld8.fill r1=[r17],16 //load r1 - ;; - ld8.fill r12=[r16],16 //load r12 - ld8.fill r13=[r17],PT(R2)-PT(R13) //load r13 - ;; - ld8 r19=[r16],PT(R3)-PT(AR_FPSR) //load ar_fpsr - ld8.fill r2=[r17],PT(AR_CCV)-PT(R2) //load r2 - ;; - ld8.fill r3=[r16] //load r3 - ld8 r18=[r17] //load ar_ccv - ;; - mov ar.fpsr=r19 - mov ar.ccv=r18 - shr.u r18=r20,16 - ;; + rsm psr.i + ;; + br.call.sptk.many b0=leave_hypervisor_tail + ;; + adds r20=PT(PR)+16,r12 + adds r8=PT(EML_UNAT)+16,r12 + ;; + ld8 r8=[r8] + ;; + mov ar.unat=r8 + ;; + lfetch [r20],PT(CR_IPSR)-PT(PR) + adds r2 = PT(B6)+16,r12 + adds r3 = PT(B7)+16,r12 + ;; + lfetch [r20] + ;; + ld8 r24=[r2],16 /* B6 */ + ld8 r25=[r3],16 /* B7 */ + ;; + ld8 r26=[r2],16 /* ar_csd */ + ld8 r27=[r3],16 /* ar_ssd */ + mov b6 = r24 + ;; + ld8.fill r8=[r2],16 + ld8.fill r9=[r3],16 + mov b7 = r25 + ;; + mov ar.csd = r26 + mov ar.ssd = r27 + ;; + ld8.fill r10=[r2],PT(R15)-PT(R10) + ld8.fill r11=[r3],PT(R14)-PT(R11) + ;; + ld8.fill r15=[r2],PT(R16)-PT(R15) + ld8.fill r14=[r3],PT(R17)-PT(R14) + ;; + ld8.fill r16=[r2],16 + ld8.fill r17=[r3],16 + ;; + ld8.fill r18=[r2],16 + ld8.fill r19=[r3],16 + ;; + ld8.fill r20=[r2],16 + ld8.fill r21=[r3],16 + ;; + ld8.fill r22=[r2],16 + ld8.fill r23=[r3],16 + ;; + ld8.fill r24=[r2],16 + ld8.fill r25=[r3],16 + ;; + ld8.fill r26=[r2],16 + ld8.fill r27=[r3],16 + ;; + ld8.fill r28=[r2],16 + ld8.fill r29=[r3],16 + ;; + ld8.fill r30=[r2],PT(F6)-PT(R30) + ld8.fill r31=[r3],PT(F7)-PT(R31) + ;; + rsm psr.i | psr.ic + // initiate turning off of interrupt and interruption collection + invala // invalidate ALAT + ;; + srlz.i // ensure interruption collection is off + ;; + bsw.0 + ;; + adds r16 = PT(CR_IPSR)+16,r12 + adds r17 = PT(CR_IIP)+16,r12 + mov r21=r13 // get current + ;; + ld8 r31=[r16],16 // load cr.ipsr + ld8 r30=[r17],16 // load cr.iip + ;; + ld8 r29=[r16],16 // load cr.ifs + ld8 r28=[r17],16 // load ar.unat + ;; + ld8 r27=[r16],16 // load ar.pfs + ld8 r26=[r17],16 // load ar.rsc + ;; + ld8 r25=[r16],16 // load ar.rnat + ld8 r24=[r17],16 // load ar.bspstore + ;; + ld8 r23=[r16],16 // load predicates + ld8 r22=[r17],16 // load b0 + ;; + ld8 r20=[r16],16 // load ar.rsc value for "loadrs" + ld8.fill r1=[r17],16 //load r1 + ;; + ld8.fill r12=[r16],16 //load r12 + ld8.fill r13=[r17],PT(R2)-PT(R13) //load r13 + ;; + ld8 r19=[r16],PT(R3)-PT(AR_FPSR) //load ar_fpsr + ld8.fill r2=[r17],PT(AR_CCV)-PT(R2) //load r2 + ;; + ld8.fill r3=[r16] //load r3 + ld8 r18=[r17] //load ar_ccv + ;; + mov ar.fpsr=r19 + mov ar.ccv=r18 + shr.u r18=r20,16 + ;; kvm_rbs_switch: - mov r19=96 + mov r19=96 kvm_dont_preserve_current_frame: /*
@@ -1208,76 +1198,76 @@ kvm_dont_preserve_current_frame: # define pReturn p7 # define Nregs 14 - alloc loc0=ar.pfs,2,Nregs-2,2,0 - shr.u loc1=r18,9 // RNaTslots <= floor(dirtySize / (64*8)) - sub r19=r19,r18 // r19 = (physStackedSize + 8) - dirtySize - ;; - mov ar.rsc=r20 // load ar.rsc to be used for "loadrs" - shladd in0=loc1,3,r19 - mov in1=0 - ;; - TEXT_ALIGN(32) + alloc loc0=ar.pfs,2,Nregs-2,2,0 + shr.u loc1=r18,9 // RNaTslots <= floor(dirtySize / (64*8)) + sub r19=r19,r18 // r19 = (physStackedSize + 8) - dirtySize + ;; + mov ar.rsc=r20 // load ar.rsc to be used for "loadrs" + shladd in0=loc1,3,r19 + mov in1=0 + ;; + TEXT_ALIGN(32) kvm_rse_clear_invalid: - alloc loc0=ar.pfs,2,Nregs-2,2,0 - cmp.lt pRecurse,p0=Nregs*8,in0 - // if more than Nregs regs left to clear, (re)curse - add out0=-Nregs*8,in0 - add out1=1,in1 // increment recursion count - mov loc1=0 - mov loc2=0 - ;; - mov loc3=0 - mov loc4=0 - mov loc5=0 - mov loc6=0 - mov loc7=0 + alloc loc0=ar.pfs,2,Nregs-2,2,0 + cmp.lt pRecurse,p0=Nregs*8,in0 + // if more than Nregs regs left to clear, (re)curse + add out0=-Nregs*8,in0 + add out1=1,in1 // increment recursion count + mov loc1=0 + mov loc2=0 + ;; + mov loc3=0 + mov loc4=0 + mov loc5=0 + mov loc6=0 + mov loc7=0 (pRecurse) br.call.dptk.few b0=kvm_rse_clear_invalid - ;; - mov loc8=0 - mov loc9=0 - cmp.ne pReturn,p0=r0,in1 - // if recursion count != 0, we need to do a br.ret - mov loc10=0 - mov loc11=0 + ;; + mov loc8=0 + mov loc9=0 + cmp.ne pReturn,p0=r0,in1 + // if recursion count != 0, we need to do a br.ret + mov loc10=0 + mov loc11=0 (pReturn) br.ret.dptk.many b0 # undef pRecurse # undef pReturn // loadrs has already been shifted - alloc r16=ar.pfs,0,0,0,0 // drop current register frame - ;; - loadrs - ;; - mov ar.bspstore=r24 - ;; - mov ar.unat=r28 - mov ar.rnat=r25 - mov ar.rsc=r26 - ;; - mov cr.ipsr=r31 - mov cr.iip=r30 - mov cr.ifs=r29 - mov ar.pfs=r27 - adds r18=VMM_VPD_BASE_OFFSET,r21 - ;; - ld8 r18=[r18] //vpd - adds r17=VMM_VCPU_ISR_OFFSET,r21 - ;; - ld8 r17=[r17] - adds r19=VMM_VPD_VPSR_OFFSET,r18 - ;; - ld8 r19=[r19] //vpsr - mov r25=r18 - adds r16= VMM_VCPU_GP_OFFSET,r21 - ;; - ld8 r16= [r16] // Put gp in r24 - movl r24=@gprel(ia64_vmm_entry) // calculate return address - ;; - add r24=r24,r16 - ;; - br.sptk.many kvm_vps_sync_write // call the service - ;; + alloc r16=ar.pfs,0,0,0,0 // drop current register frame + ;; + loadrs + ;; + mov ar.bspstore=r24 + ;; + mov ar.unat=r28 + mov ar.rnat=r25 + mov ar.rsc=r26 + ;; + mov cr.ipsr=r31 + mov cr.iip=r30 + mov cr.ifs=r29 + mov ar.pfs=r27 + adds r18=VMM_VPD_BASE_OFFSET,r21 + ;; + ld8 r18=[r18] //vpd + adds r17=VMM_VCPU_ISR_OFFSET,r21 + ;; + ld8 r17=[r17] + adds r19=VMM_VPD_VPSR_OFFSET,r18 + ;; + ld8 r19=[r19] //vpsr + mov r25=r18 + adds r16= VMM_VCPU_GP_OFFSET,r21 + ;; + ld8 r16= [r16] // Put gp in r24 + movl r24=@gprel(ia64_vmm_entry) // calculate return address + ;; + add r24=r24,r16 + ;; + br.sptk.many kvm_vps_sync_write // call the service + ;; END(ia64_leave_hypervisor) // fall through GLOBAL_ENTRY(ia64_vmm_entry)
@@ -1290,16 +1280,14 @@ GLOBAL_ENTRY(ia64_vmm_entry) * r22:b0 * r23:predicate */ - mov r24=r22 - mov r25=r18 - tbit.nz p1,p2 = r19,IA64_PSR_IC_BIT // p1=vpsr.ic - (p1) br.cond.sptk.few kvm_vps_resume_normal - (p2) br.cond.sptk.many kvm_vps_resume_handler - ;; + mov r24=r22 + mov r25=r18 + tbit.nz p1,p2 = r19,IA64_PSR_IC_BIT // p1=vpsr.ic +(p1) br.cond.sptk.few kvm_vps_resume_normal +(p2) br.cond.sptk.many kvm_vps_resume_handler + ;; END(ia64_vmm_entry) - - /* * extern u64 ia64_call_vsa(u64 proc, u64 arg1, u64 arg2, * u64 arg3, u64 arg4, u64 arg5,
@@ -1317,88 +1305,88 @@ psrsave = loc2 entry = loc3 hostret = r24 - alloc pfssave=ar.pfs,4,4,0,0 - mov rpsave=rp - adds entry=VMM_VCPU_VSA_BASE_OFFSET, r13 - ;; - ld8 entry=[entry] -1: mov hostret=ip - mov r25=in1 // copy arguments - mov r26=in2 - mov r27=in3 - mov psrsave=psr - ;; - tbit.nz p6,p0=psrsave,14 // IA64_PSR_I - tbit.nz p7,p0=psrsave,13 // IA64_PSR_IC - ;; - add hostret=2f-1b,hostret // calculate return address - add entry=entry,in0 - ;; - rsm psr.i | psr.ic - ;; - srlz.i - mov b6=entry - br.cond.sptk b6 // call the service + alloc pfssave=ar.pfs,4,4,0,0 + mov rpsave=rp + adds entry=VMM_VCPU_VSA_BASE_OFFSET, r13 + ;; + ld8 entry=[entry] +1: mov hostret=ip + mov r25=in1 // copy arguments + mov r26=in2 + mov r27=in3 + mov psrsave=psr + ;; + tbit.nz p6,p0=psrsave,14 // IA64_PSR_I + tbit.nz p7,p0=psrsave,13 // IA64_PSR_IC + ;; + add hostret=2f-1b,hostret // calculate return address + add entry=entry,in0 + ;; + rsm psr.i | psr.ic + ;; + srlz.i + mov b6=entry + br.cond.sptk b6 // call the service 2: - // Architectural sequence for enabling interrupts if necessary +// Architectural sequence for enabling interrupts if necessary (p7) ssm psr.ic - ;; + ;; (p7) srlz.i - ;; + ;; //(p6) ssm psr.i - ;; - mov rp=rpsave - mov ar.pfs=pfssave - mov r8=r31 - ;; - srlz.d - br.ret.sptk rp + ;; + mov rp=rpsave + mov ar.pfs=pfssave + mov r8=r31 + ;; + srlz.d + br.ret.sptk rp END(ia64_call_vsa) #define INIT_BSPSTORE ((4<<30)-(12<<20)-0x100) GLOBAL_ENTRY(vmm_reset_entry) - //set up ipsr, iip, vpd.vpsr, dcr - // For IPSR: it/dt/rt=1, i/ic=1, si=1, vm/bn=1 - // For DCR: all bits 0 - bsw.0 - ;; - mov r21 =r13 - adds r14=-VMM_PT_REGS_SIZE, r12 - ;; - movl r6=0x501008826000 // IPSR dt/rt/it:1;i/ic:1, si:1, vm/bn:1 - movl r10=0x8000000000000000 - adds r16=PT(CR_IIP), r14 - adds r20=PT(R1), r14 - ;; - rsm psr.ic | psr.i - ;; - srlz.i - ;; - mov ar.rsc = 0 - ;; - flushrs - ;; - mov ar.bspstore = 0 - // clear BSPSTORE - ;; - mov cr.ipsr=r6 - mov cr.ifs=r10 - ld8 r4 = [r16] // Set init iip for first run. - ld8 r1 = [r20] - ;; - mov cr.iip=r4 - adds r16=VMM_VPD_BASE_OFFSET,r13 - ;; - ld8 r18=[r16] - ;; - adds r19=VMM_VPD_VPSR_OFFSET,r18 - ;; - ld8 r19=[r19] - mov r17=r0 - mov r22=r0 - mov r23=r0 - br.cond.sptk ia64_vmm_entry - br.ret.sptk b0 + //set up ipsr, iip, vpd.vpsr, dcr + // For IPSR: it/dt/rt=1, i/ic=1, si=1, vm/bn=1 + // For DCR: all bits 0 + bsw.0 + ;; + mov r21 =r13 + adds r14=-VMM_PT_REGS_SIZE, r12 + ;; + movl r6=0x501008826000 // IPSR dt/rt/it:1;i/ic:1, si:1, vm/bn:1 + movl r10=0x8000000000000000 + adds r16=PT(CR_IIP), r14 + adds r20=PT(R1), r14 + ;; + rsm psr.ic | psr.i + ;; + srlz.i + ;; + mov ar.rsc = 0 + ;; + flushrs + ;; + mov ar.bspstore = 0 + // clear BSPSTORE + ;; + mov cr.ipsr=r6 + mov cr.ifs=r10 + ld8 r4 = [r16] // Set init iip for first run. + ld8 r1 = [r20] + ;; + mov cr.iip=r4 + adds r16=VMM_VPD_BASE_OFFSET,r13 + ;; + ld8 r18=[r16] + ;; + adds r19=VMM_VPD_VPSR_OFFSET,r18 + ;; + ld8 r19=[r19] + mov r17=r0 + mov r22=r0 + mov r23=r0 + br.cond.sptk ia64_vmm_entry + br.ret.sptk b0 END(vmm_reset_entry)
Signature · 1 line
1.6.0.3
PATCH 33/44 KVM: VMX: Conditionally request interrupt window after injecting irq
In reply to Avi KivityIf we're injecting an interrupt, and another one is pending, request an interrupt window notification so we don't have excess latency on the second interrupt.
This shouldn't happen in practice since an EOI will be issued, giving a second chance to request an interrupt window, but...
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/vmx.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/kvm/vmx.c +2 -0
@@ -3304,6 +3304,8 @@ static void vmx_intr_assist(struct kvm_vcpu *vcpu)if (vcpu->arch.interrupt.pending) {vmx_inject_irq (vcpu, vcpu->arch.interrupt.nr);kvm_timer_intr_post (vcpu, vcpu->arch.interrupt.nr); +if (kvm_cpu_has_interrupt (vcpu)) +enable_irq_window (vcpu); } }
Signature · 1 line
1.6.0.3
PATCH 34/44 x86: KVM guest: sign kvmclock as paravirt
In reply to Avi KivityFrom: Glauber Costa <[email protected]>
Currently, we only set the KVM paravirt signature in case of CONFIG_KVM_GUEST. However, it is possible to have it turned off, while CONFIG_KVM_CLOCK is turned on. This is also a paravirt case, and should be shown accordingly.
Signed-off-by Glauber Costa <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kernel/kvmclock.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-)
arch/x86/kernel/kvmclock.c +2 -0
@@ -194,5 +194,7 @@ void __init kvmclock_init(void)#endif kvm_get_preset_lpj ();clocksource_register (&kvm_clock); + pv_info.paravirt_enabled =1 ; + pv_info.name ="KVM" ; } }
Signature · 1 line
1.6.0.3
PATCH 35/44 KVM: Move ack notifier register and IRQ sourcd ID request
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
Distinguish common part for device assignment and INTx part, perparing for refactor later.
Signed-off-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- virt/kvm/kvm_main.c | 30 +++++++++++++++++++----------- 1 files changed, 19 insertions(+), 11 deletions(-)
virt/kvm/kvm_main.c +19 -11
@@ -192,16 +192,31 @@ static int kvm_vm_ioctl_assign_irq(struct kvm *kvm,return -EINVAL ; } -if (match->irq_requested) { +if (!match->irq_requested) { +INIT_WORK (&match->interrupt_work, + kvm_assigned_dev_interrupt_work_handler); +if (irqchip_in_kernel (kvm)) { +/* Register ack nofitier */ + match->ack_notifier.gsi =-1 ; + match->ack_notifier.irq_acked = + kvm_assigned_dev_ack_irq; +kvm_register_irq_ack_notifier (kvm, + &match->ack_notifier); + +/* Request IRQ source ID */ + r =kvm_request_irq_source_id (kvm); +if (r <0 ) + goto out_release; +else + match->irq_source_id = r; + } + }else { match->guest_irq = assigned_irq->guest_irq; match->ack_notifier.gsi = assigned_irq->guest_irq;mutex_unlock (&kvm->lock);return 0 ; } -INIT_WORK (&match->interrupt_work, - kvm_assigned_dev_interrupt_work_handler); -if (irqchip_in_kernel (kvm)) {if (!capable (CAP_SYS_RAWIO )) { r = -EPERM ;
@@ -214,13 +229,6 @@ static int kvm_vm_ioctl_assign_irq(struct kvm *kvm, match->host_irq = match->dev->irq; match->guest_irq = assigned_irq->guest_irq; match->ack_notifier.gsi = assigned_irq->guest_irq; - match->ack_notifier.irq_acked = kvm_assigned_dev_ack_irq; -kvm_register_irq_ack_notifier (kvm, &match->ack_notifier); - r =kvm_request_irq_source_id (kvm); -if (r <0 ) - goto out_release; -else - match->irq_source_id = r;/* Even though this is PCI, we don't want to use shared * interrupts. Sharing host devices with guest-assigned devices
Signature · 1 line
1.6.0.3
PATCH 36/44 KVM: Separate update irq to a single function
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
Separate INTx enabling part to a independence function, so that we can add MSI enabling part easily.
Signed-off-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- virt/kvm/kvm_main.c | 68 ++++++++++++++++++++++++++++---------------------- 1 files changed, 38 insertions(+), 30 deletions(-)
virt/kvm/kvm_main.c +38 -30
@@ -176,6 +176,41 @@ void kvm_free_all_assigned_devices(struct kvm *kvm) } } +static int assigned_device_update_intx (struct kvm *kvm, +struct kvm_assigned_dev_kernel *adev, +struct kvm_assigned_irq *airq) +{ +if (adev->irq_requested) { + adev->guest_irq = airq->guest_irq; + adev->ack_notifier.gsi = airq->guest_irq; +return 0 ; + } + +if (irqchip_in_kernel (kvm)) { +if (!capable (CAP_SYS_RAWIO )) +return -EPERM ; + +if (airq->host_irq) + adev->host_irq = airq->host_irq; +else + adev->host_irq = adev->dev->irq; + adev->guest_irq = airq->guest_irq; + adev->ack_notifier.gsi = airq->guest_irq; + +/* Even though this is PCI, we don't want to use shared +* interrupts. Sharing host devices with guest-assigned devices +* on the same interrupt line is not a happy situation: there +* are going to be long delays in accepting, acking, etc. +*/ +if (request_irq (adev->host_irq, kvm_assigned_dev_intr, +0 ,"kvm_assigned_intx_device" , (void *)adev)) +return -EIO ; + } + + adev->irq_requested = true; +return 0 ; +} +static int kvm_vm_ioctl_assign_irq (struct kvm *kvm,struct kvm_assigned_irq *assigned_irq)
@@ -210,39 +245,12 @@ static int kvm_vm_ioctl_assign_irq(struct kvm *kvm,else match->irq_source_id = r; } - }else { - match->guest_irq = assigned_irq->guest_irq; - match->ack_notifier.gsi = assigned_irq->guest_irq; -mutex_unlock (&kvm->lock); -return 0 ; } -if (irqchip_in_kernel (kvm)) { -if (!capable (CAP_SYS_RAWIO )) { - r = -EPERM ; - goto out_release; - } - -if (assigned_irq->host_irq) - match->host_irq = assigned_irq->host_irq; -else - match->host_irq = match->dev->irq; - match->guest_irq = assigned_irq->guest_irq; - match->ack_notifier.gsi = assigned_irq->guest_irq; - -/* Even though this is PCI, we don't want to use shared -* interrupts. Sharing host devices with guest-assigned devices -* on the same interrupt line is not a happy situation: there -* are going to be long delays in accepting, acking, etc. -*/ -if (request_irq (match->host_irq, kvm_assigned_dev_intr,0 , -"kvm_assigned_device" , (void *)match)) { - r = -EIO ; - goto out_release; - } - } + r =assigned_device_update_intx (kvm, match, assigned_irq); +if (r) + goto out_release; - match->irq_requested = true;mutex_unlock (&kvm->lock);return r; out_release:
Signature · 1 line
1.6.0.3
PATCH 37/44 KVM: Replace irq_requested with more generic irq_requested_type
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
Separate guest irq type and host irq type, for we can support guest using INTx with host using MSI (but not opposite combination).
Signed-off-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- include/linux/kvm_host.h | 4 +++- virt/kvm/kvm_main.c | 9 +++++---- 2 files changed, 8 insertions(+), 5 deletions(-)
include/linux/kvm_host.h +3 -1
@@ -307,7 +307,9 @@ struct kvm_assigned_dev_kernel {int host_devfn;int host_irq;int guest_irq; -int irq_requested; +#define KVM_ASSIGNED_DEV_GUEST_INTX (1 << 0) +#define KVM_ASSIGNED_DEV_HOST_INTX (1 << 8) +unsigned long irq_requested_type;int irq_source_id;struct pci_dev *dev;struct kvm *kvm;
virt/kvm/kvm_main.c +5 -4
@@ -140,7 +140,7 @@ static void kvm_free_assigned_device(struct kvm *kvm,struct kvm_assigned_dev_kernel *assigned_dev) { -if (irqchip_in_kernel (kvm) && assigned_dev->irq_requested) +if (irqchip_in_kernel (kvm) && assigned_dev->irq_requested_type)free_irq (assigned_dev->host_irq, (void *)assigned_dev);kvm_unregister_irq_ack_notifier (&assigned_dev->ack_notifier);
@@ -180,7 +180,7 @@ static int assigned_device_update_intx(struct kvm *kvm, struct kvm_assigned_dev_kernel *adev, struct kvm_assigned_irq *airq) { -if (adev->irq_requested) { +if (adev->irq_requested_type &KVM_ASSIGNED_DEV_GUEST_INTX ) { adev->guest_irq = airq->guest_irq; adev->ack_notifier.gsi = airq->guest_irq;return 0 ;
@@ -207,7 +207,8 @@ static int assigned_device_update_intx(struct kvm *kvm,return -EIO ; } - adev->irq_requested = true; + adev->irq_requested_type =KVM_ASSIGNED_DEV_GUEST_INTX | +KVM_ASSIGNED_DEV_HOST_INTX ;return 0 ; }
@@ -227,7 +228,7 @@ static int kvm_vm_ioctl_assign_irq(struct kvm *kvm,return -EINVAL ; } -if (!match->irq_requested) { +if (!match->irq_requested_type) {INIT_WORK (&match->interrupt_work, kvm_assigned_dev_interrupt_work_handler);if (irqchip_in_kernel (kvm)) {
Signature · 1 line
1.6.0.3
PATCH 38/44 KVM: Clean up assigned_device_update_irq
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
Signed-off-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- virt/kvm/kvm_main.c | 10 ++++------ 1 files changed, 4 insertions(+), 6 deletions(-)
virt/kvm/kvm_main.c +4 -6
@@ -180,11 +180,11 @@ static int assigned_device_update_intx(struct kvm *kvm,struct kvm_assigned_dev_kernel *adev, struct kvm_assigned_irq *airq) { -if (adev->irq_requested_type &KVM_ASSIGNED_DEV_GUEST_INTX ) { - adev->guest_irq = airq->guest_irq; - adev->ack_notifier.gsi = airq->guest_irq; + adev->guest_irq = airq->guest_irq; + adev->ack_notifier.gsi = airq->guest_irq; + +if (adev->irq_requested_type &KVM_ASSIGNED_DEV_HOST_INTX )return 0 ; - }if (irqchip_in_kernel (kvm)) {if (!capable (CAP_SYS_RAWIO ))
@@ -194,8 +194,6 @@ static int assigned_device_update_intx(struct kvm *kvm, adev->host_irq = airq->host_irq; else adev->host_irq = adev->dev->irq; - adev->guest_irq = airq->guest_irq; - adev->ack_notifier.gsi = airq->guest_irq;/* Even though this is PCI, we don't want to use shared * interrupts. Sharing host devices with guest-assigned devices
Signature · 1 line
1.6.0.3
PATCH 39/44 KVM: Add fields for MSI device assignment
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
Prepared for kvm_arch_assigned_device_msi_dispatch().
Signed-off-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- include/linux/kvm.h | 7 +++++++ include/linux/kvm_host.h | 4 ++++ 2 files changed, 11 insertions(+), 0 deletions(-)
include/linux/kvm.h +7 -0
@@ -507,10 +507,17 @@ struct kvm_assigned_irq {__u32 guest_irq;__u32 flags;union { +struct { +__u32 addr_lo; +__u32 addr_hi; +__u32 data; + } guest_msi;__u32 reserved[12 ]; }; };#define KVM_DEV_ASSIGN_ENABLE_IOMMU (1 << 0) +#define KVM_DEV_IRQ_ASSIGN_ENABLE_MSI (1 << 0) +#endif
include/linux/kvm_host.h +4 -0
@@ -16,6 +16,7 @@#include <linux/mm.h> #include <linux/preempt.h> #include <linux/marker.h> +#include <linux/msi.h> #include <asm/signal.h> #include <linux/kvm.h>
@@ -307,8 +308,11 @@ struct kvm_assigned_dev_kernel {int host_devfn;int host_irq;int guest_irq; +struct msi_msg guest_msi;#define KVM_ASSIGNED_DEV_GUEST_INTX (1 << 0) +#define KVM_ASSIGNED_DEV_GUEST_MSI (1 << 1)#define KVM_ASSIGNED_DEV_HOST_INTX (1 << 8) +#define KVM_ASSIGNED_DEV_HOST_MSI (1 << 9)unsigned long irq_requested_type;int irq_source_id;struct pci_dev *dev;
Signature · 1 line
1.6.0.3
PATCH 40/44 KVM: Export ioapic_get_delivery_bitmask
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
It would be used for MSI in device assignment, for MSI dispatch.
Signed-off-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- virt/kvm/ioapic.c | 7 ++++--- virt/kvm/ioapic.h | 2 ++ 2 files changed, 6 insertions(+), 3 deletions(-)
virt/kvm/ioapic.c +4 -3
@@ -153,8 +153,8 @@ static void ioapic_inj_nmi(struct kvm_vcpu *vcpu)kvm_vcpu_kick (vcpu); } -static u32 ioapic_get_delivery_bitmask (struct kvm_ioapic *ioapic,u8 dest, -u8 dest_mode) +u32 kvm_ioapic_get_delivery_bitmask (struct kvm_ioapic *ioapic,u8 dest, +u8 dest_mode) {u32 mask =0 ;int i;
@@ -208,7 +208,8 @@ static int ioapic_deliver(struct kvm_ioapic *ioapic, int irq)"vector=%x trig_mode=%x\n" , dest, dest_mode, delivery_mode, vector, trig_mode); - deliver_bitmask =ioapic_get_delivery_bitmask (ioapic, dest, dest_mode); + deliver_bitmask =kvm_ioapic_get_delivery_bitmask (ioapic, dest, + dest_mode);if (!deliver_bitmask) {ioapic_debug ("no target on destination\n" );return 0 ;
virt/kvm/ioapic.h +2 -0
@@ -85,5 +85,7 @@ void kvm_ioapic_update_eoi(struct kvm *kvm, int vector, int trigger_mode);int kvm_ioapic_init (struct kvm *kvm);void kvm_ioapic_set_irq (struct kvm_ioapic *ioapic,int irq,int level);void kvm_ioapic_reset (struct kvm_ioapic *ioapic); +u32 kvm_ioapic_get_delivery_bitmask (struct kvm_ioapic *ioapic,u8 dest, +u8 dest_mode);#endif
Signature · 1 line
1.6.0.3
PATCH 41/44 KVM: Add assigned_device_msi_dispatch()
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
The function is used to dispatch MSI to lapic according to MSI message address and message data.
Signed-off-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- virt/kvm/kvm_main.c | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 55 insertions(+), 0 deletions(-)
virt/kvm/kvm_main.c +55 -0
@@ -47,6 +47,10 @@#include <asm/uaccess.h> #include <asm/pgtable.h> +#ifdef CONFIG_X86 +#include <asm/msidef.h> +#endif +#ifdef KVM_COALESCED_MMIO_PAGE_OFFSET #include "coalesced_mmio.h" #endif
@@ -78,6 +82,57 @@ static long kvm_vcpu_ioctl(struct file *file, unsigned int ioctl,bool kvm_rebooting;#ifdef KVM_CAP_DEVICE_ASSIGNMENT + +#ifdef CONFIG_X86 +static void assigned_device_msi_dispatch (struct kvm_assigned_dev_kernel *dev) +{ +int vcpu_id; +struct kvm_vcpu *vcpu; +struct kvm_ioapic *ioapic =ioapic_irqchip (dev->kvm); +int dest_id = (dev->guest_msi.address_lo &MSI_ADDR_DEST_ID_MASK ) + >>MSI_ADDR_DEST_ID_SHIFT ; +int vector = (dev->guest_msi.data &MSI_DATA_VECTOR_MASK ) + >>MSI_DATA_VECTOR_SHIFT ; +int dest_mode =test_bit (MSI_ADDR_DEST_MODE_SHIFT , + (unsigned long *)&dev->guest_msi.address_lo); +int trig_mode =test_bit (MSI_DATA_TRIGGER_SHIFT , + (unsigned long *)&dev->guest_msi.data); +int delivery_mode =test_bit (MSI_DATA_DELIVERY_MODE_SHIFT , + (unsigned long *)&dev->guest_msi.data); +u32 deliver_bitmask; + +BUG_ON (!ioapic); + + deliver_bitmask =kvm_ioapic_get_delivery_bitmask (ioapic, + dest_id, dest_mode); +/* IOAPIC delivery mode value is the same as MSI here */ +switch (delivery_mode) { +case IOAPIC_LOWEST_PRIORITY : + vcpu =kvm_get_lowest_prio_vcpu (ioapic->kvm, vector, + deliver_bitmask); +if (vcpu !=NULL ) +kvm_apic_set_irq (vcpu, vector, trig_mode); +else +printk (KERN_INFO "kvm: null lowest priority vcpu!\n" ); +break ; +case IOAPIC_FIXED : +for (vcpu_id =0 ; deliver_bitmask !=0 ; vcpu_id++) { +if (!(deliver_bitmask & (1 << vcpu_id))) +continue ; + deliver_bitmask &= ~(1 << vcpu_id); + vcpu = ioapic->kvm->vcpus[vcpu_id]; +if (vcpu) +kvm_apic_set_irq (vcpu, vector, trig_mode); + } +break ; +default : +printk (KERN_INFO "kvm: unsupported MSI delivery mode\n" ); + } +} +#else +static void assigned_device_msi_dispatch (struct kvm_assigned_dev_kernel *dev) {} +#endif +static struct kvm_assigned_dev_kernel *kvm_find_assigned_dev (struct list_head *head,int assigned_dev_id) {
Signature · 1 line
1.6.0.3
PATCH 42/44 KVM: Enable MSI for device assignment
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
We enable guest MSI and host MSI support in this patch. The userspace want to enable MSI should set KVM_DEV_IRQ_ASSIGN_ENABLE_MSI in the assigned_irq's flag. Function would return -ENOTTY if can't enable MSI, userspace shouldn't set MSI Enable bit when KVM_ASSIGN_IRQ return -ENOTTY with KVM_DEV_IRQ_ASSIGN_ENABLE_MSI.
Userspace can tell the support of MSI device from #ifdef KVM_CAP_DEVICE_MSI.
Signed-off-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- include/linux/kvm.h | 3 ++ virt/kvm/kvm_main.c | 81 +++++++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 78 insertions(+), 6 deletions(-)
include/linux/kvm.h +3 -0
@@ -392,6 +392,9 @@ struct kvm_trace_rec {#endif #define KVM_CAP_IOMMU 18#define KVM_CAP_NMI 19 +#if defined(CONFIG_X86 ) +#define KVM_CAP_DEVICE_MSI 20 +#endif /* * ioctls for VM fds
virt/kvm/kvm_main.c +75 -6
111 lines not on this page · raw message
Signature · 1 line
1.6.0.3
PATCH 43/44 KVM: MSI to INTx translate
In reply to Avi KivityFrom: Sheng Yang <[email protected]>
Now we use MSI as default one, and translate MSI to INTx when guest need INTx rather than MSI. For legacy device, we provide support for non-sharing host IRQ.
Provide a parameter msi2intx for this method. The value is true by default in x86 architecture.
We can't guarantee this mode can work on every device, but for most of us tested, it works. If your device encounter some trouble with this mode, you can try set msi2intx modules parameter to 0. If the device is OK with msi2intx=0, then please report it to KVM mailing list or me. We may prepare a blacklist for the device that can't work in this mode.
Signed-off-by Sheng Yang <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- virt/kvm/kvm_main.c | 70 +++++++++++++++++++++++++++++++++++++++----------- 1 files changed, 54 insertions(+), 16 deletions(-)
virt/kvm/kvm_main.c +54 -16
@@ -64,6 +64,9 @@MODULE_AUTHOR ("Qumranet" );MODULE_LICENSE ("GPL" ); +static int msi2intx =1 ; +module_param (msi2intx, bool,0 ); +DEFINE_SPINLOCK (kvm_lock);LIST_HEAD (vm_list);
@@ -250,7 +253,8 @@ static int assigned_device_update_intx(struct kvm *kvm,return 0 ;if (irqchip_in_kernel (kvm)) { -if (adev->irq_requested_type &KVM_ASSIGNED_DEV_HOST_MSI ) { +if (!msi2intx && + adev->irq_requested_type &KVM_ASSIGNED_DEV_HOST_MSI ) {free_irq (adev->host_irq, (void *)kvm);pci_disable_msi (adev->dev); }
@@ -285,21 +289,33 @@ static int assigned_device_update_msi(struct kvm *kvm, {int r; -/* x86 don't care upper address of guest msi message addr */ - adev->guest_msi.address_lo = airq->guest_msi.addr_lo; - adev->guest_msi.data = airq->guest_msi.data; - adev->ack_notifier.gsi =-1 ; +if (airq->flags &KVM_DEV_IRQ_ASSIGN_ENABLE_MSI ) { +/* x86 don't care upper address of guest msi message addr */ + adev->irq_requested_type |=KVM_ASSIGNED_DEV_GUEST_MSI ; + adev->irq_requested_type &= ~KVM_ASSIGNED_DEV_GUEST_INTX ; + adev->guest_msi.address_lo = airq->guest_msi.addr_lo; + adev->guest_msi.data = airq->guest_msi.data; + adev->ack_notifier.gsi =-1 ; + }else if (msi2intx) { + adev->irq_requested_type |=KVM_ASSIGNED_DEV_GUEST_INTX ; + adev->irq_requested_type &= ~KVM_ASSIGNED_DEV_GUEST_MSI ; + adev->guest_irq = airq->guest_irq; + adev->ack_notifier.gsi = airq->guest_irq; + }if (adev->irq_requested_type &KVM_ASSIGNED_DEV_HOST_MSI )return 0 ;if (irqchip_in_kernel (kvm)) { -if (adev->irq_requested_type &KVM_ASSIGNED_DEV_HOST_INTX ) -free_irq (adev->host_irq, (void *)adev); - - r =pci_enable_msi (adev->dev); -if (r) -return r; +if (!msi2intx) { +if (adev->irq_requested_type & +KVM_ASSIGNED_DEV_HOST_INTX ) +free_irq (adev->host_irq, (void *)adev); + + r =pci_enable_msi (adev->dev); +if (r) +return r; + } adev->host_irq = adev->dev->irq; if (request_irq(adev->host_irq, kvm_assigned_dev_intr, 0,
@@ -307,8 +323,10 @@ static int assigned_device_update_msi(struct kvm *kvm, return -EIO; } - adev->irq_requested_type =KVM_ASSIGNED_DEV_GUEST_MSI | -KVM_ASSIGNED_DEV_HOST_MSI ; +if (!msi2intx) + adev->irq_requested_type =KVM_ASSIGNED_DEV_GUEST_MSI ; + + adev->irq_requested_type |=KVM_ASSIGNED_DEV_HOST_MSI ;return 0 ; }#endif
@@ -346,10 +364,19 @@ static int kvm_vm_ioctl_assign_irq(struct kvm *kvm, goto out_release; else match->irq_source_id = r; + +#ifdef CONFIG_X86 +/* Determine host device irq type, we can know the +* result from dev->msi_enabled */ +if (msi2intx) +pci_enable_msi (match->dev); +#endif } } -if (assigned_irq->flags &KVM_DEV_IRQ_ASSIGN_ENABLE_MSI ) { +if ((!msi2intx && + (assigned_irq->flags &KVM_DEV_IRQ_ASSIGN_ENABLE_MSI )) || + (msi2intx && match->dev->msi_enabled)) {#ifdef CONFIG_X86 r =assigned_device_update_msi (kvm, match, assigned_irq);if (r) {
@@ -362,8 +389,16 @@ static int kvm_vm_ioctl_assign_irq(struct kvm *kvm,#endif }else if (assigned_irq->host_irq ==0 && match->dev->irq ==0 ) {/* Host device IRQ 0 means don't support INTx */ -printk (KERN_WARNING "kvm: wait device to enable MSI!\n" ); - r =0 ; +if (!msi2intx) { +printk (KERN_WARNING +"kvm: wait device to enable MSI!\n" ); + r =0 ; + }else { +printk (KERN_WARNING +"kvm: failed to enable MSI device!\n" ); + r = -ENOTTY ; + goto out_release; + } }else {/* Non-sharing INTx mode */ r =assigned_device_update_intx (kvm, match, assigned_irq);
@@ -2209,6 +2244,9 @@ int kvm_init(void *opaque, unsigned int vcpu_size, kvm_preempt_ops.sched_in = kvm_sched_in; kvm_preempt_ops.sched_out = kvm_sched_out; +#ifndef CONFIG_X86 + msi2intx =0 ; +#endif return 0 ;
Signature · 1 line
1.6.0.3
PATCH 44/44 KVM: MMU: optimize set_spte for page sync
In reply to Avi KivityFrom: Marcelo Tosatti <[email protected]>
The write protect verification in set_spte is unnecessary for page sync.
Its guaranteed that, if the unsync spte was writable, the target page does not have a write protected shadow (if it had, the spte would have been write protected under mmu_lock by rmap_write_protect before).
Same reasoning applies to mark_page_dirty: the gfn has been marked as dirty via the pagefault path.
The cost of hash table and memslot lookups are quite significant if the workload is pagetable write intensive resulting in increased mmu_lock contention.
Signed-off-by Marcelo Tosatti <[email protected]>
Signed-off-by Avi Kivity <[email protected]>
--- arch/x86/kvm/mmu.c | 9 +++++++++ 1 files changed, 9 insertions(+), 0 deletions(-)
arch/x86/kvm/mmu.c +9 -0
@@ -1593,6 +1593,15 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *shadow_pte, spte |=PT_WRITABLE_MASK ; +/* +* Optimization: for pte sync, if spte was writable the hash +* lookup is unnecessary (and expensive). Write protection +* is responsibility of mmu_get_page / kvm_sync_page. +* Same reasoning can be applied to dirty page accounting. +*/ +if (!can_unsync &&is_writeble_pte (*shadow_pte)) + goto set_pte; +if (mmu_need_write_protect (vcpu, gfn, can_unsync)) {pgprintk ("%s: found shadow page for %lx, marking ro\n" , __func__, gfn);
Signature · 1 line
1.6.0.3