Nested VMX support v4
Avi,
We have addressed all of the comments, please apply.
The following patches implement nested VMX support. The patches enable a guest to use the VMX APIs in order to run its own nested guest (i.e., enable running other hypervisors which use VMX under KVM). The current patches support running Linux under a nested KVM using shadow page table (with bypass_guest_pf disabled). Reworking EPT support to mesh cleanly with the current shadow paging design per Avi's comments is a work-in-progress.
The current patches support multiple nested hypervisors, which can run multiple guests. Only 64-bit nested hypervisors are supported. SMP is supported. Additional patches for running Windows under nested KVM, and Linux under nested VMware server, are currently running in the lab, and will be sent as a follow-on patchset.
This patches were written by:
Orit Wasserman, oritw <at> il.ibm.com
Ben-Ami Yassor, benami <at> il.ibm.com
Abel Gordon, abelg <at> il.ibm.com
Muli Ben-Yehuda, muli <at> il.ibm.com
With contributions by:
Anthony Liguori, aliguori <at> us.ibm.com
Mike Day, mdday <at> us.ibm.com
This work was inspired by the nested SVM support by Alexander Graf and Joerg Roedel.
Changes since v3:
Added support for 32-bit nested guests Added support for multiple nested guests Added support for multiple nested hypervisors Implemented VMX instruction decoding Implemented CR0.TS handling for nested
PATCH 1/7 Nested VMX patch 1 implements vmon and vmoff
From: Orit Wasserman <[email protected]>
--- arch/x86/kvm/svm.c | 3 - arch/x86/kvm/vmx.c | 265 +++++++++++++++++++++++++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 11 ++- arch/x86/kvm/x86.h | 2 + 4 files changed, 274 insertions(+), 7 deletions(-)
arch/x86/kvm/svm.c +0 -3
@@ -121,9 +121,6 @@ static int npt = 1;module_param (npt, int,S_IRUGO ); -static int nested =1 ; -module_param (nested, int,S_IRUGO ); -static void svm_flush_tlb (struct kvm_vcpu *vcpu);static void svm_complete_interrupts (struct vcpu_svm *svm);
arch/x86/kvm/vmx.c +263 -2
343 lines not on this page · raw message
arch/x86/kvm/x86.c +9 -2
@@ -88,6 +88,10 @@ EXPORT_SYMBOL_GPL(kvm_x86_ops);int ignore_msrs =0 ;module_param_named (ignore_msrs, ignore_msrs, bool,S_IRUGO |S_IWUSR ); +int nested =1 ; +EXPORT_SYMBOL_GPL (nested); +module_param (nested, int,S_IRUGO ); +#define KVM_NR_SHARED_MSRS 16struct kvm_shared_msrs_global {
@@ -505,7 +509,7 @@ void kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)return ; } -if (cr4 &X86_CR4_VMXE ) { +if (cr4 &X86_CR4_VMXE && !nested) {printk (KERN_DEBUG "set_cr4: #GP, setting VMXE\n" );kvm_inject_gp (vcpu,0 );return ;
@@ -615,7 +619,10 @@ static u32 msrs_to_save[] = {#ifdef CONFIG_X86_64 MSR_CSTAR ,MSR_KERNEL_GS_BASE ,MSR_SYSCALL_MASK ,MSR_LSTAR , #endif -MSR_IA32_TSC ,MSR_IA32_PERF_STATUS ,MSR_IA32_CR_PAT ,MSR_VM_HSAVE_PA +MSR_IA32_TSC ,MSR_IA32_PERF_STATUS ,MSR_IA32_CR_PAT ,MSR_VM_HSAVE_PA , +MSR_IA32_FEATURE_CONTROL ,MSR_IA32_VMX_BASIC ,MSR_IA32_VMX_PINBASED_CTLS , +MSR_IA32_VMX_PROCBASED_CTLS ,MSR_IA32_VMX_EXIT_CTLS ,MSR_IA32_VMX_ENTRY_CTLS , +MSR_IA32_VMX_PROCBASED_CTLS2 ,MSR_IA32_VMX_EPT_VPID_CAP ,MSR_IA32_FEATURE_CONTROL };static unsigned num_msrs_to_save;
arch/x86/kvm/x86.h +2 -0
@@ -35,4 +35,6 @@ static inline bool kvm_exception_is_soft(unsigned int nr)struct kvm_cpuid_entry2 *kvm_find_cpuid_entry (struct kvm_vcpu *vcpu,u32 function,u32 index); +extern int nested; +#endif
Signature · 1 line
1.6.0.4
PATCH 2/7 Nested VMX patch 2 implements vmclear
From: Orit Wasserman <[email protected]>
--- arch/x86/kvm/vmx.c | 235 +++++++++++++++++++++++++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 5 +- arch/x86/kvm/x86.h | 3 + 3 files changed, 240 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx.c +234 -1
@@ -93,13 +93,39 @@ struct shared_msr_entry { };struct __attribute__ ((__packed__ ))level_state { +/* Has the level1 guest done vmclear? */ +bool vmclear; +}; + +/* +* This structure is mapped to guest memory. +* It is packed in order to preseve the binary content +* after live migration. +* If there are changed in the content or layout the revision_id must be updated. +*/ +struct __attribute__ ((__packed__ ))nested_vmcs_page { +u32 revision_id; +u32 abort; +struct level_state l2_state; +}; + +struct nested_vmcs_list { +struct list_head list; +gpa_t vmcs_addr; +struct vmcs *l2_vmcs; };struct nested_vmx {/* Has the level1 guest done vmxon? */ bool vmxon; +/* What is the location of the current vmcs l1 keeps for l2 */ +gpa_t current_vmptr;/* Level 1 state for switching to level 2 and back */ struct level_state *l1_state; +/* list of vmcs for each l2 guest created by l1 */ +struct list_head l2_vmcs_list; +/* l2 page corresponding to the current vmcs set by l1 */ +struct nested_vmcs_page *current_l2_page; };struct vcpu_vmx {
@@ -156,6 +182,76 @@ static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) return container_of(vcpu, struct vcpu_vmx, vcpu); } +static struct page *nested_get_page (struct kvm_vcpu *vcpu, +u64 vmcs_addr) +{ +struct page *vmcs_page =NULL ; + +down_read (¤t->mm->mmap_sem); + vmcs_page =gfn_to_page (vcpu->kvm, vmcs_addr >>PAGE_SHIFT ); +up_read (¤t->mm->mmap_sem); + +if (is_error_page (vmcs_page)) { +printk (KERN_ERR "%s error allocating page 0x%llx\n" , + __func__, vmcs_addr); +kvm_release_page_clean (vmcs_page); +return NULL ; + } + +return vmcs_page; + +} + +static int nested_map_current (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct page *vmcs_page = +nested_get_page (vcpu, vmx->nested.current_vmptr); +struct nested_vmcs_page *mapped_page; + +if (vmcs_page ==NULL ) { +printk (KERN_INFO "%s: failure in nested_get_page\n" , __func__); +return 0 ; + } + +if (vmx->nested.current_l2_page) { +printk (KERN_INFO "%s: shadow vmcs already mapped\n" , __func__); +WARN_ON (1 ); +return 0 ; + } + + mapped_page =kmap_atomic (vmcs_page,KM_USER0 ); + +if (!mapped_page) { +printk (KERN_INFO "%s: error in kmap_atomic\n" , __func__); +return 0 ; + } + + vmx->nested.current_l2_page = mapped_page; + +return 1 ; +} + +static void nested_unmap_current (struct kvm_vcpu *vcpu) +{ +struct page *page; +struct vcpu_vmx *vmx =to_vmx (vcpu); + +if (!vmx->nested.current_l2_page) { +printk (KERN_INFO "Shadow vmcs already unmapped\n" ); +WARN_ON (1 ); +return ; + } + + page =kmap_atomic_to_page (vmx->nested.current_l2_page); + +kunmap_atomic (vmx->nested.current_l2_page,KM_USER0 ); + +kvm_release_page_dirty (page); + + vmx->nested.current_l2_page =NULL ; +} +static int init_rmode (struct kvm *kvm);static u64 construct_eptp (unsigned long root_hpa);
@@ -1144,6 +1240,35 @@ static int nested_vmx_set_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data)return 0 ; } +static int read_guest_vmcs_gpa (struct kvm_vcpu *vcpu,gva_t gva,u64 *gentry) +{ +int r =0 ; +uint size; + + *gentry =0 ; + +if (is_long_mode (vcpu)) + size =sizeof (u64); +else + size =sizeof (u32); + + r =kvm_read_guest_virt (gva, gentry, + size, vcpu); +if (r) { +printk (KERN_ERR "%s cannot read guest vmcs addr %lx : %d\n" , + __func__, vcpu->arch.regs[VCPU_REGS_RAX ], r); +return r; + } + +if (!IS_ALIGNED (*gentry,PAGE_SIZE )) { +printk (KERN_DEBUG "%s addr %llx not aligned\n" , + __func__, *gentry); +return 1 ; + } + +return 0 ; +} +/* * Writes msr value into into the appropriate "register". * Returns 0 on success, non-0 otherwise.
@@ -1316,6 +1441,7 @@ static int create_l1_state(struct kvm_vcpu *vcpu) }else return 0 ; +INIT_LIST_HEAD (&(vmx->nested.l2_vmcs_list));return 0 ; }
@@ -1488,15 +1614,35 @@ static void free_vmcs(struct vmcs *vmcs)free_pages ((unsigned long )vmcs, vmcs_config.order); } +static void nested_free_current_vmcs (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct nested_vmcs_list *list_item, *n; + +list_for_each_entry_safe (list_item, n, &vmx->nested.l2_vmcs_list, list) +if (list_item->vmcs_addr == vmx->nested.current_vmptr) { +free_vmcs (list_item->l2_vmcs); +list_del (&(list_item->list)); +return ; + } +} +static void free_l1_state (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +struct nested_vmcs_list *list_item, *n;if (!vmx->nested.l1_state)return ;kfree (vmx->nested.l1_state); vmx->nested.l1_state =NULL ; + +list_for_each_entry_safe (list_item, n, &vmx->nested.l2_vmcs_list, + list) { +free_vmcs (list_item->l2_vmcs); +list_del (&(list_item->list)); + } }
@@ -3352,6 +3498,93 @@ static int handle_vmx_insn(struct kvm_vcpu *vcpu)return 1 ; } +static void clear_rflags_cf_zf (struct kvm_vcpu *vcpu) +{ +unsigned long rflags; + rflags =vmx_get_rflags (vcpu); + rflags &= ~(X86_EFLAGS_CF |X86_EFLAGS_ZF ); +vmx_set_rflags (vcpu, rflags); +} + +/* +* Decode the memory address (operand) of a vmx instruction according to Table 23-12/23-11 +* For additional information regarding offset calculation see 3.7.5 +*/ +static gva_t get_vmx_mem_address (struct kvm_vcpu *vcpu, +unsigned long exit_qualification, +u32 vmx_instruction_info) +{ +int scaling = vmx_instruction_info &3 ;/* bits 0:1 scaling */ +int addr_size = (vmx_instruction_info >>7 ) &7 ;/* bits 7:9 address size, 0=16bit, 1=32bit, 2=64bit */ +bool is_reg = vmx_instruction_info & (1u <<10 );/* bit 10 1=register operand, 0= memory */ +int seg_reg = (vmx_instruction_info >>15 ) &7 ;/* bits 15:17 segment register */ +int index_reg = (vmx_instruction_info >>18 ) &0xf ;/* bits 18:21 index register */ +bool index_is_valid = !(vmx_instruction_info & (1u <<22 ));/* bit 22 index register validity, 0=valid, 1=invalid */ +int base_reg = (vmx_instruction_info >>23 ) &0xf ;/* bits 23:26 index register */ +bool base_is_valid = !(vmx_instruction_info & (1u <<27 ));/* bit 27 base register validity, 0=valid, 1=invalid */ +gva_t addr; + +if (is_reg) +return 0 ; + +switch (addr_size) { +case 1 : + exit_qualification &=0xffffffff ;/* 32 high bits are undefied according to the spec, page 23-7 */ +break ; +case 2 : +break ; +default : +return 0 ; + } + +/* Addr = segment_base + offset */ +/* offfset = Base + [Index * Scale] + Displacement, see Figure 3-11 */ + addr =vmx_get_segment_base (vcpu, seg_reg); +if (base_is_valid) + addr +=kvm_register_read (vcpu, base_reg); +if (index_is_valid) + addr +=kvm_register_read (vcpu, index_reg)*scaling; + addr += exit_qualification;/* exit qualification holds the displacement, spec page 23-7 */ + +return addr; +} + +static int handle_vmclear (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct level_state *l2_state; +gpa_t guest_vmcs_addr; +unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); +u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO ); +gva_t vmcs_gva; + +if (!nested_vmx_check_permission (vcpu)) +return 1 ; + + vmcs_gva =get_vmx_mem_address (vcpu, exit_qualification, + vmx_instruction_info); + +if (read_guest_vmcs_gpa (vcpu, vmcs_gva, &guest_vmcs_addr)) +return 1 ; + + vmx->nested.current_vmptr = guest_vmcs_addr; +if (!nested_map_current (vcpu)) +return 1 ; + + l2_state = &(to_vmx (vcpu)->nested.current_l2_page->l2_state); + l2_state->vmclear =1 ; +nested_free_current_vmcs (vcpu); + + vmx->nested.current_vmptr =-1ull ; + +nested_unmap_current (vcpu); + +skip_emulated_instruction (vcpu); +clear_rflags_cf_zf (vcpu); + +return 1 ; +} +static int handle_vmoff (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);
@@ -3695,7 +3928,7 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { [EXIT_REASON_HLT] = handle_halt, [EXIT_REASON_INVLPG] = handle_invlpg, [EXIT_REASON_VMCALL] = handle_vmcall, - [EXIT_REASON_VMCLEAR] = handle_vmx_insn, + [EXIT_REASON_VMCLEAR] = handle_vmclear, [EXIT_REASON_VMLAUNCH] = handle_vmx_insn, [EXIT_REASON_VMPTRLD] = handle_vmx_insn, [EXIT_REASON_VMPTRST] = handle_vmx_insn,
arch/x86/kvm/x86.c +3 -2
@@ -2773,8 +2773,8 @@ static int vcpu_mmio_read(struct kvm_vcpu *vcpu, gpa_t addr, int len, void *v)return kvm_io_bus_read (&vcpu->kvm->mmio_bus, addr, len, v); } -static int kvm_read_guest_virt (gva_t addr,void *val,unsigned int bytes, -struct kvm_vcpu *vcpu) +int kvm_read_guest_virt (gva_t addr,void *val,unsigned int bytes, +struct kvm_vcpu *vcpu) {void *data = val;int r =X86EMUL_CONTINUE ;
@@ -2802,6 +2802,7 @@ static int kvm_read_guest_virt(gva_t addr, void *val, unsigned int bytes, out:return r; } +EXPORT_SYMBOL_GPL (kvm_read_guest_virt);static int kvm_write_guest_virt (gva_t addr,void *val,unsigned int bytes,struct kvm_vcpu *vcpu)
arch/x86/kvm/x86.h +3 -0
@@ -35,6 +35,9 @@ static inline bool kvm_exception_is_soft(unsigned int nr)struct kvm_cpuid_entry2 *kvm_find_cpuid_entry (struct kvm_vcpu *vcpu,u32 function,u32 index); +int kvm_read_guest_virt (gva_t addr,void *val,unsigned int bytes, +struct kvm_vcpu *vcpu); +extern int nested;#endif
Signature · 1 line
1.6.0.4
PATCH 3/7 Nested VMX patch 3 implements vmptrld and vmptrst
From: Orit Wasserman <[email protected]>
--- arch/x86/kvm/vmx.c | 292 ++++++++++++++++++++++++++++++++++++++++++++++++++-- arch/x86/kvm/x86.c | 6 +- arch/x86/kvm/x86.h | 3 + 3 files changed, 289 insertions(+), 12 deletions(-)
arch/x86/kvm/vmx.c +283 -9
@@ -92,9 +92,142 @@ struct shared_msr_entry {u64 mask; }; +struct __attribute__ ((__packed__ ))shadow_vmcs { +u16 virtual_processor_id; +u16 guest_es_selector; +u16 guest_cs_selector; +u16 guest_ss_selector; +u16 guest_ds_selector; +u16 guest_fs_selector; +u16 guest_gs_selector; +u16 guest_ldtr_selector; +u16 guest_tr_selector; +u16 host_es_selector; +u16 host_cs_selector; +u16 host_ss_selector; +u16 host_ds_selector; +u16 host_fs_selector; +u16 host_gs_selector; +u16 host_tr_selector; +u64 io_bitmap_a; +u64 io_bitmap_b; +u64 msr_bitmap; +u64 vm_exit_msr_store_addr; +u64 vm_exit_msr_load_addr; +u64 vm_entry_msr_load_addr; +u64 tsc_offset; +u64 virtual_apic_page_addr; +u64 apic_access_addr; +u64 ept_pointer; +u64 guest_physical_address; +u64 vmcs_link_pointer; +u64 guest_ia32_debugctl; +u64 guest_ia32_pat; +u64 guest_pdptr0; +u64 guest_pdptr1; +u64 guest_pdptr2; +u64 guest_pdptr3; +u64 host_ia32_pat; +u32 pin_based_vm_exec_control; +u32 cpu_based_vm_exec_control; +u32 exception_bitmap; +u32 page_fault_error_code_mask; +u32 page_fault_error_code_match; +u32 cr3_target_count; +u32 vm_exit_controls; +u32 vm_exit_msr_store_count; +u32 vm_exit_msr_load_count; +u32 vm_entry_controls; +u32 vm_entry_msr_load_count; +u32 vm_entry_intr_info_field; +u32 vm_entry_exception_error_code; +u32 vm_entry_instruction_len; +u32 tpr_threshold; +u32 secondary_vm_exec_control; +u32 vm_instruction_error; +u32 vm_exit_reason; +u32 vm_exit_intr_info; +u32 vm_exit_intr_error_code; +u32 idt_vectoring_info_field; +u32 idt_vectoring_error_code; +u32 vm_exit_instruction_len; +u32 vmx_instruction_info; +u32 guest_es_limit; +u32 guest_cs_limit; +u32 guest_ss_limit; +u32 guest_ds_limit; +u32 guest_fs_limit; +u32 guest_gs_limit; +u32 guest_ldtr_limit; +u32 guest_tr_limit; +u32 guest_gdtr_limit; +u32 guest_idtr_limit; +u32 guest_es_ar_bytes; +u32 guest_cs_ar_bytes; +u32 guest_ss_ar_bytes; +u32 guest_ds_ar_bytes; +u32 guest_fs_ar_bytes; +u32 guest_gs_ar_bytes; +u32 guest_ldtr_ar_bytes; +u32 guest_tr_ar_bytes; +u32 guest_interruptibility_info; +u32 guest_activity_state; +u32 guest_sysenter_cs; +u32 host_ia32_sysenter_cs; +unsigned long cr0_guest_host_mask; +unsigned long cr4_guest_host_mask; +unsigned long cr0_read_shadow; +unsigned long cr4_read_shadow; +unsigned long cr3_target_value0; +unsigned long cr3_target_value1; +unsigned long cr3_target_value2; +unsigned long cr3_target_value3; +unsigned long exit_qualification; +unsigned long guest_linear_address; +unsigned long guest_cr0; +unsigned long guest_cr3; +unsigned long guest_cr4; +unsigned long guest_es_base; +unsigned long guest_cs_base; +unsigned long guest_ss_base; +unsigned long guest_ds_base; +unsigned long guest_fs_base; +unsigned long guest_gs_base; +unsigned long guest_ldtr_base; +unsigned long guest_tr_base; +unsigned long guest_gdtr_base; +unsigned long guest_idtr_base; +unsigned long guest_dr7; +unsigned long guest_rsp; +unsigned long guest_rip; +unsigned long guest_rflags; +unsigned long guest_pending_dbg_exceptions; +unsigned long guest_sysenter_esp; +unsigned long guest_sysenter_eip; +unsigned long host_cr0; +unsigned long host_cr3; +unsigned long host_cr4; +unsigned long host_fs_base; +unsigned long host_gs_base; +unsigned long host_tr_base; +unsigned long host_gdtr_base; +unsigned long host_idtr_base; +unsigned long host_ia32_sysenter_esp; +unsigned long host_ia32_sysenter_eip; +unsigned long host_rsp; +unsigned long host_rip; +}; + +struct __attribute__ ((__packed__ ))level_state {/* Has the level1 guest done vmclear? */ bool vmclear; + +u64 io_bitmap_a; +u64 io_bitmap_b; +u64 msr_bitmap; + +bool first_launch; };/*
@@ -122,6 +255,8 @@ struct nested_vmx {gpa_t current_vmptr;/* Level 1 state for switching to level 2 and back */ struct level_state *l1_state; +/* Level 1 shadow vmcs for switching to level 2 and back */ +struct shadow_vmcs *l1_shadow_vmcs;/* list of vmcs for each l2 guest created by l1 */ struct list_head l2_vmcs_list;/* l2 page corresponding to the current vmcs set by l1 */
@@ -187,10 +322,7 @@ static struct page *nested_get_page(struct kvm_vcpu *vcpu, {struct page *vmcs_page =NULL ; -down_read (¤t->mm->mmap_sem); vmcs_page =gfn_to_page (vcpu->kvm, vmcs_addr >>PAGE_SHIFT ); -up_read (¤t->mm->mmap_sem); -if (is_error_page (vmcs_page)) {printk (KERN_ERR "%s error allocating page 0x%llx\n" , __func__, vmcs_addr);
@@ -832,13 +964,14 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu)if (per_cpu (current_vmcs, cpu) != vmx->vmcs) {u8 error; -per_cpu (current_vmcs, cpu) = vmx->vmcs; + asm volatile (__ex(ASM_VMX_VMPTRLD_RAX) "; setna %0" :"=g" (error) :"a" (&phys_addr),"m" (phys_addr) :"cc" ); +if (error) -printk (KERN_ERR "kvm: vmptrld %p/%llx fail\n" , +printk (KERN_ERR "kvm: vmptrld %p/%llx failed\n" , vmx->vmcs, phys_addr); }
@@ -1240,6 +1373,7 @@ static int nested_vmx_set_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data)return 0 ; } +static int read_guest_vmcs_gpa (struct kvm_vcpu *vcpu,gva_t gva,u64 *gentry) {int r =0 ;
@@ -1430,6 +1564,18 @@ static struct level_state *create_state(void)return state; } +static struct vmcs *nested_get_current_vmcs (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct nested_vmcs_list *list_item, *n; + +list_for_each_entry_safe (list_item, n, &vmx->nested.l2_vmcs_list, list) +if (list_item->vmcs_addr == vmx->nested.current_vmptr) +return list_item->l2_vmcs; + +return NULL ; +} +static int create_l1_state (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);
@@ -1441,10 +1587,75 @@ static int create_l1_state(struct kvm_vcpu *vcpu) }else return 0 ; + vmx->nested.l1_shadow_vmcs =kzalloc (PAGE_SIZE ,GFP_KERNEL ); +if (!vmx->nested.l1_shadow_vmcs) { +printk (KERN_INFO "%s could not allocate memory for l1_shadow vmcs\n" , + __func__); +kfree (vmx->nested.l1_state); +return -ENOMEM ; + } +INIT_LIST_HEAD (&(vmx->nested.l2_vmcs_list));return 0 ; } +static struct vmcs *alloc_vmcs (void ); +int create_l2_state (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct vmcs *l2_vmcs; + +if (!nested_map_current (vcpu)) { +printk (KERN_ERR "%s error mapping level 2 page" , __func__); +return -ENOMEM ; + } + + l2_vmcs =nested_get_current_vmcs (vcpu); +if (!l2_vmcs) { +struct nested_vmcs_list *new_l2_guest = + (struct nested_vmcs_list *) +kmalloc (sizeof (struct nested_vmcs_list ),GFP_KERNEL ); + +if (!new_l2_guest) { +printk (KERN_ERR "%s error could not allocate memory for a new l2 guest list item" , + __func__); +nested_unmap_current (vcpu); +return -ENOMEM ; + } + + l2_vmcs =alloc_vmcs (); + +if (!l2_vmcs) { +printk (KERN_ERR "%s error could not allocate memory for l2_vmcs" , + __func__); +kfree (new_l2_guest); +nested_unmap_current (vcpu); +return -ENOMEM ; + } + + new_l2_guest->vmcs_addr = vmx->nested.current_vmptr; + new_l2_guest->l2_vmcs = l2_vmcs; +list_add (&(new_l2_guest->list), &(vmx->nested.l2_vmcs_list)); + } + +if (cpu_has_vmx_msr_bitmap ()) + vmx->nested.current_l2_page->l2_state.msr_bitmap = +vmcs_read64 (MSR_BITMAP ); +else + vmx->nested.current_l2_page->l2_state.msr_bitmap =0 ; + + vmx->nested.current_l2_page->l2_state.io_bitmap_a = +vmcs_read64 (IO_BITMAP_A ); + vmx->nested.current_l2_page->l2_state.io_bitmap_b = +vmcs_read64 (IO_BITMAP_B ); + + vmx->nested.current_l2_page->l2_state.first_launch = true; + +nested_unmap_current (vcpu); + +return 0 ; +} +/* Just like cpu_vmxoff(), but with the __kvm_handle_fault_on_reboot() * tricks. */
@@ -1623,6 +1834,7 @@ static void nested_free_current_vmcs(struct kvm_vcpu *vcpu)if (list_item->vmcs_addr == vmx->nested.current_vmptr) {free_vmcs (list_item->l2_vmcs);list_del (&(list_item->list)); +kfree (list_item);return ; } }
@@ -1637,11 +1849,14 @@ static void free_l1_state(struct kvm_vcpu *vcpu)kfree (vmx->nested.l1_state); vmx->nested.l1_state =NULL ; +kfree (vmx->nested.l1_shadow_vmcs); + vmx->nested.l1_shadow_vmcs =NULL ;list_for_each_entry_safe (list_item, n, &vmx->nested.l2_vmcs_list, list) {free_vmcs (list_item->l2_vmcs);list_del (&(list_item->list)); +kfree (list_item); } }
@@ -3604,6 +3819,7 @@ static int handle_vmon(struct kvm_vcpu *vcpu) {struct kvm_segment cs;struct vcpu_vmx *vmx =to_vmx (vcpu); +int r =0 ;if (!nested) {pr_debug ("%s: nested vmx not enabled\n" , __func__);
@@ -3633,8 +3849,9 @@ static int handle_vmon(struct kvm_vcpu *vcpu)return 1 ; } -if (create_l1_state (vcpu)) { -printk (KERN_ERR "%s create_l1_state failed\n" , __func__); + r =create_l1_state (vcpu); +if (r) { +printk (KERN_ERR "%s create_l1_state failed: %d\n" , __func__, r);kvm_queue_exception (vcpu,UD_VECTOR );return 1 ; }
@@ -3645,6 +3862,63 @@ static int handle_vmon(struct kvm_vcpu *vcpu)return 1 ; } +static int handle_vmptrld (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +u64 guest_vmcs_addr; +gva_t vmcs_gva; +unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); +u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO ); +int r =0 ; + +if (!nested_vmx_check_permission (vcpu)) +return 1 ; + + vmcs_gva =get_vmx_mem_address (vcpu, exit_qualification, + vmx_instruction_info); + +if (read_guest_vmcs_gpa (vcpu, vmcs_gva, &guest_vmcs_addr)) +return 1 ; + +if (vmx->nested.current_vmptr != guest_vmcs_addr) { + vmx->nested.current_vmptr = guest_vmcs_addr; + r =create_l2_state (vcpu); +if (r) { +printk (KERN_ERR "%s create_l2_state failed: %d\n" , + __func__, r); +return 1 ; + } + } + +clear_rflags_cf_zf (vcpu); +skip_emulated_instruction (vcpu); +return 1 ; +} + +static int handle_vmptrst (struct kvm_vcpu *vcpu) +{ +int r =0 ; +unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); +u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO ); +gva_t vmcs_gva; + +if (!nested_vmx_check_permission (vcpu)) +return 1 ; + vmcs_gva =get_vmx_mem_address (vcpu, exit_qualification, + vmx_instruction_info); + + r =kvm_write_guest_virt (vmcs_gva, + (void *)&to_vmx (vcpu)->nested.current_vmptr, +sizeof (u64), vcpu); +if (r) { +printk (KERN_INFO "%s failed to write vmptr\n" , __func__); +return 1 ; + } +clear_rflags_cf_zf (vcpu); +skip_emulated_instruction (vcpu); +return 1 ; +} +static int handle_invlpg (struct kvm_vcpu *vcpu) {unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION );
@@ -3930,8 +4204,8 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { [EXIT_REASON_VMCALL] = handle_vmcall, [EXIT_REASON_VMCLEAR] = handle_vmclear, [EXIT_REASON_VMLAUNCH] = handle_vmx_insn, - [EXIT_REASON_VMPTRLD] = handle_vmx_insn, - [EXIT_REASON_VMPTRST] = handle_vmx_insn, + [EXIT_REASON_VMPTRLD] = handle_vmptrld, + [EXIT_REASON_VMPTRST] = handle_vmptrst, [EXIT_REASON_VMREAD] = handle_vmx_insn, [EXIT_REASON_VMRESUME] = handle_vmx_insn, [EXIT_REASON_VMWRITE] = handle_vmx_insn,
arch/x86/kvm/x86.c +3 -3
@@ -2804,8 +2804,8 @@ out: }EXPORT_SYMBOL_GPL (kvm_read_guest_virt); -static int kvm_write_guest_virt (gva_t addr,void *val,unsigned int bytes, -struct kvm_vcpu *vcpu) +int kvm_write_guest_virt (gva_t addr,void *val,unsigned int bytes, +struct kvm_vcpu *vcpu) {void *data = val;int r =X86EMUL_CONTINUE ;
@@ -2833,7 +2833,7 @@ static int kvm_write_guest_virt(gva_t addr, void *val, unsigned int bytes, out:return r; } - +EXPORT_SYMBOL_GPL (kvm_write_guest_virt); static int emulator_read_emulated(unsigned long addr, void *val,
arch/x86/kvm/x86.h +3 -0
@@ -38,6 +38,9 @@ struct kvm_cpuid_entry2 *kvm_find_cpuid_entry(struct kvm_vcpu *vcpu,int kvm_read_guest_virt (gva_t addr,void *val,unsigned int bytes,struct kvm_vcpu *vcpu); +int kvm_write_guest_virt (gva_t addr,void *val,unsigned int bytes, +struct kvm_vcpu *vcpu); +extern int nested;#endif
Signature · 1 line
1.6.0.4
PATCH 4/7 Nested VMX patch 4 implements vmread and vmwrite
From: Orit Wasserman <[email protected]>
--- arch/x86/kvm/vmx.c | 670 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 660 insertions(+), 10 deletions(-)
arch/x86/kvm/vmx.c +660 -10
@@ -239,6 +239,7 @@ struct __attribute__ ((__packed__)) level_state {struct __attribute__ ((__packed__ ))nested_vmcs_page {u32 revision_id;u32 abort; +struct shadow_vmcs shadow_vmcs;struct level_state l2_state; };
@@ -263,6 +264,55 @@ struct nested_vmx {struct nested_vmcs_page *current_l2_page; }; +enum vmcs_field_type { +VMCS_FIELD_TYPE_U16 =0 , +VMCS_FIELD_TYPE_U64 =1 , +VMCS_FIELD_TYPE_U32 =2 , +VMCS_FIELD_TYPE_ULONG =3 +}; + +#define VMCS_FIELD_LENGTH_OFFSET 13 +#define VMCS_FIELD_LENGTH_MASK 0x6000 + +/* +Returns VMCS Field type +*/ +static inline int vmcs_field_type (unsigned long field) +{ +/* For 32 bit L1 when it using the HIGH field */ +if (0x1 & field) +return VMCS_FIELD_TYPE_U32 ; + +return (VMCS_FIELD_LENGTH_MASK & field) >>13 ; +} + +/* +Returncs VMCS field size in bits +*/ +static inline int vmcs_field_size (int field_type,struct kvm_vcpu *vcpu) +{ +switch (field_type) { +case VMCS_FIELD_TYPE_U16 : +return 2 ; +case VMCS_FIELD_TYPE_U32 : +return 4 ; +case VMCS_FIELD_TYPE_U64 : +return 8 ; +case VMCS_FIELD_TYPE_ULONG : +#ifdef CONFIG_X86_64 +if (is_long_mode (vcpu)) +return 8 ; +else +return 4 ; +#else +return 4 ; +#endif + } + +printk (KERN_INFO "WARNING: invalid field type %d \n" , field_type); +return 0 ; +} +struct vcpu_vmx {struct kvm_vcpu vcpu;struct list_head local_vcpus_link;
@@ -317,6 +367,411 @@ static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) return container_of(vcpu, struct vcpu_vmx, vcpu); } +static inline struct shadow_vmcs *get_shadow_vmcs (struct kvm_vcpu *vcpu) +{ +WARN_ON (!to_vmx (vcpu)->nested.current_l2_page); +return &(to_vmx (vcpu)->nested.current_l2_page->shadow_vmcs); +} + +#define SHADOW_VMCS_OFFSET (x) offsetof(struct shadow_vmcs, x) + +static unsigned short vmcs_field_to_offset_table[HOST_RIP +1 ] = { + + [VIRTUAL_PROCESSOR_ID ] = +SHADOW_VMCS_OFFSET (virtual_processor_id), + [GUEST_ES_SELECTOR ] = +SHADOW_VMCS_OFFSET (guest_es_selector), + [GUEST_CS_SELECTOR ] = +SHADOW_VMCS_OFFSET (guest_cs_selector), + [GUEST_SS_SELECTOR ] = +SHADOW_VMCS_OFFSET (guest_ss_selector), + [GUEST_DS_SELECTOR ] = +SHADOW_VMCS_OFFSET (guest_ds_selector), + [GUEST_FS_SELECTOR ] = +SHADOW_VMCS_OFFSET (guest_fs_selector), + [GUEST_GS_SELECTOR ] = +SHADOW_VMCS_OFFSET (guest_gs_selector), + [GUEST_LDTR_SELECTOR ] = +SHADOW_VMCS_OFFSET (guest_ldtr_selector), + [GUEST_TR_SELECTOR ] = +SHADOW_VMCS_OFFSET (guest_tr_selector), + [HOST_ES_SELECTOR ] = +SHADOW_VMCS_OFFSET (host_es_selector), + [HOST_CS_SELECTOR ] = +SHADOW_VMCS_OFFSET (host_cs_selector), + [HOST_SS_SELECTOR ] = +SHADOW_VMCS_OFFSET (host_ss_selector), + [HOST_DS_SELECTOR ] = +SHADOW_VMCS_OFFSET (host_ds_selector), + [HOST_FS_SELECTOR ] = +SHADOW_VMCS_OFFSET (host_fs_selector), + [HOST_GS_SELECTOR ] = +SHADOW_VMCS_OFFSET (host_gs_selector), + [HOST_TR_SELECTOR ] = +SHADOW_VMCS_OFFSET (host_tr_selector), + [IO_BITMAP_A ] = +SHADOW_VMCS_OFFSET (io_bitmap_a), + [IO_BITMAP_A_HIGH ] = +SHADOW_VMCS_OFFSET (io_bitmap_a)+4 , + [IO_BITMAP_B ] = +SHADOW_VMCS_OFFSET (io_bitmap_b), + [IO_BITMAP_B_HIGH ] = +SHADOW_VMCS_OFFSET (io_bitmap_b)+4 , + [MSR_BITMAP ] = +SHADOW_VMCS_OFFSET (msr_bitmap), + [MSR_BITMAP_HIGH ] = +SHADOW_VMCS_OFFSET (msr_bitmap)+4 , + [VM_EXIT_MSR_STORE_ADDR ] = +SHADOW_VMCS_OFFSET (vm_exit_msr_store_addr), + [VM_EXIT_MSR_STORE_ADDR_HIGH ] = +SHADOW_VMCS_OFFSET (vm_exit_msr_store_addr)+4 , + [VM_EXIT_MSR_LOAD_ADDR ] = +SHADOW_VMCS_OFFSET (vm_exit_msr_load_addr), + [VM_EXIT_MSR_LOAD_ADDR_HIGH ] = +SHADOW_VMCS_OFFSET (vm_exit_msr_load_addr)+4 , + [VM_ENTRY_MSR_LOAD_ADDR ] = +SHADOW_VMCS_OFFSET (vm_entry_msr_load_addr), + [VM_ENTRY_MSR_LOAD_ADDR_HIGH ] = +SHADOW_VMCS_OFFSET (vm_entry_msr_load_addr)+4 , + [TSC_OFFSET ] = +SHADOW_VMCS_OFFSET (tsc_offset), + [TSC_OFFSET_HIGH ] = +SHADOW_VMCS_OFFSET (tsc_offset)+4 , + [VIRTUAL_APIC_PAGE_ADDR ] = +SHADOW_VMCS_OFFSET (virtual_apic_page_addr), + [VIRTUAL_APIC_PAGE_ADDR_HIGH ] = +SHADOW_VMCS_OFFSET (virtual_apic_page_addr)+4 , + [APIC_ACCESS_ADDR ] = +SHADOW_VMCS_OFFSET (apic_access_addr), + [APIC_ACCESS_ADDR_HIGH ] = +SHADOW_VMCS_OFFSET (apic_access_addr)+4 , + [EPT_POINTER ] = +SHADOW_VMCS_OFFSET (ept_pointer), + [EPT_POINTER_HIGH ] = +SHADOW_VMCS_OFFSET (ept_pointer)+4 , + [GUEST_PHYSICAL_ADDRESS ] = +SHADOW_VMCS_OFFSET (guest_physical_address), + [GUEST_PHYSICAL_ADDRESS_HIGH ] = +SHADOW_VMCS_OFFSET (guest_physical_address)+4 , + [VMCS_LINK_POINTER ] = +SHADOW_VMCS_OFFSET (vmcs_link_pointer), + [VMCS_LINK_POINTER_HIGH ] = +SHADOW_VMCS_OFFSET (vmcs_link_pointer)+4 , + [GUEST_IA32_DEBUGCTL ] = +SHADOW_VMCS_OFFSET (guest_ia32_debugctl), + [GUEST_IA32_DEBUGCTL_HIGH ] = +SHADOW_VMCS_OFFSET (guest_ia32_debugctl)+4 , + [GUEST_IA32_PAT ] = +SHADOW_VMCS_OFFSET (guest_ia32_pat), + [GUEST_IA32_PAT_HIGH ] = +SHADOW_VMCS_OFFSET (guest_ia32_pat)+4 , + [GUEST_PDPTR0 ] = +SHADOW_VMCS_OFFSET (guest_pdptr0), + [GUEST_PDPTR0_HIGH ] = +SHADOW_VMCS_OFFSET (guest_pdptr0)+4 , + [GUEST_PDPTR1 ] = +SHADOW_VMCS_OFFSET (guest_pdptr1), + [GUEST_PDPTR1_HIGH ] = +SHADOW_VMCS_OFFSET (guest_pdptr1)+4 , + [GUEST_PDPTR2 ] = +SHADOW_VMCS_OFFSET (guest_pdptr2), + [GUEST_PDPTR2_HIGH ] = +SHADOW_VMCS_OFFSET (guest_pdptr2)+4 , + [GUEST_PDPTR3 ] = +SHADOW_VMCS_OFFSET (guest_pdptr3), + [GUEST_PDPTR3_HIGH ] = +SHADOW_VMCS_OFFSET (guest_pdptr3)+4 , + [HOST_IA32_PAT ] = +SHADOW_VMCS_OFFSET (host_ia32_pat), + [HOST_IA32_PAT_HIGH ] = +SHADOW_VMCS_OFFSET (host_ia32_pat)+4 , + [PIN_BASED_VM_EXEC_CONTROL ] = +SHADOW_VMCS_OFFSET (pin_based_vm_exec_control), + [CPU_BASED_VM_EXEC_CONTROL ] = +SHADOW_VMCS_OFFSET (cpu_based_vm_exec_control), + [EXCEPTION_BITMAP ] = +SHADOW_VMCS_OFFSET (exception_bitmap), + [PAGE_FAULT_ERROR_CODE_MASK ] = +SHADOW_VMCS_OFFSET (page_fault_error_code_mask), + [PAGE_FAULT_ERROR_CODE_MATCH ] = +SHADOW_VMCS_OFFSET (page_fault_error_code_match), + [CR3_TARGET_COUNT ] = +SHADOW_VMCS_OFFSET (cr3_target_count), + [VM_EXIT_CONTROLS ] = +SHADOW_VMCS_OFFSET (vm_exit_controls), + [VM_EXIT_MSR_STORE_COUNT ] = +SHADOW_VMCS_OFFSET (vm_exit_msr_store_count), + [VM_EXIT_MSR_LOAD_COUNT ] = +SHADOW_VMCS_OFFSET (vm_exit_msr_load_count), + [VM_ENTRY_CONTROLS ] = +SHADOW_VMCS_OFFSET (vm_entry_controls), + [VM_ENTRY_MSR_LOAD_COUNT ] = +SHADOW_VMCS_OFFSET (vm_entry_msr_load_count), + [VM_ENTRY_INTR_INFO_FIELD ] = +SHADOW_VMCS_OFFSET (vm_entry_intr_info_field), + [VM_ENTRY_EXCEPTION_ERROR_CODE ] = +SHADOW_VMCS_OFFSET (vm_entry_exception_error_code), + [VM_ENTRY_INSTRUCTION_LEN ] = +SHADOW_VMCS_OFFSET (vm_entry_instruction_len), + [TPR_THRESHOLD ] = +SHADOW_VMCS_OFFSET (tpr_threshold), + [SECONDARY_VM_EXEC_CONTROL ] = +SHADOW_VMCS_OFFSET (secondary_vm_exec_control), + [VM_INSTRUCTION_ERROR ] = +SHADOW_VMCS_OFFSET (vm_instruction_error), + [VM_EXIT_REASON ] = +SHADOW_VMCS_OFFSET (vm_exit_reason), + [VM_EXIT_INTR_INFO ] = +SHADOW_VMCS_OFFSET (vm_exit_intr_info), + [VM_EXIT_INTR_ERROR_CODE ] = +SHADOW_VMCS_OFFSET (vm_exit_intr_error_code), + [IDT_VECTORING_INFO_FIELD ] = +SHADOW_VMCS_OFFSET (idt_vectoring_info_field), + [IDT_VECTORING_ERROR_CODE ] = +SHADOW_VMCS_OFFSET (idt_vectoring_error_code), + [VM_EXIT_INSTRUCTION_LEN ] = +SHADOW_VMCS_OFFSET (vm_exit_instruction_len), + [VMX_INSTRUCTION_INFO ] = +SHADOW_VMCS_OFFSET (vmx_instruction_info), + [GUEST_ES_LIMIT ] = +SHADOW_VMCS_OFFSET (guest_es_limit), + [GUEST_CS_LIMIT ] = +SHADOW_VMCS_OFFSET (guest_cs_limit), + [GUEST_SS_LIMIT ] = +SHADOW_VMCS_OFFSET (guest_ss_limit), + [GUEST_DS_LIMIT ] = +SHADOW_VMCS_OFFSET (guest_ds_limit), + [GUEST_FS_LIMIT ] = +SHADOW_VMCS_OFFSET (guest_fs_limit), + [GUEST_GS_LIMIT ] = +SHADOW_VMCS_OFFSET (guest_gs_limit), + [GUEST_LDTR_LIMIT ] = +SHADOW_VMCS_OFFSET (guest_ldtr_limit), + [GUEST_TR_LIMIT ] = +SHADOW_VMCS_OFFSET (guest_tr_limit), + [GUEST_GDTR_LIMIT ] = +SHADOW_VMCS_OFFSET (guest_gdtr_limit), + [GUEST_IDTR_LIMIT ] = +SHADOW_VMCS_OFFSET (guest_idtr_limit), + [GUEST_ES_AR_BYTES ] = +SHADOW_VMCS_OFFSET (guest_es_ar_bytes), + [GUEST_CS_AR_BYTES ] = +SHADOW_VMCS_OFFSET (guest_cs_ar_bytes), + [GUEST_SS_AR_BYTES ] = +SHADOW_VMCS_OFFSET (guest_ss_ar_bytes), + [GUEST_DS_AR_BYTES ] = +SHADOW_VMCS_OFFSET (guest_ds_ar_bytes), + [GUEST_FS_AR_BYTES ] = +SHADOW_VMCS_OFFSET (guest_fs_ar_bytes), + [GUEST_GS_AR_BYTES ] = +SHADOW_VMCS_OFFSET (guest_gs_ar_bytes), + [GUEST_LDTR_AR_BYTES ] = +SHADOW_VMCS_OFFSET (guest_ldtr_ar_bytes), + [GUEST_TR_AR_BYTES ] = +SHADOW_VMCS_OFFSET (guest_tr_ar_bytes), + [GUEST_INTERRUPTIBILITY_INFO ] = +SHADOW_VMCS_OFFSET (guest_interruptibility_info), + [GUEST_ACTIVITY_STATE ] = +SHADOW_VMCS_OFFSET (guest_activity_state), + [GUEST_SYSENTER_CS ] = +SHADOW_VMCS_OFFSET (guest_sysenter_cs), + [HOST_IA32_SYSENTER_CS ] = +SHADOW_VMCS_OFFSET (host_ia32_sysenter_cs), + [CR0_GUEST_HOST_MASK ] = +SHADOW_VMCS_OFFSET (cr0_guest_host_mask), + [CR4_GUEST_HOST_MASK ] = +SHADOW_VMCS_OFFSET (cr4_guest_host_mask), + [CR0_READ_SHADOW ] = +SHADOW_VMCS_OFFSET (cr0_read_shadow), + [CR4_READ_SHADOW ] = +SHADOW_VMCS_OFFSET (cr4_read_shadow), + [CR3_TARGET_VALUE0 ] = +SHADOW_VMCS_OFFSET (cr3_target_value0), + [CR3_TARGET_VALUE1 ] = +SHADOW_VMCS_OFFSET (cr3_target_value1), + [CR3_TARGET_VALUE2 ] = +SHADOW_VMCS_OFFSET (cr3_target_value2), + [CR3_TARGET_VALUE3 ] = +SHADOW_VMCS_OFFSET (cr3_target_value3), + [EXIT_QUALIFICATION ] = +SHADOW_VMCS_OFFSET (exit_qualification), + [GUEST_LINEAR_ADDRESS ] = +SHADOW_VMCS_OFFSET (guest_linear_address), + [GUEST_CR0 ] = +SHADOW_VMCS_OFFSET (guest_cr0), + [GUEST_CR3 ] = +SHADOW_VMCS_OFFSET (guest_cr3), + [GUEST_CR4 ] = +SHADOW_VMCS_OFFSET (guest_cr4), + [GUEST_ES_BASE ] = +SHADOW_VMCS_OFFSET (guest_es_base), + [GUEST_CS_BASE ] = +SHADOW_VMCS_OFFSET (guest_cs_base), + [GUEST_SS_BASE ] = +SHADOW_VMCS_OFFSET (guest_ss_base), + [GUEST_DS_BASE ] = +SHADOW_VMCS_OFFSET (guest_ds_base), + [GUEST_FS_BASE ] = +SHADOW_VMCS_OFFSET (guest_fs_base), + [GUEST_GS_BASE ] = +SHADOW_VMCS_OFFSET (guest_gs_base), + [GUEST_LDTR_BASE ] = +SHADOW_VMCS_OFFSET (guest_ldtr_base), + [GUEST_TR_BASE ] = +SHADOW_VMCS_OFFSET (guest_tr_base), + [GUEST_GDTR_BASE ] = +SHADOW_VMCS_OFFSET (guest_gdtr_base), + [GUEST_IDTR_BASE ] = +SHADOW_VMCS_OFFSET (guest_idtr_base), + [GUEST_DR7 ] = +SHADOW_VMCS_OFFSET (guest_dr7), + [GUEST_RSP ] = +SHADOW_VMCS_OFFSET (guest_rsp), + [GUEST_RIP ] = +SHADOW_VMCS_OFFSET (guest_rip), + [GUEST_RFLAGS ] = +SHADOW_VMCS_OFFSET (guest_rflags), + [GUEST_PENDING_DBG_EXCEPTIONS ] = +SHADOW_VMCS_OFFSET (guest_pending_dbg_exceptions), + [GUEST_SYSENTER_ESP ] = +SHADOW_VMCS_OFFSET (guest_sysenter_esp), + [GUEST_SYSENTER_EIP ] = +SHADOW_VMCS_OFFSET (guest_sysenter_eip), + [HOST_CR0 ] = +SHADOW_VMCS_OFFSET (host_cr0), + [HOST_CR3 ] = +SHADOW_VMCS_OFFSET (host_cr3), + [HOST_CR4 ] = +SHADOW_VMCS_OFFSET (host_cr4), + [HOST_FS_BASE ] = +SHADOW_VMCS_OFFSET (host_fs_base), + [HOST_GS_BASE ] = +SHADOW_VMCS_OFFSET (host_gs_base), + [HOST_TR_BASE ] = +SHADOW_VMCS_OFFSET (host_tr_base), + [HOST_GDTR_BASE ] = +SHADOW_VMCS_OFFSET (host_gdtr_base), + [HOST_IDTR_BASE ] = +SHADOW_VMCS_OFFSET (host_idtr_base), + [HOST_IA32_SYSENTER_ESP ] = +SHADOW_VMCS_OFFSET (host_ia32_sysenter_esp), + [HOST_IA32_SYSENTER_EIP ] = +SHADOW_VMCS_OFFSET (host_ia32_sysenter_eip), + [HOST_RSP ] = +SHADOW_VMCS_OFFSET (host_rsp), + [HOST_RIP ] = +SHADOW_VMCS_OFFSET (host_rip), +}; + +static inline unsigned short vmcs_field_to_offset (unsigned long field) +{ + +if (field >HOST_RIP || vmcs_field_to_offset_table[field] ==0 ) { +printk (KERN_ERR "invalid vmcs encoding 0x%lx\n" , field); +return -1 ; + } + +return vmcs_field_to_offset_table[field]; +} + +static inline unsigned long nested_vmcs_readl (struct kvm_vcpu *vcpu, +unsigned long field) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +unsigned long *entry; + +if (!vmx->nested.current_l2_page) { +printk (KERN_ERR "%s invalid nested vmcs\n" , __func__); +return -1 ; + } + + entry = (unsigned long *)((char *)(get_shadow_vmcs (vcpu)) + +vmcs_field_to_offset (field)); +return *entry; +} + +static inline u16 nested_vmcs_read16 (struct kvm_vcpu *vcpu, +unsigned long field) +{ +return nested_vmcs_readl (vcpu, field); +} + +static inline u32 nested_vmcs_read32 (struct kvm_vcpu *vcpu,unsigned long field) +{ +return nested_vmcs_readl (vcpu, field); +} + +static inline u64 nested_vmcs_read64 (struct kvm_vcpu *vcpu,unsigned long field) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +u64 *entry; +if (!vmx->nested.current_l2_page) { +printk (KERN_ERR "%s invalid nested vmcs\n" , __func__); +return -1 ; + } + + entry = (u64 *)((char *)(get_shadow_vmcs (vcpu)) + +vmcs_field_to_offset (field)); +return *entry; +} + +static inline void nested_vmcs_writel (struct kvm_vcpu *vcpu, +unsigned long field,unsigned long value) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +unsigned long entry = + (unsigned long )(get_shadow_vmcs (vcpu)); + +if (!vmx->nested.current_l2_page) { +printk (KERN_ERR "%s invalid nested vmcs\n" , __func__); +return ; + } + + entry +=vmcs_field_to_offset (field); + *(unsigned long *)entry = value; +} + +static inline void nested_vmcs_write16 (struct kvm_vcpu *vcpu, +unsigned long field,u16 value) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +unsigned long entry = + (unsigned long )(get_shadow_vmcs (vcpu)); + +if (!vmx->nested.current_l2_page) { +printk (KERN_ERR "%s invalid nested vmcs\n" , __func__); +return ; + } + entry +=vmcs_field_to_offset (field); + *(u16 *)entry = value; +} + +static inline void nested_vmcs_write32 (struct kvm_vcpu *vcpu, +unsigned long field,u32 value) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +unsigned long entry = + (unsigned long )(get_shadow_vmcs (vcpu)); + +if (!vmx->nested.current_l2_page) { +printk (KERN_ERR "%s invalid nested vmcs\n" , __func__); +return ; + } + entry +=vmcs_field_to_offset (field); + *(u32 *)entry = value; +} + +static inline void nested_vmcs_write64 (struct kvm_vcpu *vcpu, +unsigned long field,u64 value) +{ +#ifdef CONFIG_X86_64 +nested_vmcs_writel (vcpu, field, value); +#else /* nested: 32 bit not actually tested */ +nested_vmcs_writel (vcpu, field, value); +nested_vmcs_writel (vcpu, field+1 , value >>32 ); +#endif +} +static struct page *nested_get_page (struct kvm_vcpu *vcpu,u64 vmcs_addr) {
@@ -354,11 +809,6 @@ static int nested_map_current(struct kvm_vcpu *vcpu) mapped_page =kmap_atomic (vmcs_page,KM_USER0 ); -if (!mapped_page) { -printk (KERN_INFO "%s: error in kmap_atomic\n" , __func__); -return 0 ; - } - vmx->nested.current_l2_page = mapped_page;return 1 ;
@@ -1390,7 +1840,7 @@ static int read_guest_vmcs_gpa(struct kvm_vcpu *vcpu, gva_t gva, u64 *gentry) size, vcpu);if (r) {printk (KERN_ERR "%s cannot read guest vmcs addr %lx : %d\n" , - __func__, vcpu->arch.regs[VCPU_REGS_RAX ], r); + __func__, gva, r);return r; }
@@ -3764,6 +4214,26 @@ static gva_t get_vmx_mem_address(struct kvm_vcpu *vcpu,return addr; } +static void set_rflags_to_vmx_fail_invalid (struct kvm_vcpu *vcpu) +{ +unsigned long rflags; + rflags =vmx_get_rflags (vcpu); + rflags |=X86_EFLAGS_CF ; + rflags &= ~X86_EFLAGS_PF & ~X86_EFLAGS_AF & ~X86_EFLAGS_ZF & + ~X86_EFLAGS_SF & ~X86_EFLAGS_OF ; +vmx_set_rflags (vcpu, rflags); +} + +static void set_rflags_to_vmx_fail_valid (struct kvm_vcpu *vcpu) +{ +unsigned long rflags; + rflags =vmx_get_rflags (vcpu); + rflags |=X86_EFLAGS_ZF ; + rflags &= ~X86_EFLAGS_PF & ~X86_EFLAGS_AF & ~X86_EFLAGS_CF & + ~X86_EFLAGS_SF & ~X86_EFLAGS_OF ; +vmx_set_rflags (vcpu, rflags); + } +static int handle_vmclear (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);
@@ -3800,6 +4270,181 @@ static int handle_vmclear(struct kvm_vcpu *vcpu)return 1 ; } +static int handle_vmread_reg (struct kvm_vcpu *vcpu,int reg, +unsigned long field) +{ +u64 field_value; + +switch (vmcs_field_type (field)) { +case VMCS_FIELD_TYPE_U16 : + field_value =nested_vmcs_read16 (vcpu, field); +break ; +case VMCS_FIELD_TYPE_U32 : + field_value =nested_vmcs_read32 (vcpu, field); +break ; +case VMCS_FIELD_TYPE_U64 : + field_value =nested_vmcs_read64 (vcpu, field); +#ifdef CONFIG_X86_64 +if (!is_long_mode (vcpu)) { +kvm_register_write (vcpu, reg+1 , field_value >>32 ); + field_value = (u32 )field_value; + } +#endif +break ; +case VMCS_FIELD_TYPE_ULONG : + field_value =nested_vmcs_readl (vcpu, field); +#ifdef CONFIG_X86_64 +if (!is_long_mode (vcpu)) { +kvm_register_write (vcpu, reg+1 , field_value >>32 ); + field_value = (u32 )field_value; + } +#endif +break ; +default : +printk (KERN_INFO "%s invalid field\n" , __func__); +return 0 ; + } + +kvm_register_write (vcpu, reg, field_value); +return 1 ; +} + +static int handle_vmread_mem (struct kvm_vcpu *vcpu,gva_t gva, +unsigned long field) +{ +u64 field_value; + +switch (vmcs_field_type (field)) { +case VMCS_FIELD_TYPE_U16 : + field_value =nested_vmcs_read16 (vcpu, field); +break ; +case VMCS_FIELD_TYPE_U32 : + field_value =nested_vmcs_read32 (vcpu, field); +break ; +case VMCS_FIELD_TYPE_U64 : + field_value =nested_vmcs_read64 (vcpu, field); +break ; +case VMCS_FIELD_TYPE_ULONG : + field_value =nested_vmcs_readl (vcpu, field); +break ; +default : +printk (KERN_INFO "%s invalid field\n" , __func__); +return 0 ; + } + +kvm_write_guest_virt (gva, &field_value, +vmcs_field_size (vmcs_field_type (field), vcpu), + vcpu); +return 1 ; +} + +static int handle_vmread (struct kvm_vcpu *vcpu) +{ +unsigned long field; +int reg; +int is_reg; +unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); +u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO ); +gva_t gva =0 ; +int read_succeed; + +if (!nested_vmx_check_permission (vcpu)) +return 1 ; + +if (!nested_map_current (vcpu)) { +printk (KERN_INFO "%s invalid shadow vmcs\n" , __func__); +set_rflags_to_vmx_fail_invalid (vcpu); +return 1 ; + } + +/* decode instruction info to get the field to read and where to store its value */ +/* Bit 10, Mem/Reg (0 = memory, 1 = register) */ + is_reg = vmx_instruction_info & (1u <<10 );/* bit 10 */ + field =kvm_register_read (vcpu, (vmx_instruction_info >>28 ) &0xf );/* bits 31:28 */ + +if (is_reg) { + reg = (vmx_instruction_info >>3 ) &0xf ;/* bits 3:6 */ + read_succeed =handle_vmread_reg (vcpu, reg, field); + }else { + gva =get_vmx_mem_address (vcpu, exit_qualification, + vmx_instruction_info); + read_succeed =handle_vmread_mem (vcpu, gva, field); + } + +if (read_succeed) { +clear_rflags_cf_zf (vcpu); +skip_emulated_instruction (vcpu); + }else { +set_rflags_to_vmx_fail_valid (vcpu); +vmcs_write32 (VM_INSTRUCTION_ERROR ,12 ); + } + +nested_unmap_current (vcpu); +return 1 ; +} + + +static int handle_vmwrite (struct kvm_vcpu *vcpu) +{ +unsigned long field; +u64 field_value =0 ; +int reg; +int is_reg; +gva_t gva; +int field_type; +unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); +u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO ); + +if (!nested_vmx_check_permission (vcpu)) +return 1 ; + +if (!nested_map_current (vcpu)) { +printk (KERN_INFO "%s invalid shadow vmcs\n" , __func__); +set_rflags_to_vmx_fail_invalid (vcpu); +return 1 ; + } + +/* decode instruction info to get the field to read and where to store its value */ +/* Bit 10, Mem/Reg (0 = memory, 1 = register */ + is_reg = vmx_instruction_info & (1u <<10 );/* bit 10 */ + reg = (vmx_instruction_info >>3 ) &0xf ;/* bits 3:6 */ + field =kvm_register_read (vcpu, (vmx_instruction_info >>28 ) &0xf );/* bits 31:28 */ + field_type =vmcs_field_type (field); + +if (is_reg) + field_value =kvm_register_read (vcpu, reg); +else { + gva =get_vmx_mem_address (vcpu, exit_qualification, vmx_instruction_info); +kvm_read_guest_virt (gva, &field_value,vmcs_field_size (field_type, vcpu), vcpu); + } + +switch (field_type) { +case VMCS_FIELD_TYPE_U16 : +nested_vmcs_write16 (vcpu, field, field_value); +break ; +case VMCS_FIELD_TYPE_U32 : +nested_vmcs_write32 (vcpu, field, field_value); +break ; +case VMCS_FIELD_TYPE_U64 : +nested_vmcs_write64 (vcpu, field, field_value); +break ; +case VMCS_FIELD_TYPE_ULONG : +nested_vmcs_writel (vcpu, field, field_value); +break ; +default : +printk (KERN_INFO "%s invalid field\n" , __func__); +set_rflags_to_vmx_fail_valid (vcpu); +vmcs_write32 (VM_INSTRUCTION_ERROR ,12 ); +nested_unmap_current (vcpu); +return 1 ; + } + +clear_rflags_cf_zf (vcpu); +skip_emulated_instruction (vcpu); +nested_unmap_current (vcpu); +return 1 ; +} +static int handle_vmoff (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);
@@ -3901,15 +4546,20 @@ static int handle_vmptrst(struct kvm_vcpu *vcpu)unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION );u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO );gva_t vmcs_gva; - +uint size;if (!nested_vmx_check_permission (vcpu))return 1 ; vmcs_gva =get_vmx_mem_address (vcpu, exit_qualification, vmx_instruction_info); +if (is_long_mode (vcpu)) + size =sizeof (u64); +else + size =sizeof (u32); + r =kvm_write_guest_virt (vmcs_gva, (void *)&to_vmx (vcpu)->nested.current_vmptr, -sizeof (u64), vcpu); + size, vcpu);if (r) {printk (KERN_INFO "%s failed to write vmptr\n" , __func__);return 1 ;
@@ -4206,9 +4856,9 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { [EXIT_REASON_VMLAUNCH] = handle_vmx_insn, [EXIT_REASON_VMPTRLD] = handle_vmptrld, [EXIT_REASON_VMPTRST] = handle_vmptrst, - [EXIT_REASON_VMREAD] = handle_vmx_insn, + [EXIT_REASON_VMREAD] = handle_vmread, [EXIT_REASON_VMRESUME] = handle_vmx_insn, - [EXIT_REASON_VMWRITE] = handle_vmx_insn, + [EXIT_REASON_VMWRITE] = handle_vmwrite, [EXIT_REASON_VMOFF] = handle_vmoff, [EXIT_REASON_VMON] = handle_vmon, [EXIT_REASON_TPR_BELOW_THRESHOLD] = handle_tpr_below_threshold,
Signature · 1 line
1.6.0.4
PATCH 5/7 Nested VMX patch 5 Simplify fpu handling
From: Orit Wasserman <[email protected]>
--- arch/x86/kvm/vmx.c | 27 +++++++++++++++++---------- 1 files changed, 17 insertions(+), 10 deletions(-)
arch/x86/kvm/vmx.c +17 -10
@@ -1244,8 +1244,6 @@ static void update_exception_bitmap(struct kvm_vcpu *vcpu)u32 eb; eb = (1u <<PF_VECTOR ) | (1u <<UD_VECTOR ) | (1u <<MC_VECTOR ); -if (!vcpu->fpu_active) - eb |=1u <<NM_VECTOR ;/* * Unconditionally intercept #DB so we can maintain dr6 without * reading it every exit.
@@ -1463,10 +1461,6 @@ static void vmx_fpu_activate(struct kvm_vcpu *vcpu)if (vcpu->fpu_active)return ; vcpu->fpu_active =1 ; -vmcs_clear_bits (GUEST_CR0 ,X86_CR0_TS ); -if (vcpu->arch.cr0 &X86_CR0_TS ) -vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS ); -update_exception_bitmap (vcpu); }static void vmx_fpu_deactivate (struct kvm_vcpu *vcpu)
@@ -1474,8 +1468,6 @@ static void vmx_fpu_deactivate(struct kvm_vcpu *vcpu)if (!vcpu->fpu_active)return ; vcpu->fpu_active =0 ; -vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS ); -update_exception_bitmap (vcpu); }static unsigned long vmx_get_rflags (struct kvm_vcpu *vcpu)
@@ -2715,8 +2707,10 @@ static void vmx_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)vmx_flush_tlb (vcpu);vmcs_writel (GUEST_CR3 , guest_cr3); -if (vcpu->arch.cr0 &X86_CR0_PE ) -vmx_fpu_deactivate (vcpu); +if (vcpu->arch.cr0 &X86_CR0_PE ) { +if (guest_cr3 !=vmcs_readl (GUEST_CR3 )) +vmx_fpu_deactivate (vcpu); + } }static void vmx_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4)
@@ -5208,6 +5202,19 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu)if (vcpu->arch.switch_db_regs)get_debugreg (vcpu->arch.dr6,6 ); +if (vcpu->fpu_active) { +if (vmcs_readl (CR0_READ_SHADOW ) &X86_CR0_TS ) +vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS ); +else +vmcs_clear_bits (GUEST_CR0 ,X86_CR0_TS ); +vmcs_write32 (EXCEPTION_BITMAP , +vmcs_read32 (EXCEPTION_BITMAP ) & ~(1u <<NM_VECTOR )); + }else { +vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS ); +vmcs_write32 (EXCEPTION_BITMAP , +vmcs_read32 (EXCEPTION_BITMAP ) | (1u <<NM_VECTOR )); + } + vmx->idt_vectoring_info =vmcs_read32 (IDT_VECTORING_INFO_FIELD );if (vmx->rmode.irq.pending)fixup_rmode_irq (vmx);
Signature · 1 line
1.6.0.4
PATCH 6/7 Nested VMX patch 6 implements vmlaunch and vmresume
From: Orit Wasserman <[email protected]>
--- arch/x86/kvm/vmx.c | 890 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 873 insertions(+), 17 deletions(-)
arch/x86/kvm/vmx.c +873 -17
@@ -223,10 +223,16 @@ struct __attribute__ ((__packed__)) level_state {/* Has the level1 guest done vmclear? */ bool vmclear; +u64 shadow_efer; +unsigned long cr3; +unsigned long cr4; +u64 io_bitmap_a;u64 io_bitmap_b;u64 msr_bitmap; +int cpu; +int launched;bool first_launch; };
@@ -254,10 +260,14 @@ struct nested_vmx {bool vmxon;/* What is the location of the current vmcs l1 keeps for l2 */ gpa_t current_vmptr; +/* Are we running nested guest */ +bool nested_mode;/* Level 1 state for switching to level 2 and back */ struct level_state *l1_state;/* Level 1 shadow vmcs for switching to level 2 and back */ struct shadow_vmcs *l1_shadow_vmcs; +/* Level 1 vmcs loaded into the processor */ +struct vmcs *l1_vmcs;/* list of vmcs for each l2 guest created by l1 */ struct list_head l2_vmcs_list;/* l2 page corresponding to the current vmcs set by l1 */
@@ -287,7 +297,7 @@ static inline int vmcs_field_type(unsigned long field) }/* -Returncs VMCS field size in bits +Returns VMCS field size in bits */ static inline int vmcs_field_size (int field_type,struct kvm_vcpu *vcpu) {
@@ -313,6 +323,10 @@ static inline int vmcs_field_size(int field_type, struct kvm_vcpu *vcpu)return 0 ; } +#define NESTED_VM_EXIT_CONTROLS_MASK (~(VM_EXIT_LOAD_IA32_PAT | \ + VM_EXIT_SAVE_IA32_PAT)) +#define NESTED_VM_ENTRY_CONTROLS_MASK (~(VM_ENTRY_LOAD_IA32_PAT | \ + VM_ENTRY_IA32E_MODE))struct vcpu_vmx {struct kvm_vcpu vcpu;struct list_head local_vcpus_link;
@@ -892,7 +906,11 @@ static struct kvm_vmx_segment_field {static u64 host_efer;static void ept_save_pdptrs (struct kvm_vcpu *vcpu); + +static int nested_vmx_check_permission (struct kvm_vcpu *vcpu);static int create_l1_state (struct kvm_vcpu *vcpu); +static int create_l2_state (struct kvm_vcpu *vcpu); +static int launch_guest (struct kvm_vcpu *vcpu,bool launch);/* * Keep MSR_K6_STAR at the end, as setup_msrs() will try to optimize it
@@ -993,6 +1011,18 @@ static inline bool cpu_has_vmx_ept_2m_page(void)return !!(vmx_capability.ept &VMX_EPT_2MB_PAGE_BIT ); } +static inline int is_exception (u32 intr_info) +{ +return (intr_info & (INTR_INFO_INTR_TYPE_MASK |INTR_INFO_VALID_MASK )) + == (INTR_TYPE_HARD_EXCEPTION |INTR_INFO_VALID_MASK ); +} + +static inline int is_nmi (u32 intr_info) +{ +return (intr_info & (INTR_INFO_INTR_TYPE_MASK |INTR_INFO_VALID_MASK )) + == (INTR_TYPE_NMI_INTR |INTR_INFO_VALID_MASK ); +} +static inline int cpu_has_vmx_invept_individual_addr (void ) {return !!(vmx_capability.ept &VMX_EPT_EXTENT_INDIVIDUAL_BIT );
@@ -1049,6 +1079,51 @@ static inline bool report_flexpriority(void)return flexpriority_enabled; } +static inline int nested_cpu_has_vmx_tpr_shadow (struct kvm_vcpu *vcpu) +{ +return cpu_has_vmx_tpr_shadow () && +get_shadow_vmcs (vcpu)->cpu_based_vm_exec_control & +CPU_BASED_TPR_SHADOW ; +} + +static inline int nested_cpu_has_secondary_exec_ctrls (struct kvm_vcpu *vcpu) +{ +return cpu_has_secondary_exec_ctrls () && +get_shadow_vmcs (vcpu)->cpu_based_vm_exec_control & +CPU_BASED_ACTIVATE_SECONDARY_CONTROLS ; +} + +static inline bool nested_vm_need_virtualize_apic_accesses (struct kvm_vcpu + *vcpu) +{ +return get_shadow_vmcs (vcpu)->secondary_vm_exec_control & +SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES ; +} + +static inline int nested_cpu_has_vmx_ept (struct kvm_vcpu *vcpu) +{ +return get_shadow_vmcs (vcpu)-> + secondary_vm_exec_control &SECONDARY_EXEC_ENABLE_EPT ; +} + +static inline int nested_cpu_has_vmx_vpid (struct kvm_vcpu *vcpu) +{ +return get_shadow_vmcs (vcpu)->secondary_vm_exec_control & +SECONDARY_EXEC_ENABLE_VPID ; +} + +static inline int nested_cpu_has_vmx_pat (struct kvm_vcpu *vcpu) +{ +return get_shadow_vmcs (vcpu)->vm_entry_controls & +VM_ENTRY_LOAD_IA32_PAT ; +} + +static inline int nested_cpu_has_vmx_msr_bitmap (struct kvm_vcpu *vcpu) +{ +return get_shadow_vmcs (vcpu)->cpu_based_vm_exec_control & +CPU_BASED_USE_MSR_BITMAPS ; +} +static int __find_msr_index (struct vcpu_vmx *vmx,u32 msr) {int i;
@@ -1390,6 +1465,8 @@ static void vmx_load_host_state(struct vcpu_vmx *vmx)preempt_enable (); } +static void vmx_fpu_deactivate (struct kvm_vcpu *vcpu); +/* * Switches to specified vcpu, until a matching vcpu_put(), but assumes * vcpu mutex is already taken.
@@ -1994,6 +2071,206 @@ static void vmclear_local_vcpus(void)__vcpu_clear (vmx); } + +void prepare_vmcs_12 (struct kvm_vcpu *vcpu) +{ +struct shadow_vmcs *l2_shadow_vmcs = +get_shadow_vmcs (vcpu); + + l2_shadow_vmcs->guest_es_selector =vmcs_read16 (GUEST_ES_SELECTOR ); + l2_shadow_vmcs->guest_cs_selector =vmcs_read16 (GUEST_CS_SELECTOR ); + l2_shadow_vmcs->guest_ss_selector =vmcs_read16 (GUEST_SS_SELECTOR ); + l2_shadow_vmcs->guest_ds_selector =vmcs_read16 (GUEST_DS_SELECTOR ); + l2_shadow_vmcs->guest_fs_selector =vmcs_read16 (GUEST_FS_SELECTOR ); + l2_shadow_vmcs->guest_gs_selector =vmcs_read16 (GUEST_GS_SELECTOR ); + l2_shadow_vmcs->guest_ldtr_selector =vmcs_read16 (GUEST_LDTR_SELECTOR ); + l2_shadow_vmcs->guest_tr_selector =vmcs_read16 (GUEST_TR_SELECTOR ); + + l2_shadow_vmcs->tsc_offset =vmcs_read64 (TSC_OFFSET ); + l2_shadow_vmcs->guest_physical_address = +vmcs_read64 (GUEST_PHYSICAL_ADDRESS ); + l2_shadow_vmcs->vmcs_link_pointer =vmcs_read64 (VMCS_LINK_POINTER ); + l2_shadow_vmcs->guest_ia32_debugctl =vmcs_read64 (GUEST_IA32_DEBUGCTL ); +if (vmcs_config.vmentry_ctrl &VM_ENTRY_LOAD_IA32_PAT ) + l2_shadow_vmcs->guest_ia32_pat =vmcs_read64 (GUEST_IA32_PAT ); + l2_shadow_vmcs->cr3_target_count =vmcs_read32 (CR3_TARGET_COUNT ); + l2_shadow_vmcs->vm_entry_intr_info_field = +vmcs_read32 (VM_ENTRY_INTR_INFO_FIELD ); + l2_shadow_vmcs->vm_entry_exception_error_code = +vmcs_read32 (VM_ENTRY_EXCEPTION_ERROR_CODE ); + l2_shadow_vmcs->vm_entry_instruction_len = +vmcs_read32 (VM_ENTRY_INSTRUCTION_LEN ); + l2_shadow_vmcs->vm_instruction_error = +vmcs_read32 (VM_INSTRUCTION_ERROR ); + l2_shadow_vmcs->vm_exit_reason =vmcs_read32 (VM_EXIT_REASON ); + l2_shadow_vmcs->vm_exit_intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); + l2_shadow_vmcs->vm_exit_intr_error_code = +vmcs_read32 (VM_EXIT_INTR_ERROR_CODE ); + l2_shadow_vmcs->idt_vectoring_info_field = +vmcs_read32 (IDT_VECTORING_INFO_FIELD ); + l2_shadow_vmcs->idt_vectoring_error_code = +vmcs_read32 (IDT_VECTORING_ERROR_CODE ); + l2_shadow_vmcs->vm_exit_instruction_len = +vmcs_read32 (VM_EXIT_INSTRUCTION_LEN ); + l2_shadow_vmcs->vmx_instruction_info = +vmcs_read32 (VMX_INSTRUCTION_INFO ); + l2_shadow_vmcs->guest_es_limit =vmcs_read32 (GUEST_ES_LIMIT ); + l2_shadow_vmcs->guest_cs_limit =vmcs_read32 (GUEST_CS_LIMIT ); + l2_shadow_vmcs->guest_ss_limit =vmcs_read32 (GUEST_SS_LIMIT ); + l2_shadow_vmcs->guest_ds_limit =vmcs_read32 (GUEST_DS_LIMIT ); + l2_shadow_vmcs->guest_fs_limit =vmcs_read32 (GUEST_FS_LIMIT ); + l2_shadow_vmcs->guest_gs_limit =vmcs_read32 (GUEST_GS_LIMIT ); + l2_shadow_vmcs->guest_ldtr_limit =vmcs_read32 (GUEST_LDTR_LIMIT ); + l2_shadow_vmcs->guest_tr_limit =vmcs_read32 (GUEST_TR_LIMIT ); + l2_shadow_vmcs->guest_gdtr_limit =vmcs_read32 (GUEST_GDTR_LIMIT ); + l2_shadow_vmcs->guest_idtr_limit =vmcs_read32 (GUEST_IDTR_LIMIT ); + l2_shadow_vmcs->guest_es_ar_bytes =vmcs_read32 (GUEST_ES_AR_BYTES ); + l2_shadow_vmcs->guest_cs_ar_bytes =vmcs_read32 (GUEST_CS_AR_BYTES ); + l2_shadow_vmcs->guest_ss_ar_bytes =vmcs_read32 (GUEST_SS_AR_BYTES ); + l2_shadow_vmcs->guest_ds_ar_bytes =vmcs_read32 (GUEST_DS_AR_BYTES ); + l2_shadow_vmcs->guest_fs_ar_bytes =vmcs_read32 (GUEST_FS_AR_BYTES ); + l2_shadow_vmcs->guest_gs_ar_bytes =vmcs_read32 (GUEST_GS_AR_BYTES ); + l2_shadow_vmcs->guest_ldtr_ar_bytes =vmcs_read32 (GUEST_LDTR_AR_BYTES ); + l2_shadow_vmcs->guest_tr_ar_bytes =vmcs_read32 (GUEST_TR_AR_BYTES ); + l2_shadow_vmcs->guest_interruptibility_info = +vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ); + l2_shadow_vmcs->guest_activity_state = +vmcs_read32 (GUEST_ACTIVITY_STATE ); + l2_shadow_vmcs->guest_sysenter_cs =vmcs_read32 (GUEST_SYSENTER_CS ); + + l2_shadow_vmcs->cr4_read_shadow =vmcs_readl (CR4_READ_SHADOW ); + l2_shadow_vmcs->exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); + l2_shadow_vmcs->guest_linear_address =vmcs_readl (GUEST_LINEAR_ADDRESS ); + +if (l2_shadow_vmcs->cr0_guest_host_mask &X86_CR0_TS ) + l2_shadow_vmcs->guest_cr0 =vmcs_readl (GUEST_CR0 ); +else /* if CR0_GUEST_HOST_MASK[TS]=0 l1 should think TS was really written to CR0 */ + l2_shadow_vmcs->guest_cr0 = + (vmcs_readl (GUEST_CR0 )&~X86_CR0_TS ) | (vmcs_readl (CR0_READ_SHADOW ) &X86_CR0_TS ); + + l2_shadow_vmcs->guest_cr4 =vmcs_readl (GUEST_CR4 ); + l2_shadow_vmcs->guest_es_base =vmcs_readl (GUEST_ES_BASE ); + l2_shadow_vmcs->guest_cs_base =vmcs_readl (GUEST_CS_BASE ); + l2_shadow_vmcs->guest_ss_base =vmcs_readl (GUEST_SS_BASE ); + l2_shadow_vmcs->guest_ds_base =vmcs_readl (GUEST_DS_BASE ); + l2_shadow_vmcs->guest_fs_base =vmcs_readl (GUEST_FS_BASE ); + l2_shadow_vmcs->guest_gs_base =vmcs_readl (GUEST_GS_BASE ); + l2_shadow_vmcs->guest_ldtr_base =vmcs_readl (GUEST_LDTR_BASE ); + l2_shadow_vmcs->guest_tr_base =vmcs_readl (GUEST_TR_BASE ); + l2_shadow_vmcs->guest_gdtr_base =vmcs_readl (GUEST_GDTR_BASE ); + l2_shadow_vmcs->guest_idtr_base =vmcs_readl (GUEST_IDTR_BASE ); + l2_shadow_vmcs->guest_dr7 =vmcs_readl (GUEST_DR7 ); + l2_shadow_vmcs->guest_rsp =vmcs_readl (GUEST_RSP ); + l2_shadow_vmcs->guest_rip =vmcs_readl (GUEST_RIP ); + l2_shadow_vmcs->guest_rflags =vmcs_readl (GUEST_RFLAGS ); + l2_shadow_vmcs->guest_pending_dbg_exceptions = +vmcs_readl (GUEST_PENDING_DBG_EXCEPTIONS ); + l2_shadow_vmcs->guest_sysenter_esp =vmcs_readl (GUEST_SYSENTER_ESP ); + l2_shadow_vmcs->guest_sysenter_eip =vmcs_readl (GUEST_SYSENTER_EIP ); +} + +int load_vmcs_common (struct shadow_vmcs *src) +{ +vmcs_write16 (GUEST_ES_SELECTOR , src->guest_es_selector); +vmcs_write16 (GUEST_CS_SELECTOR , src->guest_cs_selector); +vmcs_write16 (GUEST_SS_SELECTOR , src->guest_ss_selector); +vmcs_write16 (GUEST_DS_SELECTOR , src->guest_ds_selector); +vmcs_write16 (GUEST_FS_SELECTOR , src->guest_fs_selector); +vmcs_write16 (GUEST_GS_SELECTOR , src->guest_gs_selector); +vmcs_write16 (GUEST_LDTR_SELECTOR , src->guest_ldtr_selector); +vmcs_write16 (GUEST_TR_SELECTOR , src->guest_tr_selector); + +vmcs_write64 (GUEST_IA32_DEBUGCTL , src->guest_ia32_debugctl); + +if (vmcs_config.vmentry_ctrl &VM_ENTRY_LOAD_IA32_PAT ) +vmcs_write64 (GUEST_IA32_PAT , src->guest_ia32_pat); + +if (src->vm_entry_msr_load_count <512 ) +vmcs_write32 (VM_ENTRY_MSR_LOAD_COUNT , src->vm_entry_msr_load_count); + +vmcs_write32 (VM_ENTRY_INTR_INFO_FIELD , src->vm_entry_intr_info_field); +vmcs_write32 (VM_ENTRY_EXCEPTION_ERROR_CODE , + src->vm_entry_exception_error_code); +vmcs_write32 (VM_ENTRY_INSTRUCTION_LEN , src->vm_entry_instruction_len); + +vmcs_write32 (GUEST_ES_LIMIT , src->guest_es_limit); +vmcs_write32 (GUEST_CS_LIMIT , src->guest_cs_limit); +vmcs_write32 (GUEST_SS_LIMIT , src->guest_ss_limit); +vmcs_write32 (GUEST_DS_LIMIT , src->guest_ds_limit); +vmcs_write32 (GUEST_FS_LIMIT , src->guest_fs_limit); +vmcs_write32 (GUEST_GS_LIMIT , src->guest_gs_limit); +vmcs_write32 (GUEST_LDTR_LIMIT , src->guest_ldtr_limit); +vmcs_write32 (GUEST_TR_LIMIT , src->guest_tr_limit); +vmcs_write32 (GUEST_GDTR_LIMIT , src->guest_gdtr_limit); +vmcs_write32 (GUEST_IDTR_LIMIT , src->guest_idtr_limit); +vmcs_write32 (GUEST_ES_AR_BYTES , src->guest_es_ar_bytes); +vmcs_write32 (GUEST_CS_AR_BYTES , src->guest_cs_ar_bytes); +vmcs_write32 (GUEST_SS_AR_BYTES , src->guest_ss_ar_bytes); +vmcs_write32 (GUEST_DS_AR_BYTES , src->guest_ds_ar_bytes); +vmcs_write32 (GUEST_FS_AR_BYTES , src->guest_fs_ar_bytes); +vmcs_write32 (GUEST_GS_AR_BYTES , src->guest_gs_ar_bytes); +vmcs_write32 (GUEST_LDTR_AR_BYTES , src->guest_ldtr_ar_bytes); +vmcs_write32 (GUEST_TR_AR_BYTES , src->guest_tr_ar_bytes); +vmcs_write32 (GUEST_INTERRUPTIBILITY_INFO , + src->guest_interruptibility_info); +vmcs_write32 (GUEST_ACTIVITY_STATE , src->guest_activity_state); +vmcs_write32 (GUEST_SYSENTER_CS , src->guest_sysenter_cs); + +vmcs_writel (GUEST_ES_BASE , src->guest_es_base); +vmcs_writel (GUEST_CS_BASE , src->guest_cs_base); +vmcs_writel (GUEST_SS_BASE , src->guest_ss_base); +vmcs_writel (GUEST_DS_BASE , src->guest_ds_base); +vmcs_writel (GUEST_FS_BASE , src->guest_fs_base); +vmcs_writel (GUEST_GS_BASE , src->guest_gs_base); +vmcs_writel (GUEST_LDTR_BASE , src->guest_ldtr_base); +vmcs_writel (GUEST_TR_BASE , src->guest_tr_base); +vmcs_writel (GUEST_GDTR_BASE , src->guest_gdtr_base); +vmcs_writel (GUEST_IDTR_BASE , src->guest_idtr_base); +vmcs_writel (GUEST_DR7 , src->guest_dr7); +vmcs_writel (GUEST_RSP , src->guest_rsp); +vmcs_writel (GUEST_RIP , src->guest_rip); +vmcs_writel (GUEST_RFLAGS , src->guest_rflags); +vmcs_writel (GUEST_PENDING_DBG_EXCEPTIONS , + src->guest_pending_dbg_exceptions); +vmcs_writel (GUEST_SYSENTER_ESP , src->guest_sysenter_esp); +vmcs_writel (GUEST_SYSENTER_EIP , src->guest_sysenter_eip); + +return 0 ; +} + +int load_vmcs_host_state (struct shadow_vmcs *src) +{ +vmcs_write16 (HOST_ES_SELECTOR , src->host_es_selector); +vmcs_write16 (HOST_CS_SELECTOR , src->host_cs_selector); +vmcs_write16 (HOST_SS_SELECTOR , src->host_ss_selector); +vmcs_write16 (HOST_DS_SELECTOR , src->host_ds_selector); +vmcs_write16 (HOST_FS_SELECTOR , src->host_fs_selector); +vmcs_write16 (HOST_GS_SELECTOR , src->host_gs_selector); +vmcs_write16 (HOST_TR_SELECTOR , src->host_tr_selector); + +vmcs_write64 (TSC_OFFSET , src->tsc_offset); + +if (vmcs_config.vmexit_ctrl &VM_EXIT_LOAD_IA32_PAT ) +vmcs_write64 (HOST_IA32_PAT , src->host_ia32_pat); + +vmcs_write32 (HOST_IA32_SYSENTER_CS , src->host_ia32_sysenter_cs); + +vmcs_writel (HOST_CR0 , src->host_cr0); +vmcs_writel (HOST_CR3 , src->host_cr3); +vmcs_writel (HOST_CR4 , src->host_cr4); +vmcs_writel (HOST_FS_BASE , src->host_fs_base); +vmcs_writel (HOST_GS_BASE , src->host_gs_base); +vmcs_writel (HOST_TR_BASE , src->host_tr_base); +vmcs_writel (HOST_GDTR_BASE , src->host_gdtr_base); +vmcs_writel (HOST_IDTR_BASE , src->host_idtr_base); +vmcs_writel (HOST_RSP , src->host_rsp); +vmcs_writel (HOST_RIP , src->host_rip); +vmcs_writel (HOST_IA32_SYSENTER_ESP , src->host_ia32_sysenter_esp); +vmcs_writel (HOST_IA32_SYSENTER_EIP , src->host_ia32_sysenter_eip); + +return 0 ; +} +static struct level_state *create_state (void ) {struct level_state *state =NULL ;
@@ -2301,8 +2578,6 @@ static void free_l1_state(struct kvm_vcpu *vcpu)kfree (list_item); } } - -static void free_kvm_area (void ) {int cpu;
@@ -3574,6 +3849,10 @@ static void vmx_inject_nmi(struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +if (vmx->nested.nested_mode) { +return ; + } +if (!cpu_has_virtual_nmis ()) {/* * Tracking the NMI-blocked state in software is built upon
@@ -3759,7 +4038,12 @@ static int handle_exception(struct kvm_vcpu *vcpu)return 1 ;/* already handled by vmx_vcpu_run() */ if (is_no_device (intr_info)) { +/* if l0 handled an fpu operation for l2 it's because l1 is +not interested (exception bitmap 12 does not include NM_VECTOR) +enable fpu and resume l2 (avoid switching to l1) +*/ vmx_fpu_activate (vcpu); +return 1 ; }
@@ -4151,12 +4435,6 @@ static int nested_vmx_check_permission(struct kvm_vcpu *vcpu)return 1 ; } -static int handle_vmx_insn (struct kvm_vcpu *vcpu) -{ -kvm_queue_exception (vcpu,UD_VECTOR ); -return 1 ; -} -static void clear_rflags_cf_zf (struct kvm_vcpu *vcpu) {unsigned long rflags;
@@ -4264,6 +4542,17 @@ static int handle_vmclear(struct kvm_vcpu *vcpu)return 1 ; } +static int handle_vmlaunch (struct kvm_vcpu *vcpu) +{ +return launch_guest (vcpu, true); +} + +static int handle_vmresume (struct kvm_vcpu *vcpu) +{ + +return launch_guest (vcpu, false); +} +static int handle_vmread_reg (struct kvm_vcpu *vcpu,int reg,unsigned long field) {
@@ -4504,7 +4793,7 @@ static int handle_vmon(struct kvm_vcpu *vcpu)static int handle_vmptrld (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); -u64 guest_vmcs_addr; +gpa_t guest_vmcs_addr;gva_t vmcs_gva;unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION );u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO );
@@ -4847,11 +5136,11 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { [EXIT_REASON_INVLPG] = handle_invlpg, [EXIT_REASON_VMCALL] = handle_vmcall, [EXIT_REASON_VMCLEAR] = handle_vmclear, - [EXIT_REASON_VMLAUNCH] = handle_vmx_insn, + [EXIT_REASON_VMLAUNCH] = handle_vmlaunch, [EXIT_REASON_VMPTRLD] = handle_vmptrld, [EXIT_REASON_VMPTRST] = handle_vmptrst, [EXIT_REASON_VMREAD] = handle_vmread, - [EXIT_REASON_VMRESUME] = handle_vmx_insn, + [EXIT_REASON_VMRESUME] = handle_vmresume, [EXIT_REASON_VMWRITE] = handle_vmwrite, [EXIT_REASON_VMOFF] = handle_vmoff, [EXIT_REASON_VMON] = handle_vmon,
@@ -4895,7 +5184,6 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu) = vmcs_read32(VM_INSTRUCTION_ERROR);return 0 ; } - if ((vectoring_info & VECTORING_INFO_VALID_MASK) && (exit_reason != EXIT_REASON_EXCEPTION_NMI && exit_reason != EXIT_REASON_EPT_VIOLATION &&
@@ -4903,8 +5191,7 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu)printk (KERN_WARNING "%s: unexpected, valid vectoring info " "(0x%x) and exit reason is 0x%x\n" , __func__, vectoring_info, exit_reason); - -if (unlikely (!cpu_has_virtual_nmis () && vmx->soft_vnmi_blocked)) { +if (!vmx->nested.nested_mode &&unlikely (!cpu_has_virtual_nmis () && vmx->soft_vnmi_blocked)) {if (vmx_interrupt_allowed (vcpu)) { vmx->soft_vnmi_blocked =0 ; } else if (vmx->vnmi_blocked_time > 1000000000LL &&
@@ -4951,10 +5238,13 @@ static void vmx_complete_interrupts(struct vcpu_vmx *vmx)int type;bool idtv_info_valid; - exit_intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); - vmx->exit_reason =vmcs_read32 (VM_EXIT_REASON ); +if (vmx->nested.nested_mode) +return ; + + exit_intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); +/* Handle machine checks before interrupts are enabled */ if ((vmx->exit_reason == EXIT_REASON_MCE_DURING_VMENTRY) || (vmx->exit_reason == EXIT_REASON_EXCEPTION_NMI
@@ -5068,6 +5358,7 @@ static void fixup_rmode_irq(struct vcpu_vmx *vmx)static void vmx_vcpu_run (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +u32 nested_exception_bitmap =0 ;/* Record the guest's net vcpu time for enforced NMI injections. */ if (unlikely (!cpu_has_virtual_nmis () && vmx->soft_vnmi_blocked))
@@ -5099,6 +5390,37 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu)if (vcpu->arch.switch_db_regs)set_debugreg (vcpu->arch.dr6,6 ); +if (vcpu->fpu_active) { +if (vmcs_readl (CR0_READ_SHADOW ) &X86_CR0_TS ) +vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS ); +else +vmcs_clear_bits (GUEST_CR0 ,X86_CR0_TS ); + +if (vmx->nested.nested_mode) { +if (!nested_map_current (vcpu)) { + vmx->fail =1 ; +return ; + } + + nested_exception_bitmap =get_shadow_vmcs (vcpu)-> + exception_bitmap; + +nested_unmap_current (vcpu); + } + +if (vmx->nested.nested_mode && + (nested_exception_bitmap & (1u <<NM_VECTOR ))) +vmcs_write32 (EXCEPTION_BITMAP , +vmcs_read32 (EXCEPTION_BITMAP ) | (1u <<NM_VECTOR )); +else +vmcs_write32 (EXCEPTION_BITMAP , +vmcs_read32 (EXCEPTION_BITMAP ) & ~(1u <<NM_VECTOR )); + }else { +vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS ); +vmcs_write32 (EXCEPTION_BITMAP , +vmcs_read32 (EXCEPTION_BITMAP ) | (1u <<NM_VECTOR )); + } + asm(/* Store host registers */ "push %%"R"dx; push %%"R"bp;"
@@ -5216,6 +5538,7 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu) } vmx->idt_vectoring_info =vmcs_read32 (IDT_VECTORING_INFO_FIELD ); +if (vmx->rmode.irq.pending)fixup_rmode_irq (vmx);
@@ -5300,6 +5623,11 @@ static struct kvm_vcpu *vmx_create_vcpu(struct kvm *kvm, unsigned int id) goto free_vmcs; } + vmx->nested.current_vmptr =-1ull ; + + vmx->nested.l1_state =NULL ; + vmx->nested.current_l2_page =NULL ; +return &vmx->vcpu; free_vmcs:
@@ -5388,6 +5716,534 @@ static bool vmx_gb_page_enable(void)return false; } +void save_vmcs (struct shadow_vmcs *dst) +{ + dst->guest_es_selector =vmcs_read16 (GUEST_ES_SELECTOR ); + dst->guest_cs_selector =vmcs_read16 (GUEST_CS_SELECTOR ); + dst->guest_ss_selector =vmcs_read16 (GUEST_SS_SELECTOR ); + dst->guest_ds_selector =vmcs_read16 (GUEST_DS_SELECTOR ); + dst->guest_fs_selector =vmcs_read16 (GUEST_FS_SELECTOR ); + dst->guest_gs_selector =vmcs_read16 (GUEST_GS_SELECTOR ); + dst->guest_ldtr_selector =vmcs_read16 (GUEST_LDTR_SELECTOR ); + dst->guest_tr_selector =vmcs_read16 (GUEST_TR_SELECTOR ); + dst->host_es_selector =vmcs_read16 (HOST_ES_SELECTOR ); + dst->host_cs_selector =vmcs_read16 (HOST_CS_SELECTOR ); + dst->host_ss_selector =vmcs_read16 (HOST_SS_SELECTOR ); + dst->host_ds_selector =vmcs_read16 (HOST_DS_SELECTOR ); + dst->host_fs_selector =vmcs_read16 (HOST_FS_SELECTOR ); + dst->host_gs_selector =vmcs_read16 (HOST_GS_SELECTOR ); + dst->host_tr_selector =vmcs_read16 (HOST_TR_SELECTOR ); + dst->io_bitmap_a =vmcs_read64 (IO_BITMAP_A ); + dst->io_bitmap_b =vmcs_read64 (IO_BITMAP_B ); +if (cpu_has_vmx_msr_bitmap ()) + dst->msr_bitmap =vmcs_read64 (MSR_BITMAP ); + + dst->vm_exit_msr_store_addr =vmcs_read64 (VM_EXIT_MSR_STORE_ADDR ); + dst->vm_exit_msr_load_addr =vmcs_read64 (VM_EXIT_MSR_LOAD_ADDR ); + dst->vm_entry_msr_load_addr =vmcs_read64 (VM_ENTRY_MSR_LOAD_ADDR ); + dst->tsc_offset =vmcs_read64 (TSC_OFFSET ); + dst->virtual_apic_page_addr =vmcs_read64 (VIRTUAL_APIC_PAGE_ADDR ); + dst->apic_access_addr =vmcs_read64 (APIC_ACCESS_ADDR ); +if (enable_ept) + dst->ept_pointer =vmcs_read64 (EPT_POINTER ); + + dst->guest_physical_address =vmcs_read64 (GUEST_PHYSICAL_ADDRESS ); + dst->vmcs_link_pointer =vmcs_read64 (VMCS_LINK_POINTER ); + dst->guest_ia32_debugctl =vmcs_read64 (GUEST_IA32_DEBUGCTL ); +if (vmcs_config.vmentry_ctrl &VM_ENTRY_LOAD_IA32_PAT ) + dst->guest_ia32_pat =vmcs_read64 (GUEST_IA32_PAT ); +if (enable_ept) { + dst->guest_pdptr0 =vmcs_read64 (GUEST_PDPTR0 ); + dst->guest_pdptr1 =vmcs_read64 (GUEST_PDPTR1 ); + dst->guest_pdptr2 =vmcs_read64 (GUEST_PDPTR2 ); + dst->guest_pdptr3 =vmcs_read64 (GUEST_PDPTR3 ); + } + dst->pin_based_vm_exec_control =vmcs_read32 (PIN_BASED_VM_EXEC_CONTROL ); + dst->cpu_based_vm_exec_control =vmcs_read32 (CPU_BASED_VM_EXEC_CONTROL ); + dst->exception_bitmap =vmcs_read32 (EXCEPTION_BITMAP ); + dst->page_fault_error_code_mask = +vmcs_read32 (PAGE_FAULT_ERROR_CODE_MASK ); + dst->page_fault_error_code_match = +vmcs_read32 (PAGE_FAULT_ERROR_CODE_MATCH ); + dst->cr3_target_count =vmcs_read32 (CR3_TARGET_COUNT ); + dst->vm_exit_controls =vmcs_read32 (VM_EXIT_CONTROLS ); + dst->vm_exit_msr_store_count =vmcs_read32 (VM_EXIT_MSR_STORE_COUNT ); + dst->vm_exit_msr_load_count =vmcs_read32 (VM_EXIT_MSR_LOAD_COUNT ); + dst->vm_entry_controls =vmcs_read32 (VM_ENTRY_CONTROLS ); + dst->vm_entry_msr_load_count =vmcs_read32 (VM_ENTRY_MSR_LOAD_COUNT ); + dst->vm_entry_intr_info_field =vmcs_read32 (VM_ENTRY_INTR_INFO_FIELD ); + dst->vm_entry_exception_error_code = +vmcs_read32 (VM_ENTRY_EXCEPTION_ERROR_CODE ); + dst->vm_entry_instruction_len =vmcs_read32 (VM_ENTRY_INSTRUCTION_LEN ); + dst->tpr_threshold =vmcs_read32 (TPR_THRESHOLD ); + dst->secondary_vm_exec_control =vmcs_read32 (SECONDARY_VM_EXEC_CONTROL ); +if (enable_vpid && dst->secondary_vm_exec_control & +SECONDARY_EXEC_ENABLE_VPID ) + dst->virtual_processor_id =vmcs_read16 (VIRTUAL_PROCESSOR_ID ); + dst->vm_instruction_error =vmcs_read32 (VM_INSTRUCTION_ERROR ); + dst->vm_exit_reason =vmcs_read32 (VM_EXIT_REASON ); + dst->vm_exit_intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); + dst->vm_exit_intr_error_code =vmcs_read32 (VM_EXIT_INTR_ERROR_CODE ); + dst->idt_vectoring_info_field =vmcs_read32 (IDT_VECTORING_INFO_FIELD ); + dst->idt_vectoring_error_code =vmcs_read32 (IDT_VECTORING_ERROR_CODE ); + dst->vm_exit_instruction_len =vmcs_read32 (VM_EXIT_INSTRUCTION_LEN ); + dst->vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO ); + dst->guest_es_limit =vmcs_read32 (GUEST_ES_LIMIT ); + dst->guest_cs_limit =vmcs_read32 (GUEST_CS_LIMIT ); + dst->guest_ss_limit =vmcs_read32 (GUEST_SS_LIMIT ); + dst->guest_ds_limit =vmcs_read32 (GUEST_DS_LIMIT ); + dst->guest_fs_limit =vmcs_read32 (GUEST_FS_LIMIT ); + dst->guest_gs_limit =vmcs_read32 (GUEST_GS_LIMIT ); + dst->guest_ldtr_limit =vmcs_read32 (GUEST_LDTR_LIMIT ); + dst->guest_tr_limit =vmcs_read32 (GUEST_TR_LIMIT ); + dst->guest_gdtr_limit =vmcs_read32 (GUEST_GDTR_LIMIT ); + dst->guest_idtr_limit =vmcs_read32 (GUEST_IDTR_LIMIT ); + dst->guest_es_ar_bytes =vmcs_read32 (GUEST_ES_AR_BYTES ); + dst->guest_cs_ar_bytes =vmcs_read32 (GUEST_CS_AR_BYTES ); + dst->guest_ss_ar_bytes =vmcs_read32 (GUEST_SS_AR_BYTES ); + dst->guest_ds_ar_bytes =vmcs_read32 (GUEST_DS_AR_BYTES ); + dst->guest_fs_ar_bytes =vmcs_read32 (GUEST_FS_AR_BYTES ); + dst->guest_gs_ar_bytes =vmcs_read32 (GUEST_GS_AR_BYTES ); + dst->guest_ldtr_ar_bytes =vmcs_read32 (GUEST_LDTR_AR_BYTES ); + dst->guest_tr_ar_bytes =vmcs_read32 (GUEST_TR_AR_BYTES ); + dst->guest_interruptibility_info = +vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ); + dst->guest_activity_state =vmcs_read32 (GUEST_ACTIVITY_STATE ); + dst->guest_sysenter_cs =vmcs_read32 (GUEST_SYSENTER_CS ); + dst->host_ia32_sysenter_cs =vmcs_read32 (HOST_IA32_SYSENTER_CS ); + dst->cr0_guest_host_mask =vmcs_readl (CR0_GUEST_HOST_MASK ); + dst->cr4_guest_host_mask =vmcs_readl (CR4_GUEST_HOST_MASK ); + dst->cr0_read_shadow =vmcs_readl (CR0_READ_SHADOW ); + dst->cr4_read_shadow =vmcs_readl (CR4_READ_SHADOW ); + dst->cr3_target_value0 =vmcs_readl (CR3_TARGET_VALUE0 ); + dst->cr3_target_value1 =vmcs_readl (CR3_TARGET_VALUE1 ); + dst->cr3_target_value2 =vmcs_readl (CR3_TARGET_VALUE2 ); + dst->cr3_target_value3 =vmcs_readl (CR3_TARGET_VALUE3 ); + dst->exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); + dst->guest_linear_address =vmcs_readl (GUEST_LINEAR_ADDRESS ); + dst->guest_cr0 =vmcs_readl (GUEST_CR0 ); + dst->guest_cr3 =vmcs_readl (GUEST_CR3 ); + dst->guest_cr4 =vmcs_readl (GUEST_CR4 ); + dst->guest_es_base =vmcs_readl (GUEST_ES_BASE ); + dst->guest_cs_base =vmcs_readl (GUEST_CS_BASE ); + dst->guest_ss_base =vmcs_readl (GUEST_SS_BASE ); + dst->guest_ds_base =vmcs_readl (GUEST_DS_BASE ); + dst->guest_fs_base =vmcs_readl (GUEST_FS_BASE ); + dst->guest_gs_base =vmcs_readl (GUEST_GS_BASE ); + dst->guest_ldtr_base =vmcs_readl (GUEST_LDTR_BASE ); + dst->guest_tr_base =vmcs_readl (GUEST_TR_BASE ); + dst->guest_gdtr_base =vmcs_readl (GUEST_GDTR_BASE ); + dst->guest_idtr_base =vmcs_readl (GUEST_IDTR_BASE ); + dst->guest_dr7 =vmcs_readl (GUEST_DR7 ); + dst->guest_rsp =vmcs_readl (GUEST_RSP ); + dst->guest_rip =vmcs_readl (GUEST_RIP ); + dst->guest_rflags =vmcs_readl (GUEST_RFLAGS ); + dst->guest_pending_dbg_exceptions = +vmcs_readl (GUEST_PENDING_DBG_EXCEPTIONS ); + dst->guest_sysenter_esp =vmcs_readl (GUEST_SYSENTER_ESP ); + dst->guest_sysenter_eip =vmcs_readl (GUEST_SYSENTER_EIP ); + dst->host_cr0 =vmcs_readl (HOST_CR0 ); + dst->host_cr3 =vmcs_readl (HOST_CR3 ); + dst->host_cr4 =vmcs_readl (HOST_CR4 ); + dst->host_fs_base =vmcs_readl (HOST_FS_BASE ); + dst->host_gs_base =vmcs_readl (HOST_GS_BASE ); + dst->host_tr_base =vmcs_readl (HOST_TR_BASE ); + dst->host_gdtr_base =vmcs_readl (HOST_GDTR_BASE ); + dst->host_idtr_base =vmcs_readl (HOST_IDTR_BASE ); + dst->host_ia32_sysenter_esp =vmcs_readl (HOST_IA32_SYSENTER_ESP ); + dst->host_ia32_sysenter_eip =vmcs_readl (HOST_IA32_SYSENTER_EIP ); + dst->host_rsp =vmcs_readl (HOST_RSP ); + dst->host_rip =vmcs_readl (HOST_RIP ); +if (vmcs_config.vmexit_ctrl &VM_EXIT_LOAD_IA32_PAT ) + dst->host_ia32_pat =vmcs_read64 (HOST_IA32_PAT ); +} + +int prepare_vmcs_02 (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct shadow_vmcs *src , *l1_shadow_vmcs = vmx->nested.l1_shadow_vmcs; +struct level_state *l2_state; +u32 exec_control; + + src =get_shadow_vmcs (vcpu); +if (!src) { +nested_unmap_current (vcpu); +printk (KERN_INFO "%s: Error no shadow vmcs\n" , __func__); +return 1 ; + } + +load_vmcs_common (src); + + l2_state = &(vmx->nested.current_l2_page->l2_state); + +if (l2_state->first_launch) { + +vmcs_write64 (VMCS_LINK_POINTER , src->vmcs_link_pointer); + +if (l2_state->io_bitmap_a) +vmcs_write64 (IO_BITMAP_A , l2_state->io_bitmap_a); + +if (l2_state->io_bitmap_b) +vmcs_write64 (IO_BITMAP_B , l2_state->io_bitmap_b); + +if (l2_state->msr_bitmap) +vmcs_write64 (MSR_BITMAP , l2_state->msr_bitmap); + +if (src->vm_entry_msr_load_count >0 ) { +struct page *page; + + page =nested_get_page (vcpu, + src->vm_entry_msr_load_addr); +if (!page) +return 1 ; + +vmcs_write64 (VM_ENTRY_MSR_LOAD_ADDR ,page_to_phys (page)); + +kvm_release_page_clean (page); + } + +if (nested_cpu_has_vmx_tpr_shadow (vcpu)) { +struct page *page; + + page =nested_get_page (vcpu, + src->virtual_apic_page_addr); +if (!page) +return 1 ; + +vmcs_write64 (VIRTUAL_APIC_PAGE_ADDR ,page_to_phys (page)); + +kvm_release_page_clean (page); + } + +if (nested_vm_need_virtualize_apic_accesses (vcpu)) { +struct page *page = +nested_get_page (vcpu, src->apic_access_addr); +if (!page) +return 1 ; + +vmcs_write64 (APIC_ACCESS_ADDR ,page_to_phys (page)); +kvm_release_page_clean (page); + } + + + +vmcs_write32 (PIN_BASED_VM_EXEC_CONTROL , + (l1_shadow_vmcs->pin_based_vm_exec_control | + src->pin_based_vm_exec_control)); + +vmcs_write32 (PAGE_FAULT_ERROR_CODE_MASK , + (l1_shadow_vmcs->page_fault_error_code_mask & + src->page_fault_error_code_mask)); + +vmcs_write32 (PAGE_FAULT_ERROR_CODE_MATCH , + (l1_shadow_vmcs->page_fault_error_code_match & + src->page_fault_error_code_match)); + +if (cpu_has_secondary_exec_ctrls ()) { + + exec_control = + l1_shadow_vmcs->secondary_vm_exec_control; + +if (nested_cpu_has_secondary_exec_ctrls (vcpu)) { + + exec_control |= src->secondary_vm_exec_control; + +if (!vm_need_virtualize_apic_accesses (vcpu->kvm) || + !nested_vm_need_virtualize_apic_accesses (vcpu)) + exec_control &= + ~SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES ; + } + +vmcs_write32 (SECONDARY_VM_EXEC_CONTROL , exec_control); + } + +load_vmcs_host_state (l1_shadow_vmcs); + + l2_state->first_launch = false; + } + +if (vm_need_tpr_shadow (vcpu->kvm) && +nested_cpu_has_vmx_tpr_shadow (vcpu)) +vmcs_write32 (TPR_THRESHOLD , src->tpr_threshold); + +if (enable_ept) { +if (!nested_cpu_has_vmx_ept (vcpu)) { +vmcs_write64 (EPT_POINTER , + l1_shadow_vmcs->ept_pointer); +vmcs_write64 (GUEST_PDPTR0 , + l1_shadow_vmcs->guest_pdptr0); +vmcs_write64 (GUEST_PDPTR1 , + l1_shadow_vmcs->guest_pdptr1); +vmcs_write64 (GUEST_PDPTR2 , + l1_shadow_vmcs->guest_pdptr2); +vmcs_write64 (GUEST_PDPTR3 , + l1_shadow_vmcs->guest_pdptr3); + } + } + + exec_control = l1_shadow_vmcs->cpu_based_vm_exec_control; + + exec_control &= ~CPU_BASED_VIRTUAL_INTR_PENDING ; + + exec_control &= ~CPU_BASED_VIRTUAL_NMI_PENDING ; + + exec_control &= ~CPU_BASED_TPR_SHADOW ; + + exec_control |= src->cpu_based_vm_exec_control; + +if (!vm_need_tpr_shadow (vcpu->kvm) || + src->virtual_apic_page_addr ==0 ) { + exec_control &= ~CPU_BASED_TPR_SHADOW ; +#ifdef CONFIG_X86_64 + exec_control |=CPU_BASED_CR8_STORE_EXITING | +CPU_BASED_CR8_LOAD_EXITING ; +#endif + }else if (exec_control &CPU_BASED_TPR_SHADOW ) { + +#ifdef CONFIG_X86_64 + exec_control &= ~CPU_BASED_CR8_STORE_EXITING ; + exec_control &= ~CPU_BASED_CR8_LOAD_EXITING ; +#endif + } + +vmcs_write32 (CPU_BASED_VM_EXEC_CONTROL , exec_control); + +vmcs_write32 (EXCEPTION_BITMAP , + (l1_shadow_vmcs->exception_bitmap | + src->exception_bitmap)); + +vmcs_write32 (VM_EXIT_CONTROLS , + ((l1_shadow_vmcs->vm_exit_controls & +NESTED_VM_EXIT_CONTROLS_MASK ) | src->vm_exit_controls)); + +vmcs_write32 (VM_ENTRY_CONTROLS , + (l1_shadow_vmcs->vm_entry_controls & +NESTED_VM_ENTRY_CONTROLS_MASK ) | src->vm_entry_controls); + +vmcs_write32 (VM_ENTRY_MSR_LOAD_COUNT , src->vm_entry_msr_load_count); + +vmcs_writel (CR0_GUEST_HOST_MASK , + src->cr0_guest_host_mask |X86_CR0_TS ); +vmcs_writel (CR4_GUEST_HOST_MASK , + (l1_shadow_vmcs->cr4_guest_host_mask & + src->cr4_guest_host_mask)); + +return 0 ; +} + +int switch_back_vmcs (struct kvm_vcpu *vcpu) +{ +struct shadow_vmcs *src =to_vmx (vcpu)->nested.l1_shadow_vmcs; + +if (enable_vpid && src->virtual_processor_id !=0 ) +vmcs_write16 (VIRTUAL_PROCESSOR_ID , src->virtual_processor_id); + +vmcs_write64 (IO_BITMAP_A , src->io_bitmap_a); +vmcs_write64 (IO_BITMAP_B , src->io_bitmap_b); + +if (cpu_has_vmx_msr_bitmap ()) +vmcs_write64 (MSR_BITMAP , src->msr_bitmap); + +vmcs_write64 (VIRTUAL_APIC_PAGE_ADDR , src->virtual_apic_page_addr); + +if (vm_need_virtualize_apic_accesses (vcpu->kvm)) +vmcs_write64 (APIC_ACCESS_ADDR , + src->apic_access_addr); + +if (enable_ept) { +vmcs_write64 (EPT_POINTER , src->ept_pointer); +vmcs_write64 (GUEST_PDPTR0 , src->guest_pdptr0); +vmcs_write64 (GUEST_PDPTR1 , src->guest_pdptr1); +vmcs_write64 (GUEST_PDPTR2 , src->guest_pdptr2); +vmcs_write64 (GUEST_PDPTR3 , src->guest_pdptr3); + } + +vmcs_write32 (PIN_BASED_VM_EXEC_CONTROL , src->pin_based_vm_exec_control); +vmcs_write32 (CPU_BASED_VM_EXEC_CONTROL , src->cpu_based_vm_exec_control); +vmcs_write32 (EXCEPTION_BITMAP , src->exception_bitmap); +vmcs_write32 (PAGE_FAULT_ERROR_CODE_MASK , + src->page_fault_error_code_mask); +vmcs_write32 (PAGE_FAULT_ERROR_CODE_MATCH , + src->page_fault_error_code_match); +vmcs_write32 (VM_EXIT_CONTROLS , src->vm_exit_controls); +vmcs_write32 (VM_ENTRY_CONTROLS , src->vm_entry_controls); +vmcs_write32 (VM_ENTRY_MSR_LOAD_COUNT , src->vm_entry_msr_load_count); + +if (cpu_has_secondary_exec_ctrls ()) +vmcs_write32 (SECONDARY_VM_EXEC_CONTROL , + src->secondary_vm_exec_control); + +load_vmcs_common (src); + +load_vmcs_host_state (to_vmx (vcpu)->nested.l1_shadow_vmcs); + +return 0 ; +} + +void sync_cached_regs_to_vmcs (struct kvm_vcpu *vcpu) +{ +unsigned long mask; + +if (test_bit (VCPU_REGS_RSP , (unsigned long *)&vcpu->arch.regs_dirty)) +vmcs_writel (GUEST_RSP , vcpu->arch.regs[VCPU_REGS_RSP ]); +if (test_bit (VCPU_REGS_RIP , (unsigned long *)&vcpu->arch.regs_dirty)) +vmcs_writel (GUEST_RIP , vcpu->arch.regs[VCPU_REGS_RIP ]); + + mask = ~((1 <<VCPU_REGS_RSP ) | (1 <<VCPU_REGS_RIP )); + +if (vcpu->arch.regs_dirty & mask) { +printk (KERN_INFO "WARNING: dirty cached registers regs_dirty 0x%x mask 0x%lx\n" , + vcpu->arch.regs_dirty, mask); +WARN_ON (1 ); + } + + vcpu->arch.regs_dirty =0 ; +} + +static int nested_vmx_run (struct kvm_vcpu *vcpu) +{ +/* verify that l1 has done vmptrld for l2 earlier */ +struct vcpu_vmx *vmx =to_vmx (vcpu); + +int r =0 ; +struct level_state *l2_state; + + vmx->nested.nested_mode =1 ; + +sync_cached_regs_to_vmcs (vcpu); + +save_vmcs (vmx->nested.l1_shadow_vmcs); + + vmx->nested.l1_state->shadow_efer = vcpu->arch.shadow_efer; +if (!enable_ept) + vmx->nested.l1_state->cr3 = vcpu->arch.cr3; + vmx->nested.l1_state->cr4 = vcpu->arch.cr4; + +if (!nested_map_current (vcpu)) { +set_rflags_to_vmx_fail_valid (vcpu); +return 1 ; + } + + l2_state = &(vmx->nested.current_l2_page->l2_state); + +if (cpu_has_vmx_msr_bitmap ()) + vmx->nested.l1_state->msr_bitmap =vmcs_read64 (MSR_BITMAP ); +else + vmx->nested.l1_state->msr_bitmap =0 ; + + vmx->nested.l1_state->io_bitmap_a =vmcs_read64 (IO_BITMAP_A ); + vmx->nested.l1_state->io_bitmap_b =vmcs_read64 (IO_BITMAP_B ); + vmx->nested.l1_vmcs = vmx->vmcs; + vmx->nested.l1_state->cpu = vcpu->cpu; + vmx->nested.l1_state->launched = vmx->launched; + + vmx->vmcs =nested_get_current_vmcs (vcpu); +if (!vmx->vmcs) { +printk (KERN_ERR "Missing VMCS\n" ); +set_rflags_to_vmx_fail_valid (vcpu); +return 1 ; + } + + vcpu->cpu = l2_state->cpu; + vmx->launched = l2_state->launched; + +if (l2_state->vmclear || !vmx->launched) { +vmcs_clear (vmx->vmcs); + vmx->launched =0 ; + l2_state->vmclear =0 ; + } + +vmx_vcpu_load (vcpu,get_cpu ()); +put_cpu (); + +prepare_vmcs_02 (vcpu); + +if (get_shadow_vmcs (vcpu)->vm_entry_controls & +VM_ENTRY_IA32E_MODE ) { +if (!((vcpu->arch.shadow_efer &EFER_LMA ) && + (vcpu->arch.shadow_efer &EFER_LME ))) + vcpu->arch.shadow_efer |= (EFER_LMA |EFER_LME ); + }else { +if ((vcpu->arch.shadow_efer &EFER_LMA ) || + (vcpu->arch.shadow_efer &EFER_LME )) + vcpu->arch.shadow_efer =0 ; + } + +vmx_set_cr0 (vcpu, + (get_shadow_vmcs (vcpu)->guest_cr0 & ~get_shadow_vmcs (vcpu)->cr0_guest_host_mask) | + (get_shadow_vmcs (vcpu)->cr0_read_shadow &get_shadow_vmcs (vcpu)->cr0_guest_host_mask)); + +vmx_set_cr4 (vcpu,get_shadow_vmcs (vcpu)->guest_cr4); +vmcs_writel (CR4_READ_SHADOW , +get_shadow_vmcs (vcpu)->cr4_read_shadow); + + vcpu->arch.cr0 |=X86_CR0_PG ; + +if (enable_ept && !nested_cpu_has_vmx_ept (vcpu)) { +vmcs_write32 (GUEST_CR3 ,get_shadow_vmcs (vcpu)->guest_cr3); + vmx->vcpu.arch.cr3 =get_shadow_vmcs (vcpu)->guest_cr3; + }else { +kvm_set_cr3 (vcpu,get_shadow_vmcs (vcpu)->guest_cr3); +kvm_mmu_reset_context (vcpu); + +nested_unmap_current (vcpu); + + r =kvm_mmu_load (vcpu); +if (unlikely (r)) { +printk (KERN_ERR "Error in kvm_mmu_load r %d\n" , r); +set_rflags_to_vmx_fail_valid (vcpu); +/* switch back to L1 */ + vmx->nested.nested_mode =0 ; + vmx->vmcs = vmx->nested.l1_vmcs; + vcpu->cpu = vmx->nested.l1_state->cpu; + vmx->launched = vmx->nested.l1_state->launched; + +vmx_vcpu_load (vcpu,get_cpu ()); +put_cpu (); + +return 1 ; + } + +nested_map_current (vcpu); + } + +kvm_register_write (vcpu,VCPU_REGS_RSP , +get_shadow_vmcs (vcpu)->guest_rsp); +kvm_register_write (vcpu,VCPU_REGS_RIP , +get_shadow_vmcs (vcpu)->guest_rip); + +vmcs_write32 (EXCEPTION_BITMAP , + (vmx->nested.l1_shadow_vmcs->exception_bitmap | +get_shadow_vmcs (vcpu)->exception_bitmap)); + +nested_unmap_current (vcpu); + +return 1 ; +} + +static int launch_guest (struct kvm_vcpu *vcpu,bool launch) +{ +if (!nested_vmx_check_permission (vcpu)) +return 1 ; + +if (!nested_map_current (vcpu)) +return 1 ; + +if (to_vmx (vcpu)->nested.current_l2_page->l2_state.vmclear != launch) { +set_rflags_to_vmx_fail_valid (vcpu); +nested_unmap_current (vcpu); +return 1 ; + } + +nested_unmap_current (vcpu); + +skip_emulated_instruction (vcpu); + +nested_vmx_run (vcpu); + +return 1 ; +} +static struct kvm_x86_ops vmx_x86_ops = { .cpu_has_kvm_support = cpu_has_kvm_support, .disabled_by_bios = vmx_disabled_by_bios,
Signature · 1 line
1.6.0.4
PATCH 7/7 Nested VMX patch 7 handling of nested guest exits
From: Orit Wasserman <[email protected]>
--- arch/x86/kvm/vmx.c | 521 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 515 insertions(+), 6 deletions(-)
arch/x86/kvm/vmx.c +515 -6
@@ -262,6 +262,10 @@ struct nested_vmx {gpa_t current_vmptr;/* Are we running nested guest */ bool nested_mode; +/* L1 requested VMLAUNCH or VMRESUME but we didn't run L2 yet */ +bool nested_run_pending; +/* flag indicating if there was a valid IDT after exiting from l2 */ +bool valid_idt_vectoring_info;/* Level 1 state for switching to level 2 and back */ struct level_state *l1_state;/* Level 1 shadow vmcs for switching to level 2 and back */
@@ -908,9 +912,16 @@ static u64 host_efer;static void ept_save_pdptrs (struct kvm_vcpu *vcpu);static int nested_vmx_check_permission (struct kvm_vcpu *vcpu); +static int nested_vmx_check_exception (struct vcpu_vmx *vmx,unsigned nr, +bool has_error_code,u32 error_code); +static int nested_vmx_intr (struct kvm_vcpu *vcpu);static int create_l1_state (struct kvm_vcpu *vcpu);static int create_l2_state (struct kvm_vcpu *vcpu);static int launch_guest (struct kvm_vcpu *vcpu,bool launch); +static int nested_vmx_exit_handled_msr (struct kvm_vcpu *vcpu); +static int nested_vmx_exit_handled (struct kvm_vcpu *vcpu,bool kvm_override); +static int nested_vmx_vmexit (struct kvm_vcpu *vcpu, +bool is_interrupt);/* * Keep MSR_K6_STAR at the end, as setup_msrs() will try to optimize it
@@ -1467,6 +1478,8 @@ static void vmx_load_host_state(struct vcpu_vmx *vmx)static void vmx_fpu_deactivate (struct kvm_vcpu *vcpu); +int load_vmcs_host_state (struct shadow_vmcs *src); +/* * Switches to specified vcpu, until a matching vcpu_put(), but assumes * vcpu mutex is already taken.
@@ -1503,6 +1516,7 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu)if (vcpu->cpu != cpu) {struct descriptor_table dt;unsigned long sysenter_esp; +struct shadow_vmcs *l1_shadow_vmcs = vmx->nested.l1_shadow_vmcs; vcpu->cpu = cpu;/*
@@ -1525,6 +1539,22 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) new_offset =vmcs_read64 (TSC_OFFSET ) + delta;vmcs_write64 (TSC_OFFSET , new_offset); } + +if (l1_shadow_vmcs !=NULL ) { + l1_shadow_vmcs->host_tr_base = +vmcs_readl (HOST_TR_BASE ); + l1_shadow_vmcs->host_gdtr_base = +vmcs_readl (HOST_GDTR_BASE ); + l1_shadow_vmcs->host_ia32_sysenter_esp = +vmcs_readl (HOST_IA32_SYSENTER_ESP ); + +if (tsc_this < vcpu->arch.host_tsc) + l1_shadow_vmcs->tsc_offset = +vmcs_read64 (TSC_OFFSET ); + +if (vmx->nested.nested_mode) +load_vmcs_host_state (l1_shadow_vmcs); + } } }
@@ -1611,6 +1641,9 @@ static void vmx_queue_exception(struct kvm_vcpu *vcpu, unsigned nr,struct vcpu_vmx *vmx =to_vmx (vcpu);u32 intr_info = nr |INTR_INFO_VALID_MASK ; +if (nested_vmx_check_exception (vmx, nr, has_error_code, error_code)) +return ; +if (has_error_code) {vmcs_write32 (VM_ENTRY_EXCEPTION_ERROR_CODE , error_code); intr_info |=INTR_INFO_DELIVER_CODE_MASK ;
@@ -2185,9 +2218,6 @@ int load_vmcs_common(struct shadow_vmcs *src)if (vmcs_config.vmentry_ctrl &VM_ENTRY_LOAD_IA32_PAT )vmcs_write64 (GUEST_IA32_PAT , src->guest_ia32_pat); -if (src->vm_entry_msr_load_count <512 ) -vmcs_write32 (VM_ENTRY_MSR_LOAD_COUNT , src->vm_entry_msr_load_count); -vmcs_write32 (VM_ENTRY_INTR_INFO_FIELD , src->vm_entry_intr_info_field);vmcs_write32 (VM_ENTRY_EXCEPTION_ERROR_CODE , src->vm_entry_exception_error_code);
@@ -3794,6 +3824,11 @@ static void enable_irq_window(struct kvm_vcpu *vcpu) {u32 cpu_based_vm_exec_control; +if (to_vmx (vcpu)->nested.nested_mode) { +nested_vmx_intr (vcpu); +return ; + } + cpu_based_vm_exec_control =vmcs_read32 (CPU_BASED_VM_EXEC_CONTROL ); cpu_based_vm_exec_control |=CPU_BASED_VIRTUAL_INTR_PENDING ;vmcs_write32 (CPU_BASED_VM_EXEC_CONTROL , cpu_based_vm_exec_control);
@@ -3922,6 +3957,11 @@ static void vmx_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked)static int vmx_interrupt_allowed (struct kvm_vcpu *vcpu) { +if (to_vmx (vcpu)->nested.nested_mode) { +if (!nested_vmx_intr (vcpu)) +return 0 ; + } +return (vmcs_readl (GUEST_RFLAGS ) &X86_EFLAGS_IF ) && !(vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ) & (GUEST_INTR_STATE_STI |GUEST_INTR_STATE_MOV_SS ));
@@ -4042,6 +4082,10 @@ static int handle_exception(struct kvm_vcpu *vcpu)not interested (exception bitmap 12 does not include NM_VECTOR) enable fpu and resume l2 (avoid switching to l1) */ + +if (vmx->nested.nested_mode) + vmx->nested.nested_run_pending =1 ;/* removing this line cause hung on boot of l2*/ +vmx_fpu_activate (vcpu);return 1 ;
@@ -4169,7 +4213,33 @@ static int handle_cr(struct kvm_vcpu *vcpu)trace_kvm_cr_write (cr, val);switch (cr) {case 0 : -kvm_set_cr0 (vcpu, val); +if (to_vmx (vcpu)->nested.nested_mode) { +/* assume only X86_CR0_TS is handled by l0 */ +long new_cr0 =vmcs_readl (GUEST_CR0 ); +long new_cr0_read_shadow =vmcs_readl (CR0_READ_SHADOW ); + +vmx_fpu_deactivate (vcpu); + +if (val &X86_CR0_TS ) { + new_cr0 |=X86_CR0_TS ; + new_cr0_read_shadow |=X86_CR0_TS ; + vcpu->arch.cr0 |=X86_CR0_TS ; + }else { + new_cr0 &= ~X86_CR0_TS ; + new_cr0_read_shadow &= ~X86_CR0_TS ; + vcpu->arch.cr0 &=X86_CR0_TS ; + } + +vmcs_writel (GUEST_CR0 , new_cr0); +vmcs_writel (CR0_READ_SHADOW , new_cr0_read_shadow); + +if (!(val &X86_CR0_TS ) || !(val &X86_CR0_PE )) +vmx_fpu_activate (vcpu); + +to_vmx (vcpu)->nested.nested_run_pending =1 ; + }else +kvm_set_cr0 (vcpu, val); +skip_emulated_instruction (vcpu);return 1 ;case 3 :
@@ -4196,8 +4266,15 @@ static int handle_cr(struct kvm_vcpu *vcpu)break ;case 2 :/* clts */ vmx_fpu_deactivate (vcpu); - vcpu->arch.cr0 &= ~X86_CR0_TS ; -vmcs_writel (CR0_READ_SHADOW , vcpu->arch.cr0); +if (to_vmx (vcpu)->nested.nested_mode) { +vmcs_writel (GUEST_CR0 ,vmcs_readl (GUEST_CR0 ) & ~X86_CR0_TS ); +vmcs_writel (CR0_READ_SHADOW ,vmcs_readl (CR0_READ_SHADOW ) & ~X86_CR0_TS ); + vcpu->arch.cr0 &= ~X86_CR0_TS ; +to_vmx (vcpu)->nested.nested_run_pending =1 ; + }else { + vcpu->arch.cr0 &= ~X86_CR0_TS ; +vmcs_writel (CR0_READ_SHADOW , vcpu->arch.cr0); + }vmx_fpu_activate (vcpu);skip_emulated_instruction (vcpu);return 1 ;
@@ -5173,6 +5250,17 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu)if (vmx->emulation_required && emulate_invalid_guest_state)return handle_invalid_guest_state (vcpu); +if (exit_reason ==EXIT_REASON_VMLAUNCH || + exit_reason ==EXIT_REASON_VMRESUME ) + vmx->nested.nested_run_pending =1 ; +else + vmx->nested.nested_run_pending =0 ; + +if (vmx->nested.nested_mode &&nested_vmx_exit_handled (vcpu, true)) { +nested_vmx_vmexit (vcpu, false); +return 1 ; + } +/* Access CR3 don't cause VMExit in paging mode, so we need * to sync with guest real CR3. */ if (enable_ept &&is_paging (vcpu))
@@ -5347,6 +5435,60 @@ static void fixup_rmode_irq(struct vcpu_vmx *vmx) | vmx->rmode.irq.vector; } +static int nested_handle_valid_idt (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +int irq; +int type; +int errCodeValid; +u32 idt_vectoring_info; +u32 guest_intr; +bool nmi_window_open; +bool interrupt_window_open; + +if (vmx->nested.valid_idt_vectoring_info) { + idt_vectoring_info =vmcs_read32 (IDT_VECTORING_INFO_FIELD ); + irq = idt_vectoring_info &VECTORING_INFO_VECTOR_MASK ; + type = idt_vectoring_info &VECTORING_INFO_TYPE_MASK ; + errCodeValid = idt_vectoring_info & +VECTORING_INFO_DELIVER_CODE_MASK ; + + guest_intr =vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ); + nmi_window_open = + !(guest_intr & (GUEST_INTR_STATE_STI | +GUEST_INTR_STATE_MOV_SS | +GUEST_INTR_STATE_NMI )); + + interrupt_window_open = + ((vmcs_readl (GUEST_RFLAGS ) &X86_EFLAGS_IF ) && + !(guest_intr & (GUEST_INTR_STATE_STI | +GUEST_INTR_STATE_MOV_SS ))); + +if (type ==INTR_TYPE_EXT_INTR && !interrupt_window_open) { +printk (KERN_INFO "IDT ignored, l2 interrupt window closed!\n" ); +return 0 ; + } + +if (type ==INTR_TYPE_NMI_INTR && !nmi_window_open) { +printk (KERN_INFO "IDT ignored, l2 nmi window closed!\n" ); +return 0 ; + } + +vmcs_write32 (VM_ENTRY_INTR_INFO_FIELD , + irq | type |INTR_INFO_VALID_MASK | errCodeValid); + + +vmcs_write32 (VM_ENTRY_INSTRUCTION_LEN , +vmcs_read32 (VM_EXIT_INSTRUCTION_LEN )); + +if (errCodeValid) +vmcs_write32 (VM_ENTRY_EXCEPTION_ERROR_CODE , +vmcs_read32 (IDT_VECTORING_ERROR_CODE )); + } + +return 1 ; +} +#ifdef CONFIG_X86_64 #define R "r"#define Q "q"
@@ -5358,8 +5500,17 @@ static void fixup_rmode_irq(struct vcpu_vmx *vmx)static void vmx_vcpu_run (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +int r;u32 nested_exception_bitmap =0 ; +if (vmx->nested.nested_mode) { + r =nested_handle_valid_idt (vcpu); +if (!r) { + vmx->fail =1 ; +return ; + } + } +/* Record the guest's net vcpu time for enforced NMI injections. */ if (unlikely (!cpu_has_virtual_nmis () && vmx->soft_vnmi_blocked)) vmx->entry_time =ktime_get ();
@@ -5539,6 +5690,9 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu) vmx->idt_vectoring_info =vmcs_read32 (IDT_VECTORING_INFO_FIELD ); + vmx->nested.valid_idt_vectoring_info = vmx->nested.nested_mode && + (vmx->idt_vectoring_info &VECTORING_INFO_VALID_MASK ); +if (vmx->rmode.irq.pending)fixup_rmode_irq (vmx);
@@ -6191,6 +6345,7 @@ static int nested_vmx_run(struct kvm_vcpu *vcpu) r =kvm_mmu_load (vcpu);if (unlikely (r)) {printk (KERN_ERR "Error in kvm_mmu_load r %d\n" , r); +nested_vmx_vmexit (vcpu, false);set_rflags_to_vmx_fail_valid (vcpu);/* switch back to L1 */ vmx->nested.nested_mode =0 ;
@@ -6244,6 +6399,360 @@ static int launch_guest(struct kvm_vcpu *vcpu, bool launch)return 1 ; } +static int nested_vmx_vmexit (struct kvm_vcpu *vcpu, +bool is_interrupt) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct level_state *l2_state; +int efer_offset; + +if (!vmx->nested.nested_mode) { +printk (KERN_INFO "WARNING: %s called but not in nested mode\n" , + __func__); +return 0 ; + } + +sync_cached_regs_to_vmcs (vcpu); + +if (!nested_map_current (vcpu)) { +printk (KERN_INFO "Error mapping shadow vmcs\n" ); +set_rflags_to_vmx_fail_valid (vcpu); +return 1 ; + } + + l2_state = &(vmx->nested.current_l2_page->l2_state); +prepare_vmcs_12 (vcpu); +if (is_interrupt) +get_shadow_vmcs (vcpu)->vm_exit_reason = +EXIT_REASON_EXTERNAL_INTERRUPT ; + + l2_state->launched = vmx->launched; + l2_state->cpu = vcpu->cpu; + +nested_unmap_current (vcpu); + + vmx->vmcs = vmx->nested.l1_vmcs; + vcpu->cpu = vmx->nested.l1_state->cpu; + vmx->launched = vmx->nested.l1_state->launched; + +vmx_vcpu_load (vcpu,get_cpu ()); +put_cpu (); + + vcpu->arch.shadow_efer = vmx->nested.l1_state->shadow_efer; +if ((vcpu->arch.shadow_efer &EFER_LMA ) && + !(vcpu->arch.shadow_efer &EFER_SCE )) + vcpu->arch.shadow_efer |=EFER_SCE ; + + efer_offset =__find_msr_index (vmx,MSR_EFER ); +if (update_transition_efer (vmx, efer_offset)) +wrmsrl (MSR_EFER , vmx->guest_msrs[efer_offset].data); + +vmx_set_cr0 (vcpu, vmx->nested.l1_shadow_vmcs->cr0_read_shadow); +vmx_set_cr4 (vcpu, vmx->nested.l1_state->cr4); + +if (enable_ept) { + vcpu->arch.cr3 = vmx->nested.l1_shadow_vmcs->guest_cr3; +vmcs_write32 (GUEST_CR3 , vmx->nested.l1_shadow_vmcs->guest_cr3); + }else { +kvm_set_cr3 (vcpu, vmx->nested.l1_state->cr3); + } + +if (!nested_map_current (vcpu)) { +printk (KERN_INFO "Error mapping shadow vmcs\n" ); +set_rflags_to_vmx_fail_valid (vcpu); +return 1 ; + } + +switch_back_vmcs (vcpu); + + vmx->nested.l1_shadow_vmcs->guest_cr0 =get_shadow_vmcs (vcpu)->host_cr0; + +nested_unmap_current (vcpu); + +kvm_register_write (vcpu,VCPU_REGS_RSP , + vmx->nested.l1_shadow_vmcs->guest_rsp); +kvm_register_write (vcpu,VCPU_REGS_RIP , + vmx->nested.l1_shadow_vmcs->guest_rip); + + vmx->nested.nested_mode =0 ; + +kvm_mmu_reset_context (vcpu); +kvm_mmu_load (vcpu); + +if (unlikely (vmx->fail)) { + vmx->fail =0 ; +set_rflags_to_vmx_fail_valid (vcpu); + }else +clear_rflags_cf_zf (vcpu); + +return 0 ; +} + +static int nested_vmx_exit_handled_msr (struct kvm_vcpu *vcpu) +{ +if (to_vmx (vcpu)->nested.nested_mode) { +struct page *msr_page =NULL ; +u32 msr_index = vcpu->arch.regs[VCPU_REGS_RCX ]; +u32 exit_code =vmcs_read32 (VM_EXIT_REASON ); +struct shadow_vmcs *l2svmcs =get_shadow_vmcs (vcpu); + +if (!cpu_has_vmx_msr_bitmap () + || !nested_cpu_has_vmx_msr_bitmap (vcpu)) +return 1 ; + + msr_page =nested_get_page (vcpu, + l2svmcs->msr_bitmap); + +if (!msr_page) { +printk (KERN_INFO "%s error in nested_get_page\n" , + __func__); +return 0 ; + } + +switch (exit_code) { +case EXIT_REASON_MSR_READ : +if (msr_index <=0x1fff ) { +if (test_bit (msr_index, + (unsigned long *)(msr_page + +0x000 ))) +return 1 ; + }else if ((msr_index >=0xc0000000 ) && + (msr_index <=0xc0001fff )) { + msr_index &=0x1fff ; +if (test_bit (msr_index, + (unsigned long *)(msr_page + +0x400 ))) +return 1 ; + } +break ; +case EXIT_REASON_MSR_WRITE : +if (msr_index <=0x1fff ) { +if (test_bit (msr_index, + (unsigned long *)(msr_page + +0x800 ))) +return 1 ; + }else if ((msr_index >=0xc0000000 ) && + (msr_index <=0xc0001fff )) { + msr_index &=0x1fff ; +if (test_bit (msr_index, + (unsigned long *)(msr_page + +0xc00 ))) +return 1 ; + } +break ; + } + } + +return 0 ; +} + +static int nested_vmx_exit_handled (struct kvm_vcpu *vcpu,bool kvm_override) +{ +u32 exit_code =vmcs_read32 (VM_EXIT_REASON ); +struct vcpu_vmx *vmx =to_vmx (vcpu); +u32 intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); +struct shadow_vmcs *l2svmcs; + +int r =0 ; + +if (vmx->nested.nested_run_pending) +return 0 ; + +if (unlikely (vmx->fail)) { +printk (KERN_INFO "%s failed vm entry %x\n" , + __func__,vmcs_read32 (VM_INSTRUCTION_ERROR )); +return 1 ; + } + +if (kvm_override) { +switch (exit_code) { +case EXIT_REASON_EXTERNAL_INTERRUPT : +return 0 ; +case EXIT_REASON_EXCEPTION_NMI : +if (!is_exception (intr_info)) +return 0 ; + +if (is_page_fault (intr_info) && (!enable_ept)) +return 0 ; + +break ; +case EXIT_REASON_EPT_VIOLATION : +if (enable_ept) +return 0 ; + +break ; + } + } + + +if (!nested_map_current (vcpu)) +return 0 ; + + l2svmcs =get_shadow_vmcs (vcpu); + +switch (exit_code) { +case EXIT_REASON_INVLPG : +if (l2svmcs->cpu_based_vm_exec_control & +CPU_BASED_INVLPG_EXITING ) + r =1 ; +break ; +case EXIT_REASON_MSR_READ : +case EXIT_REASON_MSR_WRITE : + r =nested_vmx_exit_handled_msr (vcpu); +break ; +case EXIT_REASON_CR_ACCESS : { +unsigned long exit_qualification = +vmcs_readl (EXIT_QUALIFICATION ); +int cr = exit_qualification &15 ; +int reg = (exit_qualification >>8 ) &15 ; +unsigned long val =kvm_register_read (vcpu, reg); + +switch ((exit_qualification >>4 ) &3 ) { +case 0 :/* mov to cr */ +switch (cr) { +case 0 : +if (l2svmcs->cr0_guest_host_mask & + (val ^ l2svmcs->cr0_read_shadow)) + r =1 ; +break ; +case 3 : +if (l2svmcs->cpu_based_vm_exec_control & +CPU_BASED_CR3_LOAD_EXITING ) + r =1 ; +break ; +case 4 : +if (l2svmcs->cr4_guest_host_mask & + (l2svmcs->cr4_read_shadow ^ val)) + r =1 ; +break ; +case 8 : +if (l2svmcs->cpu_based_vm_exec_control & +CPU_BASED_CR8_LOAD_EXITING ) + r =1 ; +break ; + } +break ; +case 2 :/* clts */ +if (l2svmcs->cr0_guest_host_mask &X86_CR0_TS ) + r =1 ; +break ; +case 1 :/*mov from cr*/ +switch (cr) { +case 0 : + r =1 ; +case 3 : +if (l2svmcs->cpu_based_vm_exec_control & +CPU_BASED_CR3_STORE_EXITING ) + r =1 ; +break ; +case 4 : + r =1 ; +break ; +case 8 : +if (l2svmcs->cpu_based_vm_exec_control & +CPU_BASED_CR8_STORE_EXITING ) + r =1 ; +break ; + } +break ; +case 3 :/* lmsw */ +if (l2svmcs->cr0_guest_host_mask & + (val ^ l2svmcs->cr0_read_shadow)) + r =1 ; +break ; + } +break ; + } +case EXIT_REASON_DR_ACCESS : { +if (l2svmcs->cpu_based_vm_exec_control & +CPU_BASED_MOV_DR_EXITING ) + r =1 ; +break ; + } + +case EXIT_REASON_EXCEPTION_NMI : { + +if (is_external_interrupt (intr_info) && + (l2svmcs->pin_based_vm_exec_control & +PIN_BASED_EXT_INTR_MASK )) + r =1 ; +else if (is_nmi (intr_info) && + (l2svmcs->pin_based_vm_exec_control & +PIN_BASED_NMI_EXITING )) + r =1 ; +else if (is_exception (intr_info) && + (l2svmcs->exception_bitmap & + (1u << (intr_info &INTR_INFO_VECTOR_MASK )))) + r =1 ; +else if (is_page_fault (intr_info)) + r =1 ; +break ; + } + +case EXIT_REASON_EXTERNAL_INTERRUPT : +if (l2svmcs->pin_based_vm_exec_control & +PIN_BASED_EXT_INTR_MASK ) + r =1 ; +break ; +default : + r =1 ; + } +nested_unmap_current (vcpu); + +return r; +} + +static int nested_vmx_check_exception (struct vcpu_vmx *vmx,unsigned nr, +bool has_error_code,u32 error_code) +{ +if (vmx->nested.nested_mode) { +if (nested_vmx_exit_handled (&vmx->vcpu, false)) { +nested_vmx_vmexit (&vmx->vcpu, false); +if (!nested_map_current (&vmx->vcpu)) +return 1 ; +get_shadow_vmcs (&vmx->vcpu)->vm_exit_reason = +EXIT_REASON_EXCEPTION_NMI ; +get_shadow_vmcs (&vmx->vcpu)->vm_exit_intr_info = + (nr |INTR_TYPE_HARD_EXCEPTION + | (has_error_code ? +INTR_INFO_DELIVER_CODE_MASK :0 ) + |INTR_INFO_VALID_MASK ); + +if (has_error_code) +get_shadow_vmcs (&vmx->vcpu)-> + vm_exit_intr_error_code = error_code; +nested_unmap_current (&vmx->vcpu); +return 1 ; + } + } +return 0 ; +} + +static int nested_vmx_intr (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); + +if (!nested_map_current (vcpu)) +return 0 ; + +if (get_shadow_vmcs (vcpu)->pin_based_vm_exec_control & +PIN_BASED_EXT_INTR_MASK ) { + +if (vmx->nested.nested_run_pending) { +nested_unmap_current (vcpu); +return 0 ; + } + +nested_unmap_current (vcpu); +nested_vmx_vmexit (vcpu, true); +return 1 ; + } + +nested_unmap_current (vcpu); + +return 0 ; +} +static struct kvm_x86_ops vmx_x86_ops = { .cpu_has_kvm_support = cpu_has_kvm_support, .disabled_by_bios = vmx_disabled_by_bios,
Signature · 1 line
1.6.0.4
Re: [PATCH 7/7] Nested VMX patch 7 handling of nested guest exits
Quoted 3 lines · On 12/10/2009 08:38 PM, [email protected] wrote:
From: Orit Wasserman<[email protected]>
(changelog)
Quoted 11 lines
arch/x86/kvm/vmx.c +7
-0@@ -1525,6 +1539,22 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) new_offset =vmcs_read64 (TSC_OFFSET ) + delta;vmcs_write64 (TSC_OFFSET , new_offset); } + +if (l1_shadow_vmcs !=NULL ) { + l1_shadow_vmcs->host_tr_base = +vmcs_readl (HOST_TR_BASE ); + l1_shadow_vmcs->host_gdtr_base = +vmcs_readl (HOST_GDTR_BASE ); + l1_shadow_vmcs->host_ia32_sysenter_esp =
arch/x86/kvm/vmx.c static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 1558
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+vmcs_readl (HOST_IA32_SYSENTER_ESP ); + +if (tsc_this< vcpu->arch.host_tsc) + l1_shadow_vmcs->tsc_offset = +vmcs_read64 (TSC_OFFSET ); + +if (vmx->nested.nested_mode) +load_vmcs_host_state (l1_shadow_vmcs); + }
Please share this code with non-nested vmcs setup.
arch/x86/kvm/vmx.c static void enable_irq_window(struct kvm_vcpu *vcpu) · line 3831
Quoted 9 lines
arch/x86/kvm/vmx.c +4
-0@@ -3794,6 +3824,11 @@ static void enable_irq_window(struct kvm_vcpu *vcpu) {u32 cpu_based_vm_exec_control; +if (to_vmx (vcpu)->nested.nested_mode) { +nested_vmx_intr (vcpu); +return ; + }
I think this happens too late? enable_irq_window() is called after we've given up on injecting the interrupt because interrupts are disabled. But if we're running a guest, we can vmexit and inject the interrupt. This code will only vmexit.
Hm, I see the vmexit code has an in_interrupt case, but I'd like this to be more regular: adjust vmx_interrupt_allowed() to allow interrupts if in a guest, and vmx_inject_irq() to force the vmexit. This way interrupts have a single code path.
arch/x86/kvm/vmx.c static void vmx_set_nmi_mask(struct kvm_vcpu *vcpu, bool masked) · line 3963
Quoted 8 lines
arch/x86/kvm/vmx.c +4
-0static int vmx_interrupt_allowed (struct kvm_vcpu *vcpu) { +if (to_vmx (vcpu)->nested.nested_mode) { +if (!nested_vmx_intr (vcpu)) +return 0 ; + }
... and you do that... so I wonder why the changes to enable_irq_window() are needed?
arch/x86/kvm/vmx.c static int handle_exception(struct kvm_vcpu *vcpu) · line 4089
Quoted 13 lines
arch/x86/kvm/vmx.c +5
-0+return (vmcs_readl (GUEST_RFLAGS )&X86_EFLAGS_IF )&& !(vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO )& (GUEST_INTR_STATE_STI |GUEST_INTR_STATE_MOV_SS ));@@ -4042,6 +4082,10 @@ static int handle_exception(struct kvm_vcpu *vcpu)not interested (exception bitmap 12 does not include NM_VECTOR) enable fpu and resume l2 (avoid switching to l1) */ + +if (vmx->nested.nested_mode) + vmx->nested.nested_run_pending =1 ;/* removing this line cause hung on boot of l2*/ +
This indicates a hack?
Quoted 19 lines
arch/x86/kvm/vmx.c +0
-0vmx_fpu_activate (vcpu);return 1;
arch/x86/kvm/vmx.c +10
-1@@ -4169,7 +4213,33 @@ static int handle_cr(struct kvm_vcpu *vcpu)trace_kvm_cr_write (cr, val);switch (cr) {case 0 : -kvm_set_cr0 (vcpu, val); +if (to_vmx (vcpu)->nested.nested_mode) { +/* assume only X86_CR0_TS is handled by l0 */ +long new_cr0 =vmcs_readl (GUEST_CR0 ); +long new_cr0_read_shadow =vmcs_readl (CR0_READ_SHADOW ); + +vmx_fpu_deactivate (vcpu); + +if (val&X86_CR0_TS ) { + new_cr0 |=X86_CR0_TS ; + new_cr0_read_shadow |=X86_CR0_TS ;
arch/x86/kvm/vmx.c static int handle_cr(struct kvm_vcpu *vcpu) · line 4236
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+ vcpu->arch.cr0 |=X86_CR0_TS ; + }else { + new_cr0&= ~X86_CR0_TS ; + new_cr0_read_shadow&= ~X86_CR0_TS ; + vcpu->arch.cr0&=X86_CR0_TS ; + } + +vmcs_writel (GUEST_CR0 , new_cr0); +vmcs_writel (CR0_READ_SHADOW , new_cr0_read_shadow);
Don't you need to #vmexit if the new cr0 violates the cr0_bits_always_on constraint, or if it changes bits in cr0 that the guest intercepts?
arch/x86/kvm/vmx.c static int handle_cr(struct kvm_vcpu *vcpu)
Quoted 6 lines
arch/x86/kvm/vmx.c +5
-0+ +if (!(val&X86_CR0_TS ) || !(val&X86_CR0_PE )) +vmx_fpu_activate (vcpu); + +to_vmx (vcpu)->nested.nested_run_pending =1 ;
Please split into a function.
Quoted 8 lines
arch/x86/kvm/vmx.c +3
-0+ }else +kvm_set_cr0 (vcpu, val); +skip_emulated_instruction(vcpu); return 1; case 3:arch/x86/kvm/vmx.c +0
-0@@ -4196,8 +4266,15 @@ static int handle_cr(struct kvm_vcpu *vcpu)break ;
arch/x86/kvm/vmx.c static int handle_cr(struct kvm_vcpu *vcpu) · line 4274
Quoted 10 lines
arch/x86/kvm/vmx.c +5
-2case 2 :/* clts */ vmx_fpu_deactivate (vcpu); - vcpu->arch.cr0&= ~X86_CR0_TS ; -vmcs_writel (CR0_READ_SHADOW , vcpu->arch.cr0); +if (to_vmx (vcpu)->nested.nested_mode) { +vmcs_writel (GUEST_CR0 ,vmcs_readl (GUEST_CR0 )& ~X86_CR0_TS ); +vmcs_writel (CR0_READ_SHADOW ,vmcs_readl (CR0_READ_SHADOW )& ~X86_CR0_TS ); + vcpu->arch.cr0&= ~X86_CR0_TS ; +to_vmx (vcpu)->nested.nested_run_pending =1 ;
Won't the guest want to intercept this some time?
arch/x86/kvm/vmx.c static void fixup_rmode_irq(struct vcpu_vmx *vmx) · line 5439
Quoted 9 lines
arch/x86/kvm/vmx.c +1
-0/* Access CR3 don't cause VMExit in paging mode, so we need * to sync with guest real CR3. */ if (enable_ept&&is_paging (vcpu))@@ -5347,6 +5435,60 @@ static void fixup_rmode_irq(struct vcpu_vmx *vmx) | vmx->rmode.irq.vector; } +static int nested_handle_valid_idt (struct kvm_vcpu *vcpu)
I asked for this to be renamed.
arch/x86/kvm/vmx.c static void fixup_rmode_irq(struct vcpu_vmx *vmx) · line 5508
Quoted 13 lines
arch/x86/kvm/vmx.c +3
-0#ifdef CONFIG_X86_64 #define R "r"#define Q "q"@@ -5358,8 +5500,17 @@ static void fixup_rmode_irq(struct vcpu_vmx *vmx)static void vmx_vcpu_run (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +int r;u32 nested_exception_bitmap =0 ; +if (vmx->nested.nested_mode) { + r =nested_handle_valid_idt (vcpu);
This will cause vmread()s before the launch of state that is not saved. This means it is broken on migration or after set_regs().
In general we follow the following pattern:
read from memory
vmwrite
vmlaunch/vmresume
vmread
write to memory
loop
There are exceptions where we allow state to be cached, mostly registers. But we keep accessors for them so that save/restore works.
Quoted 47 lines
arch/x86/kvm/vmx.c +46
-0+ +static int nested_vmx_exit_handled_msr (struct kvm_vcpu *vcpu) +{ +if (to_vmx (vcpu)->nested.nested_mode) { +struct page *msr_page =NULL ; +u32 msr_index = vcpu->arch.regs[VCPU_REGS_RCX ]; +u32 exit_code =vmcs_read32 (VM_EXIT_REASON ); +struct shadow_vmcs *l2svmcs =get_shadow_vmcs (vcpu); + +if (!cpu_has_vmx_msr_bitmap () + || !nested_cpu_has_vmx_msr_bitmap (vcpu)) +return 1 ; + + msr_page =nested_get_page (vcpu, + l2svmcs->msr_bitmap); + +if (!msr_page) { +printk (KERN_INFO "%s error in nested_get_page\n" , + __func__); +return 0 ; + } + +switch (exit_code) { +case EXIT_REASON_MSR_READ : +if (msr_index<=0x1fff ) { +if (test_bit (msr_index, + (unsigned long *)(msr_page + +0x000 ))) +return 1 ; + }else if ((msr_index>=0xc0000000 )&& + (msr_index<=0xc0001fff )) { + msr_index&=0x1fff ; +if (test_bit (msr_index, + (unsigned long *)(msr_page + +0x400 ))) +return 1 ; + } +break ; +case EXIT_REASON_MSR_WRITE : +if (msr_index<=0x1fff ) { +if (test_bit (msr_index, + (unsigned long *)(msr_page + +0x800 ))) +return 1 ; + }else if ((msr_index>=0xc0000000 )&& + (msr_index<=0xc0001fff )) {
arch/x86/kvm/vmx.c static int launch_guest(struct kvm_vcpu *vcpu, bool launch) · line 6545
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+ msr_index&=0x1fff ; +if (test_bit (msr_index, + (unsigned long *)(msr_page + +0xc00 ))) +return 1 ; + } +break ; + } + } +
Please refactor with a single test_bit, just calculate the offsets differently (+400*8 for high msrs, +800*8 for writes).
Quoted 11 lines
arch/x86/kvm/vmx.c +10
-0+ +static int nested_vmx_exit_handled (struct kvm_vcpu *vcpu,bool kvm_override) +{ +u32 exit_code =vmcs_read32 (VM_EXIT_REASON ); +struct vcpu_vmx *vmx =to_vmx (vcpu); +u32 intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); +struct shadow_vmcs *l2svmcs; + +int r =0 ; +
arch/x86/kvm/vmx.c static int launch_guest(struct kvm_vcpu *vcpu, bool launch) · line 6567
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+if (vmx->nested.nested_run_pending) +return 0 ; + +if (unlikely (vmx->fail)) { +printk (KERN_INFO "%s failed vm entry %x\n" , + __func__,vmcs_read32 (VM_INSTRUCTION_ERROR )); +return 1 ; + } + +if (kvm_override) {
What's kvm_override?
Quoted 126 lines
arch/x86/kvm/vmx.c +124
-0+switch (exit_code) { +case EXIT_REASON_EXTERNAL_INTERRUPT : +return 0 ; +case EXIT_REASON_EXCEPTION_NMI : +if (!is_exception (intr_info)) +return 0 ; + +if (is_page_fault (intr_info)&& (!enable_ept)) +return 0 ; + +break ; +case EXIT_REASON_EPT_VIOLATION : +if (enable_ept) +return 0 ; + +break ; + } + } + + +if (!nested_map_current (vcpu)) +return 0 ; + + l2svmcs =get_shadow_vmcs (vcpu); + +switch (exit_code) { +case EXIT_REASON_INVLPG : +if (l2svmcs->cpu_based_vm_exec_control& +CPU_BASED_INVLPG_EXITING ) + r =1 ; +break ; +case EXIT_REASON_MSR_READ : +case EXIT_REASON_MSR_WRITE : + r =nested_vmx_exit_handled_msr (vcpu); +break ; +case EXIT_REASON_CR_ACCESS : { +unsigned long exit_qualification = +vmcs_readl (EXIT_QUALIFICATION ); +int cr = exit_qualification&15 ; +int reg = (exit_qualification>>8 )&15 ; +unsigned long val =kvm_register_read (vcpu, reg); + +switch ((exit_qualification>>4 )&3 ) { +case 0 :/* mov to cr */ +switch (cr) { +case 0 : +if (l2svmcs->cr0_guest_host_mask& + (val ^ l2svmcs->cr0_read_shadow)) + r =1 ; +break ; +case 3 : +if (l2svmcs->cpu_based_vm_exec_control& +CPU_BASED_CR3_LOAD_EXITING ) + r =1 ; +break ; +case 4 : +if (l2svmcs->cr4_guest_host_mask& + (l2svmcs->cr4_read_shadow ^ val)) + r =1 ; +break ; +case 8 : +if (l2svmcs->cpu_based_vm_exec_control& +CPU_BASED_CR8_LOAD_EXITING ) + r =1 ; +break ; + } +break ; +case 2 :/* clts */ +if (l2svmcs->cr0_guest_host_mask&X86_CR0_TS ) + r =1 ; +break ; +case 1 :/*mov from cr*/ +switch (cr) { +case 0 : + r =1 ; +case 3 : +if (l2svmcs->cpu_based_vm_exec_control& +CPU_BASED_CR3_STORE_EXITING ) + r =1 ; +break ; +case 4 : + r =1 ; +break ; +case 8 : +if (l2svmcs->cpu_based_vm_exec_control& +CPU_BASED_CR8_STORE_EXITING ) + r =1 ; +break ; + } +break ; +case 3 :/* lmsw */ +if (l2svmcs->cr0_guest_host_mask& + (val ^ l2svmcs->cr0_read_shadow)) + r =1 ; +break ; + } +break ; + } +case EXIT_REASON_DR_ACCESS : { +if (l2svmcs->cpu_based_vm_exec_control& +CPU_BASED_MOV_DR_EXITING ) + r =1 ; +break ; + } + +case EXIT_REASON_EXCEPTION_NMI : { + +if (is_external_interrupt (intr_info)&& + (l2svmcs->pin_based_vm_exec_control& +PIN_BASED_EXT_INTR_MASK )) + r =1 ; +else if (is_nmi (intr_info)&& + (l2svmcs->pin_based_vm_exec_control& +PIN_BASED_NMI_EXITING )) + r =1 ; +else if (is_exception (intr_info)&& + (l2svmcs->exception_bitmap& + (1u << (intr_info&INTR_INFO_VECTOR_MASK )))) + r =1 ; +else if (is_page_fault (intr_info)) + r =1 ; +break ; + } +
arch/x86/kvm/vmx.c static int launch_guest(struct kvm_vcpu *vcpu, bool launch) · line 6701
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+case EXIT_REASON_EXTERNAL_INTERRUPT : +if (l2svmcs->pin_based_vm_exec_control& +PIN_BASED_EXT_INTR_MASK ) + r =1 ; +break ; +default : + r =1 ; + } +nested_unmap_current (vcpu); +
Please move these to the normal handlers so it is possible to follow the code.
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: [PATCH 6/7] Nested VMX patch 6 implements vmlaunch and vmresume
In reply to [email protected]arch/x86/kvm/vmx.c static inline int vmcs_field_type(unsigned long field) · line 302
Quoted 11 lines · On 12/10/2009 08:38 PM, [email protected] wrote:
From: Orit Wasserman<[email protected]>
arch/x86/kvm/vmx.c +1
-1@@ -287,7 +297,7 @@ static inline int vmcs_field_type(unsigned long field) }/* -Returncs VMCS field size in bits +Returns VMCS field size in bits */
Fold.
arch/x86/kvm/vmx.c static inline int vmcs_field_size(int field_type, struct kvm_vcpu *vcpu) · line 329
Quoted 10 lines
arch/x86/kvm/vmx.c +0
-0static inline int vmcs_field_size (int field_type,struct kvm_vcpu *vcpu){arch/x86/kvm/vmx.c +4
-0@@ -313,6 +323,10 @@ static inline int vmcs_field_size(int field_type, struct kvm_vcpu *vcpu)return 0 ; } +#define NESTED_VM_EXIT_CONTROLS_MASK (~(VM_EXIT_LOAD_IA32_PAT | \ + VM_EXIT_SAVE_IA32_PAT)) +#define NESTED_VM_ENTRY_CONTROLS_MASK (~(VM_ENTRY_LOAD_IA32_PAT | \ +
I think a whitelist is better here, so if we add a new feature and forget it here, we don't introduce a vulnerability.
arch/x86/kvm/vmx.c static inline bool report_flexpriority(void) · line 1087
Quoted 8 lines
arch/x86/kvm/vmx.c +6
-0+static inline int nested_cpu_has_vmx_tpr_shadow (struct kvm_vcpu *vcpu) +{ +return cpu_has_vmx_tpr_shadow ()&& +get_shadow_vmcs (vcpu)->cpu_based_vm_exec_control& +CPU_BASED_TPR_SHADOW ; +}
bools are better.
Quoted 5 lines
arch/x86/kvm/vmx.c +4
-0+ +static inline int nested_cpu_has_secondary_exec_ctrls (struct kvm_vcpu *vcpu) +{ + return cpu_has_secondary_exec_ctrls()&&
arch/x86/kvm/vmx.c static inline bool report_flexpriority(void) · line 1101
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+get_shadow_vmcs (vcpu)->cpu_based_vm_exec_control& +CPU_BASED_ACTIVATE_SECONDARY_CONTROLS ; +} + +static inline bool nested_vm_need_virtualize_apic_accesses (struct kvm_vcpu + *vcpu) +{ +return get_shadow_vmcs (vcpu)->secondary_vm_exec_control& +SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES ; +}
Need to check for secondary controls first.
arch/x86/kvm/vmx.c static inline bool report_flexpriority(void) · line 1113
Quoted 13 lines
arch/x86/kvm/vmx.c +12
-0+ +static inline int nested_cpu_has_vmx_ept (struct kvm_vcpu *vcpu) +{ +return get_shadow_vmcs (vcpu)-> + secondary_vm_exec_control&SECONDARY_EXEC_ENABLE_EPT ; +} + +static inline int nested_cpu_has_vmx_vpid (struct kvm_vcpu *vcpu) +{ +return get_shadow_vmcs (vcpu)->secondary_vm_exec_control& +SECONDARY_EXEC_ENABLE_VPID ; +}
A helper nested_check_2ndary_control() can help reduce duplication.
arch/x86/kvm/vmx.c static inline bool report_flexpriority(void) · line 1119
Quoted 7 lines
arch/x86/kvm/vmx.c +6
-0+ +static inline int nested_cpu_has_vmx_pat (struct kvm_vcpu *vcpu) +{ +return get_shadow_vmcs (vcpu)->vm_entry_controls& +VM_ENTRY_LOAD_IA32_PAT ; +}
Suggest dropping PAT support for now (it's optional in the spec IIRC, and doesn't help much).
arch/x86/kvm/vmx.c static void vmclear_local_vcpus(void) · line 2076
Quoted 5 lines
arch/x86/kvm/vmx.c +3
-0+ +void prepare_vmcs_12 (struct kvm_vcpu *vcpu) +{
Not sure what this does. From the name, it appears to prepare a vmcs. From the contents, it appears to read the vmcs and save it into a shadow vmcs.
Quoted 57 lines
arch/x86/kvm/vmx.c +56
-0+struct shadow_vmcs *l2_shadow_vmcs = +get_shadow_vmcs (vcpu); + + l2_shadow_vmcs->guest_es_selector =vmcs_read16 (GUEST_ES_SELECTOR ); + l2_shadow_vmcs->guest_cs_selector =vmcs_read16 (GUEST_CS_SELECTOR ); + l2_shadow_vmcs->guest_ss_selector =vmcs_read16 (GUEST_SS_SELECTOR ); + l2_shadow_vmcs->guest_ds_selector =vmcs_read16 (GUEST_DS_SELECTOR ); + l2_shadow_vmcs->guest_fs_selector =vmcs_read16 (GUEST_FS_SELECTOR ); + l2_shadow_vmcs->guest_gs_selector =vmcs_read16 (GUEST_GS_SELECTOR ); + l2_shadow_vmcs->guest_ldtr_selector =vmcs_read16 (GUEST_LDTR_SELECTOR ); + l2_shadow_vmcs->guest_tr_selector =vmcs_read16 (GUEST_TR_SELECTOR ); + + l2_shadow_vmcs->tsc_offset =vmcs_read64 (TSC_OFFSET ); + l2_shadow_vmcs->guest_physical_address = +vmcs_read64 (GUEST_PHYSICAL_ADDRESS ); + l2_shadow_vmcs->vmcs_link_pointer =vmcs_read64 (VMCS_LINK_POINTER ); + l2_shadow_vmcs->guest_ia32_debugctl =vmcs_read64 (GUEST_IA32_DEBUGCTL ); +if (vmcs_config.vmentry_ctrl&VM_ENTRY_LOAD_IA32_PAT ) + l2_shadow_vmcs->guest_ia32_pat =vmcs_read64 (GUEST_IA32_PAT ); + l2_shadow_vmcs->cr3_target_count =vmcs_read32 (CR3_TARGET_COUNT ); + l2_shadow_vmcs->vm_entry_intr_info_field = +vmcs_read32 (VM_ENTRY_INTR_INFO_FIELD ); + l2_shadow_vmcs->vm_entry_exception_error_code = +vmcs_read32 (VM_ENTRY_EXCEPTION_ERROR_CODE ); + l2_shadow_vmcs->vm_entry_instruction_len = +vmcs_read32 (VM_ENTRY_INSTRUCTION_LEN ); + l2_shadow_vmcs->vm_instruction_error = +vmcs_read32 (VM_INSTRUCTION_ERROR ); + l2_shadow_vmcs->vm_exit_reason =vmcs_read32 (VM_EXIT_REASON ); + l2_shadow_vmcs->vm_exit_intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); + l2_shadow_vmcs->vm_exit_intr_error_code = +vmcs_read32 (VM_EXIT_INTR_ERROR_CODE ); + l2_shadow_vmcs->idt_vectoring_info_field = +vmcs_read32 (IDT_VECTORING_INFO_FIELD ); + l2_shadow_vmcs->idt_vectoring_error_code = +vmcs_read32 (IDT_VECTORING_ERROR_CODE ); + l2_shadow_vmcs->vm_exit_instruction_len = +vmcs_read32 (VM_EXIT_INSTRUCTION_LEN ); + l2_shadow_vmcs->vmx_instruction_info = +vmcs_read32 (VMX_INSTRUCTION_INFO ); + l2_shadow_vmcs->guest_es_limit =vmcs_read32 (GUEST_ES_LIMIT ); + l2_shadow_vmcs->guest_cs_limit =vmcs_read32 (GUEST_CS_LIMIT ); + l2_shadow_vmcs->guest_ss_limit =vmcs_read32 (GUEST_SS_LIMIT ); + l2_shadow_vmcs->guest_ds_limit =vmcs_read32 (GUEST_DS_LIMIT ); + l2_shadow_vmcs->guest_fs_limit =vmcs_read32 (GUEST_FS_LIMIT ); + l2_shadow_vmcs->guest_gs_limit =vmcs_read32 (GUEST_GS_LIMIT ); + l2_shadow_vmcs->guest_ldtr_limit =vmcs_read32 (GUEST_LDTR_LIMIT ); + l2_shadow_vmcs->guest_tr_limit =vmcs_read32 (GUEST_TR_LIMIT ); + l2_shadow_vmcs->guest_gdtr_limit =vmcs_read32 (GUEST_GDTR_LIMIT ); + l2_shadow_vmcs->guest_idtr_limit =vmcs_read32 (GUEST_IDTR_LIMIT ); + l2_shadow_vmcs->guest_es_ar_bytes =vmcs_read32 (GUEST_ES_AR_BYTES ); + l2_shadow_vmcs->guest_cs_ar_bytes =vmcs_read32 (GUEST_CS_AR_BYTES ); + l2_shadow_vmcs->guest_ss_ar_bytes =vmcs_read32 (GUEST_SS_AR_BYTES ); + l2_shadow_vmcs->guest_ds_ar_bytes =vmcs_read32 (GUEST_DS_AR_BYTES ); + l2_shadow_vmcs->guest_fs_ar_bytes =vmcs_read32 (GUEST_FS_AR_BYTES ); + l2_shadow_vmcs->guest_gs_ar_bytes =vmcs_read32 (GUEST_GS_AR_BYTES );
arch/x86/kvm/vmx.c static void vmclear_local_vcpus(void) · line 2142
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+ l2_shadow_vmcs->guest_ldtr_ar_bytes =vmcs_read32 (GUEST_LDTR_AR_BYTES ); + l2_shadow_vmcs->guest_tr_ar_bytes =vmcs_read32 (GUEST_TR_AR_BYTES ); + l2_shadow_vmcs->guest_interruptibility_info = +vmcs_read32 (GUEST_INTERRUPTIBILITY_INFO ); + l2_shadow_vmcs->guest_activity_state = +vmcs_read32 (GUEST_ACTIVITY_STATE ); + l2_shadow_vmcs->guest_sysenter_cs =vmcs_read32 (GUEST_SYSENTER_CS ); + + l2_shadow_vmcs->cr4_read_shadow =vmcs_readl (CR4_READ_SHADOW ); + l2_shadow_vmcs->exit_qualification =vmcs_readl (EXIT_QUALIFICATION );
If EXIT_QUALIFICATION is a physical address, don't you need to translate it? vmx will store a host physical address and we need a guest physical address.
arch/x86/kvm/vmx.c static void vmclear_local_vcpus(void) · line 2143
Quoted 2 lines
arch/x86/kvm/vmx.c +1
-0+ l2_shadow_vmcs->guest_linear_address =vmcs_readl (GUEST_LINEAR_ADDRESS );
Not all processors support this.
arch/x86/kvm/vmx.c static void vmclear_local_vcpus(void) · line 2151
Quoted 9 lines
arch/x86/kvm/vmx.c +8
-0+ +if (l2_shadow_vmcs->cr0_guest_host_mask&X86_CR0_TS ) + l2_shadow_vmcs->guest_cr0 =vmcs_readl (GUEST_CR0 ); +else /* if CR0_GUEST_HOST_MASK[TS]=0 l1 should think TS was really written to CR0 */ + l2_shadow_vmcs->guest_cr0 = + (vmcs_readl (GUEST_CR0 )&~X86_CR0_TS ) | (vmcs_readl (CR0_READ_SHADOW )&X86_CR0_TS ); + + l2_shadow_vmcs->guest_cr4 =vmcs_readl (GUEST_CR4 );
GUEST_CR4 will be different from the guest's view of it (for example, without EPT CR4.PAE will always be set.
We also use CR4_GUEST_HOST_MASK, I think you need to account for that.
Quoted 26 lines
arch/x86/kvm/vmx.c +25
-0+ l2_shadow_vmcs->guest_es_base =vmcs_readl (GUEST_ES_BASE ); + l2_shadow_vmcs->guest_cs_base =vmcs_readl (GUEST_CS_BASE ); + l2_shadow_vmcs->guest_ss_base =vmcs_readl (GUEST_SS_BASE ); + l2_shadow_vmcs->guest_ds_base =vmcs_readl (GUEST_DS_BASE ); + l2_shadow_vmcs->guest_fs_base =vmcs_readl (GUEST_FS_BASE ); + l2_shadow_vmcs->guest_gs_base =vmcs_readl (GUEST_GS_BASE ); + l2_shadow_vmcs->guest_ldtr_base =vmcs_readl (GUEST_LDTR_BASE ); + l2_shadow_vmcs->guest_tr_base =vmcs_readl (GUEST_TR_BASE ); + l2_shadow_vmcs->guest_gdtr_base =vmcs_readl (GUEST_GDTR_BASE ); + l2_shadow_vmcs->guest_idtr_base =vmcs_readl (GUEST_IDTR_BASE ); + l2_shadow_vmcs->guest_dr7 =vmcs_readl (GUEST_DR7 ); + l2_shadow_vmcs->guest_rsp =vmcs_readl (GUEST_RSP ); + l2_shadow_vmcs->guest_rip =vmcs_readl (GUEST_RIP ); + l2_shadow_vmcs->guest_rflags =vmcs_readl (GUEST_RFLAGS ); + l2_shadow_vmcs->guest_pending_dbg_exceptions = +vmcs_readl (GUEST_PENDING_DBG_EXCEPTIONS ); + l2_shadow_vmcs->guest_sysenter_esp =vmcs_readl (GUEST_SYSENTER_ESP ); + l2_shadow_vmcs->guest_sysenter_eip =vmcs_readl (GUEST_SYSENTER_EIP ); +} + +int load_vmcs_common (struct shadow_vmcs *src) +{ +vmcs_write16 (GUEST_ES_SELECTOR , src->guest_es_selector); +vmcs_write16 (GUEST_CS_SELECTOR , src->guest_cs_selector); +vmcs_write16 (GUEST_SS_SELECTOR , src->guest_ss_selector);
arch/x86/kvm/vmx.c static void vmclear_local_vcpus(void) · line 2186
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+vmcs_write16 (GUEST_DS_SELECTOR , src->guest_ds_selector); +vmcs_write16 (GUEST_FS_SELECTOR , src->guest_fs_selector); +vmcs_write16 (GUEST_GS_SELECTOR , src->guest_gs_selector); +vmcs_write16 (GUEST_LDTR_SELECTOR , src->guest_ldtr_selector); +vmcs_write16 (GUEST_TR_SELECTOR , src->guest_tr_selector); + +vmcs_write64 (GUEST_IA32_DEBUGCTL , src->guest_ia32_debugctl); + +if (vmcs_config.vmentry_ctrl&VM_ENTRY_LOAD_IA32_PAT ) +vmcs_write64 (GUEST_IA32_PAT , src->guest_ia32_pat);
This is dangerous, the guest can cause inconsistent page types and processor lockups.
arch/x86/kvm/vmx.c static void vmclear_local_vcpus(void) · line 2189
Quoted 4 lines
arch/x86/kvm/vmx.c +3
-0+ +if (src->vm_entry_msr_load_count<512 ) +vmcs_write32 (VM_ENTRY_MSR_LOAD_COUNT , src->vm_entry_msr_load_count);
Ditto. MSRs have to be validated. This feature is optional, right? Suggest we don't support it for now.
arch/x86/kvm/vmx.c static void vmclear_local_vcpus(void) · line 2234
Quoted 3 lines
arch/x86/kvm/vmx.c +2
-0+vmcs_writel (GUEST_PENDING_DBG_EXCEPTIONS , + src->guest_pending_dbg_exceptions);
I think this is a new field.
arch/x86/kvm/vmx.c static void free_l1_state(struct kvm_vcpu *vcpu) · line 2581
Quoted 10 lines
static struct level_state *create_state(void) { struct level_state *state = NULL;arch/x86/kvm/vmx.c +0
-2@@ -2301,8 +2578,6 @@ static void free_l1_state(struct kvm_vcpu *vcpu)kfree (list_item); } } - -
?
arch/x86/kvm/vmx.c static void vmx_inject_nmi(struct kvm_vcpu *vcpu) · line 3856
Quoted 9 lines
arch/x86/kvm/vmx.c +4
-0@@ -3574,6 +3849,10 @@ static void vmx_inject_nmi(struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +if (vmx->nested.nested_mode) { +return ; + } +
No braces on single-line if statements in kernel code.
arch/x86/kvm/vmx.c static int handle_exception(struct kvm_vcpu *vcpu) · line 4045
Quoted 12 lines
arch/x86/kvm/vmx.c +0
-0if (!cpu_has_virtual_nmis ()) {/* * Tracking the NMI-blocked state in software is built upon
arch/x86/kvm/vmx.c +4
-0@@ -3759,7 +4038,12 @@ static int handle_exception(struct kvm_vcpu *vcpu)return 1 ;/* already handled by vmx_vcpu_run() */ if (is_no_device (intr_info)) { +/* if l0 handled an fpu operation for l2 it's because l1 is +not interested (exception bitmap 12 does not include NM_VECTOR) +enable fpu and resume l2 (avoid switching to l1) +*/
Use standard comment style please.
Quoted 8 lines
arch/x86/kvm/vmx.c +1
-0vmx_fpu_activate (vcpu); +return 1 ; }
arch/x86/kvm/vmx.c static int handle_vmon(struct kvm_vcpu *vcpu) · line 4797
Quoted 6 lines
arch/x86/kvm/vmx.c +1
-1@@ -4504,7 +4793,7 @@ static int handle_vmon(struct kvm_vcpu *vcpu)static int handle_vmptrld (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); -u64 guest_vmcs_addr; +gpa_t guest_vmcs_addr;
Fold.
arch/x86/kvm/vmx.c static int vmx_handle_exit(struct kvm_vcpu *vcpu) · line 5187
Quoted 6 lines
arch/x86/kvm/vmx.c +0
-1@@ -4895,7 +5184,6 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu) = vmcs_read32(VM_INSTRUCTION_ERROR);return 0 ; } -
?
arch/x86/kvm/vmx.c static int vmx_handle_exit(struct kvm_vcpu *vcpu) · line 5195
Quoted 11 lines
(quoted diff) +0
-0if ((vectoring_info& VECTORING_INFO_VALID_MASK)&& (exit_reason != EXIT_REASON_EXCEPTION_NMI&&
exit_reason!= EXIT_REASON_EPT_VIOLATION&&arch/x86/kvm/vmx.c +1
-2@@ -4903,8 +5191,7 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu)printk (KERN_WARNING "%s: unexpected, valid vectoring info " "(0x%x) and exit reason is 0x%x\n" , __func__, vectoring_info, exit_reason); - -if (unlikely (!cpu_has_virtual_nmis ()&& vmx->soft_vnmi_blocked)) { +if (!vmx->nested.nested_mode&&unlikely (!cpu_has_virtual_nmis ()&& vmx->soft_vnmi_blocked)) {
Why is this different for nested mode? At least, you have to check if the guest is intercepting nmis.
Quoted 4 lines
arch/x86/kvm/vmx.c +0
-0if (vmx_interrupt_allowed (vcpu)) { vmx->soft_vnmi_blocked =0 ;} else if (vmx->vnmi_blocked_time> 1000000000LL&&
arch/x86/kvm/vmx.c static void vmx_complete_interrupts(struct vcpu_vmx *vmx) · line 5245
Quoted 10 lines
arch/x86/kvm/vmx.c +2
-2@@ -4951,10 +5238,13 @@ static void vmx_complete_interrupts(struct vcpu_vmx *vmx)int type;bool idtv_info_valid; - exit_intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); - vmx->exit_reason =vmcs_read32 (VM_EXIT_REASON ); +if (vmx->nested.nested_mode) +return ;
Again, I think you have to check if the guest is intercepting interrupts.
Quoted 34 lines
arch/x86/kvm/vmx.c +3
-0+ + exit_intr_info =vmcs_read32 (VM_EXIT_INTR_INFO ); +/* Handle machine checks before interrupts are enabled */ if ((vmx->exit_reason == EXIT_REASON_MCE_DURING_VMENTRY)|| (
vmx->exit_reason==EXIT_REASON_EXCEPTION_NMIarch/x86/kvm/vmx.c +17
-0@@ -5068,6 +5358,7 @@ static void fixup_rmode_irq(struct vcpu_vmx *vmx)static void vmx_vcpu_run (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +u32 nested_exception_bitmap =0 ;/* Record the guest's net vcpu time for enforced NMI injections. */ if (unlikely (!cpu_has_virtual_nmis ()&& vmx->soft_vnmi_blocked))@@ -5099,6 +5390,37 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu)if (vcpu->arch.switch_db_regs)set_debugreg (vcpu->arch.dr6,6 ); +if (vcpu->fpu_active) { +if (vmcs_readl (CR0_READ_SHADOW )&X86_CR0_TS ) +vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS ); +else +vmcs_clear_bits (GUEST_CR0 ,X86_CR0_TS ); + +if (vmx->nested.nested_mode) { +if (!nested_map_current (vcpu)) { + vmx->fail =1 ; +return ; + } + + nested_exception_bitmap =get_shadow_vmcs (vcpu)-> + exception_bitmap; + +nested_unmap_current (vcpu);
arch/x86/kvm/vmx.c static void vmx_vcpu_run(struct kvm_vcpu *vcpu) · line 5418
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+ } + +if (vmx->nested.nested_mode&& + (nested_exception_bitmap& (1u <<NM_VECTOR ))) +vmcs_write32 (EXCEPTION_BITMAP , +vmcs_read32 (EXCEPTION_BITMAP ) | (1u <<NM_VECTOR )); +else +vmcs_write32 (EXCEPTION_BITMAP , +vmcs_read32 (EXCEPTION_BITMAP )& ~(1u <<NM_VECTOR ));
I'd like to see generalized handling of the exception bitmap in update_exception_bitmap(), not something ad-hoc.
arch/x86/kvm/vmx.c static void vmx_vcpu_run(struct kvm_vcpu *vcpu)
Quoted 6 lines
arch/x86/kvm/vmx.c +5
-0+ }else { +vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS ); +vmcs_write32 (EXCEPTION_BITMAP , +vmcs_read32 (EXCEPTION_BITMAP ) | (1u <<NM_VECTOR )); + }
This looks confused wrt the previous patch.
Quoted 13 lines
arch/x86/kvm/vmx.c +11
-0+void save_vmcs (struct shadow_vmcs *dst) +{ + dst->guest_es_selector =vmcs_read16 (GUEST_ES_SELECTOR ); + dst->guest_cs_selector =vmcs_read16 (GUEST_CS_SELECTOR ); + dst->guest_ss_selector =vmcs_read16 (GUEST_SS_SELECTOR ); + dst->guest_ds_selector =vmcs_read16 (GUEST_DS_SELECTOR ); + dst->guest_fs_selector =vmcs_read16 (GUEST_FS_SELECTOR ); + dst->guest_gs_selector =vmcs_read16 (GUEST_GS_SELECTOR ); + dst->guest_ldtr_selector =vmcs_read16 (GUEST_LDTR_SELECTOR ); + dst->guest_tr_selector =vmcs_read16 (GUEST_TR_SELECTOR ); + dst->host_es_selector =vmcs_read16 (HOST_ES_SELECTOR );
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 5739
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+ dst->host_cs_selector =vmcs_read16 (HOST_CS_SELECTOR ); + dst->host_ss_selector =vmcs_read16 (HOST_SS_SELECTOR ); + dst->host_ds_selector =vmcs_read16 (HOST_DS_SELECTOR ); + dst->host_fs_selector =vmcs_read16 (HOST_FS_SELECTOR ); + dst->host_gs_selector =vmcs_read16 (HOST_GS_SELECTOR ); + dst->host_tr_selector =vmcs_read16 (HOST_TR_SELECTOR ); + dst->io_bitmap_a =vmcs_read64 (IO_BITMAP_A ); + dst->io_bitmap_b =vmcs_read64 (IO_BITMAP_B ); +if (cpu_has_vmx_msr_bitmap ()) + dst->msr_bitmap =vmcs_read64 (MSR_BITMAP );
Instead of reading and writing the host parts, which don't change, I'd like to see the vmcs host initialization factored out and reused.
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 5743
Quoted 5 lines
arch/x86/kvm/vmx.c +4
-0+ + dst->vm_exit_msr_store_addr =vmcs_read64 (VM_EXIT_MSR_STORE_ADDR ); + dst->vm_exit_msr_load_addr =vmcs_read64 (VM_EXIT_MSR_LOAD_ADDR ); + dst->vm_entry_msr_load_addr =vmcs_read64 (VM_ENTRY_MSR_LOAD_ADDR );
We don't use those.
Quoted 14 lines
arch/x86/kvm/vmx.c +13
-0+} + +int prepare_vmcs_02 (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct shadow_vmcs *src , *l1_shadow_vmcs = vmx->nested.l1_shadow_vmcs; +struct level_state *l2_state; +u32 exec_control; + + src =get_shadow_vmcs (vcpu); +if (!src) { +nested_unmap_current (vcpu); +printk (KERN_INFO "%s: Error no shadow vmcs\n" , __func__);
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 5881
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+return 1 ; + } + +load_vmcs_common (src); + + l2_state =&(vmx->nested.current_l2_page->l2_state); + +if (l2_state->first_launch) { + +vmcs_write64 (VMCS_LINK_POINTER , src->vmcs_link_pointer);
Looks wrong. The address needs translation at least. Not sure what it does?
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 5887
Quoted 7 lines
arch/x86/kvm/vmx.c +6
-0+ +if (l2_state->io_bitmap_a) +vmcs_write64 (IO_BITMAP_A , l2_state->io_bitmap_a); + +if (l2_state->io_bitmap_b) +vmcs_write64 (IO_BITMAP_B , l2_state->io_bitmap_b);
Address translation? Also, need to mask with the host's I/O bitmap.
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 5890
Quoted 4 lines
arch/x86/kvm/vmx.c +3
-0+ +if (l2_state->msr_bitmap) +vmcs_write64 (MSR_BITMAP , l2_state->msr_bitmap);
Need to mask with the host's msr bitmap.
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 5902
Quoted 13 lines
arch/x86/kvm/vmx.c +12
-0+ +if (src->vm_entry_msr_load_count>0 ) { +struct page *page; + + page =nested_get_page (vcpu, + src->vm_entry_msr_load_addr); +if (!page) +return 1 ; + +vmcs_write64 (VM_ENTRY_MSR_LOAD_ADDR ,page_to_phys (page)); + +kvm_release_page_clean (page);
I don't see how we can trust the guest's msr autoload.
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 5941
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+ +vmcs_write32 (PAGE_FAULT_ERROR_CODE_MASK , + (l1_shadow_vmcs->page_fault_error_code_mask& + src->page_fault_error_code_mask)); + +vmcs_write32 (PAGE_FAULT_ERROR_CODE_MATCH , + (l1_shadow_vmcs->page_fault_error_code_match& + src->page_fault_error_code_match)); +
I don't think this is right. If both masks are 1 but host match is 1 and guest match is 0, we will trap only on the guest's idea of PFEC matching.
Also, if the guest has bit 14 clear in EXCEPTION_BITMAP, this needs to be done completely differently.
I didn't see where PFEC matchin is handled in the exception handler?
Quoted 13 lines
arch/x86/kvm/vmx.c +12
-0+if (cpu_has_secondary_exec_ctrls ()) { + + exec_control = + l1_shadow_vmcs->secondary_vm_exec_control; + +if (nested_cpu_has_secondary_exec_ctrls (vcpu)) { + + exec_control |= src->secondary_vm_exec_control; + +if (!vm_need_virtualize_apic_accesses (vcpu->kvm) || + !nested_vm_need_virtualize_apic_accesses (vcpu)) + exec_control&=
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 5963
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+ ~SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES ; + } + +vmcs_write32 (SECONDARY_VM_EXEC_CONTROL , exec_control); + } + +load_vmcs_host_state (l1_shadow_vmcs); + + l2_state->first_launch = false; + }
How are all those calculations redone if the guest changes one of those controls?
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 5967
Quoted 5 lines
arch/x86/kvm/vmx.c +4
-0+ +if (vm_need_tpr_shadow (vcpu->kvm)&& +nested_cpu_has_vmx_tpr_shadow (vcpu)) +vmcs_write32 (TPR_THRESHOLD , src->tpr_threshold);
If the guest doesn't trap interrupts, we need to stay with the host tpr threshold.
Quoted 6 lines
arch/x86/kvm/vmx.c +5
-0+ +if (enable_ept) { +if (!nested_cpu_has_vmx_ept (vcpu)) { +vmcs_write64 (EPT_POINTER , + l1_shadow_vmcs->ept_pointer);
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 5982
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+vmcs_write64 (GUEST_PDPTR0 , + l1_shadow_vmcs->guest_pdptr0); +vmcs_write64 (GUEST_PDPTR1 , + l1_shadow_vmcs->guest_pdptr1); +vmcs_write64 (GUEST_PDPTR2 , + l1_shadow_vmcs->guest_pdptr2); +vmcs_write64 (GUEST_PDPTR3 , + l1_shadow_vmcs->guest_pdptr3); + } + }
This version doesn't support ept, so please drop.
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 5992
Quoted 11 lines
arch/x86/kvm/vmx.c +10
-0+ + exec_control = l1_shadow_vmcs->cpu_based_vm_exec_control; + + exec_control&= ~CPU_BASED_VIRTUAL_INTR_PENDING ; + + exec_control&= ~CPU_BASED_VIRTUAL_NMI_PENDING ; + + exec_control&= ~CPU_BASED_TPR_SHADOW ; + + exec_control |= src->cpu_based_vm_exec_control;
Need to whitelist good bits.
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+void sync_cached_regs_to_vmcs (struct kvm_vcpu *vcpu) +{ +unsigned long mask; + +if (test_bit (VCPU_REGS_RSP , (unsigned long *)&vcpu->arch.regs_dirty)) +vmcs_writel (GUEST_RSP , vcpu->arch.regs[VCPU_REGS_RSP ]); +if (test_bit (VCPU_REGS_RIP , (unsigned long *)&vcpu->arch.regs_dirty)) +vmcs_writel (GUEST_RIP , vcpu->arch.regs[VCPU_REGS_RIP ]); +
arch/x86/kvm/vmx.c static bool vmx_gb_page_enable(void) · line 6101
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+ mask = ~((1 <<VCPU_REGS_RSP ) | (1 <<VCPU_REGS_RIP )); + +if (vcpu->arch.regs_dirty& mask) { +printk (KERN_INFO "WARNING: dirty cached registers regs_dirty 0x%x mask 0x%lx\n" , + vcpu->arch.regs_dirty, mask); +WARN_ON (1 ); + } + + vcpu->arch.regs_dirty =0 ; +}
Split into a separate patch (and use in existing code which already does this).
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: [PATCH 5/7] Nested VMX patch 5 Simplify fpu handling
In reply to [email protected]Quoted 3 lines · On 12/10/2009 08:38 PM, [email protected] wrote:
From: Orit Wasserman<[email protected]>
What exactly is the simplification? Is it intended to have a functional change?
Quoted 38 lines
--- arch/x86/kvm/vmx.c | 27 +++++++++++++++++---------- 1 files changed, 17 insertions(+), 10 deletions(-)arch/x86/kvm/vmx.c +0
-8@@ -1244,8 +1244,6 @@ static void update_exception_bitmap(struct kvm_vcpu *vcpu)u32 eb; eb = (1u <<PF_VECTOR ) | (1u <<UD_VECTOR ) | (1u <<MC_VECTOR ); -if (!vcpu->fpu_active) - eb |=1u <<NM_VECTOR ;/* * Unconditionally intercept #DB so we can maintain dr6 without * reading it every exit. @@ -1463,10 +1461,6 @@ static void vmx_fpu_activate(struct kvm_vcpu *vcpu)if (vcpu->fpu_active)return ; vcpu->fpu_active =1 ; -vmcs_clear_bits (GUEST_CR0 ,X86_CR0_TS ); -if (vcpu->arch.cr0&X86_CR0_TS ) -vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS ); -update_exception_bitmap (vcpu); }static void vmx_fpu_deactivate (struct kvm_vcpu *vcpu)@@ -1474,8 +1468,6 @@ static void vmx_fpu_deactivate(struct kvm_vcpu *vcpu)if (!vcpu->fpu_active)return ; vcpu->fpu_active =0 ; -vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS ); -update_exception_bitmap (vcpu); }static unsigned long vmx_get_rflags (struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx.c static void vmx_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3) · line 2714
Quoted 10 lines
arch/x86/kvm/vmx.c +4
-2@@ -2715,8 +2707,10 @@ static void vmx_set_cr3(struct kvm_vcpu *vcpu, unsigned long cr3)vmx_flush_tlb (vcpu);vmcs_writel (GUEST_CR3 , guest_cr3); -if (vcpu->arch.cr0&X86_CR0_PE ) -vmx_fpu_deactivate (vcpu); +if (vcpu->arch.cr0&X86_CR0_PE ) { +if (guest_cr3 !=vmcs_readl (GUEST_CR3 )) +vmx_fpu_deactivate (vcpu); + }
Why the added cr3 check? It may make sense, but it isn't a simplification.
Quoted 8 lines
arch/x86/kvm/vmx.c +3
-0static void vmx_set_cr4 (struct kvm_vcpu *vcpu,unsigned long cr4)@@ -5208,6 +5202,19 @@ static void vmx_vcpu_run(struct kvm_vcpu *vcpu)if (vcpu->arch.switch_db_regs)get_debugreg (vcpu->arch.dr6,6 ); +if (vcpu->fpu_active) { +if (vmcs_readl (CR0_READ_SHADOW )&X86_CR0_TS ) +vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS );
arch/x86/kvm/vmx.c static void vmx_vcpu_run(struct kvm_vcpu *vcpu) · line 5217
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+else +vmcs_clear_bits (GUEST_CR0 ,X86_CR0_TS ); +vmcs_write32 (EXCEPTION_BITMAP , +vmcs_read32 (EXCEPTION_BITMAP )& ~(1u <<NM_VECTOR )); + }else { +vmcs_set_bits (GUEST_CR0 ,X86_CR0_TS ); +vmcs_write32 (EXCEPTION_BITMAP , +vmcs_read32 (EXCEPTION_BITMAP ) | (1u <<NM_VECTOR )); + }
This is executed unconditionally, so the vmreads/vmwrites take place every time. Need to cache the previous fpu_active state and only take action if it changed.
Since this is a large piece of code, move it to a function.
Please post this as the first patch (or better, separately), so I can apply it independently of the rest.
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: [PATCH 4/7] Nested VMX patch 4 implements vmread and vmwrite
In reply to [email protected]Quoted 52 lines · On 12/10/2009 08:38 PM, [email protected] wrote:
From: Orit Wasserman<[email protected]>
--- arch/x86/kvm/vmx.c | 670 +++++++++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 660 insertions(+), 10 deletions(-)arch/x86/kvm/vmx.c +31
-0@@ -239,6 +239,7 @@ struct __attribute__ ((__packed__)) level_state {struct __attribute__ ((__packed__ ))nested_vmcs_page {u32 revision_id;u32 abort; +struct shadow_vmcs shadow_vmcs;struct level_state l2_state; };@@ -263,6 +264,55 @@ struct nested_vmx {struct nested_vmcs_page *current_l2_page; }; +enum vmcs_field_type { +VMCS_FIELD_TYPE_U16 =0 , +VMCS_FIELD_TYPE_U64 =1 , +VMCS_FIELD_TYPE_U32 =2 , +VMCS_FIELD_TYPE_ULONG =3 +}; + +#define VMCS_FIELD_LENGTH_OFFSET 13 +#define VMCS_FIELD_LENGTH_MASK 0x6000 + +/* +Returns VMCS Field type +*/ +static inline int vmcs_field_type (unsigned long field) +{ +/* For 32 bit L1 when it using the HIGH field */ +if (0x1 & field) +return VMCS_FIELD_TYPE_U32 ; + +return (VMCS_FIELD_LENGTH_MASK & field)>>13 ; +} + +/* +Returncs VMCS field size in bits +*/ +static inline int vmcs_field_size (int field_type,struct kvm_vcpu *vcpu) +{ +switch (field_type) { +case VMCS_FIELD_TYPE_U16 : +return 2 ;
arch/x86/kvm/vmx.c struct nested_vmx { · line 306
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+case VMCS_FIELD_TYPE_U32 : +return 4 ; +case VMCS_FIELD_TYPE_U64 : +return 8 ; +case VMCS_FIELD_TYPE_ULONG : +#ifdef CONFIG_X86_64 +if (is_long_mode (vcpu)) +return 8 ; +else
Can replace with #endif
arch/x86/kvm/vmx.c struct nested_vmx {
Quoted 5 lines
arch/x86/kvm/vmx.c +4
-0+return 4 ; +#else +return 4 ; +#endif
... and drop the previous three lines.
arch/x86/kvm/vmx.c struct nested_vmx {
Quoted 5 lines
arch/x86/kvm/vmx.c +4
-0+ } + +printk (KERN_INFO "WARNING: invalid field type %d \n" , field_type); +return 0 ;
Can this happen? The field is only two bits wide.
Quoted 4 lines
arch/x86/kvm/vmx.c +2
-0+static inline struct shadow_vmcs *get_shadow_vmcs (struct kvm_vcpu *vcpu) +{
arch/x86/kvm/vmx.c static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) · line 381
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+WARN_ON (!to_vmx (vcpu)->nested.current_l2_page); +return &(to_vmx (vcpu)->nested.current_l2_page->shadow_vmcs); +} + +#define SHADOW_VMCS_OFFSET (x) offsetof(struct shadow_vmcs, x) + +static unsigned short vmcs_field_to_offset_table[HOST_RIP +1 ] = { + + [VIRTUAL_PROCESSOR_ID ] = +SHADOW_VMCS_OFFSET (virtual_processor_id),
Keep on one line, you can use a shorter macro name if it helps. This table is just noise.
arch/x86/kvm/vmx.c static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) · line 671
Quoted 8 lines
arch/x86/kvm/vmx.c +7
-0+ +static inline unsigned short vmcs_field_to_offset (unsigned long field) +{ + +if (field>HOST_RIP || vmcs_field_to_offset_table[field] ==0 ) { +printk (KERN_ERR "invalid vmcs encoding 0x%lx\n" , field); +return -1 ;
This will be converted to 0xffff.
Quoted 9 lines
arch/x86/kvm/vmx.c +8
-0+ } + +return vmcs_field_to_offset_table[field]; +} + +static inline unsigned long nested_vmcs_readl (struct kvm_vcpu *vcpu, +unsigned long field) +{
arch/x86/kvm/vmx.c static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) · line 689
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+struct vcpu_vmx *vmx =to_vmx (vcpu); +unsigned long *entry; + +if (!vmx->nested.current_l2_page) { +printk (KERN_ERR "%s invalid nested vmcs\n" , __func__); +return -1 ; + } + + entry = (unsigned long *)((char *)(get_shadow_vmcs (vcpu)) + +vmcs_field_to_offset (field));
Error check?
arch/x86/kvm/vmx.c static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) · line 714
Quoted 12 lines
arch/x86/kvm/vmx.c +11
-0+static inline u64 nested_vmcs_read64 (struct kvm_vcpu *vcpu,unsigned long field) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +u64 *entry; +if (!vmx->nested.current_l2_page) { +printk (KERN_ERR "%s invalid nested vmcs\n" , __func__); +return -1 ; + } + + entry = (u64 *)((char *)(get_shadow_vmcs (vcpu)) + +vmcs_field_to_offset (field));
Need to support the 'high' part of 64-bit fields.
Quoted 4 lines
arch/x86/kvm/vmx.c +2
-0+return *entry; +}
arch/x86/kvm/vmx.c static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) · line 772
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+ +static inline void nested_vmcs_write64 (struct kvm_vcpu *vcpu, +unsigned long field,u64 value) +{ +#ifdef CONFIG_X86_64 +nested_vmcs_writel (vcpu, field, value); +#else /* nested: 32 bit not actually tested */ +nested_vmcs_writel (vcpu, field, value); +nested_vmcs_writel (vcpu, field+1 , value>>32 ); +#endif
High field support needed.
arch/x86/kvm/vmx.c static int nested_map_current(struct kvm_vcpu *vcpu) · line 812
Quoted 13 lines
static struct page *nested_get_page(struct
kvm_vcpu*vcpu,
u64vmcs_addr)
{arch/x86/kvm/vmx.c +0
-5@@ -354,11 +809,6 @@ static int nested_map_current(struct kvm_vcpu *vcpu) mapped_page =kmap_atomic (vmcs_page,KM_USER0 ); -if (!mapped_page) { -printk (KERN_INFO "%s: error in kmap_atomic\n" , __func__); -return 0 ; - } -
Fold.
arch/x86/kvm/vmx.c static int read_guest_vmcs_gpa(struct kvm_vcpu *vcpu, gva_t gva, u64 *gentry) · line 1844
Quoted 10 lines
arch/x86/kvm/vmx.c +0
-0vmx->nested.current_l2_page = mapped_page;return 1;
arch/x86/kvm/vmx.c +1
-1@@ -1390,7 +1840,7 @@ static int read_guest_vmcs_gpa(struct kvm_vcpu *vcpu, gva_t gva, u64 *gentry) size, vcpu);if (r) {printk (KERN_ERR "%s cannot read guest vmcs addr %lx : %d\n" , - __func__, vcpu->arch.regs[VCPU_REGS_RAX ], r); + __func__, gva, r);
Fold.
Quoted 15 lines
arch/x86/kvm/vmx.c +11
-0@@ -3764,6 +4214,26 @@ static gva_t get_vmx_mem_address(struct kvm_vcpu *vcpu,return addr; } +static void set_rflags_to_vmx_fail_invalid (struct kvm_vcpu *vcpu) +{ +unsigned long rflags; + rflags =vmx_get_rflags (vcpu); + rflags |=X86_EFLAGS_CF ; + rflags&= ~X86_EFLAGS_PF & ~X86_EFLAGS_AF & ~X86_EFLAGS_ZF & + ~X86_EFLAGS_SF & ~X86_EFLAGS_OF ; +vmx_set_rflags (vcpu, rflags); +} + +static void set_rflags_to_vmx_fail_valid (struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx.c static gva_t get_vmx_mem_address(struct kvm_vcpu *vcpu, · line 4237
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+{ +unsigned long rflags; + rflags =vmx_get_rflags (vcpu); + rflags |=X86_EFLAGS_ZF ; + rflags&= ~X86_EFLAGS_PF & ~X86_EFLAGS_AF & ~X86_EFLAGS_CF & + ~X86_EFLAGS_SF & ~X86_EFLAGS_OF ; +vmx_set_rflags (vcpu, rflags); + } +
These are needed much earlier.
Quoted 59 lines
arch/x86/kvm/vmx.c +57
-0+static int handle_vmread_reg (struct kvm_vcpu *vcpu,int reg, +unsigned long field) +{ +u64 field_value; + +switch (vmcs_field_type (field)) { +case VMCS_FIELD_TYPE_U16 : + field_value =nested_vmcs_read16 (vcpu, field); +break ; +case VMCS_FIELD_TYPE_U32 : + field_value =nested_vmcs_read32 (vcpu, field); +break ; +case VMCS_FIELD_TYPE_U64 : + field_value =nested_vmcs_read64 (vcpu, field); +#ifdef CONFIG_X86_64 +if (!is_long_mode (vcpu)) { +kvm_register_write (vcpu, reg+1 , field_value>>32 ); + field_value = (u32 )field_value; + } +#endif +break ; +case VMCS_FIELD_TYPE_ULONG : + field_value =nested_vmcs_readl (vcpu, field); +#ifdef CONFIG_X86_64 +if (!is_long_mode (vcpu)) { +kvm_register_write (vcpu, reg+1 , field_value>>32 ); + field_value = (u32 )field_value; + } +#endif +break ; +default : +printk (KERN_INFO "%s invalid field\n" , __func__); +return 0 ; + } + +kvm_register_write (vcpu, reg, field_value); +return 1 ; +} + +static int handle_vmread_mem (struct kvm_vcpu *vcpu,gva_t gva, +unsigned long field) +{ +u64 field_value; + +switch (vmcs_field_type (field)) { +case VMCS_FIELD_TYPE_U16 : + field_value =nested_vmcs_read16 (vcpu, field); +break ; +case VMCS_FIELD_TYPE_U32 : + field_value =nested_vmcs_read32 (vcpu, field); +break ; +case VMCS_FIELD_TYPE_U64 : + field_value =nested_vmcs_read64 (vcpu, field); +break ; +case VMCS_FIELD_TYPE_ULONG : + field_value =nested_vmcs_readl (vcpu, field); +break ;
arch/x86/kvm/vmx.c static int handle_vmclear(struct kvm_vcpu *vcpu) · line 4339
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+default : +printk (KERN_INFO "%s invalid field\n" , __func__); +return 0 ; + } + +kvm_write_guest_virt (gva,&field_value, +vmcs_field_size (vmcs_field_type (field), vcpu), + vcpu); +return 1 ; +}
Looks like a lot of code duplication. You can probably do this with a single function, and write either to a register or memory at the end.
Quoted 25 lines
arch/x86/kvm/vmx.c +24
-0+ +static int handle_vmread (struct kvm_vcpu *vcpu) +{ +unsigned long field; +int reg; +int is_reg; +unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); +u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO ); +gva_t gva =0 ; +int read_succeed; + +if (!nested_vmx_check_permission (vcpu)) +return 1 ; + +if (!nested_map_current (vcpu)) { +printk (KERN_INFO "%s invalid shadow vmcs\n" , __func__); +set_rflags_to_vmx_fail_invalid (vcpu); +return 1 ; + } + +/* decode instruction info to get the field to read and where to store its value */ +/* Bit 10, Mem/Reg (0 = memory, 1 = register) */ + is_reg = vmx_instruction_info& (1u <<10 );/* bit 10 */ + field =kvm_register_read (vcpu, (vmx_instruction_info>>28 )&0xf );/* bits 31:28 */
arch/x86/kvm/vmx.c static int handle_vmclear(struct kvm_vcpu *vcpu) · line 4373
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+ +if (is_reg) { + reg = (vmx_instruction_info>>3 )&0xf ;/* bits 3:6 */ + read_succeed =handle_vmread_reg (vcpu, reg, field); + }else { + gva =get_vmx_mem_address (vcpu, exit_qualification, + vmx_instruction_info); + read_succeed =handle_vmread_mem (vcpu, gva, field); + } +
This, too, can go into a separate function instead of being duplicated all over.
Quoted 24 lines
arch/x86/kvm/vmx.c +13
-1+if (read_succeed) { +clear_rflags_cf_zf (vcpu); +skip_emulated_instruction (vcpu); + }else { +set_rflags_to_vmx_fail_valid (vcpu); +vmcs_write32 (VM_INSTRUCTION_ERROR ,12 ); + } + +nested_unmap_current (vcpu); +return 1 ; +} +static int handle_vmoff (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -3901,15 +4546,20 @@ static int handle_vmptrst(struct kvm_vcpu *vcpu)unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION );u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO );gva_t vmcs_gva; - +uint size;
arch/x86/kvm/vmx.c static int handle_vmptrst(struct kvm_vcpu *vcpu) · line 4559
Quoted 10 lines
arch/x86/kvm/vmx.c +4
-0if (!nested_vmx_check_permission (vcpu))return 1 ; vmcs_gva =get_vmx_mem_address (vcpu, exit_qualification, vmx_instruction_info); +if (is_long_mode (vcpu)) + size =sizeof (u64); +else + size =sizeof (u32);
I think the vmpointers are always 64-bit.
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: [PATCH 3/7] Nested VMX patch 3 implements vmptrld and vmptrst
In reply to [email protected]Quoted 4 lines · On 12/10/2009 08:38 PM, [email protected] wrote:
arch/x86/kvm/vmx.c +2
-0+ +
arch/x86/kvm/vmx.c struct shared_msr_entry { · line 231
Quoted 10 lines
arch/x86/kvm/vmx.c +6
-0struct __attribute__ ((__packed__ ))level_state {/* Has the level1 guest done vmclear? */ bool vmclear; + +u64 io_bitmap_a; +u64 io_bitmap_b; +u64 msr_bitmap; + +bool first_launch; };
Please keep things naturally aligned.
Quoted 11 lines
/*
arch/x86/kvm/vmx.c +2
-0@@ -122,6 +255,8 @@ struct nested_vmx {gpa_t current_vmptr;/* Level 1 state for switching to level 2 and back */ struct level_state *l1_state; +/* Level 1 shadow vmcs for switching to level 2 and back */ +struct shadow_vmcs *l1_shadow_vmcs;/* list of vmcs for each l2 guest created by l1 */ struct list_head l2_vmcs_list;/* l2 page corresponding to the current vmcs set by l1 */
arch/x86/kvm/vmx.c static struct page *nested_get_page(struct kvm_vcpu *vcpu, · line 326
Quoted 7 lines
arch/x86/kvm/vmx.c +0
-2@@ -187,10 +322,7 @@ static struct page *nested_get_page(struct kvm_vcpu *vcpu, {struct page *vmcs_page =NULL ; -down_read (¤t->mm->mmap_sem); vmcs_page =gfn_to_page (vcpu->kvm, vmcs_addr>>PAGE_SHIFT ); -up_read (¤t->mm->mmap_sem);
Fold this into the patch that introduced the problem.
arch/x86/kvm/vmx.c static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 969
Quoted 12 lines
arch/x86/kvm/vmx.c +0
-1-if (is_error_page (vmcs_page)) { printk(KERN_ERR "%s error allocating page 0x%llx\n",
__func__,vmcs_addr);arch/x86/kvm/vmx.c +1
-1@@ -832,13 +964,14 @@ static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu)if (per_cpu (current_vmcs, cpu) != vmx->vmcs) {u8 error; -per_cpu (current_vmcs, cpu) = vmx->vmcs; +
Please avoid pointless whitespace changes.
arch/x86/kvm/vmx.c static void vmx_vcpu_load(struct kvm_vcpu *vcpu, int cpu) · line 977
Quoted 9 lines
arch/x86/kvm/vmx.c +2
-1asm volatile (__ex(ASM_VMX_VMPTRLD_RAX) "; setna %0" :"=g" (error) :"a" (&phys_addr),"m" (phys_addr) :"cc" ); +if (error) -printk (KERN_ERR "kvm: vmptrld %p/%llx fail\n" , +printk (KERN_ERR "kvm: vmptrld %p/%llx failed\n" , vmx->vmcs, phys_addr);
Fold.
Quoted 34 lines
arch/x86/kvm/vmx.c +23
-0+static int create_l1_state (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -1441,10 +1587,75 @@ static int create_l1_state(struct kvm_vcpu *vcpu) }else return 0 ; + vmx->nested.l1_shadow_vmcs =kzalloc (PAGE_SIZE ,GFP_KERNEL ); +if (!vmx->nested.l1_shadow_vmcs) { +printk (KERN_INFO "%s could not allocate memory for l1_shadow vmcs\n" , + __func__); +kfree (vmx->nested.l1_state); +return -ENOMEM ; + } +INIT_LIST_HEAD (&(vmx->nested.l2_vmcs_list));return 0 ; } +static struct vmcs *alloc_vmcs (void ); +int create_l2_state (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct vmcs *l2_vmcs; + +if (!nested_map_current (vcpu)) { +printk (KERN_ERR "%s error mapping level 2 page" , __func__); +return -ENOMEM ; + } + + l2_vmcs =nested_get_current_vmcs (vcpu); +if (!l2_vmcs) { + struct nested_vmcs_list *new_l2_guest =
arch/x86/kvm/vmx.c static int create_l1_state(struct kvm_vcpu *vcpu) · line 1625
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+ (struct nested_vmcs_list *) +kmalloc (sizeof (struct nested_vmcs_list ),GFP_KERNEL ); + +if (!new_l2_guest) { +printk (KERN_ERR "%s error could not allocate memory for a new l2 guest list item" , + __func__); +nested_unmap_current (vcpu); +return -ENOMEM ; + }
Can the list grow without bounds?
Quoted 17 lines
arch/x86/kvm/vmx.c +17
-0+ + l2_vmcs =alloc_vmcs (); + +if (!l2_vmcs) { +printk (KERN_ERR "%s error could not allocate memory for l2_vmcs" , + __func__); +kfree (new_l2_guest); +nested_unmap_current (vcpu); +return -ENOMEM ; + } + + new_l2_guest->vmcs_addr = vmx->nested.current_vmptr; + new_l2_guest->l2_vmcs = l2_vmcs; +list_add (&(new_l2_guest->list),&(vmx->nested.l2_vmcs_list)); + } + +if (cpu_has_vmx_msr_bitmap ())
arch/x86/kvm/vmx.c static int create_l1_state(struct kvm_vcpu *vcpu)
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+ vmx->nested.current_l2_page->l2_state.msr_bitmap = +vmcs_read64 (MSR_BITMAP ); + else + vmx->nested.current_l2_page->l2_state.msr_bitmap = 0; + + vmx->nested.current_l2_page->l2_state.io_bitmap_a = +vmcs_read64 (IO_BITMAP_A ); + vmx->nested.current_l2_page->l2_state.io_bitmap_b = +vmcs_read64 (IO_BITMAP_B );
Don't understand why these reads are needed.
Quoted 11 lines
arch/x86/kvm/vmx.c +8
-0+ + vmx->nested.current_l2_page->l2_state.first_launch = true; + +nested_unmap_current (vcpu); + +return 0 ; +} +
arch/x86/kvm/vmx.c static int handle_vmon(struct kvm_vcpu *vcpu) · line 3855
Quoted 9 lines
arch/x86/kvm/vmx.c +3
-2@@ -3633,8 +3849,9 @@ static int handle_vmon(struct kvm_vcpu *vcpu)return 1 ; } -if (create_l1_state (vcpu)) { -printk (KERN_ERR "%s create_l1_state failed\n" , __func__); + r =create_l1_state (vcpu); +if (r) { +printk (KERN_ERR "%s create_l1_state failed: %d\n" , __func__, r);
Move this to the original patch.
Quoted 29 lines
arch/x86/kvm/vmx.c +0
-0kvm_queue_exception (vcpu,UD_VECTOR );return 1 ;}arch/x86/kvm/vmx.c +22
-0@@ -3645,6 +3862,63 @@ static int handle_vmon(struct kvm_vcpu *vcpu)return 1 ; } +static int handle_vmptrld (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +u64 guest_vmcs_addr; +gva_t vmcs_gva; +unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); +u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO ); +int r =0 ; + +if (!nested_vmx_check_permission (vcpu)) +return 1 ; + + vmcs_gva =get_vmx_mem_address (vcpu, exit_qualification, + vmx_instruction_info); + +if (read_guest_vmcs_gpa (vcpu, vmcs_gva,&guest_vmcs_addr)) +return 1 ; + +if (vmx->nested.current_vmptr != guest_vmcs_addr) { + vmx->nested.current_vmptr = guest_vmcs_addr; + r =create_l2_state (vcpu); +if (r) {
arch/x86/kvm/vmx.c static int handle_vmon(struct kvm_vcpu *vcpu) · line 3896
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+printk (KERN_ERR "%s create_l2_state failed: %d\n" , + __func__, r); +return 1 ; + } + } + +clear_rflags_cf_zf (vcpu); +skip_emulated_instruction (vcpu); +return 1 ;
No set_rflags() on error?
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: [PATCH 2/7] Nested VMX patch 2 implements vmclear
In reply to [email protected]Quoted 13 lines · On 12/10/2009 08:38 PM, [email protected] wrote:
From: Orit Wasserman<[email protected]>
--- arch/x86/kvm/vmx.c | 235 +++++++++++++++++++++++++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 5 +- arch/x86/kvm/x86.h | 3 + 3 files changed, 240 insertions(+), 3 deletions(-)arch/x86/kvm/vmx.c +0
-0
arch/x86/kvm/vmx.c struct shared_msr_entry { · line 99
Quoted 7 lines
arch/x86/kvm/vmx.c +3
-0@@ -93,13 +93,39 @@ struct shared_msr_entry { };struct __attribute__ ((__packed__ ))level_state { +/* Has the level1 guest done vmclear? */ +bool vmclear; +};
Suggest calling it launch_state and using an enum. We can have three states: uninitialized, clear, and launched. Not sure if this is really required by the spec.
Do we need vmclear in l1_state?
arch/x86/kvm/vmx.c struct shared_msr_entry { · line 117
Quoted 9 lines
arch/x86/kvm/vmx.c +8
-0+struct __attribute__ ((__packed__ ))nested_vmcs_page { +u32 revision_id; +u32 abort; +struct level_state l2_state; +}; + +struct nested_vmcs_list { +struct list_head list;
'link'
Quoted 21 lines
arch/x86/kvm/vmx.c +8
-0+gpa_t vmcs_addr; +struct vmcs *l2_vmcs; };struct nested_vmx {/* Has the level1 guest done vmxon? */ bool vmxon; +/* What is the location of the current vmcs l1 keeps for l2 */ +gpa_t current_vmptr;/* Level 1 state for switching to level 2 and back */ struct level_state *l1_state; +/* list of vmcs for each l2 guest created by l1 */ +struct list_head l2_vmcs_list; +/* l2 page corresponding to the current vmcs set by l1 */ +struct nested_vmcs_page *current_l2_page; };struct vcpu_vmx {@@ -156,6 +182,76 @@ static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) return container_of(vcpu, struct vcpu_vmx, vcpu); }
arch/x86/kvm/vmx.c static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) · line 193
Quoted 10 lines
arch/x86/kvm/vmx.c +8
-0+static struct page *nested_get_page (struct kvm_vcpu *vcpu, +u64 vmcs_addr) +{ +struct page *vmcs_page =NULL ; + +down_read (¤t->mm->mmap_sem); + vmcs_page =gfn_to_page (vcpu->kvm, vmcs_addr>>PAGE_SHIFT ); +up_read (¤t->mm->mmap_sem);
gfn_to_page() doesn't need mmap_sem (and may deadlock if you take it).
Quoted 27 lines
arch/x86/kvm/vmx.c +27
-0+ +if (is_error_page (vmcs_page)) { +printk (KERN_ERR "%s error allocating page 0x%llx\n" , + __func__, vmcs_addr); +kvm_release_page_clean (vmcs_page); +return NULL ; + } + +return vmcs_page; + +} + +static int nested_map_current (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct page *vmcs_page = +nested_get_page (vcpu, vmx->nested.current_vmptr); +struct nested_vmcs_page *mapped_page; + +if (vmcs_page ==NULL ) { +printk (KERN_INFO "%s: failure in nested_get_page\n" , __func__); +return 0 ; + } + +if (vmx->nested.current_l2_page) { +printk (KERN_INFO "%s: shadow vmcs already mapped\n" , __func__); +WARN_ON (1 );
arch/x86/kvm/vmx.c static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) · line 257
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+return 0 ; + } + + mapped_page =kmap_atomic (vmcs_page,KM_USER0 ); + +if (!mapped_page) { +printk (KERN_INFO "%s: error in kmap_atomic\n" , __func__); +return 0 ; + }
kmap_atomic() can't fail.
Quoted 5 lines
arch/x86/kvm/vmx.c +5
-0+ + vmx->nested.current_l2_page = mapped_page; + +return 1 ; +}
arch/x86/kvm/vmx.c static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) · line 257
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+ +static void nested_unmap_current (struct kvm_vcpu *vcpu) +{ +struct page *page; +struct vcpu_vmx *vmx =to_vmx (vcpu); + +if (!vmx->nested.current_l2_page) { +printk (KERN_INFO "Shadow vmcs already unmapped\n" ); +WARN_ON (1 );
Use BUG_ON(), since this can't happen unless there's a bug.
Quoted 21 lines
arch/x86/kvm/vmx.c +12
-0+return ; + } + + page =kmap_atomic_to_page (vmx->nested.current_l2_page); + +kunmap_atomic (vmx->nested.current_l2_page,KM_USER0 ); + +kvm_release_page_dirty (page); + + vmx->nested.current_l2_page =NULL ; +} +static int init_rmode(struct kvm *kvm); static u64 construct_eptp(unsigned long root_hpa);arch/x86/kvm/vmx.c +2
-0@@ -1144,6 +1240,35 @@ static int nested_vmx_set_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data)return 0 ; } +static int read_guest_vmcs_gpa (struct kvm_vcpu *vcpu,gva_t gva,u64 *gentry) +{
arch/x86/kvm/vmx.c static int nested_vmx_set_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data) · line 1254
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+int r =0 ; +uint size; + + *gentry =0 ; + +if (is_long_mode (vcpu)) + size =sizeof (u64); +else + size =sizeof (u32);
I think the gpa is always 64 bit, regardless of the current mode.
arch/x86/kvm/vmx.c static int nested_vmx_set_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data)
Quoted 7 lines
arch/x86/kvm/vmx.c +6
-0+ + r =kvm_read_guest_virt (gva, gentry, + size, vcpu); +if (r) { +printk (KERN_ERR "%s cannot read guest vmcs addr %lx : %d\n" , + __func__, vcpu->arch.regs[VCPU_REGS_RAX ], r);
RAX may not be relevant. Just return, and the user can disassemble the instructions and see for themselves.
Quoted 24 lines
arch/x86/kvm/vmx.c +22
-0+return r; + } + +if (!IS_ALIGNED (*gentry,PAGE_SIZE )) { +printk (KERN_DEBUG "%s addr %llx not aligned\n" , + __func__, *gentry); +return 1 ; + } + +return 0 ; +} + +/* +* Decode the memory address (operand) of a vmx instruction according to Table 23-12/23-11 +* For additional information regarding offset calculation see 3.7.5 +*/ +static gva_t get_vmx_mem_address (struct kvm_vcpu *vcpu, +unsigned long exit_qualification, +u32 vmx_instruction_info) +{ +int scaling = vmx_instruction_info&3 ;/* bits 0:1 scaling */ +int addr_size = (vmx_instruction_info>>7 )&7 ;/* bits 7:9 address size, 0=16bit, 1=32bit, 2=64bit */
arch/x86/kvm/vmx.c static int nested_vmx_set_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data)
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+bool is_reg = vmx_instruction_info& (1u <<10 );/* bit 10 1=register operand, 0= memory */ +int seg_reg = (vmx_instruction_info>>15 )&7 ;/* bits 15:17 segment register */ +int index_reg = (vmx_instruction_info>>18 )&0xf ;/* bits 18:21 index register */ +bool index_is_valid = !(vmx_instruction_info& (1u <<22 ));/* bit 22 index register validity, 0=valid, 1=invalid */ +int base_reg = (vmx_instruction_info>>23 )&0xf ;/* bits 23:26 index register */ +bool base_is_valid = !(vmx_instruction_info& (1u <<27 ));/* bit 27 base register validity, 0=valid, 1=invalid */ +gva_t addr; + +if (is_reg) +return 0 ;
Should #UD.
Quoted 9 lines
arch/x86/kvm/vmx.c +8
-0+ +switch (addr_size) { +case 1 : + exit_qualification&=0xffffffff ;/* 32 high bits are undefied according to the spec, page 23-7 */ +break ; +case 2 : +break ; +default :
arch/x86/kvm/vmx.c static int handle_vmx_insn(struct kvm_vcpu *vcpu) · line 3546
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+return 0 ; + } + +/* Addr = segment_base + offset */ +/* offfset = Base + [Index * Scale] + Displacement, see Figure 3-11 */ + addr =vmx_get_segment_base (vcpu, seg_reg); +if (base_is_valid) + addr +=kvm_register_read (vcpu, base_reg); +if (index_is_valid) + addr +=kvm_register_read (vcpu, index_reg)*scaling;
Shouldn't this be a shift?
Wish we had something like that for emulate.c.
Quoted 10 lines
arch/x86/kvm/vmx.c +9
-0+ addr += exit_qualification;/* exit qualification holds the displacement, spec page 23-7 */ + +return addr; +} + +static int handle_vmclear (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct level_state *l2_state;
arch/x86/kvm/vmx.c static int handle_vmx_insn(struct kvm_vcpu *vcpu) · line 3565
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+gpa_t guest_vmcs_addr; +unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); +u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO ); +gva_t vmcs_gva; + +if (!nested_vmx_check_permission (vcpu)) +return 1 ; + + vmcs_gva =get_vmx_mem_address (vcpu, exit_qualification, + vmx_instruction_info);
I think you can let get_vmx_mem_address() do the vmread()s, simpler.
arch/x86/kvm/vmx.c static int handle_vmx_insn(struct kvm_vcpu *vcpu) · line 3576
Quoted 12 lines
arch/x86/kvm/vmx.c +11
-0+ +if (read_guest_vmcs_gpa (vcpu, vmcs_gva,&guest_vmcs_addr)) +return 1 ; + + vmx->nested.current_vmptr = guest_vmcs_addr; +if (!nested_map_current (vcpu)) +return 1 ; + + l2_state =&(to_vmx (vcpu)->nested.current_l2_page->l2_state); + l2_state->vmclear =1 ; +nested_free_current_vmcs (vcpu);
Why free? Isn't the purpose of the list to keep those active?
arch/x86/kvm/vmx.c static int handle_vmx_insn(struct kvm_vcpu *vcpu) · line 3587
Quoted 12 lines
arch/x86/kvm/vmx.c +11
-0+ + vmx->nested.current_vmptr =-1ull ; + +nested_unmap_current (vcpu); + +skip_emulated_instruction (vcpu); +clear_rflags_cf_zf (vcpu); + +return 1 ; +} +
As usual, if you can split some of the infrastructure into separate patches, it would help review.
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: [PATCH 2/7] Nested VMX patch 2 implements vmclear
In reply to [email protected]Quoted 272 lines · On Thu, Dec 10, 2009 at 08:38:24PM +0200, [email protected] wrote:
From: Orit Wasserman <[email protected]>
--- arch/x86/kvm/vmx.c | 235 +++++++++++++++++++++++++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 5 +- arch/x86/kvm/x86.h | 3 + 3 files changed, 240 insertions(+), 3 deletions(-)arch/x86/kvm/vmx.c +205
-0@@ -93,13 +93,39 @@ struct shared_msr_entry { };struct __attribute__ ((__packed__ ))level_state { +/* Has the level1 guest done vmclear? */ +bool vmclear; +}; + +/* +* This structure is mapped to guest memory. +* It is packed in order to preseve the binary content +* after live migration. +* If there are changed in the content or layout the revision_id must be updated. +*/ +struct __attribute__ ((__packed__ ))nested_vmcs_page { +u32 revision_id; +u32 abort; +struct level_state l2_state; +}; + +struct nested_vmcs_list { +struct list_head list; +gpa_t vmcs_addr; +struct vmcs *l2_vmcs; };struct nested_vmx {/* Has the level1 guest done vmxon? */ bool vmxon; +/* What is the location of the current vmcs l1 keeps for l2 */ +gpa_t current_vmptr;/* Level 1 state for switching to level 2 and back */ struct level_state *l1_state; +/* list of vmcs for each l2 guest created by l1 */ +struct list_head l2_vmcs_list; +/* l2 page corresponding to the current vmcs set by l1 */ +struct nested_vmcs_page *current_l2_page; };struct vcpu_vmx {@@ -156,6 +182,76 @@ static inline struct vcpu_vmx *to_vmx(struct kvm_vcpu *vcpu) return container_of(vcpu, struct vcpu_vmx, vcpu); } +static struct page *nested_get_page (struct kvm_vcpu *vcpu, +u64 vmcs_addr) +{ +struct page *vmcs_page =NULL ; + +down_read (¤t->mm->mmap_sem); + vmcs_page =gfn_to_page (vcpu->kvm, vmcs_addr >>PAGE_SHIFT ); +up_read (¤t->mm->mmap_sem); + +if (is_error_page (vmcs_page)) { +printk (KERN_ERR "%s error allocating page 0x%llx\n" , + __func__, vmcs_addr); +kvm_release_page_clean (vmcs_page); +return NULL ; + } + +return vmcs_page; + +} + +static int nested_map_current (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct page *vmcs_page = +nested_get_page (vcpu, vmx->nested.current_vmptr); +struct nested_vmcs_page *mapped_page; + +if (vmcs_page ==NULL ) { +printk (KERN_INFO "%s: failure in nested_get_page\n" , __func__); +return 0 ; + } + +if (vmx->nested.current_l2_page) { +printk (KERN_INFO "%s: shadow vmcs already mapped\n" , __func__); +WARN_ON (1 ); +return 0 ; + } + + mapped_page =kmap_atomic (vmcs_page,KM_USER0 ); + +if (!mapped_page) { +printk (KERN_INFO "%s: error in kmap_atomic\n" , __func__); +return 0 ; + } + + vmx->nested.current_l2_page = mapped_page; + +return 1 ; +} + +static void nested_unmap_current (struct kvm_vcpu *vcpu) +{ +struct page *page; +struct vcpu_vmx *vmx =to_vmx (vcpu); + +if (!vmx->nested.current_l2_page) { +printk (KERN_INFO "Shadow vmcs already unmapped\n" ); +WARN_ON (1 ); +return ; + } + + page =kmap_atomic_to_page (vmx->nested.current_l2_page); + +kunmap_atomic (vmx->nested.current_l2_page,KM_USER0 ); + +kvm_release_page_dirty (page); + + vmx->nested.current_l2_page =NULL ; +} +static int init_rmode (struct kvm *kvm);static u64 construct_eptp (unsigned long root_hpa);@@ -1144,6 +1240,35 @@ static int nested_vmx_set_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data)return 0 ; } +static int read_guest_vmcs_gpa (struct kvm_vcpu *vcpu,gva_t gva,u64 *gentry) +{ +int r =0 ; +uint size; + + *gentry =0 ; + +if (is_long_mode (vcpu)) + size =sizeof (u64); +else + size =sizeof (u32); + + r =kvm_read_guest_virt (gva, gentry, + size, vcpu); +if (r) { +printk (KERN_ERR "%s cannot read guest vmcs addr %lx : %d\n" , + __func__, vcpu->arch.regs[VCPU_REGS_RAX ], r); +return r; + } + +if (!IS_ALIGNED (*gentry,PAGE_SIZE )) { +printk (KERN_DEBUG "%s addr %llx not aligned\n" , + __func__, *gentry); +return 1 ; + } + +return 0 ; +} +/* * Writes msr value into into the appropriate "register". * Returns 0 on success, non-0 otherwise. @@ -1316,6 +1441,7 @@ static int create_l1_state(struct kvm_vcpu *vcpu) }else return 0 ; +INIT_LIST_HEAD (&(vmx->nested.l2_vmcs_list));return 0 ; }@@ -1488,15 +1614,35 @@ static void free_vmcs(struct vmcs *vmcs)free_pages ((unsigned long )vmcs, vmcs_config.order); } +static void nested_free_current_vmcs (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct nested_vmcs_list *list_item, *n; + +list_for_each_entry_safe (list_item, n, &vmx->nested.l2_vmcs_list, list) +if (list_item->vmcs_addr == vmx->nested.current_vmptr) { +free_vmcs (list_item->l2_vmcs); +list_del (&(list_item->list)); +return ; + } +} +static void free_l1_state (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu); +struct nested_vmcs_list *list_item, *n;if (!vmx->nested.l1_state)return ;kfree (vmx->nested.l1_state); vmx->nested.l1_state =NULL ; + +list_for_each_entry_safe (list_item, n, &vmx->nested.l2_vmcs_list, + list) { +free_vmcs (list_item->l2_vmcs); +list_del (&(list_item->list)); + } }@@ -3352,6 +3498,93 @@ static int handle_vmx_insn(struct kvm_vcpu *vcpu)return 1 ; } +static void clear_rflags_cf_zf (struct kvm_vcpu *vcpu) +{ +unsigned long rflags; + rflags =vmx_get_rflags (vcpu); + rflags &= ~(X86_EFLAGS_CF |X86_EFLAGS_ZF ); +vmx_set_rflags (vcpu, rflags); +} + +/* +* Decode the memory address (operand) of a vmx instruction according to Table 23-12/23-11 +* For additional information regarding offset calculation see 3.7.5 +*/ +static gva_t get_vmx_mem_address (struct kvm_vcpu *vcpu, +unsigned long exit_qualification, +u32 vmx_instruction_info) +{ +int scaling = vmx_instruction_info &3 ;/* bits 0:1 scaling */ +int addr_size = (vmx_instruction_info >>7 ) &7 ;/* bits 7:9 address size, 0=16bit, 1=32bit, 2=64bit */ +bool is_reg = vmx_instruction_info & (1u <<10 );/* bit 10 1=register operand, 0= memory */ +int seg_reg = (vmx_instruction_info >>15 ) &7 ;/* bits 15:17 segment register */ +int index_reg = (vmx_instruction_info >>18 ) &0xf ;/* bits 18:21 index register */ +bool index_is_valid = !(vmx_instruction_info & (1u <<22 ));/* bit 22 index register validity, 0=valid, 1=invalid */ +int base_reg = (vmx_instruction_info >>23 ) &0xf ;/* bits 23:26 index register */ +bool base_is_valid = !(vmx_instruction_info & (1u <<27 ));/* bit 27 base register validity, 0=valid, 1=invalid */ +gva_t addr; + +if (is_reg) +return 0 ; + +switch (addr_size) { +case 1 : + exit_qualification &=0xffffffff ;/* 32 high bits are undefied according to the spec, page 23-7 */ +break ; +case 2 : +break ; +default : +return 0 ; + } + +/* Addr = segment_base + offset */ +/* offfset = Base + [Index * Scale] + Displacement, see Figure 3-11 */ + addr =vmx_get_segment_base (vcpu, seg_reg); +if (base_is_valid) + addr +=kvm_register_read (vcpu, base_reg); +if (index_is_valid) + addr +=kvm_register_read (vcpu, index_reg)*scaling; + addr += exit_qualification;/* exit qualification holds the displacement, spec page 23-7 */ + +return addr; +} + +static int handle_vmclear (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); +struct level_state *l2_state; +gpa_t guest_vmcs_addr; +unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION ); +u32 vmx_instruction_info =vmcs_read32 (VMX_INSTRUCTION_INFO ); +gva_t vmcs_gva;
arch/x86/kvm/vmx.c static int handle_vmx_insn(struct kvm_vcpu *vcpu) · line 3569
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+ +if (!nested_vmx_check_permission (vcpu)) +return 1 ; + + vmcs_gva =get_vmx_mem_address (vcpu, exit_qualification, + vmx_instruction_info); + +if (read_guest_vmcs_gpa (vcpu, vmcs_gva, &guest_vmcs_addr)) +return 1 ; +
Should check that vmcs address is 4K aligned and given address is not equal to vmxon pointer.
arch/x86/kvm/vmx.c static int handle_vmx_insn(struct kvm_vcpu *vcpu) · line 3570
Quoted 1 line
arch/x86/kvm/vmx.c +1
-0+ vmx->nested.current_vmptr = guest_vmcs_addr;
vmclear doesn't change current vmcs pointer.
arch/x86/kvm/vmx.c static int handle_vmx_insn(struct kvm_vcpu *vcpu) · line 3579
Quoted 9 lines
arch/x86/kvm/vmx.c +9
-0+if (!nested_map_current (vcpu)) +return 1 ; + + l2_state = &(to_vmx (vcpu)->nested.current_l2_page->l2_state); + l2_state->vmclear =1 ; +nested_free_current_vmcs (vcpu); + + vmx->nested.current_vmptr =-1ull ; +
vmclear reset current vmcs pointer to -1 only if it was called with current vmcs pointer as an argument.
Quoted 64 lines
arch/x86/kvm/vmx.c +9
-1+nested_unmap_current (vcpu); + +skip_emulated_instruction (vcpu); +clear_rflags_cf_zf (vcpu); + +return 1 ; +} +static int handle_vmoff (struct kvm_vcpu *vcpu) {struct vcpu_vmx *vmx =to_vmx (vcpu);@@ -3695,7 +3928,7 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { [EXIT_REASON_HLT] = handle_halt, [EXIT_REASON_INVLPG] = handle_invlpg, [EXIT_REASON_VMCALL] = handle_vmcall, - [EXIT_REASON_VMCLEAR] = handle_vmx_insn, + [EXIT_REASON_VMCLEAR] = handle_vmclear, [EXIT_REASON_VMLAUNCH] = handle_vmx_insn, [EXIT_REASON_VMPTRLD] = handle_vmx_insn, [EXIT_REASON_VMPTRST] = handle_vmx_insn,arch/x86/kvm/x86.c +3
-2@@ -2773,8 +2773,8 @@ static int vcpu_mmio_read(struct kvm_vcpu *vcpu, gpa_t addr, int len, void *v)return kvm_io_bus_read (&vcpu->kvm->mmio_bus, addr, len, v); } -static int kvm_read_guest_virt (gva_t addr,void *val,unsigned int bytes, -struct kvm_vcpu *vcpu) +int kvm_read_guest_virt (gva_t addr,void *val,unsigned int bytes, +struct kvm_vcpu *vcpu) {void *data = val;int r =X86EMUL_CONTINUE ;@@ -2802,6 +2802,7 @@ static int kvm_read_guest_virt(gva_t addr, void *val, unsigned int bytes, out:return r; } +EXPORT_SYMBOL_GPL (kvm_read_guest_virt);static int kvm_write_guest_virt (gva_t addr,void *val,unsigned int bytes,struct kvm_vcpu *vcpu)arch/x86/kvm/x86.h +3
-0@@ -35,6 +35,9 @@ static inline bool kvm_exception_is_soft(unsigned int nr)struct kvm_cpuid_entry2 *kvm_find_cpuid_entry (struct kvm_vcpu *vcpu,u32 function,u32 index); +int kvm_read_guest_virt (gva_t addr,void *val,unsigned int bytes, +struct kvm_vcpu *vcpu); +extern int nested;#endif Signature · 6 lines
1.6.0.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gleb.
Re: [PATCH 1/7] Nested VMX patch 1 implements vmon and vmoff
In reply to [email protected]Quoted 3 lines · On 12/10/2009 08:38 PM, [email protected] wrote:
From: Orit Wasserman<[email protected]>
Missing changelog entry. Please use the format common to all kvm patches.
arch/x86/kvm/svm.c static int npt = 1; · line 124
Quoted 12 lines
arch/x86/kvm/svm.c +0
-3@@ -121,9 +121,6 @@ static int npt = 1;module_param (npt, int,S_IRUGO ); -static int nested =1 ; -module_param (nested, int,S_IRUGO ); -
Separate moving 'nested' into a different patch.
arch/x86/kvm/svm.c static int npt = 1;
Quoted 9 lines
arch/x86/kvm/svm.c +8
-0+struct __attribute__ ((__packed__ ))level_state { +}; + +struct nested_vmx { +/* Has the level1 guest done vmxon? */ +bool vmxon; +/* Level 1 state for switching to level 2 and back */ +struct level_state *l1_state;
If this doesn't grow too large, can keep it as a member instead of a pointer.
Quoted 20 lines
arch/x86/kvm/svm.c +7
-0+}; +static inline struct vcpu_vmx *to_vmx (struct kvm_vcpu *vcpu)@@ -201,6 +214,7 @@ static struct kvm_vmx_segment_field {static u64 host_efer;static void ept_save_pdptrs (struct kvm_vcpu *vcpu); +static int create_l1_state (struct kvm_vcpu *vcpu);/* * Keep MSR_K6_STAR at the end, as setup_msrs() will try to optimize it @@ -961,6 +975,95 @@ static void guest_write_tsc(u64 guest_tsc, u64 host_tsc) }/* +* Handles msr read for nested virtualization +*/ +static int nested_vmx_get_msr (struct kvm_vcpu *vcpu,u32 msr_index, +u64 *pdata)
arch/x86/kvm/svm.c static void guest_write_tsc(u64 guest_tsc, u64 host_tsc) · line 991
Quoted 10 lines
arch/x86/kvm/svm.c +9
-0+{ +u64 vmx_msr =0 ; + +switch (msr_index) { +case MSR_IA32_FEATURE_CONTROL : + *pdata =0 ; +break ; +case MSR_IA32_VMX_BASIC : + *pdata =0 ;
Not needed.
arch/x86/kvm/svm.c static void guest_write_tsc(u64 guest_tsc, u64 host_tsc) · line 994
Quoted 3 lines
arch/x86/kvm/svm.c +2
-0+rdmsrl (MSR_IA32_VMX_BASIC , vmx_msr); + *pdata = (vmx_msr&0x00ffffcfffffffff );
Please use symbolic constants.
arch/x86/kvm/svm.c static void guest_write_tsc(u64 guest_tsc, u64 host_tsc) · line 1001
Quoted 7 lines
arch/x86/kvm/svm.c +6
-0+break ; +case MSR_IA32_VMX_PINBASED_CTLS : +rdmsrl (MSR_IA32_VMX_PINBASED_CTLS , vmx_msr); + *pdata = (PIN_BASED_EXT_INTR_MASK & vmcs_config.pin_based_exec_ctrl) | + (PIN_BASED_NMI_EXITING & vmcs_config.pin_based_exec_ctrl) | + (PIN_BASED_VIRTUAL_NMIS & vmcs_config.pin_based_exec_ctrl);
Don't understand. You read vmx_msr and then use vmcs_config?
Quoted 25 lines
arch/x86/kvm/svm.c +24
-0+case MSR_IA32_VMX_PROCBASED_CTLS : + { + u32 vmx_msr_high, vmx_msr_low; + u32 control = CPU_BASED_HLT_EXITING | +#ifdef CONFIG_X86_64 + CPU_BASED_CR8_LOAD_EXITING | + CPU_BASED_CR8_STORE_EXITING | +#endif + CPU_BASED_CR3_LOAD_EXITING | + CPU_BASED_CR3_STORE_EXITING | + CPU_BASED_USE_IO_BITMAPS | + CPU_BASED_MOV_DR_EXITING | + CPU_BASED_USE_TSC_OFFSETING | + CPU_BASED_INVLPG_EXITING | + CPU_BASED_TPR_SHADOW | + CPU_BASED_USE_MSR_BITMAPS | + CPU_BASED_ACTIVATE_SECONDARY_CONTROLS; + + rdmsr(MSR_IA32_VMX_PROCBASED_CTLS, vmx_msr_low, vmx_msr_high); + + control&= vmx_msr_high; /* bit == 0 in high word ==> must be zero */ + control |= vmx_msr_low; /* bit == 1 in low word ==> must be one */ + + *pdata = (CPU_BASED_HLT_EXITING& control) |
arch/x86/kvm/svm.c static void guest_write_tsc(u64 guest_tsc, u64 host_tsc) · line 1035
Quoted 10 lines
arch/x86/kvm/svm.c +10
-0+#ifdef CONFIG_X86_64 + (CPU_BASED_CR8_LOAD_EXITING & control) | + (CPU_BASED_CR8_STORE_EXITING & control) | +#endif + (CPU_BASED_CR3_LOAD_EXITING & control) | + (CPU_BASED_CR3_STORE_EXITING & control) | + (CPU_BASED_USE_IO_BITMAPS & control) | + (CPU_BASED_MOV_DR_EXITING & control) | + (CPU_BASED_USE_TSC_OFFSETING & control) | + (CPU_BASED_INVLPG_EXITING & control) ;
What about the high word of the msr? Will it always allow 0?
Quoted 6 lines
arch/x86/kvm/svm.c +3
-0/* +* Writes msr value for nested virtualization +* Returns 0 on success, non-0 otherwise. +*/
arch/x86/kvm/svm.c static void guest_write_tsc(u64 guest_tsc, u64 host_tsc)
Quoted 10 lines
arch/x86/kvm/svm.c +10
-0+static int nested_vmx_set_msr (struct kvm_vcpu *vcpu,u32 msr_index,u64 data) +{ +switch (msr_index) { +case MSR_IA32_FEATURE_CONTROL : +if ((data& (FEATURE_CONTROL_LOCKED | +FEATURE_CONTROL_VMXON_ENABLED )) + != (FEATURE_CONTROL_LOCKED | +FEATURE_CONTROL_VMXON_ENABLED )) +return 1 ; +break ;
Need to trap if unsupported bits are set.
Need a way for userspace to write these msrs, so that live migration to an older kvm can work. We do the same thing with cpuid - userspace sets cpuid to values that are common across the migration cluster.
arch/x86/kvm/svm.c
Quoted 7 lines
arch/x86/kvm/svm.c +6
-0+static void free_l1_state (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); + +if (!vmx->nested.l1_state) +return ;
Check isn't needed, kfree() likes NULLs.
Quoted 13 lines
arch/x86/kvm/svm.c +6
-0+ +kfree (vmx->nested.l1_state); + vmx->nested.l1_state =NULL ; +} + +struct kvm_shared_msrs_global {
arch/x86/kvm/svm.c void kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) · line 515
Quoted 8 lines
arch/x86/kvm/svm.c +1
-1@@ -505,7 +509,7 @@ void kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)return ; } -if (cr4&X86_CR4_VMXE ) { +if (cr4&X86_CR4_VMXE && !nested) {printk (KERN_DEBUG "set_cr4: #GP, setting VMXE\n" );kvm_inject_gp (vcpu,0 );return ;
Some bits are required to be set when VMXE is enabled.
Please split the MSR changes into a separate patch. Even cr4 is better on its own.
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: [PATCH 1/7] Nested VMX patch 1 implements vmon and vmoff
In reply to [email protected]Quoted 223 lines · On Thu, Dec 10, 2009 at 08:38:23PM +0200, [email protected] wrote:
From: Orit Wasserman <[email protected]>
--- arch/x86/kvm/svm.c | 3 - arch/x86/kvm/vmx.c | 265 +++++++++++++++++++++++++++++++++++++++++++++++++++- arch/x86/kvm/x86.c | 11 ++- arch/x86/kvm/x86.h | 2 + 4 files changed, 274 insertions(+), 7 deletions(-)arch/x86/kvm/svm.c +0
-3@@ -121,9 +121,6 @@ static int npt = 1;module_param (npt, int,S_IRUGO ); -static int nested =1 ; -module_param (nested, int,S_IRUGO ); -static void svm_flush_tlb (struct kvm_vcpu *vcpu);static void svm_complete_interrupts (struct vcpu_svm *svm);arch/x86/kvm/vmx.c +143
-0@@ -92,6 +92,16 @@ struct shared_msr_entry {u64 mask; }; +struct __attribute__ ((__packed__ ))level_state { +}; + +struct nested_vmx { +/* Has the level1 guest done vmxon? */ +bool vmxon; +/* Level 1 state for switching to level 2 and back */ +struct level_state *l1_state; +}; +struct vcpu_vmx {struct kvm_vcpu vcpu;struct list_head local_vcpus_link;@@ -136,6 +146,9 @@ struct vcpu_vmx {ktime_t entry_time;s64 vnmi_blocked_time;u32 exit_reason; + +/* Nested vmx */ +struct nested_vmx nested; };static inline struct vcpu_vmx *to_vmx (struct kvm_vcpu *vcpu)@@ -201,6 +214,7 @@ static struct kvm_vmx_segment_field {static u64 host_efer;static void ept_save_pdptrs (struct kvm_vcpu *vcpu); +static int create_l1_state (struct kvm_vcpu *vcpu);/* * Keep MSR_K6_STAR at the end, as setup_msrs() will try to optimize it @@ -961,6 +975,95 @@ static void guest_write_tsc(u64 guest_tsc, u64 host_tsc) }/* +* Handles msr read for nested virtualization +*/ +static int nested_vmx_get_msr (struct kvm_vcpu *vcpu,u32 msr_index, +u64 *pdata) +{ +u64 vmx_msr =0 ; + +switch (msr_index) { +case MSR_IA32_FEATURE_CONTROL : + *pdata =0 ; +break ; +case MSR_IA32_VMX_BASIC : + *pdata =0 ; +rdmsrl (MSR_IA32_VMX_BASIC , vmx_msr); + *pdata = (vmx_msr &0x00ffffcfffffffff ); +break ; +case MSR_IA32_VMX_PINBASED_CTLS : +rdmsrl (MSR_IA32_VMX_PINBASED_CTLS , vmx_msr); + *pdata = (PIN_BASED_EXT_INTR_MASK & vmcs_config.pin_based_exec_ctrl) | + (PIN_BASED_NMI_EXITING & vmcs_config.pin_based_exec_ctrl) | + (PIN_BASED_VIRTUAL_NMIS & vmcs_config.pin_based_exec_ctrl); +break ; +case MSR_IA32_VMX_PROCBASED_CTLS : + { +u32 vmx_msr_high, vmx_msr_low; + u32 control = CPU_BASED_HLT_EXITING | +#ifdef CONFIG_X86_64 +CPU_BASED_CR8_LOAD_EXITING | +CPU_BASED_CR8_STORE_EXITING | +#endif +CPU_BASED_CR3_LOAD_EXITING | +CPU_BASED_CR3_STORE_EXITING | +CPU_BASED_USE_IO_BITMAPS | +CPU_BASED_MOV_DR_EXITING | +CPU_BASED_USE_TSC_OFFSETING | +CPU_BASED_INVLPG_EXITING | +CPU_BASED_TPR_SHADOW | +CPU_BASED_USE_MSR_BITMAPS | +CPU_BASED_ACTIVATE_SECONDARY_CONTROLS ; + +rdmsr (MSR_IA32_VMX_PROCBASED_CTLS , vmx_msr_low, vmx_msr_high); + + control &= vmx_msr_high;/* bit == 0 in high word ==> must be zero */ + control |= vmx_msr_low;/* bit == 1 in low word ==> must be one */ + + *pdata = (CPU_BASED_HLT_EXITING & control) | +#ifdef CONFIG_X86_64 + (CPU_BASED_CR8_LOAD_EXITING & control) | + (CPU_BASED_CR8_STORE_EXITING & control) | +#endif + (CPU_BASED_CR3_LOAD_EXITING & control) | + (CPU_BASED_CR3_STORE_EXITING & control) | + (CPU_BASED_USE_IO_BITMAPS & control) | + (CPU_BASED_MOV_DR_EXITING & control) | + (CPU_BASED_USE_TSC_OFFSETING & control) | + (CPU_BASED_INVLPG_EXITING & control) ; + +if (cpu_has_secondary_exec_ctrls ()) + *pdata |=CPU_BASED_ACTIVATE_SECONDARY_CONTROLS ; + +if (vm_need_tpr_shadow (vcpu->kvm)) + *pdata |=CPU_BASED_TPR_SHADOW ; + break; + } +case MSR_IA32_VMX_EXIT_CTLS : + *pdata =0 ; +#ifdef CONFIG_X86_64 + *pdata |=VM_EXIT_HOST_ADDR_SPACE_SIZE ; +#endif +break ; +case MSR_IA32_VMX_ENTRY_CTLS : + *pdata =0 ; +break ; +case MSR_IA32_VMX_PROCBASED_CTLS2 : + *pdata =0 ; +if (vm_need_virtualize_apic_accesses (vcpu->kvm)) + *pdata |=SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES ; +break ; +case MSR_IA32_VMX_EPT_VPID_CAP : + *pdata =0 ; +break ; +default : +return 1 ; + } + +return 0 ; +} + +/* * Reads an msr value (of 'msr_index') into 'pdata'. * Returns 0 on success, non-0 otherwise. * Assumes vcpu_load() was already called. @@ -1004,6 +1107,9 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 *pdata)break ;default :vmx_load_host_state (to_vmx (vcpu)); +if (nested && + !nested_vmx_get_msr (vcpu, msr_index, &data)) +break ; msr =find_msr_entry (to_vmx (vcpu), msr_index);if (msr) {vmx_load_host_state (to_vmx (vcpu));@@ -1018,6 +1124,27 @@ static int vmx_get_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 *pdata) }/* +* Writes msr value for nested virtualization +* Returns 0 on success, non-0 otherwise. +*/ +static int nested_vmx_set_msr (struct kvm_vcpu *vcpu,u32 msr_index,u64 data) +{ +switch (msr_index) { +case MSR_IA32_FEATURE_CONTROL : +if ((data & (FEATURE_CONTROL_LOCKED | +FEATURE_CONTROL_VMXON_ENABLED )) + != (FEATURE_CONTROL_LOCKED | +FEATURE_CONTROL_VMXON_ENABLED )) +return 1 ; +break ; +default : +return 1 ; + } + +return 0 ; +} + +/* * Writes msr value into into the appropriate "register". * Returns 0 on success, non-0 otherwise. * Assumes vcpu_load() was already called. @@ -1067,6 +1194,9 @@ static int vmx_set_msr(struct kvm_vcpu *vcpu, u32 msr_index, u64 data) }/* Otherwise falls through to kvm_set_msr_common */ default : +if (nested && + !nested_vmx_set_msr (vcpu, msr_index, data)) +break ; msr =find_msr_entry (vmx, msr_index);if (msr) {vmx_load_host_state (vmx);@@ -1163,6 +1293,31 @@ static void vmclear_local_vcpus(void)__vcpu_clear (vmx); } +static struct level_state *create_state (void ) +{ +struct level_state *state =NULL ; + + state =kzalloc (sizeof (struct level_state ),GFP_KERNEL ); +if (!state) { +printk (KERN_INFO "Error create level state\n" ); +return NULL ; + } +return state; +} + +static int create_l1_state (struct kvm_vcpu *vcpu)
arch/x86/kvm/vmx.c static void vmclear_local_vcpus(void) · line 1318
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+{ +struct vcpu_vmx *vmx =to_vmx (vcpu); + +if (!vmx->nested.l1_state) { + vmx->nested.l1_state =create_state (); +if (!vmx->nested.l1_state) +return -ENOMEM ; + }else +return 0 ; +
Else clause is not needed.
Quoted 108 lines
arch/x86/kvm/vmx.c +88
-0+return 0 ; +}/* Just like cpu_vmxoff(), but with the __kvm_handle_fault_on_reboot() * tricks. @@ -1333,6 +1488,18 @@ static void free_vmcs(struct vmcs *vmcs)free_pages ((unsigned long )vmcs, vmcs_config.order); } +static void free_l1_state (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); + +if (!vmx->nested.l1_state) +return ; + +kfree (vmx->nested.l1_state); + vmx->nested.l1_state =NULL ; +} + +static void free_kvm_area (void ) {int cpu;@@ -3146,12 +3313,105 @@ static int handle_vmcall(struct kvm_vcpu *vcpu)return 1 ; } +/* +* Check to see if vcpu can execute vmx command +* Inject the corrseponding exception +*/ +static int nested_vmx_check_permission (struct kvm_vcpu *vcpu) +{ +struct kvm_segment cs; +struct vcpu_vmx *vmx =to_vmx (vcpu); + +vmx_get_segment (vcpu, &cs,VCPU_SREG_CS ); + +if (!vmx->nested.vmxon) { +pr_debug ("%s: vmx not on\n" , __func__); +kvm_queue_exception (vcpu,UD_VECTOR ); +return 0 ; + } + +if ((vmx_get_rflags (vcpu) &X86_EFLAGS_VM ) || + (is_long_mode (vcpu) && !cs.l)) { +pr_debug ("%s: invalid mode cs.l %d is_long mode %d\n" , + __func__, cs.l,is_long_mode (vcpu)); +kvm_queue_exception (vcpu,UD_VECTOR ); +return 0 ; + } + +if (vmx_get_cpl (vcpu)) { +kvm_inject_gp (vcpu,0 ); +return 0 ; + } + +return 1 ; +} +static int handle_vmx_insn (struct kvm_vcpu *vcpu) {kvm_queue_exception (vcpu,UD_VECTOR );return 1 ; } +static int handle_vmoff (struct kvm_vcpu *vcpu) +{ +struct vcpu_vmx *vmx =to_vmx (vcpu); + +if (!nested_vmx_check_permission (vcpu)) +return 1 ; + + vmx->nested.vmxon =0 ; + +free_l1_state (vcpu); + +skip_emulated_instruction (vcpu); +return 1 ; +} + +static int handle_vmon (struct kvm_vcpu *vcpu) +{ +struct kvm_segment cs; +struct vcpu_vmx *vmx =to_vmx (vcpu); + +if (!nested) { +pr_debug ("%s: nested vmx not enabled\n" , __func__); +kvm_queue_exception (vcpu,UD_VECTOR ); +return 1 ; + } + +vmx_get_segment (vcpu, &cs,VCPU_SREG_CS ); + +if (!(vcpu->arch.cr4 &X86_CR4_VMXE ) || + !(vcpu->arch.cr0 &X86_CR0_PE ) || + (vmx_get_rflags (vcpu) &X86_EFLAGS_VM )) { +kvm_queue_exception (vcpu,UD_VECTOR ); +printk (KERN_INFO "%s invalid register state\n" , __func__); +return 1 ; + } + +if (is_long_mode (vcpu) && !cs.l) { +kvm_queue_exception (vcpu,UD_VECTOR ); +printk (KERN_INFO "%s invalid register state\n" , __func__); +return 1 ; + }
arch/x86/kvm/vmx.c static int handle_vmcall(struct kvm_vcpu *vcpu) · line 3405
Quoted 10 lines
arch/x86/kvm/vmx.c +10
-0+ +if (vmx_get_cpl (vcpu)) { +printk (KERN_INFO "%s no permission\n" , __func__); +kvm_inject_gp (vcpu,0 ); +return 1 ; + } + +if (create_l1_state (vcpu)) { +printk (KERN_ERR "%s create_l1_state failed\n" , __func__); +kvm_queue_exception (vcpu,UD_VECTOR );
Should we send UD exception if there is internal error? May be doing VMfailinvalid would be more appropriate? Also this function doesn't handle all errors (address alignment, version checking, register operand).
Quoted 85 lines
arch/x86/kvm/vmx.c +12
-2+return 1 ; + } + + vmx->nested.vmxon =1 ; + +skip_emulated_instruction (vcpu); +return 1 ; +} +static int handle_invlpg (struct kvm_vcpu *vcpu) {unsigned long exit_qualification =vmcs_readl (EXIT_QUALIFICATION );@@ -3442,8 +3702,8 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { [EXIT_REASON_VMREAD] = handle_vmx_insn, [EXIT_REASON_VMRESUME] = handle_vmx_insn, [EXIT_REASON_VMWRITE] = handle_vmx_insn, - [EXIT_REASON_VMOFF] = handle_vmx_insn, - [EXIT_REASON_VMON] = handle_vmx_insn, + [EXIT_REASON_VMOFF] = handle_vmoff, + [EXIT_REASON_VMON] = handle_vmon, [EXIT_REASON_TPR_BELOW_THRESHOLD] = handle_tpr_below_threshold, [EXIT_REASON_APIC_ACCESS] = handle_apic_access, [EXIT_REASON_WBINVD] = handle_wbinvd,@@ -3823,6 +4083,7 @@ static void vmx_free_vcpu(struct kvm_vcpu *vcpu)if (vmx->vpid !=0 )__clear_bit (vmx->vpid, vmx_vpid_bitmap);spin_unlock (&vmx_vpid_lock); +free_l1_state (vcpu);vmx_free_vmcs (vcpu);kfree (vmx->guest_msrs);kvm_vcpu_uninit (vcpu);arch/x86/kvm/x86.c +9
-2@@ -88,6 +88,10 @@ EXPORT_SYMBOL_GPL(kvm_x86_ops);int ignore_msrs =0 ;module_param_named (ignore_msrs, ignore_msrs, bool,S_IRUGO |S_IWUSR ); +int nested =1 ; +EXPORT_SYMBOL_GPL (nested); +module_param (nested, int,S_IRUGO ); +#define KVM_NR_SHARED_MSRS 16struct kvm_shared_msrs_global {@@ -505,7 +509,7 @@ void kvm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)return ; } -if (cr4 &X86_CR4_VMXE ) { +if (cr4 &X86_CR4_VMXE && !nested) {printk (KERN_DEBUG "set_cr4: #GP, setting VMXE\n" );kvm_inject_gp (vcpu,0 );return ;@@ -615,7 +619,10 @@ static u32 msrs_to_save[] = {#ifdef CONFIG_X86_64 MSR_CSTAR ,MSR_KERNEL_GS_BASE ,MSR_SYSCALL_MASK ,MSR_LSTAR , #endif -MSR_IA32_TSC ,MSR_IA32_PERF_STATUS ,MSR_IA32_CR_PAT ,MSR_VM_HSAVE_PA +MSR_IA32_TSC ,MSR_IA32_PERF_STATUS ,MSR_IA32_CR_PAT ,MSR_VM_HSAVE_PA , +MSR_IA32_FEATURE_CONTROL ,MSR_IA32_VMX_BASIC ,MSR_IA32_VMX_PINBASED_CTLS , +MSR_IA32_VMX_PROCBASED_CTLS ,MSR_IA32_VMX_EXIT_CTLS ,MSR_IA32_VMX_ENTRY_CTLS , +MSR_IA32_VMX_PROCBASED_CTLS2 ,MSR_IA32_VMX_EPT_VPID_CAP ,MSR_IA32_FEATURE_CONTROL };static unsigned num_msrs_to_save;arch/x86/kvm/x86.h +2
-0@@ -35,4 +35,6 @@ static inline bool kvm_exception_is_soft(unsigned int nr)struct kvm_cpuid_entry2 *kvm_find_cpuid_entry (struct kvm_vcpu *vcpu,u32 function,u32 index); +extern int nested; +#endif Signature · 6 lines
1.6.0.4 -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gleb.
Re: [PATCH 1/7] Nested VMX patch 1 implements vmon and vmoff
Quoted 9 lines · Gleb Natapov
+
arch/x86/kvm/vmx.c static int handle_vmcall(struct kvm_vcpu *vcpu) · line 3405
Quoted 4 lines
arch/x86/kvm/vmx.c +3
-0+if (create_l1_state (vcpu)) { +printk (KERN_ERR "%s create_l1_state failed\n" , __func__); +kvm_queue_exception (vcpu,UD_VECTOR );Should we send UD exception if there is internal error? May be doing VMfailinvalid would be more appropriate? Also this function doesn't handle all errors (address alignment, version checking, register operand).
Host errors like -ENOMEM should propagate to the return code of ioctl().
Signature · 1 line
error compiling committee.c: too many arguments to function
Re: [PATCH 1/7] Nested VMX patch 1 implements vmon and vmoff
Quoted 10 lines · Avi Kivity
Quoted 7 lines · Gleb Natapov
+
arch/x86/kvm/vmx.c static int handle_vmcall(struct kvm_vcpu *vcpu) · line 3405
Quoted 3 lines
arch/x86/kvm/vmx.c +3
-0+if (create_l1_state (vcpu)) { +printk (KERN_ERR "%s create_l1_state failed\n" , __func__); +kvm_queue_exception (vcpu,UD_VECTOR );Should we send UD exception if there is internal error? May be doing VMfailinvalid would be more appropriate? Also this function doesn't handle all errors (address alignment, version checking, register operand).
Quoted 2 lines
Host errors like
-ENOMEMshould propagate to the return code ofioctl().
Yes, even better.
--
Gleb.
Re: [PATCH 1/7] Nested VMX patch 1 implements vmon and vmoff
In reply to Gleb NatapovQuoted 3 lines · Gleb Natapov
arch/x86/kvm/x86.c EXPORT_SYMBOL_GPL(kvm_x86_ops); · line 92
Quoted 3 lines
arch/x86/kvm/x86.c +2
-0+int nested =1 ; +EXPORT_SYMBOL_GPL (nested);
Unless this is a lot better tested and audited wouldn't it make more sense to default it to off?
I don't think it's a big burden to let users set a special knob for this, but it would be a big problem if there was some kind of jail break hidden in there that could be exploited by malicious guests.
Since VMX was not originally designed to be nested that wouldn't surprise me.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Re: [PATCH 1/7] Nested VMX patch 1 implements vmon and vmoff
Quoted 10 lines · Andi Kleen
Quoted 4 lines · Gleb Natapov
arch/x86/kvm/x86.c EXPORT_SYMBOL_GPL(kvm_x86_ops); · line 92
Quoted 4 lines
arch/x86/kvm/x86.c +2
-0+int nested =1 ; +EXPORT_SYMBOL_GPL (nested);Unless this is a lot better tested and audited wouldn't it make more sense to default it to off?
This is actually a move of an existing svm-only variable, which defaults to enabled. Nested svm has been tested for a while.
Quoted 4 lines
I don't think it's a big burden to let users set a special knob for this, but it would be a big problem if there was some kind of jail break hidden in there that could be exploited by malicious guests.
True. It makes sense to have different defaults of vmx and svm.
Quoted 2 lines
Since VMX was not originally designed to be nested that wouldn't surprise me.
vmx was designed to correct the non-virtualizability of x86. It would have been criminal to design it without nesting in mind, especially given all the prior art.
vmx does support nesting, albeit not very efficiently.
Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
Re: [PATCH 1/7] Nested VMX patch 1 implements vmon and vmoff
Quoted 8 lines · Avi Kivity
Quoted 2 lines
Since VMX was not originally designed to be nested that wouldn't surprise me.
vmx was designed to correct the non-virtualizability of x86. It would have been criminal to design it without nesting in mind, especially given all the prior art.
vmx does support nesting, albeit not very efficiently.
I would say that VMX only supports nesting if you define "supports" as "does not make it impossible". The fact that VMX operations in executed in non-root mode are trapped is welcome, but there's so much more that could be done in hardware to make nesting "better supported" that I would hesitate to say that the current generation of VMX supports nesting.
Cheers,
Muli
Signature · 5 lines
Muli Ben-Yehuda | [email protected] | +972-4-8281080 Manager, Virtualization and Systems Architecture Master Inventor, IBM Research -- Haifa Second Workshop on I/O Virtualization (WIOV '10): http://sysrun.haifa.il.ibm.com/hrl/wiov2010/
Re: [PATCH 1/7] Nested VMX patch 1 implements vmon and vmoff
Quoted 8 lines · Muli Ben-Yehuda
I would say that VMX only supports nesting if you define "supports" as "does not make it impossible". The fact that VMX operations in executed in non-root mode are trapped is welcome, but there's so much more that could be done in hardware to make nesting "better supported" that I would hesitate to say that the current generation of VMX supports nesting.
I would phrase it as "there is so much less the hardware can do to make nesting better supported" (such as avoid using vmread/vmwrite). As nested svm shows, you don't really need much (nested paging is very important though).
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 4 lines · On 12/10/2009 08:38 PM, [email protected] wrote:
Avi,
We have addressed all of the comments, please apply.
I'm afraid there is still a lot of work remaining.
Quoted 4 lines
This work was inspired by the nested SVM support by Alexander Graf and Joerg Roedel.
Please try to make this as readable as the svm work. That means smaller patches, detailed changelogs, reduced forward declarations. I would appreciate documentation of the data structures. I am still confused about the triplication in kvm_vcpu, l1_state, and l2_state - there shouldn't be three, just two.
Signature · 1 line
error compiling committee.c: too many arguments to function