Introduce KVM_{GET|SET}_SREGS2 and fix PDPTR migration
Subsystems (4)
v2 cover Introduce KVM_{GET|SET}_SREGS2 and fix PDPTR migration
This patch set aims to fix few flaws that were discovered in KVM_{GET|SET}_SREGS on x86:
* There is no support for reading/writing PDPTRs although these are considered to be part of the guest state.
* There is useless interrupt bitmap which isn't needed
* No support for future extensions (via flags and such)
Also if the user doesn't use the new SREG2 api, the PDPTR load after migration is now done on KVM_REQ_GET_NESTED_STATE_PAGES to at least read them correctly in cases when guest memory map is not up to date when nested state is loaded.
This patch series was tested by doing nested migration test of 32 bit PAE L1 + 32 bit PAE L2 on AMD and Intel and by nested migration test of 64 bit L1 + 32 bit PAE L2 on AMD. The later test currently fails on Intel (regardless of my patches).
Changes from V1:
- move only PDPTRS load to KVM_REQ_GET_NESTED_STATE_PAGES on VMX
- rebase on top of kvm/queue
- improve the KVM_GET_SREGS2 to have flag for PDPTRS and remove padding
Patches to qemu will be send soon as well.
Best regards,
Maxim Levitskky
Maxim Levitsky (6):
KVM: nSVM: refactor the CR3 reload on migration
KVM: nVMX: delay loading of PDPTRs to KVM_REQ_GET_NESTED_STATE_PAGES
KVM: x86: introduce kvm_register_clear_available
KVM: x86: Introduce KVM_GET_SREGS2 / KVM_SET_SREGS2
KVM: nSVM: avoid loading PDPTRs after migration when possible
KVM: nVMX: avoid loading PDPTRs after migration when possible
Documentation/virt/kvm/api.rst | 48 +++++++++++ arch/x86/include/asm/kvm_host.h | 7 ++ arch/x86/include/uapi/asm/kvm.h | 13 +++ arch/x86/kvm/kvm_cache_regs.h | 12 +++ arch/x86/kvm/svm/nested.c | 33 ++++++-- arch/x86/kvm/svm/svm.c | 6 +- arch/x86/kvm/vmx/nested.c | 24 ++++-- arch/x86/kvm/x86.c | 139 ++++++++++++++++++++++++++------ include/uapi/linux/kvm.h | 4 + 9 files changed, 246 insertions(+), 40 deletions(-)
Signature · 1 line
2.26.2
PATCH v2 1/6 KVM: nSVM: refactor the CR3 reload on migration
Document the actual reason why we need to do it on migration and move the call to svm_set_nested_state to be closer to VMX code.
To avoid loading the PDPTRs from possibly not up to date memory map, in nested_svm_load_cr3 after the move, move this code to .get_nested_state_pages.
--- arch/x86/kvm/svm/nested.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-)
arch/x86/kvm/svm/nested.c +25 -7
@@ -385,12 +385,12 @@ static inline bool nested_npt_enabled(struct vcpu_svm *svm)* if we are emulating VM-Entry into a guest with NPT enabled. */ static int nested_svm_load_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3, -bool nested_npt) +bool nested_npt,bool reload_pdptrs) {if (CC (kvm_vcpu_is_illegal_gpa (vcpu, cr3)))return -EINVAL ; -if (!nested_npt &&is_pae_paging (vcpu) && +if (reload_pdptrs && !nested_npt &&is_pae_paging (vcpu) && (cr3 !=kvm_read_cr3 (vcpu) ||pdptrs_changed (vcpu))) {if (CC (!load_pdptrs (vcpu, vcpu->arch.walk_mmu, cr3)))return -EINVAL ;
@@ -576,7 +576,7 @@ int enter_svm_guest_mode(struct kvm_vcpu *vcpu, u64 vmcb12_gpa,nested_vmcb02_prepare_save (svm, vmcb12); ret =nested_svm_load_cr3 (&svm->vcpu, vmcb12->save.cr3, -nested_npt_enabled (svm)); +nested_npt_enabled (svm), true);if (ret)return ret;
@@ -806,7 +806,7 @@ int nested_svm_vmexit(struct vcpu_svm *svm)nested_svm_uninit_mmu_context (vcpu); - rc =nested_svm_load_cr3 (vcpu, svm->vmcb->save.cr3, false); + rc =nested_svm_load_cr3 (vcpu, svm->vmcb->save.cr3, false, true);if (rc)return 1 ;
@@ -1291,6 +1291,19 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu, !nested_vmcb_valid_sregs(vcpu, save)) goto out_free; +/* +* While the nested guest CR3 is already checked and set by +* KVM_SET_SREGS, it was set when nested state was yet loaded, +* thus MMU might not be initialized correctly. +* Set it again to fix this. +*/ + + ret =nested_svm_load_cr3 (&svm->vcpu, vcpu->arch.cr3, +nested_npt_enabled (svm), false); +if (WARN_ON_ONCE (ret)) + goto out_free; + +/* * All checks done, we can enter guest mode. Userspace provides * vmcb12.control, which will be combined with L1 and stored into
@@ -1343,9 +1356,14 @@ static bool svm_get_nested_state_pages(struct kvm_vcpu *vcpu)if (WARN_ON (!is_guest_mode (vcpu)))return true; -if (nested_svm_load_cr3 (&svm->vcpu, vcpu->arch.cr3, -nested_npt_enabled (svm))) -return false; +if (!nested_npt_enabled (svm) &&is_pae_paging (vcpu)) +/* +* Reload the guest's PDPTRs since after a migration +* the guest CR3 might be restored prior to setting the nested +* state which can lead to a load of wrong PDPTRs. +*/ +if (CC (!load_pdptrs (vcpu, vcpu->arch.walk_mmu, vcpu->arch.cr3))) +return false;if (!nested_svm_vmrun_msrpm (svm)) { vcpu->run->exit_reason =KVM_EXIT_INTERNAL_ERROR ;
Signature · 1 line
2.26.2
PATCH v2 2/6 KVM: nVMX: delay loading of PDPTRs to KVM_REQ_GET_NESTED_STATE_PAGES
In reply to Maxim LevitskySimilar to the rest of guest page accesses after a migration, this access should be delayed to KVM_REQ_GET_NESTED_STATE_PAGES.
--- arch/x86/kvm/vmx/nested.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-)
arch/x86/kvm/vmx/nested.c +18 -5
@@ -1105,7 +1105,8 @@ static bool nested_vmx_transition_mmu_sync(struct kvm_vcpu *vcpu)* Exit Qualification (for a VM-Entry consistency check VM-Exit) is assigned to * @entry_failure_code. */ -static int nested_vmx_load_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3,bool nested_ept, +static int nested_vmx_load_cr3 (struct kvm_vcpu *vcpu,unsigned long cr3, +bool nested_ept,bool reload_pdptrs,enum vm_entry_failure_code *entry_failure_code) {if (CC (kvm_vcpu_is_illegal_gpa (vcpu, cr3))) {
@@ -1117,7 +1118,7 @@ static int nested_vmx_load_cr3(struct kvm_vcpu *vcpu, unsigned long cr3, bool ne* If PAE paging and EPT are both on, CR3 is not used by the CPU and * must not be dereferenced. */ -if (!nested_ept &&is_pae_paging (vcpu) && +if (reload_pdptrs && !nested_ept &&is_pae_paging (vcpu) && (cr3 !=kvm_read_cr3 (vcpu) ||pdptrs_changed (vcpu))) {if (CC (!load_pdptrs (vcpu, vcpu->arch.walk_mmu, cr3))) { *entry_failure_code =ENTRY_FAIL_PDPTE ;
@@ -2488,6 +2489,7 @@ static void prepare_vmcs02_rare(struct vcpu_vmx *vmx, struct vmcs12 *vmcs12)* is assigned to entry_failure_code on failure. */ static int prepare_vmcs02 (struct kvm_vcpu *vcpu,struct vmcs12 *vmcs12, +bool from_vmentry,enum vm_entry_failure_code *entry_failure_code) {struct vcpu_vmx *vmx =to_vmx (vcpu);
@@ -2572,7 +2574,7 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12,/* Shadow page tables on either EPT or shadow page tables. */ if (nested_vmx_load_cr3 (vcpu, vmcs12->guest_cr3,nested_cpu_has_ept (vmcs12), - entry_failure_code)) + from_vmentry, entry_failure_code))return -EINVAL ;/*
@@ -3120,6 +3122,17 @@ static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu)struct page *page;u64 hpa; +if (!nested_cpu_has_ept (vmcs12) &&is_pae_paging (vcpu)) { +/* +* Reload the guest's PDPTRs since after a migration +* the guest CR3 might be restored prior to setting the nested +* state which can lead to a load of wrong PDPTRs. +*/ +if (CC (!load_pdptrs (vcpu, vcpu->arch.walk_mmu, vcpu->arch.cr3))) +return false; + } + +if (nested_cpu_has2 (vmcs12,SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES )) {/* * Translate L1 physical address to host physical
@@ -3356,7 +3369,7 @@ enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu,if (vmcs12->cpu_based_vm_exec_control &CPU_BASED_USE_TSC_OFFSETTING ) vcpu->arch.tsc_offset += vmcs12->tsc_offset; -if (prepare_vmcs02 (vcpu, vmcs12, &entry_failure_code)) { +if (prepare_vmcs02 (vcpu, vmcs12, from_vmentry, &entry_failure_code)) { exit_reason.basic =EXIT_REASON_INVALID_STATE ; vmcs12->exit_qualification = entry_failure_code; goto vmentry_fail_vmexit_guest_mode;
@@ -4205,7 +4218,7 @@ static void load_vmcs12_host_state(struct kvm_vcpu *vcpu,* Only PDPTE load can fail as the value of cr3 was checked on entry and * couldn't have changed. */ -if (nested_vmx_load_cr3 (vcpu, vmcs12->host_cr3, false, &ignored)) +if (nested_vmx_load_cr3 (vcpu, vmcs12->host_cr3, false, true, &ignored))nested_vmx_abort (vcpu,VMX_ABORT_LOAD_HOST_PDPTE_FAIL );nested_vmx_transition_tlb_flush (vcpu, vmcs12, false);
Signature · 1 line
2.26.2
PATCH v2 3/6 KVM: x86: introduce kvm_register_clear_available
In reply to Maxim LevitskySmall refactoring that will be used in the next patch.
--- arch/x86/kvm/kvm_cache_regs.h | 7 +++++++ arch/x86/kvm/svm/svm.c | 6 ++---- 2 files changed, 9 insertions(+), 4 deletions(-)
arch/x86/kvm/kvm_cache_regs.h +7 -0
@@ -55,6 +55,13 @@ static inline void kvm_register_mark_available(struct kvm_vcpu *vcpu,__set_bit (reg, (unsigned long *)&vcpu->arch.regs_avail); } +static inline void kvm_register_clear_available (struct kvm_vcpu *vcpu, +enum kvm_reg reg) +{ +__clear_bit (reg, (unsigned long *)&vcpu->arch.regs_avail); +__clear_bit (reg, (unsigned long *)&vcpu->arch.regs_dirty); +} +static inline void kvm_register_mark_dirty (struct kvm_vcpu *vcpu,enum kvm_reg reg) {
arch/x86/kvm/svm/svm.c +2 -4
@@ -3868,10 +3868,8 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) vcpu->arch.apf.host_apf_flags =kvm_read_and_reset_apf_flags (); -if (npt_enabled) { - vcpu->arch.regs_avail &= ~(1 <<VCPU_EXREG_PDPTR ); - vcpu->arch.regs_dirty &= ~(1 <<VCPU_EXREG_PDPTR ); - } +if (npt_enabled) +kvm_register_clear_available (vcpu,VCPU_EXREG_PDPTR );/* * We need to handle MC intercepts here before the vcpu has a chance to
Signature · 1 line
2.26.2
PATCH v2 4/6 KVM: x86: Introduce KVM_GET_SREGS2 / KVM_SET_SREGS2
In reply to Maxim LevitskyThis is a new version of KVM_GET_SREGS / KVM_SET_SREGS.
It has the following changes:
* Has flags for future extensions
* Has vcpu's PDPTS, which allows to save/restore them on migration.
* Lacks obsolete interrupt bitmap (done now via KVM_SET_VCPU_EVENTS)
New capability, KVM_CAP_SREGS2 is added to signal
the userspace of this ioctl.
--- Documentation/virt/kvm/api.rst | 48 +++++++++++ arch/x86/include/asm/kvm_host.h | 7 ++ arch/x86/include/uapi/asm/kvm.h | 13 +++ arch/x86/kvm/kvm_cache_regs.h | 5 ++ arch/x86/kvm/x86.c | 139 ++++++++++++++++++++++++++------ include/uapi/linux/kvm.h | 4 + 6 files changed, 192 insertions(+), 24 deletions(-)
Documentation/virt/kvm/api.rst +48 -0
@@ -5021,6 +5021,54 @@ see KVM_XEN_VCPU_SET_ATTR above. The KVM_XEN_VCPU_ATTR_TYPE_RUNSTATE_ADJUST type may not be used with the KVM_XEN_VCPU_GET_ATTR ioctl. + +4.131 KVM_GET_SREGS2 +------------------ + +:Capability: KVM_CAP_SREGS2 +:Architectures: x86 +:Type: vcpu ioctl +:Parameters: struct kvm_sregs2 (out) +:Returns: 0 on success, -1 on error + +Reads special registers from the vcpu. +This ioctl (when supported) replaces the KVM_GET_SREGS. + +:: + +struct kvm_sregs2 { + /* out (KVM_GET_SREGS2) / in (KVM_SET_SREGS2) */ + struct kvm_segment cs, ds, es, fs, gs, ss; + struct kvm_segment tr, ldt; + struct kvm_dtable gdt, idt; + __u64 cr0, cr2, cr3, cr4, cr8; + __u64 efer; + __u64 apic_base; + __u64 flags; + __u64 pdptrs[4]; +}; + +flags values for ``kvm_sregs2``: + +``KVM_SREGS2_FLAGS_PDPTRS_VALID`` + + Indicates thats the struct contain valid PDPTR values. + + +4.132 KVM_SET_SREGS2 +------------------ + +:Capability: KVM_CAP_SREGS2 +:Architectures: x86 +:Type: vcpu ioctl +:Parameters: struct kvm_sregs2 (in) +:Returns: 0 on success, -1 on error + +Writes special registers into the vcpu. +See KVM_GET_SREGS2 for the data structures. +This ioctl (when supported) replaces the KVM_SET_SREGS. + + 5. The kvm_run structure ========================
arch/x86/include/asm/kvm_host.h +7 -0
@@ -849,6 +849,13 @@ struct kvm_vcpu_arch {/* Protected Guests */ bool guest_state_protected; + +/* +* Do we need to reload the PDPTRs when entering the nested state? +* Set if the userspace used the KVM_SET_SREGS to set CR3 +* and thus didn't load the PDPTRs from the migration state. +*/ +bool pdptr_reload_on_nesting_needed; };struct kvm_lpage_info {
arch/x86/include/uapi/asm/kvm.h +13 -0
@@ -159,6 +159,19 @@ struct kvm_sregs {__u64 interrupt_bitmap[(KVM_NR_INTERRUPTS +63 ) /64 ]; }; +struct kvm_sregs2 { +/* out (KVM_GET_SREGS2) / in (KVM_SET_SREGS2) */ +struct kvm_segment cs, ds, es, fs, gs, ss; +struct kvm_segment tr, ldt; +struct kvm_dtable gdt, idt; +__u64 cr0, cr2, cr3, cr4, cr8; +__u64 efer; +__u64 apic_base; +__u64 flags; +__u64 pdptrs[4 ]; +}; +#define KVM_SREGS2_FLAGS_PDPTRS_VALID 1 +/* for KVM_GET_FPU and KVM_SET_FPU */ struct kvm_fpu {__u8 fpr[8 ][16 ];
arch/x86/kvm/kvm_cache_regs.h +5 -0
@@ -125,6 +125,11 @@ static inline u64 kvm_pdptr_read(struct kvm_vcpu *vcpu, int index)return vcpu->arch.walk_mmu->pdptrs[index]; } +static inline void kvm_pdptr_write (struct kvm_vcpu *vcpu,int index,u64 value) +{ + vcpu->arch.walk_mmu->pdptrs[index] = value; +} +static inline ulong kvm_read_cr0_bits (struct kvm_vcpu *vcpu,ulong mask) {ulong tmask = mask &KVM_POSSIBLE_CR0_GUEST_BITS ;
arch/x86/kvm/x86.c +115 -24
229 lines not on this page · raw message
include/uapi/linux/kvm.h +4 -0
@@ -1082,6 +1082,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_SGX_ATTRIBUTE 196#define KVM_CAP_VM_COPY_ENC_CONTEXT_FROM 197#define KVM_CAP_EXIT_HYPERCALL 198 +#define KVM_CAP_SREGS2 199#ifdef KVM_CAP_IRQ_ROUTING
@@ -1620,6 +1621,9 @@ struct kvm_xen_hvm_attr {#define KVM_XEN_VCPU_GET_ATTR _IOWR(KVMIO, 0xca, struct kvm_xen_vcpu_attr)#define KVM_XEN_VCPU_SET_ATTR _IOW(KVMIO, 0xcb, struct kvm_xen_vcpu_attr) +#define KVM_GET_SREGS2 _IOR(KVMIO, 0xcc, struct kvm_sregs2) +#define KVM_SET_SREGS2 _IOW(KVMIO, 0xcd, struct kvm_sregs2) +struct kvm_xen_vcpu_attr {__u16 type;__u16 pad[3 ];
Signature · 1 line
2.26.2
Paolo Bonzini
maintainerRe: [PATCH v2 4/6] KVM: x86: Introduce KVM_GET_SREGS2 / KVM_SET_SREGS2
arch/x86/kvm/x86.c static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) · line 10121
Quoted 11 lines · Maxim Levitsky
arch/x86/kvm/x86.c +11
-0+if (sregs2->flags &KVM_SREGS2_FLAGS_PDPTRS_VALID ) { + +if (!is_pae_paging (vcpu)) +return -EINVAL ; + +for (i =0 ; i <4 ; i++) +kvm_pdptr_write (vcpu, i, sregs2->pdptrs[i]); + +kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); + mmu_reset_needed =1 ; + }
I think this should also have
else {
if (is_pae_paging(vcpu))
return -EINVAL;
}but perhaps even better, check it at the beginning:
if ((sregs->cr4 & X86_CR4_PAE) &&
!!(sregs->efer & EFER_LMA) == !!(sregs2->flags & KVM_SREGS2_FLAGS_PDPTRS_VALID))
return -EINVAL;which technically means the flag is redundant, but there is some value in having the flag and not allowing the user to shoot itself in the foot.
Paolo
Re: [PATCH v2 4/6] KVM: x86: Introduce KVM_GET_SREGS2 / KVM_SET_SREGS2
Quoted 15 lines · Paolo Bonzini
arch/x86/kvm/x86.c static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) · line 10121
Quoted 11 lines · Maxim Levitsky
arch/x86/kvm/x86.c +11
-0+if (sregs2->flags &KVM_SREGS2_FLAGS_PDPTRS_VALID ) { + +if (!is_pae_paging (vcpu)) +return -EINVAL ; + +for (i =0 ; i <4 ; i++) +kvm_pdptr_write (vcpu, i, sregs2->pdptrs[i]); + +kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); + mmu_reset_needed =1 ; + }I think this should also have
Quoted 4 lines
else {
if (is_pae_paging(vcpu))
return -EINVAL;
}What about the case when we migrate from qemu that doesn't use this ioctl to qemu that does?
In this case assuming that the new qemu does use SREGS2 ioctls, the PDPTR data will not be present
in the migration stream and thus qemu will call this ioctl without this flag set.
I think I should in this case load the pdptrs from memory, Or I should make qemu not use this ioctl in this. What do you prefer?
Thanks for pointing this bug out though!
I haven't thought about this case well enough.
Best regards,
Maxim Levitsky
Quoted 12 lines
but perhaps even better, check it at the beginning:
if ((sregs->cr4 & X86_CR4_PAE) && !!(sregs->efer & EFER_LMA) == !!(sregs2->flags & KVM_SREGS2_FLAGS_PDPTRS_VALID)) return -EINVAL;which technically means the flag is redundant, but there is some value in having the flag and not allowing the user to shoot itself in the foot.
Paolo
Paolo Bonzini
maintainerRe: [PATCH v2 4/6] KVM: x86: Introduce KVM_GET_SREGS2 / KVM_SET_SREGS2
Quoted 16 lines · Maxim Levitsky
Quoted 15 lines · Paolo Bonzini
arch/x86/kvm/x86.c static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) · line 10121
Quoted 11 lines · Maxim Levitsky
arch/x86/kvm/x86.c +11
-0+if (sregs2->flags &KVM_SREGS2_FLAGS_PDPTRS_VALID ) { + +if (!is_pae_paging (vcpu)) +return -EINVAL ; + +for (i =0 ; i <4 ; i++) +kvm_pdptr_write (vcpu, i, sregs2->pdptrs[i]); + +kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); + mmu_reset_needed =1 ; + }I think this should also have
Quoted 8 lines
Quoted 4 lines
else { if (is_pae_paging(vcpu)) return -EINVAL; }What about the case when we migrate from qemu that doesn't use this ioctl to qemu that does?
Right, that makes sense but then the "else" branch should do the same as KVM_SET_SREGS. Maybe add a "load_pdptrs" bool to __set_sregs_common?
Paolo
Re: [PATCH v2 4/6] KVM: x86: Introduce KVM_GET_SREGS2 / KVM_SET_SREGS2
Quoted 22 lines · Paolo Bonzini
Quoted 21 lines · Maxim Levitsky
Quoted 19 lines · Paolo Bonzini
arch/x86/kvm/x86.c static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) · line 10121
Quoted 11 lines · Maxim Levitsky
arch/x86/kvm/x86.c +11
-0+if (sregs2->flags &KVM_SREGS2_FLAGS_PDPTRS_VALID ) { + +if (!is_pae_paging (vcpu)) +return -EINVAL ; + +for (i =0 ; i <4 ; i++) +kvm_pdptr_write (vcpu, i, sregs2->pdptrs[i]); + +kvm_register_mark_dirty (vcpu,VCPU_EXREG_PDPTR ); + mmu_reset_needed =1 ; + }I think this should also have
else { if (is_pae_paging(vcpu)) return -EINVAL; }
Quoted 5 lines
Quoted 2 lines
What about the case when we migrate from qemu that doesn't use this ioctl to qemu that does?
Right, that makes sense but then the "else" branch should do the same as
KVM_SET_SREGS. Maybe add a "load_pdptrs" bool to__set_sregs_common?
Yes, I'll do something like that.
Thanks,
Best regards,
Maxim Levitsky
Quoted 3 lines
Paolo
PATCH v2 5/6 KVM: nSVM: avoid loading PDPTRs after migration when possible
In reply to Maxim Levitskyif new KVM_*_SREGS2 ioctls are used, the PDPTRs are a part of the migration state and are correctly restored by those ioctls.
--- arch/x86/kvm/svm/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/nested.c +2 -1
@@ -1356,7 +1356,8 @@ static bool svm_get_nested_state_pages(struct kvm_vcpu *vcpu)if (WARN_ON (!is_guest_mode (vcpu)))return true; -if (!nested_npt_enabled (svm) &&is_pae_paging (vcpu)) +if (vcpu->arch.pdptr_reload_on_nesting_needed && + !nested_npt_enabled (svm) &&is_pae_paging (vcpu))/* * Reload the guest's PDPTRs since after a migration * the guest CR3 might be restored prior to setting the nested
Signature · 1 line
2.26.2
PATCH v2 6/6 KVM: nVMX: avoid loading PDPTRs after migration when possible
In reply to Maxim Levitskyif new KVM_*_SREGS2 ioctls are used, the PDPTRs are a part of the migration state and are correctly restored by those ioctls.
--- arch/x86/kvm/vmx/nested.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/vmx/nested.c +2 -1
@@ -3122,7 +3122,8 @@ static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu)struct page *page;u64 hpa; -if (!nested_cpu_has_ept (vmcs12) &&is_pae_paging (vcpu)) { +if (vcpu->arch.pdptr_reload_on_nesting_needed && + !nested_cpu_has_ept (vmcs12) &&is_pae_paging (vcpu)) {/* * Reload the guest's PDPTRs since after a migration * the guest CR3 might be restored prior to setting the nested
Signature · 1 line
2.26.2
Sean Christopherson
maintainerQuoted 20 lines · Maxim Levitsky
This patch set aims to fix few flaws that were discovered in KVM_{GET|SET}_SREGS on x86:
* There is no support for reading/writing PDPTRs although these are considered to be part of the guest state.
* There is useless interrupt bitmap which isn't needed
* No support for future extensions (via flags and such)
Also if the user doesn't use the new SREG2 api, the PDPTR load after migration is now done on
KVM_REQ_GET_NESTED_STATE_PAGESto at least read them correctly in cases when guest memory map is not up to date when nested state is loaded.This patch series was tested by doing nested migration test of 32 bit PAE L1 + 32 bit PAE L2 on AMD and Intel and by nested migration test of 64 bit L1 + 32 bit PAE L2 on AMD. The later test currently fails on Intel (regardless of my patches).
Quoted 7 lines
Changes from V1:
- move only PDPTRS load toKVM_REQ_GET_NESTED_STATE_PAGESon VMX
- rebase on top of kvm/queue
- improve theKVM_GET_SREGS2to have flag for PDPTRS and remove paddingPatches to qemu will be send soon as well.
How did you want to handle integration with the removal of pdptrs_changed()?
Quoted 27 lines · Sean Christopherson
Quoted 26 lines · Maxim Levitsky
This patch set aims to fix few flaws that were discovered in KVM_{GET|SET}_SREGS on x86:
* There is no support for reading/writing PDPTRs although these are considered to be part of the guest state.
* There is useless interrupt bitmap which isn't needed
* No support for future extensions (via flags and such)
Also if the user doesn't use the new SREG2 api, the PDPTR load after migration is now done on
KVM_REQ_GET_NESTED_STATE_PAGESto at least read them correctly in cases when guest memory map is not up to date when nested state is loaded.This patch series was tested by doing nested migration test of 32 bit PAE L1 + 32 bit PAE L2 on AMD and Intel and by nested migration test of 64 bit L1 + 32 bit PAE L2 on AMD. The later test currently fails on Intel (regardless of my patches).
Changes from V1:
- move only PDPTRS load toKVM_REQ_GET_NESTED_STATE_PAGESon VMX
- rebase on top of kvm/queue
- improve theKVM_GET_SREGS2to have flag for PDPTRS and remove padding
Quoted 7 lines
Quoted 1 line
Patches to qemu will be send soon as well.
How did you want to handle integration with the removal of
pdptrs_changed()?
Hi!
Sorry that I missed your mail. I will take a look in a day or so at this, and I don't envision any significant trouble with removal of pdptrs_changed, since it is only an optimization anyway.
Thanks,
Best regards,
Maxim Levitsky